HonahX commented on code in PR #12177:
URL: https://github.com/apache/iceberg/pull/12177#discussion_r1952099384
##########
parquet/src/main/java/org/apache/iceberg/parquet/TypeToMessageType.java:
##########
@@ -71,6 +89,10 @@ public GroupType struct(StructType struct, Type.Repetition
repetition, int id, S
Types.GroupBuilder<GroupType> builder = Types.buildGroup(repetition);
for (NestedField field : struct.fields()) {
+ if (ignoreUnknownFields
+ && field.type().typeId() ==
org.apache.iceberg.types.Type.TypeID.UNKNOWN) {
+ continue;
Review Comment:
My interpretation of `unknown` type:
https://iceberg.apache.org/spec/#semi-structured-types
> Default / null column type used when a more specific type is not known
—is that `unknown` should only be used as a column type, not as an inner
type within a list or map. In fact, we can’t even "ignore" the element field of
a list in Parquet.
Not sure if we should update the spec to make this more explicit, since
lists can currently take "any" type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
