pvary commented on code in PR #17320:
URL: https://github.com/apache/iceberg/pull/17320#discussion_r3978427119
##########
parquet/src/main/java/org/apache/iceberg/parquet/ParquetValueReaders.java:
##########
@@ -231,6 +235,90 @@ public static ParquetValueReader<?>
replaceWithMetadataReader(
return reader;
}
+ /**
+ * Builds a struct reader from the expected fields, in field order. A field
present in the file
+ * uses its column reader; a field missing from the file uses a metadata or
partition constant, or
+ * its initial default. When no expected field reads a file column, the
struct reader is wrapped
+ * so a presence column supplies the definition level.
+ */
+ public static <T> ParquetValueReader<T> structReader(
+ MessageType fileSchema,
+ String[] structPath,
+ List<Types.NestedField> expectedFields,
+ Map<Integer, ParquetValueReader<?>> readersById,
+ Map<Integer, ?> idToConstant,
+ BiFunction<org.apache.iceberg.types.Type, Object, Object>
convertConstant,
+ Function<List<ParquetValueReader<?>>, ParquetValueReader<T>>
newStructReader) {
+ int constantDefinitionLevel = fileSchema.getMaxDefinitionLevel(structPath);
+
+ List<ParquetValueReader<?>> readers =
Lists.newArrayListWithExpectedSize(expectedFields.size());
+ for (Types.NestedField field : expectedFields) {
+ int id = field.fieldId();
+ ParquetValueReader<?> reader =
+ replaceWithMetadataReader(id, readersById.get(id), idToConstant,
constantDefinitionLevel);
+ readers.add(defaultReader(field, reader, constantDefinitionLevel,
convertConstant));
+ }
+
+ ParquetValueReader<T> reader = newStructReader.apply(readers);
+ ColumnDescriptor presence =
+ presenceColumn(
+ fileSchema, structPath, constantDefinitionLevel, expectedFields,
readersById);
+
+ return presence == null
+ ? reader
+ : withPresence(reader, presence,
fileSchema.getMaxRepetitionLevel(structPath));
Review Comment:
nit: could we just inline `new PresenceReader<>(reader, presence,
repetitionLevel)`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
