wombatu-kun commented on code in PR #19926:
URL: https://github.com/apache/hudi/pull/19926#discussion_r4623630397
##########
hudi-hadoop-common/src/main/java/org/apache/hudi/common/util/ParquetUtils.java:
##########
@@ -452,10 +452,27 @@ public HoodieFileFormat getFormat() {
*/
@Override
public List<GenericRecord> readAvroRecords(HoodieStorage storage,
StoragePath filePath) {
+ // A schema-less read takes the schema from the file; the copy drops any
read schema or
+ // projection a previous caller left on the shared configuration.
+ Configuration conf = storage.getConf().unwrapCopyAs(Configuration.class);
+ conf.unset("parquet.avro.read.schema");
+ conf.unset(AvroReadSupport.AVRO_REQUESTED_PROJECTION);
+ return readAvroRecords(conf, filePath);
+ }
+
+ @Override
+ public List<GenericRecord> readAvroRecords(HoodieStorage storage,
StoragePath filePath, HoodieSchema schema) {
+ // The read schema goes on a private copy so the caller's configuration is
left untouched.
+ Configuration conf = storage.getConf().unwrapCopyAs(Configuration.class);
+ AvroReadSupport.setAvroReadSchema(conf, schema.toAvroSchema());
Review Comment:
The schema-supplied `readAvroRecords` overload inherits
`AVRO_REQUESTED_PROJECTION` on its private copy, so a stale projection can omit
stored fields and replace them with defaults. Please unset the projection there
and extend `TestParquetUtils` to cover a narrow projection followed by an
explicit-schema read.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]