danny0405 commented on code in PR #19205:
URL: https://github.com/apache/hudi/pull/19205#discussion_r3773937705
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/BaseCreateHandle.java:
##########
@@ -171,6 +183,23 @@ protected HoodieRecord<T> updateFileName(HoodieRecord<T>
record, HoodieSchema sc
return record.prependMetaFields(schema, targetSchema, metadataValues,
prop);
}
+ /**
+ * Blank out {@code _hoodie_file_name} on a record being copied forward
under a mode that does not
+ * populate it.
+ *
+ * <p>Clearing rather than leaving the column alone is deliberate. The
record here came from the
+ * previous base file, so it can still carry a file name written while the
table was on
+ * {@code ALL} — and that name points at a file this record no longer lives
in. Passing null
+ * through {@link MetadataValues} would not do it either: {@code
updateMetadataValuesInternal}
+ * skips null entries (HoodieAvroIndexedRecord:383), so the stale value
would survive. Hence the
+ * explicit {@code updateMetaField}.
+ */
+ private HoodieRecord<T> clearFileName(HoodieRecord<T> record, HoodieSchema
schema, HoodieSchema targetSchema, Properties prop) {
Review Comment:
I would say the clear is not necessary, the flag `preserveMetadata` servers
mainly for table service rewrite like compaction or clustering which should do
a spitting image rewrite of the existing data files, it does not need to care
about the meta fields population.
Also we need to adjust the logic in `updateFileName` to only set up the file
name if the meta field is already there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
