cshuo commented on code in PR #19205:
URL: https://github.com/apache/hudi/pull/19205#discussion_r3643626290
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/io/storage/row/HoodieRowCreateHandle.java:
##########
@@ -158,10 +159,45 @@ public HoodieRowCreateHandle(HoodieTable table,
* @throws IOException
*/
public void write(InternalRow row) throws IOException {
- if (populateMetaFields) {
- writeRow(row);
- } else {
- writeRowNoMetaFields(row);
+ switch (metaFieldsMode) {
+ case ALL:
+ writeRow(row);
+ break;
+ case NONE:
+ writeRowNoMetaFields(row);
+ break;
+ default:
+ writeRowSelectiveMetaFields(row);
+ break;
+ }
+ }
+
+ /**
+ * Selective meta-field write path: populate only the meta columns opted in
via
+ * {@code hoodie.meta.fields.mode} — {@code _hoodie_commit_time} and/or
{@code _hoodie_file_name}.
+ * The other meta columns stay null on disk. Record key is never populated
in this path, so the
+ * record key is not registered with the write support (bloom filter / RLI
hooks are meaningless
+ * without the record-key column).
+ */
+ private void writeRowSelectiveMetaFields(InternalRow row) {
+ try {
+ UTF8String[] metaFields = new UTF8String[5];
+ if (metaFieldsMode.isCommitTimePopulated()) {
+ metaFields[HoodieRecord.COMMIT_TIME_METADATA_FIELD_ORD] =
shouldPreserveHoodieMetadata
+ ? row.getUTF8String(HoodieRecord.COMMIT_TIME_METADATA_FIELD_ORD) :
commitTime;
+ }
+ if (metaFieldsMode.isFileNamePopulated()) {
+ metaFields[HoodieRecord.FILENAME_META_FIELD_ORD] =
shouldPreserveHoodieMetadata
Review Comment:
Row-writer clustering copies the old _hoodie_file_name when preserving
metadata. Rewritten records then reference replaced files. The file name should
always be updated to the new output file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
