rangareddy commented on issue #17298: URL: https://github.com/apache/hudi/issues/17298#issuecomment-5365817725
This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-8423). **Findings: partially delivered. The read side now supports the record merger; the write side does not.** **Read side - done.** `hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/cdc/CDCFileGroupIterator.scala` resolves the merger from the reader context and threads it through the merge: - `:130` - `private lazy val recordMerger: HoodieRecordMerger = readerContext.getRecordMerger().get()` - `:96` - branches on `recordMerger.getMergingStrategy == PAYLOAD_BASED_MERGE_STRATEGY_UUID` - `:105-106` - passes `readerContext.getMergeMode`, `isPartialMergeEnabled`, `Option.of(recordMerger)`, `payloadClass` and `partialUpdateModeOpt` into the buffered merge That came in via #13444 (HUDI-9526, *"Use HoodieFileGroupReader throughout the CDC flow"*, `fffa3efaab8d`) and #13694 (HUDI-9695, *"Use BufferedRecordMerger for spark cdc reader"*, `7aa92cb32038`) - neither keyed to this ticket, which is why it still reads as untouched. **Write side - not done.** `hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/cdc/HoodieCDCLogger.java`, and its `HoodieNativeCDCLogger` / `HoodieAvroNativeCDCLogger` implementations, contain no reference to `RecordMergeMode`, `HoodieRecordMerger` or a payload class. CDC log-block generation is still merger-unaware. So the remaining scope for this ticket is the write half only. Worth narrowing the description to that, so it does not get re-triaged as wholly outstanding. Keeping this open. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
