hudi-agent commented on code in PR #19749:
URL: https://github.com/apache/hudi/pull/19749#discussion_r3864259496
##########
hudi-common/src/main/java/org/apache/hudi/common/model/HoodieAvroRecordMerger.java:
##########
@@ -71,10 +71,11 @@ public <T> BufferedRecord<T> merge(BufferedRecord<T> older,
BufferedRecord<T> ne
if (updatedRecord == previousAvroData || updatedRecord ==
HoodieRecord.SENTINEL) {
return older;
}
- if (updatedRecord == newerAvroRecord) {
- // simply return the newer record instead of creating a new record
- return newer;
- }
+ // Do not short-circuit to `newer` when updatedRecord ==
newerAvroRecord. Some payloads
+ // (e.g. PostgresDebeziumAvroPayload's TOAST backfill via
mergeToastedValuesIfPresent)
+ // mutate the incoming Avro record in place and return the same
reference. The engine-native
+ // `newer` record is a distinct object that does not reflect those
in-place mutations, so
+ // returning it would drop the merge result. Always rebuild the result
from updatedRecord.
// Construct a new BufferedRecord with updated value
T resultRecord = recordContext.convertAvroRecord(updatedRecord);
Review Comment:
🤖 Removing the shortcut means the "newer wins" case now always rebuilds via
`fromEngineRecord(..., updatedValue.isEmpty())`. Since we're inside `if
(updatedValue.isPresent())`, `isEmpty()` is always false, so the result's
`HoodieOperation` is always null — whereas `return newer` preserved it.
`UpdateProcessor.CallbackProcessor` (lines 176-179) routes
`onUpdate`/`onInsert` off `result.getHoodieOperation()`, so
`isUpdateAfter(null)`/`isInsert(null)` are both false and neither callback
fires. Could this silently skip `BaseFileUpdateCallback` index/metadata updates
for newer-wins records during compaction on CUSTOM merge-mode tables?
@nsivabalan could you confirm whether preserving `newer`'s HoodieOperation
matters here?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
