lokeshj1703 opened a new issue, #19748:
URL: https://github.com/apache/hudi/issues/19748
### Problem
On a table-version-6 MOR table using `PostgresDebeziumAvroPayload`, an
unchanged Postgres TOAST column is emitted by Debezium as the sentinel
`__debezium_unavailable_value`, and the payload merge is supposed to restore
the prior column value at read time. Instead, the raw sentinel leaks to readers.
Root cause: `PostgresDebeziumAvroPayload.mergeToastedValuesIfPresent`
backfills the sentinel by mutating the incoming Avro record in place and
returning the same reference. `HoodieAvroRecordMerger.merge` then has an
identity shortcut:
```java
if (updatedRecord == newerAvroRecord) {
return newer; // engine-native record, never received the in-place
backfill
}
```
The check assumes "same reference implies unchanged" and returns the
engine-native `newer` record, which never received the in-place mutation. The
corrected value is computed and then discarded, so the raw sentinel is returned.
Only v6 x 1.x is affected: v9 reroutes Postgres to the FILL_UNAVAILABLE
partial-update framework (a different merger), and 0.x used the payload's Avro
result directly.
### Expected
The read on a v6 MOR Postgres CDC table restores the prior value for an
unchanged TOAST column, rather than returning the raw
`__debezium_unavailable_value` sentinel.
### Note
This is one of two approaches to the same bug. See #19280, which instead
changes `PostgresDebeziumAvroPayload` to return a new record on backfill
(keeping the merger shortcut). This issue/PR takes the merger-side approach
(remove the shortcut), which also protects any other payload that mutates the
incoming record in place. The two should be reconciled; only one is needed.
### Environment
- Hudi version: master (1.x)
- Table version 6, MOR, PostgresDebeziumAvroPayload.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
