junmuz opened a new pull request, #10702:
URL: https://github.com/apache/paimon/pull/10702
### Purpose
- Adds a new config option changelog-producer.preserve-sequence-on-retract
that accepts a comma-separated list of column names whose values should be
taken from the incoming event (rather than the stored row) when producing
changelog retraction records (-U, -D)
- This is useful when delete or update events carry an event timestamp
that downstream consumers need — e.g., external systems like Cassandra that
rely on WRITETIME for conflict resolution
- Only supported with the lookup changelog producer
- This also allows alignment with FlinkCDC that emits retract events with
db event timestamps.
- Added clearly in document not to use the preserved field as aggregated
values or filtering keys.
### Tests
- [x] Unit tests for delete retraction preserving the event's sequence field
value
- [x] Unit tests for update retraction (-U) carrying the new event's
sequence field value
- [x] Unit tests confirming original behavior is preserved when the option
is not set
### Manual verification
- Tested out after configuration, the table correctly produces the changelog
events.
```
CREATE TABLE test.test_table (
id INT,
name STRING,
updated_at TIMESTAMP,
op_type STRING
) TBLPROPERTIES (
'primary-key' = 'id',
'changelog-producer' = 'lookup',
'write-only' = 'false',
'changelog-producer.preserve-field-on-retract' = 'updated_at',
'rowkind.field' = 'op_type'
)
```
```
+-------+---+-------+--------------------------+
|rowkind|id |name |updated_at |
+-------+---+-------+--------------------------+
|-D |1 |one_new|2026-09-09 15:11:45.082928|
|+U |1 |one_new|2026-09-09 15:11:03.804031|
|-U |1 |one |2026-09-09 15:11:03.804031|
|+I |1 |one |2026-09-09 15:10:31.101137|
+-------+---+-------+--------------------------+
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
