Hi Paimon devs, I'd like to propose adding Flink FLIP-510 support to Paimon's primary-key table sink, and I've opened an issue and a PR for it:
- Issue: https://github.com/apache/paimon/issues/8498 - PR: https://github.com/apache/paimon/pull/8452 Motivation ---------- FLIP-510 ("Drop ChangelogNormalize for operations which don't need it") lets a sink advertise a "keyOnlyDeletes" capability, so the Flink planner can drop the upstream ChangelogNormalize node when the source produces deletes by key. This is especially useful for CDC sources, where UPDATE_AFTER already carries the full row: dropping ChangelogNormalize removes its state and overhead with no loss of correctness. Proposed approach ----------------- - Advertise the FLIP-510 keyOnlyDeletes capability on primary-key table sinks. - Gated behind a new table option `sink.key-only-deletes.enabled` (default false), so existing plans are unchanged unless explicitly opted in. - The API only exists in Flink 2.1+, so the behavior is wired through ChangelogModeUtils in paimon-flink1-common (no-op) and paimon-flink2-common (sets the flag), keeping paimon-flink-common compiling against both Flink 1.x and 2.x. I've tested this in a real environment and ChangelogNormalize is correctly dropped when the option is enabled. CI on the PR is green. I'd appreciate feedback on the approach, and would be grateful if a committer could take a look / assign the issue. Thanks! Irene
