discivigour opened a new pull request, #10700: URL: https://github.com/apache/paimon/pull/10700
### Purpose Concurrent writers or compactors can restore the same fixed-bucket deletion vector (DV) index and prepare separate replacements. Since a fixed bucket replaces its entire DV index, a later stale commit can discard previously committed deletion marks and expose duplicate primary keys. This change: - Emits explicit DELETE entries for the previously prepared DV index when replacing or clearing it, for both compaction and direct DV DML. - Reuses the existing file/DV dependency checks for `HASH_FIXED`, with additional bucket-level checks for initial ADDs and legacy empty indexes. - Distinguishes unchanged, replacement, and removal results so clearing an index no longer writes an empty DV file, and intermediate compaction outputs do not become published dependencies. Stale results are rejected and must be recomputed from the latest state. The checks remain conservative: metadata-only level upgrades with DVs may also be rejected, which can block compaction. ### Tests - Adds regression coverage for concurrent compactions involving the same or different files, same-key outputs across levels, stale rewrite inputs, and direct DV deletes. - Covers DV removal, intermediate compaction results, consecutive prepares, writer state restoration, and commit retries. - Tests were not run for the current branch head. - `git diff --check origin/master...HEAD` passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
