JingsongLi opened a new pull request, #1101: URL: https://github.com/apache/paimon-rust/pull/1101
## Purpose Allow the existing PyPaimon writes for deletion-vector primary-key tables to use Native across partial-update, aggregation, and postpone fixed-bucket routing. ## Changes - Remove the partial-update/aggregation DV rejection from the KV writer and the redundant private `deletion_vectors_enabled` configuration field. The existing shared merge functions still fold each write buffer into sorted level-0 files. - Allow postpone fixed-bucket writers on DV tables. Reuse the normal writer, existing bucket layout restore, overwrite rules, and commit checks. - Preserve Java's batch visibility: ordinary DV scans hide uncompacted level-0 files; `deletion-vectors.merge-on-read=true` reads and merges them. - Replace old rejection tests with write/commit/read coverage. Cover fixed/dynamic buckets, INPUT changelog before folding, sequence restoration, multiple buffer flushes, post-merge predicates with omitted filter columns, and fixed postpone append/overwrite. - Extend the mixed level-1/level-0 deletion-vector reader test to partial-update and aggregation, proving that deleted records are filtered before merging and cannot be resurrected. ## Java semantics `MergeTreeWriter.flushWriteBuffer` writes level-0 data through the configured merge function and writes INPUT changelog before folding. DV maintenance is owned by the surrounding file-store write path, rather than a restriction on the buffer reducer. `CoreOptions.batchScanSkipLevel0` and `AbstractBatchTableScan` define the default visibility and merge-on-read override. `KeyValueFileStore.newPostponeFixedBucketWrite` uses a write-only fixed-bucket writer and does not prohibit deletion vectors. This change adds no compaction implementation, new format, or public API. Existing validation for invalid merge configurations remains in place. ## Validation - Reproduced both old rejection paths with new behavioral tests before removing the guards. - `cargo test -p paimon --test dv_merge_engine_write_test`: 4 tests pass, exercising multiple engines and bucket/changelog configurations. - `cargo test -p paimon --lib kv_file`: 66 tests pass. - `cargo test -p paimon --lib postpone_fixed`: 26 tests pass. - `cargo +1.98.0 clippy -p paimon -p pypaimon_rust --all-targets -- -D warnings` passes. - `cargo fmt --all --check` and `git diff --check` pass. - Rebuilt the Python extension and ran 337 PyPaimon regressions with Native plan/read/write/commit enabled: all pass, including 51 new REST integration cases. Counters: plans=488, reads=363, writes=387, commits=160. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
