JingsongLi opened a new pull request, #1081: URL: https://github.com/apache/paimon-rust/pull/1081
## Purpose PyPaimon already supports updates of raw BLOB, ARRAY<BLOB> and MAP<X, BLOB> columns on data-evolution tables. Rust rejected those columns, so their update operations could not use Native execution. Move this existing writer capability into Rust core. Java has no corresponding public raw-Blob update writer API; the authoritative persisted format and fallback semantics are specified by `BlobUpdateTest`, `BlobFallbackRecordReader` and the Blob element serializers. ## Changes - Build sparse Blob deltas without reading unchanged payloads. Unchanged positions use Java placeholder length -2; an explicit NULL uses -1. With no historical provider for the field ID, write real NULLs across the full normal-file range. - Split raw Blob columns into dedicated writers, independently roll physical files, retain continuous row IDs, and prepare deltas with the 0/0 snapshot-sequence sentinel. - Determine update/delete anchors from normal files and attach overlapping dedicated files. Ordinary-column rewrites exclude unrelated Blob payloads and their independent physical ranges. - Resolve historical providers by field ID. Dropping and adding a column with the same name cannot reuse the old column's baseline. - Reuse the core URI copy path for row Blob streams. The Python binding only adapts a reader factory and propagates its original exception. - Accept row map entries as a list of key/value structs to preserve Java's nullable map keys, including duplicate-null-key validation. Ordinary Arrow Map input keeps its existing representation. - Keep prepared and published files when an updater is aborted or a commit fails. Explicit TableCommit.abort retains its existing caller-controlled semantics. ## Verification - `cargo +1.98.0 fmt --all -- --check` - `cargo +1.98.0 clippy --locked --all-targets --workspace --features fulltext,vortex -- -D warnings` - `cargo test --locked -p paimon --all-targets --features fulltext,vortex` - 13 Blob update integration tests, plus wire-tag, sparse-cursor and nullable-key tests. - Local Python extension build with maturin and 181 PyPaimon Blob update tests. Nullable map-key output is decoded independently by Python's existing Blob reader. - PyPaimon CI test scope: Native 9270 passed, 275 skipped; classic 6648 passed, 269 skipped. The final Blob-specific additions were also verified separately against the rebuilt extension. The Python enablement PR depends on this core change. No additional dependencies, formats or compaction capabilities are introduced. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
