JingsongLi opened a new pull request, #10454: URL: https://github.com/apache/paimon/pull/10454
## Purpose Enable Native execution for PyPaimon's existing raw BLOB, ARRAY<BLOB> and MAP<X, BLOB> update capabilities on data-evolution tables. Depends on https://github.com/apache/paimon-rust/pull/1081. This is a draft until that core implementation lands. Native CI continues to build Rust main; it is not redirected to a fork or a feature branch. ## Changes - Remove the raw-Blob update eligibility restriction while retaining the packed-video restriction. - Use existing core APIs for direct/grouped/incremental row-ID updates, predicate assignments, key upserts and MERGE operations. - Transport row Blob objects as lazy, operation-local URI sources. Rust owns copying, record serialization, rolling and partial-file metadata. Python does not materialize Blob streams. - Preserve Java's nullable map keys through a list-of-entry-structs transport; core writes the Java map record layout and validates key uniqueness. - Retain the existing row-aware upsert path for custom Blob streams, so overwritten input streams are never opened before key deduplication. - Preserve prepared commit files on abort and later failures. ## Tests 181 Native Blob update tests cover scalar/array/string-key map/integer-key map values, batch/stream operations, row and Arrow inputs, predicates, upserts, MERGE, explicit NULL versus unchanged placeholders, independently rolled files, schema addition/drop-and-add, cross-column overlap, streaming failures/retry and prepared-file ownership. Tests block Python update fallbacks where Native execution is expected. Nullable-key files are decoded by the existing Python Blob reader as an independent format check. Local verification with the companion Rust extension: - Native CI test scope: **9270 passed, 275 skipped**, including non-zero counters for every update family. - Classic CI test scope: **6648 passed, 269 skipped**. - Final Blob-specific suite: **181 passed** against the rebuilt companion extension. - flake8 on all changed Python files and git diff --check. The classic local run used a temporary, uncommitted pytest hook to reset tqdm's inherited lock after fork on macOS. CI configuration and test coverage are unchanged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
