JingsongLi opened a new pull request, #10753: URL: https://github.com/apache/paimon/pull/10753
### Purpose Keep the public write/update APIs focused on configuration, validation and delegation. Cover direct row-ID update, row-ID delete and shard-updater construction with the existing Native implementation, and use Rust's writer metadata for postpone bucket planning. ### Changes - Select and cache an execution backend before direct TableUpdateByRowId, TableDeleteByRowId and ShardTableUpdator construction loads or rewrites target files. Rust already provides these mutation APIs. - Remove Pure schema/routing/file-rewrite/assignment private-method forwarding from the public writers, updates, upserts and row-ID classes. Python executors invoke their own private helpers. Row-ID update/delete executors own their captured index and staging state. - Preserve existing public method signatures, class paths, callbacks and public subclass/instance hooks, including write_arrow_batch overrides used by Arrow and Pandas input. - Once Python is selected, keep nested updater/deleter resources in Python, including all local MERGE actions. Avoid a second Native preparation after row-ID factory fallback. Execution failures are never replayed in another backend. - Route direct PostponeBucketPlanner metadata loading through dispatch. Native uses the Core write builder's bucket_plan() getter, preserving writer access without adding SELECT authorization. Read the Arrow columns positionally so a partition named total_buckets remains valid. - Preserve Ray's broadcast precomputed indexes/anchor ranges, their captured snapshots and the existing no-per-worker-manifest-rescan behavior. Those precomputed worker paths remain explicitly Python; this PR does not claim Native support for transporting that private index. - Keep public validation, shared Arrow/row inputs, bucket arithmetic, transport metadata and resource helpers outside execution. Update the execution-boundary documentation. ### Dependency The writer-owned bucket_plan getter requires https://github.com/apache/paimon-rust/pull/1200. Keep this PR draft until that dependency merges. Native CI continues building Rust main; no SDK-version probes or compatibility shims were added. ### Tests - Final affected-module + write-suite verification: 74 modules, 1766 passed and 3055 skipped. The distributed input module ran in a separate process: 6 passed. Total: 1772 passed, 3055 skipped across 75 modules. - PyArrow 16 focused compatibility/shard/backend-selection regressions: 93 passed. - Added real round trips for writers without public private-method hooks, selected-Python update/upsert/delete/shard/MERGE operations, fresh-process Native import isolation, failure-without-replay, single preparation, captured metadata reuse and bucket-plan transport. - Native integration cases exercise the three direct mutation constructors and verify committed results without initializing Python executors. These extension-dependent tests are skipped locally. - Changed-file Flake8, Python 3.6 grammar checks (72 files), and git diff --check passed. - Independent cross-reviews found and repaired public batch-hook dispatch, repeated Native preparation, Pure MERGE backend reselection and duplicate Arrow field-name handling. No remaining actionable issue was found in the final passes. - No Native extension was installed or rebuilt locally. CI runs the Native integration tests after the Rust dependency merges. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
