JingsongLi opened a new pull request, #10757: URL: https://github.com/apache/paimon/pull/10757
## Purpose Complete the plan-execution and structured-conflict interfaces needed to run existing Ray row-ID mutations and vector index builds through Rust Core, while preserving Python public APIs. Depends on apache/paimon-rust#1202. This PR remains draft until that dependency lands. Native CI continues to build Rust main; no SDK compatibility probes, Python replay, or CI branch override is added. ## Changes - Capture the resolved snapshot and complete logical file groups once. Ray update, delete, self-MERGE update and row-ID read routing share neutral serializable metadata, instead of explicitly constructing Python file indexes in the Native path. - Route captured update/delete plans through dispatch to the new typed SDK `Plan` / `with_plan` interfaces. Preserve the worker's current environment and the captured read view. Preparation/execution authorization failures propagate without replay. - Keep full physical groups for mutations when scalar-index planning returns an IndexedSplit. Preserve QueryAuth wrappers, deletion vectors and canonical groups. Fill only missing canonical bucket paths while retaining external file locations, matching Java's split/file path model. - Execute Core-planned vindex shards with Ray. Each worker builds exactly one captured shard; publication stays with the existing caller. Collect messages in plan order. On worker or result-delivery failure, drain dispatched tasks and retain their outputs. - Share structured row-ID conflicts between execution backends, including pickle support and current DELETE layout fields. Native translates only the actual typed layout conflict; opaque/unknown publication errors remain errors. - Delegate the pre-rewrite logical history check to the selected backend without creating a Python publisher. Preserve Java's distinction between compactions and logical DML conflicts. - Keep existing public constructors and methods, explicit Python execution, and the existing commit ownership rules. No new dependencies. ## Verification - Python 3.13 / Arrow 19: affected Ray, protocol, authorization and execution-boundary aggregate: **267 passed, 3 skipped**. - Final plan/wire/conflict protocol suite after external-path coverage: **71 passed, 3 skipped**; backend-selection suite: **14 passed**. - Arrow 16 protocol checks passed; changed Python files pass flake8 and Python 3.6 grammar checks. - Paired Rust Core tests, actual PyO3 interface/error tests, and Rust 1.98 Core/binding Clippy passed. The three skipped cases are real SDK integration tests for new plan interfaces. The local Python environment has no new SDK, and the extension was not rebuilt or installed. These cases are included for Native CI after the Rust dependency merges. Existing real Ray regressions and backend-independent protocol tests ran locally; mocked transport tests are not counted as real SDK execution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
