JingsongLi opened a new pull request, #10778: URL: https://github.com/apache/paimon/pull/10778
### Purpose Move the remaining distributed row-ID range lookup and deletion grouping behind execution dispatch, and finish placing the corresponding Python bucket execution implementations under `execution/python`. Public PyPaimon and Ray API signatures remain unchanged. ### Changes - Give captured row-ID plans matching Python and Native `route`, `route_by_bucket` and read-only `select` execution contracts. - Delegate Native routing to Core's existing `RowIdPlan` with Arrow inputs. Worker serialization drops only the SDK handle and reconstructs the same captured typed plan; it does not replan, reselect a backend or replay execution errors in Python. - Remove repeated NumPy range lookup from Ray read/update/delete callbacks and replace pickle/base64 deletion keys with typed per-partition/bucket routing IDs. Keep Ray scheduling and existing worker authorization, liveness checks and commit messages. - Keep complete file groups, deletion vectors, authorization, paths and snapshot metadata during read selection. - Match Core's normal-anchor ownership when a Blob/vector file spans multiple groups: attach it to overlapping groups without joining anchors or filling row-ID gaps. Preserve planning work and LIMIT statistics while copying and refreshing the selected plan's final output counters, including after pickle. - Move Python-only bucket hashing, HASH index restoration and stateful assignment into `execution/python`. Keep shared limits, validation and read-view configuration outside execution. Update internal imports and packaging without compatibility reexports or SDK version probes. ### Verification - **5 real Ray cases passed**: cross-file reads and updates, updates with deletion vectors, partial-column evolution files and Blob reads. - **295 current-SDK integration tests passed**, exercising routing, plan serialization, captured-snapshot updates/deletes, selected reads and existing distributed bucket execution. No Native extension was installed or rebuilt for these Python tests. - Independent Python row-ID regression collection: **87 passed, 4 skipped** without an installed SDK; those Native paths passed in the joint collection. Bucket regressions: **158 passed** (collections overlap). - Actual wheel-content validation passed. Changed-file Flake8 and diff checks passed. Depends on https://github.com/apache/paimon-rust/pull/1209. This PR will remain draft until the companion Rust API reaches main. Native CI continues to build Rust main; no CI pin or compatibility fallback is added. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
