JingsongLi opened a new pull request, #10691: URL: https://github.com/apache/paimon/pull/10691
## What is the purpose of the change? Depends on https://github.com/apache/paimon-rust/pull/1183. Native CI continues to build Rust main; this PR stays draft until that dependency is merged. Delegate Native split execution to Rust Core. PyPaimon retains public option resolution, split/schema transport and output adapters, while Rust owns split grouping, bounded concurrency, ordering, global LIMIT, BLOB fan-out and cancellation. ## Brief change log - Call the Native reader once with all splits and the resolved execution options. - Remove the Native Python worker pool, queues, byte-balancing algorithm and shared row-quota implementation (about 300 lines of product code). - Preserve ordered materialization and InternalRow iteration; streaming batches can be emitted by whichever split group is ready first. - Propagate early close to the Core stream for serial as well as parallel Native readers. - Preserve Pure reader behavior and setup-fallback resource limits. Consumption errors still surface without replaying the read through Python. - Replace tests of private Native Python scheduling with adapter-contract regressions; scheduling and quota behavior are covered by Rust Core tests. ## Tests - Targeted Python read, plan, parallel and BLOB/sliced-read suites: 211 passed, 25 skipped, plus 44 subtests. Native-only integration cases were skipped because the extension is absent locally. - InternalRow iterator adapter mocks: 11 passed. - Changed Python files pass Flake8, Python 3.6 syntax checks and `git diff --check`. No Native extension was rebuilt or installed locally. CI continues to build Rust main, with no version probes or compatibility fallback for the new execution keywords. This PR will remain draft until the matching Rust Core PR is merged. The remaining Python write/update input adapters and commit lifecycle are outside this read-execution change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
