JingsongLi opened a new pull request, #1140: URL: https://github.com/apache/paimon-rust/pull/1140
## Purpose PyPaimon already exposes partition, bucket and file-statistics pruning counters in `ReadBuilder.explain()`, but Native plans currently return only snapshot and split metadata. The lightweight core `ScanTrace` cannot provide the complete funnel because normal scans skip manifests/Avro blocks and may stop constructing splits at LIMIT. ## Changes - Add explicit `TableScan::plan_with_stats()` and combined incremental statistics planning in Rust core, returning the plan and read-only metadata-entry counters from one pass. - Count partition filtering before bucket filtering, consistent with Java's entry filtering order. Include schema-aware ADD and DELETE statistics before manifest netting, and count selected Data Evolution file references before LIMIT. - Keep ordinary `plan()` and lightweight `plan_with_trace()` optimizations, including early bucket/range/sidecar pruning and LIMIT stopping. - Expose the core result through thin Python bindings. Collecting complete statistics reads more manifest entries and does not open data files. These counters describe metadata entries rather than visible rows. This adds no formats or read capabilities that PyPaimon does not already support. ## Verification - `cargo test -p paimon --lib table::table_scan`: 134 passed. - Python binding `tests/test_read.py`: 151 passed. - Companion PyPaimon explain, REST Native statistics, authorization, incremental, distribution and Native integration tests: 276 passed, 4 skipped, 115 subtests passed with Native planning/reading enabled. - `cargo +1.98.0 fmt --all -- --check`. - `cargo +1.98.0 clippy --locked --all-targets --workspace --features fulltext,vortex -- -D warnings`. Regression coverage includes zero/positive LIMIT, empty/pinned/historical snapshots, partition plus bucket pruning, cross-schema ADD/DELETE netting, sparse Data Evolution projections, incremental Delta/Changelog plans, captured authorization policies, and full 1000-entry statistics while ordinary sidecar planning still decodes fewer entries. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
