JingsongLi opened a new pull request, #1197:
URL: https://github.com/apache/paimon-rust/pull/1197
### Purpose
Allow PyPaimon's existing file-scanner API to use Rust Core rather than
constructing a Python executor. Preserve the separation between raw manifest
metadata and query results.
### Brief change log
- Add `TableScan::plan_files` and `FileScanPlan` for complete manifest
entries and the selected snapshot ID. File metadata planning precedes
projection, LIMIT, row selection, global-index refinement and split
distribution.
- Use the existing snapshot/time-travel selection and share incremental AUTO
event-source selection with row planning. Keep merged ADD/DELETE entries,
historical schema statistics, data-evolution column files and Blob metadata.
- Authorize all columns before reading metadata, require an unrestricted
whole-table grant and recheck REST table identity after planning, including
empty results.
- Add independent level-zero and pending-bucket policies for the existing
Python scanner controls. Unconfigured queries retain their current defaults.
Explicit L0 reads keep overlapping primary-key versions together through split
packing and sharding.
- Expose typed file plans and existing `ManifestEntry` values through the
Python bindings. No custom Python manifest-source or follow-up-scanner callback
is added.
### Tests
- Rust 1.98.0 targeted runtime tests: **16 passed, 0 failed**.
- Core raw file metadata planning: 3 passed.
- Explicit L0 key merging and sharding for FirstRow and deletion-vector
deduplication, including pending buckets: 1 passed.
- Incremental timestamp scans: 7 passed.
- REST raw file planning and writer authorization: 4 passed.
- Python binding typed metadata transport and independent policies: 1
passed.
- `cargo +1.98.0 clippy --locked --all-targets --workspace --features
fulltext,vortex -- -D warnings`: passed.
- `cargo +1.98.0 fmt --all -- --check` and `git diff --check`: passed.
- No Native Python extension was built or installed for these tests.
### API and Format
Adds `TableScan::plan_files`, `TableScan::with_file_filters`, `FileScanPlan`
and corresponding Python bindings. No storage-format change or new
configuration option.
### Documentation
Rust API documentation and Python type stubs describe the file-planning and
authorization boundary. PyPaimon's integration is in
https://github.com/apache/paimon/pull/10742.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
