JingsongLi opened a new pull request, #1086: URL: https://github.com/apache/paimon-rust/pull/1086
## Summary PyPaimon already supports source-backed primary-key BTREE and BITMAP indexes. Its Native scan currently plans data files in Rust, then loads and evaluates these indexes in Python. This change moves that refinement into Rust core without adding binding APIs. - Resolve scalar index definitions and per-column reader options using Java's `PrimaryKeyIndexDefinitions` and `CoreOptions` rules. - Validate compacted source coverage per partition, bucket, field and data level, following `PkSortedBucketIndexState`. Preserve retired-source offsets, leave uncovered files readable and reject ambiguous payload coverage. - Reuse the existing scalar index scanner, cache group queries and localize positions before combining fields. Preserve entire merge-required splits so an old matching version cannot replace a newer version. - Preserve deletion-file alignment and bucket-local/external paths. Invalid positions, excessive fragmentation and index read failures fall back to reading the data file. - Interpret KV IndexedSplit ranges as physical file positions, including files carrying stable row IDs, as in Java's KV reader. The automatic refinement applies to ordinary batch reads. Streaming, incremental, write-side scans and query-auth planning retain their existing paths. This adds no index-building or compaction functionality. ## Validation - 322 targeted Rust unit tests passed across PK sorted indexes, global index scanning, table scanning, KV reading, table reading and file reading; one existing test ignored. - `cargo +1.98.0 clippy --locked --all-targets --workspace --features fulltext,vortex -- -D warnings` passed. - `cargo fmt --all --check` passed. - 15 REST Native interoperability tests passed against Java-generated BTREE/BITMAP files, including time travel, compound predicates, reader limits/shards, unavailable payloads, tracked row IDs and a newer nonmatching PK version. - Independent core/integration reviews and a test mutation audit completed; identified owner/position boundary test gaps were added. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
