JingsongLi opened a new pull request, #1153: URL: https://github.com/apache/paimon-rust/pull/1153
### Purpose PyPaimon already supports scalar filters over flattened nested ROW projections, such as projecting `profile.detail.score` and filtering `profile_detail_score = 7`. The Native path currently cannot resolve that leaf and falls back to Python. Add nested ROW predicates to the core read pipeline, following the current Java `NestedFieldTransform`: retain the enclosing root as an input, resolve separate path components with stable field IDs, mask NULL ancestors, and avoid interpreting parent-column statistics or indexes as child metadata. ### Brief change log - Add `PredicateBuilder::nested` and a distinct `Predicate::NestedLeaf`; retain the original field identities through physical-file remapping. - Reuse exact scalar residual evaluation for nested comparisons, ranges, membership and string predicates, including parent-null masking and promoted fields. - Widen predicate ROW inputs and restore the requested nested projection after filtering in ordinary reads, schema evolution, row tracking, Data Evolution and managed BLOB reads. - Keep partition, bucket, file/global-index and parent-statistics pruning conservative for nested leaves; preserve exact filtering after primary-key merge. - Resolve optional `nested_path` in the Python predicate binding through the core builder and reuse existing typed literal conversion. ### Tests - Reproduced the baseline Native fallback while the existing Python read returned the matching row. - Core: 107 tests selected by `nested_`, 264 by `predicate`, 86 DataEvolutionReader tests, 38 DataFileReader tests, and 7 managed BLOB reader tests passed (these groups overlap). - Python/Native integration on the built extension: 970 passed, 2 skipped, 35 subtests passed. Assertions reject Python scan/read fallback; cover append/PK, batch/stream, NULL parents, scalar operators, projection changes, field promotion/rename/drop-readd, aliases colliding with real columns, partition/bucket pruning and Boolean composition. - `cargo +1.98.0 clippy --locked -p paimon -p pypaimon_rust --lib --tests -- -D warnings` and formatting checks passed. ### API and Format New core builder method and predicate variant. The Python filter dictionary accepts an optional `nested_path` list alongside the physical root `field`. No storage format changes. ### Documentation API documentation describes path components, field identity and NULL-parent semantics. Companion PyPaimon PR will retain physical projection paths and add Native regression coverage; its CI continues building Rust main. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
