JingsongLi opened a new pull request, #10637: URL: https://github.com/apache/paimon/pull/10637
### Purpose Keep existing PyPaimon nested ROW projection filters on Native scans and reads, rather than falling back when the projected leaf name is not a top-level table field. Depends on https://github.com/apache/paimon-rust/pull/1153. This PR remains draft until that core support is merged. Native CI continues building Rust main; no fallback or revision pin is introduced to bypass the dependency. The existing Python API is unchanged: project `profile.detail.score`, then filter `profile_detail_score` using the builder's predicate builder. ### Changes - Retain each list projection's physical source path on predicates, including top-level columns renamed by collision suffixes. Preserve paths through index/literal rewrites and serialization. - Pass physical roots and descendant paths to the Rust core predicate builder; Rust owns path validation, typed literals, exact evaluation and schema evolution. - Align Python with the current Java `NestedFieldTransform` behavior: nested leaves cannot prune using an unrelated parent/scalar column's stats, partition/bucket keys or indexes. - Fix `a.b` / real `a_b` collisions in Python exact filtering and Native transport. Preserve correct AND/OR pruning, row tracking and explicit partition-filter validation. ### Tests - Reproduced a real baseline Native fallback and guarded new Native tests against Python scan/read fallback. - New coverage spans append/PK tables, batch/stream reads, NULL ancestors, comparisons/string predicates/ranges/membership, projection changes and LIMIT, field promotion/rename/drop-readd, pickle, and collision cases with full statistics, separate files, partitioning, 8 buckets and AND/OR. - Combined Python/Native regression: 970 passed, 2 skipped, 35 subtests passed, with 993 Native plans and 1159 Native reads exercised. - Additional review regressions: 96 collision cases, 26 partition tests and 354 existing predicate/projection/auth/index/bucket tests passed (groups overlap). - Flake8 and `git diff --check` passed. Native tests ran against the built companion extension; extension rebuilds were completed before starting Native test runs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
