LuciferYang opened a new pull request, #2906:
URL: https://github.com/apache/iceberg-rust/pull/2906
## Which issue does this PR close?
- Closes #2432.
## What changes are included in this PR?
A predicate on a primitive leaf inside a struct (e.g. `person.age > 25`)
fails while building the Arrow row filter with `Leaf column ... isn't a root
column in Parquet schema`. `PredicateConverter::bound_reference` resolved the
field to its Parquet leaf index and rejected it when that leaf's column root
was a group, and `project_column` only handled top-level columns.
This resolves the referenced field to its Parquet column path (root to leaf)
instead of a single leaf index, and has `project_column` walk that path through
the projected `RecordBatch`, descending `StructArray` children by name to reach
the primitive leaf. `ProjectionMask::leaves` preserves the Parquet nesting, so
a predicate on `nested.value` sees `nested` as a `StructArray` holding the leaf.
### Scope
Only top-level and struct-nested primitive leaves can carry a predicate:
`Schema::build_accessors` builds no accessor for list/map interior fields, so
`Reference::bind` rejects them before the row filter is ever built. Every path
segment before the leaf is therefore a struct. This matches the Java reader,
which reads a struct-nested field normally.
Row-group and page-index pruning (`RowGroupMetricsEvaluator` /
`PageIndexEvaluator`) resolve nested leaves by field id and never errored on
this path; they are out of scope here and can be looked at separately.
## Are these changes tested?
- `predicate_visitor`: unit coverage for the nested-leaf conversion (a
`group person { int32 age }` Parquet schema, the shape that used to be
rejected).
- `row_filter`: end-to-end reads of real Parquet files for a single-nested
(`person.age`) and a double-nested (`person.address.zip`) struct leaf, so the
projected `RecordBatch` shape comes from arrow-rs rather than a hand-built
batch; and a bind-time check that predicates on list-element and map-value
fields are rejected before reaching the row filter.
All three predicate tests fail on the pre-fix code and pass after. `cargo
test -p iceberg --lib`, `cargo fmt --check`, and `cargo clippy --all-targets
--all-features -- -D warnings` all pass. No public API signature changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
