viirya commented on issue #5699: URL: https://github.com/apache/arrow-rs/issues/5699#issuecomment-3047584029
> The `ProjectionMask` includes all leaf columns (so the `[aa, bb, cc]` in the example above can be selected in the predicate) > However, I think they would come back as their own columns (and I am not sure how nullability would work, etc) > The first step for this ticket is probably to work up an example: > 1. Write a struct array with multiple fields into a parquet file > 2. Write a test that tries to apply an ArrowPredicate to only one field and see what happens Hmm, I thought that I provided enough context in the description, but seems there is still confusion on this issue. This issue originated from some works on iceberg-rs, specially this is the discussion around nested type https://github.com/apache/iceberg-rust/pull/295#discussion_r1547592586. If to avoid read the discussion, in short, although we can use `ProjectionMask` to access leaf columns, you cannot get "flatten" values in `RowFilter` API. In other words, you still get nested values where the nested struct is pruned to include the leaf columns. To verify how the API works and if it fits for the filter pushdown work, I wrote a test https://github.com/apache/arrow-rs/pull/5600 at that time. I think the test does what @alamb asked. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
