yihua opened a new pull request, #763:
URL: https://github.com/apache/hudi-rs/pull/763
## Description
The COW parquet scan now takes row-filter pushdown, filter reordering, and
page-index use from the session's parquet options instead of hardcoding them
on, so a Hudi table scan behaves like a plain parquet registration of the same
files under the same session config. The scan also only installs predicates for
conjuncts DataFusion's physical filter-pushdown rule cannot absorb itself:
pushable conjuncts reach the scan through that rule exactly once (for stats
pruning, and as a row filter when the session enables pushdown), so the merged
scan predicate now carries each conjunct once in both modes. Plan test anchors
are updated for the resulting plan shape (the struct-field access is still
computed inside the scan, under an optimizer-extracted name). Part of the query
analysis work in #713.
## How are the changes test-covered
- [ ] N/A
- [x] Automated tests (unit and/or integration tests)
- [ ] Manual tests
- [ ] Details are described below
New test pins that the filter stays in a `FilterExec` under default session
config, moves into the scan when `pushdown_filters` is enabled, and that the
scan predicate carries each conjunct exactly once in both modes (the once-only
assertions fail on the previous behavior). Existing plan tests keep pinning
struct-field predicate and projection pushdown; the full hudi-datafusion suite
passes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
