Gabriel39 opened a new pull request, #65965:
URL: https://github.com/apache/doris/pull/65965
### What problem does this PR solve?
Issue Number: N/A
Related PR: #65921
Problem Summary:
The V2 Parquet scanner already evaluates single-column predicates round by
round, but a multi-column residual (including the children of a compound `AND`)
still caused every predicate column to be materialized before expression
evaluation. Expression short circuiting therefore happened after Parquet
decode/IO and could not avoid later-only columns.
This PR follows expression-triggered lazy materialization:
- preserve conjunct order and split safe compound `AND` residuals into
ordered expression stages;
- record each stage's slot dependencies and materialize only the next
reachable columns;
- read later columns with the surviving selection, or skip them entirely
when an earlier stage rejects the batch;
- prefetch only the first reachable predicate stage;
- retain the original eager path for stateful/error-sensitive expressions
that are unsafe on selected rows;
- add unit coverage and a Release microbenchmark scenario based on the
Parquet benchmark framework from #65921.
Correctness/counter validation (6 input rows):
- an earlier two-column `AND` child filters all rows: `ReaderReadRows=12`,
`ReaderSkipRows=6` (the third column is never decoded);
- when 3 rows survive the first residual: the later column reads only those
3 rows (`ReaderReadRows=15`, `ReaderSelectRows=3`, `ReaderSkipRows=3`).
Release microbenchmark:
```text
CPU: Intel Xeon Platinum 8457C, pinned to CPU 8
Build: RELEASE
Filter:
^ParquetReader/complex_residual_scan/plain/null_10/alternating/sel_10/
Protocol: 3 warmups, then A-B-B-A; 5 repetitions per group; min_time=1s
Pair 1 median CPU: before 1,377,041 ns; after 1,318,811 ns (-4.23%)
Pair 2 median CPU: before 2,140,937 ns; after 2,124,862 ns (-0.75%)
Paired geometric normalization: -2.50% CPU time
Selected rows: 1,460 in every run
```
The host had high concurrent load and CPU scaling enabled, so the paired
groups and run conditions are reported explicitly rather than relying on wall
time. Both A/B pairs improve in the same direction.
### Release note
None
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- [x] Manual test (Release microbenchmark above)
- [ ] No need to test or manual test.
- Behavior changed:
- [x] No. The SQL result is unchanged; only predicate-column
materialization timing changes.
- [ ] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
