mrhhsg opened a new pull request, #70133:
URL: https://github.com/apache/doris/pull/70133
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: After predicate evaluation, segment scans always read
deferred physical output columns by selected row IDs, even when very few
candidate rows were filtered. This can make low-filtering scans perform
unnecessary sparse reads. Add a forwarded `lazy_read_ratio` session variable
(0–100, default 50). For eligible query scans, read output columns in batches
when the filtering ratio does not exceed the threshold and a contiguous
candidate group is available, then compact them using the final row selection.
Retain selected-rowid reads when the threshold is exceeded, a pushed-down LIMIT
reduces the result, or every candidate chunk is sparse. Non-query scans and
scans with virtual columns retain the existing read path. The condition cache
continues to mark only final selected rows.
### Release note
Add `lazy_read_ratio` to control adaptive reads of physical output columns
in eligible segment scans. LIMIT-truncated and wholly sparse candidate batches
avoid unnecessary full-batch output reads.
### Check List (For Author)
- Test: Unit Test and Regression test
- ASAN BE build: `BUILD_TYPE=ASAN ./build.sh --be -j 32` — passed.
- FE build and Checkstyle: `BUILD_TYPE=ASAN ./build.sh --fe` — passed.
- Focused SegmentIterator BE unit tests — 39/39 passed.
- `./run-fe-ut.sh --run org.apache.doris.qe.VariableMgrTest` — 17/17
passed.
- Targeted SQL regression with a worktree-specific configuration:
segment output ratio, nested-column pruning, and condition-cache LIMIT suites —
3/3 passed. The segment output expected results were generated by the
regression runner and then verified without generation.
- Clang formatting/check, build hygiene, changed-line clang-tidy, and
`git diff --check` — passed.
- No local performance benchmark, full CI run, or mixed-version runtime
test was performed.
- Behavior changed: Yes (eligible scans select the output read mode per
batch; LIMIT-truncated or wholly sparse batches use final selected row IDs).
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
