bobhan1 opened a new pull request, #70558: URL: https://github.com/apache/doris/pull/70558
### Background Query page read-ahead builds column byte windows after pruning, reads merged file ranges asynchronously, and serves the original page slices to PageIO. The current implementation coordinates all columns through `SegmentReadAhead`; its FileReader adapter retains column-window pointers and updates them when buffered pages are consumed. Consumed ranges can enter the existing File Cache writeback and hole-fill pipeline. ### What problem does this PR solve? Issue Number: None Related PR: #67292 (introduced query page read-ahead) PR Type: Refactor Problem Summary: Move page prediction and range submission into a controller owned by each eligible `FileColumnIterator`. Callers pass row selections, filtering dependencies and scan direction, allowing each column to maintain its own window. PageIO receives an explicit optional buffer source, eliminating the specialized FileReader and its callbacks into column windows. Regular scans, point-query missing-column reads and row-store reads use the same services and iterator interface. ### Details - **Change map:** 30 files; production code +838/-939 lines and tests +805/-1211 lines. There are no generated files or documentation changes. The production changes comprise the column controller/resources, the IO slice registry, and scan/point-query/PageIO integration. Tests migrate the previous coordinator contracts and add ownership and cache-race coverage. - **Moves vs. logic changes:** `ColumnReadAheadController` owns per-column metadata preparation, window advancement, Page Cache probes, range planning and submission. `ReadAheadRangeBuffer` owns exact slices of scheduled ranges and their once-only consumption notification. `ColumnReadAheadResources` shares the original reader, query context, configuration and writeback factory without coordinating columns. Remove `SegmentReadAhead`, its FileReader adapter and column-owner pointers. Reuse the existing window, range planner, scheduler, hole-fill and cache-writer algorithms. - **Behavior change:** Each column independently merges and submits its own pages. Cross-column GET coalescing and shared exact-page registrations are removed, so request count and admission order can change. Preserve query/BE byte budgets, query cancellation, original page boundaries, page-driven window advancement, synchronous fallback and the existing pre-checksum consumed-range notification. Point-query columns still submit before decoding starts. Prefetch eligibility and legacy-prefetch exclusion are handled at the iterator boundary; complex columns and extracted Variant paths retain their existing readers. `plan_calls` counts nonempty per-column plans. - **Compatibility:** No storage-format, RPC, configuration or default-value changes. ColumnReader remains shared metadata; query-specific windows and buffers stay with each iterator. The existing Cloud/query and dual-enable-switch requirements remain in effect. - **Performance impact:** Candidate pages are prepared once per scan column and subsequent physical page accesses advance its window. Cross-column merge removal can increase GET count; cluster and object-store performance measurements are pending. - **Rollback:** Revert this commit. ### Testing - Production ASAN/PCH build (`CMAKE_BUILD_TYPE=ASAN`, `MAKE_TEST=OFF`, `ENABLE_PCH=ON`): `./build.sh --be -j100` passed, including the glibc 2.17 compatibility gate. - Targeted ASAN/PCH BE tests: **234 tests in 17 suites passed**, with no failures or skips. Covers window planning, real compressed-page reads, reverse batches, Page Cache hits after planning, partial admission, checksum fallback, an in-flight read outliving its destroyed column controller, shared query budgets/cancellation, point-query switch fallback, count emission, TopN range refresh, and cache writeback. ```bash ./run-be-ut.sh --run -j100 --filter='ColumnReadAheadTest.*:ColumnReaderTest.*:SegmentIteratorReadAheadTest.*:RowIdReadAheadTest.*:ReadAheadRangeBufferTest.*:FileRangeReadSchedulerTest.*:FileRangePlannerTest.*:AsyncCachedRemoteFileReaderTest.*:*PointQuery*:*CountEmit*:SegmentIteratorTopNRuntimeFilterTest.*:QueryContextReadAheadTest.*:RangeCacheWritebackTest.*:RangeWritebackDispatcherTest.*' ``` - Repository clang-format 16, `check-format.sh`, `check-build-hygiene.sh` and `git diff --check` passed. - No cluster or performance run. Clang-tidy was not completed. ### Notes for Reviewers Start with `ColumnReadAheadController`, `ReadAheadRangeBuffer`, and the before/after hooks in `FileColumnIterator::_read_data_page()`. The default-disabled iterator eligibility is set by `Segment::new_column_iterator()` so a scalar physical child or extracted Variant leaf does not automatically enter the new path. ### Release note None ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: No new public API or configuration. - Pre-submission status: Partially verified: production build and targeted tests pass; cluster behavior and performance remain unverified. ### AI Disclosure - Tool: Codex (GPT-6) - Generated: implementation, tests and this description - Human review: Pending -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
