xiangfu0 opened a new pull request, #19513: URL: https://github.com/apache/pinot/pull/19513
## What `ValueBasedSegmentPruner` now overrides `prune(List, QueryContext, ExecutorService)` and prunes across the query executor when there are enough segments to be worth it. It previously implemented only the two-arg `prune`, so `SegmentPruner`'s three-arg default silently discarded the executor it was handed. `BloomFilterSegmentPruner` already overrode it and pruned in parallel — the pruner that runs *first*, over the full segment set, did not. The parallel path mirrors the bloom-filter one: strided tasks over the segment list, each with its own `ValueCache` and data-source cache (neither is thread-safe, and both are scoped to a single segment at a time regardless), run through `QueryMultiThreadingUtils.runTasksWithDeadline` so the query deadline and cancellation are honored. Below `TARGET_NUM_SEGMENTS_PER_THREAD`, or with no executor, it stays serial. `TARGET_NUM_SEGMENTS_PER_THREAD` moves to the parent; `BloomFilterSegmentPruner` was shadowing it with the same value. ## Why This loop runs on the calling thread for **every segment the server holds**, before any per-segment parallelism starts. On a server holding 44,780 segments it is the longest single-threaded stretch of the query — and the default pruner chain is `ColumnValueSegmentPruner, BloomFilterSegmentPruner, SelectionQuerySegmentPruner`, so the serial one runs first and on the largest input. Stacked on #19511, which removes the per-segment *work* in that loop (it was materializing a column to read min/max). This removes the serialisation. They are independent: either helps alone, and the combination is what makes pruning proportional to cores rather than to segments held. ## Behaviour change Segments come back in a different order than they went in. That is already true of `BloomFilterSegmentPruner`, which runs immediately after this pruner in the default chain, so nothing downstream can have depended on the input order surviving pruning. ## Tests `ColumnValueSegmentPrunerTest#testParallelPruningSelectsTheSameSegments` builds 40 segments — half matching the predicate, half prunable — and asserts the parallel path selects exactly the same set as the serial one. 52 pruner tests pass; spotless, checkstyle and license clean. ## Stack Part 12, based on #19511. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
