a34729t opened a new pull request, #19280: URL: https://github.com/apache/pinot/pull/19280
Per https://github.com/apache/pinot/issues/19279, it would be convenient to have a query parameter that disables the IN clause pruning threshold. ------- ## Summary * Add forceInPredicatePruning query option to force IN-predicate segment pruning regardless of IN-clause size, overriding the server inpredicate.threshold (default 10) on a per-query basis ## Usage Added a per-query boolean query option `forceInPredicatePruning`: ```sql SET forceInPredicatePruning=true; SELECT ... WHERE id IN (a, b, ..., 500 values) ``` When true, the pruners always attempt to prune segments for IN predicates, regardless of the clause size. Default is false, preserving existing behavior. _Note that pruning is per-segment, so if we are testing on for example a single segment, there is no pruning. In a larger cluster however pruning will be observed._ ## Testing Unit tests cover: * `QueryOptionsUtils` getter with and without the option * End-to-end pruning of a >10-value IN list in both` ColumnValueSegmentPrunerTest` and `BloomFilterSegmentPrunerTest`. Also tested in my local Pinot cluster setup and ran a few queries to validate query plans and behavior (both SSE and MSE). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
