somu-imply commented on a change in pull request #12195:
URL: https://github.com/apache/druid/pull/12195#discussion_r800292972
##########
File path:
sql/src/main/java/org/apache/druid/sql/calcite/run/NativeQueryMaker.java
##########
@@ -122,6 +127,32 @@ public boolean feature(QueryFeature feature)
);
}
}
+ int numFilters =
plannerContext.getPlannerConfig().getMaxNumericInFilters();
+
+ // special corner case handling for numeric IN filters
+ // in case of query containing IN (v1, v2, v3,...) where Vi is numeric
+ // a BoundFilter is created internally for each of the values
+ // whereas when Vi s are String the Filters are converted as BoundFilter
to SelectorFilter to InFilter
+ // which takes lesser processing for bitmaps
+ // So in a case where user executes a query with multiple numeric INs,
flame graph shows BoundFilter.getBitmapResult
+ // and BoundFilter.match predicate eating up processing time which stalls
a historical for a query with large number
+ // of numeric INs (> 10K). In such cases user should change the query to
specify the IN clauses as String
+ // Instead of IN(v1,v2,v3) user should specify IN('v1','v2','v3')
+ if (numFilters != PlannerConfig.NUM_FILTER_NOT_USED) {
+ if (query.getFilter() instanceof OrDimFilter) {
Review comment:
I am thinking in lines of a very simple function that can traverse the
entire filter tree would look something like
```
private int countFilters(DimFilter filter)
{
int count = 0;
if (filter instanceof BoundDimFilter) {
count += 1;
} else if (filter instanceof OrDimFilter) {
for (DimFilter e : ((OrDimFilter) filter).getFields()) {
count += countFilters(e);
}
} else if (filter instanceof AndDimFilter) {
for (DimFilter e : ((AndDimFilter) filter).getFields()) {
count += countFilters(e);
}
}
return count;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
