xiangfu0 commented on code in PR #19539:
URL: https://github.com/apache/pinot/pull/19539#discussion_r4114884539
##########
pinot-broker/src/main/java/org/apache/pinot/broker/routing/segmentpruner/SinglePartitionColumnSegmentPruner.java:
##########
@@ -105,6 +125,35 @@ public Set<String> prune(BrokerRequest brokerRequest,
Set<String> segments) {
return selectedSegments;
}
+ private Set<String> pruneWithPreparedPredicate(Expression filterExpression,
Set<String> segments) {
+ Set<String> selectedSegments = new HashSet<>();
+ List<PreparedPredicate> predicates = new ArrayList<>();
+ for (String segment : segments) {
+ SegmentPartitionInfo partitionInfo = _partitionInfoMap.get(segment);
+ if (partitionInfo == null || partitionInfo ==
SegmentPartitionUtils.INVALID_PARTITION_INFO) {
+ selectedSegments.add(segment);
+ continue;
+ }
+ PartitionFunction function = partitionInfo.getPartitionFunction();
+ int functionHashCode = function.hashCode();
+ PreparedPredicate predicate = null;
+ for (PreparedPredicate candidate : predicates) {
Review Comment:
I tried the whole-predicate ID-set approach in a temporary full-prune JMH
variant (256 segments, one or two functions; 3x1s warmups, 5x1s measurements,
two forks on JDK 25). It helps wide filters: homogeneous IN(16) went from 5.70
to 3.10 us/op and OR(4) from 6.43 to 2.73. But it regressed the common EQUALS
case from 2.46 to 2.99 (+21%) and early-hit IN(128) from 5.86 to 6.70 (+14%;
two functions: 6.49 to 9.56, +47%). Omitting rejected literals also changes the
existing short-circuit/error behavior (covered by the invalid-number tests);
preserving it needs a fallback path. Given those regressions and the extra
path, I kept lazy preparation for this minimal PR. No benchmark source is
committed.
##########
pinot-broker/src/main/java/org/apache/pinot/broker/routing/segmentpruner/SinglePartitionColumnSegmentPruner.java:
##########
@@ -105,6 +131,28 @@ public Set<String> prune(BrokerRequest brokerRequest,
Set<String> segments) {
return selectedSegments;
}
+ private Set<String> pruneWithPreparedPredicate(Expression filterExpression,
Set<String> segments) {
+ Set<String> selectedSegments = new HashSet<>();
+ Map<PartitionFunction, PreparedPredicate> predicates = new HashMap<>();
Review Comment:
Implemented the small query-local list, promoting to HashMap after eight
distinct functions (6f6e8de). In the full 256-segment prune benchmark, the
one-function cases improved from 4.31 to 2.46 us/op for EQUALS, 8.23 to 5.70
for IN(16), and 4.94 to 2.47 for AND(4); two-function EQUALS improved from 4.63
to 2.99. Homogeneous EQUALS allocation fell from 688 to 552 B/op. A focused
test exercises nine distinct functions and verifies ID reuse after promotion.
These are local JDK 25 measurements, not a claim about every workload.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
