costin commented on PR #16166:
URL: https://github.com/apache/lucene/pull/16166#issuecomment-4603191421
Thanks for the catch @romseygeek, the MatchAllDocsQuery filter was
unnecessary and was optimized away, so the old benchmark was effectively always
testing the plain query path.
I've updated the benchmark with a queryShape parameter:
- plain: standalone SortedNumericDocValuesField.newSlowRangeQuery: exercises
the new batch evaluation path directly
- conjunction: range query wrapped in a BooleanQuery with a real filter
(SortedNumericDocValuesField.newSlowRangeQuery("filter", 0, 0) matching ~50% of
docs): shows behavior when the range query is a confirming clause inside a
conjunction.
Results (AMD EPYC c5a.2xlarge, JDK 25, 1M docs, vpd=4, fixed cardinality,
dense)
### Plain query shape
| pattern | selectivity | baseline (ops/s) | candidate (ops/s) | speedup |
|---------|-------------|------------------:|-------------------:|--------:|
| clustered | 0.01 | 4,764 ± 324 | 7,104 ± 72 | **1.49x** |
| clustered | 0.1 | 4,769 ± 175 | 7,949 ± 111 | **1.67x** |
| clustered | 0.5 | 6,503 ± 136 | 19,230 ± 951 | **2.96x** |
| random | 0.01 | 34.4 ± 0.2 | 51.9 ± 1.0 | **1.51x** |
| random | 0.1 | 32.3 ± 0.4 | 45.5 ± 0.2 | **1.41x** |
| random | 0.5 | 36.5 ± 0.2 | 54.1 ± 1.8 | **1.48x** |
### Conjunction query shape
| pattern | selectivity | baseline (ops/s) | candidate (ops/s) | speedup |
|---------|-------------|------------------:|-------------------:|--------:|
| clustered | 0.01 | 6,004 ± 222 | 6,020 ± 41 | ~1.0x |
| clustered | 0.1 | 3,029 ± 119 | 3,120 ± 25 | ~1.0x |
| clustered | 0.5 | 995 ± 35 | 1,031 ± 31 | ~1.0x |
| random | 0.01 | 47.4 ± 0.5 | 48.1 ± 0.6 | ~1.0x |
| random | 0.1 | 45.3 ± 1.1 | 42.6 ± 0.5 | ~1.0x |
| random | 0.5 | 44.9 ± 0.9 | 50.5 ± 1.6 | ~1.0x |
The plain query shape shows improvements but the conjuction does not since
`ConjunctionDISI` still does per-doc evaluation. I can follow-up with a
separate PR after this one and #16146 get merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
