rajat315315 commented on PR #16431:
URL: https://github.com/apache/lucene/pull/16431#issuecomment-5121440072
I have performed benchmarking on wikimedia dataset using luceneutil and
below are the findings.
I have used a different query dataset than the one it uses as I didn't want
to run tests on Fuzzy queries and other types.
I was focused on Function Queries where results are modified by the function
that the user defines, more specifically monotonically increasing or decreasing.
The benchmark evaluates **20 distinct query permutations** formed by
combining **4 scoring function categories** with **5 query template
categories** across 100 query instances per category (total: 1,936 task
iterations across 5 JVM processes).
### A. Scoring Function Categories
1. **`Inc_*` (Monotonically Increasing Function)**
* **Formula**: $f(x) = x$ on numeric field `lastMod_skipper`.
* **Skipper Bound**: Upper bound evaluated from `skipper.maxValue(0)`.
2. **`DecNeg_*` (Monotonically Decreasing Negation Function)**
* **Formula**: $g(x) = -x$ on numeric field `lastMod_skipper`.
* **Skipper Bound**: Upper bound evaluated from `skipper.minValue(0)`.
3. **`DecInv_*` (Monotonically Decreasing Inverse Function)**
* **Formula**: $g(x) = \frac{100,000}{x}$ on numeric field
`lastMod_skipper`.
* **Skipper Bound**: Upper bound evaluated from `skipper.minValue(0)`.
4. **`TwoFields_*` (Multi-Field Composite Function)**
* **Formula**: Combines values from two fields (`lastMod_skipper +
dayOfYear_skipper`).
### B. Query Template Categories
1. **`HighTerm`**: Single-term query matching high-frequency corpus words
(e.g., `ref`, `title`, `name`, `cite`, `year`).
2. **`MedTerm`**: Single-term query matching medium-frequency technical
words (e.g., `algorithm`, `database`, `vector`, `search`, `lucene`).
3. **`LowTerm`**: Single-term query matching low-frequency rare words (e.g.,
`quantum`, `qubit`, `singularity`, `hologram`).
4. **`AndHighHigh`**: Conjunctive boolean query (`AND`) between two
high-frequency terms (e.g., `body:ref AND body:title`).
5. **`OrHighHigh`**: Disjunctive boolean query (`OR`) between two
high-frequency terms (e.g., `body:ref OR body:title`).
---
## 3. Detailed Benchmark Performance Results
| Row Name (Permutation) | Query Type Description |
Scoring Function | Baseline QPS (`main`) | Candidate QPS (`feature/...`) | QPS
Diff (%) | P100 Max Latency Reduction |
| :----------------------------| :--------------------------------------|
:------------------| :---------------------:| :-----------------------------:|
:------------:| :--------------------------:|
| **`DecNeg_HighTerm`** | Single High-Frequency Term Query |
$g(x) = -x$ | 157.67 | **215.48** |
**+36.7%** | -20.4% |
| **`Inc_MedTerm`** | Single Medium-Frequency Term Query |
$f(x) = x$ | 355.48 | **485.73** |
**+36.6%** | -69.4% |
| **`DecInv_OrHighHigh`** | Boolean OR Query (`term1 OR term2`) |
$g(x) = 100k / x$ | 52.03 | **69.62** |
**+33.8%** | -19.1% |
| **`Inc_LowTerm`** | Single Low-Frequency Term Query |
$f(x) = x$ | 541.12 | **722.21** |
**+33.5%** | **-40.6%** |
| **`Inc_OrHighHigh`** | Boolean OR Query (`term1 OR term2`) |
$f(x) = x$ | 42.70 | **55.23** |
**+29.3%** | -18.9% |
| **`TwoFields_AndHighHigh`** | Boolean AND Query (`term1 AND term2`) |
Multi-Field Sum | 57.95 | **72.49** |
**+25.1%** | **-39.0%** |
| **`DecInv_MedTerm`** | Single Medium-Frequency Term Query |
$g(x) = 100k / x$ | 277.27 | **344.99** |
**+24.4%** | +1.3% |
| **`DecNeg_OrHighHigh`** | Boolean OR Query (`term1 OR term2`) |
$g(x) = -x$ | 48.72 | **60.49** |
**+24.2%** | -12.6% |
| **`Inc_HighTerm`** | Single High-Frequency Term Query |
$f(x) = x$ | 100.48 | **124.42** |
**+23.8%** | +28.5% |
| **`DecNeg_AndHighHigh`** | Boolean AND Query (`term1 AND term2`) |
$g(x) = -x$ | 52.39 | **64.34** |
**+22.8%** | **-59.6%** |
| **`TwoFields_HighTerm`** | Single High-Frequency Term Query |
Multi-Field Sum | 125.21 | **153.51** |
**+22.6%** | +22.5% |
| **`DecInv_HighTerm`** | Single High-Frequency Term Query |
$g(x) = 100k / x$ | 117.79 | **143.95** |
**+22.2%** | +11.1% |
| **`Inc_AndHighHigh`** | Boolean AND Query (`term1 AND term2`) |
$f(x) = x$ | 97.83 | **116.20** |
**+18.8%** | -21.7% |
| **`DecInv_AndHighHigh`** | Boolean AND Query (`term1 AND term2`) |
$g(x) = 100k / x$ | 46.07 | **51.64** |
**+12.1%** | **-47.2%** |
| **`TwoFields_LowTerm`** | Single Low-Frequency Term Query |
Multi-Field Sum | 645.58 | **664.31** |
**+2.9%** | +61.7% |
| **`DecNeg_MedTerm`** | Single Medium-Frequency Term Query |
$g(x) = -x$ | 598.86 | **613.38** |
**+2.4%** | -35.8% |
| **`TwoFields_MedTerm`** | Single Medium-Frequency Term Query |
Multi-Field Sum | 313.73 | 309.34 |
**-1.4%** | **-54.0%** |
| **`TwoFields_OrHighHigh`** | Boolean OR Query (`term1 OR term2`) |
Multi-Field Sum | 97.82 | 92.51 |
**-5.4%** | **-66.4%** |
| **`DecInv_LowTerm`** | Single Low-Frequency Term Query |
$g(x) = 100k / x$ | 756.62 | 710.34 |
**-6.1%** | +169.1% |
| **`DecNeg_LowTerm`** | Single Low-Frequency Term Query |
$g(x) = -x$ | 713.80 | 616.24 |
**-13.7%** | -19.1% |
---
## 4. Key Performance Takeaways
1. **Massive QPS Gains Across Monotonic Functions**:
* Monotonically increasing (`Inc_*`), decreasing (`DecNeg_*`), and
inverse (`DecInv_*`) functions experience **+22.2% to +36.7% higher
throughput** across high/medium frequency terms and boolean queries.
2. **Dramatic Max Latency Reductions**:
* Max tail latency (`P100`) dropped by **-66.4%** on
`TwoFields_OrHighHigh` (from 186.1ms to 62.5ms) and **-59.6%** on
`DecNeg_AndHighHigh` (from 592.7ms to 239.7ms).
3. **4096-Block Size Efficiency**:
* Using 4096 doc-values block size allows Lucene to prune entire 4K
document blocks in a single WAND threshold evaluation without per-doc
calculation overhead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
