rishabhdaim commented on PR #3174: URL: https://github.com/apache/jackrabbit-oak/pull/3174#issuecomment-5911940739
## Benchmark re-run after the eviction-listener fix OAK-12425 head `a11587c20d`, benchmark `DocumentCacheBenchmark` from #3174 at `25aa31eda0`. Both policies now forward evictions to a listener set, exactly as `DocumentNodeStoreBuilder` does. Settings: 10,000-entry capacity, 2M operations, 16 threads, `-Xmx2g`, mean of 3 runs. ### Caffeine maximumWeight × 1.0 | scenario | working set | LIRS ops/s | Caffeine ops/s | Δ ops/s | LIRS hit% | Caffeine hit% | |---|---|---:|---:|---:|---:|---:| | STEADY_STATE | 0.5x | 19,066,040 | 17,344,764 | -9% | 100.0 | 100.0 | | STEADY_STATE | 1.0x | 12,549,385 | 17,113,698 | +36% | 99.7 | 100.0 | | STEADY_STATE | 2.0x | 8,321,610 | 10,269,268 | +23% | 87.5 | 87.9 | | STEADY_STATE | 5.0x | 5,264,256 | 8,074,341 | +53% | 70.8 | 74.3 | | STEADY_STATE | 10.0x | 4,081,043 | 4,010,552 | -2% | 38.5 | 42.6 | | CHURN | 0.5x | 16,326,530 | 17,854,212 | +9% | 100.0 | 100.0 | | CHURN | 1.0x | 12,362,329 | 15,122,933 | +22% | 97.8 | 100.0 | | CHURN | 2.0x | 6,358,537 | 6,527,151 | +3% | 62.1 | 64.9 | | CHURN | 5.0x | 3,578,263 | 3,710,089 | +4% | 21.9 | 27.5 | | CHURN | 10.0x | 2,938,242 | 2,626,955 | -11% | 8.7 | 13.3 | | INVALIDATION | 0.5x | 15,067,423 | 7,853,725 | -48% | 93.8 | 93.8 | | INVALIDATION | 1.0x | 13,536,603 | 7,014,511 | -48% | 93.4 | 93.8 | | INVALIDATION | 2.0x | 8,595,456 | 6,192,896 | -28% | 82.0 | 80.0 | | INVALIDATION | 5.0x | 6,197,165 | 4,984,333 | -20% | 64.3 | 61.4 | | INVALIDATION | 10.0x | 4,436,787 | 3,768,283 | -15% | 35.5 | 38.2 | | CONCURRENT | 0.5x | 20,895,743 | 109,371,953 | +423% | 100.0 | 100.0 | | CONCURRENT | 1.0x | 21,258,202 | 132,682,160 | +524% | 99.7 | 100.0 | | CONCURRENT | 2.0x | 17,292,795 | 22,033,073 | +27% | 87.5 | 88.2 | | CONCURRENT | 5.0x | 12,819,804 | 15,355,834 | +20% | 70.8 | 74.7 | | CONCURRENT | 10.0x | 8,112,766 | 7,363,155 | -9% | 38.5 | 42.3 | ### Caffeine maximumWeight × 1.25 | scenario | working set | LIRS ops/s | Caffeine ops/s | Δ ops/s | LIRS hit% | Caffeine hit% | |---|---|---:|---:|---:|---:|---:| | STEADY_STATE | 0.5x | 19,179,573 | 36,570,679 | +91% | 100.0 | 100.0 | | STEADY_STATE | 1.0x | 12,111,118 | 16,065,231 | +33% | 99.7 | 100.0 | | STEADY_STATE | 2.0x | 9,144,367 | 11,017,139 | +20% | 87.5 | 90.8 | | STEADY_STATE | 5.0x | 6,240,117 | 8,705,257 | +40% | 70.8 | 80.3 | | STEADY_STATE | 10.0x | 4,658,370 | 5,427,746 | +17% | 38.5 | 52.0 | | CHURN | 0.5x | 16,496,335 | 38,962,177 | +136% | 100.0 | 100.0 | | CHURN | 1.0x | 14,179,232 | 16,774,460 | +18% | 97.8 | 100.0 | | CHURN | 2.0x | 7,949,484 | 9,251,195 | +16% | 62.1 | 74.1 | | CHURN | 5.0x | 4,591,126 | 3,918,122 | -15% | 21.9 | 32.0 | | CHURN | 10.0x | 3,423,001 | 3,122,652 | -9% | 8.7 | 15.6 | | INVALIDATION | 0.5x | 16,582,471 | 11,619,115 | -30% | 93.8 | 93.8 | | INVALIDATION | 1.0x | 15,456,814 | 7,579,324 | -51% | 93.4 | 93.8 | | INVALIDATION | 2.0x | 9,259,889 | 6,983,597 | -25% | 82.0 | 84.2 | | INVALIDATION | 5.0x | 6,420,126 | 5,476,147 | -15% | 64.3 | 68.1 | | INVALIDATION | 10.0x | 4,642,930 | 4,304,089 | -7% | 35.5 | 44.8 | | CONCURRENT | 0.5x | 19,550,875 | 266,307,588 | +1262% | 100.0 | 100.0 | | CONCURRENT | 1.0x | 21,840,401 | 55,363,198 | +153% | 99.7 | 100.0 | | CONCURRENT | 2.0x | 16,988,408 | 27,788,361 | +64% | 87.5 | 91.5 | | CONCURRENT | 5.0x | 12,167,053 | 16,884,738 | +39% | 70.8 | 79.5 | | CONCURRENT | 10.0x | 8,321,228 | 8,477,137 | +2% | 38.6 | 51.4 | ### Summary - **Hit rate:** Caffeine's is equal or higher everywhere except INVALIDATION at 2–5× the working set (−2 to −3 pp at equal size). With +25% capacity, which matches the 256→320 MB default, it is higher everywhere, by up to +13 pp. - **Concurrent reads:** Caffeine is much faster (+20% to +520% at equal size, +39% to +1262% with +25%). The only exception is 10× the working set at equal size (−9%). - **Single-threaded INVALIDATION:** Caffeine is 7–51% slower. This is an explicit invalidate-and-reload micro-path; hit rates are unaffected. - **Always-registered eviction listener:** no measurable overhead. The trends match the earlier runs. - **Noise:** single-threaded numbers at 0.5× vary between runs (e.g. STEADY_STATE 0.5×: −9% vs +91%), so treat single-digit deltas as noise. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
