[
https://issues.apache.org/jira/browse/LUCENE-8213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900160#comment-16900160
]
Atri Sharma commented on LUCENE-8213:
-------------------------------------
I raised a PR for the same. The performance number from newly enhanced
luceneutil for wikimedium10m are:
Latencies:
|Task ('Wildcard', None)||P50 Base 2.045201||P50 Cmp 2.089539||Pct Diff
2.16790427933||P90 Base 18.845334||P90 Cmp 35.346911||Pct Diff
87.5631973411||P99 Base 83.02344||P99 Cmp 48.300884||Pct Diff
-41.8225937157||P999 Base 83.02344||P999 Cmp 48.300884||Pct Diff
-41.8225937157||P100 Base 249.902876||P100 Cmp 87.512667||Pct Diff
-64.9813285862||
||Task ('HighTermDayOfYearSort', 'DayOfYear')||P50 Base 4.295828||P50 Cmp
4.727759||Pct Diff 10.0546623375||P90 Base 9.037488||P90 Cmp 55.639159||Pct
Diff 515.648496573||P99 Base 82.149576||P99 Cmp 81.261365||Pct Diff
-1.08121191033||P999 Base 82.149576||P999 Cmp 81.261365||Pct Diff
-1.08121191033||P100 Base 86.642014||P100 Cmp 168.84768||Pct Diff
94.8796804285||
||Task ('MedSloppyPhrase', None)||P50 Base 9.18549||P50 Cmp 8.683321||Pct Diff
-5.46698107559||P90 Base 29.233836||P90 Cmp 30.984274||Pct Diff
5.98771232075||P99 Base 34.303039||P99 Cmp 35.978633||Pct Diff
4.88468091705||P999 Base 34.303039||P999 Cmp 35.978633||Pct Diff
4.88468091705||P100 Base 181.426025||P100 Cmp 261.742214||Pct Diff
44.2693869306||
||Task ('OrHighHigh', None)||P50 Base 20.997779||P50 Cmp 16.938239||Pct Diff
-19.3331875719||P90 Base 26.989668||P90 Cmp 29.711731||Pct Diff
10.0855742279||P99 Base 71.1345||P99 Cmp 72.914457||Pct Diff
2.50224152837||P999 Base 71.1345||P999 Cmp 72.914457||Pct Diff
2.50224152837||P100 Base 288.85441||P100 Cmp 203.02949||Pct Diff
-29.7121723016||
||Task ('MedPhrase', None)||P50 Base 6.935508||P50 Cmp 6.676061||Pct Diff
-3.74085070625||P90 Base 8.834132||P90 Cmp 7.366097||Pct Diff
-16.6177616545||P99 Base 61.645788||P99 Cmp 59.423887||Pct Diff
-3.60430302229||P999 Base 61.645788||P999 Cmp 59.423887||Pct Diff
-3.60430302229||P100 Base 65.592528||P100 Cmp 63.493249||Pct Diff
-3.20048496987||
||Task ('LowSpanNear', None)||P50 Base 23.256239||P50 Cmp 23.17936||Pct Diff
-0.330573658105||P90 Base 33.890598||P90 Cmp 34.205568||Pct Diff
0.929372801271||P99 Base 34.958863||P99 Cmp 34.857876||Pct Diff
-0.288873811485||P999 Base 34.958863||P999 Cmp 34.857876||Pct Diff
-0.288873811485||P100 Base 96.937787||P100 Cmp 121.889403||Pct Diff
25.7398242442||
||Task ('Fuzzy2', None)||P50 Base 25.45292||P50 Cmp 25.25128||Pct Diff
-0.792207730979||P90 Base 79.376572||P90 Cmp 106.649481||Pct Diff
34.3588899254||P99 Base 108.933154||P99 Cmp 122.051216||Pct Diff
12.0423044026||P999 Base 108.933154||P999 Cmp 122.051216||Pct Diff
12.0423044026||P100 Base 212.373308||P100 Cmp 209.138442||Pct Diff
-1.52319800942||
||Task ('OrNotHighHigh', None)||P50 Base 1.903331||P50 Cmp 2.16024||Pct Diff
13.4978624317||P90 Base 4.890325||P90 Cmp 4.723459||Pct Diff -3.4121658581||P99
Base 102.556452||P99 Cmp 102.641448||Pct Diff 0.0828772820651||P999 Base
102.556452||P999 Cmp 102.641448||Pct Diff 0.0828772820651||P100 Base
226.783706||P100 Cmp 308.709148||Pct Diff 36.1249242483||
||Task ('OrHighNotLow', None)||P50 Base 1.434646||P50 Cmp 1.52378||Pct Diff
6.21296124619||P90 Base 3.905319||P90 Cmp 4.569729||Pct Diff 17.0129507986||P99
Base 6.321682||P99 Cmp 7.281513||Pct Diff 15.1831585328||P999 Base
6.321682||P999 Cmp 7.281513||Pct Diff 15.1831585328||P100 Base 7.720665||P100
Cmp 15.035781||Pct Diff 94.7472270847||
||Task ('BrowseMonthSSDVFacets', None)||P50 Base 93.940495||P50 Cmp
93.939183||Pct Diff -0.00139662879145||P90 Base 102.50354||P90 Cmp
98.604983||Pct Diff -3.80333888956||P99 Base 103.572854||P99 Cmp
106.785928||Pct Diff 3.10223564951||P999 Base 103.572854||P999 Cmp
106.785928||Pct Diff 3.10223564951||P100 Base 283.457123||P100 Cmp
244.054099||Pct Diff -13.9008762888||
||Task ('Fuzzy1', None)||P50 Base 26.559456||P50 Cmp 29.050383||Pct Diff
9.37868230434||P90 Base 159.424881||P90 Cmp 171.063113||Pct Diff
7.30013529068||P99 Base 339.7673||P99 Cmp 179.733118||Pct Diff
-47.1011136151||P999 Base 339.7673||P999 Cmp 179.733118||Pct Diff
-47.1011136151||P100 Base 417.349072||P100 Cmp 395.168736||Pct Diff
-5.31457657105||
||Task ('HighSloppyPhrase', None)||P50 Base 9.489382||P50 Cmp 9.980939||Pct
Diff 5.18007389733||P90 Base 14.424659||P90 Cmp 15.315198||Pct Diff
6.17372653315||P99 Base 37.046395||P99 Cmp 31.348423||Pct Diff
-15.380638251||P999 Base 37.046395||P999 Cmp 31.348423||Pct Diff
-15.380638251||P100 Base 51.797966||P100 Cmp 33.660774||Pct Diff
-35.0152590934||
||Task ('OrNotHighMed', None)||P50 Base 1.605631||P50 Cmp 1.549948||Pct Diff
-3.46798236955||P90 Base 16.030506||P90 Cmp 11.175798||Pct Diff
-30.2841844169||P99 Base 63.933462||P99 Cmp 63.33348||Pct Diff
-0.938447537848||P999 Base 63.933462||P999 Cmp 63.33348||Pct Diff
-0.938447537848||P100 Base 176.946354||P100 Cmp 141.426012||Pct Diff
-20.0740739761||
||Task ('LowPhrase', None)||P50 Base 5.294719||P50 Cmp 5.398776||Pct Diff
1.96529787511||P90 Base 14.581045||P90 Cmp 14.201875||Pct Diff
-2.6004309019||P99 Base 28.226324||P99 Cmp 17.160324||Pct Diff
-39.2045382884||P999 Base 28.226324||P999 Cmp 17.160324||Pct Diff
-39.2045382884||P100 Base 170.879755||P100 Cmp 141.494236||Pct Diff
-17.1966064675||
||Task ('AndHighLow', None)||P50 Base 1.407978||P50 Cmp 1.356986||Pct Diff
-3.62164749733||P90 Base 4.452746||P90 Cmp 6.676942||Pct Diff
49.9511088214||P99 Base 69.005964||P99 Cmp 69.618355||Pct Diff
0.887446482162||P999 Base 69.005964||P999 Cmp 69.618355||Pct Diff
0.887446482162||P100 Base 302.690572||P100 Cmp 288.210249||Pct Diff
-4.7838698458||
||Task ('LowTerm', None)||P50 Base 0.94373||P50 Cmp 0.984236||Pct Diff
4.29211744885||P90 Base 1.776916||P90 Cmp 1.95572||Pct Diff 10.0626028467||P99
Base 2.130259||P99 Cmp 2.967897||Pct Diff 39.3209464201||P999 Base
2.130259||P999 Cmp 2.967897||Pct Diff 39.3209464201||P100 Base 3.471183||P100
Cmp 11.055948||Pct Diff 218.50663016||
||Task ('BrowseDayOfYearSSDVFacets', None)||P50 Base 126.336025||P50 Cmp
125.101777||Pct Diff -0.976956493605||P90 Base 135.381181||P90 Cmp
137.867944||Pct Diff 1.8368601763||P99 Base 144.330923||P99 Cmp 144.977097||Pct
Diff 0.447703088547||P999 Base 144.330923||P999 Cmp 144.977097||Pct Diff
0.447703088547||P100 Base 295.783063||P100 Cmp 302.394898||Pct Diff
2.23536632995||
||Task ('AndHighHigh', None)||P50 Base 5.089657||P50 Cmp 5.20707||Pct Diff
2.30689415809||P90 Base 6.822905||P90 Cmp 8.306661||Pct Diff 21.7466900096||P99
Base 7.038109||P99 Cmp 9.130519||Pct Diff 29.7297185934||P999 Base
7.038109||P999 Cmp 9.130519||Pct Diff 29.7297185934||P100 Base 111.532214||P100
Cmp 36.808683||Pct Diff -66.9972632302||
||Task ('HighTermMonthSort', 'Month')||P50 Base 2.42148||P50 Cmp 2.464948||Pct
Diff 1.79510051704||P90 Base 4.411412||P90 Cmp 4.012971||Pct Diff
-9.03205141574||P99 Base 4.993632||P99 Cmp 4.667476||Pct Diff -6.53143844||P999
Base 4.993632||P999 Cmp 4.667476||Pct Diff -6.53143844||P100 Base
98.675367||P100 Cmp 94.641052||Pct Diff -4.08847225265||
||Task Respell||P50 Base 17.339473||P50 Cmp 16.495659||Pct Diff
-4.86643394525||P90 Base 202.898846||P90 Cmp 190.452875||Pct Diff
-6.13407678031||P99 Base 232.699615||P99 Cmp 206.63496||Pct Diff
-11.2009875908||P999 Base 232.699615||P999 Cmp 206.63496||Pct Diff
-11.2009875908||P100 Base 259.083639||P100 Cmp 261.411887||Pct Diff
0.89864725113||
||Task ('MedTerm', None)||P50 Base 1.044861||P50 Cmp 1.046421||Pct Diff
0.149302155981||P90 Base 10.095498||P90 Cmp 2.086543||Pct Diff
-79.3319457842||P99 Base 31.02937||P99 Cmp 4.311224||Pct Diff
-86.1059892611||P999 Base 31.02937||P999 Cmp 4.311224||Pct Diff
-86.1059892611||P100 Base 37.048313||P100 Cmp 30.992555||Pct Diff
-16.3455701748||
||Task ('HighIntervalsOrdered', None)||P50 Base 18.204528||P50 Cmp
18.50288||Pct Diff 1.63888896213||P90 Base 28.38481||P90 Cmp 28.186101||Pct
Diff -0.70005400776||P99 Base 28.553312||P99 Cmp 30.147999||Pct Diff
5.58494580243||P999 Base 28.553312||P999 Cmp 30.147999||Pct Diff
5.58494580243||P100 Base 85.526627||P100 Cmp 248.898567||Pct Diff
191.018804004||
||Task ('HighSpanNear', None)||P50 Base 19.317909||P50 Cmp 18.448229||Pct Diff
-4.50193651911||P90 Base 26.886654||P90 Cmp 27.011485||Pct Diff
0.464286110127||P99 Base 27.066426||P99 Cmp 28.519568||Pct Diff
5.36879896888||P999 Base 27.066426||P999 Cmp 28.519568||Pct Diff
5.36879896888||P100 Base 27.949757||P100 Cmp 28.532982||Pct Diff 2.08669077159||
||Task ('LowSloppyPhrase', None)||P50 Base 17.995202||P50 Cmp 17.321666||Pct
Diff -3.74286434795||P90 Base 27.136218||P90 Cmp 23.990793||Pct Diff
-11.5912431128||P99 Base 27.177319||P99 Cmp 25.467183||Pct Diff
-6.29251178161||P999 Base 27.177319||P999 Cmp 25.467183||Pct Diff
-6.29251178161||P100 Base 37.041781||P100 Cmp 26.387058||Pct Diff
-28.7640677968||
||Task ('OrHighNotHigh', None)||P50 Base 1.721541||P50 Cmp 1.683098||Pct Diff
-2.23305747583||P90 Base 2.334157||P90 Cmp 2.147467||Pct Diff
-7.99817664364||P99 Base 3.127609||P99 Cmp 3.205909||Pct Diff
2.50350986968||P999 Base 3.127609||P999 Cmp 3.205909||Pct Diff
2.50350986968||P100 Base 10.177133||P100 Cmp 5.218394||Pct Diff -48.7243214764||
||Task ('OrNotHighLow', None)||P50 Base 1.467553||P50 Cmp 1.4717||Pct Diff
0.282579232232||P90 Base 11.130311||P90 Cmp 9.27055||Pct Diff
-16.7089760565||P99 Base 12.92663||P99 Cmp 12.305485||Pct Diff
-4.80515803423||P999 Base 12.92663||P999 Cmp 12.305485||Pct Diff
-4.80515803423||P100 Base 41.801941||P100 Cmp 42.736573||Pct Diff
2.23585789952||
||Task ('MedSpanNear', None)||P50 Base 8.050703||P50 Cmp 8.0656||Pct Diff
0.18503974125||P90 Base 12.37128||P90 Cmp 11.972162||Pct Diff
-3.22616576458||P99 Base 13.276762||P99 Cmp 13.410559||Pct Diff
1.00775324586||P999 Base 13.276762||P999 Cmp 13.410559||Pct Diff
1.00775324586||P100 Base 389.340697||P100 Cmp 524.654134||Pct Diff
34.7545062827||
||Task ('BrowseDateTaxoFacets', None)||P50 Base 429.062505||P50 Cmp
428.859358||Pct Diff -0.0473467146704||P90 Base 436.499827||P90 Cmp
441.680746||Pct Diff 1.18692349447||P99 Base 458.955056||P99 Cmp
443.442175||Pct Diff -3.38004360061||P999 Base 458.955056||P999 Cmp
443.442175||Pct Diff -3.38004360061||P100 Base 516.629721||P100 Cmp
518.247598||Pct Diff 0.313159877227||
||Task ('HighPhrase', None)||P50 Base 4.346807||P50 Cmp 4.173837||Pct Diff
-3.9792426947||P90 Base 5.762266||P90 Cmp 6.035056||Pct Diff 4.7340751017||P99
Base 7.09179||P99 Cmp 8.179865||Pct Diff 15.3427413953||P999 Base 7.09179||P999
Cmp 8.179865||Pct Diff 15.3427413953||P100 Base 25.682734||P100 Cmp
26.872466||Pct Diff 4.63241958586||
||Task ('OrHighLow', None)||P50 Base 2.360377||P50 Cmp 2.373989||Pct Diff
0.576687537626||P90 Base 2.920517||P90 Cmp 4.037781||Pct Diff
38.2556923997||P99 Base 4.226431||P99 Cmp 4.511819||Pct Diff
6.75245851642||P999 Base 4.226431||P999 Cmp 4.511819||Pct Diff
6.75245851642||P100 Base 48.928035||P100 Cmp 13.15068||Pct Diff -73.1224031376||
||Task ('BrowseDayOfYearTaxoFacets', None)||P50 Base 0.168513||P50 Cmp
0.182987||Pct Diff 8.58924830725||P90 Base 0.204691||P90 Cmp 0.247465||Pct Diff
20.8968640536||P99 Base 15.455504||P99 Cmp 15.315246||Pct Diff
-0.907495478633||P999 Base 15.455504||P999 Cmp 15.315246||Pct Diff
-0.907495478633||P100 Base 16.866213||P100 Cmp 19.937509||Pct Diff
18.2097546142||
||Task ('OrHighMed', None)||P50 Base 3.574777||P50 Cmp 3.231059||Pct Diff
-9.6150892769||P90 Base 24.188598||P90 Cmp 4.638494||Pct Diff
-80.8236343421||P99 Base 46.693976||P99 Cmp 8.113618||Pct Diff
-82.6238442406||P999 Base 46.693976||P999 Cmp 8.113618||Pct Diff
-82.6238442406||P100 Base 269.409786||P100 Cmp 23.431229||Pct Diff
-91.3027550529||
||Task ('IntNRQ', None)||P50 Base 4.011057||P50 Cmp 4.923845||Pct Diff
22.7567945307||P90 Base 13.346714||P90 Cmp 12.29044||Pct Diff
-7.91411279211||P99 Base 85.073004||P99 Cmp 67.905295||Pct Diff
-20.1799727208||P999 Base 85.073004||P999 Cmp 67.905295||Pct Diff
-20.1799727208||P100 Base 191.744926||P100 Cmp 194.853114||Pct Diff
1.62100143396||
||Task ('BrowseMonthTaxoFacets', None)||P50 Base 0.170601||P50 Cmp
0.172844||Pct Diff 1.31476368837||P90 Base 0.212197||P90 Cmp 0.260394||Pct Diff
22.7133277096||P99 Base 0.227793||P99 Cmp 4.692225||Pct Diff
1959.86356034||P999 Base 0.227793||P999 Cmp 4.692225||Pct Diff
1959.86356034||P100 Base 10.00639||P100 Cmp 9.966375||Pct Diff -0.399894467435||
||Task ('AndHighMed', None)||P50 Base 3.367039||P50 Cmp 3.095126||Pct Diff
-8.07573063454||P90 Base 6.445771||P90 Cmp 4.890087||Pct Diff
-24.1349560821||P99 Base 6.646047||P99 Cmp 6.78425||Pct Diff
2.07947671751||P999 Base 6.646047||P999 Cmp 6.78425||Pct Diff
2.07947671751||P100 Base 42.892862||P100 Cmp 96.998233||Pct Diff 126.140734092||
||Task ('OrHighNotMed', None)||P50 Base 1.75091||P50 Cmp 1.851148||Pct Diff
5.72490876173||P90 Base 3.059898||P90 Cmp 3.291761||Pct Diff 7.57747480472||P99
Base 5.304103||P99 Cmp 4.755972||Pct Diff -10.3340941909||P999 Base
5.304103||P999 Cmp 4.755972||Pct Diff -10.3340941909||P100 Base 21.934264||P100
Cmp 16.361938||Pct Diff -25.4046636805||
||Task ('Prefix3', None)||P50 Base 15.061956||P50 Cmp 14.031055||Pct Diff
-6.84440321031||P90 Base 28.636916||P90 Cmp 25.283538||Pct Diff
-11.7099830163||P99 Base 425.912794||P99 Cmp 443.836014||Pct Diff
4.20819009255||P999 Base 425.912794||P999 Cmp 443.836014||Pct Diff
4.20819009255||P100 Base 497.136127||P100 Cmp 551.753833||Pct Diff
10.9864689033||
||Task PKLookup||P50 Base 11.334723||P50 Cmp 10.857995||Pct Diff
-4.20590781089||P90 Base 89.284901||P90 Cmp 32.062856||Pct Diff
-64.0892741764||P99 Base 334.811782||P99 Cmp 73.551516||Pct Diff
-78.0319809654||P999 Base 334.811782||P999 Cmp 73.551516||Pct Diff
-78.0319809654||P100 Base 363.062965||P100 Cmp 418.731587||Pct Diff
15.3330489107||
||Task ('HighTerm', None)||P50 Base 1.148459||P50 Cmp 1.212344||Pct Diff
5.56267137094||P90 Base 1.54||P90 Cmp 1.915795||Pct Diff 24.4022727273||P99
Base 2.138094||P99 Cmp 3.134936||Pct Diff 46.622926775||P999 Base
2.138094||P999 Cmp 3.134936||Pct Diff 46.622926775||P100 Base 2.722231||P100
Cmp 153.54749||Pct Diff 5540.50185308||
> Cache costly subqueries asynchronously
> --------------------------------------
>
> Key: LUCENE-8213
> URL: https://issues.apache.org/jira/browse/LUCENE-8213
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/query/scoring
> Affects Versions: 7.2.1
> Reporter: Amir Hadadi
> Priority: Minor
> Labels: performance
> Time Spent: 20m
> Remaining Estimate: 0h
>
> IndexOrDocValuesQuery allows to combine costly range queries with a selective
> lead iterator in an optimized way. However, the range query at some point
> gets cached by a querying thread in LRUQueryCache, which negates the
> optimization of IndexOrDocValuesQuery for that specific query.
> It would be nice to see an asynchronous caching implementation in such cases,
> so that queries involving IndexOrDocValuesQuery would have consistent
> performance characteristics.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
