[ https://issues.apache.org/jira/browse/LUCENE-10374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475438#comment-17475438 ]
Greg Miller commented on LUCENE-10374: -------------------------------------- It appears that LUCENE-10350 was responsible for the nightly benchmark regressions but I can't reason about how it would cause a regression. Both [~gf2121] and I saw significant performance improvements associated with these same benchmark tasks when running locally (results below; and note that this benchmark is "reversed" in that it was trying to measure the impact of reverting LUCENE-10350... so the baseline has the change and the candidate reverts it). So it remains a mystery why the discrepancy would exist. {code:java} TaskQPS baseline StdDevQPS candidate StdDev Pct diff p-value BrowseDayOfYearTaxoFacets 30.81 (24.0%) 14.56 (4.5%) -52.7% ( -65% - -31%) 0.000 BrowseDateTaxoFacets 30.38 (23.9%) 14.52 (4.5%) -52.2% ( -65% - -31%) 0.000 BrowseMonthTaxoFacets 30.08 (20.9%) 15.73 (3.9%) -47.7% ( -59% - -29%) 0.000 BrowseRandomLabelTaxoFacets 23.21 (24.6%) 12.43 (4.4%) -46.4% ( -60% - -23%) 0.000 MedTermDayTaxoFacets 34.20 (4.5%) 33.65 (4.8%) -1.6% ( -10% - 7%) 0.273 TermDTSort 185.78 (20.4%) 183.85 (20.8%) -1.0% ( -35% - 50%) 0.873 AndHighHigh 53.56 (3.3%) 53.04 (3.3%) -1.0% ( -7% - 5%) 0.361 LowPhrase 97.47 (3.4%) 97.06 (2.1%) -0.4% ( -5% - 5%) 0.634 HighSpanNear 8.59 (4.2%) 8.55 (4.4%) -0.4% ( -8% - 8%) 0.761 OrHighLow 668.71 (1.6%) 666.14 (2.4%) -0.4% ( -4% - 3%) 0.546 AndHighMed 227.67 (1.8%) 226.80 (2.1%) -0.4% ( -4% - 3%) 0.533 OrHighMedDayTaxoFacets 7.26 (5.2%) 7.24 (5.8%) -0.4% ( -10% - 11%) 0.832 OrHighMed 117.01 (3.7%) 116.66 (2.9%) -0.3% ( -6% - 6%) 0.774 MedSpanNear 49.68 (4.0%) 49.55 (4.4%) -0.3% ( -8% - 8%) 0.847 LowSpanNear 45.83 (3.1%) 45.72 (2.7%) -0.2% ( -5% - 5%) 0.796 MedPhrase 95.22 (3.2%) 95.12 (2.2%) -0.1% ( -5% - 5%) 0.897 OrHighHigh 36.26 (3.7%) 36.22 (3.2%) -0.1% ( -6% - 7%) 0.918 IntNRQ 97.48 (1.4%) 97.57 (1.1%) 0.1% ( -2% - 2%) 0.822 MedSloppyPhrase 28.73 (2.4%) 28.76 (2.8%) 0.1% ( -4% - 5%) 0.912 BrowseRandomLabelSSDVFacets 9.48 (3.8%) 9.49 (3.5%) 0.1% ( -6% - 7%) 0.929 OrNotHighHigh 884.42 (3.0%) 885.49 (3.2%) 0.1% ( -5% - 6%) 0.902 LowIntervalsOrdered 90.92 (4.1%) 91.10 (3.9%) 0.2% ( -7% - 8%) 0.878 OrNotHighMed 1089.50 (2.2%) 1091.92 (3.0%) 0.2% ( -4% - 5%) 0.788 OrHighNotHigh 826.99 (3.7%) 829.46 (3.3%) 0.3% ( -6% - 7%) 0.787 HighIntervalsOrdered 7.48 (6.8%) 7.50 (6.5%) 0.3% ( -12% - 14%) 0.885 MedTerm 1890.85 (2.9%) 1896.72 (2.6%) 0.3% ( -5% - 5%) 0.721 MedIntervalsOrdered 6.54 (4.5%) 6.57 (4.2%) 0.4% ( -7% - 9%) 0.770 AndHighHighDayTaxoFacets 15.93 (3.1%) 16.00 (3.0%) 0.4% ( -5% - 6%) 0.662 LowTerm 1977.37 (2.7%) 1986.22 (3.5%) 0.4% ( -5% - 6%) 0.648 LowSloppyPhrase 86.15 (3.9%) 86.56 (4.4%) 0.5% ( -7% - 9%) 0.720 HighTerm 1494.75 (3.1%) 1501.82 (2.9%) 0.5% ( -5% - 6%) 0.622 Fuzzy2 67.47 (2.0%) 67.80 (2.1%) 0.5% ( -3% - 4%) 0.450 OrHighNotLow 1388.49 (3.0%) 1395.34 (2.4%) 0.5% ( -4% - 5%) 0.560 AndHighMedDayTaxoFacets 89.88 (2.3%) 90.38 (2.1%) 0.6% ( -3% - 5%) 0.418 BrowseMonthSSDVFacets 14.54 (20.6%) 14.63 (21.8%) 0.6% ( -34% - 54%) 0.927 HighSloppyPhrase 11.53 (4.3%) 11.60 (5.1%) 0.6% ( -8% - 10%) 0.678 HighTermTitleBDVSort 122.18 (14.6%) 122.97 (17.4%) 0.6% ( -27% - 38%) 0.898 HighPhrase 432.20 (3.3%) 435.35 (2.4%) 0.7% ( -4% - 6%) 0.426 Fuzzy1 81.65 (2.1%) 82.25 (2.2%) 0.7% ( -3% - 5%) 0.280 OrHighNotMed 1060.43 (3.0%) 1068.54 (3.0%) 0.8% ( -5% - 6%) 0.422 Respell 63.22 (2.5%) 63.78 (2.4%) 0.9% ( -3% - 5%) 0.255 AndHighLow 1108.01 (2.6%) 1118.78 (3.1%) 1.0% ( -4% - 6%) 0.287 PKLookup 171.13 (3.0%) 173.05 (4.8%) 1.1% ( -6% - 9%) 0.380 HighTermDayOfYearSort 37.22 (25.8%) 37.84 (22.8%) 1.7% ( -37% - 67%) 0.827 Wildcard 85.54 (5.3%) 87.11 (5.0%) 1.8% ( -8% - 12%) 0.262 OrNotHighLow 938.56 (3.1%) 955.98 (3.0%) 1.9% ( -4% - 8%) 0.054 Prefix3 146.06 (9.7%) 150.33 (8.5%) 2.9% ( -13% - 23%) 0.307 HighTermMonthSort 123.10 (14.9%) 126.94 (16.8%) 3.1% ( -24% - 40%) 0.534 BrowseDayOfYearSSDVFacets 12.11 (9.5%) 12.56 (12.9%) 3.7% ( -17% - 28%) 0.297{code} > Track down the "browse" taxonomy faceting qps regression > -------------------------------------------------------- > > Key: LUCENE-10374 > URL: https://issues.apache.org/jira/browse/LUCENE-10374 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/facet > Reporter: Greg Miller > Priority: Major > > We need to track down the source of the regression observed here: > [https://home.apache.org/~mikemccand/lucenebench/2022.01.10.18.03.12.html.] > Some details on the regression hunting are in > [https://github.com/apache/lucene/pull/597.] -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org