[ https://issues.apache.org/jira/browse/LUCENE-10346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17468009#comment-17468009 ]
ASF subversion and git services commented on LUCENE-10346: ---------------------------------------------------------- Commit 01f5e7bb7b4f5efb5330ab97896008c83daef657 in lucene's branch refs/heads/branch_9x from gf2121 [ https://gitbox.apache.org/repos/asf?p=lucene.git;h=01f5e7b ] LUCENE-10346: Specially treat SingletonSortedNumericDocValues in FastTaxonomyFacetCounts#countAll() (#574) > Specially treat SingletonSortedNumericDocValues in > FastTaxonomyFacetCounts#countAll() > ------------------------------------------------------------------------------------- > > Key: LUCENE-10346 > URL: https://issues.apache.org/jira/browse/LUCENE-10346 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/facet > Reporter: Feng Guo > Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > CPU profile often tells {{SingletonSortedNumericDocValues#nextDoc()}} is > using a high percentage of CPU when running luceneutil, but the {{nextDoc()}} > of dense cases should be rather simple. So I suspect that it is too many > layers of abstraction (and wrap) that cause the stress of JVM. Unwraping it > to {{NumericDocvalues}} shows around 30% speed up. > {code:java} > TaskQPS baseline StdDevQPS > my_modified_version StdDev Pct diff p-value > HighTermTitleBDVSort 132.24 (20.6%) 125.67 > (9.9%) -5.0% ( -29% - 32%) 0.330 > LowTerm 1424.13 (3.2%) 1381.34 > (4.4%) -3.0% ( -10% - 4%) 0.014 > OrHighNotHigh 707.82 (3.3%) 687.49 > (6.0%) -2.9% ( -11% - 6%) 0.062 > TermDTSort 155.32 (10.9%) 151.02 > (10.2%) -2.8% ( -21% - 20%) 0.406 > OrNotHighMed 618.46 (3.7%) 602.65 > (4.4%) -2.6% ( -10% - 5%) 0.047 > Fuzzy1 76.22 (5.3%) 74.71 > (6.6%) -2.0% ( -13% - 10%) 0.293 > HighTermMonthSort 174.89 (10.4%) 171.45 > (10.6%) -2.0% ( -20% - 21%) 0.554 > OrHighNotMed 776.08 (4.9%) 761.70 > (7.8%) -1.9% ( -13% - 11%) 0.367 > HighTermDayOfYearSort 56.23 (10.7%) 55.26 > (10.9%) -1.7% ( -21% - 22%) 0.615 > MedTerm 1449.48 (3.7%) 1425.87 > (5.1%) -1.6% ( -10% - 7%) 0.250 > OrNotHighHigh 687.92 (4.9%) 677.06 > (5.5%) -1.6% ( -11% - 9%) 0.339 > OrHighNotLow 742.99 (4.7%) 732.23 > (5.9%) -1.4% ( -11% - 9%) 0.390 > OrNotHighLow 789.37 (2.7%) 778.80 > (4.7%) -1.3% ( -8% - 6%) 0.270 > HighPhrase 75.84 (2.2%) 75.14 > (3.0%) -0.9% ( -6% - 4%) 0.269 > HighSloppyPhrase 20.71 (5.9%) 20.56 > (5.2%) -0.7% ( -11% - 11%) 0.678 > IntNRQ 106.38 (18.4%) 105.67 > (18.2%) -0.7% ( -31% - 44%) 0.908 > OrHighMed 45.10 (1.5%) 44.83 > (1.8%) -0.6% ( -3% - 2%) 0.261 > MedSpanNear 192.49 (2.5%) 191.51 > (3.5%) -0.5% ( -6% - 5%) 0.593 > OrHighLow 489.82 (5.5%) 487.79 > (5.7%) -0.4% ( -11% - 11%) 0.815 > MedSloppyPhrase 27.33 (2.9%) 27.22 > (2.3%) -0.4% ( -5% - 5%) 0.623 > MedPhrase 208.94 (2.9%) 208.09 > (3.7%) -0.4% ( -6% - 6%) 0.696 > Respell 71.84 (2.4%) 71.55 > (2.4%) -0.4% ( -5% - 4%) 0.600 > OrHighHigh 36.26 (1.3%) 36.13 > (1.1%) -0.4% ( -2% - 2%) 0.344 > BrowseMonthSSDVFacets 15.95 (2.7%) 15.90 > (2.5%) -0.4% ( -5% - 5%) 0.672 > AndHighMed 85.83 (2.2%) 85.53 > (2.7%) -0.3% ( -5% - 4%) 0.658 > Prefix3 123.15 (2.6%) 122.74 > (2.5%) -0.3% ( -5% - 4%) 0.678 > Fuzzy2 76.41 (4.7%) 76.23 > (4.2%) -0.2% ( -8% - 9%) 0.867 > BrowseDayOfYearSSDVFacets 14.52 (2.4%) 14.49 > (2.2%) -0.2% ( -4% - 4%) 0.747 > MedIntervalsOrdered 56.39 (4.2%) 56.27 > (4.1%) -0.2% ( -8% - 8%) 0.871 > HighIntervalsOrdered 9.29 (4.7%) 9.27 > (4.4%) -0.2% ( -8% - 9%) 0.896 > AndHighMedDayTaxoFacets 119.76 (2.5%) 119.53 > (2.9%) -0.2% ( -5% - 5%) 0.831 > HighSpanNear 20.89 (2.0%) 20.85 > (2.3%) -0.2% ( -4% - 4%) 0.803 > LowIntervalsOrdered 45.51 (4.9%) 45.47 > (4.8%) -0.1% ( -9% - 10%) 0.952 > LowPhrase 64.17 (2.6%) 64.14 > (2.6%) -0.1% ( -5% - 5%) 0.951 > LowSpanNear 104.45 (2.2%) 104.41 > (1.9%) -0.0% ( -4% - 4%) 0.959 > Wildcard 103.83 (2.8%) 103.80 > (2.8%) -0.0% ( -5% - 5%) 0.970 > AndHighHigh 42.33 (2.6%) 42.33 > (2.4%) -0.0% ( -4% - 5%) 0.991 > BrowseRandomLabelSSDVFacets 10.62 (2.5%) 10.62 > (1.8%) 0.0% ( -4% - 4%) 0.981 > AndHighHighDayTaxoFacets 29.75 (2.3%) 29.76 > (2.7%) 0.1% ( -4% - 5%) 0.949 > MedTermDayTaxoFacets 26.56 (3.0%) 26.58 > (2.5%) 0.1% ( -5% - 5%) 0.945 > AndHighLow 1012.26 (4.5%) 1013.62 > (4.3%) 0.1% ( -8% - 9%) 0.923 > LowSloppyPhrase 78.82 (6.8%) 79.03 > (6.0%) 0.3% ( -11% - 14%) 0.897 > PKLookup 204.09 (3.0%) 204.82 > (2.9%) 0.4% ( -5% - 6%) 0.703 > OrHighMedDayTaxoFacets 14.53 (3.4%) 14.59 > (2.7%) 0.4% ( -5% - 6%) 0.694 > HighTerm 1607.26 (5.2%) 1623.99 > (5.6%) 1.0% ( -9% - 12%) 0.543 > BrowseRandomLabelTaxoFacets 11.93 (6.9%) 15.52 > (2.5%) 30.1% ( 19% - 42%) 0.000 > BrowseDateTaxoFacets 13.46 (9.0%) 18.28 > (3.6%) 35.8% ( 21% - 53%) 0.000 > BrowseDayOfYearTaxoFacets 13.59 (9.1%) 18.53 > (3.6%) 36.3% ( 21% - 53%) 0.000 > BrowseMonthTaxoFacets 13.93 (10.9%) 19.70 > (14.9%) 41.4% ( 14% - 75%) 0.000 > {code} > *Baseline* > {code:java} > PERCENT CPU SAMPLES STACK > 3.85% 12316 > org.apache.lucene.facet.sortedset.SortedSetDocValuesFacetCounts#countOneSegment() > 3.78% 12076 > org.apache.lucene.util.packed.DirectReader$DirectPackedReader20#get() > 3.72% 11905 > org.apache.lucene.index.SingletonSortedNumericDocValues#nextDoc() > 2.88% 9199 > org.apache.lucene.queries.intervals.OrderedIntervalsSource$OrderedIntervalIterator#nextInterval() > 2.31% 7380 > org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts#countAll() > 2.27% 7270 > org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$20#ordValue() > 2.25% 7211 > org.apache.lucene.facet.taxonomy.IntTaxonomyFacets#increment() > 2.23% 7139 > org.apache.lucene.index.SingletonSortedNumericDocValues#nextValue() > 1.88% 6006 java.nio.Buffer#checkIndex() > 1.86% 5965 jdk.internal.misc.Unsafe#convEndian() > 1.85% 5916 > org.apache.lucene.util.packed.DirectReader$DirectPackedReader4#get() > 1.72% 5491 > org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$EverythingEnum#nextPosition() > 1.49% 4780 java.nio.DirectByteBuffer#ix() > 1.42% 4548 java.nio.Buffer#scope() > 1.40% 4465 > org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$4#longValue() > 1.39% 4434 > org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$EverythingEnum#advance() > 1.33% 4254 > org.apache.lucene.store.ByteBufferGuard#ensureValid() > 1.32% 4219 > org.apache.lucene.util.packed.DirectReader$DirectPackedReader12#get() > 1.28% 4109 > org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$20#nextDoc() > 1.28% 4089 > jdk.internal.misc.ScopedMemoryAccess#getByteInternal() > 1.16% 3709 > org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$BlockImpactsPostingsEnum#advance() > 1.10% 3517 org.apache.lucene.store.ByteBufferGuard#getInt() > 1.07% 3427 > org.apache.lucene.codecs.lucene90.Lucene90NormsProducer$3#longValue() > 0.98% 3149 org.apache.lucene.search.ConjunctionDISI#doNext() > 0.98% 3120 > org.apache.lucene.codecs.lucene90.Lucene90PostingsReader#findFirstGreater() > 0.93% 2969 > org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$3#longValue() > 0.92% 2927 org.apache.lucene.store.ByteBufferGuard#getByte() > 0.88% 2828 com.carrotsearch.hppc.IntIntHashMap#indexOf() > 0.82% 2635 > org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$BlockImpactsDocsEnum#advance() > 0.82% 2633 > org.apache.lucene.search.similarities.BM25Similarity$BM25Scorer#score() > {code} > *Candidate* > {code:java} > PERCENT CPU SAMPLES STACK > 4.15% 12823 > org.apache.lucene.util.packed.DirectReader$DirectPackedReader20#get() > 3.94% 12186 > org.apache.lucene.facet.sortedset.SortedSetDocValuesFacetCounts#countOneSegment() > 3.32% 10266 > org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts#countAll() > 2.98% 9208 > org.apache.lucene.queries.intervals.OrderedIntervalsSource$OrderedIntervalIterator#nextInterval() > 2.38% 7351 > org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$20#ordValue() > 2.07% 6386 > org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$DenseNumericDocValues#nextDoc() > 1.85% 5723 > org.apache.lucene.facet.taxonomy.IntTaxonomyFacets#increment() > 1.81% 5600 jdk.internal.misc.Unsafe#convEndian() > 1.81% 5588 > org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$EverythingEnum#nextPosition() > 1.75% 5409 java.nio.Buffer#checkIndex() > 1.72% 5310 > org.apache.lucene.util.packed.DirectReader$DirectPackedReader4#get() > 1.50% 4631 java.nio.Buffer#scope() > 1.44% 4437 > jdk.internal.misc.ScopedMemoryAccess#getByteInternal() > 1.43% 4408 > org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$EverythingEnum#advance() > 1.39% 4297 java.nio.DirectByteBuffer#ix() > 1.39% 4280 > org.apache.lucene.util.packed.DirectReader$DirectPackedReader12#get() > 1.33% 4111 > org.apache.lucene.store.ByteBufferGuard#ensureValid() > 1.31% 4052 > org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$20#nextDoc() > 1.29% 3974 > org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$4#longValue() > 1.22% 3761 > org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$BlockImpactsPostingsEnum#advance() > 1.13% 3502 > org.apache.lucene.codecs.lucene90.Lucene90NormsProducer$3#longValue() > 1.04% 3219 org.apache.lucene.search.ConjunctionDISI#doNext() > 1.00% 3099 > org.apache.lucene.codecs.lucene90.Lucene90PostingsReader#findFirstGreater() > 0.99% 3067 > org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$3#longValue() > 0.99% 3052 org.apache.lucene.store.ByteBufferGuard#getInt() > 0.89% 2762 > org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$BlockImpactsDocsEnum#advance() > 0.87% 2690 > org.apache.lucene.search.similarities.BM25Similarity$BM25Scorer#score() > 0.86% 2663 org.apache.lucene.store.ByteBufferGuard#getByte() > 0.80% 2476 > org.apache.lucene.codecs.lucene90.ForUtil#expand8() > 0.78% 2420 > org.apache.lucene.codecs.lucene90.Lucene90PostingsReader$EverythingEnum#skipPositions() > {code} -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org