[ https://issues.apache.org/jira/browse/LUCENE-5425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888463#comment-13888463 ]
Lei Wang commented on LUCENE-5425: ---------------------------------- I'm not exactly sure how to run the facets benchmarks. I did a run on ant run-task -Dtask.alg=conf/facets.alg, and changed "SearchSameRdr" Search > : 40 to 40000, to get stable results. I'm not sure how to read the results also..., but the numbers looks quite similar between trunk and the docidset on my box. trunk: [java] ------------> Report sum by Prefix (Search) and Round (4 about 4 out of 42) [java] Operation round facets runCnt recsPerRun rec/s elapsedSec avgUsedMem avgTotalMem [java] SearchSameRdr_40000 0 true 1 40000 7,155.64 5.59 31,610,096 51,212,288 [java] SearchSameRdr_40000 - 1 false - - 1 - - 40000 - 8,814.46 - - 4.54 - 33,534,008 - 49,209,344 [java] SearchSameRdr_40000 2 true 1 40000 9,088.84 4.40 35,673,136 48,373,760 [java] SearchSameRdr_40000 - 3 false - - 1 - - 40000 - 9,045.68 - - 4.42 - 35,279,544 - 47,661,056 [java] [java] [java] ------------> Report sum by Prefix (Populate) and Round (4 about 4 out of 42) [java] Operation round facets runCnt recsPerRun rec/s elapsedSec avgUsedMem avgTotalMem [java] Populate 0 true 1 21578 2,489.96 8.67 31,369,696 51,212,288 [java] Populate - - 1 false - - 1 - - 21578 - 3,973.85 - - 5.43 - 33,272,104 - 49,209,344 [java] Populate 2 true 1 21578 4,216.92 5.12 32,701,392 48,373,760 [java] Populate - - 3 false - - 1 - - 21578 - 4,366.25 - - 4.94 - 35,064,408 - 47,661,056 [java] [java] [java] ------------> Report sum by Prefix (MAddDocs) and Round (4 about 4 out of 42) [java] Operation round facets runCnt recsPerRun rec/s elapsedSec avgUsedMem avgTotalMem [java] MAddDocs_Exhaust 0 true 1 21578 3,469.13 6.22 24,536,720 51,212,288 [java] MAddDocs_Exhaust - 1 false - - 1 - - 21578 - 4,845.72 - - 4.45 - 34,857,920 - 49,209,344 [java] MAddDocs_Exhaust 2 true 1 21578 5,129.07 4.21 29,209,256 48,373,760 [java] MAddDocs_Exhaust - 3 false - - 1 - - 21578 - 5,259.08 - - 4.10 - 25,845,424 - 47,661,056 With the patch, but I changed the OpenBitSet to FixedBitSet, and use bits.iterator() to return iterator (It's still an OpenBitSetIterator), the result: [java] ------------> Report sum by Prefix (Search) and Round (4 about 4 out of 42) [java] Operation round facets runCnt recsPerRun rec/s elapsedSec avgUsedMem avgTotalMem [java] SearchSameRdr_40000 0 true 1 40000 7,280.67 5.49 25,424,104 51,113,984 [java] SearchSameRdr_40000 - 1 false - - 1 - - 40000 - 8,689.98 - - 4.60 - 31,356,960 - 49,053,696 [java] SearchSameRdr_40000 2 true 1 40000 9,157.51 4.37 38,849,248 47,632,384 [java] SearchSameRdr_40000 - 3 false - - 1 - - 40000 - 9,097.11 - - 4.40 - 39,840,912 - 46,465,024 [java] [java] [java] ------------> Report sum by Prefix (Populate) and Round (4 about 4 out of 42) [java] Operation round facets runCnt recsPerRun rec/s elapsedSec avgUsedMem avgTotalMem [java] Populate 0 true 1 21578 2,465.21 8.75 25,187,152 51,113,984 [java] Populate - - 1 false - - 1 - - 21578 - 2,651.19 - - 8.14 - 30,985,904 - 49,053,696 [java] Populate 2 true 1 21578 4,247.64 5.08 38,656,320 47,632,384 [java] Populate - - 3 false - - 1 - - 21578 - 4,298.41 - - 5.02 - 39,355,912 - 46,465,024 [java] [java] [java] ------------> Report sum by Prefix (MAddDocs) and Round (4 about 4 out of 42) [java] Operation round facets runCnt recsPerRun rec/s elapsedSec avgUsedMem avgTotalMem [java] MAddDocs_Exhaust 0 true 1 21578 3,404.01 6.34 34,015,968 51,113,984 [java] MAddDocs_Exhaust - 1 false - - 1 - - 21578 - 3,062.88 - - 7.05 - 30,420,848 - 49,053,696 [java] MAddDocs_Exhaust 2 true 1 21578 5,147.42 4.19 28,833,976 47,632,384 [java] MAddDocs_Exhaust - 3 false - - 1 - - 21578 - 5,129.07 - - 4.21 - 37,117,288 - 46,465,024 > Make creation of FixedBitSet in FacetsCollector overridable > ----------------------------------------------------------- > > Key: LUCENE-5425 > URL: https://issues.apache.org/jira/browse/LUCENE-5425 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/facet > Affects Versions: 4.6 > Reporter: John Wang > Attachments: facetscollector.patch, facetscollector.patch > > > In FacetsCollector, creation of bits in MatchingDocs are allocated per query. > For large indexes where maxDocs are large creating a bitset of maxDoc bits > will be expensive and would great a lot of garbage. > Attached patch is to allow for this allocation customizable while maintaining > current behavior. -- This message was sent by Atlassian JIRA (v6.1.5#6160) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org