[
https://issues.apache.org/jira/browse/LUCENE-5425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13890911#comment-13890911
]
John Wang commented on LUCENE-5425:
-----------------------------------
Hi Shai:
Our motivation for generalization also comes from performance: Large index
size (e.g. large maxDoc) and very high query throughput. Creating a BitSet of
maxDoc bits per query is not feasible. So being able to cache in a thread local
would work better (num threads is low due to async nature of our system), which
is what the first patch is about.
If we were to reuse the bitset, we would have to zero out the bits before
reuse. We found given our doc size, zeroing out the bitset is expensive,
specifically, 4x that of creating the bits in the first place.
So if we can abstract this out, we can plugin more optimized
implementations. We can put together a diciest impl that would perform much
better than FixedBitSet.
Having said this, I am happy to break this into multiple patches/issues.
Thanks
-John
> Make creation of FixedBitSet in FacetsCollector overridable
> -----------------------------------------------------------
>
> Key: LUCENE-5425
> URL: https://issues.apache.org/jira/browse/LUCENE-5425
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/facet
> Affects Versions: 4.6
> Reporter: John Wang
> Attachments: facetscollector.patch, facetscollector.patch,
> fixbitset.patch
>
>
> In FacetsCollector, creation of bits in MatchingDocs are allocated per query.
> For large indexes where maxDocs are large creating a bitset of maxDoc bits
> will be expensive and would great a lot of garbage.
> Attached patch is to allow for this allocation customizable while maintaining
> current behavior.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
