[
https://issues.apache.org/jira/browse/LUCENE-7905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085423#comment-16085423
]
Dawid Weiss commented on LUCENE-7905:
-------------------------------------
bq. I do feel like how we compare terms in the PQ is inefficient, and we should
be able to do something like what radix sort does, because at any given time,
the terms in the queue likely share long common prefixes yet we keep
inefficiently re-comparing those long common prefixes.
You could do it with radix-like sort, but you'd need all the terms from all the
TermEnums; the pq has the advantage of scanning through them on the fly,
progressively?
> Optimizations for OrdinalMap
> ----------------------------
>
> Key: LUCENE-7905
> URL: https://issues.apache.org/jira/browse/LUCENE-7905
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Fix For: 7.1
>
> Attachments: LUCENE-7905.patch, LUCENE-7905.patch,
> LUCENE-7905-specialized.patch
>
>
> {{OrdinalMap}} is a useful class to quickly map per-segment ordinals to
> global space, but it's fairly costly to build, which must typically be done
> on every NRT refresh.
> I'm using it quite heavily in two different places, one for
> {{SortedSetDocValuesFacetCounts}}, and another custom usage, and I found some
> small optimizations to improve its construction time.
> I switched it to use a simple priority queue to merge the terms instead of
> the more general {{MultiTermsEnum}}, which does extra work since it must also
> provide postings, implement seekExact, etc.
> I also pulled {{OrdinalMap}} out into its own oal.index class.
> When testing construction time for my case the patch is ~16% faster (159.9s
> -> 134.2s) in one case with 91.4 M terms and ~9% faster (115.6s -> 105.7s) in
> another case with 26.6 M terms.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
