[
https://issues.apache.org/jira/browse/LUCENE-7994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16206176#comment-16206176
]
Dawid Weiss commented on LUCENE-7994:
-------------------------------------
The key value rehash function is pretty simplistic in those implementations.
I've had bad experiences with collisions on such trivial functions in real life
(in HPPC); these can vary from slow-downs to actual practical deadlocks (not to
mention intentional adversaries) [1].
The current implementation in HPPC uses a different key mixing strategy [2],
combined with a unique per-instance seed to minimize the practical impact of
such clashes. The performance cost is there, but it's not huge... something to
consider?
[1] http://issues.carrot2.org/browse/HPPC-80
http://issues.carrot2.org/browse/HPPC-103
[2]
https://github.com/carrotsearch/hppc/blob/master/hppc/src/main/java/com/carrotsearch/hppc/BitMixer.java
> Use int/int hash map for int taxonomy facet counts
> --------------------------------------------------
>
> Key: LUCENE-7994
> URL: https://issues.apache.org/jira/browse/LUCENE-7994
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Fix For: master (8.0), 7.2
>
> Attachments: LUCENE-7994.patch
>
>
> Int taxonomy facets today always count into a dense {{int[]}}, which is
> wasteful in cases where the number of unique facet labels is high and the
> size of the current result set is small.
> I factored the native hash map from LUCENE-7927 and use a simple heuristic
> (customizable by the user by subclassing) to decide up front whether to count
> sparse or dense. I also made loading of the large children and siblings
> {{int[]}} lazy, so that they are only instantiated if you really need them.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
