[
https://issues.apache.org/jira/browse/LUCENE-5604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13967781#comment-13967781
]
Michael McCandless commented on LUCENE-5604:
--------------------------------------------
I ran performance tests on first 5M Wikipedia "medium" (1 KB sized)
docs and Geonames (sources for the benchmark are all in luceneutil):
{noformat}
Wiki first 5M docs, no merge policy, 64 MB RAM buffer, 4 indexing threads,
default codec:
trunk: 136.985 sec, 189729244 conflicts
murmur: 134.156 sec, 164990724 conflicts
Geonames, no merge policy, 64 MB RAM buffer, 4 indexing threads, default codec:
trunk: 167.354 sec, 236051203 conflicts
murmur: 168.101 sec, 179747265 conflicts
{noformat}
Net/net the indexing time is the same (within noise of run-to-run).
The conflict count is how many times we had to probe in the open
addressed hash table inside BytesRefHash, and Murmur3 gives a nice
reduction (~ 13-24%). I think we should switch.
> Should we switch BytesRefHash to MurmurHash3?
> ---------------------------------------------
>
> Key: LUCENE-5604
> URL: https://issues.apache.org/jira/browse/LUCENE-5604
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Fix For: 4.9, 5.0
>
> Attachments: LUCENE-5604.patch
>
>
> MurmurHash3 has better hashing distribution than the current hash function we
> use for BytesRefHash which is a simple multiplicative function with 31
> multiplier (same as Java's String.hashCode, but applied to bytes not chars).
> Maybe we should switch ...
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
