[
https://issues.apache.org/jira/browse/SOLR-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14488125#comment-14488125
]
Yonik Seeley commented on SOLR-6820:
------------------------------------
At first I couldn't understand why such a large number of buckets was needed...
the math for two documents wanting to use the same bucket at the same time
should be similar to the birthday problem:
http://en.wikipedia.org/wiki/Birthday_problem , with "n" being the number of
indexing threads, and "d" being the number of buckets.
But then I realized... if one add takes a long time because lucene used the
thread to flush a segment, then other indexing threads will quickly pile up on
the same bucket. Basically, and indexing thread quickly indexes documents into
random slots until it accidentally hits the same bucket as the blocked thread,
and then it stops.
I guess having lots of buckets is the best work-around we can do for now (but
it still doesn't cure the problem). The root cure would be a way for Lucene to
not use client threads for long running index operations.
> The sync on the VersionInfo bucket in DistributedUpdateProcesser#addDocument
> appears to be a large bottleneck when using replication.
> -------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-6820
> URL: https://issues.apache.org/jira/browse/SOLR-6820
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
> Reporter: Mark Miller
> Assignee: Timothy Potter
> Attachments: SOLR-6820.patch, threads.png
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
