[
https://issues.apache.org/jira/browse/SOLR-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12993331#comment-12993331
]
Yonik Seeley commented on SOLR-2342:
------------------------------------
I did a test on my laptop (2 cores, 4 virtual cores via hyper-threading)
using StreamingUpdateSolrServer with 1,2,4,8, and 16 threads.
The documents were just 1M unique ids, and the throughput ranged anywhere from
60,000 docs per second, to 67000.
The timing of different runs varied up to 6% for the same number of threads on
the client, but I couldn't detect a difference when the ReadWriteLock was
switched to fair mode.
> Lock starvation can cause commit to never run when many clients are adding
> docs
> -------------------------------------------------------------------------------
>
> Key: SOLR-2342
> URL: https://issues.apache.org/jira/browse/SOLR-2342
> Project: Solr
> Issue Type: Bug
> Components: update
> Reporter: Michael McCandless
> Priority: Minor
>
> I have a stress test, where 100 clients add 100 1MB docs and then call commit
> in the end. It's a falldown test (try to make Solr fall down) and nowhere
> near "actual" usage.
> But, after some initial commits that succeed, I'm seeing later commits always
> time out (client side timeout @ 10 minutes). Watching Solr's logging, no
> commit ever runs.
> Looking at the stack traces in the threads, this is not deadlock: the
> add/update calls are running, and new segments are being flushed to the index.
> Digging in the code a bit, we use ReentrantReadWriteLock, with add/update
> acquiring the readLock and commit acquiring the writeLock. But, according to
> the jdocs, the writeLock isn't given any priority over the readLock (unless
> you set fairness, which we don't). So I think this explains the starvation?
> However, this is not a real world use case (most apps would/should call
> commit less often, and from on client). Also, we could set fairness, but it
> seems to have some performance penalty, and I'm not sure we should penalize
> the "normal" case for this unusual one. EG see here (thanks Mark):
> http://www.javaspecialists.eu/archive/Issue165.html.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
