[
https://issues.apache.org/jira/browse/SOLR-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978559#action_12978559
]
Johannes commented on SOLR-1711:
--------------------------------
We are still seeing the same issue with Solr1.4.1
We get into this situation when all the runner threads die due to a broken
pipe, while the BlockingQueue is still full. All of the producer threads are
all blocked on the BlockingQueue.put() method. Since the runners are spawned by
the producers, which are all blocked, runner threads never get created to drain
the queue.
Here's a potential fix. In the runner code, replace these lines:
// remove it from the list of running things...
synchronized (runners) {
runners.remove( this );
}
with these lines:
// remove it from the list of running things unless we are the last
runner and the queue is full...
synchronized (runners) {
if (runners.size() == 1 && queue.remainingCapacity() == 0) {
// keep this runner alive
scheduler.execute(this);
} else {
runners.remove( this );
}
}
> Race condition in
> org/apache/solr/client/solrj/impl/StreamingUpdateSolrServer.java
> ----------------------------------------------------------------------------------
>
> Key: SOLR-1711
> URL: https://issues.apache.org/jira/browse/SOLR-1711
> Project: Solr
> Issue Type: Bug
> Components: clients - java
> Affects Versions: 1.4, 1.5
> Reporter: Attila Babo
> Priority: Critical
> Fix For: 1.4.1, 1.5, 3.1, 4.0
>
> Attachments: StreamingUpdateSolrServer.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> While inserting a large pile of documents using StreamingUpdateSolrServer
> there is a race condition as all Runner instances stop processing while the
> blocking queue is full. With a high performance client this could happen
> quite often, there is no way to recover from it at the client side.
> In StreamingUpdateSolrServer there is a BlockingQueue called queue to store
> UpdateRequests, there are up to threadCount number of workers threads from
> StreamingUpdateSolrServer.Runner to read that queue and push requests to a
> Solr instance. If at one point the BlockingQueue is empty all workers stop
> processing it and pushing the collected content to Solr which could be a time
> consuming process, sometimes all worker threads are waiting for Solr. If at
> this time the client fills the BlockingQueue to full all worker threads will
> quit without processing any further and the main thread will block forever.
> There is a simple, well tested patch attached to handle this situation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
