[
https://issues.apache.org/jira/browse/SOLR-906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan McKinley updated SOLR-906:
-------------------------------
Attachment: SOLR-906-StreamingHttpSolrServer.patch
Here is an updated version that lets you specify how many threads should work
on emptying the Queue. It also adds tests to make sure it passes all the same
tests that CommonsHttpSolrServer and EmbeddedSolrServer already pass. That
is, it is a drop in replacement and passes all existing tests.
One big change is that calling commit or optimize with waitSearcher=true:
1. blocks adding new docs to the Queue
2. waits for the Queue to empty (send all docs)
3. waits for <commit waitSearcher=true /> to return
4. unblocks everything
5. finally continues execution.
My threading chops are not great, so I may be doing something really strange.
It would be good to get some more eyes on this!
> Buffered / Streaming SolrServer implementaion
> ---------------------------------------------
>
> Key: SOLR-906
> URL: https://issues.apache.org/jira/browse/SOLR-906
> Project: Solr
> Issue Type: New Feature
> Components: clients - java
> Reporter: Ryan McKinley
> Fix For: 1.4
>
> Attachments: SOLR-906-StreamingHttpSolrServer.patch,
> StreamingHttpSolrServer.java
>
>
> While indexing lots of documents, the CommonsHttpSolrServer add(
> SolrInputDocument ) is less then optimal. This makes a new request for each
> document.
> With a "StreamingHttpSolrServer", documents are buffered and then written to
> a single open Http connection.
> For related discussion see:
> http://www.nabble.com/solr-performance-tt9055437.html#a20833680
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
