[ 
https://issues.apache.org/jira/browse/SOLR-7333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389653#comment-14389653
 ] 

Timothy Potter commented on SOLR-7333:
--------------------------------------

Here's the original comment I made about this issue:

https://issues.apache.org/jira/browse/SOLR-6816?focusedCommentId=14233700&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14233700

> Make the poll queue time configurable and use knowledge that a batch is being 
> processed to poll efficiently
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: SOLR-7333
>                 URL: https://issues.apache.org/jira/browse/SOLR-7333
>             Project: Solr
>          Issue Type: Sub-task
>          Components: SolrCloud
>            Reporter: Timothy Potter
>            Assignee: Timothy Potter
>
> {{StreamingSolrClients}} uses {{ConcurrentUpdateSolrServer}} to stream 
> documents from leader to replica, by default it sets the {{pollQueueTime}} 
> for CUSS to 0 so that we don't impose an unnecessary wait when processing 
> single document updates or the last doc in a batch. However, the downside is 
> that replicas receive many more update requests than leaders; I've seen up to 
> 40x number of update requests between replica and leader.
> If we're processing a batch of docs, then ideally the poll queue time should 
> be greater than 0 up until the last doc is pulled off the queue. If we're 
> processing a single doc, then the poll queue time should always be 0 as we 
> don't want the thread to wait unnecessarily for another doc that won't come.
> Rather than force indexing applications to provide this optional parameter in 
> an update request, it would be better for server-side code that can detect 
> whether an update request is a single document or batch of documents to 
> override this value internally, i.e. it'll be 0 by default, but since 
> {{JavaBinUpdateRequestCodec}} can determine when it's seen the last doc in a 
> batch, it can override the pollQueueTime to something greater than 0.
> This means that current indexing clients will see a boost when doing batch 
> updates without making any changes on their side.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to