[ https://issues.apache.org/jira/browse/SOLR-7333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389653#comment-14389653 ]
Timothy Potter commented on SOLR-7333: -------------------------------------- Here's the original comment I made about this issue: https://issues.apache.org/jira/browse/SOLR-6816?focusedCommentId=14233700&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14233700 > Make the poll queue time configurable and use knowledge that a batch is being > processed to poll efficiently > ----------------------------------------------------------------------------------------------------------- > > Key: SOLR-7333 > URL: https://issues.apache.org/jira/browse/SOLR-7333 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud > Reporter: Timothy Potter > Assignee: Timothy Potter > > {{StreamingSolrClients}} uses {{ConcurrentUpdateSolrServer}} to stream > documents from leader to replica, by default it sets the {{pollQueueTime}} > for CUSS to 0 so that we don't impose an unnecessary wait when processing > single document updates or the last doc in a batch. However, the downside is > that replicas receive many more update requests than leaders; I've seen up to > 40x number of update requests between replica and leader. > If we're processing a batch of docs, then ideally the poll queue time should > be greater than 0 up until the last doc is pulled off the queue. If we're > processing a single doc, then the poll queue time should always be 0 as we > don't want the thread to wait unnecessarily for another doc that won't come. > Rather than force indexing applications to provide this optional parameter in > an update request, it would be better for server-side code that can detect > whether an update request is a single document or batch of documents to > override this value internally, i.e. it'll be 0 by default, but since > {{JavaBinUpdateRequestCodec}} can determine when it's seen the last doc in a > batch, it can override the pollQueueTime to something greater than 0. > This means that current indexing clients will see a boost when doing batch > updates without making any changes on their side. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org