Timothy Potter created SOLR-7333: ------------------------------------ Summary: Make the poll queue time configurable and use knowledge that a batch is being processed to poll efficiently Key: SOLR-7333 URL: https://issues.apache.org/jira/browse/SOLR-7333 Project: Solr Issue Type: Sub-task Components: SolrCloud Reporter: Timothy Potter Assignee: Timothy Potter
{{StreamingSolrClients}} uses {{ConcurrentUpdateSolrServer}} to stream documents from leader to replica, by default it sets the {{pollQueueTime}} for CUSS to 0 so that we don't impose an unnecessary wait when processing single document updates or the last doc in a batch. However, the downside is that replicas receive many more update requests than leaders; I've seen up to 40x number of update requests between replica and leader. If we're processing a batch of docs, then ideally the poll queue time should be greater than 0 up until the last doc is pulled off the queue. If we're processing a single doc, then the poll queue time should always be 0 as we don't want the thread to wait unnecessarily for another doc that won't come. Rather than force indexing applications to provide this optional parameter in an update request, it would be better for server-side code that can detect whether an update request is a single document or batch of documents to override this value internally, i.e. it'll be 0 by default, but since {{JavaBinUpdateRequestCodec}} can determine when it's seen the last doc in a batch, it can override the pollQueueTime to something greater than 0. This means that current indexing clients will see a boost when doing batch updates without making any changes on their side. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org