[ https://issues.apache.org/jira/browse/SOLR-11443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16201107#comment-16201107 ]
Scott Blum edited comment on SOLR-11443 at 10/11/17 10:47 PM: -------------------------------------------------------------- it might be more clear to do the `processedNodes.add(head.first());` _after_ calling processQueueItem, maybe conditionally based on whether onWriteAfter was called. IE: {code} Set<String> processedNodes = new HashSet<>(); String[] curNode = new String[1]; while (!queue.isEmpty()) { for (Pair<String, byte[]> head : queue) { curNode[0] = head.first(); byte[] data = head.second(); final ZkNodeProps message = ZkNodeProps.load(data); log.debug("processMessage: queueSize: {}, message = {} current state version: {}", stateUpdateQueue.getStats().getQueueLength(), message, clusterState.getZkClusterStateVersion()); // The callback always be called on this thread clusterState = processQueueItem(message, clusterState, zkStateWriter, true, new ZkStateWriter.ZkWriteCallback() { @Override public void onWriteBefore() throws Exception { stateUpdateQueue.remove(processedNodes); processedNodes.clear(); } @Override public void onWriteAfter() throws Exception { processedNodes.add(curNode[0]); stateUpdateQueue.remove(processedNodes); processedNodes.clear(); curNode[0] = null; } }); } if (curNode[0] != null) { // e.g. onWriteAfter was not called processedNodes.add(curNode[0]); } queue = new LinkedList<>(stateUpdateQueue.peekElements(1000, 100, node -> !processedNodes.contains(node))); } {code} was (Author: dragonsinth): it might be more clear to do the `processedNodes.add(head.first());` _after_ calling processQueueItem, maybe conditionally based on whether onWriteAfter was called. > Remove the usage of workqueue for Overseer > ------------------------------------------ > > Key: SOLR-11443 > URL: https://issues.apache.org/jira/browse/SOLR-11443 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Reporter: Cao Manh Dat > Assignee: Cao Manh Dat > Attachments: SOLR-11443.patch, SOLR-11443.patch > > > If we can remove the usage of workqueue, We can save a lot of IO blocking in > Overseer, hence boost performance a lot. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org