[
https://issues.apache.org/jira/browse/SOLR-11443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16201107#comment-16201107
]
Scott Blum edited comment on SOLR-11443 at 10/11/17 10:47 PM:
--------------------------------------------------------------
it might be more clear to do the `processedNodes.add(head.first());` _after_
calling processQueueItem, maybe conditionally based on whether onWriteAfter was
called. IE:
{code}
Set<String> processedNodes = new HashSet<>();
String[] curNode = new String[1];
while (!queue.isEmpty()) {
for (Pair<String, byte[]> head : queue) {
curNode[0] = head.first();
byte[] data = head.second();
final ZkNodeProps message = ZkNodeProps.load(data);
log.debug("processMessage: queueSize: {}, message = {} current
state version: {}", stateUpdateQueue.getStats().getQueueLength(), message,
clusterState.getZkClusterStateVersion());
// The callback always be called on this thread
clusterState = processQueueItem(message, clusterState,
zkStateWriter, true, new ZkStateWriter.ZkWriteCallback() {
@Override
public void onWriteBefore() throws Exception {
stateUpdateQueue.remove(processedNodes);
processedNodes.clear();
}
@Override
public void onWriteAfter() throws Exception {
processedNodes.add(curNode[0]);
stateUpdateQueue.remove(processedNodes);
processedNodes.clear();
curNode[0] = null;
}
});
}
if (curNode[0] != null) {
// e.g. onWriteAfter was not called
processedNodes.add(curNode[0]);
}
queue = new LinkedList<>(stateUpdateQueue.peekElements(1000, 100,
node -> !processedNodes.contains(node)));
}
{code}
was (Author: dragonsinth):
it might be more clear to do the `processedNodes.add(head.first());` _after_
calling processQueueItem, maybe conditionally based on whether onWriteAfter was
called.
> Remove the usage of workqueue for Overseer
> ------------------------------------------
>
> Key: SOLR-11443
> URL: https://issues.apache.org/jira/browse/SOLR-11443
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Cao Manh Dat
> Assignee: Cao Manh Dat
> Attachments: SOLR-11443.patch, SOLR-11443.patch
>
>
> If we can remove the usage of workqueue, We can save a lot of IO blocking in
> Overseer, hence boost performance a lot.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
