[ https://issues.apache.org/jira/browse/SOLR-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17270973#comment-17270973 ]
Ilan Ginzburg commented on SOLR-14928: -------------------------------------- Very preliminary numbers running a Collection creation load from JMeter. 10 threads creating 10 collections each. Each collection has 2 shards with 2 replicas each. The cluster is composed of 3 nodes. All running on a MacBook Pro (created using the "cloud" example) but each given 10GB heap. The Overseer based cluster state change: * Average collection creation time: 10559ms * Min collection creation time: 8003ms * Max collection creation time: 14403ms The distributed cluster state change: * Average collection creation time: 7414ms * Min collection creation time: 5945ms * Max collection creation time: 10503ms I will rebase on latest master, cleanup the code a bit, run some more testing ans will be ready for a PR... (current non rebased branch is [here|https://github.com/murblanc/lucene-solr/tree/SOLR-14928_ClusterStateUpdater]) > Remove Overseer ClusterStateUpdater > ----------------------------------- > > Key: SOLR-14928 > URL: https://issues.apache.org/jira/browse/SOLR-14928 > Project: Solr > Issue Type: Sub-task > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud > Reporter: Ilan Ginzburg > Assignee: Ilan Ginzburg > Priority: Major > Labels: cluster, collection-api, overseer > > Remove the Overseer {{ClusterStateUpdater}} thread and associated Zookeeper > queue at {{<_chroot_>/overseer/queue}}. > Change cluster state updates so that each (Collection API) command execution > does the update directly in Zookeeper using optimistic locking (Compare and > Swap on the {{state.json}} Zookeeper files). > Following this change cluster state updates would still be happening only > from the Overseer node (that's where Collection API commands are executing), > but the code will be ready for distribution once such commands can be > executed by any node (other work done in the context of parent task > SOLR-14927). > See the [Cluster State > Updater|https://docs.google.com/document/d/1u4QHsIHuIxlglIW6hekYlXGNOP0HjLGVX5N6inkj6Ok/edit#heading=h.ymtfm3p518c] > section in the Removing Overseer doc. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org