I'm not sure this is an edge case at all. LazyCollectionRef goes to Zookeeper for any collection that does _not_ have a replica on the Solr node in question IIUC. There's even a comment in the "get()" method:
// TODO: consider limited caching So consider someone sending HTTP request to update documents to Solr. Or even queries. Or fronting the cluster with a load-balancer. Any request that ends up on a Solr node that does not host at least one replica for that collection will cause that node to go to ZK to get the collection state _even if it's just been fetched_. I've seen a situation "in the field" where there are over 1,500 threads on a Solr client going to ZK to get a collection state. I suspect there's something _else_ going on here that causes the SolrJ client (this particular client is certainly using SolrJ) to send the updates to the wrong Solr node or something, but still this is egregious. We really shouldn't need to pull the state from ZK so often in that case. OTOH, we don't want to cache the collection state (and set a watch) for any collection ever referenced by a Solr instance or we might as well go back to the single clusterstate.json.... Or I completely misunderstand the situation, wouldn't be the first time. On Tue, Apr 26, 2016 at 11:06 AM, Scott Blum (JIRA) <[email protected]> wrote: > > [ > https://issues.apache.org/jira/browse/SOLR-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15258576#comment-15258576 > ] > > Scott Blum commented on SOLR-9014: > ---------------------------------- > > Correct, the edge case is what we're guarding against here. I would be > completely on board with making collectionStates() cheap, but we would need > to go update all the call sites to guard against that edge case; ie, using a > key returned from collectionStates(), all the call sites need to expect that > getCollection(key) might return null. > >> Deprecate and reduce usage of ClusterState methods which may make calls to >> ZK via the lazy collection reference >> --------------------------------------------------------------------------------------------------------------- >> >> Key: SOLR-9014 >> URL: https://issues.apache.org/jira/browse/SOLR-9014 >> Project: Solr >> Issue Type: Improvement >> Components: SolrCloud >> Reporter: Shalin Shekhar Mangar >> Fix For: master, 6.1 >> >> Attachments: SOLR-9014.patch >> >> >> ClusterState has a bunch of methods such as getSlice and getReplica which >> internally call getCollectionOrNull that ends up making a call to ZK via the >> lazy collection reference. Many classes use these methods even though a >> DocCollection object is available. In such cases, multiple redundant calls >> to ZooKeeper can happen if the collection is not watched locally. This is >> especially true for Overseer classes which operate on all collections. >> We should audit all usages of these methods and replace them with calls to >> appropriate DocCollection methods. > > > > -- > This message was sent by Atlassian JIRA > (v6.3.4#6332) > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
