Arcadius: Note that one of the more recent changes is "per collection states" in ZK. So rather than have one huge clusterstate.json that gets passed out to to all collection on any change, the listeners can now listen only to specific collections.
Reduces the "thundering herd" problem. Best, Erick On Fri, Sep 4, 2015 at 12:39 AM, Arcadius Ahouansou <arcad...@menelic.com> wrote: > Hello Shawn. > This question was raised because IMHO, apart from leader election, there > are other load-generating activities such as all 10 solrj > clients+solrCloudNodes listening to changes on clusterstate.json/state.json > and downloading the whole file in case there is a change... And this would > have happened on zk1 only if we did not shuffle... That's the theory. > I could test this and see. > On Sep 4, 2015 6:27 AM, "Shawn Heisey" <apa...@elyograg.org> wrote: > >> On 9/3/2015 9:47 PM, Arcadius Ahouansou wrote: >> > Let's say we have 10 SolrJ clients all configured with >> > zkhost=zk1:port,zk2:port,zk3:port >> > >> > For each of the 10 SolrJ clients, would it make a difference in term of >> > load on zk1 (the server on the list) if we shuffle around the order of >> the >> > ZK servers in zkHost or is it all the same? >> > >> > I would have thought that shuffling would lower load on zk1. >> >> I don't think this is going to make much difference. Here's why, >> assuming that my understanding of how it all works is correct: >> >> One of the things zookeeper does is manage elections. It helps figure >> out which member of a cluster is the leader. I think Zookeeper uses >> this concept internally, too. One of the hosts in the ensemble will be >> elected to be the leader, which accepts all input and replicates it to >> the other members of the cluster. All of the clients will be talking to >> the leader first, no matter what order the hosts are listed. >> >> If my understanding of how this works is flawed, then what I just said >> is probably wrong. >> >> Thanks, >> Shawn >> >>
