Dominique:

In a word, “yes”. You’ve got it. A common misunderstanding is that ZK is 
actively involved in queries/updates/whatever. Basically, what ZK is 
responsible for is maintaining collection-wide resources, i.e. the current 
state of all the replicas, config files, etc., your “global configuration" and 
"collection configuration”, which should change very rarely thus rarely 
generate writes.

The “collection state” (including your “nodes state”) information changes more 
frequently and generates more writes as nodes come up and down, go into 
recovery, etc. That said, for a cluster where all the replicas are “active” and 
don’t go away or go into recovery etc, ZK won’t do any writes.

So the consequence is that when you power up a cluster, there will be a flurry 
of write operations managed by the Overseer, but after all the replicas are up, 
write activity should pretty much cease.

As long as the state is steady, i.e. no replicas changing state, each 
individual Solr node has a copy of the relevant collection’s “state.json” znode 
and has all the information it needs to query or index without asking Zookeeper 
without _either_ reading or writing to ZK.

One rather obscure cause for ZK writes is when using “schemaless” mode. When a 
new field is detected, the schema (and thus the collection’s configuration) is 
changed, which generates writes..

Best,
Erick


> On Nov 15, 2019, at 12:06 PM, Dominique Bejean <dominique.bej...@eolya.fr> 
> wrote:
> 
> Hi,
> 
> I would like to be certain to understand how Solr use Zookeeper and more
> precisely when Solr write into Zookeeper.
> 
> Solr stores various informations in ZK
> 
>   - globale configuration (autoscaling, security.json)
>   - collection configuration (configs)
>   - collections state (state.json, leaders, ...)
>   - nodes state (live_nodes, overseer)
> 
> 
> Writes in Zk occur when
> 
>   - a zookeeper member start or stop
>   - a solr node start or stop
>   - a configuration is loaded
>   - a collection is created, deleted or updated (nearly all call to
>   collection, core or config API)
> 
> 
> Write do not occur during
> 
>   - SolrJ client creation
>   - indexing data (Solrj, HTTP, DIH, ...)
>   - searching (Solrj, HTTP)
> 
> 
> In conclusion, if Solr nodes are stable (no failure, no maintenance), no
> calls to  collection, core or config API are done, so there is nearly no
> writes to ZK.
> 
> Is it correct ?
> 
> 
> Regards
> 
> Dominique

Reply via email to