Your thoughts regarding Gossip are correct. There could be a time that nodes in the cluster hold different views of the ring locally.
In the case of bootstrapping, 1. The joining node updates its status to BOOT before streaming data and waits for a certain delay in order to populate the update in the cluster. 2. The replica nodes calculate the pending ranges, and the joining node can get new writes for the token ranges, meanwhile data streaming is on-going. 3. When the other nodes learn the joining node has finished bootstrapping via gossip (after some delay), the new node starts to serve the read as well. (I was wrong about not getting any client traffic earlier. The joining node accepts write, but no read) - Yifan On Tue, Jan 26, 2021 at 11:36 AM Han <keepsim...@gmail.com> wrote: > >>> I'm particularly trying to understand the fault-tolerant part of >>> updating Token Ring state on every node >> >> The new node only joins the ring (updates the rings state) when the data >> streaming (bootstrapping) is successful. Otherwise, the existing ring >> remains as is, the joining node remains in JOINING state, and it won't get >> any client traffic. If I understand the question correctly. >> > > Thanks Yifan for your response. > > The ring state update is propagated via gossip, so it is "eventually > consistent". This means there is a time period when some existing node has > the new token ring but other existing nodes still have the old ring info, > right? Is it true that other operations (e.g. replications) are still > going on between nodes, even when their local token rings info are not > consistent? > > From the new node point of view, even when it is successful, at the end > of `StorageService::joinTokenRing`, it is possible that some existing nodes > have not updated their token ring yet, is this correct? > > Thanks > Han > > > >> >> Hopefully, the answers help. >> >> - Yifan >> >> On Sun, Jan 24, 2021 at 1:00 PM Han <keepsim...@gmail.com> wrote: >> >>> Hi, >>> >>> I wanted to understand how the bootstrapping (add a new node) works. My >>> understanding is that the first step is Token Allocation and the new node >>> will get a number of tokens. >>> >>> My question is: >>> >>> How / when do the existing nodes update their Token Ring state? and is >>> that different between the seed node and non-seed node? >>> >>> I'm particularly trying to understand the fault-tolerant part of >>> updating Token Ring state on every node, but couldn't find relevant info by >>> searching. >>> >>> Any info or pointers are appreciated. >>> >>> Thanks! >>> Han >>> >>>
