When a new node joins the ring, it needs to own new token ranges. This should be unique to the new node and we don’t want to end up in a situation where two nodes joining simultaneously can own same range (and ideally evenly distributed). Cassandra has this 2 minute wait rule for gossip state to propagate before a node is added. But this on its does not guarantees that token ranges can’t overlap. See this ticket for more details https://issues.apache.org/jira/browse/CASSANDRA-7069 To overcome this issue, the approach was to only allow one node joining at a time.
When you replace a dead node the new token range selection does not applies as the replacing node just owns the token ranges of the dead node. I think that’s why the restriction of only replacing one node at a time does not applies in this case. Thanks Alok Dwivedi Senior Consultant https://www.instaclustr.com/platform/ From: Fd Habash <fmhab...@gmail.com> Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org> Date: Wednesday, 1 May 2019 at 06:18 To: "user@cassandra.apache.org" <user@cassandra.apache.org> Subject: Bootstrapping to Replace a Dead Node vs. Adding a New Node: Consistency Guarantees Reviewing the documentation & based on my testing, using C* 2.2.8, I was not able to extend the cluster by adding multiple nodes simultaneously. I got an error message … Other bootstrapping/leaving/moving nodes detected, cannot bootstrap while cassandra.consistent.rangemovement is true I understand this is to force a node to bootstrap from the former owner of the range when adding a node as part of extending the cluster. However, I was able to bootstrap multiple nodes to replace dead nodes. C* did not complain about it. Is consistent range movement & the guarantee it offers to bootstrap from primary range owner not applicable when bootstrapping to replace dead nodes? ---------------- Thank you
