Jonathan Shook created CASSANDRA-11688:
------------------------------------------

             Summary: Replace_address should sanity check prior node state 
before migrating tokens
                 Key: CASSANDRA-11688
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11688
             Project: Cassandra
          Issue Type: Improvement
            Reporter: Jonathan Shook


During a node replacement, a customer used an ip address associated with a 
different node than the intended one. The result was that both nodes remained 
active after the node came up. This caused several other issues which were 
difficult to diagnose, including invalid gossip state, etc.

Replace_address should be more robust in this scenario. It would be much more 
user friendly if the replace_address logic would first do some basic sanity 
checks, possibly to include:

- Pinging the other node to see if it is indeed “down”, if the address is 
different than all local interface addresses
- Checking gossip state of the node to verify that it is not known to peers.

It may even be safest to require that both address reachability and gossip 
state are required to show the replace_address as down by default before 
allowing any token migration or other replace_address actions to occur.

In the case that the replace_address is not ready to be replaced, the log 
should indicate that you are trying to replace an active node, and cassandra 
should refuse to start.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to