Hello Cassandra-users, I was running some tests today. My end goal was to learn more about replacing a dead node in a live Cassandra cluster with minimal disruption to the existing cluster and figure out a better and faster way of doing the same.
I am running a package installation of the following version of Cassandra. [centos@rj-cassandra-1 testcf-97896450869d11e6a84c4381bf5c5035]$ nodetool version ReleaseVersion: 2.1.12 I setup a 4 node Cassandra in the lab. I got one non-seed node (lets say node1) down by issuing a 'sudo service cassandra stop'. Then following following instructions from this link <https://docs.datastax.com/en/cassandra/2.1/cassandra/operations/opsReplaceNode.html>, I tried to replace node1 with the JMX option -Dcassandra.replace_address=<ip_address_of_node1>. However, when I do this the bootstrap fails with the following error in the log: ERROR [main] 2016-09-30 23:54:17,104 CassandraDaemon.java:579 - Exception encountered during startup java.lang.RuntimeException: Unable to gossip with any seeds at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1337) ~[apache-cassandra-2.1.12.jar:2.1.12] at org.apache.cassandra.service.StorageService.prepareReplacementInfo(StorageService.java:512) ~[apache-cassandra-2.1.12.jar:2.1.12] at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:783) ~[apache-cassandra-2.1.12.jar:2.1.12] at org.apache.cassandra.service.StorageService.initServer(StorageService.java:721) ~[apache-cassandra-2.1.12.jar:2.1.12] at org.apache.cassandra.service.StorageService.initServer(StorageService.java:612) ~[apache-cassandra-2.1.12.jar:2.1.12] at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:387) [apache-cassandra-2.1.12.jar:2.1.12] at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:562) [apache-cassandra-2.1.12.jar:2.1.12] at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:651) [apache-cassandra-2.1.12.jar:2.1.12] WARN [StorageServiceShutdownHook] 2016-09-30 23:54:17,109 Gossiper.java:1454 - No local state or state is in silent shutdown, not announcing shutdown INFO [StorageServiceShutdownHook] 2016-09-30 23:54:17,109 MessagingService.java:734 - Waiting for messaging service to quiesce INFO [ACCEPT-/10.7.0.232] 2016-09-30 23:54:17,110 MessagingService.java:1018 - MessagingService has terminated the accept() thread How do I recover from this error message ? ------------------------ Rajath Subramanyam
