Hi there, I have a dead node in our cluster, which is a wired state right now, and can not be removed from cluster.
The nodestatus shows: Datacenter: DC1 =============== Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- Address Load Tokens Owns Host ID Rack DN 10.210.165.55 ? 256 ? null r1 I tried the unsafeAssassinateEndpoint, but got exception like: 2015-09-18_23:21:40.79760 INFO 23:21:40 InetAddress /10.210.165.55 is now DOWN 2015-09-18_23:21:40.80667 ERROR 23:21:40 Exception in thread Thread[GossipStage:1,5,main] 2015-09-18_23:21:40.80668 java.lang.NullPointerException: null 2015-09-18_23:21:40.80669 at org.apache.cassandra.service.StorageService.getApplicationStateValue(StorageService.java:1584) ~[apache-cassandra-2.1.8+git20150804.076b0b1.jar:2.1.8+git20150804.076b0b1] 2015-09-18_23:21:40.80669 at org.apache.cassandra.service.StorageService.getTokensFor(StorageService.java:1592) ~[apache-cassandra-2.1.8+git20150804.076b0b1.jar:2.1.8+git20150804.076b0b1] 2015-09-18_23:21:40.80670 at org.apache.cassandra.service.StorageService.handleStateLeft(StorageService.java:1822) ~[apache-cassandra-2.1.8+git20150804.076b0b1.jar:2.1.8+git20150804.076b0b1] 2015-09-18_23:21:40.80671 at org.apache.cassandra.service.StorageService.onChange(StorageService.java:1495) ~[apache-cassandra-2.1.8+git20150804.076b0b1.jar:2.1.8+git20150804.076b0b1] 2015-09-18_23:21:40.80671 at org.apache.cassandra.service.StorageService.onJoin(StorageService.java:2121) ~[apache-cassandra-2.1.8+git20150804.076b0b1.jar:2.1.8+git20150804.076b0b1] 2015-09-18_23:21:40.80672 at org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1009) ~[apache-cassandra-2.1.8+git20150804.076b0b1.jar:2.1.8+git20150804.076b0b1] 2015-09-18_23:21:40.80673 at org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1113) ~[apache-cassandra-2.1.8+git20150804.076b0b1.jar:2.1.8+git20150804.076b0b1] 2015-09-18_23:21:40.80673 at org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:49) ~[apache-cassandra-2.1.8+git20150804.076b0b1.jar:2.1.8+git20150804.076b0b1] 2015-09-18_23:21:40.80673 at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62) ~[apache-cassandra-2.1.8+git20150804.076b0b1.jar:2.1.8+git20150804.076b0b1] 2015-09-18_23:21:40.80674 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_45] 2015-09-18_23:21:40.80674 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_45] 2015-09-18_23:21:40.80674 at java.lang.Thread.run(Thread.java:744) ~[na:1.7.0_45] 2015-09-18_23:21:40.85812 WARN 23:21:40 Not marking nodes down due to local pause of 10852378435 > 5000000000 Any suggestions about how to remove it? Thanks. -- Dikang