[ https://issues.apache.org/jira/browse/CASSANDRA-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068725#comment-13068725 ]
paul cannon commented on CASSANDRA-2496: ---------------------------------------- Ok, nodes do indeed infinitely retry the replication confirmation in some cases, but it appears it's not just when the former removal coordinator has restarted in the interim- it seems to be when the removetoken is reissued to another, new removal coordinator. In this case, I get this traceback every 10 seconds: {noformat} ERROR [MiscStage:9] 2011-07-20 23:42:06,599 AbstractCassandraDaemon.java (line 113) Fatal exception in thread Thread[MiscStage:9,5,main] java.lang.AssertionError at org.apache.cassandra.service.StorageService.confirmReplication(StorageService.java:2088) at org.apache.cassandra.streaming.ReplicationFinishedVerbHandler.doVerb(ReplicationFinishedVerbHandler.java:38) at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:72) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) {noformat} I'll look into this. Second, it seems that moving/joining nodes can take over the removed token fine, once the removetoken is complete. I haven't tried having a node take over the removed token while the removal is ongoing- I assume we can just document that that probably isn't a great idea? > Gossip should handle 'dead' states > ---------------------------------- > > Key: CASSANDRA-2496 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2496 > Project: Cassandra > Issue Type: Bug > Components: Core > Reporter: Brandon Williams > Assignee: Brandon Williams > Attachments: 0001-Rework-token-removal-process.txt, > 0002-add-2115-back.txt, 0003-update-gossip-related-comments.patch.txt, > 0004-do-REMOVING_TOKEN-REMOVED_TOKEN.patch.txt, > 0005-drain-self-if-removetoken-d-elsewhere.patch.txt > > > For background, see CASSANDRA-2371 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira