[ https://issues.apache.org/jira/browse/CASSANDRA-3466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147070#comment-13147070 ]
Jonas Borgström commented on CASSANDRA-3466: -------------------------------------------- Brandon, you're right. After commenting out those two lines I can no longer reproduce the hint delivery problem. The AssertionError is still there though and I managed to reproduce it with my two node setup. I'll attach a tarball with my config, DEBUG-level log files and data directories shortly. What I did was basically: # Start a new cluster with a patched 1.0.2 and create keyspace and column family. # Bring down node2, trigger hints on node1 # Bring up node2, notice that hints are delivered, YAY! # Bring down node1, trigger hints on node2 # Bring up node1, noticed AssertionError in log file for node1 (node1/system.log:2036) To me it looks like the assertionError on node1 is triggered when node2 connects to deliver its hints. > Hinted handoff not working after rolling upgrade from 0.8.7 to 1.0.2 > -------------------------------------------------------------------- > > Key: CASSANDRA-3466 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3466 > Project: Cassandra > Issue Type: Bug > Affects Versions: 1.0.0 > Reporter: Jonas Borgström > Assignee: Brandon Williams > Labels: hintedhandoff > Fix For: 1.0.3 > > > While testing rolling upgrades from 0.8.7 to 1.0.2 on a test cluster I've > noticed that hinted hand-off didn't always work properly. Hints generated on > an upgraded node does not seem to be delivered to other newly upgraded nodes > once they rejoin the ring. They only way I've found to get a node to deliver > its hints is to restart it. > Here's some steps to reproduce this issue: > 1. Install cassandra 0.8.7 on node1 and node2 using default settings. > 2. Create keyspace foo with {replication_factor: 2}. Create column family bar > 3. Shutdown node2 > 4. Insert data into bar and verify that HintsColumnFamily on node2 contains > hints > 5. Start node2 and verify that hinted handoff is performed and > HintsColumnFamily becomes empty again. > 6. Upgrade and restart node1 > 7. Shutdown node2 > 8. Insert data into bar and verify that HintsColumnFamily on node2 contains > hints > 9. Upgrade and start node2 > 10. Notice that hinted handoff is *not* performed when "node2" comes back. > (Only if node1 is restarted) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira