[ 
https://issues.apache.org/jira/browse/CASSANDRA-3466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146124#comment-13146124
 ] 

Jonas Borgström commented on CASSANDRA-3466:
--------------------------------------------

> Do you observe the same behavior if both nodes start life as 1.x?

No. Hinted handoff seems to work well if I either start with 1.0.2 nodes or 
restart all nodes once all nodes have been upgraded to 1.0.2. Probably 
something in the gossip state confusing hinted hand-offs.

> Is node2 down long enough for node1 failure detector to notice? (i.e. shows 
> as DOWN in nodetool ring)

Yes. This from the node1 log:
{code}
Node2 is shut down
  INFO 19:39:08,331 InetAddress /127.0.0.2 is now dead.
Hint is triggered using set bar[x][x]='x'
  INFO 19:40:52,126 Node /127.0.0.2 has restarted, now UP
  INFO 19:40:52,127 InetAddress /127.0.0.2 is now UP
  INFO 19:40:52,127 Node /127.0.0.2 state jump to normal
Nothing happens, hint stays in HintsColumnFamily forever
{code}

Calling deliverHints() using JMX also does not seem to work. Restarting node1 
will though...

                
> Hinted handoff not working after rolling upgrade from 0.8.7 to 1.0.2
> --------------------------------------------------------------------
>
>                 Key: CASSANDRA-3466
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3466
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.0.2
>            Reporter: Jonas Borgström
>              Labels: hintedhandoff
>
> While testing rolling upgrades from 0.8.7 to 1.0.2 on a test cluster I've 
> noticed that hinted hand-off didn't always work properly. Hints generated on 
> an upgraded node does not seem to be delivered to other newly upgraded nodes 
> once they rejoin the ring. They only way I've found to get a node to deliver 
> its hints is to restart it.
> Here's some steps to reproduce this issue:
> 1. Install cassandra 0.8.7 on node1 and node2 using default settings.
> 2. Create keyspace foo with {replication_factor: 2}. Create column family bar
> 3. Shutdown node2 
> 4. Insert data into bar and verify that HintsColumnFamily on node2 contains 
> hints
> 5. Start node2 and verify that hinted handoff is performed and 
> HintsColumnFamily becomes empty again.
> 6. Upgrade and restart node1
> 7. Shutdown node2 
> 8. Insert data into bar and verify that HintsColumnFamily on node2 contains 
> hints
> 9. Upgrade and start node2
> 10. Notice that hinted handoff is *not* performed when "node2" comes back. 
> (Only if node1 is restarted)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


Reply via email to