[
https://issues.apache.org/jira/browse/CASSANDRA-2948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis reassigned CASSANDRA-2948:
-----------------------------------------
Assignee: Brandon Williams
> Nodetool move fails to stream out data from moved node to new endpoint.
> -----------------------------------------------------------------------
>
> Key: CASSANDRA-2948
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2948
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.2
> Reporter: Sam Overton
> Assignee: Brandon Williams
> Priority: Critical
> Labels: move, streaming
> Attachments: 2948.patch
>
>
> When moving a node in the ring with nodetool move, that node streams its data
> to itself instead of to the new endpoint responsible for its old range.
> Steps to reproduce:
> * Create a cluster (A,B,C,D) with tokens (0,4,8,C) using
> ByteOrderedPartitioner
> * Create a keyspace and CF with RF=1
> * Insert keys (2,6,A,E). This should put one key on each node.
> * Move node A to token 7. This should cause:
> + node C streams key 6 to node A
> + node A streams key E to node B
> However instead, node A streams key E to itself.
> Selected log messages from node A:
> INFO [RMI TCP Connection(6)-10.2.129.41] 2011-07-26 16:29:17,075
> StorageService.java (line 1878) Moving miles/10.2.129.41 from
> Token(bytes[00]) to Token(bytes[07]).
> DEBUG [RMI TCP Connection(6)-10.2.129.41] 2011-07-26 16:29:17,080
> StorageService.java (line 1941) Table ks: work map
> {/10.2.129.16=[(Token(bytes[04]),Token(bytes[07])]]}.
> INFO [RMI TCP Connection(6)-10.2.129.41] 2011-07-26 16:29:17,080
> StorageService.java (line 1946) Sleeping 30000 ms before start
> streaming/fetching ranges.
> ...
> INFO [RMI TCP Connection(6)-10.2.129.41] 2011-07-26 16:29:46,728
> StorageService.java (line 522) Moving: fetching new ranges and streaming old
> ranges
> DEBUG [RMI TCP Connection(6)-10.2.129.41] 2011-07-26 16:29:46,728
> StorageService.java (line 1960) [Move->STREAMING] Work Map:
> {ks={(Token(bytes[0c]),Token(bytes[00])]=[miles/10.2.129.41]}}
> DEBUG [RMI TCP Connection(6)-10.2.129.41] 2011-07-26 16:29:46,729
> StorageService.java (line 1965) [Move->FETCHING] Work Map:
> {ks={/10.2.129.16=[(Token(bytes[04]),Token(bytes[07])]]}}
> DEBUG [RMI TCP Connection(6)-10.2.129.41] 2011-07-26 16:29:46,730
> StorageService.java (line 2411) Requesting from /10.2.129.16 ranges
> (Token(bytes[04]),Token(bytes[07])]
> ...
> INFO [StreamStage:1] 2011-07-26 16:29:46,737 StreamOut.java (line 90)
> Beginning transfer to miles/10.2.129.41
> DEBUG [StreamStage:1] 2011-07-26 16:29:46,737 StreamOut.java (line 91) Ranges
> are (Token(bytes[0c]),Token(bytes[00])]
> This appears to be caused because in StorageService.move we call
> Gossiper.instance.addLocalApplicationState(ApplicationState.STATUS,
> valueFactory.moving(newToken));
> and then get the new token metadata in order to calculate where the new
> endpoint is that we should stream to
> TokenMetadata tokenMetaClone = tokenMetadata_.cloneAfterAllSettled();
> however, in addLocalApplicationState there is no notification broadcast for
> the change in local state, so tokenMetadata_ never updates the list of moving
> nodes, and the tokenMetaClone is still the state of the ring from before the
> move.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
