[jira] [Updated] (CASSANDRA-8132) Save or stream hints to a safe place in node replacement
[ https://issues.apache.org/jira/browse/CASSANDRA-8132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-8132: Component/s: Hints > Save or stream hints to a safe place in node replacement > > > Key: CASSANDRA-8132 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8132 > Project: Cassandra > Issue Type: New Feature > Components: Hints >Reporter: Minh Do >Assignee: Minh Do >Priority: Major > Fix For: 4.x > > > Often, we need to replace a node with a new instance in the cloud environment > where we have all nodes are still alive. To be safe without losing data, we > usually make sure all hints are gone before we do this operation. > Replacement means we just want to shutdown C* process on a node and bring up > another instance to take over that node's token. > However, if a node to be replaced has a lot of stored hints, its > HintedHandofManager seems very slow to send the hints to other nodes. In our > case, we tried to replace a node and had to wait for several days before its > stored hints are clear out. As mentioned above, we need all hints on this > node to clear out before we can terminate it and replace it by a new > instance/machine. > Since this is not a decommission, I am proposing that we have the same > hints-streaming mechanism as in the decommission code. Furthermore, there > needs to be a cmd for NodeTool to trigger this. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-8132) Save or stream hints to a safe place in node replacement
[ https://issues.apache.org/jira/browse/CASSANDRA-8132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-8132: -- Fix Version/s: (was: 2.1.x) 3.x > Save or stream hints to a safe place in node replacement > > > Key: CASSANDRA-8132 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8132 > Project: Cassandra > Issue Type: New Feature >Reporter: Minh Do >Assignee: Minh Do > Fix For: 3.x > > > Often, we need to replace a node with a new instance in the cloud environment > where we have all nodes are still alive. To be safe without losing data, we > usually make sure all hints are gone before we do this operation. > Replacement means we just want to shutdown C* process on a node and bring up > another instance to take over that node's token. > However, if a node to be replaced has a lot of stored hints, its > HintedHandofManager seems very slow to send the hints to other nodes. In our > case, we tried to replace a node and had to wait for several days before its > stored hints are clear out. As mentioned above, we need all hints on this > node to clear out before we can terminate it and replace it by a new > instance/machine. > Since this is not a decommission, I am proposing that we have the same > hints-streaming mechanism as in the decommission code. Furthermore, there > needs to be a cmd for NodeTool to trigger this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8132) Save or stream hints to a safe place in node replacement
[ https://issues.apache.org/jira/browse/CASSANDRA-8132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-8132: Issue Type: New Feature (was: Sub-task) Parent: (was: CASSANDRA-9427) > Save or stream hints to a safe place in node replacement > > > Key: CASSANDRA-8132 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8132 > Project: Cassandra > Issue Type: New Feature >Reporter: Minh Do >Assignee: Minh Do > Fix For: 2.1.x > > > Often, we need to replace a node with a new instance in the cloud environment > where we have all nodes are still alive. To be safe without losing data, we > usually make sure all hints are gone before we do this operation. > Replacement means we just want to shutdown C* process on a node and bring up > another instance to take over that node's token. > However, if a node to be replaced has a lot of stored hints, its > HintedHandofManager seems very slow to send the hints to other nodes. In our > case, we tried to replace a node and had to wait for several days before its > stored hints are clear out. As mentioned above, we need all hints on this > node to clear out before we can terminate it and replace it by a new > instance/machine. > Since this is not a decommission, I am proposing that we have the same > hints-streaming mechanism as in the decommission code. Furthermore, there > needs to be a cmd for NodeTool to trigger this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8132) Save or stream hints to a safe place in node replacement
[ https://issues.apache.org/jira/browse/CASSANDRA-8132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-8132: - Issue Type: Sub-task (was: Improvement) Parent: CASSANDRA-9427 > Save or stream hints to a safe place in node replacement > > > Key: CASSANDRA-8132 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8132 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Minh Do >Assignee: Minh Do > Fix For: 2.1.x > > > Often, we need to replace a node with a new instance in the cloud environment > where we have all nodes are still alive. To be safe without losing data, we > usually make sure all hints are gone before we do this operation. > Replacement means we just want to shutdown C* process on a node and bring up > another instance to take over that node's token. > However, if a node to be replaced has a lot of stored hints, its > HintedHandofManager seems very slow to send the hints to other nodes. In our > case, we tried to replace a node and had to wait for several days before its > stored hints are clear out. As mentioned above, we need all hints on this > node to clear out before we can terminate it and replace it by a new > instance/machine. > Since this is not a decommission, I am proposing that we have the same > hints-streaming mechanism as in the decommission code. Furthermore, there > needs to be a cmd for NodeTool to trigger this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8132) Save or stream hints to a safe place in node replacement
[ https://issues.apache.org/jira/browse/CASSANDRA-8132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Minh Do updated CASSANDRA-8132: --- Due Date: 15/Jan/15 (was: 28/Nov/14) > Save or stream hints to a safe place in node replacement > > > Key: CASSANDRA-8132 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8132 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Minh Do >Assignee: Minh Do > Fix For: 2.1.3 > > > Often, we need to replace a node with a new instance in the cloud environment > where we have all nodes are still alive. To be safe without losing data, we > usually make sure all hints are gone before we do this operation. > Replacement means we just want to shutdown C* process on a node and bring up > another instance to take over that node's token. > However, if a node to be replaced has a lot of stored hints, its > HintedHandofManager seems very slow to send the hints to other nodes. In our > case, we tried to replace a node and had to wait for several days before its > stored hints are clear out. As mentioned above, we need all hints on this > node to clear out before we can terminate it and replace it by a new > instance/machine. > Since this is not a decommission, I am proposing that we have the same > hints-streaming mechanism as in the decommission code. Furthermore, there > needs to be a cmd for NodeTool to trigger this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8132) Save or stream hints to a safe place in node replacement
[ https://issues.apache.org/jira/browse/CASSANDRA-8132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-8132: -- Fix Version/s: (was: 2.1.2) 2.1.3 > Save or stream hints to a safe place in node replacement > > > Key: CASSANDRA-8132 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8132 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Minh Do >Assignee: Minh Do > Fix For: 2.1.3 > > > Often, we need to replace a node with a new instance in the cloud environment > where we have all nodes are still alive. To be safe without losing data, we > usually make sure all hints are gone before we do this operation. > Replacement means we just want to shutdown C* process on a node and bring up > another instance to take over that node's token. > However, if a node to be replaced has a lot of stored hints, its > HintedHandofManager seems very slow to send the hints to other nodes. In our > case, we tried to replace a node and had to wait for several days before its > stored hints are clear out. As mentioned above, we need all hints on this > node to clear out before we can terminate it and replace it by a new > instance/machine. > Since this is not a decommission, I am proposing that we have the same > hints-streaming mechanism as in the decommission code. Furthermore, there > needs to be a cmd for NodeTool to trigger this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8132) Save or stream hints to a safe place in node replacement
[ https://issues.apache.org/jira/browse/CASSANDRA-8132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-8132: Fix Version/s: (was: 2.1.1) 2.1.2 > Save or stream hints to a safe place in node replacement > > > Key: CASSANDRA-8132 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8132 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Minh Do >Assignee: Minh Do > Fix For: 2.1.2 > > > Often, we need to replace a node with a new instance in the cloud environment > where we have all nodes are still alive. To be safe without losing data, we > usually make sure all hints are gone before we do this operation. > Replacement means we just want to shutdown C* process on a node and bring up > another instance to take over that node's token. > However, if a node to be replaced has a lot of stored hints, its > HintedHandofManager seems very slow to send the hints to other nodes. In our > case, we tried to replace a node and had to wait for several days before its > stored hints are clear out. As mentioned above, we need all hints on this > node to clear out before we can terminate it and replace it by a new > instance/machine. > Since this is not a decommission, I am proposing that we have the same > hints-streaming mechanism as in the decommission code. Furthermore, there > needs to be a cmd for NodeTool to trigger this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8132) Save or stream hints to a safe place in node replacement
[ https://issues.apache.org/jira/browse/CASSANDRA-8132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Minh Do updated CASSANDRA-8132: --- Description: Often, we need to replace a node with a new instance in the cloud environment where we have all nodes are still alive. To be safe without losing data, we usually make sure all hints are gone before we do this operation. Replacement means we just want to shutdown C* process on a node and bring up another instance to take over that node's token. However, if a node to be replaced has a lot of stored hints, its HintedHandofManager seems very slow to send the hints to other nodes. In our case, we tried to replace a node and had to wait for several days before its stored hints are clear out. As mentioned above, we need all hints on this node to clear out before we can terminate it and replace it by a new instance/machine. Since this is not a decommission, I am proposing that we have the same hints-streaming mechanism as in the decommission code. Furthermore, there needs to be a cmd for NodeTool to trigger this. was: Often, we need to replace a node with a new instance in the cloud environment where we have all nodes are still alive. To be safe without losing data, we usually make sure all hints are gone before we do this operation. Replacement means we just want to shutdown C* process on a node and bring up another instance to take over that node's token. However, if a node to be replaced has a lot of stored hints, its HintedHandofManager seems very slow to send the hints to other nodes. In our case, we tried to replace a node and had to wait for several days before its stored hints are clear out. As mentioned above, we need all hints on this node to clear out before we can terminate it and replace it by a new node. Since this is not a decommission, I am proposing that we have the same hints-streaming mechanism as in the decommission code. Furthermore, there needs to be a cmd for NodeTool to trigger this. > Save or stream hints to a safe place in node replacement > > > Key: CASSANDRA-8132 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8132 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Minh Do >Assignee: Minh Do > Fix For: 2.1.1 > > > Often, we need to replace a node with a new instance in the cloud environment > where we have all nodes are still alive. To be safe without losing data, we > usually make sure all hints are gone before we do this operation. > Replacement means we just want to shutdown C* process on a node and bring up > another instance to take over that node's token. > However, if a node to be replaced has a lot of stored hints, its > HintedHandofManager seems very slow to send the hints to other nodes. In our > case, we tried to replace a node and had to wait for several days before its > stored hints are clear out. As mentioned above, we need all hints on this > node to clear out before we can terminate it and replace it by a new > instance/machine. > Since this is not a decommission, I am proposing that we have the same > hints-streaming mechanism as in the decommission code. Furthermore, there > needs to be a cmd for NodeTool to trigger this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8132) Save or stream hints to a safe place in node replacement
[ https://issues.apache.org/jira/browse/CASSANDRA-8132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Minh Do updated CASSANDRA-8132: --- Description: Often, we need to replace a node with a new instance in the cloud environment where we have all nodes are still alive. To be safe without losing data, we usually make sure all hints are gone before we do this operation. Replacement means we just want to shutdown C* process on a node and bring up another instance to take over that node's token. However, if a node to be replaced has a lot of stored hints, its HintedHandofManager seems very slow to send the hints to other nodes. In our case, we tried to replace a node and had to wait for several days before its stored hints are clear out. As mentioned above, we need all hints on this node to clear out before we can terminate it and replace it by a new node. Since this is not a decommission, I am proposing that we have the same hints-streaming mechanism as in the decommission code. Furthermore, there needs to be a cmd for NodeTool to trigger this. was: Often, we need to replace a node with a new instance in the cloud environment where we have all nodes are still alive. To be safe without losing data, we usually make sure all hints are gone before we do this operation. Replacement means we just want to shutdown C* process on a node and bring up another instance to take over that node's token. However, if a node has a lot of stored hints, HintedHandofManager seems very slow to play the hints. In our case, we tried to replace a node and had to wait for several days. Since this is not a decommission, I am proposing that we have the same hints-streaming mechanism as in the decommission code. Furthermore, there needs to be a cmd for NodeTool to trigger this. > Save or stream hints to a safe place in node replacement > > > Key: CASSANDRA-8132 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8132 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Minh Do >Assignee: Minh Do > Fix For: 2.1.1 > > > Often, we need to replace a node with a new instance in the cloud environment > where we have all nodes are still alive. To be safe without losing data, we > usually make sure all hints are gone before we do this operation. > Replacement means we just want to shutdown C* process on a node and bring up > another instance to take over that node's token. > However, if a node to be replaced has a lot of stored hints, its > HintedHandofManager seems very slow to send the hints to other nodes. In our > case, we tried to replace a node and had to wait for several days before its > stored hints are clear out. As mentioned above, we need all hints on this > node to clear out before we can terminate it and replace it by a new node. > Since this is not a decommission, I am proposing that we have the same > hints-streaming mechanism as in the decommission code. Furthermore, there > needs to be a cmd for NodeTool to trigger this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
