[jira] [Commented] (CASSANDRA-7446) Batchlog should be streamed to a different node on decom
[ https://issues.apache.org/jira/browse/CASSANDRA-7446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171094#comment-14171094 ] Aleksey Yeschenko commented on CASSANDRA-7446: -- bq. Is this understanding correct? Yup. And the commit looks good so far. Batchlog should be streamed to a different node on decom Key: CASSANDRA-7446 URL: https://issues.apache.org/jira/browse/CASSANDRA-7446 Project: Cassandra Issue Type: Bug Reporter: Aleksey Yeschenko Assignee: Branimir Lambov Fix For: 2.0.11, 2.1.1 Just like we stream hints on decom, we should also stream the contents of the batchlog - even though we do replicate the batch to at least two nodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7446) Batchlog should be streamed to a different node on decom
[ https://issues.apache.org/jira/browse/CASSANDRA-7446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14169558#comment-14169558 ] Jason Brown commented on CASSANDRA-7446: Hmm, I'm kinda -1 on this, as the coordinator will have no idea who now has those batchlog entries (after they've been streamed off), so we're pretty much guaranteed that the new owner of the batchlog entries will replay them. If anything, maybe the decommissioned node should just ahead and send them. This being said, in SS.decomission, we already wait RING_DELAY before performing any unbootstrap() work (and we've already gossiped that we're leaving). And by the time we've streamed off the data (and the hints) it's most likely that any batchlog entries we would have been either deleted or replayed as streaming a non-trivial amount of data will take some amount of time greater than the batchlog replay timeout. Batchlog should be streamed to a different node on decom Key: CASSANDRA-7446 URL: https://issues.apache.org/jira/browse/CASSANDRA-7446 Project: Cassandra Issue Type: Bug Reporter: Aleksey Yeschenko Assignee: Branimir Lambov Just like we stream hints on decom, we should also stream the contents of the batchlog - even though we do replicate the batch to at least two nodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7446) Batchlog should be streamed to a different node on decom
[ https://issues.apache.org/jira/browse/CASSANDRA-7446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14169565#comment-14169565 ] Aleksey Yeschenko commented on CASSANDRA-7446: -- You are probably right. I think what I came up is the wrong fix the problem - but the problem itself is real. Now I think what we should do is to force batchlog replay on decommission instead (and do it before we stream away the hints). Batchlog should be streamed to a different node on decom Key: CASSANDRA-7446 URL: https://issues.apache.org/jira/browse/CASSANDRA-7446 Project: Cassandra Issue Type: Bug Reporter: Aleksey Yeschenko Assignee: Branimir Lambov Just like we stream hints on decom, we should also stream the contents of the batchlog - even though we do replicate the batch to at least two nodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7446) Batchlog should be streamed to a different node on decom
[ https://issues.apache.org/jira/browse/CASSANDRA-7446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14169577#comment-14169577 ] Jason Brown commented on CASSANDRA-7446: Agreed about letting the decommissioning node replay those batchlogs it has, but why not give if it enough time to do it naturally - that is, wait until after the data and hints have streamed? I suspect enough time will have elapsed in the non-trivial case that the batchlogs would have done what they normally do. Forcing the replay as the last (or one of the very last) steps makes sense, and gives the coordinator a few extra moments to get the right thing done. However, and because I'm lazy right now, we need to double check that coordinator still sends the batchlog delete command while the decom'ing node is LEAVING (LEFT, of course, is totally different). Batchlog should be streamed to a different node on decom Key: CASSANDRA-7446 URL: https://issues.apache.org/jira/browse/CASSANDRA-7446 Project: Cassandra Issue Type: Bug Reporter: Aleksey Yeschenko Assignee: Branimir Lambov Just like we stream hints on decom, we should also stream the contents of the batchlog - even though we do replicate the batch to at least two nodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7446) Batchlog should be streamed to a different node on decom
[ https://issues.apache.org/jira/browse/CASSANDRA-7446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14169601#comment-14169601 ] Jason Brown commented on CASSANDRA-7446: bq. I prefer to not trust anything that's based purely on timeouts I think we're in agreement here, I'm just arguing to force the batchlogs last :) Batchlog should be streamed to a different node on decom Key: CASSANDRA-7446 URL: https://issues.apache.org/jira/browse/CASSANDRA-7446 Project: Cassandra Issue Type: Bug Reporter: Aleksey Yeschenko Assignee: Branimir Lambov Just like we stream hints on decom, we should also stream the contents of the batchlog - even though we do replicate the batch to at least two nodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7446) Batchlog should be streamed to a different node on decom
[ https://issues.apache.org/jira/browse/CASSANDRA-7446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14169812#comment-14169812 ] Jason Brown commented on CASSANDRA-7446: bq. batchlog replay might write a few hints Ah, good call. Batchlog should be streamed to a different node on decom Key: CASSANDRA-7446 URL: https://issues.apache.org/jira/browse/CASSANDRA-7446 Project: Cassandra Issue Type: Bug Reporter: Aleksey Yeschenko Assignee: Branimir Lambov Just like we stream hints on decom, we should also stream the contents of the batchlog - even though we do replicate the batch to at least two nodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
