[jira] [Commented] (CASSANDRA-8739) Don't check for overlap with sstables that have had their start positions moved in LCS
[ https://issues.apache.org/jira/browse/CASSANDRA-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306145#comment-14306145 ] Benedict commented on CASSANDRA-8739: - I assume this is another race condition problem, because we shouldn't be able to compact any files that have had their starts moved? The only way it should be a problem is if we abort, and then mark compacting based on the starts during the move? For CASSANDRA-8689 I intend to make it so we only permit markCompacting to succeed on sstables in the live set. I'm tempted to make it only succeed if the _exact instance_ is the one in the live set, so that if you're looking at a stale copy of the file (i.e. one with moved starts) you fail, and reselect your candidates. Would this solve this problem? > Don't check for overlap with sstables that have had their start positions > moved in LCS > -- > > Key: CASSANDRA-8739 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8739 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 2.1.3 > > > When picking compaction candidates in LCS, we check that we won't cause any > overlap in the higher level. Problem is that we compare the files that have > had their start positions moved meaning we can cause overlap. We need to also > include the tmplink files when checking this. > Note that in 2.1 overlap is not as big problem as earlier, if adding an > sstable would cause overlap, we send it back to L0 instead, meaning we do a > bit more compaction but we never actually have overlap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8739) Don't check for overlap with sstables that have had their start positions moved in LCS
[ https://issues.apache.org/jira/browse/CASSANDRA-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14307159#comment-14307159 ] Benedict commented on CASSANDRA-8739: - My question is more how this then causes problems, if it fails to markCompacting the candidates it selects? > Don't check for overlap with sstables that have had their start positions > moved in LCS > -- > > Key: CASSANDRA-8739 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8739 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 2.1.3 > > Attachments: 0001-8739.patch > > > When picking compaction candidates in LCS, we check that we won't cause any > overlap in the higher level. Problem is that we compare the files that have > had their start positions moved meaning we can cause overlap. We need to also > include the tmplink files when checking this. > Note that in 2.1 overlap is not as big problem as earlier, if adding an > sstable would cause overlap, we send it back to L0 instead, meaning we do a > bit more compaction but we never actually have overlap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8739) Don't check for overlap with sstables that have had their start positions moved in LCS
[ https://issues.apache.org/jira/browse/CASSANDRA-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14307164#comment-14307164 ] Marcus Eriksson commented on CASSANDRA-8739: we pick a bunch of files for compaction (those are not currently compacting), then we make sure that by compacting those into L1, we don't cause any overlap in L1. We do that by checking first/last keys in the sstables > Don't check for overlap with sstables that have had their start positions > moved in LCS > -- > > Key: CASSANDRA-8739 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8739 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 2.1.3 > > Attachments: 0001-8739.patch > > > When picking compaction candidates in LCS, we check that we won't cause any > overlap in the higher level. Problem is that we compare the files that have > had their start positions moved meaning we can cause overlap. We need to also > include the tmplink files when checking this. > Note that in 2.1 overlap is not as big problem as earlier, if adding an > sstable would cause overlap, we send it back to L0 instead, meaning we do a > bit more compaction but we never actually have overlap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8739) Don't check for overlap with sstables that have had their start positions moved in LCS
[ https://issues.apache.org/jira/browse/CASSANDRA-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14307172#comment-14307172 ] Benedict commented on CASSANDRA-8739: - Oh, right. Yes, now you've spelled it out, I can see you were clearly stating it before and I was just missing it. > Don't check for overlap with sstables that have had their start positions > moved in LCS > -- > > Key: CASSANDRA-8739 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8739 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 2.1.3 > > Attachments: 0001-8739.patch > > > When picking compaction candidates in LCS, we check that we won't cause any > overlap in the higher level. Problem is that we compare the files that have > had their start positions moved meaning we can cause overlap. We need to also > include the tmplink files when checking this. > Note that in 2.1 overlap is not as big problem as earlier, if adding an > sstable would cause overlap, we send it back to L0 instead, meaning we do a > bit more compaction but we never actually have overlap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8739) Don't check for overlap with sstables that have had their start positions moved in LCS
[ https://issues.apache.org/jira/browse/CASSANDRA-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14327989#comment-14327989 ] Carl Yeksigian commented on CASSANDRA-8739: --- I'm not sure how the change to the way it calculates the L0 compacting makes a difference; it seems like they should be the same. The new L0 overlapping check makes sense. > Don't check for overlap with sstables that have had their start positions > moved in LCS > -- > > Key: CASSANDRA-8739 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8739 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 2.1.4 > > Attachments: 0001-8739.patch > > > When picking compaction candidates in LCS, we check that we won't cause any > overlap in the higher level. Problem is that we compare the files that have > had their start positions moved meaning we can cause overlap. We need to also > include the tmplink files when checking this. > Note that in 2.1 overlap is not as big problem as earlier, if adding an > sstable would cause overlap, we send it back to L0 instead, meaning we do a > bit more compaction but we never actually have overlap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8739) Don't check for overlap with sstables that have had their start positions moved in LCS
[ https://issues.apache.org/jira/browse/CASSANDRA-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14345034#comment-14345034 ] Marcus Eriksson commented on CASSANDRA-8739: the new compacting L0 calculation takes the sstable *instances* from the datatracker compacting set - these instances are not the same as the ones in LCS L0 (the LCS L0 can have had their start positions moved), hoping to fix that in CASSANDRA-8764 > Don't check for overlap with sstables that have had their start positions > moved in LCS > -- > > Key: CASSANDRA-8739 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8739 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 2.1.4 > > Attachments: 0001-8739.patch > > > When picking compaction candidates in LCS, we check that we won't cause any > overlap in the higher level. Problem is that we compare the files that have > had their start positions moved meaning we can cause overlap. We need to also > include the tmplink files when checking this. > Note that in 2.1 overlap is not as big problem as earlier, if adding an > sstable would cause overlap, we send it back to L0 instead, meaning we do a > bit more compaction but we never actually have overlap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8739) Don't check for overlap with sstables that have had their start positions moved in LCS
[ https://issues.apache.org/jira/browse/CASSANDRA-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14345036#comment-14345036 ] Benedict commented on CASSANDRA-8739: - I'm hoping to fix this in CASSANDRA-8568 also > Don't check for overlap with sstables that have had their start positions > moved in LCS > -- > > Key: CASSANDRA-8739 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8739 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 2.1.4 > > Attachments: 0001-8739.patch > > > When picking compaction candidates in LCS, we check that we won't cause any > overlap in the higher level. Problem is that we compare the files that have > had their start positions moved meaning we can cause overlap. We need to also > include the tmplink files when checking this. > Note that in 2.1 overlap is not as big problem as earlier, if adding an > sstable would cause overlap, we send it back to L0 instead, meaning we do a > bit more compaction but we never actually have overlap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8739) Don't check for overlap with sstables that have had their start positions moved in LCS
[ https://issues.apache.org/jira/browse/CASSANDRA-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14349862#comment-14349862 ] Carl Yeksigian commented on CASSANDRA-8739: --- I missed that before. Makes sense; +1 on the patch. > Don't check for overlap with sstables that have had their start positions > moved in LCS > -- > > Key: CASSANDRA-8739 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8739 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 2.1.4 > > Attachments: 0001-8739.patch > > > When picking compaction candidates in LCS, we check that we won't cause any > overlap in the higher level. Problem is that we compare the files that have > had their start positions moved meaning we can cause overlap. We need to also > include the tmplink files when checking this. > Note that in 2.1 overlap is not as big problem as earlier, if adding an > sstable would cause overlap, we send it back to L0 instead, meaning we do a > bit more compaction but we never actually have overlap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
