[jira] [Commented] (CASSANDRA-12212) system.compactions_in_progress needs to be used on first upgrade to 3.0
[
https://issues.apache.org/jira/browse/CASSANDRA-12212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15381613#comment-15381613
]
Stefania commented on CASSANDRA-12212:
--
Was there an actual problem caused by this? We delete unfinished temporary
files in {{SystemKeyspace.migrateDataDirs()}}.
As for final files, consulting {{system.compactions_in_progress}} would only
partially help because the old files still exist when the entry in the table is
deleted by {{CompactionTask.runMyThrow()}}. There is a time interval between
temporary new files being promoted to final new files and the deletion of the
entry in the table, this would be protected, but the time interval between the
deletion of the entry and the deletion of old files (which could be much longer
if other references to the sstables exist) is not protected. As far as it was
understood during the development of 7066, the only adverse effect of this is a
redundant compaction since counters no longer double-count in 2.1.
Further, since CASSANDRA-10079 we also cancel compactions in progress in
nodetool drain, so if people follow the upgrade procedure there won't be any
entry in the table or left-overs on disk.
> system.compactions_in_progress needs to be used on first upgrade to 3.0
> ---
>
> Key: CASSANDRA-12212
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12212
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Jeremiah Jordan
>Assignee: Stefania
> Fix For: 3.0.x, 3.x
>
>
> CASSANDRA-7066 removed the system.compactions_in_progress table and replaced
> it with the new transaction system. But system.compactions_in_progress needs
> to be consulted for the first startup after upgrading from 2.1 to 3.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12212) system.compactions_in_progress needs to be used on first upgrade to 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-12212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15389600#comment-15389600 ] Jeremiah Jordan commented on CASSANDRA-12212: - I have just been told that not using compactions in progress can result in bad things happening to counters and possibly other places. If that is no longer the case case in 2.1 great. But if that is no longer the case why do we still track them there? My thought here was purely "if we hadn't upgraded to 3.0 we would be consulting compactions in progress" so the first upgrade to 3.0 should consult compactions in progress. > system.compactions_in_progress needs to be used on first upgrade to 3.0 > --- > > Key: CASSANDRA-12212 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12212 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Jeremiah Jordan >Assignee: Stefania > Fix For: 3.0.x, 3.x > > > CASSANDRA-7066 removed the system.compactions_in_progress table and replaced > it with the new transaction system. But system.compactions_in_progress needs > to be consulted for the first startup after upgrading from 2.1 to 3.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12212) system.compactions_in_progress needs to be used on first upgrade to 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-12212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15391432#comment-15391432 ] Stefania commented on CASSANDRA-12212: -- I'm pretty sure that in 2.1 we no longer store deltas for counters in memtables or sstables, and so we should be able to tolerate an old and a compacted sstable but, just to be sure, can you confirm [~iamaleksey]? bq. But if that is no longer the case why do we still track them there? I think it'ss just to clean-up compaction left overs regardless of counters. > system.compactions_in_progress needs to be used on first upgrade to 3.0 > --- > > Key: CASSANDRA-12212 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12212 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Jeremiah Jordan >Assignee: Stefania > Fix For: 3.0.x, 3.x > > > CASSANDRA-7066 removed the system.compactions_in_progress table and replaced > it with the new transaction system. But system.compactions_in_progress needs > to be consulted for the first startup after upgrading from 2.1 to 3.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12212) system.compactions_in_progress needs to be used on first upgrade to 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-12212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15394741#comment-15394741 ] Aleksey Yeschenko commented on CASSANDRA-12212: --- We don't for counters created in 2.1 and later. The remains of local shards from pre-2.1 can remain for an unbounded time, however, which may or may not be relevant to the ticket. > system.compactions_in_progress needs to be used on first upgrade to 3.0 > --- > > Key: CASSANDRA-12212 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12212 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Jeremiah Jordan >Assignee: Stefania > Fix For: 3.0.x, 3.x > > > CASSANDRA-7066 removed the system.compactions_in_progress table and replaced > it with the new transaction system. But system.compactions_in_progress needs > to be consulted for the first startup after upgrading from 2.1 to 3.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12212) system.compactions_in_progress needs to be used on first upgrade to 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-12212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401670#comment-15401670 ] Stefania commented on CASSANDRA-12212: -- Thanks for clarifying. If I understood correctly, given that when local shards get reconciled with global shards we take the global shard, and counter mutations in 2.1+ create global shards, we shouldn't have any compactions in progress that need to reconcile two local shards unless people have been using 2.1 only for a very short period of time. So the only legitimate case would be the case when 2.1 is installed as an intermediate release before upgrading to 3.0. Is this accurate? > system.compactions_in_progress needs to be used on first upgrade to 3.0 > --- > > Key: CASSANDRA-12212 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12212 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Jeremiah Jordan >Assignee: Stefania > Fix For: 3.0.x, 3.x > > > CASSANDRA-7066 removed the system.compactions_in_progress table and replaced > it with the new transaction system. But system.compactions_in_progress needs > to be consulted for the first startup after upgrading from 2.1 to 3.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12212) system.compactions_in_progress needs to be used on first upgrade to 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-12212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15402014#comment-15402014 ] Aleksey Yeschenko commented on CASSANDRA-12212: --- Mostly correct. "only for a very short period of time" is not true. Pre-2.1 counter cells can remain in the system potentially forever, as there is no guarantee old counter values would ever be overridden. > system.compactions_in_progress needs to be used on first upgrade to 3.0 > --- > > Key: CASSANDRA-12212 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12212 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Jeremiah Jordan >Assignee: Stefania > Fix For: 3.0.x, 3.x > > > CASSANDRA-7066 removed the system.compactions_in_progress table and replaced > it with the new transaction system. But system.compactions_in_progress needs > to be consulted for the first startup after upgrading from 2.1 to 3.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12212) system.compactions_in_progress needs to be used on first upgrade to 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-12212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15403252#comment-15403252 ] Stefania commented on CASSANDRA-12212: -- I understood that local shards can remain in the system potentially forever. My point was that if they do get overridden, then it would be with a global shard, and therefore in a potential interrupted compaction we'd have one local shard in one sstable and one global shard in another sstable, resolved to the global shard in the compacted sstable. If we restarted in 3.0 with these 3 sstables still around, the global shard would prevail in a read operation and so no harm in having both old and new sstables. The only potential scenario is two local shards, for the same counter, present into two different sstables, which were never compacted and hence unlikely to be present in an interrupted compaction just before the upgrade to 3.0. To implement this, it is mostly a copy and paste exercise, the trickiest part is to design a test for it. However, I think the benefits are limited, given that it is a rare scenario, and besides a nodetool drain will stop compactions, and the time interval between the deletion of the compactions_in_progress entry and the deletion of old sstable files is still not covered by the compactions_in_progress entry. So my inclination is to close this as won't fix, but if people think we should still read compactions_in_progress when upgrading to 3.0, then we can do so. > system.compactions_in_progress needs to be used on first upgrade to 3.0 > --- > > Key: CASSANDRA-12212 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12212 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Jeremiah Jordan >Assignee: Stefania > Fix For: 3.0.x, 3.x > > > CASSANDRA-7066 removed the system.compactions_in_progress table and replaced > it with the new transaction system. But system.compactions_in_progress needs > to be consulted for the first startup after upgrading from 2.1 to 3.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12212) system.compactions_in_progress needs to be used on first upgrade to 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-12212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15436888#comment-15436888 ] Aleksey Yeschenko commented on CASSANDRA-12212: --- Go ahead with Won't Fix. > system.compactions_in_progress needs to be used on first upgrade to 3.0 > --- > > Key: CASSANDRA-12212 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12212 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Jeremiah Jordan >Assignee: Stefania > Fix For: 3.0.x, 3.x > > > CASSANDRA-7066 removed the system.compactions_in_progress table and replaced > it with the new transaction system. But system.compactions_in_progress needs > to be consulted for the first startup after upgrading from 2.1 to 3.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
