[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[
https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522950#comment-16522950
]
Jay Zhuang commented on CASSANDRA-6434:
---
Is there a follow-up ticket or plan to make {{only_purge_repaired_tombstones}}
default and get ride of GCGS? As the purgeable tombstones are not repairable,
so keeping the unrepaired tombstones (purgeable) seems not helpful.
> Repair-aware gc grace period
> -
>
> Key: CASSANDRA-6434
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6434
> Project: Cassandra
> Issue Type: New Feature
>Reporter: sankalp kohli
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 3.0 beta 1
>
>
> Since the reason for gcgs is to ensure that we don't purge tombstones until
> every replica has been notified, it's redundant in a world where we're
> tracking repair times per sstable (and repairing frequentily), i.e., a world
> where we default to incremental repair a la CASSANDRA-5351.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681095#comment-14681095 ] Yuki Morishita commented on CASSANDRA-6434: --- dtest seems good. +1. > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 beta 1 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680453#comment-14680453 ] Yuki Morishita commented on CASSANDRA-6434: --- Patch looks good to me. testall seems OK, waiting for another dtest run to make sure. Little more comments on new unit test (RepairedDataTombstoneTest) will be helpful. There is commented lines in the test that I guess it's just for seeing data and can be removed. I think it is good to add brief lines in NEWS.txt's new feature section about this functionality before you commit. > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 beta 1 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662320#comment-14662320 ] Marcus Eriksson commented on CASSANDRA-6434: rebased on 3.0 here: https://github.com/krummas/cassandra/commits/marcuse/6434-3.0 > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 beta 1 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14647494#comment-14647494 ] Marcus Eriksson commented on CASSANDRA-6434: bq. the option probably belongs to compaction params, not as a standalone thing. good point, fixed and pushed > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 beta 1 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[
https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14647427#comment-14647427
]
Aleksey Yeschenko commented on CASSANDRA-6434:
--
And no need to check for column presence (if we were to keep it a separate
column) - right now, {{row.has("only_purge_repaired_tombstones")}} will
*always* be true.
> Repair-aware gc grace period
> -
>
> Key: CASSANDRA-6434
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6434
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
>Reporter: sankalp kohli
>Assignee: Marcus Eriksson
> Fix For: 3.0 beta 1
>
>
> Since the reason for gcgs is to ensure that we don't purge tombstones until
> every replica has been notified, it's redundant in a world where we're
> tracking repair times per sstable (and repairing frequentily), i.e., a world
> where we default to incremental repair a la CASSANDRA-5351.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[
https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14647425#comment-14647425
]
Aleksey Yeschenko commented on CASSANDRA-6434:
--
The changes to Thrift should most definitely be reverted.
There is no point in migrating the option in {{LegacySchemaMigrator}} - it's a
new 3.0 feature, it will never ever be present in old schema.
And the option probably belongs to compaction params, not as a standalone thing.
> Repair-aware gc grace period
> -
>
> Key: CASSANDRA-6434
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6434
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
>Reporter: sankalp kohli
>Assignee: Marcus Eriksson
> Fix For: 3.0 beta 1
>
>
> Since the reason for gcgs is to ensure that we don't purge tombstones until
> every replica has been notified, it's redundant in a world where we're
> tracking repair times per sstable (and repairing frequentily), i.e., a world
> where we default to incremental repair a la CASSANDRA-5351.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14647402#comment-14647402 ] Marcus Eriksson commented on CASSANDRA-6434: bq. Couldn't we use a non-optimal (but much simpler and hopefully good enough) solution for that? Typically, unrepaired sstables should, by design, be fairly recent. So what if we said that we only purge tombstone if it's older than gcGrace and its localDeletionTime is older than then oldest localDeletion in the unrepaired sstables used by the query? That is a much better idea, first stab at that here: https://github.com/krummas/cassandra/commits/marcuse/6434-trunk-2 > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 beta 1 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14618749#comment-14618749 ] Sylvain Lebresne commented on CASSANDRA-6434: - bq. Could we just never drop a tombstone on the read path? There is 2 (linked) reasons we drop purgeable tombstones on reads: # it's a minor optimization: the sooner we get rid of stuff we don't need, the better. # it makes sure we don't throw a TombstoneOverwhelming because of them (which *has* happened in the wild and is not terribly nice because we want tombstone overwhelming exception to mean "you've done something wrong while modeling" while purgeable tombstones are just an artifact of compaction lagging behind, which could be temporary and not a huge deal). > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 beta 1 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14618741#comment-14618741 ] Jeremiah Jordan commented on CASSANDRA-6434: Could we just never drop a tombstone on the read path? If its still on disk, then keep it? > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 beta 1 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[
https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14618737#comment-14618737
]
Sylvain Lebresne commented on CASSANDRA-6434:
-
Ok, so as I said above, I'm not really a fan of keeping for every tombstone if
it's from a repaired or unrepaired sstable. Tombstones do not necessarily
originate from a sstable, and so putting this information everywhere is
confusing and more complex/widely impactful than it should be.
Now, as you said, the only problem is the read path. Couldn't we use a
non-optimal (but much simpler and hopefully good enough) solution for that?
Typically, unrepaired sstables should, by design, be fairly recent. So what if
we said that we only purge tombstone if it's older than gcGrace *and* its
{{localDeletionTime}} is older than then oldest {{localDeletion}} in the
unrepaired sstables used by the query? We'll be guaranteed to only purge
tombstone from repaired sstables, and while we might include a few tombstones
from repaired sstables that we shouldn't, that's unlikely to be a whole lot
(and it's technically ok to do so anyway), and is probably not a big price to
pay for the extra safety.
> Repair-aware gc grace period
> -
>
> Key: CASSANDRA-6434
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6434
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
>Reporter: sankalp kohli
>Assignee: Marcus Eriksson
> Fix For: 3.0 beta 1
>
>
> Since the reason for gcgs is to ensure that we don't purge tombstones until
> every replica has been notified, it's redundant in a world where we're
> tracking repair times per sstable (and repairing frequentily), i.e., a world
> where we default to incremental repair a la CASSANDRA-5351.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14618689#comment-14618689 ] Jeremiah Jordan commented on CASSANDRA-6434: It lets you reduce GC grace, not remove it completely. So it doesn't need to be 7 days anymore, but does need to be longer than the hint window and some other things. So maybe we can reduce it to a couple hours, or one day or something, and let the "don't drop un-repiared" make sure we never drop one too early. This also solves a "safety" issue. If something happens to your cluster such that you can't repair it in gc_grace, you aren't screwed, which is something people have to worry about now. > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 beta 1 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14618679#comment-14618679 ] Marcus Eriksson commented on CASSANDRA-6434: Yes, it is really only extra safety for now, if something happens so that repairs cannot run for gc_grace for example > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 beta 1 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14618670#comment-14618670 ] Sylvain Lebresne commented on CASSANDRA-6434: - I'm confused. That ticket description sounds like it's about lowering the lifetime of tombstones in the system but you make it sounds like it's not about that anymore. If this doesn't allow to lower gc_grace, then why do we need that extra safety? Is it just really only extra safety (to be clear, I'm really not a fan of the changes to LivenessInfo and DeletionTime and I'd like to see if we can avoid it, but before that I need to understand why we're doing this ticket in the first place). > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 beta 1 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14618662#comment-14618662 ] Marcus Eriksson commented on CASSANDRA-6434: or, we guarantee to atleast keep them for gc_grace seconds (due to hints, paxos etc mentioned above), but then also until they have been repaired > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 beta 1 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14618633#comment-14618633 ] Marcus Eriksson commented on CASSANDRA-6434: It is mostly as a safety measure, we guarantee that we keep around tombstones until they have been (incrementally) repaired, no matter what gc_grace is set to > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 beta 1 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14618626#comment-14618626 ] Sylvain Lebresne commented on CASSANDRA-6434: - Just to make sure I'm on the right page, what's the idea of the general solution? Is it that people using incremental repair would activate the "only purge tombstones from repaired sstables" option and be then able to set gcGrace to 0? > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 beta 1 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14618530#comment-14618530 ] Marcus Eriksson commented on CASSANDRA-6434: patch here: https://github.com/krummas/cassandra/commits/marcuse/6434-trunk and I'd like some initial feedback on the approach from [~slebresne] if possible > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 beta 1 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600708#comment-14600708 ] Marcus Eriksson commented on CASSANDRA-6434: working on it - on top of CASSANDRA-8099, so mostly trying to digest that code so far > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 beta 1 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600705#comment-14600705 ] sankalp kohli commented on CASSANDRA-6434: -- [~krummas] Any updates on this? > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 beta 1 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498107#comment-14498107 ] sankalp kohli commented on CASSANDRA-6434: -- Yes we can do that. Once this is donein the future we can talk about lowering gc grace default and also may be renaming it. > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496985#comment-14496985 ] Marcus Eriksson commented on CASSANDRA-6434: So, due to the complications mentioned above, I suggest we keep everything basically the same it is today, with the added security of never dropping tombstones when doing unrepaired compactions. Ie, we keep gc_grace_seconds, but only use it when we do compactions within the repaired data wdyt [~kohlisankalp]? > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14386284#comment-14386284 ] Marcus Eriksson commented on CASSANDRA-6434: [~kohlisankalp] no updates, I'll do some more research in what we can actually do here > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14325049#comment-14325049 ] sankalp kohli commented on CASSANDRA-6434: -- [~krummas] Any update on this. We need this patch so that we don't have to keep patches. > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14206162#comment-14206162 ] Marcus Eriksson commented on CASSANDRA-6434: Hint TTL is set to the smallest gcgs among the column families included in the mutation to make sure we don't resurrect any data by replaying an old hint. After this ticket we would only drop tombstones if the sstable is repaired, repairing the node will also make sure that it has received the data it should have. If we then receive a hint for a range that is from before the last repair, we could safely ignore it. CL.ANY makes this more unclean since a node might not actually have all the data it should after a repair, but, if we keep a safety window of max_hint_window_in_ms during which we always keep the tombstones, repaired or not, we would probably be safe. (and it would solve [~jjordan]s issue above as well) I most likely missed/ignored some corner case (like that last comment on CASSANDRA-3620 and batches generating new hints etc.. rage) > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14205328#comment-14205328 ] Aleksey Yeschenko commented on CASSANDRA-6434: -- That, and there is the similar issue of hints, batches, and paxos. Similar ideas have already been proposed before, but this issue killed them - see, for example, CASSANDRA-3620. I don't see how to work around that at the moment. > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14205304#comment-14205304 ] Jeremiah Jordan commented on CASSANDRA-6434: I think we need to make sure we still have some kind of grace period for tombstones. An extreme edge case here, but: 1. user insert Y at time 10. 2. Insert gets held up in coordinator or something. 3. User inserts delete for Y at time 11 to a different node. 4. Incremental repair happens. 5. compaction happens (dropping the tombstone) because it was repaired. 6. insert at time 10 comes through. If you have a gcgs > timeout period the data from step 6 will be ignored (though you probably want it even longer than the timeout, as client retries with same old timestamp could happen). If you don't have gcgs, the data could suddenly show up even though a delete went through. So as much as I like the idea of just removing gc grace, I think we really need to keep it as the minimum time tombstones will stay around. But we could make the default 30 minutes or something if we have the repair aware tombstone removal. > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14070452#comment-14070452 ] Marcus Eriksson commented on CASSANDRA-6434: hmm, I think we were just being cautious with this feature in 2.1, should be fine in 3.0 though - created CASSANDRA-7586 > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14070390#comment-14070390 ] Jonathan Ellis commented on CASSANDRA-6434: --- Why couldn't we just add that information to non-incremental repairs? > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14070380#comment-14070380 ] Marcus Eriksson commented on CASSANDRA-6434: bq. "don't drop tombstones at all until it's repaired" yes, of course, but we need to support people who don't run incremental repairs - we don't tag sstables as repaired when doing old-style repairs > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14070370#comment-14070370 ] Jonathan Ellis commented on CASSANDRA-6434: --- bq. I don't think that would work since max(repairtime) is dependent on compaction Actually we don't really care *when* it was repaired; as long as it *has* been repaired. So really the logic should be, "if I'm compacting a repaired sstable, and it doesn't shadow any data in other sstables, then I can drop the ts immediately." bq. Lets use gcgs on unrepaired data until we always mark sstables as repaired. I suppose we could do that, but the alternative of "don't drop tombstones at all until it's repaired" is more technically correct. > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14070067#comment-14070067 ] Marcus Eriksson commented on CASSANDRA-6434: Hmm yes, [~kohlisankalp]s approach of dropping tombstones immediately (if the key is not in any other sstable) works during compaction since we never compact repaired and unrepaired data together. When doing incremental repair we should probably include all tombstones. It is during reads we need an actual value on gcBefore. bq. We could approximate it as max(repairtime) for any sstable that may contain the key. Unless I misunderstand your suggestion, I don't think that would work since max(repairtime) is dependent on compaction - the resulting sstable gets the worst case (oldest) repair time among the involved sstables, meaning different nodes will disagree on when the last repair was done for a key. I'll unlink CASSANDRA-5839 - it would not work looking up actual per-key repair times during reads anyway. Lets use gcgs on unrepaired data until we always mark sstables as repaired. > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068592#comment-14068592 ] Jonathan Ellis commented on CASSANDRA-6434: --- We could approximate it as max(repairtime) for any sstable that may contain the key. (And we already have the logic that says, "checking bloom filters is expensive so avoid it until the last minute.") > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068437#comment-14068437 ] Marcus Eriksson commented on CASSANDRA-6434: Blocks on CASSANDRA-5839 - once we have the repair history we can figure out actual repair times for keys > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14051487#comment-14051487 ] Jonathan Ellis commented on CASSANDRA-6434: --- I'd actually prefer to use actual-repair-time instead of gcgs approximation even for non-incremental repair. The latter means you could cause "reappearing" data if the approximation is too optimistic (which was what Sankalp originally filed this ticket for; I've modified his description). > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14051181#comment-14051181 ] Marcus Eriksson commented on CASSANDRA-6434: Hmm, I guess the only reason to keep gcgs is for people who don't run incremental repair (with incremental repair we can, as [~kohlisankalp] mentions above set gcgs to 0 when compacting repaired data). Since we don't make incremental repairs default until 3.0, we should probably keep gcgs there for people not using inc repair and then, in 3.1 (or 4.0), both remove gcgs and make full repairs mark data as repaired. > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6434) Repair aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13884358#comment-13884358 ] sankalp kohli commented on CASSANDRA-6434: -- Once CASSANDRA-5351 is fixed, I think we can drop tombstones of repaired stables if the original data has been compacted. We don't need to keep track of last repair time. > Repair aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Priority: Minor > > If we don't run repair every gc grace period, forgotten delete problem can > happen. This can be very bad for some use cases. > To avoid this, the only way is to guaranty that we run repair successfully > across the cluster every gc grace period. > This is operationally very hard to achieve when we are dealing with lot of > nodes. > Also repair can fail for many reasons like machine failures, one stable which > is bad, etc. > So one solution to this is to add a new optional feature(disable by default) > which only delete tombstones if repair has successfully run on it instead of > relying on gc grace period. We can track the last successful repair time in a > system key space. > This feature will be very useful for use cases which cannot tolerate data > reappearing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
