[ https://issues.apache.org/jira/browse/CASSANDRA-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144901#comment-16144901 ]
Romain GERARD edited comment on CASSANDRA-13418 at 8/29/17 8:09 AM: -------------------------------------------------------------------- bq. 2. only enabling unsafe_aggressive_sstable_expiration When looking for sstables expired, you will *ignore *the overlaps and only look locally if the current sstable is eligible. When looking for sstables to compact, you will *not ignore* the overlaps and look globally if the current sstable is eligible. bq. 1. enabling both When looking for sstables expired, you will *ignore *the overlaps and only look locally if the current sstable is eligible. When looking for sstables to compact, you will *ignore* the overlaps and look globally if the current sstable is eligible. I made a new version of the patch with uncheckedTombstoneCompaction disabled and a warning message. https://github.com/criteo-forks/cassandra/commit/800ab325cbf7d9d4d5e60e2b959918426e121815 the diff {noformat} diff --git a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java index 43c90c7042..d21222c484 100644 --- a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java +++ b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java @@ -67,9 +67,9 @@ public class TimeWindowCompactionStrategy extends AbstractCompactionStrategy else logger.debug("Enabling tombstone compactions for TWCS"); - if (this.options.ignoreOverlaps) - this.uncheckedTombstoneCompaction = true; - + if(this.options.ignoreOverlaps && !this.uncheckedTombstoneCompaction) { + logger.warn("You are running with sstables overlapping checks disabled but without unchecked tombstone compaction, check that this what you want"); + } } {noformat} was (Author: rgerard): 2. only enabling unsafe_aggressive_sstable_expiration When looking for sstables expired, you will *ignore *the overlaps and only look locally if the current sstable is eligible. When looking for sstables to compact, you will *not ignore* the overlaps and look globally if the current sstable is eligible. bq. 1. enabling both When looking for sstables expired, you will *ignore *the overlaps and only look locally if the current sstable is eligible. When looking for sstables to compact, you will *ignore* the overlaps and look globally if the current sstable is eligible. I made a new version of the patch with uncheckedTombstoneCompaction disabled and a warning message. https://github.com/criteo-forks/cassandra/commit/800ab325cbf7d9d4d5e60e2b959918426e121815 the diff {noformat} diff --git a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java index 43c90c7042..d21222c484 100644 --- a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java +++ b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java @@ -67,9 +67,9 @@ public class TimeWindowCompactionStrategy extends AbstractCompactionStrategy else logger.debug("Enabling tombstone compactions for TWCS"); - if (this.options.ignoreOverlaps) - this.uncheckedTombstoneCompaction = true; - + if(this.options.ignoreOverlaps && !this.uncheckedTombstoneCompaction) { + logger.warn("You are running with sstables overlapping checks disabled but without unchecked tombstone compaction, check that this what you want"); + } } {noformat} > Allow TWCS to ignore overlaps when dropping fully expired sstables > ------------------------------------------------------------------ > > Key: CASSANDRA-13418 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13418 > Project: Cassandra > Issue Type: Improvement > Components: Compaction > Reporter: Corentin Chary > Labels: twcs > Attachments: twcs-cleanup.png > > > http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html explains it well. If > you really want read-repairs you're going to have sstables blocking the > expiration of other fully expired SSTables because they overlap. > You can set unchecked_tombstone_compaction = true or tombstone_threshold to a > very low value and that will purge the blockers of old data that should > already have expired, thus removing the overlaps and allowing the other > SSTables to expire. > The thing is that this is rather CPU intensive and not optimal. If you have > time series, you might not care if all your data doesn't exactly expire at > the right time, or if data re-appears for some time, as long as it gets > deleted as soon as it can. And in this situation I believe it would be really > beneficial to allow users to simply ignore overlapping SSTables when looking > for fully expired ones. > To the question: why would you need read-repairs ? > - Full repairs basically take longer than the TTL of the data on my dataset, > so this isn't really effective. > - Even with a 10% chances of doing a repair, we found out that this would be > enough to greatly reduce entropy of the most used data (and if you have > timeseries, you're likely to have a dashboard doing the same important > queries over and over again). > - LOCAL_QUORUM is too expensive (need >3 replicas), QUORUM is too slow. > I'll try to come up with a patch demonstrating how this would work, try it on > our system and report the effects. > cc: [~adejanovski], [~rgerard] as I know you worked on similar issues already. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org