[jira] [Updated] (CASSANDRA-18507) Partial compaction can resurrect deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-18507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-18507:
--
Status: Ready to Commit (was: Review In Progress)
> Partial compaction can resurrect deleted data
> -
>
> Key: CASSANDRA-18507
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18507
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Tobias Lindaaker
>Assignee: Tobias Lindaaker
>Priority: Normal
>
> If there isn't enough disk space available to compact all existing sstables,
> Cassandra will attempt to perform a partial compaction by removing sstables
> from the set of candidate sstables to be compacted, starting with the largest
> one. It is possible that the sstable removed from the set of sstables to
> compact contains data for which there are tombstones in another (more recent)
> sstable. Since the overlaps between sstables is computed when the
> {{CompactionController}} is created, and the {{CompactionController}} is
> created before the removal of any sstables from the set of sstables to be
> compacted this computed overlap will be outdated when checking which sstables
> are covered by certain tombstones. This leads to the faulty conclusion that
> the tombstones can be pruned during the compaction, causing the data to be
> resurrected.
> The issue is present in Cassandra 4.0 and 4.1. Cassandra 3.11 creates the
> {{CompactionController}} after the set of sstables to compact has been
> reduced, and is thus not affected. {{trunk}} does not appear to support
> partial compactions at all, but instead refuses to compact when the disk is
> full.
> This regression appears to have been introduced by CASSANDRA-13068.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18507) Partial compaction can resurrect deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-18507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-18507:
--
Fix Version/s: 4.0.10
4.1.2
5.0
Since Version: 4.0.0
Source Control Link:
https://github.com/apache/cassandra/commit/1053e3b475829c7f2d0dc4ab59322d5819d1496a
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Partial compaction can resurrect deleted data
> -
>
> Key: CASSANDRA-18507
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18507
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Tobias Lindaaker
>Assignee: Tobias Lindaaker
>Priority: Normal
> Fix For: 4.0.10, 4.1.2, 5.0
>
>
> If there isn't enough disk space available to compact all existing sstables,
> Cassandra will attempt to perform a partial compaction by removing sstables
> from the set of candidate sstables to be compacted, starting with the largest
> one. It is possible that the sstable removed from the set of sstables to
> compact contains data for which there are tombstones in another (more recent)
> sstable. Since the overlaps between sstables is computed when the
> {{CompactionController}} is created, and the {{CompactionController}} is
> created before the removal of any sstables from the set of sstables to be
> compacted this computed overlap will be outdated when checking which sstables
> are covered by certain tombstones. This leads to the faulty conclusion that
> the tombstones can be pruned during the compaction, causing the data to be
> resurrected.
> The issue is present in Cassandra 4.0 and 4.1. Cassandra 3.11 creates the
> {{CompactionController}} after the set of sstables to compact has been
> reduced, and is thus not affected. {{trunk}} does not appear to support
> partial compactions at all, but instead refuses to compact when the disk is
> full.
> This regression appears to have been introduced by CASSANDRA-13068.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18507) Partial compaction can resurrect deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-18507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-18507:
--
Reviewers: David Capwell, Marcus Eriksson, David Capwell
David Capwell, Marcus Eriksson, David Capwell (was: David
Capwell, Marcus Eriksson)
Status: Review In Progress (was: Patch Available)
Overall the patch LGTM... its not clear why the test doesn't work on
[~maedhroz] and my laptop, but think [~marcuse] can help here.
For the core patch, I am +1. I do think we will want this test for trunk just
to make sure this regression doesn't come back.
> Partial compaction can resurrect deleted data
> -
>
> Key: CASSANDRA-18507
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18507
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Tobias Lindaaker
>Assignee: Tobias Lindaaker
>Priority: Normal
>
> If there isn't enough disk space available to compact all existing sstables,
> Cassandra will attempt to perform a partial compaction by removing sstables
> from the set of candidate sstables to be compacted, starting with the largest
> one. It is possible that the sstable removed from the set of sstables to
> compact contains data for which there are tombstones in another (more recent)
> sstable. Since the overlaps between sstables is computed when the
> {{CompactionController}} is created, and the {{CompactionController}} is
> created before the removal of any sstables from the set of sstables to be
> compacted this computed overlap will be outdated when checking which sstables
> are covered by certain tombstones. This leads to the faulty conclusion that
> the tombstones can be pruned during the compaction, causing the data to be
> resurrected.
> The issue is present in Cassandra 4.0 and 4.1. Cassandra 3.11 creates the
> {{CompactionController}} after the set of sstables to compact has been
> reduced, and is thus not affected. {{trunk}} does not appear to support
> partial compactions at all, but instead refuses to compact when the disk is
> full.
> This regression appears to have been introduced by CASSANDRA-13068.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18507) Partial compaction can resurrect deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-18507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tobias Lindaaker updated CASSANDRA-18507:
-
Description:
If there isn't enough disk space available to compact all existing sstables,
Cassandra will attempt to perform a partial compaction by removing sstables
from the set of candidate sstables to be compacted, starting with the largest
one. It is possible that the sstable removed from the set of sstables to
compact contains data for which there are tombstones in another (more recent)
sstable. Since the overlaps between sstables is computed when the
{{CompactionController}} is created, and the {{CompactionController}} is
created before the removal of any sstables from the set of sstables to be
compacted this computed overlap will be outdated when checking which sstables
are covered by certain tombstones. This leads to the faulty conclusion that the
tombstones can be pruned during the compaction, causing the data to be
resurrected.
The issue is present in Cassandra 4.0 and 4.1. Cassandra 3.11 creates the
{{CompactionController}} after the set of sstables to compact has been reduced,
and is thus not affected. {{trunk}} does not appear to support partial
compactions at all, but instead refuses to compact when the disk is full.
This regression appears to have been introduced by CASSANDRA-13068.
was:
If there isn't enough disk space available to compact all existing sstables,
Cassandra will attempt to perform a partial compaction by removing sstables
from the set of candidate sstables to be compacted, starting with the largest
one. It is possible that the sstable removed from the set of sstables to
compact contains data for which there are tombstones in another (more recent)
sstable. Since the overlaps between sstables is computed when the
{{CompactionController}} is created, and the {{CompactionController}} is
created before the removal of any sstables from the set of sstables to be
compacted this computed overlap will be outdated when checking which sstables
are covered by certain tombstones. This leads to the faulty conclusion that the
tombstones can be pruned during the compaction, causing the data to be
resurrected.
The issue is present in Cassandra 4.0 and 4.1. Cassandra 3.11 creates the
{{CompactionController}} after the set of sstables to compact has been reduced,
and is thus not affected. {{trunk}} does not appear to support partial
compactions at all, but instead refuses to compact when the disk is full.
This regression appears to have been introduced by CASSANDRA-18507.
> Partial compaction can resurrect deleted data
> -
>
> Key: CASSANDRA-18507
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18507
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Tobias Lindaaker
>Assignee: Tobias Lindaaker
>Priority: Normal
>
> If there isn't enough disk space available to compact all existing sstables,
> Cassandra will attempt to perform a partial compaction by removing sstables
> from the set of candidate sstables to be compacted, starting with the largest
> one. It is possible that the sstable removed from the set of sstables to
> compact contains data for which there are tombstones in another (more recent)
> sstable. Since the overlaps between sstables is computed when the
> {{CompactionController}} is created, and the {{CompactionController}} is
> created before the removal of any sstables from the set of sstables to be
> compacted this computed overlap will be outdated when checking which sstables
> are covered by certain tombstones. This leads to the faulty conclusion that
> the tombstones can be pruned during the compaction, causing the data to be
> resurrected.
> The issue is present in Cassandra 4.0 and 4.1. Cassandra 3.11 creates the
> {{CompactionController}} after the set of sstables to compact has been
> reduced, and is thus not affected. {{trunk}} does not appear to support
> partial compactions at all, but instead refuses to compact when the disk is
> full.
> This regression appears to have been introduced by CASSANDRA-13068.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18507) Partial compaction can resurrect deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-18507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tobias Lindaaker updated CASSANDRA-18507:
-
Description:
If there isn't enough disk space available to compact all existing sstables,
Cassandra will attempt to perform a partial compaction by removing sstables
from the set of candidate sstables to be compacted, starting with the largest
one. It is possible that the sstable removed from the set of sstables to
compact contains data for which there are tombstones in another (more recent)
sstable. Since the overlaps between sstables is computed when the
{{CompactionController}} is created, and the {{CompactionController}} is
created before the removal of any sstables from the set of sstables to be
compacted this computed overlap will be outdated when checking which sstables
are covered by certain tombstones. This leads to the faulty conclusion that the
tombstones can be pruned during the compaction, causing the data to be
resurrected.
The issue is present in Cassandra 4.0 and 4.1. Cassandra 3.11 creates the
{{CompactionController}} after the set of sstables to compact has been reduced,
and is thus not affected. {{trunk}} does not appear to support partial
compactions at all, but instead refuses to compact when the disk is full.
This regression appears to have been introduced by CASSANDRA-18507.
was:
If there isn't enough disk space available to compact all existing sstables,
Cassandra will attempt to perform a partial compaction by removing sstables
from the set of candidate sstables to be compacted, starting with the largest
one. It is possible that the sstable removed from the set of sstables to
compact contains data for which there are tombstones in another (more recent)
sstable. Since the overlaps between sstables is computed when the
{{CompactionController}} is created, and the {{CompactionController}} is
created before the removal of any sstables from the set of sstables to be
compacted this computed overlap will be outdated when checking which sstables
are covered by certain tombstones. This leads to the faulty conclusion that the
tombstones can be pruned during the compaction, causing the data to be
resurrected.
The issue is present in Cassandra 4.0 and 4.1. Cassandra 3.11 creates the
{{CompactionController}} after the set of sstables to compact has been reduced,
and is thus not affected. {{trunk}} does not appear to support partial
compactions at all, but instead refuses to compact when the disk is full.
> Partial compaction can resurrect deleted data
> -
>
> Key: CASSANDRA-18507
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18507
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Tobias Lindaaker
>Assignee: Tobias Lindaaker
>Priority: Normal
>
> If there isn't enough disk space available to compact all existing sstables,
> Cassandra will attempt to perform a partial compaction by removing sstables
> from the set of candidate sstables to be compacted, starting with the largest
> one. It is possible that the sstable removed from the set of sstables to
> compact contains data for which there are tombstones in another (more recent)
> sstable. Since the overlaps between sstables is computed when the
> {{CompactionController}} is created, and the {{CompactionController}} is
> created before the removal of any sstables from the set of sstables to be
> compacted this computed overlap will be outdated when checking which sstables
> are covered by certain tombstones. This leads to the faulty conclusion that
> the tombstones can be pruned during the compaction, causing the data to be
> resurrected.
> The issue is present in Cassandra 4.0 and 4.1. Cassandra 3.11 creates the
> {{CompactionController}} after the set of sstables to compact has been
> reduced, and is thus not affected. {{trunk}} does not appear to support
> partial compactions at all, but instead refuses to compact when the disk is
> full.
> This regression appears to have been introduced by CASSANDRA-18507.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18507) Partial compaction can resurrect deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-18507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremiah Jordan updated CASSANDRA-18507:
Test and Documentation Plan: Unit Tests
Status: Patch Available (was: Open)
> Partial compaction can resurrect deleted data
> -
>
> Key: CASSANDRA-18507
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18507
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Tobias Lindaaker
>Assignee: Tobias Lindaaker
>Priority: Normal
>
> If there isn't enough disk space available to compact all existing sstables,
> Cassandra will attempt to perform a partial compaction by removing sstables
> from the set of candidate sstables to be compacted, starting with the largest
> one. It is possible that the sstable removed from the set of sstables to
> compact contains data for which there are tombstones in another (more recent)
> sstable. Since the overlaps between sstables is computed when the
> {{CompactionController}} is created, and the {{CompactionController}} is
> created before the removal of any sstables from the set of sstables to be
> compacted this computed overlap will be outdated when checking which sstables
> are covered by certain tombstones. This leads to the faulty conclusion that
> the tombstones can be pruned during the compaction, causing the data to be
> resurrected.
> The issue is present in Cassandra 4.0 and 4.1. Cassandra 3.11 creates the
> {{CompactionController}} after the set of sstables to compact has been
> reduced, and is thus not affected. {{trunk}} does not appear to support
> partial compactions at all, but instead refuses to compact when the disk is
> full.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18507) Partial compaction can resurrect deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-18507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremiah Jordan updated CASSANDRA-18507:
Bug Category: Parent values: Correctness(12982)Level 1 values:
Unrecoverable Corruption / Loss(13161)
Complexity: Normal
Component/s: Local/Compaction
Discovered By: User Report
Severity: Critical
Assignee: Tobias Lindaaker
Status: Open (was: Triage Needed)
> Partial compaction can resurrect deleted data
> -
>
> Key: CASSANDRA-18507
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18507
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Tobias Lindaaker
>Assignee: Tobias Lindaaker
>Priority: Normal
>
> If there isn't enough disk space available to compact all existing sstables,
> Cassandra will attempt to perform a partial compaction by removing sstables
> from the set of candidate sstables to be compacted, starting with the largest
> one. It is possible that the sstable removed from the set of sstables to
> compact contains data for which there are tombstones in another (more recent)
> sstable. Since the overlaps between sstables is computed when the
> {{CompactionController}} is created, and the {{CompactionController}} is
> created before the removal of any sstables from the set of sstables to be
> compacted this computed overlap will be outdated when checking which sstables
> are covered by certain tombstones. This leads to the faulty conclusion that
> the tombstones can be pruned during the compaction, causing the data to be
> resurrected.
> The issue is present in Cassandra 4.0 and 4.1. Cassandra 3.11 creates the
> {{CompactionController}} after the set of sstables to compact has been
> reduced, and is thus not affected. {{trunk}} does not appear to support
> partial compactions at all, but instead refuses to compact when the disk is
> full.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18507) Partial compaction can resurrect deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-18507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tobias Lindaaker updated CASSANDRA-18507:
-
Description:
If there isn't enough disk space available to compact all existing sstables,
Cassandra will attempt to perform a partial compaction by removing sstables
from the set of candidate sstables to be compacted, starting with the largest
one. It is possible that the sstable removed from the set of sstables to
compact contains data for which there are tombstones in another (more recent)
sstable. Since the overlaps between sstables is computed when the
{{CompactionController}} is created, and the {{CompactionController}} is
created before the removal of any sstables from the set of sstables to be
compacted this computed overlap will be outdated when checking which sstables
are covered by certain tombstones. This leads to the faulty conclusion that the
tombstones can be pruned during the compaction, causing the data to be
resurrected.
The issue is present in Cassandra 4.0 and 4.1. Cassandra 3.11 creates the
{{CompactionController}} after the set of sstables to compact has been reduced,
and is thus not affected. {{trunk}} does not appear to support partial
compactions at all, but instead refuses to compact when the disk is full.
was:If there isn't enough disk space available to compact all existing
sstables, Cassandra will attempt to perform a partial compaction by removing
sstables from the set of candidate sstables to be compacted, starting with the
largest one. It is possible that the sstable removed from the set of sstables
to compact contains data for which there are tombstones in another (more
recent) sstable. Since the overlaps between sstables is computed when the
{{CompactionController}} is created, and the {{CompactionController}} is
created before the removal of any sstables from the set of sstables to be
compacted this computed overlap will be outdated when checking which sstables
are covered by certain tombstones. This leads to the faulty conclusion that the
tombstones can be pruned during the compaction, causing the data to be
resurrected.
> Partial compaction can resurrect deleted data
> -
>
> Key: CASSANDRA-18507
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18507
> Project: Cassandra
> Issue Type: Bug
>Reporter: Tobias Lindaaker
>Priority: Normal
>
> If there isn't enough disk space available to compact all existing sstables,
> Cassandra will attempt to perform a partial compaction by removing sstables
> from the set of candidate sstables to be compacted, starting with the largest
> one. It is possible that the sstable removed from the set of sstables to
> compact contains data for which there are tombstones in another (more recent)
> sstable. Since the overlaps between sstables is computed when the
> {{CompactionController}} is created, and the {{CompactionController}} is
> created before the removal of any sstables from the set of sstables to be
> compacted this computed overlap will be outdated when checking which sstables
> are covered by certain tombstones. This leads to the faulty conclusion that
> the tombstones can be pruned during the compaction, causing the data to be
> resurrected.
> The issue is present in Cassandra 4.0 and 4.1. Cassandra 3.11 creates the
> {{CompactionController}} after the set of sstables to compact has been
> reduced, and is thus not affected. {{trunk}} does not appear to support
> partial compactions at all, but instead refuses to compact when the disk is
> full.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
