[ https://issues.apache.org/jira/browse/CASSANDRA-6008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13811374#comment-13811374 ]
Nikolai Grigoriev commented on CASSANDRA-6008: ---------------------------------------------- Just upgraded to 2.0.2. Now I have this problem on every single node, my entire cluster is dead. Not a single node would start. > Getting 'This should never happen' error at startup due to sstables missing > --------------------------------------------------------------------------- > > Key: CASSANDRA-6008 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6008 > Project: Cassandra > Issue Type: Bug > Components: Core > Reporter: John Carrino > Assignee: Yuki Morishita > Fix For: 2.0.3 > > > Exception encountered during startup: "Unfinished compactions reference > missing sstables. This should never happen since compactions are marked > finished before we start removing the old sstables" > This happens when sstables that have been compacted away are removed, but > they still have entries in the system.compactions_in_progress table. > Normally this should not happen because the entries in > system.compactions_in_progress are deleted before the old sstables are > deleted. > However at startup recovery time, old sstables are deleted (NOT BEFORE they > are removed from the compactions_in_progress table) and then after that is > done it does a truncate using SystemKeyspace.discardCompactionsInProgress > We ran into a case where the disk filled up and the node died and was bounced > and then failed to truncate this table on startup, and then got stuck hitting > this exception in ColumnFamilyStore.removeUnfinishedCompactionLeftovers. > Maybe on startup we can delete from this table incrementally as we clean > stuff up in the same way that compactions delete from this table before they > delete old sstables. -- This message was sent by Atlassian JIRA (v6.1#6144)