[
https://issues.apache.org/jira/browse/CASSANDRA-7368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14026867#comment-14026867
]
Francois Richard commented on CASSANDRA-7368:
---------------------------------------------
The scrub of the sstables completed and the problems does not seem to happen
again. Coming back to the original problem, why did compaction totally stop on
this node even if we have the following setting:
concurrent_compactors: 26.
Should there be more resilience at the compaction level to keep on going when
some SSTABLE are corrupted?
> Compaction stops after org.apache.cassandra.io.sstable.CorruptSSTableException
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-7368
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7368
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: OS: RHEL 6.5
> Cassandra version: 1.2.16
> Reporter: Francois Richard
>
> Hi,
> We are getting a case where compaction stops totally on a node after an
> exception related to: org.apache.cassandra.io.sstable.CorruptSSTableException.
> nodetool compactionstats remains at the same level for hours:
> {code}
> pending tasks: 1451
> compaction type keyspace column family completed
> total unit progress
> Compaction SyncCoreContactPrefixBytesIndex
> 257799931 376785179 bytes 68.42%
> Active compaction remaining time : n/a
> {code}
> Here is the exception log:
> {code}
> ERROR [Deserialize
> SSTableReader(path='/home/y/var/cassandra/data/SyncCore/ContactPrefixBytesIndex/SyncCore-ContactPrefixBytesIndex-ic-116118-Data.db')]
> 2014-06-09 06:39:37,570 CassandraDaemon.java (line 191) Exception in thread
> Thread[Deserialize
> SSTableReader(path='/home/y/var/cassandra/data/SyncCore/ContactPrefixBytesIndex/SyncCore-ContactPrefixBytesIndex-ic-116118-Data.db'),1,main]
> org.apache.cassandra.io.sstable.CorruptSSTableException: java.io.IOException:
> dataSize of 7421941880990663551 starting at 257836699 would be larger than
> file
> /home/y/var/cassandra/data/SyncCore/ContactPrefixBytesIndex/SyncCore-ContactPrefixBytesIndex-ic-116118-Data.db
> length 376785179
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.<init>(SSTableIdentityIterator.java:167)
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.<init>(SSTableIdentityIterator.java:83)
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.<init>(SSTableIdentityIterator.java:69)
> at
> org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.next(SSTableScanner.java:180)
> at
> org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.next(SSTableScanner.java:155)
> at
> org.apache.cassandra.io.sstable.SSTableScanner.next(SSTableScanner.java:142)
> at
> org.apache.cassandra.io.sstable.SSTableScanner.next(SSTableScanner.java:38)
> at
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy$LeveledScanner.computeNext(LeveledCompactionStrategy.java:238)
> at
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy$LeveledScanner.computeNext(LeveledCompactionStrategy.java:207)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> --
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
