[ https://issues.apache.org/jira/browse/CASSANDRA-5930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13756170#comment-13756170 ]
Jeff Potter commented on CASSANDRA-5930: ---------------------------------------- We're seeing this too -- slightly different stack trace, which I'll include here in case it's of use. WARNING: Non-fatal error reading row (stacktrace follows) Exception in thread "main" java.io.IOError: java.lang.IllegalArgumentException at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:244) at org.apache.cassandra.tools.StandaloneScrubber.main(StandaloneScrubber.java:125) Caused by: java.lang.IllegalArgumentException at java.nio.Buffer.limit(Buffer.java:247) at org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:51) at org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:60) at org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:78) at org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:31) at org.apache.cassandra.db.ArrayBackedSortedColumns.addColumn(ArrayBackedSortedColumns.java:128) at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:114) at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:109) at org.apache.cassandra.db.ColumnFamily.addAtom(ColumnFamily.java:219) at org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumnsFromSSTable(ColumnFamilySerializer.java:149) at org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:234) at org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:114) at org.apache.cassandra.db.compaction.PrecompactedRow.<init>(PrecompactedRow.java:98) at org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:160) at org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:166) at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:173) ... 1 more > Offline scrubs can choke on broken files > ---------------------------------------- > > Key: CASSANDRA-5930 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5930 > Project: Cassandra > Issue Type: Bug > Reporter: Jeremiah Jordan > Assignee: Jason Brown > Priority: Minor > > There are cases where offline scrub can hit an exception and die, like: > {noformat} > WARNING: Non-fatal error reading row (stacktrace follows) > Exception in thread "main" java.io.IOError: java.io.IOError: > java.io.EOFException > at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:242) > at > org.apache.cassandra.tools.StandaloneScrubber.main(StandaloneScrubber.java:121) > Caused by: java.io.IOError: java.io.EOFException > at > org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:116) > at > org.apache.cassandra.db.compaction.PrecompactedRow.<init>(PrecompactedRow.java:99) > at > org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:176) > at > org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:182) > at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:171) > ... 1 more > Caused by: java.io.EOFException > at java.io.RandomAccessFile.readFully(RandomAccessFile.java:399) > at java.io.RandomAccessFile.readFully(RandomAccessFile.java:377) > at > org.apache.cassandra.utils.BytesReadTracker.readFully(BytesReadTracker.java:95) > at > org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:401) > at > org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:363) > at > org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:120) > at > org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:37) > at > org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumns(ColumnFamilySerializer.java:144) > at > org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:234) > at > org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:112) > ... 5 more > {noformat} > Since the purpose of offline scrub is to fix broken stuff, it should be more > resilient to broken stuff... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira