[
https://issues.apache.org/jira/browse/CASSANDRA-1248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050076#comment-13050076
]
Patricio Echague commented on CASSANDRA-1248:
---------------------------------------------
David, just out of curiosity. Are commits logs and Data partition all in the
same volume?
if not, what partition ran out of space?
> ArrayIndexOutOfBoundsException thrown when reading partially-written sstable
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-1248
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1248
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 0.6.3
> java version "1.6.0_0"
> IcedTea6 1.3.1 (6b12-0ubuntu6.6) Runtime Environment (build 1.6.0_0-b12)
> OpenJDK 64-Bit Server VM (build 1.6.0_0-b12, mixed mode)
> Ubuntu 8.10
> 6 node cluster
> RF==3
> Reporter: David King
> Assignee: Patricio Echague
> Fix For: 0.8.2
>
>
> A node died by running out of disk space. After growing the volume, trying to
> execute reads dies with:
> {quote}
> ERROR 11:02:51,957 Uncaught exception in thread
> Thread[ROW-READ-STAGE:10,5,main]
> java.lang.ArrayIndexOutOfBoundsException
> at
> org.apache.cassandra.io.util.BufferedRandomAccessFile.read(BufferedRandomAccessFile.java:326)
> at java.io.RandomAccessFile.readFully(RandomAccessFile.java:414)
> at java.io.DataInputStream.readUTF(DataInputStream.java:609)
> at java.io.RandomAccessFile.readUTF(RandomAccessFile.java:938)
> at
> org.apache.cassandra.db.filter.SSTableSliceIterator$ColumnGroupReader.<init>(SSTableSliceIterator.java:125)
> at
> org.apache.cassandra.db.filter.SSTableSliceIterator.<init>(SSTableSliceIterator.java:59)
> at
> org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:63)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:859)
> at
> org.apache.cassandra.db.ColumnFamilyStore.cacheRow(ColumnFamilyStore.java:747)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:772)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:739)
> at org.apache.cassandra.db.Table.getRow(Table.java:381)
> at
> org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:56)
> at
> org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:70)
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:40)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:636)
> {quote}
> The clients then die with TimeoutExceptions.
> It might be better to return an exception to the client immediately instead
> of letting them die, and it would be helpful if this exception were
> accompanied with a message indicating *which* SStable is malformed.
> This command does *not* turn up the failed table (nor does the same with
> sstablekeys):
> {quote}
> ri@pmc08:/cassandra/data$ for x in $(ls -t */*-Data.db); do ls -lh $x;
> /usr/local/cassandra/bin/sstable2json $x > /dev/null; done
> {quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
