[ https://issues.apache.org/jira/browse/CASSANDRA-5761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13713432#comment-13713432 ]
Andriy Yevsyukov commented on CASSANDRA-5761: --------------------------------------------- We don't use StorageProxy and json2sstable and SSTableUnsortedWriter. Before start we create CFs skeletons where we set up mandatory columns and secondary indexes using CQL3 All the insertions we do only with plain Hector. > Issue with secondary index sstable. > ----------------------------------- > > Key: CASSANDRA-5761 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5761 > Project: Cassandra > Issue Type: Bug > Components: Core > Affects Versions: 1.2.5 > Reporter: Andriy Yevsyukov > Priority: Critical > > With Cassandra 1.2.5 having issue very similar to > [CASSANDRA-5225|https://issues.apache.org/jira/browse/CASSANDRA-5225] but for > secondary index sstable. Every query that uses this index fails in Hector > with ConnectionTimeout but cassandra log says that reason is: > {noformat} > ERROR [ReadStage:55803] 2013-07-15 12:11:35,392 CassandraDaemon.java (line > 175) Exception in thread Thread[ReadStage:55803,5,main] > java.lang.RuntimeException: > org.apache.cassandra.io.sstable.CorruptSSTableException: > org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid > column name length 0 > (/data/cassandra/data/betting/events/betting-events.events_sport_type_idx-ic-1-Data.db, > 19658 bytes remaining) > at > org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1582) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > Caused by: org.apache.cassandra.io.sstable.CorruptSSTableException: > org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid > column name length 0 > (/data/cassandra/data/betting/events/betting-events.events_sport_type_idx-ic-1-Data.db, > 19658 bytes remaining) > at > org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:108) > at > org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:39) > at > com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) > at > com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) > at > org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:90) > at > org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:171) > at > org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:154) > at > org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:143) > at > org.apache.cassandra.utils.MergeIterator$ManyToOne.<init>(MergeIterator.java:86) > at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:45) > at > org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:134) > at > org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:84) > at > org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:293) > at > org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:65) > at > org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1357) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1214) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1126) > at > org.apache.cassandra.db.index.keys.KeysSearcher$1.computeNext(KeysSearcher.java:140) > at > org.apache.cassandra.db.index.keys.KeysSearcher$1.computeNext(KeysSearcher.java:109) > at > com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) > at > com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) > at > org.apache.cassandra.db.ColumnFamilyStore.filter(ColumnFamilyStore.java:1466) > at > org.apache.cassandra.db.index.keys.KeysSearcher.search(KeysSearcher.java:82) > at > org.apache.cassandra.db.index.SecondaryIndexManager.search(SecondaryIndexManager.java:548) > at > org.apache.cassandra.db.ColumnFamilyStore.search(ColumnFamilyStore.java:1454) > at > org.apache.cassandra.service.RangeSliceVerbHandler.executeLocally(RangeSliceVerbHandler.java:44) > at > org.apache.cassandra.service.StorageProxy$LocalRangeSliceRunnable.runMayThrow(StorageProxy.java:1076) > at > org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1578) > ... 3 more > Caused by: org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: > invalid column name length 0 > (/data/cassandra/data/betting/events/betting-events.events_sport_type_idx-ic-1-Data.db, > 19658 bytes remaining) > at > org.apache.cassandra.db.ColumnSerializer$CorruptColumnException.create(ColumnSerializer.java:148) > at > org.apache.cassandra.db.RangeTombstone$Serializer.deserializeBody(RangeTombstone.java:279) > at > org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:90) > at > org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:73) > at > org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:104) > ... 30 more > {noformat} > We cleared CF but we have a lot of insertion and in few hours the issue > happened again. Are there any advices? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira