nodetool scrub will repair out of order rows in the source SSTables for the compaction process. Or you can stop the node and use the offline bin/sstablescrub tool
Not sure how they got there, there was a ticket for similar problems in 1.1.1 Cheers ----------------- Aaron Morton Freelance Cassandra Consultant New Zealand @aaronmorton http://www.thelastpickle.com On 30/04/2013, at 1:31 AM, Oleg Dulin <oleg.du...@gmail.com> wrote: > We saw this exception with 1.1.1 and also with 1.1.11 (we upgraded for > unrelated reasons, to fix the FD leak during slice queries) -- name of the CF > replaced with "*" for confidentiality: > > 10419 ERROR [CompactionExecutor:36] 2013-04-29 07:50:49,060 > AbstractCassandraDaemon.java (line 132) Exception in thread T > hread[CompactionExecutor:36,1,main] > 10420 java.lang.RuntimeException: Last written key > DecoratedKey(138024912283272996716128964353306009224, 6138633035613062 > 2d616666362d376330612d666531662d373738616630636265396535) >= current key > DecoratedKey(12706537740594940274338371890 1402082101, > 64323962636163652d646561372d333039322d386166322d663064346132363963386131) > writing into *-tmp-hf-7372-Data.db > 10421 at > org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:134) > > 10422 at > org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:153) > 10423 at > org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:160) > > 10424 at > org.apache.cassandra.db.compaction.LeveledCompactionTask.execute(LeveledCompactionTask.java:50) > > 10425 at > org.apache.cassandra.db.compaction.CompactionManager$2.runMayThrow(CompactionManager.java:164) > > 10426 at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) > 10427 at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > 10428 at > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > 10429 at java.util.concurrent.FutureTask.run(FutureTask.java:166) > 10430 at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > 10431 at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > 10432 at java.lang.Thread.run(Thread.java:722) > > > ANy thoughts ? Should I be concerned about data being lost ? > > > -- > Regards, > Oleg Dulin > NYC Java Big Data Engineer > http://www.olegdulin.com/ > >
