[ https://issues.apache.org/jira/browse/CASSANDRA-4456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419912#comment-13419912 ]
Michael Theroux commented on CASSANDRA-4456: -------------------------------------------- I just hit this problem myself today, on a single node in a six node cluster. I was running nodetool repair, and it halted with this exception in the log. I was monitoring the repair pretty closely. A couple of observations: 1) It happened while compaction of the same column family was happening simultaneously 2) When I re-ran it, it worked. Note: I am not a cassandra developer, but I looked at the code. A highly uneducated guess is that an sstable was compacted and deleted while validation was expecting it to be there? > AssertionError in ColumnFamilyStore.getOverlappingSSTables() during repair > -------------------------------------------------------------------------- > > Key: CASSANDRA-4456 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4456 > Project: Cassandra > Issue Type: Bug > Components: Core > Affects Versions: 1.1.2 > Environment: Ubuntu 11.04 64-bit > Reporter: Mike Heffner > > We have hit the following exception on several nodes while running repairs > across our 1.1.2 ring. We've observed it happen on either the node executing > the repair or a participating replica in the repair operation. The result in > either case is that the repair hangs. > ERROR [ValidationExecutor:9] 2012-07-21 01:54:03,019 > AbstractCassandraDaemon.java (line 134) Exception in thread > Thread[ValidationExecutor:9,1,main] > java.lang.AssertionError > at > org.apache.cassandra.db.ColumnFamilyStore.getOverlappingSSTables(ColumnFamilyStore.java:874) > at > org.apache.cassandra.db.compaction.CompactionController.<init>(CompactionController.java:69) > at > org.apache.cassandra.db.compaction.CompactionManager$ValidationCompactionIterable.<init>(CompactionManager.java:834) > at > org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:698) > at > org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:68) > at > org.apache.cassandra.db.compaction.CompactionManager$8.call(CompactionManager.java:438) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > In building this ring we migrated sstables from an identical 0.8.8 ring by: > 1. Creating the schema on our new 1.1.2 ring. > 2. Rsyncing over sstables from 0.8.8 ring. > 3. Renaming the sstables to match the directory and file naming structure of > 1.1.x. > 4. Ran nodetool refresh <keyspace> <cf> for each CF across each node. > 5. Ran nodetool upgradesstables for each CF across each node. > When those steps had completed, we began rolling repairs. Not all of the > repair operations have hit the exception -- some of the repairs have > completed successfully. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira