[ 
https://issues.apache.org/jira/browse/CASSANDRA-7145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-7145:
---------------------------------------

    Attachment: 0001-avoid-marking-compacted-sstables-as-compacting.patch

If we have a situation where this happens (in sequence);

# We ask LeveledManifest for a new CompactionCandidate
# LCS returns a CompactionCandidate containing sstables marked as compacting (a 
bug)
# The compaction that held one of the sstables we marked in #2 finishes and 
removes the files that were included in the compaction
# We successfully mark the compacted sstable as compacting (it is no longer 
marked as compacting in the View)
# FileNotFoundException once we start trying to compact

Attached patch 
* removes a case in LCS where we could return compacting sstables in a 
CompactionCandidate
* makes sure we can't mark compacted sstables as compacting

It would be much appreciated if anyone that can reproduce this could try with 
the attached patch to see if the problem goes away.

> FileNotFoundException during compaction
> ---------------------------------------
>
>                 Key: CASSANDRA-7145
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7145
>             Project: Cassandra
>          Issue Type: Bug
>         Environment: CentOS 6.3, Datastax Enterprise 4.0.1 (Cassandra 2.0.5), 
> Java 1.7.0_55
>            Reporter: PJ
>            Assignee: Marcus Eriksson
>             Fix For: 2.0.10
>
>         Attachments: 
> 0001-avoid-marking-compacted-sstables-as-compacting.patch, compaction - 
> FileNotFoundException.txt, repair - RuntimeException.txt, startup - 
> AssertionError.txt
>
>
> I can't finish any compaction because my nodes always throw a 
> "FileNotFoundException". I've already tried the following but nothing helped:
> 1. nodetool flush
> 2. nodetool repair (ends with RuntimeException; see attachment)
> 3. node restart (via dse cassandra-stop)
> Whenever I restart the nodes, another type of exception is logged (see 
> attachment) somewhere near the end of startup process. This particular 
> exception doesn't seem to be critical because the nodes still manage to 
> finish the startup and become online.
> I don't have specific steps to reproduce the problem that I'm experiencing 
> with compaction and repair. I'm in the middle of migrating 4.8 billion rows 
> from MySQL via SSTableLoader. 
> Some things that may or may not be relevant:
> 1. I didn't drop and recreate the keyspace (so probably not related to 
> CASSANDRA-4857)
> 2. I do the bulk-loading in batches of 1 to 20 millions rows. When a batch 
> reaches 100% total progress (i.e. starts to build secondary index), I kill 
> the sstableloader process and cancel the index build
> 3. I restart the nodes occasionally. It's possible that there is an on-going 
> compaction during one of those restarts.
> Related StackOverflow question (mine): 
> http://stackoverflow.com/questions/23435847/filenotfoundexception-during-compaction



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Reply via email to