[
https://issues.apache.org/jira/browse/CASSANDRA-8743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14363965#comment-14363965
]
Joshua McKenzie commented on CASSANDRA-8743:
--------------------------------------------
[~tamarn] No luck reproducing on this side on 2.1.2.
3-node cluster with all files stored over NFS mounted drive, killed a node in
the middle of a long-running stress and cleared its data, brought it back up
and repaired on it. Doing this repeatedly and not seeing any exceptions.
If you can consistently reproduce this, if you could try out 2.1-HEAD or even
better, trunk, that would be very helpful. Also, if you could provide more
information about your use case and topology that would help as well.
> Repair on NFS in version 2.1.2
> ------------------------------
>
> Key: CASSANDRA-8743
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8743
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Tamar Nirenberg
> Assignee: Joshua McKenzie
> Priority: Minor
>
> Running repair over NFS in Cassandra 2.1.2 encounters this error and crashes
> the ring:
> ERROR [ValidationExecutor:2] 2015-01-22 11:48:14,811 Validator.java:232 -
> Failed creating a merkle tree for [repair
> #c84c7c70-a21b-11e4-aeca-19e6d7fa2595 on ATTRIBUTES/LINKS,
> (11621838520493020277529637175352775759,11853478749048239324667887059881170862]],
> /10.1.234.63 (see log for details)
> ERROR [ValidationExecutor:2] 2015-01-22 11:48:14,827 CassandraDaemon.java:153
> - Exception in thread Thread[ValidationExecutor:2,1,main]
> org.apache.cassandra.io.FSWriteError:
> java.nio.file.DirectoryNotEmptyException:
> /exlibris/cassandra/local/data/data/ATTRIBUTES/LINKS/snapshots/c84c7c70-a21b-11e4-aeca-19e6d7fa2595
> at
> org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:135)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.io.util.FileUtils.deleteRecursive(FileUtils.java:381)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.db.Directories.clearSnapshot(Directories.java:547)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.db.ColumnFamilyStore.clearSnapshot(ColumnFamilyStore.java:2223)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:939)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:97)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:557)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ~[na:1.7.0_71]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> ~[na:1.7.0_71]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_71]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
> Caused by: java.nio.file.DirectoryNotEmptyException:
> /exlibris/cassandra/local/data/data/ATTRIBUTES/LINKS/snapshots/c84c7c70-a21b-11e4-aeca-19e6d7fa2595
> at
> sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:242)
> ~[na:1.7.0_71]
> at
> sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
> ~[na:1.7.0_71]
> at java.nio.file.Files.delete(Files.java:1079) ~[na:1.7.0_71]
> at
> org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:131)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> ... 10 common frames omitted
> ERROR [ValidationExecutor:2] 2015-01-22 11:48:14,829 StorageService.java:383
> - Stopping gossiper
> WARN [ValidationExecutor:2] 2015-01-22 11:48:14,829 StorageService.java:291
> - Stopping gossip by operator request
> INFO [ValidationExecutor:2] 2015-01-22 11:48:14,829 Gossiper.java:1318 -
> Announcing shutdown
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
