[ https://issues.apache.org/jira/browse/CASSANDRA-8743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14339087#comment-14339087 ]
Joshua McKenzie commented on CASSANDRA-8743: -------------------------------------------- The following: {noformat} org.apache.cassandra.io.FSWriteError: java.nio.file.DirectoryNotEmptyException: /exlibris/cassandra/local/data/data/ATTRIBUTES/LINKS/snapshots/c84c7c70-a21b-11e4-aeca-19e6d7fa2595 {noformat} states that NFS isn't performing the deletes on the remote file store at the time we think it is and/or are asking for it. Given NTFS' intolerance of shared operations on files, I suspect NFS will also have restrictions that will make our reliance on deleteWithConfirm/renameWithConfirm problematic beyond just the snapshot during Merkle tree creation. What's the use case where you're storing Cassandra data files over an NFS share rather than on a local filesystem? > Repair on NFS in version 2.1.2 > ------------------------------ > > Key: CASSANDRA-8743 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8743 > Project: Cassandra > Issue Type: Bug > Components: Core > Reporter: Tamar Nirenberg > Assignee: Joshua McKenzie > Priority: Minor > > Running repair over NFS in Cassandra 2.1.2 encounters this error and crashes > the ring: > ERROR [ValidationExecutor:2] 2015-01-22 11:48:14,811 Validator.java:232 - > Failed creating a merkle tree for [repair > #c84c7c70-a21b-11e4-aeca-19e6d7fa2595 on ATTRIBUTES/LINKS, > (11621838520493020277529637175352775759,11853478749048239324667887059881170862]], > /10.1.234.63 (see log for details) > ERROR [ValidationExecutor:2] 2015-01-22 11:48:14,827 CassandraDaemon.java:153 > - Exception in thread Thread[ValidationExecutor:2,1,main] > org.apache.cassandra.io.FSWriteError: > java.nio.file.DirectoryNotEmptyException: > /exlibris/cassandra/local/data/data/ATTRIBUTES/LINKS/snapshots/c84c7c70-a21b-11e4-aeca-19e6d7fa2595 > at > org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:135) > ~[apache-cassandra-2.1.2.jar:2.1.2] > at > org.apache.cassandra.io.util.FileUtils.deleteRecursive(FileUtils.java:381) > ~[apache-cassandra-2.1.2.jar:2.1.2] > at > org.apache.cassandra.db.Directories.clearSnapshot(Directories.java:547) > ~[apache-cassandra-2.1.2.jar:2.1.2] > at > org.apache.cassandra.db.ColumnFamilyStore.clearSnapshot(ColumnFamilyStore.java:2223) > ~[apache-cassandra-2.1.2.jar:2.1.2] > at > org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:939) > ~[apache-cassandra-2.1.2.jar:2.1.2] > at > org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:97) > ~[apache-cassandra-2.1.2.jar:2.1.2] > at > org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:557) > ~[apache-cassandra-2.1.2.jar:2.1.2] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > ~[na:1.7.0_71] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > ~[na:1.7.0_71] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_71] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71] > Caused by: java.nio.file.DirectoryNotEmptyException: > /exlibris/cassandra/local/data/data/ATTRIBUTES/LINKS/snapshots/c84c7c70-a21b-11e4-aeca-19e6d7fa2595 > at > sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:242) > ~[na:1.7.0_71] > at > sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103) > ~[na:1.7.0_71] > at java.nio.file.Files.delete(Files.java:1079) ~[na:1.7.0_71] > at > org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:131) > ~[apache-cassandra-2.1.2.jar:2.1.2] > ... 10 common frames omitted > ERROR [ValidationExecutor:2] 2015-01-22 11:48:14,829 StorageService.java:383 > - Stopping gossiper > WARN [ValidationExecutor:2] 2015-01-22 11:48:14,829 StorageService.java:291 > - Stopping gossip by operator request > INFO [ValidationExecutor:2] 2015-01-22 11:48:14,829 Gossiper.java:1318 - > Announcing shutdown -- This message was sent by Atlassian JIRA (v6.3.4#6332)