[ https://issues.apache.org/jira/browse/CASSANDRA-9525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14567754#comment-14567754 ]
Aleksey Yeschenko commented on CASSANDRA-9525: ---------------------------------------------- cc [~benedict] > Commitlog allocation failure doesn't stop the entire node. > ---------------------------------------------------------- > > Key: CASSANDRA-9525 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9525 > Project: Cassandra > Issue Type: Bug > Components: Core > Reporter: Jeremiah Jordan > Fix For: 2.1.x > > > If there is an error opening a commit log segment, the thread dies, but it > doesn't stop the whole node. > Got the following on a node: > {noformat} > ERROR [COMMIT-LOG-ALLOCATOR] 2015-05-29 17:06:45,722 CommitLog.java:397 - > Failed managing commit log segments. Commit disk failure policy is stop; > terminating thread > org.apache.cassandra.io.FSWriteError: java.io.FileNotFoundException: > /var/lib/cassandra/commitlog/CommitLog-4-1432937194590.log (Permission denied) > at > org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:177) > ~[cassandra-all-2.1.5.jar:2.1.5] > at > org.apache.cassandra.db.commitlog.CommitLogSegmentManager$4.call(CommitLogSegmentManager.java:397) > ~[cassandra-all-2.1.5.jar:2.1.5] > at > org.apache.cassandra.db.commitlog.CommitLogSegmentManager$4.call(CommitLogSegmentManager.java:394) > ~[cassandra-all-2.1.5.jar:2.1.5] > at > org.apache.cassandra.db.commitlog.CommitLogSegmentManager$1.runMayThrow(CommitLogSegmentManager.java:152) > ~[cassandra-all-2.1.5.jar:2.1.5] > at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > [cassandra-all-2.1.5.jar:2.1.5] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45] > Caused by: java.io.FileNotFoundException: > /var/lib/cassandra/commitlog/CommitLog-4-1432937194590.log (Permission denied) > at java.io.RandomAccessFile.open0(Native Method) ~[na:1.8.0_45] > at java.io.RandomAccessFile.open(RandomAccessFile.java:316) > ~[na:1.8.0_45] > at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243) > ~[na:1.8.0_45] > at > org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:155) > ~[cassandra-all-2.1.5.jar:2.1.5] > ... 5 common frames omitted > {noformat} > And the node stayed kind of up, didn't notice something wrong until the node > died OOM because some threads were dead and others weren't. -- This message was sent by Atlassian JIRA (v6.3.4#6332)