[ https://issues.apache.org/jira/browse/CASSANDRA-19329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17812449#comment-17812449 ]
Brandon Williams edited comment on CASSANDRA-19329 at 1/31/24 1:34 AM: ----------------------------------------------------------------------- In 18260, compaction is preemptively declaring there is not enough space, which is different from receiving an error when trying to write to the disk. edit: but actually, both are proactive errors and not _really_ out of space yet, and I'm not yet sure why one is a RuntimeError and the other is a proper exception that triggers the policy. was (Author: brandon.williams): In 18260, compaction is preemptively declaring there is not enough space, which is different from receiving an error when trying to write to the disk. > Repair can trigger the disk_failure_policy > ------------------------------------------ > > Key: CASSANDRA-19329 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19329 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Repair > Reporter: Meredith Silvernail > Priority: Normal > > While a cluster was running a repair, we observed Insufficient disk space > would unexpectedly shut down gossip instead of skipping the repair and give > the below error. It would seem the right behavior would be to skip and go on > to the next one, as nodetool garbagecollect and cleanup would do. Instead > the two nodes that were experiencing the error continued to stay in > UP-PARTIAL state with no gossip running. > > {quote}2024-01-20 18:37:50,716 [ERROR] [CompactionExecutor:2] cluster_id=9 > ip_address=10.0.0.1 CassandraDaemon.java:581 - Exception in thread > Thread[CompactionExecutor:2,1,main] > org.apache.cassandra.io.FSDiskFullWriteError: java.io.IOException: > Insufficient disk space to write 625503268755 bytes into the foobar keyspace > at > org.apache.cassandra.db.Directories.getWriteableLocation(Directories.java:443) > at > org.apache.cassandra.db.Directories.getWriteableLocationAsFile(Directories.java:339) > at > org.apache.cassandra.db.compaction.CompactionManager.antiCompactGroup(CompactionManager.java:1556) > at > org.apache.cassandra.db.compaction.CompactionManager.doAntiCompaction(CompactionManager.java:1522) > at > org.apache.cassandra.db.compaction.CompactionManager.performAnticompaction(CompactionManager.java:788) > at > org.apache.cassandra.db.compaction.CompactionManager$9.runMayThrow(CompactionManager.java:702) > at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > at > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > at java.base/java.lang.Thread.run(Thread.java:834) > Caused by: java.io.IOException: Insufficient disk space to write 625503268755 > bytes into the foobar keyspace > at > org.apache.cassandra.io.FSDiskFullWriteError.<init>(FSDiskFullWriteError.java:27) > ... 13 common frames omitted > {quote} -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org