[ https://issues.apache.org/jira/browse/CASSANDRA-13123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16186704#comment-16186704 ]
Jeff Jirsa commented on CASSANDRA-13123: ---------------------------------------- Hi folks, Pretty sure this commit breaks {{CommitLogSegmentManagerTest}} - have seen a pretty sharp rise in failures, and reverting this commit seems to solve them. {code} [junit] INFO 23:34:52 Initializing CommitLogTest.Standard1 [junit] INFO 23:34:52 Initializing CommitLogTest.Standard2 [junit] ------------- ---------------- --------------- [junit] Testcase: testShutdownWithPendingTasks(org.apache.cassandra.db.commitlog.CommitLogSegmentManagerTest): FAILED [junit] null [junit] junit.framework.AssertionFailedError [junit] at org.apache.cassandra.db.Keyspace.open(Keyspace.java:105) [junit] at org.apache.cassandra.db.commitlog.CommitLogSegmentManagerTest.testShutdownWithPendingTasks(CommitLogSegmentManagerTest.java:147) [junit] at org.jboss.byteman.contrib.bmunit.BMUnitRunner$10.evaluate(BMUnitRunner.java:371) [junit] at org.jboss.byteman.contrib.bmunit.BMUnitRunner$6.evaluate(BMUnitRunner.java:241) [junit] at org.jboss.byteman.contrib.bmunit.BMUnitRunner$1.evaluate(BMUnitRunner.java:75) [junit] [junit] [junit] Testcase: testCompressedCommitLogBackpressure(org.apache.cassandra.db.commitlog.CommitLogSegmentManagerTest): FAILED [junit] expected:<3> but was:<1> [junit] junit.framework.AssertionFailedError: expected:<3> but was:<1> [junit] at org.apache.cassandra.Util.spinAssertEquals(Util.java:535) [junit] at org.apache.cassandra.db.commitlog.CommitLogSegmentManagerTest.testCompressedCommitLogBackpressure(CommitLogSegmentManagerTest.java:112) [junit] at org.jboss.byteman.contrib.bmunit.BMUnitRunner$9.evaluate(BMUnitRunner.java:342) [junit] at org.jboss.byteman.contrib.bmunit.BMUnitRunner$6.evaluate(BMUnitRunner.java:241) [junit] at org.jboss.byteman.contrib.bmunit.BMUnitRunner$1.evaluate(BMUnitRunner.java:75) [junit] [junit] [junit] Test org.apache.cassandra.db.commitlog.CommitLogSegmentManagerTest FAILED [delete] Deleting directory /Users/jjirsa/Desktop/Dev/cassandra/build/test/cassandra/commitlog:0 [delete] Deleting directory /Users/jjirsa/Desktop/Dev/cassandra/build/test/cassandra/data:0 [delete] Deleting directory /Users/jjirsa/Desktop/Dev/cassandra/build/test/cassandra/saved_caches:0 [junitreport] Processing /Users/jjirsa/Desktop/Dev/cassandra/build/test/TESTS-TestSuites.xml to /var/folders/nq/4w83hn7s3h13dc5wxmvcdn9w0000gn/T/null1048031913 [junitreport] Loading stylesheet jar:file:/usr/local/ant/lib/ant-junit.jar!/org/apache/tools/ant/taskdefs/optional/junit/xsl/junit-frames.xsl [junitreport] Transform time: 256ms [junitreport] Deleting: /var/folders/nq/4w83hn7s3h13dc5wxmvcdn9w0000gn/T/null1048031913 {code} > Draining a node might fail to delete all inactive commitlogs > ------------------------------------------------------------ > > Key: CASSANDRA-13123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13123 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths > Reporter: Jan Urbański > Assignee: Jan Urbański > Fix For: 3.0.15, 3.11.1, 4.0 > > Attachments: 13123-2.2.8.txt, 13123-3.0.10.txt, 13123-3.9.txt, > 13123-trunk.txt > > > After issuing a drain command, it's possible that not all of the inactive > commitlogs are removed. > The drain command shuts down the CommitLog instance, which in turn shuts down > the CommitLogSegmentManager. This has the effect of discarding any pending > management tasks it might have, like the removal of inactive commitlogs. > This in turn leads to an excessive amount of commitlogs being left behind > after a drain and a lengthy recovery after a restart. With a fleet of dozens > of nodes, each of them leaving several GB of commitlogs after a drain and > taking up to two minutes to recover them on restart, the additional time > required to restart the entire fleet becomes noticeable. > This problem is not present in 3.x or trunk because of the CLSM rewrite done > in CASSANDRA-8844. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org