[jira] [Commented] (CASSANDRA-8515) Commit log stop policy not enforced correctly during startup
[ https://issues.apache.org/jira/browse/CASSANDRA-8515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680551#comment-14680551 ] Aleksey Yeschenko commented on CASSANDRA-8515: -- Breaks it on OS X as well. Don't know if we had checked with cassci before committing, but I don't see a link here. Also, there is now overlap between {{CommitLogFailurePolicyTest}} and {{CommitLogTest}}. It seems to me that the former can be removed entirely. > Commit log stop policy not enforced correctly during startup > - > > Key: CASSANDRA-8515 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8515 > Project: Cassandra > Issue Type: Bug >Reporter: Richard Low >Assignee: Paulo Motta >Priority: Minor > Fix For: 2.1.9, 2.2.1, 3.0 beta 1 > > > If the commit log directory has no free space, Cassandra hangs on startup. > The main thread is waiting: > {code} > "main" prio=9 tid=0x7fefe400f800 nid=0x1303 waiting on condition > [0x00010b9c1000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0007dc8c5fc8> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > org.apache.cassandra.db.commitlog.CommitLogAllocator.fetchSegment(CommitLogAllocator.java:137) > at > org.apache.cassandra.db.commitlog.CommitLog.activateNextSegment(CommitLog.java:299) > at org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:73) > at > org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:53) > at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:360) > at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:339) > at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:211) > at > org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:699) > at > org.apache.cassandra.cql3.QueryProcessor.processInternal(QueryProcessor.java:208) > at > org.apache.cassandra.db.SystemKeyspace.updateSchemaVersion(SystemKeyspace.java:390) > - locked <0x0007de2f2ce0> (a java.lang.Class for > org.apache.cassandra.db.SystemKeyspace) > at org.apache.cassandra.config.Schema.updateVersion(Schema.java:384) > at > org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:532) > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:270) > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496) > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585) > {code} > but COMMIT-LOG-ALLOCATOR is RUNNABLE: > {code} > "COMMIT-LOG-ALLOCATOR" prio=9 tid=0x7fefe5402800 nid=0x7513 in > Object.wait() [0x000118252000] >java.lang.Thread.State: RUNNABLE > at > org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:116) > at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > at java.lang.Thread.run(Thread.java:745) > {code} > but making no progress. > This behaviour has changed since 1.2 (see CASSANDRA-5737). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8515) Commit log stop policy not enforced correctly during startup
[ https://issues.apache.org/jira/browse/CASSANDRA-8515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680582#comment-14680582 ] Benedict commented on CASSANDRA-8515: - Yep, although it's possible I merge upstream badly. Or perhaps that these tests were not present in 2.1, since it was all pretty clean. [for reference|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-8515-2.1-testall/lastCompletedBuild/testReport/] Still, definitely my bad for not confirming each of the upstream merges passed. > Commit log stop policy not enforced correctly during startup > - > > Key: CASSANDRA-8515 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8515 > Project: Cassandra > Issue Type: Bug >Reporter: Richard Low >Assignee: Paulo Motta >Priority: Minor > Fix For: 2.1.9, 2.2.1, 3.0 beta 1 > > > If the commit log directory has no free space, Cassandra hangs on startup. > The main thread is waiting: > {code} > "main" prio=9 tid=0x7fefe400f800 nid=0x1303 waiting on condition > [0x00010b9c1000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0007dc8c5fc8> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > org.apache.cassandra.db.commitlog.CommitLogAllocator.fetchSegment(CommitLogAllocator.java:137) > at > org.apache.cassandra.db.commitlog.CommitLog.activateNextSegment(CommitLog.java:299) > at org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:73) > at > org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:53) > at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:360) > at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:339) > at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:211) > at > org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:699) > at > org.apache.cassandra.cql3.QueryProcessor.processInternal(QueryProcessor.java:208) > at > org.apache.cassandra.db.SystemKeyspace.updateSchemaVersion(SystemKeyspace.java:390) > - locked <0x0007de2f2ce0> (a java.lang.Class for > org.apache.cassandra.db.SystemKeyspace) > at org.apache.cassandra.config.Schema.updateVersion(Schema.java:384) > at > org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:532) > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:270) > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496) > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585) > {code} > but COMMIT-LOG-ALLOCATOR is RUNNABLE: > {code} > "COMMIT-LOG-ALLOCATOR" prio=9 tid=0x7fefe5402800 nid=0x7513 in > Object.wait() [0x000118252000] >java.lang.Thread.State: RUNNABLE > at > org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:116) > at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > at java.lang.Thread.run(Thread.java:745) > {code} > but making no progress. > This behaviour has changed since 1.2 (see CASSANDRA-5737). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8515) Commit log stop policy not enforced correctly during startup
[ https://issues.apache.org/jira/browse/CASSANDRA-8515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680662#comment-14680662 ] Paulo Motta commented on CASSANDRA-8515: Sorry for not including test links to all branches before. [CASSANDRA-8992|https://issues.apache.org/jira/browse/CASSANDRA-8992] extracted commit log failure policy tests from class {{CommitLogTest}} to {{CommitLogFailurePolicyTests}} on 2.2+. Since this patch (CASSANDRA-8115) was originally made on 2.1, it fixed existing tests on {{CommitLogTests}}, but when that was merged up, the class {{CommitLogFailurePolicyTest}} remained unchanged with broken tests. One option would be to place all {{CommitLogFailurePolicyTest}} back on {{CommitLogTests}}, but I opted for leaving them in a separate class (as is right now), since they are specific tests only for the failure policies, and not for overall commit log functioning. Fix patch is available on the following links for review: * [2.2 patch|https://github.com/apache/cassandra/compare/cassandra-2.2...pauloricardomg:8515-2.2-fix] * [3.0 patch|https://github.com/apache/cassandra/compare/cassandra-3.0...pauloricardomg:8515-3.0-fix] * [trunk patch|https://github.com/apache/cassandra/compare/trunk...pauloricardomg:8515-trunk-fix] Tests will be available soon on the following links: * [2.2 dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-8515-2.2-fix-dtest/] * [2.2 testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-8515-2.2-fix-testall/] * [3.0 dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-8515-3.0-fix-dtest/] * [3.0 testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-8515-3.0-fix-testall/] * [trunk dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-8515-trunk-fix-dtest/] * [trunk testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-8515-trunk-fix-testall/] > Commit log stop policy not enforced correctly during startup > - > > Key: CASSANDRA-8515 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8515 > Project: Cassandra > Issue Type: Bug >Reporter: Richard Low >Assignee: Paulo Motta >Priority: Minor > Fix For: 2.1.9, 2.2.1, 3.0 beta 1 > > > If the commit log directory has no free space, Cassandra hangs on startup. > The main thread is waiting: > {code} > "main" prio=9 tid=0x7fefe400f800 nid=0x1303 waiting on condition > [0x00010b9c1000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0007dc8c5fc8> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > org.apache.cassandra.db.commitlog.CommitLogAllocator.fetchSegment(CommitLogAllocator.java:137) > at > org.apache.cassandra.db.commitlog.CommitLog.activateNextSegment(CommitLog.java:299) > at org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:73) > at > org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:53) > at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:360) > at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:339) > at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:211) > at > org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:699) > at > org.apache.cassandra.cql3.QueryProcessor.processInternal(QueryProcessor.java:208) > at > org.apache.cassandra.db.SystemKeyspace.updateSchemaVersion(SystemKeyspace.java:390) > - locked <0x0007de2f2ce0> (a java.lang.Class for > org.apache.cassandra.db.SystemKeyspace) > at org.apache.cassandra.config.Schema.updateVersion(Schema.java:384) > at > org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:532) > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:270) > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496) > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585) > {code} > but COMMIT-LOG-ALLOCATOR is RUNNABLE: > {code} > "COMMIT-LOG-ALLOCATOR" prio=9 tid=0x7fefe5402800 nid=0x7513 in > Object.wait() [0x000118252000] >java.lang.Thread.State: RUNNABLE > at > org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAll
[jira] [Commented] (CASSANDRA-8515) Commit log stop policy not enforced correctly during startup
[ https://issues.apache.org/jira/browse/CASSANDRA-8515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681465#comment-14681465 ] Benedict commented on CASSANDRA-8515: - [~pauloricardomg]: something funny seems to have happened during all of those runs. I'm not sure what, but none of them completed successfully. it's possible it's a CI issue, but if you could investigate that would be appreciated. > Commit log stop policy not enforced correctly during startup > - > > Key: CASSANDRA-8515 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8515 > Project: Cassandra > Issue Type: Bug >Reporter: Richard Low >Assignee: Paulo Motta >Priority: Minor > Fix For: 2.1.9, 2.2.1, 3.0 beta 1 > > > If the commit log directory has no free space, Cassandra hangs on startup. > The main thread is waiting: > {code} > "main" prio=9 tid=0x7fefe400f800 nid=0x1303 waiting on condition > [0x00010b9c1000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0007dc8c5fc8> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > org.apache.cassandra.db.commitlog.CommitLogAllocator.fetchSegment(CommitLogAllocator.java:137) > at > org.apache.cassandra.db.commitlog.CommitLog.activateNextSegment(CommitLog.java:299) > at org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:73) > at > org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:53) > at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:360) > at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:339) > at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:211) > at > org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:699) > at > org.apache.cassandra.cql3.QueryProcessor.processInternal(QueryProcessor.java:208) > at > org.apache.cassandra.db.SystemKeyspace.updateSchemaVersion(SystemKeyspace.java:390) > - locked <0x0007de2f2ce0> (a java.lang.Class for > org.apache.cassandra.db.SystemKeyspace) > at org.apache.cassandra.config.Schema.updateVersion(Schema.java:384) > at > org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:532) > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:270) > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496) > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585) > {code} > but COMMIT-LOG-ALLOCATOR is RUNNABLE: > {code} > "COMMIT-LOG-ALLOCATOR" prio=9 tid=0x7fefe5402800 nid=0x7513 in > Object.wait() [0x000118252000] >java.lang.Thread.State: RUNNABLE > at > org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:116) > at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > at java.lang.Thread.run(Thread.java:745) > {code} > but making no progress. > This behaviour has changed since 1.2 (see CASSANDRA-5737). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8515) Commit log stop policy not enforced correctly during startup
[ https://issues.apache.org/jira/browse/CASSANDRA-8515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14694492#comment-14694492 ] Paulo Motta commented on CASSANDRA-8515: Unit tests are now available. 2.2 dtests worked fine. 3.0 and trunk dtests are being ABORTED, but the same is happening with upstream branches. Unit tests worked fine on Windows locally. Since the changes are only related to unit tests, I think we can safely commit. > Commit log stop policy not enforced correctly during startup > - > > Key: CASSANDRA-8515 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8515 > Project: Cassandra > Issue Type: Bug >Reporter: Richard Low >Assignee: Paulo Motta >Priority: Minor > Fix For: 2.1.9, 2.2.1, 3.0 beta 1 > > > If the commit log directory has no free space, Cassandra hangs on startup. > The main thread is waiting: > {code} > "main" prio=9 tid=0x7fefe400f800 nid=0x1303 waiting on condition > [0x00010b9c1000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0007dc8c5fc8> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) > at > org.apache.cassandra.db.commitlog.CommitLogAllocator.fetchSegment(CommitLogAllocator.java:137) > at > org.apache.cassandra.db.commitlog.CommitLog.activateNextSegment(CommitLog.java:299) > at org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:73) > at > org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:53) > at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:360) > at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:339) > at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:211) > at > org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:699) > at > org.apache.cassandra.cql3.QueryProcessor.processInternal(QueryProcessor.java:208) > at > org.apache.cassandra.db.SystemKeyspace.updateSchemaVersion(SystemKeyspace.java:390) > - locked <0x0007de2f2ce0> (a java.lang.Class for > org.apache.cassandra.db.SystemKeyspace) > at org.apache.cassandra.config.Schema.updateVersion(Schema.java:384) > at > org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:532) > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:270) > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496) > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585) > {code} > but COMMIT-LOG-ALLOCATOR is RUNNABLE: > {code} > "COMMIT-LOG-ALLOCATOR" prio=9 tid=0x7fefe5402800 nid=0x7513 in > Object.wait() [0x000118252000] >java.lang.Thread.State: RUNNABLE > at > org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:116) > at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > at java.lang.Thread.run(Thread.java:745) > {code} > but making no progress. > This behaviour has changed since 1.2 (see CASSANDRA-5737). -- This message was sent by Atlassian JIRA (v6.3.4#6332)