[jira] [Commented] (CASSANDRA-8515) Commit log stop policy not enforced correctly during startup
[
https://issues.apache.org/jira/browse/CASSANDRA-8515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14694492#comment-14694492
]
Paulo Motta commented on CASSANDRA-8515:
Unit tests are now available. 2.2 dtests worked fine. 3.0 and trunk dtests are
being ABORTED, but the same is happening with upstream branches. Unit tests
worked fine on Windows locally.
Since the changes are only related to unit tests, I think we can safely commit.
> Commit log stop policy not enforced correctly during startup
> -
>
> Key: CASSANDRA-8515
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8515
> Project: Cassandra
> Issue Type: Bug
>Reporter: Richard Low
>Assignee: Paulo Motta
>Priority: Minor
> Fix For: 2.1.9, 2.2.1, 3.0 beta 1
>
>
> If the commit log directory has no free space, Cassandra hangs on startup.
> The main thread is waiting:
> {code}
> "main" prio=9 tid=0x7fefe400f800 nid=0x1303 waiting on condition
> [0x00010b9c1000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0007dc8c5fc8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> org.apache.cassandra.db.commitlog.CommitLogAllocator.fetchSegment(CommitLogAllocator.java:137)
> at
> org.apache.cassandra.db.commitlog.CommitLog.activateNextSegment(CommitLog.java:299)
> at org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:53)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:360)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:339)
> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:211)
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:699)
> at
> org.apache.cassandra.cql3.QueryProcessor.processInternal(QueryProcessor.java:208)
> at
> org.apache.cassandra.db.SystemKeyspace.updateSchemaVersion(SystemKeyspace.java:390)
> - locked <0x0007de2f2ce0> (a java.lang.Class for
> org.apache.cassandra.db.SystemKeyspace)
> at org.apache.cassandra.config.Schema.updateVersion(Schema.java:384)
> at
> org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:532)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:270)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585)
> {code}
> but COMMIT-LOG-ALLOCATOR is RUNNABLE:
> {code}
> "COMMIT-LOG-ALLOCATOR" prio=9 tid=0x7fefe5402800 nid=0x7513 in
> Object.wait() [0x000118252000]
>java.lang.Thread.State: RUNNABLE
> at
> org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:116)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> but making no progress.
> This behaviour has changed since 1.2 (see CASSANDRA-5737).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8515) Commit log stop policy not enforced correctly during startup
[
https://issues.apache.org/jira/browse/CASSANDRA-8515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681465#comment-14681465
]
Benedict commented on CASSANDRA-8515:
-
[~pauloricardomg]: something funny seems to have happened during all of those
runs. I'm not sure what, but none of them completed successfully.
it's possible it's a CI issue, but if you could investigate that would be
appreciated.
> Commit log stop policy not enforced correctly during startup
> -
>
> Key: CASSANDRA-8515
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8515
> Project: Cassandra
> Issue Type: Bug
>Reporter: Richard Low
>Assignee: Paulo Motta
>Priority: Minor
> Fix For: 2.1.9, 2.2.1, 3.0 beta 1
>
>
> If the commit log directory has no free space, Cassandra hangs on startup.
> The main thread is waiting:
> {code}
> "main" prio=9 tid=0x7fefe400f800 nid=0x1303 waiting on condition
> [0x00010b9c1000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0007dc8c5fc8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> org.apache.cassandra.db.commitlog.CommitLogAllocator.fetchSegment(CommitLogAllocator.java:137)
> at
> org.apache.cassandra.db.commitlog.CommitLog.activateNextSegment(CommitLog.java:299)
> at org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:53)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:360)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:339)
> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:211)
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:699)
> at
> org.apache.cassandra.cql3.QueryProcessor.processInternal(QueryProcessor.java:208)
> at
> org.apache.cassandra.db.SystemKeyspace.updateSchemaVersion(SystemKeyspace.java:390)
> - locked <0x0007de2f2ce0> (a java.lang.Class for
> org.apache.cassandra.db.SystemKeyspace)
> at org.apache.cassandra.config.Schema.updateVersion(Schema.java:384)
> at
> org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:532)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:270)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585)
> {code}
> but COMMIT-LOG-ALLOCATOR is RUNNABLE:
> {code}
> "COMMIT-LOG-ALLOCATOR" prio=9 tid=0x7fefe5402800 nid=0x7513 in
> Object.wait() [0x000118252000]
>java.lang.Thread.State: RUNNABLE
> at
> org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:116)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> but making no progress.
> This behaviour has changed since 1.2 (see CASSANDRA-5737).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8515) Commit log stop policy not enforced correctly during startup
[
https://issues.apache.org/jira/browse/CASSANDRA-8515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680662#comment-14680662
]
Paulo Motta commented on CASSANDRA-8515:
Sorry for not including test links to all branches before.
[CASSANDRA-8992|https://issues.apache.org/jira/browse/CASSANDRA-8992] extracted
commit log failure policy tests from class {{CommitLogTest}} to
{{CommitLogFailurePolicyTests}} on 2.2+. Since this patch (CASSANDRA-8115) was
originally made on 2.1, it fixed existing tests on {{CommitLogTests}}, but when
that was merged up, the class {{CommitLogFailurePolicyTest}} remained unchanged
with broken tests.
One option would be to place all {{CommitLogFailurePolicyTest}} back on
{{CommitLogTests}}, but I opted for leaving them in a separate class (as is
right now), since they are specific tests only for the failure policies, and
not for overall commit log functioning.
Fix patch is available on the following links for review:
* [2.2
patch|https://github.com/apache/cassandra/compare/cassandra-2.2...pauloricardomg:8515-2.2-fix]
* [3.0
patch|https://github.com/apache/cassandra/compare/cassandra-3.0...pauloricardomg:8515-3.0-fix]
* [trunk
patch|https://github.com/apache/cassandra/compare/trunk...pauloricardomg:8515-trunk-fix]
Tests will be available soon on the following links:
* [2.2
dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-8515-2.2-fix-dtest/]
* [2.2
testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-8515-2.2-fix-testall/]
* [3.0
dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-8515-3.0-fix-dtest/]
* [3.0
testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-8515-3.0-fix-testall/]
* [trunk
dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-8515-trunk-fix-dtest/]
* [trunk
testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-8515-trunk-fix-testall/]
> Commit log stop policy not enforced correctly during startup
> -
>
> Key: CASSANDRA-8515
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8515
> Project: Cassandra
> Issue Type: Bug
>Reporter: Richard Low
>Assignee: Paulo Motta
>Priority: Minor
> Fix For: 2.1.9, 2.2.1, 3.0 beta 1
>
>
> If the commit log directory has no free space, Cassandra hangs on startup.
> The main thread is waiting:
> {code}
> "main" prio=9 tid=0x7fefe400f800 nid=0x1303 waiting on condition
> [0x00010b9c1000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0007dc8c5fc8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> org.apache.cassandra.db.commitlog.CommitLogAllocator.fetchSegment(CommitLogAllocator.java:137)
> at
> org.apache.cassandra.db.commitlog.CommitLog.activateNextSegment(CommitLog.java:299)
> at org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:53)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:360)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:339)
> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:211)
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:699)
> at
> org.apache.cassandra.cql3.QueryProcessor.processInternal(QueryProcessor.java:208)
> at
> org.apache.cassandra.db.SystemKeyspace.updateSchemaVersion(SystemKeyspace.java:390)
> - locked <0x0007de2f2ce0> (a java.lang.Class for
> org.apache.cassandra.db.SystemKeyspace)
> at org.apache.cassandra.config.Schema.updateVersion(Schema.java:384)
> at
> org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:532)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:270)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585)
> {code}
> but COMMIT-LOG-ALLOCATOR is RUNNABLE:
> {code}
> "COMMIT-LOG-ALLOCATOR" prio=9 tid=0x7fefe5402800 nid=0x7513 in
> Object.wait() [0x000118252000]
>java.lang.Thread.State: RUNNABLE
> at
> org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAll
[jira] [Commented] (CASSANDRA-8515) Commit log stop policy not enforced correctly during startup
[
https://issues.apache.org/jira/browse/CASSANDRA-8515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680582#comment-14680582
]
Benedict commented on CASSANDRA-8515:
-
Yep, although it's possible I merge upstream badly. Or perhaps that these tests
were not present in 2.1, since it was all pretty clean.
[for
reference|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-8515-2.1-testall/lastCompletedBuild/testReport/]
Still, definitely my bad for not confirming each of the upstream merges passed.
> Commit log stop policy not enforced correctly during startup
> -
>
> Key: CASSANDRA-8515
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8515
> Project: Cassandra
> Issue Type: Bug
>Reporter: Richard Low
>Assignee: Paulo Motta
>Priority: Minor
> Fix For: 2.1.9, 2.2.1, 3.0 beta 1
>
>
> If the commit log directory has no free space, Cassandra hangs on startup.
> The main thread is waiting:
> {code}
> "main" prio=9 tid=0x7fefe400f800 nid=0x1303 waiting on condition
> [0x00010b9c1000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0007dc8c5fc8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> org.apache.cassandra.db.commitlog.CommitLogAllocator.fetchSegment(CommitLogAllocator.java:137)
> at
> org.apache.cassandra.db.commitlog.CommitLog.activateNextSegment(CommitLog.java:299)
> at org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:53)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:360)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:339)
> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:211)
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:699)
> at
> org.apache.cassandra.cql3.QueryProcessor.processInternal(QueryProcessor.java:208)
> at
> org.apache.cassandra.db.SystemKeyspace.updateSchemaVersion(SystemKeyspace.java:390)
> - locked <0x0007de2f2ce0> (a java.lang.Class for
> org.apache.cassandra.db.SystemKeyspace)
> at org.apache.cassandra.config.Schema.updateVersion(Schema.java:384)
> at
> org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:532)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:270)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585)
> {code}
> but COMMIT-LOG-ALLOCATOR is RUNNABLE:
> {code}
> "COMMIT-LOG-ALLOCATOR" prio=9 tid=0x7fefe5402800 nid=0x7513 in
> Object.wait() [0x000118252000]
>java.lang.Thread.State: RUNNABLE
> at
> org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:116)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> but making no progress.
> This behaviour has changed since 1.2 (see CASSANDRA-5737).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8515) Commit log stop policy not enforced correctly during startup
[
https://issues.apache.org/jira/browse/CASSANDRA-8515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680551#comment-14680551
]
Aleksey Yeschenko commented on CASSANDRA-8515:
--
Breaks it on OS X as well.
Don't know if we had checked with cassci before committing, but I don't see a
link here.
Also, there is now overlap between {{CommitLogFailurePolicyTest}} and
{{CommitLogTest}}. It seems to me that the former can be removed entirely.
> Commit log stop policy not enforced correctly during startup
> -
>
> Key: CASSANDRA-8515
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8515
> Project: Cassandra
> Issue Type: Bug
>Reporter: Richard Low
>Assignee: Paulo Motta
>Priority: Minor
> Fix For: 2.1.9, 2.2.1, 3.0 beta 1
>
>
> If the commit log directory has no free space, Cassandra hangs on startup.
> The main thread is waiting:
> {code}
> "main" prio=9 tid=0x7fefe400f800 nid=0x1303 waiting on condition
> [0x00010b9c1000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0007dc8c5fc8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> org.apache.cassandra.db.commitlog.CommitLogAllocator.fetchSegment(CommitLogAllocator.java:137)
> at
> org.apache.cassandra.db.commitlog.CommitLog.activateNextSegment(CommitLog.java:299)
> at org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLog.(CommitLog.java:53)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:360)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:339)
> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:211)
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:699)
> at
> org.apache.cassandra.cql3.QueryProcessor.processInternal(QueryProcessor.java:208)
> at
> org.apache.cassandra.db.SystemKeyspace.updateSchemaVersion(SystemKeyspace.java:390)
> - locked <0x0007de2f2ce0> (a java.lang.Class for
> org.apache.cassandra.db.SystemKeyspace)
> at org.apache.cassandra.config.Schema.updateVersion(Schema.java:384)
> at
> org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:532)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:270)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585)
> {code}
> but COMMIT-LOG-ALLOCATOR is RUNNABLE:
> {code}
> "COMMIT-LOG-ALLOCATOR" prio=9 tid=0x7fefe5402800 nid=0x7513 in
> Object.wait() [0x000118252000]
>java.lang.Thread.State: RUNNABLE
> at
> org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:116)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> but making no progress.
> This behaviour has changed since 1.2 (see CASSANDRA-5737).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
