[jira] [Updated] (CASSANDRA-15438) MerkleTrees viriables renaming tree -> trees
[ https://issues.apache.org/jira/browse/CASSANDRA-15438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiangwang updated CASSANDRA-15438: -- Description: It's confusing that the virable tree is sometimes MerkleTrees instead of MerkleTree. I'd like to rename "MerkleTrees tree" to "MerkleTrees trees" to make the code easier to read. was:Rename "MerkleTrees tree" to "MerkleTrees trees" > MerkleTrees viriables renaming tree -> trees > > > Key: CASSANDRA-15438 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15438 > Project: Cassandra > Issue Type: Improvement >Reporter: xiangwang >Assignee: xiangwang >Priority: Normal > Attachments: 0001-Rename-MerkleTrees-tree-to-MerkleTrees-trees.patch > > > It's confusing that the virable tree is sometimes MerkleTrees instead of > MerkleTree. > I'd like to rename "MerkleTrees tree" to "MerkleTrees trees" to make the code > easier to read. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15722) Attach cluster type to the error message in cassandra-diff
[ https://issues.apache.org/jira/browse/CASSANDRA-15722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15722: Reviewers: Marcus Eriksson, Marcus Eriksson (was: Marcus Eriksson) Marcus Eriksson, Marcus Eriksson Status: Review In Progress (was: Patch Available) > Attach cluster type to the error message in cassandra-diff > -- > > Key: CASSANDRA-15722 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15722 > Project: Cassandra > Issue Type: Improvement > Components: Tool/diff >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The exceptions get logged in the cassandra-diff do not contain cluster type. > It is hard to tell whether the exceptions are caused by requesting the SOURCE > or TARGET cluster. > The error message could be more informative by including the cluster type. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15722) Attach cluster type to the error message in cassandra-diff
[ https://issues.apache.org/jira/browse/CASSANDRA-15722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15722: Status: Ready to Commit (was: Review In Progress) > Attach cluster type to the error message in cassandra-diff > -- > > Key: CASSANDRA-15722 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15722 > Project: Cassandra > Issue Type: Improvement > Components: Tool/diff >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The exceptions get logged in the cassandra-diff do not contain cluster type. > It is hard to tell whether the exceptions are caused by requesting the SOURCE > or TARGET cluster. > The error message could be more informative by including the cluster type. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15722) Attach cluster type to the error message in cassandra-diff
[ https://issues.apache.org/jira/browse/CASSANDRA-15722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15722: Fix Version/s: 4.0-alpha Source Control Link: https://github.com/apache/cassandra-diff/commit/c8670b0fdcfebd0edbf9e43ef3cc6d46cde36b0f Resolution: Fixed Status: Resolved (was: Ready to Commit) +1, merged, thanks! > Attach cluster type to the error message in cassandra-diff > -- > > Key: CASSANDRA-15722 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15722 > Project: Cassandra > Issue Type: Improvement > Components: Tool/diff >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-alpha > > Time Spent: 10m > Remaining Estimate: 0h > > The exceptions get logged in the cassandra-diff do not contain cluster type. > It is hard to tell whether the exceptions are caused by requesting the SOURCE > or TARGET cluster. > The error message could be more informative by including the cluster type. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15725) Add support for adding custom Verbs
Marcus Eriksson created CASSANDRA-15725: --- Summary: Add support for adding custom Verbs Key: CASSANDRA-15725 URL: https://issues.apache.org/jira/browse/CASSANDRA-15725 Project: Cassandra Issue Type: Improvement Reporter: Marcus Eriksson Assignee: Marcus Eriksson It should be possible to safely add custom/internal Verbs - without risking conflicts when new ones are added. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15667) StreamResultFuture check for completeness is inconsistent, leading to races
[
https://issues.apache.org/jira/browse/CASSANDRA-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076236#comment-17076236
]
Massimiliano Tomassi edited comment on CASSANDRA-15667 at 4/14/20, 11:06 AM:
-
||Pull Request||CI Links||
|[PR
4.0\|[https://github.com/maxtomassi/cassandra/pull/1]]|[Circle-CI\|[https://app.circleci.com/pipelines/github/maxtomassi/cassandra?branch=15667-4.0]]|
|[PR
3.0\|[https://github.com/maxtomassi/cassandra/pull/2|https://github.com/maxtomassi/cassandra/pull/1]]|[Circle-CI\|[https://app.circleci.com/pipelines/github/maxtomassi/cassandra?branch=15667-3.0|https://app.circleci.com/pipelines/github/maxtomassi/cassandra?branch=15667-4.0]]|
|[PR
3.11\|[https://github.com/maxtomassi/cassandra/pull/3|https://github.com/maxtomassi/cassandra/pull/1]]|[Circle-CI\|[https://app.circleci.com/pipelines/github/maxtomassi/cassandra?branch=15667-3.11|https://app.circleci.com/pipelines/github/maxtomassi/cassandra?branch=15667-4.0]]|
was (Author: maxtomassi):
PR: [https://github.com/maxtomassi/cassandra/pull/1]
CircleCI:
[https://app.circleci.com/pipelines/github/maxtomassi/cassandra?branch=15667-4.0]
(JVM dtests failed running)
> StreamResultFuture check for completeness is inconsistent, leading to races
> ---
>
> Key: CASSANDRA-15667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15667
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: Massimiliano Tomassi
>Priority: Normal
> Fix For: 4.0
>
> Attachments: log_bootstrap_resumable
>
>
> {{StreamResultFuture#maybeComplete()}} uses
> {{StreamCoordinator#hasActiveSessions()}} to determine if all sessions are
> completed, but then accesses each session state via
> {{StreamCoordinator#getAllSessionInfo()}}: this is inconsistent, as the
> former relies on the actual {{StreamSession}} state, while the latter on the
> {{SessionInfo}} state, and the two are concurrently updated with no
> coordination whatsoever.
> This leads to races, i.e. apparent in some dtest spurious failures, such as
> {{TestBootstrap.resumable_bootstrap_test}} in CASSANDRA-15614 cc
> [~e.dimitrova].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15667) StreamResultFuture check for completeness is inconsistent, leading to races
[
https://issues.apache.org/jira/browse/CASSANDRA-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076236#comment-17076236
]
Massimiliano Tomassi edited comment on CASSANDRA-15667 at 4/14/20, 11:16 AM:
-
||Pull Request||CI Links||
|[PR
4.0|https://github.com/maxtomassi/cassandra/pull/1]|[Circle-CI|https://app.circleci.com/pipelines/github/maxtomassi/cassandra?branch=15667-4.0]|
|[PR
3.0|https://github.com/maxtomassi/cassandra/pull/2|https://github.com/maxtomassi/cassandra/pull/12]|[Circle-CI|https://app.circleci.com/pipelines/github/maxtomassi/cassandra?branch=15667-3.0]|
|[PR
3.11|https://github.com/maxtomassi/cassandra/pull/3]|[Circle-CI|https://app.circleci.com/pipelines/github/maxtomassi/cassandra?branch=15667-3.11]|
was (Author: maxtomassi):
||Pull Request||CI Links||
|[PR
4.0\|[https://github.com/maxtomassi/cassandra/pull/1]]|[Circle-CI\|[https://app.circleci.com/pipelines/github/maxtomassi/cassandra?branch=15667-4.0]]|
|[PR
3.0\|[https://github.com/maxtomassi/cassandra/pull/2|https://github.com/maxtomassi/cassandra/pull/1]]|[Circle-CI\|[https://app.circleci.com/pipelines/github/maxtomassi/cassandra?branch=15667-3.0|https://app.circleci.com/pipelines/github/maxtomassi/cassandra?branch=15667-4.0]]|
|[PR
3.11\|[https://github.com/maxtomassi/cassandra/pull/3|https://github.com/maxtomassi/cassandra/pull/1]]|[Circle-CI\|[https://app.circleci.com/pipelines/github/maxtomassi/cassandra?branch=15667-3.11|https://app.circleci.com/pipelines/github/maxtomassi/cassandra?branch=15667-4.0]]|
> StreamResultFuture check for completeness is inconsistent, leading to races
> ---
>
> Key: CASSANDRA-15667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15667
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: Massimiliano Tomassi
>Priority: Normal
> Fix For: 4.0
>
> Attachments: log_bootstrap_resumable
>
>
> {{StreamResultFuture#maybeComplete()}} uses
> {{StreamCoordinator#hasActiveSessions()}} to determine if all sessions are
> completed, but then accesses each session state via
> {{StreamCoordinator#getAllSessionInfo()}}: this is inconsistent, as the
> former relies on the actual {{StreamSession}} state, while the latter on the
> {{SessionInfo}} state, and the two are concurrently updated with no
> coordination whatsoever.
> This leads to races, i.e. apparent in some dtest spurious failures, such as
> {{TestBootstrap.resumable_bootstrap_test}} in CASSANDRA-15614 cc
> [~e.dimitrova].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15667) StreamResultFuture check for completeness is inconsistent, leading to races
[
https://issues.apache.org/jira/browse/CASSANDRA-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083119#comment-17083119
]
Massimiliano Tomassi commented on CASSANDRA-15667:
--
[~jasonstack], sorry for the delay. I've updated my first comment with links to
PRs against 3.0 and 3.11 too.
> StreamResultFuture check for completeness is inconsistent, leading to races
> ---
>
> Key: CASSANDRA-15667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15667
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: Massimiliano Tomassi
>Priority: Normal
> Fix For: 4.0
>
> Attachments: log_bootstrap_resumable
>
>
> {{StreamResultFuture#maybeComplete()}} uses
> {{StreamCoordinator#hasActiveSessions()}} to determine if all sessions are
> completed, but then accesses each session state via
> {{StreamCoordinator#getAllSessionInfo()}}: this is inconsistent, as the
> former relies on the actual {{StreamSession}} state, while the latter on the
> {{SessionInfo}} state, and the two are concurrently updated with no
> coordination whatsoever.
> This leads to races, i.e. apparent in some dtest spurious failures, such as
> {{TestBootstrap.resumable_bootstrap_test}} in CASSANDRA-15614 cc
> [~e.dimitrova].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15726) buffer pool may NPE with concurrent release
ZhaoYang created CASSANDRA-15726:
Summary: buffer pool may NPE with concurrent release
Key: CASSANDRA-15726
URL: https://issues.apache.org/jira/browse/CASSANDRA-15726
Project: Cassandra
Issue Type: Bug
Reporter: ZhaoYang
Assignee: ZhaoYang
This can be reproduced by running {{LongBufferPoolTest}}, 1 out 5 runs..
{code:java}
java.lang.NullPointerException
at
org.apache.cassandra.utils.memory.BufferPool$Chunk.access$1300(BufferPool.java:836)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.lambda$remove$1(BufferPool.java:716)
at
org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.removeIf(BufferPool.java:460)
at
org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.access$1500(BufferPool.java:304)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.remove(BufferPool.java:716)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.put(BufferPool.java:590)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.recycle(BufferPool.java:709)
at
org.apache.cassandra.utils.memory.BufferPool$Chunk.recycle(BufferPool.java:909)
at
org.apache.cassandra.utils.memory.BufferPool$Chunk.tryRecycle(BufferPool.java:903)
at
org.apache.cassandra.utils.memory.BufferPool$Chunk.release(BufferPool.java:896)
at
org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.removeIf(BufferPool.java:465)
at
org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.access$1500(BufferPool.java:304)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.addChunk(BufferPool.java:736)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.addChunkFromParent(BufferPool.java:725)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.tryGetInternal(BufferPool.java:691)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.tryGet(BufferPool.java:679)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.access$000(BufferPool.java:518)
at
org.apache.cassandra.utils.memory.BufferPool.tryGet(BufferPool.java:120)
at
org.apache.cassandra.utils.memory.LongBufferPoolTest$2.testOne(LongBufferPoolTest.java:497)
at
org.apache.cassandra.utils.memory.LongBufferPoolTest$TestUntil.call(LongBufferPoolTest.java:558)
at
org.apache.cassandra.utils.memory.LongBufferPoolTest$TestUntil.call(LongBufferPoolTest.java:538)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
{code}
The cause is that:
* When evicting a normal chunk from a full MicroQueueOfChunks, local pool will
try to remove corresponding tiny chunks, via {{MicroQueueOfChunks#removeIf}}.
* If matching tiny chunk is found, tiny {{chunk.release()}} is called
immediately before moving null chunk to the back of the queue.
* Due to concurrent release from different threads, tiny {{chunk.release()}}
may cause its parent normal chunk, aka. the evicted chunk in #1, to be removed
from local pool and causes tiny pool to remove corresponding tiny chunks again
in {{LocalPool#remove()}}.
* In {{MicroQueueOfChunks#removeIf}}, due to previous in-progress
{{removeIf}}, it throws NPE as it violate MicroQueueOfChunks's assumption which
requires null chunks to be put at the back of queue.
| [patch|https://github.com/apache/cassandra/pull/537] |
[CI|https://circleci.com/workflow-run/a97317a0-ef21-4c01-9a97-82eaf28d7faf]|
The fix to put null chunks to the back of queue before releasing any chunks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15726) buffer pool may throw NPE with concurrent release due to in-progress tiny pool eviction
[
https://issues.apache.org/jira/browse/CASSANDRA-15726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang updated CASSANDRA-15726:
-
Summary: buffer pool may throw NPE with concurrent release due to
in-progress tiny pool eviction (was: buffer pool may NPE with concurrent
release)
> buffer pool may throw NPE with concurrent release due to in-progress tiny
> pool eviction
> ---
>
> Key: CASSANDRA-15726
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15726
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> This can be reproduced by running {{LongBufferPoolTest}}, 1 out 5 runs..
> {code:java}
> java.lang.NullPointerException
> at
> org.apache.cassandra.utils.memory.BufferPool$Chunk.access$1300(BufferPool.java:836)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.lambda$remove$1(BufferPool.java:716)
> at
> org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.removeIf(BufferPool.java:460)
> at
> org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.access$1500(BufferPool.java:304)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.remove(BufferPool.java:716)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.put(BufferPool.java:590)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.recycle(BufferPool.java:709)
> at
> org.apache.cassandra.utils.memory.BufferPool$Chunk.recycle(BufferPool.java:909)
> at
> org.apache.cassandra.utils.memory.BufferPool$Chunk.tryRecycle(BufferPool.java:903)
> at
> org.apache.cassandra.utils.memory.BufferPool$Chunk.release(BufferPool.java:896)
> at
> org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.removeIf(BufferPool.java:465)
> at
> org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.access$1500(BufferPool.java:304)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.addChunk(BufferPool.java:736)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.addChunkFromParent(BufferPool.java:725)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.tryGetInternal(BufferPool.java:691)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.tryGet(BufferPool.java:679)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.access$000(BufferPool.java:518)
> at
> org.apache.cassandra.utils.memory.BufferPool.tryGet(BufferPool.java:120)
>
> at
> org.apache.cassandra.utils.memory.LongBufferPoolTest$2.testOne(LongBufferPoolTest.java:497)
> at
> org.apache.cassandra.utils.memory.LongBufferPoolTest$TestUntil.call(LongBufferPoolTest.java:558)
> at
> org.apache.cassandra.utils.memory.LongBufferPoolTest$TestUntil.call(LongBufferPoolTest.java:538)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> The cause is that:
> * When evicting a normal chunk from a full MicroQueueOfChunks, local pool
> will try to remove corresponding tiny chunks, via
> {{MicroQueueOfChunks#removeIf}}.
> * If matching tiny chunk is found, tiny {{chunk.release()}} is called
> immediately before moving null chunk to the back of the queue.
> * Due to concurrent release from different threads, tiny {{chunk.release()}}
> may cause its parent normal chunk, aka. the evicted chunk in #1, to be
> removed from local pool and causes tiny pool to remove corresponding tiny
> chunks again in {{LocalPool#remove()}}.
> * In {{MicroQueueOfChunks#removeIf}}, due to previous in-progress
> {{removeIf}}, it throws NPE as it violate MicroQueueOfChunks's assumption
> which requires null chunks to be put at the back of queue.
>
> | [patch|https://github.com/apache/cassandra/pull/537] |
> [CI|https://circleci.com/workflow-run/a97317a0-ef21-4c01-9a97-82eaf28d7faf]|
> The fix to put null chunks to the back of queue before releasing any chunks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15667) StreamResultFuture check for completeness is inconsistent, leading to races
[
https://issues.apache.org/jira/browse/CASSANDRA-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083146#comment-17083146
]
Sergio Bossa commented on CASSANDRA-15667:
--
[~maxtomassi], [~e.dimitrova], thanks for your PRs. I'm +1 to both.
> StreamResultFuture check for completeness is inconsistent, leading to races
> ---
>
> Key: CASSANDRA-15667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15667
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: Massimiliano Tomassi
>Priority: Normal
> Fix For: 4.0
>
> Attachments: log_bootstrap_resumable
>
>
> {{StreamResultFuture#maybeComplete()}} uses
> {{StreamCoordinator#hasActiveSessions()}} to determine if all sessions are
> completed, but then accesses each session state via
> {{StreamCoordinator#getAllSessionInfo()}}: this is inconsistent, as the
> former relies on the actual {{StreamSession}} state, while the latter on the
> {{SessionInfo}} state, and the two are concurrently updated with no
> coordination whatsoever.
> This leads to races, i.e. apparent in some dtest spurious failures, such as
> {{TestBootstrap.resumable_bootstrap_test}} in CASSANDRA-15614 cc
> [~e.dimitrova].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15726) buffer pool may NPE with concurrent release
[
https://issues.apache.org/jira/browse/CASSANDRA-15726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang updated CASSANDRA-15726:
-
Bug Category: Parent values: Correctness(12982)
Complexity: Normal
Component/s: Legacy/Core
Discovered By: Unit Test
Fix Version/s: 4.0-alpha
Severity: Normal
Status: Open (was: Triage Needed)
[~aleksey] [~benedict] do you mind reviewing?
> buffer pool may NPE with concurrent release
> ---
>
> Key: CASSANDRA-15726
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15726
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> This can be reproduced by running {{LongBufferPoolTest}}, 1 out 5 runs..
> {code:java}
> java.lang.NullPointerException
> at
> org.apache.cassandra.utils.memory.BufferPool$Chunk.access$1300(BufferPool.java:836)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.lambda$remove$1(BufferPool.java:716)
> at
> org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.removeIf(BufferPool.java:460)
> at
> org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.access$1500(BufferPool.java:304)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.remove(BufferPool.java:716)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.put(BufferPool.java:590)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.recycle(BufferPool.java:709)
> at
> org.apache.cassandra.utils.memory.BufferPool$Chunk.recycle(BufferPool.java:909)
> at
> org.apache.cassandra.utils.memory.BufferPool$Chunk.tryRecycle(BufferPool.java:903)
> at
> org.apache.cassandra.utils.memory.BufferPool$Chunk.release(BufferPool.java:896)
> at
> org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.removeIf(BufferPool.java:465)
> at
> org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.access$1500(BufferPool.java:304)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.addChunk(BufferPool.java:736)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.addChunkFromParent(BufferPool.java:725)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.tryGetInternal(BufferPool.java:691)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.tryGet(BufferPool.java:679)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.access$000(BufferPool.java:518)
> at
> org.apache.cassandra.utils.memory.BufferPool.tryGet(BufferPool.java:120)
>
> at
> org.apache.cassandra.utils.memory.LongBufferPoolTest$2.testOne(LongBufferPoolTest.java:497)
> at
> org.apache.cassandra.utils.memory.LongBufferPoolTest$TestUntil.call(LongBufferPoolTest.java:558)
> at
> org.apache.cassandra.utils.memory.LongBufferPoolTest$TestUntil.call(LongBufferPoolTest.java:538)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> The cause is that:
> * When evicting a normal chunk from a full MicroQueueOfChunks, local pool
> will try to remove corresponding tiny chunks, via
> {{MicroQueueOfChunks#removeIf}}.
> * If matching tiny chunk is found, tiny {{chunk.release()}} is called
> immediately before moving null chunk to the back of the queue.
> * Due to concurrent release from different threads, tiny {{chunk.release()}}
> may cause its parent normal chunk, aka. the evicted chunk in #1, to be
> removed from local pool and causes tiny pool to remove corresponding tiny
> chunks again in {{LocalPool#remove()}}.
> * In {{MicroQueueOfChunks#removeIf}}, due to previous in-progress
> {{removeIf}}, it throws NPE as it violate MicroQueueOfChunks's assumption
> which requires null chunks to be put at the back of queue.
>
> | [patch|https://github.com/apache/cassandra/pull/537] |
> [CI|https://circleci.com/workflow-run/a97317a0-ef21-4c01-9a97-82eaf28d7faf]|
> The fix to put null chunks to the back of queue before releasing any chunks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15726) buffer pool may throw NPE with concurrent release due to in-progress tiny pool eviction
[
https://issues.apache.org/jira/browse/CASSANDRA-15726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang updated CASSANDRA-15726:
-
Test and Documentation Plan: With patch, cannot reproduce anymore on
{{LongBufferPoolTest}}
Status: Patch Available (was: Open)
> buffer pool may throw NPE with concurrent release due to in-progress tiny
> pool eviction
> ---
>
> Key: CASSANDRA-15726
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15726
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> This can be reproduced by running {{LongBufferPoolTest}}, 1 out 5 runs..
> {code:java}
> java.lang.NullPointerException
> at
> org.apache.cassandra.utils.memory.BufferPool$Chunk.access$1300(BufferPool.java:836)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.lambda$remove$1(BufferPool.java:716)
> at
> org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.removeIf(BufferPool.java:460)
> at
> org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.access$1500(BufferPool.java:304)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.remove(BufferPool.java:716)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.put(BufferPool.java:590)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.recycle(BufferPool.java:709)
> at
> org.apache.cassandra.utils.memory.BufferPool$Chunk.recycle(BufferPool.java:909)
> at
> org.apache.cassandra.utils.memory.BufferPool$Chunk.tryRecycle(BufferPool.java:903)
> at
> org.apache.cassandra.utils.memory.BufferPool$Chunk.release(BufferPool.java:896)
> at
> org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.removeIf(BufferPool.java:465)
> at
> org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.access$1500(BufferPool.java:304)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.addChunk(BufferPool.java:736)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.addChunkFromParent(BufferPool.java:725)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.tryGetInternal(BufferPool.java:691)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.tryGet(BufferPool.java:679)
> at
> org.apache.cassandra.utils.memory.BufferPool$LocalPool.access$000(BufferPool.java:518)
> at
> org.apache.cassandra.utils.memory.BufferPool.tryGet(BufferPool.java:120)
>
> at
> org.apache.cassandra.utils.memory.LongBufferPoolTest$2.testOne(LongBufferPoolTest.java:497)
> at
> org.apache.cassandra.utils.memory.LongBufferPoolTest$TestUntil.call(LongBufferPoolTest.java:558)
> at
> org.apache.cassandra.utils.memory.LongBufferPoolTest$TestUntil.call(LongBufferPoolTest.java:538)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> The cause is that:
> * When evicting a normal chunk from a full MicroQueueOfChunks, local pool
> will try to remove corresponding tiny chunks, via
> {{MicroQueueOfChunks#removeIf}}.
> * If matching tiny chunk is found, tiny {{chunk.release()}} is called
> immediately before moving null chunk to the back of the queue.
> * Due to concurrent release from different threads, tiny {{chunk.release()}}
> may cause its parent normal chunk, aka. the evicted chunk in #1, to be
> removed from local pool and causes tiny pool to remove corresponding tiny
> chunks again in {{LocalPool#remove()}}.
> * In {{MicroQueueOfChunks#removeIf}}, due to previous in-progress
> {{removeIf}}, it throws NPE as it violate MicroQueueOfChunks's assumption
> which requires null chunks to be put at the back of queue.
>
> | [patch|https://github.com/apache/cassandra/pull/537] |
> [CI|https://circleci.com/workflow-run/a97317a0-ef21-4c01-9a97-82eaf28d7faf]|
> The fix to put null chunks to the back of queue before releasing any chunks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15666) Race condition when completing stream sessions
[
https://issues.apache.org/jira/browse/CASSANDRA-15666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang updated CASSANDRA-15666:
-
Test and Documentation Plan:
Added interceptor to verify stream messages and state transition.
CI: [https://circleci.com/workflow-run/d7e2fdab-7c5e-4653-a6c2-07515c0eb205]
dtest failure "repair_test.py" is fixed in
[https://github.com/apache/cassandra-dtest/pull/63]
was:
Added interceptor to verify stream messages and state transition.
CI: [https://circleci.com/workflow-run/deca618c-1f9c-4e3b-b4f1-f0fb1a8aac7f]
dtest failure "repair_test.py" is fixed in
[https://github.com/apache/cassandra-dtest/pull/63]
> Race condition when completing stream sessions
> --
>
> Key: CASSANDRA-15666
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15666
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: ZhaoYang
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{StreamSession#prepareAsync()}} executes, as the name implies,
> asynchronously from the IO thread: this opens up for race conditions between
> the sending of the {{PrepareSynAckMessage}} and the call to
> {{StreamSession#maybeCompleted()}}. I.e., the following could happen:
> 1) Node A sends {{PrepareSynAckMessage}} from the {{prepareAsync()}} thread.
> 2) Node B receives it and starts streaming.
> 3) Node A receives the streamed file and sends {{ReceivedMessage}}.
> 4) At this point, if this was the only file to stream, both nodes are ready
> to close the session via {{maybeCompleted()}}, but:
> a) Node A will call it twice from both the IO thread and the thread at #1,
> closing the session and its channels.
> b) Node B will attempt to send a {{CompleteMessage}}, but will fail because
> the session has been closed in the meantime.
> There are other subtle variations of the pattern above, depending on the
> order of concurrently sent/received messages.
> I believe the best fix would be to modify the message exchange so that:
> 1) Only the "follower" is allowed to send the {{CompleteMessage}}.
> 2) Only the "initiator" is allowed to close the session and its channels
> after receiving the {{CompleteMessage}}.
> By doing so, the message exchange logic would be easier to reason about,
> which is overall a win anyway.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch master updated (3252755 -> 8b128ff)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git. from 3252755 Add Ubuntu 19.10 with JDK11 Image new cde3ecc Update .gitignore to ignore .idea new 8b128ff Update default Docker image used for testing to Ubuntu 19.10 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .gitignore| 1 + jenkins-dsl/cassandra_job_dsl_seed.groovy | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] 01/02: Update .gitignore to ignore .idea
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git commit cde3ecc3301ae0887917db2db05469225cd5c2da Author: Eduard Tudenhoefner AuthorDate: Thu Apr 9 12:28:17 2020 +0200 Update .gitignore to ignore .idea --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 7bb4cf8..5b88705 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1,2 @@ dist/cassandra* +.idea - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] 02/02: Update default Docker image used for testing to Ubuntu 19.10
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
commit 8b128ffbc1496a60ea174cfaca4b7a0eb1c41da0
Author: Eduard Tudenhoefner
AuthorDate: Thu Apr 9 12:28:49 2020 +0200
Update default Docker image used for testing to Ubuntu 19.10
---
jenkins-dsl/cassandra_job_dsl_seed.groovy | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index 26430d2..845ba4f 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -54,7 +54,7 @@ def dtestTargets = ['dtest', 'dtest-novnode',
'dtest-offheap', 'dtest-large']
if(binding.hasVariable("CASSANDRA_DTEST_TEST_TARGETS")) {
dtestTargets = "${CASSANDRA_DTEST_TEST_TARGETS}".split(",")
}
-def dtestDockerImage = 'spod/cassandra-testing-ubuntu18-java11'
+def dtestDockerImage = 'nastra/cassandra-testing-ubuntu1910-java11'
if(binding.hasVariable("CASSANDRA_DOCKER_IMAGE")) {
dtestDockerImage = "${CASSANDRA_DOCKER_IMAGE}"
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch master updated: added Docker image for Debian Buster
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/master by this push:
new 1f7f4ba added Docker image for Debian Buster
1f7f4ba is described below
commit 1f7f4baa37139d2d31284921305e7b7c77af8bbd
Author: Stefan Miklosovic
AuthorDate: Mon Apr 6 10:28:00 2020 +0200
added Docker image for Debian Buster
---
README.md | 8 +++-
docker/build-debs.sh | 25 +++--
docker/buster-image.docker | 42 ++
3 files changed, 72 insertions(+), 3 deletions(-)
diff --git a/README.md b/README.md
index 249037a..de88ae6 100644
--- a/README.md
+++ b/README.md
@@ -17,11 +17,17 @@
```docker build -t cass-build-rpms -f docker/centos7-image.docker docker/```
The image will contain a clone of the Apache git repository by default.
Using a different repository is possible by adding the `--build-arg
CASSANDRA_GIT_URL=https://github.com/myuser/cassandra.git` parameter. All
successive builds will be executed based on the repository cloned during docker
image creation.
2. Run build script through docker (specify branch, e.g. cassandra-3.0 and
version, e.g. 3.0.11):
- * Debian:
+ * Debian Jessie:
```docker run --rm -v `pwd`/dist:/dist `docker images -f
label=org.cassandra.buildenv=jessie -q` /home/build/build-debs.sh
```
+ * Debian Buster
+```docker run --rm -v `pwd`/dist:/dist `docker images -f
label=org.cassandra.buildenv=buster -q` /home/build/build-debs.sh
```
* RPM:
```docker run --rm -v `pwd`/dist:/dist `docker images -f
label=org.cassandra.buildenv=centos -q` /home/build/build-rpms.sh
```
+For the build by Debian Buster, you have the possibility to build Cassandra
either by Java 8 (default) or by Java 11. You control the Java version like
following. If you want to build with Java 8, just omit that last option.
+
+```docker run --rm -v `pwd`/dist:/dist `docker images -f
label=org.cassandra.buildenv=buster -q` /home/build/build-debs.sh
11```
+
You should find newly created Debian and RPM packages in the `dist` directory.
### Note about versioning
diff --git a/docker/build-debs.sh b/docker/build-debs.sh
index 7da1d97..5aeec0a 100755
--- a/docker/build-debs.sh
+++ b/docker/build-debs.sh
@@ -1,12 +1,25 @@
#!/bin/bash -x
set -e
-if [ "$#" -ne 1 ]; then
- echo "$0 "
+if [ "$#" -lt 1 ]; then
+ echo "$0 "
+ echo "if Java version is not set, it is set to 8 by default, choose from 8
or 11"
exit 1
fi
CASSANDRA_BRANCH=$1
+JAVA_VERSION=$2
+
+if [ "$JAVA_VERSION" = "" ]; then
+JAVA_VERSION=8
+fi
+
+regx_java_version="(8|11)"
+
+if [[ ! "$JAVA_VERSION" =~ $regx_java_version ]]; then
+ echo "Error: Java version is not set to 8 nor 11, it is set to
$JAVA_VERSION"
+ exit 1
+fi
cd $CASSANDRA_DIR
git fetch
@@ -76,6 +89,14 @@ if [ $buildxml_version != $git_version ]; then
exit 4
fi
+if [ $JAVA_VERSION = "11" ]; then
+ sudo update-java-alternatives --set java-1.11.0-openjdk-amd64
+ export CASSANDRA_USE_JDK11=true
+ echo "Cassandra will be built with Java 11"
+else
+ echo "Cassandra will be built with Java 8"
+fi
+
# Install build dependencies and build package
echo "y" | sudo mk-build-deps --install
dpkg-buildpackage -uc -us
diff --git a/docker/buster-image.docker b/docker/buster-image.docker
new file mode 100644
index 000..08106ab
--- /dev/null
+++ b/docker/buster-image.docker
@@ -0,0 +1,42 @@
+FROM debian:buster
+
+ENV DEB_DIST_DIR=/dist
+ENV BUILD_HOME=/home/build
+ENV CASSANDRA_DIR=$BUILD_HOME/cassandra
+
+LABEL org.cassandra.buildenv=buster
+
+VOLUME ${DEB_DIST_DIR}
+
+# install deps
+RUN apt-get update && apt-get -y install \
+ ant \
+ build-essential \
+ curl \
+ devscripts \
+ git \
+ sudo \
+ python-sphinx \
+ python-sphinx-rtd-theme
+
+RUN echo 'deb http://ftp.debian.org/debian stretch main' >>
/etc/apt/sources.list \
+&& apt-get update \
+&& apt-get install -y --no-install-recommends openjdk-8-jdk openjdk-11-jdk
\
+&& sed -i '$d' /etc/apt/sources.list \
+&& apt-get update \
+&& update-java-alternatives --set java-1.8.0-openjdk-amd64
+
+# create and change to build user
+RUN adduser --disabled-login --gecos build build && gpasswd -a build sudo
+RUN echo "build ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/build && \
+ chmod 0440 /etc/sudoers.d/build
+
+USER build
+
+# clone Cassandra and cache maven artifacts
+ARG CASSANDRA_GIT_URL=https://gitbox.apache.org/repos/asf/cassandra.git
+RUN git clone ${CASSANDRA_GIT_URL} ${CASSANDRA_DIR}
+WORKDIR ${CASSANDRA_DIR}
+RUN ant maven-ant-tasks-retrieve-build
+
+COPY build-debs.sh $BUILD_HOME/
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.or
[jira] [Updated] (CASSANDRA-15726) buffer pool may throw NPE with concurrent release due to in-progress tiny pool eviction
[
https://issues.apache.org/jira/browse/CASSANDRA-15726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang updated CASSANDRA-15726:
-
Description:
This can be reproduced by running {{LongBufferPoolTest}}, 1 out 5 runs..
{code:java}
java.lang.NullPointerException
at
org.apache.cassandra.utils.memory.BufferPool$Chunk.access$1300(BufferPool.java:836)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.lambda$remove$1(BufferPool.java:716)
at
org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.removeIf(BufferPool.java:460)
at
org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.access$1500(BufferPool.java:304)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.remove(BufferPool.java:716)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.put(BufferPool.java:590)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.recycle(BufferPool.java:709)
at
org.apache.cassandra.utils.memory.BufferPool$Chunk.recycle(BufferPool.java:909)
at
org.apache.cassandra.utils.memory.BufferPool$Chunk.tryRecycle(BufferPool.java:903)
at
org.apache.cassandra.utils.memory.BufferPool$Chunk.release(BufferPool.java:896)
at
org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.removeIf(BufferPool.java:465)
at
org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.access$1500(BufferPool.java:304)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.addChunk(BufferPool.java:736)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.addChunkFromParent(BufferPool.java:725)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.tryGetInternal(BufferPool.java:691)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.tryGet(BufferPool.java:679)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.access$000(BufferPool.java:518)
at
org.apache.cassandra.utils.memory.BufferPool.tryGet(BufferPool.java:120)
at
org.apache.cassandra.utils.memory.LongBufferPoolTest$2.testOne(LongBufferPoolTest.java:497)
at
org.apache.cassandra.utils.memory.LongBufferPoolTest$TestUntil.call(LongBufferPoolTest.java:558)
at
org.apache.cassandra.utils.memory.LongBufferPoolTest$TestUntil.call(LongBufferPoolTest.java:538)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
{code}
The cause is that:
* When evicting a normal chunk from a full MicroQueueOfChunks, local pool will
try to remove corresponding tiny chunks, via {{MicroQueueOfChunks#removeIf}}.
* If matching tiny chunk is found, tiny {{chunk.release()}} is called
immediately before moving null chunk to the back of the queue.
* Due to concurrent release from different threads, tiny {{chunk.release()}}
may cause its parent normal chunk, aka. the evicted chunk in #1, to be removed
from local pool and causes tiny pool to remove corresponding tiny chunks again
in {{LocalPool#remove()}}.
* In {{MicroQueueOfChunks#removeIf}}, due to previous in-progress
{{removeIf}}, it throws NPE as it violate MicroQueueOfChunks's assumption which
requires null chunks to be put at the back of queue.
|[patch|https://github.com/apache/cassandra/pull/537]|[CI|https://circleci.com/workflow-run/a97317a0-ef21-4c01-9a97-82eaf28d7faf]|
The fix is to put null chunks to the back of queue before releasing any chunks.
was:
This can be reproduced by running {{LongBufferPoolTest}}, 1 out 5 runs..
{code:java}
java.lang.NullPointerException
at
org.apache.cassandra.utils.memory.BufferPool$Chunk.access$1300(BufferPool.java:836)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.lambda$remove$1(BufferPool.java:716)
at
org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.removeIf(BufferPool.java:460)
at
org.apache.cassandra.utils.memory.BufferPool$MicroQueueOfChunks.access$1500(BufferPool.java:304)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.remove(BufferPool.java:716)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.put(BufferPool.java:590)
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.recycle(BufferPool.java:709)
at
org.apache.cassandra.utils.memory.BufferPool$Chunk.recycle(BufferPool.java:909)
at
org.apache.cassandra.utils.memory.BufferPool$Chunk.tryRecycle(BufferPool.java:903)
at
org.apache.cassandra.utils.memory.BufferPool$Chunk.release(BufferPool.java:896)
at
org.apache.cassandra.uti
[jira] [Commented] (CASSANDRA-15691) DOC - Review Testing section, update builds info to ci-cassandra.apache.org
[
https://issues.apache.org/jira/browse/CASSANDRA-15691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083234#comment-17083234
]
Michael Semb Wever commented on CASSANDRA-15691:
I think there's a few things that can be added/improved to the {{testing}} and
{{ci}} doc page now.
The testing page should only be about testing, not our CI systems. Therefore
the section about CircleCI should be moved to the CI page.
There is more to the unit tests than the long tests. Eg cdc, compression, burn,
stress, fqltool, long, cqlsh and jvm tests. The test page should reflect that.
Likewise there is more to the dtests now.
The Performance section needs an update too. Is cstar_perf still used by
anyone? tlp-stress and nosqlbench should be mentioned.
The CI page should not be only about Jenkins, so should be renamed to something
more appropriate, like "CI Environments".
http://cassandra.apache.org/doc/latest/development/ci.html
The patches page
(http://cassandra.apache.org/doc/latest/development/patches.html) should link
to the CI page, indicating how to include their use when submitting a patch to
a ticket.
> DOC - Review Testing section, update builds info to ci-cassandra.apache.org
> ---
>
> Key: CASSANDRA-15691
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15691
> Project: Cassandra
> Issue Type: Task
> Components: Documentation/Website
>Reporter: Erick Ramirez
>Priority: Normal
>
> h2. Background
> While discussing doc updates to CASSANDRA-15466 on [ASF
> Slack|https://the-asf.slack.com/archives/CK23JSY2K/p1585985233187200], I
> discovered that the
> [Testing|http://cassandra.apache.org/doc/latest/development/testing.html]
> page requires a review.
> h2. Sample section
> [~mck] advised that the linked CI server here:
> bq. Using dtests helps us to prevent regression bugs by continually executing
> tests on the [CI server|https://builds.apache.org/] against new patches.
> should be https://ci-cassandra.apache.org/.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15686) Improvements in circle CI default config
[
https://issues.apache.org/jira/browse/CASSANDRA-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083255#comment-17083255
]
Michael Semb Wever commented on CASSANDRA-15686:
Who introduced the {{get-cores}} and {{get-mem}} ant targets?
This indicates that the unit tests were runner-safe at some point in the past,
and have since been broken…
It would be nice to return this functionality (automatic runner calculation) to
build.xml in trunk.
Are the breakages just around a few code changes that has been introduced since
the {{get-cores}} and {{get-mem}} ant targets were added? for example running
multiple nodes on the one ipaddress but with different ports?
> Improvements in circle CI default config
>
>
> Key: CASSANDRA-15686
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15686
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Kevin Gallardo
>Priority: Normal
>
> I have been looking at and played around with the [default CircleCI
> config|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml],
> a few comments/questions regarding the following topics:
> * Python dtests do not run successfully (200-300 failures) on {{medium}}
> instances, they seem to only run with small flaky failures on {{large}}
> instances or higher
> * Python Upgrade tests:
> ** Do not seem to run without many failures on any instance types / any
> parallelism setting
> ** Do not seem to parallelize well, it seems each container is going to
> download multiple C* versions
> ** Additionally it seems the configuration is not up to date, as currently
> we get errors because {{JAVA8_HOME}} is not set
> * Unit tests do not seem to parallelize optimally, number of test runners do
> not reflect the available CPUs on the container. Ideally if # of runners == #
> of CPUs, build time is improved, on any type of instances.
> ** For instance when using the current configuration, running on medium
> instances, build will use 1 junit test runner, but 2 CPUs are available. If
> using 2 runners, the build time is reduced from 19min (at the current main
> config of parallelism=4) to 12min.
> * There are some typos in the file, some dtests say "Run Unit Tests" but
> they are JVM dtests (see
> [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1077],
>
> [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1386])
> So some ways to process these would be:
> * Do the Python dtests run successfully for anyone on {{medium}} instances?
> If not, would it make sense to bump them to {{large}} so that they can be run
> successfully?
> * Does anybody ever run the python upgrade tests on CircleCI and what is the
> configuration that makes it work?
> * Would it make sense to either hardcode the number of test runners in the
> unit tests with `-Dtest.runners` in the config file to reflect the number of
> CPUs on the instances, or change the build so that it is able to detect the
> appropriate number of core available automatically?
> Additionally, it seems this default config file (config.yml) is not as well
> maintained as the
> [{{config-2_1.yml}}|https://github.com/apache/cassandra/blob/trunk/.circleci/config-2_1.yml]
> (+its lowres/highres) version in the same folder (from CASSANDRA-14806).
> What is the reasoning for maintaining these 2 versions of the build? Could
> the better maintained version be used as the default? We could generate a
> lowres version of the new config-2_1.yml, and rename it {{config.yml}} so
> that it gets picked up by CircleCI automatically instead of the current
> default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15686) Improvements in circle CI default config
[
https://issues.apache.org/jira/browse/CASSANDRA-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083259#comment-17083259
]
Michael Semb Wever commented on CASSANDRA-15686:
bq. Who introduced the get-cores and get-mem ant targets?
See CASSANDRA-13078
> Improvements in circle CI default config
>
>
> Key: CASSANDRA-15686
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15686
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Kevin Gallardo
>Priority: Normal
>
> I have been looking at and played around with the [default CircleCI
> config|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml],
> a few comments/questions regarding the following topics:
> * Python dtests do not run successfully (200-300 failures) on {{medium}}
> instances, they seem to only run with small flaky failures on {{large}}
> instances or higher
> * Python Upgrade tests:
> ** Do not seem to run without many failures on any instance types / any
> parallelism setting
> ** Do not seem to parallelize well, it seems each container is going to
> download multiple C* versions
> ** Additionally it seems the configuration is not up to date, as currently
> we get errors because {{JAVA8_HOME}} is not set
> * Unit tests do not seem to parallelize optimally, number of test runners do
> not reflect the available CPUs on the container. Ideally if # of runners == #
> of CPUs, build time is improved, on any type of instances.
> ** For instance when using the current configuration, running on medium
> instances, build will use 1 junit test runner, but 2 CPUs are available. If
> using 2 runners, the build time is reduced from 19min (at the current main
> config of parallelism=4) to 12min.
> * There are some typos in the file, some dtests say "Run Unit Tests" but
> they are JVM dtests (see
> [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1077],
>
> [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1386])
> So some ways to process these would be:
> * Do the Python dtests run successfully for anyone on {{medium}} instances?
> If not, would it make sense to bump them to {{large}} so that they can be run
> successfully?
> * Does anybody ever run the python upgrade tests on CircleCI and what is the
> configuration that makes it work?
> * Would it make sense to either hardcode the number of test runners in the
> unit tests with `-Dtest.runners` in the config file to reflect the number of
> CPUs on the instances, or change the build so that it is able to detect the
> appropriate number of core available automatically?
> Additionally, it seems this default config file (config.yml) is not as well
> maintained as the
> [{{config-2_1.yml}}|https://github.com/apache/cassandra/blob/trunk/.circleci/config-2_1.yml]
> (+its lowres/highres) version in the same folder (from CASSANDRA-14806).
> What is the reasoning for maintaining these 2 versions of the build? Could
> the better maintained version be used as the default? We could generate a
> lowres version of the new config-2_1.yml, and rename it {{config.yml}} so
> that it gets picked up by CircleCI automatically instead of the current
> default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Improve logging around incremental repairs
This is an automated email from the ASF dual-hosted git repository.
marcuse pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 78c7853 Improve logging around incremental repairs
78c7853 is described below
commit 78c785367f2836960ae26e5e5ef258162ef30ec3
Author: Marcus Eriksson
AuthorDate: Tue Feb 25 14:10:22 2020 +0100
Improve logging around incremental repairs
Patch by marcuse; reviewed by Sam Tunnicliffe for CASSANDRA-15599
---
CHANGES.txt| 1 +
.../cassandra/db/compaction/CompactionManager.java | 28 --
.../db/compaction/PendingRepairManager.java| 2 +-
.../db/repair/CassandraValidationIterator.java | 15 +---
.../org/apache/cassandra/repair/RepairJob.java | 19 +--
.../cassandra/repair/RepairMessageVerbHandler.java | 7 +++---
.../apache/cassandra/repair/RepairRunnable.java| 4 ++--
.../org/apache/cassandra/repair/RepairSession.java | 3 ++-
.../consistent/CoordinatorMessagingTest.java | 1 -
9 files changed, 50 insertions(+), 30 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 38fef18..d125138 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0-alpha4
+ * Improve logging around incremental repair (CASSANDRA-15599)
* Do not check cdc_raw_directory filesystem space if CDC disabled
(CASSANDRA-15688)
* Replace array iterators with get by index (CASSANDRA-15394)
* Minimize BTree iterator allocations (CASSANDRA-15389)
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 7924a1f..2b5ffe6 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -1486,7 +1486,7 @@ public class CompactionManager implements
CompactionManagerMBean
BooleanSupplier isCancelled)
{
int originalCount = txn.originals().size();
-logger.info("Performing anticompaction on {} sstables", originalCount);
+logger.info("Performing anticompaction on {} sstables for {}",
originalCount, pendingRepair);
//Group SSTables
Set sstables = txn.originals();
@@ -1509,9 +1509,8 @@ public class CompactionManager implements
CompactionManagerMBean
antiCompactedSSTableCount += antiCompacted;
}
}
-
-String format = "Anticompaction completed successfully, anticompacted
from {} to {} sstable(s).";
-logger.info(format, originalCount, antiCompactedSSTableCount);
+String format = "Anticompaction completed successfully, anticompacted
from {} to {} sstable(s) for {}.";
+logger.info(format, originalCount, antiCompactedSSTableCount,
pendingRepair);
}
@VisibleForTesting
@@ -1537,7 +1536,7 @@ public class CompactionManager implements
CompactionManagerMBean
return 0;
}
-logger.info("Anticompacting {}", txn);
+logger.info("Anticompacting {} in {}.{} for {}", txn.originals(),
cfs.keyspace.getName(), cfs.getTableName(), pendingRepair);
Set sstableAsSet = txn.originals();
File destination =
cfs.getDirectories().getWriteableLocationAsFile(cfs.getExpectedCompactedFileSize(sstableAsSet,
OperationType.ANTICOMPACTION));
@@ -1627,8 +1626,6 @@ public class CompactionManager implements
CompactionManagerMBean
}
}
-List anticompactedSSTables = new ArrayList<>();
-
fullWriter.prepareToCommit();
transWriter.prepareToCommit();
unrepairedWriter.prepareToCommit();
@@ -1636,16 +1633,23 @@ public class CompactionManager implements
CompactionManagerMBean
txn.obsoleteOriginals();
txn.prepareToCommit();
-anticompactedSSTables.addAll(fullWriter.finished());
-anticompactedSSTables.addAll(transWriter.finished());
-anticompactedSSTables.addAll(unrepairedWriter.finished());
+List fullSSTables = new
ArrayList<>(fullWriter.finished());
+List transSSTables = new
ArrayList<>(transWriter.finished());
+List unrepairedSSTables = new
ArrayList<>(unrepairedWriter.finished());
fullWriter.commit();
transWriter.commit();
unrepairedWriter.commit();
txn.commit();
-
-return anticompactedSSTables.size();
+logger.info("Anticompacted {} in {}.{} to full = {}, transient =
{}, unrepaired = {} for {}",
+sstableAsSet,
+cfs.keyspace.getName(),
+cfs.getTableName(),
+fullSSTables,
+transSSTables,
+unrepairedSSTa
[jira] [Updated] (CASSANDRA-15599) Improve logging around incremental repair
[ https://issues.apache.org/jira/browse/CASSANDRA-15599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15599: Source Control Link: https://github.com/apache/cassandra/commit/78c785367f2836960ae26e5e5ef258162ef30ec3 Resolution: Fixed Status: Resolved (was: Ready to Commit) committed, thanks test runs; [unit tests|https://circleci.com/gh/krummas/cassandra/3148], [jvm dtests|https://circleci.com/gh/krummas/cassandra/3147], [dtests novnodes|https://circleci.com/gh/krummas/cassandra/3149], [dtests vnodes|https://circleci.com/gh/krummas/cassandra/3150] > Improve logging around incremental repair > - > > Key: CASSANDRA-15599 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15599 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Repair >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0-alpha > > > Some repair logging messages omit the parent session id and we don't log > which sstables are created by anticompaction. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15599) Improve logging around incremental repair
[ https://issues.apache.org/jira/browse/CASSANDRA-15599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15599: Reviewers: Sam Tunnicliffe (was: Blake Eggleston, Sam Tunnicliffe) > Improve logging around incremental repair > - > > Key: CASSANDRA-15599 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15599 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Repair >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0-alpha > > > Some repair logging messages omit the parent session id and we don't log > which sstables are created by anticompaction. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15718) Improve BatchMetricsTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083351#comment-17083351
]
Stephen Mallette commented on CASSANDRA-15718:
--
I've pushed some changes to my branch (link in description) addressing the
points that were made in the initial review. I converted to quick theories
rather than my custom use of {{Random}}. I wasn't quite sure how to tune the
parameters to the test exactly as running these tests over larger parameter
spaces adds to the total run time of the test. I ended up settling on
parameters that routinely ran in less than one to two seconds total for the
entire test class. If someone thinks there is value in tuning quick theories to
run more examples with wider parameters it's easy enough to change. For my own
purposes/testing, I've run these tests with significantly wider testing
parameters and they pass without issues.
As for this point:
> the batch has distinctPartitions mutations, so shouldn't max reflect that?
I spent a fair amount of time trying to understand exactly what was happening
there and I could still have it wrong but I think that the semantics of {{<=}}
for the assertion might have been correct, though the numbers I asserted in my
test were originally wrong. Given the underlying use of
{{DecayingEstimatedHistogramReservoir}} for the batch metrics it seems that
it's possible for the actual value recorded in the reservoir to be different
from the results returned from {{getMin()}} and {[getMax()}} so even though we
know the {{distinctPartitions}} I'm not sure that we know exactly what the
returned value might be. Of course, it's entirely possible that I've completely
misunderstood something here.
Looking forward to the next round of review on this and if some aspect of this
can be further improved.
> Improve BatchMetricsTest
> -
>
> Key: CASSANDRA-15718
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15718
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Stephen Mallette
>Assignee: Stephen Mallette
>Priority: Normal
>
> As noted in CASSANDRA-15582 {{BatchMetricsTest}} should test
> {{BatchStatement.Type.COUNTER}} to cover all the {{BatchMetrics}}. Some
> changes were introduced to make this improvement at:
> https://github.com/apache/cassandra/compare/trunk...spmallette:CASSANDRA-15582-trunk-batchmetrics
> and the following suggestions were made in review (in addition to the
> suggestion that a separate JIRA be created for this change) by [~dcapwell]:
> {quote}
> * I like the usage of BatchStatement.Type for the tests
> * honestly feel quick theories is better than random, but glad you added the
> seed to all asserts =). Would still be better as a quick theories test since
> you basically wrote a property anyways!
> *
> https://github.com/apache/cassandra/compare/trunk...spmallette:CASSANDRA-15582-trunk-batchmetrics#diff-8948cec1f9d33f10b15c38de80141548R131
> feel you should rename to expectedPartitionsPerLoggedBatch
> {Count,Logged,Unlogged}
> * . pre is what the value is, post is what the value is expected to be
> (rather than what it is).
> *
> *
> https://github.com/apache/cassandra/compare/trunk...spmallette:CASSANDRA-15582-trunk-batchmetrics#diff-8948cec1f9d33f10b15c38de80141548R150
> this doesn't look correct. the batch has distinctPartitions mutations, so
> shouldn't max reflect that? I ran the current test in a debugger and see that
> that is the case (aka current test is wrong).
> most of the comments are nit picks, but the last one looks like a test bug to
> me
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15724) Modify cassandra.yaml and replace the default data location, still data stored in default location
[
https://issues.apache.org/jira/browse/CASSANDRA-15724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083370#comment-17083370
]
Jon Meredith commented on CASSANDRA-15724:
--
Are you editing the correct configuration file? From the filenames/output above
it looks like you're running RedHat and perhaps eding the wrong file.
When the service is running, check the arguments of the running process with
something like {{ps auxwww | grep CassandraDaemon}} what does
{{-Dcassandra.config}} point to?
> Modify cassandra.yaml and replace the default data location, still data
> stored in default location
> --
>
> Key: CASSANDRA-15724
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15724
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Boopalan
>Priority: Normal
>
> I have created a separated directory (/mnt/) for cassandra data, commit log,
> hint and saved_cache. But data are not created on created directory, stored
> in default location /var/lib/cassandra/data.
>
> *Configuration File:*
> [root@node-master cassandra]# cat /etc/cassandra/default.conf/cassandra.yaml
> | grep /mnt/
> hints_directory: */mnt/*hints
> - */mnt/*data
> commitlog_directory: */mnt/*commitlog
> cdc_raw_directory: */mnt/*cdc_raw
> saved_caches_directory: */mnt/*saved_caches
>
> Please help me to resolve this issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15724) Modify cassandra.yaml and replace the default data location, still data stored in default location
[ https://issues.apache.org/jira/browse/CASSANDRA-15724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Meredith updated CASSANDRA-15724: - Status: Awaiting Feedback (was: Triage Needed) > Modify cassandra.yaml and replace the default data location, still data > stored in default location > -- > > Key: CASSANDRA-15724 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15724 > Project: Cassandra > Issue Type: Bug > Components: Local/SSTable >Reporter: Boopalan >Priority: Normal > > I have created a separated directory (/mnt/) for cassandra data, commit log, > hint and saved_cache. But data are not created on created directory, stored > in default location /var/lib/cassandra/data. > > *Configuration File:* > [root@node-master cassandra]# cat /etc/cassandra/default.conf/cassandra.yaml > | grep /mnt/ > hints_directory: */mnt/*hints > - */mnt/*data > commitlog_directory: */mnt/*commitlog > cdc_raw_directory: */mnt/*cdc_raw > saved_caches_directory: */mnt/*saved_caches > > Please help me to resolve this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15686) Improvements in circle CI default config
[
https://issues.apache.org/jira/browse/CASSANDRA-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083259#comment-17083259
]
Michael Semb Wever edited comment on CASSANDRA-15686 at 4/14/20, 4:18 PM:
--
bq. Who introduced the get-cores and get-mem ant targets?
See CASSANDRA-13078
bq. {code}org.apache.cassandra.exceptions.ConfigurationException:
127.0.0.1:7014 is in use by another process. Change
listen_address:storage_port in cassandra.yaml to values that do not conflict
with other services{code}
[~dcapwell], I've
[witnessed|https://ci-cassandra.apache.org/job/Cassandra-devbranch/37/testReport/junit/org.apache.cassandra.distributed.test/BootstrapTest/bootstrapTest/]
this in single runner test environments as well.
Inside the pipeline it is found
[here|https://ci-cassandra.apache.org/job/Cassandra-devbranch-jvm-dtest/28/testReport/junit/org.apache.cassandra.distributed.test/BootstrapTest/bootstrapTest/]
with the console
[here|https://ci-cassandra.apache.org/job/Cassandra-devbranch-jvm-dtest/28/console].
was (Author: michaelsembwever):
bq. Who introduced the get-cores and get-mem ant targets?
See CASSANDRA-13078
> Improvements in circle CI default config
>
>
> Key: CASSANDRA-15686
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15686
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Kevin Gallardo
>Priority: Normal
>
> I have been looking at and played around with the [default CircleCI
> config|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml],
> a few comments/questions regarding the following topics:
> * Python dtests do not run successfully (200-300 failures) on {{medium}}
> instances, they seem to only run with small flaky failures on {{large}}
> instances or higher
> * Python Upgrade tests:
> ** Do not seem to run without many failures on any instance types / any
> parallelism setting
> ** Do not seem to parallelize well, it seems each container is going to
> download multiple C* versions
> ** Additionally it seems the configuration is not up to date, as currently
> we get errors because {{JAVA8_HOME}} is not set
> * Unit tests do not seem to parallelize optimally, number of test runners do
> not reflect the available CPUs on the container. Ideally if # of runners == #
> of CPUs, build time is improved, on any type of instances.
> ** For instance when using the current configuration, running on medium
> instances, build will use 1 junit test runner, but 2 CPUs are available. If
> using 2 runners, the build time is reduced from 19min (at the current main
> config of parallelism=4) to 12min.
> * There are some typos in the file, some dtests say "Run Unit Tests" but
> they are JVM dtests (see
> [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1077],
>
> [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1386])
> So some ways to process these would be:
> * Do the Python dtests run successfully for anyone on {{medium}} instances?
> If not, would it make sense to bump them to {{large}} so that they can be run
> successfully?
> * Does anybody ever run the python upgrade tests on CircleCI and what is the
> configuration that makes it work?
> * Would it make sense to either hardcode the number of test runners in the
> unit tests with `-Dtest.runners` in the config file to reflect the number of
> CPUs on the instances, or change the build so that it is able to detect the
> appropriate number of core available automatically?
> Additionally, it seems this default config file (config.yml) is not as well
> maintained as the
> [{{config-2_1.yml}}|https://github.com/apache/cassandra/blob/trunk/.circleci/config-2_1.yml]
> (+its lowres/highres) version in the same folder (from CASSANDRA-14806).
> What is the reasoning for maintaining these 2 versions of the build? Could
> the better maintained version be used as the default? We could generate a
> lowres version of the new config-2_1.yml, and rename it {{config.yml}} so
> that it gets picked up by CircleCI automatically instead of the current
> default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14801) calculatePendingRanges no longer safe for multiple adjacent range movements
[
https://issues.apache.org/jira/browse/CASSANDRA-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083399#comment-17083399
]
Aleksandr Sorokoumov commented on CASSANDRA-14801:
--
[~fcofdezc] reviewed the PR and based on his suggestion I re-wrote the
randomized test using QuickTheories. As the run time of that test ended
comparable to the other unit tests, I included it in the suite as well
([a0246e|https://github.com/apache/cassandra/pull/495/commits/a0246e1e8ba481b98f76896e5005e9a8cc277586]).
> calculatePendingRanges no longer safe for multiple adjacent range movements
> ---
>
> Key: CASSANDRA-14801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14801
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Coordination, Legacy/Distributed Metadata
>Reporter: Benedict Elliott Smith
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0, 4.0-beta
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Correctness depended upon the narrowing to a {{Set}},
> which we no longer do - we maintain a collection of all {{Replica}}. Our
> {{RangesAtEndpoint}} collection built by {{getPendingRanges}} can as a result
> contain the same endpoint multiple times; and our {{EndpointsForToken}}
> obtained by {{TokenMetadata.pendingEndpointsFor}} may fail to be constructed,
> resulting in cluster-wide failures for writes to the affected token ranges
> for the duration of the range movement.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14801) calculatePendingRanges no longer safe for multiple adjacent range movements
[
https://issues.apache.org/jira/browse/CASSANDRA-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Sorokoumov updated CASSANDRA-14801:
-
Reviewers: Francisco Fernandez
> calculatePendingRanges no longer safe for multiple adjacent range movements
> ---
>
> Key: CASSANDRA-14801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14801
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Coordination, Legacy/Distributed Metadata
>Reporter: Benedict Elliott Smith
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0, 4.0-beta
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Correctness depended upon the narrowing to a {{Set}},
> which we no longer do - we maintain a collection of all {{Replica}}. Our
> {{RangesAtEndpoint}} collection built by {{getPendingRanges}} can as a result
> contain the same endpoint multiple times; and our {{EndpointsForToken}}
> obtained by {{TokenMetadata.pendingEndpointsFor}} may fail to be constructed,
> resulting in cluster-wide failures for writes to the affected token ranges
> for the duration of the range movement.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15674) liveDiskSpaceUsed and totalDiskSpaceUsed get corrupted if IndexSummaryRedistribution gets interrupted
[ https://issues.apache.org/jira/browse/CASSANDRA-15674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083438#comment-17083438 ] David Capwell commented on CASSANDRA-15674: --- bq. We should probably stick to consistent nomenclature, of commit/abort, rather than introduce commit/rollback? Fixed. > liveDiskSpaceUsed and totalDiskSpaceUsed get corrupted if > IndexSummaryRedistribution gets interrupted > - > > Key: CASSANDRA-15674 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15674 > Project: Cassandra > Issue Type: Bug > Components: Local/Compaction, Observability/Metrics >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-alpha > > > IndexSummaryRedistribution is a compaction task and as such extends Holder > and supports cancelation by throwing a CompactionInterruptedException. The > issue is that IndexSummaryRedistribution tries to use transactions, but > mutates the sstable in-place; transaction is unable to roll back. > This would be fine (only updates summary) if it wasn’t for the fact the task > attempts to also mutate the two metrics liveDiskSpaceUsed and > totalDiskSpaceUsed, since these can’t be rolled back any cancelation could > corrupt these metrics. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15724) Modify cassandra.yaml and replace the default data location, still data stored in default location
[ https://issues.apache.org/jira/browse/CASSANDRA-15724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083446#comment-17083446 ] Boopalan commented on CASSANDRA-15724: --- Thanks for your response [~jmeredithco] Below is the output: [root@node-master ~]# ps -aux | grep -i cassandra cassand+ 21207 0.6 14.5 12104404 9574560 ? SLl Mar28 151:17 /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/bin/java -Xloggc:/var/log/*cassandra*/gc.log -ea -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -XX:+HeapDumpOnOutOfMemoryError -Xss256k -XX:StringTableSize=103 -XX:+AlwaysPreTouch -XX:-UseBiasedLocking -XX:+UseTLAB -XX:+ResizeTLAB -XX:+UseNUMA -XX:+PerfDisableSharedMem -Djava.net.preferIPv4Stack=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSWaitDuration=1 -XX:+CMSParallelInitialMarkEnabled -XX:+CMSEdenChunksRecordAlways -XX:+CMSClassUnloadingEnabled -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -XX:+PrintPromotionFailure -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=10M -Xms8192M -Xmx8192M -Xmn2048M -XX:+UseCondCardMark -XX:CompileCommandFile=/etc/*cassandra*/conf/hotspot_compiler -javaagent:/usr/share/*cassandra*/lib/jamm-0.3.0.jar -D*cassandra*.jmx.local.port=7199 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.password.file=/etc/*cassandra*/jmxremote.password -Djava.library.path=/usr/share/*cassandra*/lib/sigar-bin -XX:OnOutOfMemoryError=kill -9 %p -Dlogback.configurationFile=logback.xml -D*cassandra*.logdir=/var/log/*cassandra* -D*cassandra*.storagedir= -D*cassandra*-pidfile=/var/run/*cassandra*/*cassandra*.pid -cp /etc/*cassandra*/conf:/usr/share/*cassandra*/lib/airline-0.6.jar:/usr/share/*cassandra*/lib/antlr-runtime-3.5.2.jar:/usr/share/*cassandra*/lib/asm-5.0.4.jar:/usr/share/*cassandra*/lib/caffeine-2.2.6.jar:/usr/share/*cassandra*/lib/*cassandra*-driver-core-3.0.1-shaded.jar:/usr/share/*cassandra*/lib/commons-cli-1.1.jar:/usr/share/*cassandra*/lib/commons-codec-1.9.jar:/usr/share/*cassandra*/lib/commons-lang3-3.1.jar:/usr/share/*cassandra*/lib/commons-math3-3.2.jar:/usr/share/*cassandra*/lib/compress-lzf-0.8.4.jar:/usr/share/*cassandra*/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/*cassandra*/lib/concurrent-trees-2.4.0.jar:/usr/share/*cassandra*/lib/disruptor-3.0.1.jar:/usr/share/*cassandra*/lib/ecj-4.4.2.jar:/usr/share/*cassandra*/lib/guava-18.0.jar:/usr/share/*cassandra*/lib/HdrHistogram-2.1.9.jar:/usr/share/*cassandra*/lib/high-scale-lib-1.0.6.jar:/usr/share/*cassandra*/lib/hppc-0.5.4.jar:/usr/share/*cassandra*/lib/jackson-core-asl-1.9.13.jar:/usr/share/*cassandra*/lib/jackson-mapper-asl-1.9.13.jar:/usr/share/*cassandra*/lib/jamm-0.3.0.jar:/usr/share/*cassandra*/lib/javax.inject.jar:/usr/share/*cassandra*/lib/jbcrypt-0.3m.jar:/usr/share/*cassandra*/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/*cassandra*/lib/jctools-core-1.2.1.jar:/usr/share/*cassandra*/lib/jflex-1.6.0.jar:/usr/share/*cassandra*/lib/jna-4.2.2.jar:/usr/share/*cassandra*/lib/joda-time-2.4.jar:/usr/share/*cassandra*/lib/json-simple-1.1.jar:/usr/share/*cassandra*/lib/jstackjunit-0.0.1.jar:/usr/share/*cassandra*/lib/libthrift-0.9.2.jar:/usr/share/*cassandra*/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/*cassandra*/lib/logback-classic-1.1.3.jar:/usr/share/*cassandra*/lib/logback-core-1.1.3.jar:/usr/share/*cassandra*/lib/lz4-1.3.0.jar:/usr/share/*cassandra*/lib/metrics-core-3.1.5.jar:/usr/share/*cassandra*/lib/metrics-jvm-3.1.5.jar:/usr/share/*cassandra*/lib/metrics-logback-3.1.5.jar:/usr/share/*cassandra*/lib/netty-all-4.0.44.Final.jar:/usr/share/*cassandra*/lib/ohc-core-0.4.4.jar:/usr/share/*cassandra*/lib/ohc-core-j8-0.4.4.jar:/usr/share/*cassandra*/lib/reporter-config3-3.0.3.jar:/usr/share/*cassandra*/lib/reporter-config-base-3.0.3.jar:/usr/share/*cassandra*/lib/sigar-1.6.4.jar:/usr/share/*cassandra*/lib/slf4j-api-1.7.7.jar:/usr/share/*cassandra*/lib/snakeyaml-1.11.jar:/usr/share/*cassandra*/lib/snappy-java-1.1.1.7.jar:/usr/share/*cassandra*/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/*cassandra*/lib/ST4-4.0.8.jar:/usr/share/*cassandra*/lib/stream-2.5.2.jar:/usr/share/*cassandra*/lib/thrift-server-0.3.7.jar:/usr/share/*cassandra*/apache-*cassandra*-3.11.6.jar:/usr/share/*cassandra*/apache-*cassandra*-thrift-3.11.6.jar:/usr/share/*cassandra*/stress.jar: org.apache.*cassandra*.service.*Cassandra*Daemon > Modify cassandra.yaml and replace the default data location, still data > stored in default location > -- > > Key: CASSANDRA-15724 > URL: https://issues.apache.org/jira/br
[jira] [Commented] (CASSANDRA-15724) Modify cassandra.yaml and replace the default data location, still data stored in default location
[
https://issues.apache.org/jira/browse/CASSANDRA-15724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083448#comment-17083448
]
Boopalan commented on CASSANDRA-15724:
---
{noformat}
ps auxwww | grep CassandraDaemon
cassand+ 21207 0.6 14.5 12104348 9576924 ? SLl Mar28 151:40
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/bin/java
-Xloggc:/var/log/cassandra/gc.log -ea -XX:+UseThreadPriorities
-XX:ThreadPriorityPolicy=42 -XX:+HeapDumpOnOutOfMemoryError -Xss256k
-XX:StringTableSize=103 -XX:+AlwaysPreTouch -XX:-UseBiasedLocking
-XX:+UseTLAB -XX:+ResizeTLAB -XX:+UseNUMA -XX:+PerfDisableSharedMem
-Djava.net.preferIPv4Stack=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1
-XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly
-XX:CMSWaitDuration=1 -XX:+CMSParallelInitialMarkEnabled
-XX:+CMSEdenChunksRecordAlways -XX:+CMSClassUnloadingEnabled
-XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintHeapAtGC
-XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime
-XX:+PrintPromotionFailure -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10
-XX:GCLogFileSize=10M -Xms8192M -Xmx8192M -Xmn2048M -XX:+UseCondCardMark
-XX:CompileCommandFile=/etc/cassandra/conf/hotspot_compiler
-javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar
-Dcassandra.jmx.local.port=7199
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password
-Djava.library.path=/usr/share/cassandra/lib/sigar-bin
-XX:OnOutOfMemoryError=kill -9 %p -Dlogback.configurationFile=logback.xml
-Dcassandra.logdir=/var/log/cassandra -Dcassandra.storagedir=
-Dcassandra-pidfile=/var/run/cassandra/cassandra.pid -cp
/etc/cassandra/conf:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.9.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.13.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.13.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jctools-core-1.2.1.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.2.2.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/jstackjunit-0.0.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.5.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.5.jar:/usr/share/cassandra/lib/metrics-logback-3.1.5.jar:/usr/share/cassandra/lib/netty-all-4.0.44.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.4.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.4.jar:/usr/share/cassandra/lib/reporter-config3-3.0.3.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.3.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.11.6.jar:/usr/share/cassandra/apache-cassandra-thrift-3.11.6.jar:/usr/share/cassandra/stress.jar:
org.apache.cassandra.service.CassandraDaemon
root 92798 0.0 0.0 112688 996 pts/2 S+ 22:36 0:00 grep
--color=auto CassandraDaemon {noformat}
> Modify cassandra.yaml and replace the default data location, still data
> stored in default location
> --
>
> Key: CASSANDRA-15724
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15724
> Project: Cassandra
> Issue Type: Bug
> Components: L
[jira] [Comment Edited] (CASSANDRA-15724) Modify cassandra.yaml and replace the default data location, still data stored in default location
[
https://issues.apache.org/jira/browse/CASSANDRA-15724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083446#comment-17083446
]
Boopalan edited comment on CASSANDRA-15724 at 4/14/20, 5:08 PM:
-
Thanks for your response [~jmeredithco]
Below is the output:
{noformat}
ps auxwww | grep CassandraDaemon
cassand+ 21207 0.6 14.5 12104348 9576924 ? SLl Mar28 151:40
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/bin/java
-Xloggc:/var/log/cassandra/gc.log -ea -XX:+UseThreadPriorities
-XX:ThreadPriorityPolicy=42 -XX:+HeapDumpOnOutOfMemoryError -Xss256k
-XX:StringTableSize=103 -XX:+AlwaysPreTouch -XX:-UseBiasedLocking
-XX:+UseTLAB -XX:+ResizeTLAB -XX:+UseNUMA -XX:+PerfDisableSharedMem
-Djava.net.preferIPv4Stack=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1
-XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly
-XX:CMSWaitDuration=1 -XX:+CMSParallelInitialMarkEnabled
-XX:+CMSEdenChunksRecordAlways -XX:+CMSClassUnloadingEnabled
-XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintHeapAtGC
-XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime
-XX:+PrintPromotionFailure -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10
-XX:GCLogFileSize=10M -Xms8192M -Xmx8192M -Xmn2048M -XX:+UseCondCardMark
-XX:CompileCommandFile=/etc/cassandra/conf/hotspot_compiler
-javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar
-Dcassandra.jmx.local.port=7199
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password
-Djava.library.path=/usr/share/cassandra/lib/sigar-bin
-XX:OnOutOfMemoryError=kill -9 %p -Dlogback.configurationFile=logback.xml
-Dcassandra.logdir=/var/log/cassandra -Dcassandra.storagedir=
-Dcassandra-pidfile=/var/run/cassandra/cassandra.pid -cp
/etc/cassandra/conf:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.9.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.13.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.13.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jctools-core-1.2.1.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.2.2.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/jstackjunit-0.0.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.5.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.5.jar:/usr/share/cassandra/lib/metrics-logback-3.1.5.jar:/usr/share/cassandra/lib/netty-all-4.0.44.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.4.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.4.jar:/usr/share/cassandra/lib/reporter-config3-3.0.3.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.3.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.11.6.jar:/usr/share/cassandra/apache-cassandra-thrift-3.11.6.jar:/usr/share/cassandra/stress.jar:
org.apache.cassandra.service.CassandraDaemon
root 92798 0.0 0.0 112688 996 pts/2 S+ 22:36 0:00 grep
--color=auto CassandraDaemon {noformat}
was (Author: boopalan.m):
Thanks for your response [~jmeredithco]
Below is the output:
[root@node-master ~]# ps -aux | grep -i cassandra
cassand+ 21207 0.6 14.5 12104404 9574560 ? SLl Mar28 151:17
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre/bin/java
-Xl
[jira] [Commented] (CASSANDRA-15686) Improvements in circle CI default config
[
https://issues.apache.org/jira/browse/CASSANDRA-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083462#comment-17083462
]
David Capwell commented on CASSANDRA-15686:
---
[~mck] your link is jvm dtest
(org.apache.cassandra.distributed.test.BootstrapTest.bootstrapTest) which would
require more work to support concurrent runners (doable). That test looks to
start a cluster, tear it down, then start another one; my guess is the second
one fails since the first isn't fully dead yet? This looks like a bug to me.
bq. This indicates that the unit tests were runner-safe at some point in the
past, and have since been broken…
https://issues.apache.org/jira/browse/CASSANDRA-13078?focusedCommentId=16039394&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16039394
bq. any thoughts on how this would interact with (for example) circleci or asf
jenkins? -- Jeff
bq. It's fine we would just override it for CircleCI so it's 1. For ASF Jenkins
it really depends on how big those boxes, but I am pretty sure we can pick a
number > 1. A quad-core box can handle 4 just fine. -- Ariel
Looks like Circle CI has always been 1 runner. They talk about updating
Jenkins but that didn't look to happen? If CI never ran with runners > 1 and
it was only locally a few times, then I wouldn't say that this is a regression
as failures are not consistent so plausible to not be seen the few times this
was used?
bq. It would be nice to return this functionality (automatic runner
calculation) to build.xml in trunk.
If builds get faster, then +1 from me.
bq. Are the breakages just around a few code changes that has been introduced
since the get-cores and get-mem ant targets were added? for example running
multiple nodes on the one ipaddress but with different ports?
org.apache.cassandra.net.ConnectionTest#doTestManual doesn't use random ports
so would conflict with anything trying to use the default port (not sure what
does that) but doesn't look bad to change that for that test, but we would also
need to know which test also did that.
Are there other tests? Not sure, but I wouldn't be opposed to changing to
concurrent runners and mark the failures as blockers for alpha (like we do
flaky tests)
> Improvements in circle CI default config
>
>
> Key: CASSANDRA-15686
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15686
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Kevin Gallardo
>Priority: Normal
>
> I have been looking at and played around with the [default CircleCI
> config|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml],
> a few comments/questions regarding the following topics:
> * Python dtests do not run successfully (200-300 failures) on {{medium}}
> instances, they seem to only run with small flaky failures on {{large}}
> instances or higher
> * Python Upgrade tests:
> ** Do not seem to run without many failures on any instance types / any
> parallelism setting
> ** Do not seem to parallelize well, it seems each container is going to
> download multiple C* versions
> ** Additionally it seems the configuration is not up to date, as currently
> we get errors because {{JAVA8_HOME}} is not set
> * Unit tests do not seem to parallelize optimally, number of test runners do
> not reflect the available CPUs on the container. Ideally if # of runners == #
> of CPUs, build time is improved, on any type of instances.
> ** For instance when using the current configuration, running on medium
> instances, build will use 1 junit test runner, but 2 CPUs are available. If
> using 2 runners, the build time is reduced from 19min (at the current main
> config of parallelism=4) to 12min.
> * There are some typos in the file, some dtests say "Run Unit Tests" but
> they are JVM dtests (see
> [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1077],
>
> [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1386])
> So some ways to process these would be:
> * Do the Python dtests run successfully for anyone on {{medium}} instances?
> If not, would it make sense to bump them to {{large}} so that they can be run
> successfully?
> * Does anybody ever run the python upgrade tests on CircleCI and what is the
> configuration that makes it work?
> * Would it make sense to either hardcode the number of test runners in the
> unit tests with `-Dtest.runners` in the config file to reflect the number of
> CPUs on the instances, or change the build so that it is able to detect the
> appropriate number of core available automatically?
> Additionally, it seems this default config file (config.yml) is not as well
> maintained as the
> [{{config-2_1.yml}}|https://github.
[jira] [Comment Edited] (CASSANDRA-15686) Improvements in circle CI default config
[
https://issues.apache.org/jira/browse/CASSANDRA-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083479#comment-17083479
]
Michael Semb Wever edited comment on CASSANDRA-15686 at 4/14/20, 6:06 PM:
--
bq. They talk about updating Jenkins but that didn't look to happen?
Multiple runners were introduced in Jenkins CI with CASSANDRA-13078.
We only recently undid it, in CASSANDRA-15651, when the code went from `ant
test` (and `ant test-all`) to testclasslist. The former targets depend on
`get-cores` and `get-mem, but the latter do not.
was (Author: michaelsembwever):
bq. They talk about updating Jenkins but that didn't look to happen?
Multiple runners were introduced in Jenkins CI with CASSANDRA-13078.
We only [recently undid it|CASSANDRA-15651] when the code went from `ant test`
(and `ant test-all`) to testclasslist. The former targets depend on `get-cores`
and `get-mem, but the latter do not.
> Improvements in circle CI default config
>
>
> Key: CASSANDRA-15686
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15686
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Kevin Gallardo
>Priority: Normal
>
> I have been looking at and played around with the [default CircleCI
> config|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml],
> a few comments/questions regarding the following topics:
> * Python dtests do not run successfully (200-300 failures) on {{medium}}
> instances, they seem to only run with small flaky failures on {{large}}
> instances or higher
> * Python Upgrade tests:
> ** Do not seem to run without many failures on any instance types / any
> parallelism setting
> ** Do not seem to parallelize well, it seems each container is going to
> download multiple C* versions
> ** Additionally it seems the configuration is not up to date, as currently
> we get errors because {{JAVA8_HOME}} is not set
> * Unit tests do not seem to parallelize optimally, number of test runners do
> not reflect the available CPUs on the container. Ideally if # of runners == #
> of CPUs, build time is improved, on any type of instances.
> ** For instance when using the current configuration, running on medium
> instances, build will use 1 junit test runner, but 2 CPUs are available. If
> using 2 runners, the build time is reduced from 19min (at the current main
> config of parallelism=4) to 12min.
> * There are some typos in the file, some dtests say "Run Unit Tests" but
> they are JVM dtests (see
> [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1077],
>
> [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1386])
> So some ways to process these would be:
> * Do the Python dtests run successfully for anyone on {{medium}} instances?
> If not, would it make sense to bump them to {{large}} so that they can be run
> successfully?
> * Does anybody ever run the python upgrade tests on CircleCI and what is the
> configuration that makes it work?
> * Would it make sense to either hardcode the number of test runners in the
> unit tests with `-Dtest.runners` in the config file to reflect the number of
> CPUs on the instances, or change the build so that it is able to detect the
> appropriate number of core available automatically?
> Additionally, it seems this default config file (config.yml) is not as well
> maintained as the
> [{{config-2_1.yml}}|https://github.com/apache/cassandra/blob/trunk/.circleci/config-2_1.yml]
> (+its lowres/highres) version in the same folder (from CASSANDRA-14806).
> What is the reasoning for maintaining these 2 versions of the build? Could
> the better maintained version be used as the default? We could generate a
> lowres version of the new config-2_1.yml, and rename it {{config.yml}} so
> that it gets picked up by CircleCI automatically instead of the current
> default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15686) Improvements in circle CI default config
[
https://issues.apache.org/jira/browse/CASSANDRA-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083479#comment-17083479
]
Michael Semb Wever commented on CASSANDRA-15686:
bq. They talk about updating Jenkins but that didn't look to happen?
Multiple runners were introduced in Jenkins CI with CASSANDRA-13078.
We only [recently undid it|CASSANDRA-15651] when the code went from `ant test`
(and `ant test-all`) to testclasslist. The former targets depend on `get-cores`
and `get-mem, but the latter do not.
> Improvements in circle CI default config
>
>
> Key: CASSANDRA-15686
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15686
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Kevin Gallardo
>Priority: Normal
>
> I have been looking at and played around with the [default CircleCI
> config|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml],
> a few comments/questions regarding the following topics:
> * Python dtests do not run successfully (200-300 failures) on {{medium}}
> instances, they seem to only run with small flaky failures on {{large}}
> instances or higher
> * Python Upgrade tests:
> ** Do not seem to run without many failures on any instance types / any
> parallelism setting
> ** Do not seem to parallelize well, it seems each container is going to
> download multiple C* versions
> ** Additionally it seems the configuration is not up to date, as currently
> we get errors because {{JAVA8_HOME}} is not set
> * Unit tests do not seem to parallelize optimally, number of test runners do
> not reflect the available CPUs on the container. Ideally if # of runners == #
> of CPUs, build time is improved, on any type of instances.
> ** For instance when using the current configuration, running on medium
> instances, build will use 1 junit test runner, but 2 CPUs are available. If
> using 2 runners, the build time is reduced from 19min (at the current main
> config of parallelism=4) to 12min.
> * There are some typos in the file, some dtests say "Run Unit Tests" but
> they are JVM dtests (see
> [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1077],
>
> [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1386])
> So some ways to process these would be:
> * Do the Python dtests run successfully for anyone on {{medium}} instances?
> If not, would it make sense to bump them to {{large}} so that they can be run
> successfully?
> * Does anybody ever run the python upgrade tests on CircleCI and what is the
> configuration that makes it work?
> * Would it make sense to either hardcode the number of test runners in the
> unit tests with `-Dtest.runners` in the config file to reflect the number of
> CPUs on the instances, or change the build so that it is able to detect the
> appropriate number of core available automatically?
> Additionally, it seems this default config file (config.yml) is not as well
> maintained as the
> [{{config-2_1.yml}}|https://github.com/apache/cassandra/blob/trunk/.circleci/config-2_1.yml]
> (+its lowres/highres) version in the same folder (from CASSANDRA-14806).
> What is the reasoning for maintaining these 2 versions of the build? Could
> the better maintained version be used as the default? We could generate a
> lowres version of the new config-2_1.yml, and rename it {{config.yml}} so
> that it gets picked up by CircleCI automatically instead of the current
> default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Exclude JNA from Chronicle's dependencies to avoid bringing wrong versions into consumers of Cassandra as a library
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 52a55d8 Exclude JNA from Chronicle's dependencies to avoid bringing wrong versions into consumers of Cassandra as a library 52a55d8 is described below commit 52a55d821a2c4352dcbca013386e3cb45e376eed Author: Mick Semb Wever AuthorDate: Wed Apr 8 13:03:51 2020 +0200 Exclude JNA from Chronicle's dependencies to avoid bringing wrong versions into consumers of Cassandra as a library patch by Doug Rohrer; reviewed Ryan Svihla, Jon Meredith, Mick Semb Wever for CASSANDRA-15647 --- build.xml | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/build.xml b/build.xml index f22f4a9..cdc3c64 100644 --- a/build.xml +++ b/build.xml @@ -592,7 +592,11 @@ - + + + + + - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15647) Mismatching dependencies between cassandra dist and cassandra-all pom
[ https://issues.apache.org/jira/browse/CASSANDRA-15647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15647: --- Status: Ready to Commit (was: Review In Progress) > Mismatching dependencies between cassandra dist and cassandra-all pom > - > > Key: CASSANDRA-15647 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15647 > Project: Cassandra > Issue Type: Bug > Components: Build, Dependencies >Reporter: Marvin Froeder >Assignee: Ryan Svihla >Priority: Normal > Fix For: 4.0-beta > > > I noticed that the cassandra distribution (tar.gz) dependencies doesn't match > the dependency list for the cassandra-all that is available at maven central. > Cassandra distribution only includes jna 4.2.2. > But, the maven dependency also include jna-platform 4.4.0 > Breakdown of relevant maven dependencies: > ``` > [INFO] +- org.apache.cassandra:cassandra-all:jar:4.0-alpha3:provided > [INFO] | +- net.java.dev.jna:jna:jar:4.2.2:provided > [INFO] | +- net.openhft:chronicle-threads:jar:1.16.0:provided > [INFO] | | \- net.openhft:affinity:jar:3.1.7:provided > [INFO] | | \- net.java.dev.jna:jna-platform:jar:4.4.0:provided > ``` > As you can see, jna is a direct dependency and jna-platform is a transitive > dependency from chronicle-threads. > I expected this issue to had been fixed by > https://github.com/apache/cassandra/pull/240/, but this change seem to have > being reverted, as no longer in trunk. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15647) Mismatching dependencies between cassandra dist and cassandra-all pom
[ https://issues.apache.org/jira/browse/CASSANDRA-15647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15647: --- Fix Version/s: (was: 4.0-beta) 4.0-alpha Since Version: 4.0-alpha Source Control Link: https://github.com/apache/cassandra/commit/52a55d821a2c4352dcbca013386e3cb45e376eed Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as 52a55d821a2c4352dcbca013386e3cb45e376eed > Mismatching dependencies between cassandra dist and cassandra-all pom > - > > Key: CASSANDRA-15647 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15647 > Project: Cassandra > Issue Type: Bug > Components: Build, Dependencies >Reporter: Marvin Froeder >Assignee: Ryan Svihla >Priority: Normal > Fix For: 4.0-alpha > > > I noticed that the cassandra distribution (tar.gz) dependencies doesn't match > the dependency list for the cassandra-all that is available at maven central. > Cassandra distribution only includes jna 4.2.2. > But, the maven dependency also include jna-platform 4.4.0 > Breakdown of relevant maven dependencies: > ``` > [INFO] +- org.apache.cassandra:cassandra-all:jar:4.0-alpha3:provided > [INFO] | +- net.java.dev.jna:jna:jar:4.2.2:provided > [INFO] | +- net.openhft:chronicle-threads:jar:1.16.0:provided > [INFO] | | \- net.openhft:affinity:jar:3.1.7:provided > [INFO] | | \- net.java.dev.jna:jna-platform:jar:4.4.0:provided > ``` > As you can see, jna is a direct dependency and jna-platform is a transitive > dependency from chronicle-threads. > I expected this issue to had been fixed by > https://github.com/apache/cassandra/pull/240/, but this change seem to have > being reverted, as no longer in trunk. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15686) Improvements in circle CI default config
[
https://issues.apache.org/jira/browse/CASSANDRA-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083494#comment-17083494
]
David Capwell commented on CASSANDRA-15686:
---
Ah so was used very recently. Interesting.
> Improvements in circle CI default config
>
>
> Key: CASSANDRA-15686
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15686
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Kevin Gallardo
>Priority: Normal
>
> I have been looking at and played around with the [default CircleCI
> config|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml],
> a few comments/questions regarding the following topics:
> * Python dtests do not run successfully (200-300 failures) on {{medium}}
> instances, they seem to only run with small flaky failures on {{large}}
> instances or higher
> * Python Upgrade tests:
> ** Do not seem to run without many failures on any instance types / any
> parallelism setting
> ** Do not seem to parallelize well, it seems each container is going to
> download multiple C* versions
> ** Additionally it seems the configuration is not up to date, as currently
> we get errors because {{JAVA8_HOME}} is not set
> * Unit tests do not seem to parallelize optimally, number of test runners do
> not reflect the available CPUs on the container. Ideally if # of runners == #
> of CPUs, build time is improved, on any type of instances.
> ** For instance when using the current configuration, running on medium
> instances, build will use 1 junit test runner, but 2 CPUs are available. If
> using 2 runners, the build time is reduced from 19min (at the current main
> config of parallelism=4) to 12min.
> * There are some typos in the file, some dtests say "Run Unit Tests" but
> they are JVM dtests (see
> [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1077],
>
> [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1386])
> So some ways to process these would be:
> * Do the Python dtests run successfully for anyone on {{medium}} instances?
> If not, would it make sense to bump them to {{large}} so that they can be run
> successfully?
> * Does anybody ever run the python upgrade tests on CircleCI and what is the
> configuration that makes it work?
> * Would it make sense to either hardcode the number of test runners in the
> unit tests with `-Dtest.runners` in the config file to reflect the number of
> CPUs on the instances, or change the build so that it is able to detect the
> appropriate number of core available automatically?
> Additionally, it seems this default config file (config.yml) is not as well
> maintained as the
> [{{config-2_1.yml}}|https://github.com/apache/cassandra/blob/trunk/.circleci/config-2_1.yml]
> (+its lowres/highres) version in the same folder (from CASSANDRA-14806).
> What is the reasoning for maintaining these 2 versions of the build? Could
> the better maintained version be used as the default? We could generate a
> lowres version of the new config-2_1.yml, and rename it {{config.yml}} so
> that it gets picked up by CircleCI automatically instead of the current
> default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15659) Better support of Python 3 for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-15659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15659: --- Status: Ready to Commit (was: Review In Progress) > Better support of Python 3 for cqlsh > > > Key: CASSANDRA-15659 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15659 > Project: Cassandra > Issue Type: Task > Components: Tool/cqlsh >Reporter: Stefan Miklosovic >Assignee: Eduard Tudenhoefner >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-alpha > > Time Spent: 40m > Remaining Estimate: 0h > > h2. From mailing list: > [https://lists.apache.org/thread.html/r377099b632c62b641e4feef5b738084fc5369b0c7157fae867853597%40%3Cdev.cassandra.apache.org%3E] > > As of today (24/3/2020) and current trunk, there is Python 3.6 supported (1) > but there is not any 3.6 version ootb in Debian for example. E.g. Buster has > Python 3.7 and other (recent) releases have version 2.7. This means that if > one wants to use Python 3 in Debian, he has to use 3.6 but it is not in the > repository so he has to download / compile / install it on his own. > There should be some sane Python 3 version supported which is as well present > in Debian repository (or requirement to run with 3.6 should be relaxed) . > (1) > [https://github.com/apache/cassandra/blob/bf9a1d487b9ba469e8d740cf7d1cd419535a7e79/bin/cqlsh#L57-L65] > h2. Summary of work that was done: > I relaxed the requirement of *cqlsh* only working with Python 2.7 & 3.6 by > allowing Python 3.6+. > Note that I left the constraint for Python 3.6 being the minimum Python3 > version. > As [~ptbannister] pointed out, we could remove the Python 3.6 min version > once we remove Python 2.7 support, as otherwise testing with lots of > different Python versions will get costly. > 2 Dockerfiles were added in *pylib* for minimal local testing of *cqlsh* > starting up with Python 3.7 & 3.8 and that both revealed > CASSANDRA-15572 and CASSANDRA-15573. > CASSANDRA-15572 was fixed here as it was a one-liner. And I'm going to > tackle CASSANDRA-15573 later. > Python 3.8 testing was added to the CircleCI config so that we can actually > see what else breaks with newer Python versions. > A new Docker images with Ubuntu 19.10 was required for testing > ([https://github.com/apache/cassandra-builds/pull/17]). This docker image > sets up Python 2.7/3.6/3.7/3.8 with their respective virtual environments, > which are then being used by the CircleCI yaml. > The image *spod/cassandra-testing-ubuntu1810-java11-w-dependencies:20190306* > couldn't be updated unfortunately because it can't be built anymore, due to > Ubuntu 18.10 being EOL. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15659) Better support of Python 3 for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-15659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15659: --- Source Control Link: https://github.com/apache/cassandra/commit/c05443c0980cb51720ba0503f26f084c1538729c Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as c05443c0980cb51720ba0503f26f084c1538729c > Better support of Python 3 for cqlsh > > > Key: CASSANDRA-15659 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15659 > Project: Cassandra > Issue Type: Task > Components: Tool/cqlsh >Reporter: Stefan Miklosovic >Assignee: Eduard Tudenhoefner >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-alpha > > Time Spent: 40m > Remaining Estimate: 0h > > h2. From mailing list: > [https://lists.apache.org/thread.html/r377099b632c62b641e4feef5b738084fc5369b0c7157fae867853597%40%3Cdev.cassandra.apache.org%3E] > > As of today (24/3/2020) and current trunk, there is Python 3.6 supported (1) > but there is not any 3.6 version ootb in Debian for example. E.g. Buster has > Python 3.7 and other (recent) releases have version 2.7. This means that if > one wants to use Python 3 in Debian, he has to use 3.6 but it is not in the > repository so he has to download / compile / install it on his own. > There should be some sane Python 3 version supported which is as well present > in Debian repository (or requirement to run with 3.6 should be relaxed) . > (1) > [https://github.com/apache/cassandra/blob/bf9a1d487b9ba469e8d740cf7d1cd419535a7e79/bin/cqlsh#L57-L65] > h2. Summary of work that was done: > I relaxed the requirement of *cqlsh* only working with Python 2.7 & 3.6 by > allowing Python 3.6+. > Note that I left the constraint for Python 3.6 being the minimum Python3 > version. > As [~ptbannister] pointed out, we could remove the Python 3.6 min version > once we remove Python 2.7 support, as otherwise testing with lots of > different Python versions will get costly. > 2 Dockerfiles were added in *pylib* for minimal local testing of *cqlsh* > starting up with Python 3.7 & 3.8 and that both revealed > CASSANDRA-15572 and CASSANDRA-15573. > CASSANDRA-15572 was fixed here as it was a one-liner. And I'm going to > tackle CASSANDRA-15573 later. > Python 3.8 testing was added to the CircleCI config so that we can actually > see what else breaks with newer Python versions. > A new Docker images with Ubuntu 19.10 was required for testing > ([https://github.com/apache/cassandra-builds/pull/17]). This docker image > sets up Python 2.7/3.6/3.7/3.8 with their respective virtual environments, > which are then being used by the CircleCI yaml. > The image *spod/cassandra-testing-ubuntu1810-java11-w-dependencies:20190306* > couldn't be updated unfortunately because it can't be built anymore, due to > Ubuntu 18.10 being EOL. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15659) Better support of Python 3 for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-15659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083537#comment-17083537 ] Michael Semb Wever commented on CASSANDRA-15659: [~yukim], we didn't exactly honour your request of keeping this ticket as an epic here, and only working with sub-tasks. Do you have any other tickets in mind? We can at least keep linking them back to this, at least during the 4.0 stabilisation phase. > Better support of Python 3 for cqlsh > > > Key: CASSANDRA-15659 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15659 > Project: Cassandra > Issue Type: Task > Components: Tool/cqlsh >Reporter: Stefan Miklosovic >Assignee: Eduard Tudenhoefner >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-alpha > > Time Spent: 40m > Remaining Estimate: 0h > > h2. From mailing list: > [https://lists.apache.org/thread.html/r377099b632c62b641e4feef5b738084fc5369b0c7157fae867853597%40%3Cdev.cassandra.apache.org%3E] > > As of today (24/3/2020) and current trunk, there is Python 3.6 supported (1) > but there is not any 3.6 version ootb in Debian for example. E.g. Buster has > Python 3.7 and other (recent) releases have version 2.7. This means that if > one wants to use Python 3 in Debian, he has to use 3.6 but it is not in the > repository so he has to download / compile / install it on his own. > There should be some sane Python 3 version supported which is as well present > in Debian repository (or requirement to run with 3.6 should be relaxed) . > (1) > [https://github.com/apache/cassandra/blob/bf9a1d487b9ba469e8d740cf7d1cd419535a7e79/bin/cqlsh#L57-L65] > h2. Summary of work that was done: > I relaxed the requirement of *cqlsh* only working with Python 2.7 & 3.6 by > allowing Python 3.6+. > Note that I left the constraint for Python 3.6 being the minimum Python3 > version. > As [~ptbannister] pointed out, we could remove the Python 3.6 min version > once we remove Python 2.7 support, as otherwise testing with lots of > different Python versions will get costly. > 2 Dockerfiles were added in *pylib* for minimal local testing of *cqlsh* > starting up with Python 3.7 & 3.8 and that both revealed > CASSANDRA-15572 and CASSANDRA-15573. > CASSANDRA-15572 was fixed here as it was a one-liner. And I'm going to > tackle CASSANDRA-15573 later. > Python 3.8 testing was added to the CircleCI config so that we can actually > see what else breaks with newer Python versions. > A new Docker images with Ubuntu 19.10 was required for testing > ([https://github.com/apache/cassandra-builds/pull/17]). This docker image > sets up Python 2.7/3.6/3.7/3.8 with their respective virtual environments, > which are then being used by the CircleCI yaml. > The image *spod/cassandra-testing-ubuntu1810-java11-w-dependencies:20190306* > couldn't be updated unfortunately because it can't be built anymore, due to > Ubuntu 18.10 being EOL. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15581) 4.0 quality testing: Compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-15581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083538#comment-17083538 ] Paulo Motta commented on CASSANDRA-15581: - Just a heads up that I ended up getting dragged into something else and will not be able to take this. Anyone willing to work on this feel free to take it. > 4.0 quality testing: Compaction > --- > > Key: CASSANDRA-15581 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15581 > Project: Cassandra > Issue Type: Task > Components: Test/dtest >Reporter: Josh McKenzie >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0-rc > > > Alongside the local and distributed read/write paths, we'll also want to > validate compaction. CASSANDRA-6696 introduced substantial > changes/improvements that require testing (esp. JBOD). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15573) Python 3.8 fails to execute cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-15573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15573:
---
Reviewers: Dinesh Joshi, Michael Semb Wever, Michael Semb Wever (was:
Dinesh Joshi, Michael Semb Wever)
Dinesh Joshi, Michael Semb Wever, Michael Semb Wever (was:
Dinesh Joshi, Michael Semb Wever)
Status: Review In Progress (was: Patch Available)
> Python 3.8 fails to execute cqlsh
> -
>
> Key: CASSANDRA-15573
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15573
> Project: Cassandra
> Issue Type: Sub-task
> Components: Tool/cqlsh
>Reporter: Yuki Morishita
>Assignee: Eduard Tudenhoefner
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Python 3.8 renamed sre_parse.Pattern to sre_parse.State (see
> [https://bugs.python.org/issue34681] and corresponding pull request
> [https://github.com/python/cpython/pull/9310])
> So when executing cqlsh with Python 3.8, it throws error:
> {code:java}
> Traceback (most recent call last):
> File ".\bin\cqlsh.py", line 175, in
> from cqlshlib import cql3handling, cqlhandling, pylexotron, sslhandling,
> cqlshhandling
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\cql3handling.py", line 19,
> in
> from cqlshlib.cqlhandling import CqlParsingRuleSet, Hint
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\cqlhandling.py", line 23,
> in
> from cqlshlib import pylexotron, util
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\pylexotron.py", line 342,
> in
> class ParsingRuleSet:
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\pylexotron.py", line 343,
> in ParsingRuleSet
> RuleSpecScanner = SaferScanner([
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\saferscanner.py", line 74,
> in __init__
> s = re.sre_parse.Pattern()
> AttributeError: module 'sre_parse' has no attribute 'Pattern'
> {code}
> h2. Summary of Work that was done
> Added a Python 3.8 compatible SaferScanner implementation ([diff
> here|https://github.com/apache/cassandra/pull/518/commits/2e6813f0ef5817e5d8d655052d61ce75a5fc062c]).
> Note that the changes from CASSANDRA-15659 are required in order to verify
> that the issue is fixed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Add Python3.8 compatible SaferScanner
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 24c8a21 Add Python3.8 compatible SaferScanner 24c8a21 is described below commit 24c8a21c1c131abd89c6b646343ff098d1b3263b Author: Eduard Tudenhoefner AuthorDate: Mon Apr 6 17:36:03 2020 +0200 Add Python3.8 compatible SaferScanner This is necessary because Python 3.8 renamed `sre_parse.Pattern` to `sre_parse.State` (see https://bugs.python.org/issue34681 and https://github.com/python/cpython/pull/9310/files for details) patch by Eduard Tudenhöfner, reviewed by Mick Semb Wever for CASSANDRA-15573 --- CHANGES.txt| 2 +- pylib/cqlshlib/saferscanner.py | 17 + 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/CHANGES.txt b/CHANGES.txt index 958ee99..5dd25e4 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,5 +1,5 @@ 4.0-alpha4 - * Allow cqlsh to run with Python2.7/Python3.6+ (CASSANDRA-15659) + * Allow cqlsh to run with Python2.7/Python3.6+ (CASSANDRA-15659,CASSANDRA-15573) * Improve logging around incremental repair (CASSANDRA-15599) * Do not check cdc_raw_directory filesystem space if CDC disabled (CASSANDRA-15688) * Replace array iterators with get by index (CASSANDRA-15394) diff --git a/pylib/cqlshlib/saferscanner.py b/pylib/cqlshlib/saferscanner.py index e559f1c..75b5df7 100644 --- a/pylib/cqlshlib/saferscanner.py +++ b/pylib/cqlshlib/saferscanner.py @@ -21,6 +21,7 @@ import re import six from sre_constants import BRANCH, SUBPATTERN, GROUPREF, GROUPREF_IGNORE, GROUPREF_EXISTS +from sys import version_info class SaferScannerBase(re.Scanner): @@ -82,4 +83,20 @@ class Py36SaferScanner(SaferScannerBase): self.scanner = re.sre_compile.compile(p) +class Py38SaferScanner(SaferScannerBase): + +def __init__(self, lexicon, flags=0): +self.lexicon = lexicon +p = [] +s = re.sre_parse.State() +s.flags = flags +for phrase, action in lexicon: +gid = s.opengroup() +p.append(re.sre_parse.SubPattern(s, [(SUBPATTERN, (gid, 0, 0, re.sre_parse.parse(phrase, flags))), ])) +s.closegroup(gid, p[-1]) +p = re.sre_parse.SubPattern(s, [(BRANCH, (None, p))]) +self.p = p +self.scanner = re.sre_compile.compile(p) + SaferScanner = Py36SaferScanner if six.PY3 else Py2SaferScanner +SaferScanner = Py38SaferScanner if version_info >= (3, 8) else SaferScanner - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15573) Python 3.8 fails to execute cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-15573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15573:
---
Status: Ready to Commit (was: Review In Progress)
> Python 3.8 fails to execute cqlsh
> -
>
> Key: CASSANDRA-15573
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15573
> Project: Cassandra
> Issue Type: Sub-task
> Components: Tool/cqlsh
>Reporter: Yuki Morishita
>Assignee: Eduard Tudenhoefner
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Python 3.8 renamed sre_parse.Pattern to sre_parse.State (see
> [https://bugs.python.org/issue34681] and corresponding pull request
> [https://github.com/python/cpython/pull/9310])
> So when executing cqlsh with Python 3.8, it throws error:
> {code:java}
> Traceback (most recent call last):
> File ".\bin\cqlsh.py", line 175, in
> from cqlshlib import cql3handling, cqlhandling, pylexotron, sslhandling,
> cqlshhandling
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\cql3handling.py", line 19,
> in
> from cqlshlib.cqlhandling import CqlParsingRuleSet, Hint
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\cqlhandling.py", line 23,
> in
> from cqlshlib import pylexotron, util
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\pylexotron.py", line 342,
> in
> class ParsingRuleSet:
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\pylexotron.py", line 343,
> in ParsingRuleSet
> RuleSpecScanner = SaferScanner([
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\saferscanner.py", line 74,
> in __init__
> s = re.sre_parse.Pattern()
> AttributeError: module 'sre_parse' has no attribute 'Pattern'
> {code}
> h2. Summary of Work that was done
> Added a Python 3.8 compatible SaferScanner implementation ([diff
> here|https://github.com/apache/cassandra/pull/518/commits/2e6813f0ef5817e5d8d655052d61ce75a5fc062c]).
> Note that the changes from CASSANDRA-15659 are required in order to verify
> that the issue is fixed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15573) Python 3.8 fails to execute cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-15573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15573:
---
Source Control Link:
https://github.com/apache/cassandra/commit/24c8a21c1c131abd89c6b646343ff098d1b3263b
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed as 24c8a21c1c131abd89c6b646343ff098d1b3263b
> Python 3.8 fails to execute cqlsh
> -
>
> Key: CASSANDRA-15573
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15573
> Project: Cassandra
> Issue Type: Sub-task
> Components: Tool/cqlsh
>Reporter: Yuki Morishita
>Assignee: Eduard Tudenhoefner
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Python 3.8 renamed sre_parse.Pattern to sre_parse.State (see
> [https://bugs.python.org/issue34681] and corresponding pull request
> [https://github.com/python/cpython/pull/9310])
> So when executing cqlsh with Python 3.8, it throws error:
> {code:java}
> Traceback (most recent call last):
> File ".\bin\cqlsh.py", line 175, in
> from cqlshlib import cql3handling, cqlhandling, pylexotron, sslhandling,
> cqlshhandling
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\cql3handling.py", line 19,
> in
> from cqlshlib.cqlhandling import CqlParsingRuleSet, Hint
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\cqlhandling.py", line 23,
> in
> from cqlshlib import pylexotron, util
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\pylexotron.py", line 342,
> in
> class ParsingRuleSet:
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\pylexotron.py", line 343,
> in ParsingRuleSet
> RuleSpecScanner = SaferScanner([
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\saferscanner.py", line 74,
> in __init__
> s = re.sre_parse.Pattern()
> AttributeError: module 'sre_parse' has no attribute 'Pattern'
> {code}
> h2. Summary of Work that was done
> Added a Python 3.8 compatible SaferScanner implementation ([diff
> here|https://github.com/apache/cassandra/pull/518/commits/2e6813f0ef5817e5d8d655052d61ce75a5fc062c]).
> Note that the changes from CASSANDRA-15659 are required in order to verify
> that the issue is fixed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15724) Modify cassandra.yaml and replace the default data location, still data stored in default location
[
https://issues.apache.org/jira/browse/CASSANDRA-15724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083563#comment-17083563
]
Jon Meredith commented on CASSANDRA-15724:
--
Looks like {{-Dcassandra.config}} is not set, so the server will use the
default {{cassandra.yaml}} (in the working directory the cassandra process is
started from).
One way you could fix that is editing {{jvm.options}} in /etc/cassandra and
adjusting this line
{code:java}
# The directory location of the cassandra.yaml file.
-Dcassandra.config=/etc/cassandra/default.conf/cassandra.yaml
{code}
Personally I would take a copy of the default config and place it there, but I
haven't looked at the redhat packaging closely to see how it is played out.
The chosen location should be logged during startup, e.g.
{code:java}
INFO [main] 2020-04-14 13:39:29,141 YamlConfigurationLoader.java:89 -
Configuration location: file:/etc/cassandra/cassandra.yaml {code}
I suspect this is just a configuration issue rather than a bug. Please let me
know how you get on and I'll close this issue if you're able to find and parse
the configuration file.
> Modify cassandra.yaml and replace the default data location, still data
> stored in default location
> --
>
> Key: CASSANDRA-15724
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15724
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Boopalan
>Priority: Normal
>
> I have created a separated directory (/mnt/) for cassandra data, commit log,
> hint and saved_cache. But data are not created on created directory, stored
> in default location /var/lib/cassandra/data.
>
> *Configuration File:*
> [root@node-master cassandra]# cat /etc/cassandra/default.conf/cassandra.yaml
> | grep /mnt/
> hints_directory: */mnt/*hints
> - */mnt/*data
> commitlog_directory: */mnt/*commitlog
> cdc_raw_directory: */mnt/*cdc_raw
> saved_caches_directory: */mnt/*saved_caches
>
> Please help me to resolve this issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15685) flaky testWithMismatchingPending - org.apache.cassandra.distributed.test.PreviewRepairTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15685:
--
Status: Open (was: Patch Available)
> flaky testWithMismatchingPending -
> org.apache.cassandra.distributed.test.PreviewRepairTest
> --
>
> Key: CASSANDRA-15685
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15685
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Kevin Gallardo
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-beta
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Observed in:
> https://app.circleci.com/pipelines/github/newkek/cassandra/34/workflows/1c6b157d-13c3-48a9-85fb-9fe8c153256b/jobs/191/tests
> Failure:
> {noformat}
> testWithMismatchingPending -
> org.apache.cassandra.distributed.test.PreviewRepairTest
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.distributed.test.PreviewRepairTest.testWithMismatchingPending(PreviewRepairTest.java:97)
> {noformat}
> [Circle
> CI|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FCASSANDRA-15685]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15685) flaky testWithMismatchingPending - org.apache.cassandra.distributed.test.PreviewRepairTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083698#comment-17083698
]
David Capwell commented on CASSANDRA-15685:
---
moved out of PA since I don't think the patch I sent solves this completely.
The main issue is we need a notification which is lossy, so we need to move
this to the completion message and poll for that in the tests (aka can't use
nodetool).
> flaky testWithMismatchingPending -
> org.apache.cassandra.distributed.test.PreviewRepairTest
> --
>
> Key: CASSANDRA-15685
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15685
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Kevin Gallardo
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-beta
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Observed in:
> https://app.circleci.com/pipelines/github/newkek/cassandra/34/workflows/1c6b157d-13c3-48a9-85fb-9fe8c153256b/jobs/191/tests
> Failure:
> {noformat}
> testWithMismatchingPending -
> org.apache.cassandra.distributed.test.PreviewRepairTest
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.distributed.test.PreviewRepairTest.testWithMismatchingPending(PreviewRepairTest.java:97)
> {noformat}
> [Circle
> CI|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FCASSANDRA-15685]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15727) Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if initial connection version incorrect
Jon Meredith created CASSANDRA-15727:
Summary: Internode messaging connection setup between 4.0 and
legacy SSL 3.0 fails if initial connection version incorrect
Key: CASSANDRA-15727
URL: https://issues.apache.org/jira/browse/CASSANDRA-15727
Project: Cassandra
Issue Type: Bug
Components: Messaging/Internode
Reporter: Jon Meredith
Assignee: Jon Meredith
This was discovered while testing upgrading an SSL enabled cluster from 3.0 to
4.0. The 3.0 cluster was configured to only listen on the ssl storage port.
When the upgraded 4.0 node started it received a gossip messsage that triggered
a shadow round before it had correctly set the messaging versions for the other
endpoints.
Sending the message created the connection, but because the endpoint defaulted
to {code}VERSION_40{code} the initial connect attempt was to the regular
{code}storage_port{code}. The 3.0 node was only listening on the
{code}ssl_storage_port{code}, so the connection was refused and the
{code}OutboundCOnnection.onFailure{code} handler was called. As the shadow
gossip round had queued up a message, the {code}hasPending{code} branch was
followed and the connection was rescheduled, however the port is never
recalculated as the original settings are used so it always fails.
Meanwhile, the node discovered information about peers through inbound
connection and gossip updating the messaging version for the endpoint which
could have been used to make a valid connection.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15727) Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if initial connection version incorrect
[
https://issues.apache.org/jira/browse/CASSANDRA-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Meredith updated CASSANDRA-15727:
-
Bug Category: Parent values: Availability(12983)Level 1 values:
Unavailable(12994)
Complexity: Low Hanging Fruit
Discovered By: Adhoc Test
Fix Version/s: 4.0-beta
Severity: Normal
Status: Open (was: Triage Needed)
> Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if
> initial connection version incorrect
> -
>
> Key: CASSANDRA-15727
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15727
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-beta
>
>
> This was discovered while testing upgrading an SSL enabled cluster from 3.0
> to 4.0. The 3.0 cluster was configured to only listen on the ssl storage
> port. When the upgraded 4.0 node started it received a gossip messsage that
> triggered a shadow round before it had correctly set the messaging versions
> for the other endpoints.
> Sending the message created the connection, but because the endpoint
> defaulted to {code}VERSION_40{code} the initial connect attempt was to the
> regular {code}storage_port{code}. The 3.0 node was only listening on the
> {code}ssl_storage_port{code}, so the connection was refused and the
> {code}OutboundCOnnection.onFailure{code} handler was called. As the shadow
> gossip round had queued up a message, the {code}hasPending{code} branch was
> followed and the connection was rescheduled, however the port is never
> recalculated as the original settings are used so it always fails.
> Meanwhile, the node discovered information about peers through inbound
> connection and gossip updating the messaging version for the endpoint which
> could have been used to make a valid connection.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15727) Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if initial connection version incorrect
[
https://issues.apache.org/jira/browse/CASSANDRA-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CASSANDRA-15727:
---
Labels: pull-request-available (was: )
> Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if
> initial connection version incorrect
> -
>
> Key: CASSANDRA-15727
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15727
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-beta
>
>
> This was discovered while testing upgrading an SSL enabled cluster from 3.0
> to 4.0. The 3.0 cluster was configured to only listen on the ssl storage
> port. When the upgraded 4.0 node started it received a gossip messsage that
> triggered a shadow round before it had correctly set the messaging versions
> for the other endpoints.
> Sending the message created the connection, but because the endpoint
> defaulted to {code}VERSION_40{code} the initial connect attempt was to the
> regular {code}storage_port{code}. The 3.0 node was only listening on the
> {code}ssl_storage_port{code}, so the connection was refused and the
> {code}OutboundCOnnection.onFailure{code} handler was called. As the shadow
> gossip round had queued up a message, the {code}hasPending{code} branch was
> followed and the connection was rescheduled, however the port is never
> recalculated as the original settings are used so it always fails.
> Meanwhile, the node discovered information about peers through inbound
> connection and gossip updating the messaging version for the endpoint which
> could have been used to make a valid connection.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15727) Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if initial connection version incorrect
[
https://issues.apache.org/jira/browse/CASSANDRA-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Meredith updated CASSANDRA-15727:
-
Test and Documentation Plan: Reproduced in ConnectionTest.
testPendingOutboundConnectionUpdatesMessageVersionOnReconnectAttempt and
checked the fix.
Status: Patch Available (was: In Progress)
Patch here (opened as a PR with CircleCI hi-res config added on top)
https://github.com/apache/cassandra/commit/e2bafa15bd228da06e3cf30e6ef860c18cc7b03b
> Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if
> initial connection version incorrect
> -
>
> Key: CASSANDRA-15727
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15727
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-beta
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This was discovered while testing upgrading an SSL enabled cluster from 3.0
> to 4.0. The 3.0 cluster was configured to only listen on the ssl storage
> port. When the upgraded 4.0 node started it received a gossip messsage that
> triggered a shadow round before it had correctly set the messaging versions
> for the other endpoints.
> Sending the message created the connection, but because the endpoint
> defaulted to {code}VERSION_40{code} the initial connect attempt was to the
> regular {code}storage_port{code}. The 3.0 node was only listening on the
> {code}ssl_storage_port{code}, so the connection was refused and the
> {code}OutboundCOnnection.onFailure{code} handler was called. As the shadow
> gossip round had queued up a message, the {code}hasPending{code} branch was
> followed and the connection was rescheduled, however the port is never
> recalculated as the original settings are used so it always fails.
> Meanwhile, the node discovered information about peers through inbound
> connection and gossip updating the messaging version for the endpoint which
> could have been used to make a valid connection.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15727) Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if initial connection version incorrect
[
https://issues.apache.org/jira/browse/CASSANDRA-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083755#comment-17083755
]
Jon Meredith edited comment on CASSANDRA-15727 at 4/15/20, 3:14 AM:
Patch here (opened as a PR with CircleCI hi-res config added on top)
https://github.com/apache/cassandra/commit/2356e86bc43494af7f54a1006242a1db42957d82
was (Author: jmeredithco):
Patch here (opened as a PR with CircleCI hi-res config added on top)
https://github.com/apache/cassandra/commit/e2bafa15bd228da06e3cf30e6ef860c18cc7b03b
> Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if
> initial connection version incorrect
> -
>
> Key: CASSANDRA-15727
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15727
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-beta
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This was discovered while testing upgrading an SSL enabled cluster from 3.0
> to 4.0. The 3.0 cluster was configured to only listen on the ssl storage
> port. When the upgraded 4.0 node started it received a gossip messsage that
> triggered a shadow round before it had correctly set the messaging versions
> for the other endpoints.
> Sending the message created the connection, but because the endpoint
> defaulted to {code}VERSION_40{code} the initial connect attempt was to the
> regular {code}storage_port{code}. The 3.0 node was only listening on the
> {code}ssl_storage_port{code}, so the connection was refused and the
> {code}OutboundCOnnection.onFailure{code} handler was called. As the shadow
> gossip round had queued up a message, the {code}hasPending{code} branch was
> followed and the connection was rescheduled, however the port is never
> recalculated as the original settings are used so it always fails.
> Meanwhile, the node discovered information about peers through inbound
> connection and gossip updating the messaging version for the endpoint which
> could have been used to make a valid connection.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15727) Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if initial connection version incorrect
[
https://issues.apache.org/jira/browse/CASSANDRA-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083758#comment-17083758
]
Jon Meredith commented on CASSANDRA-15727:
--
CircleCI
[Java8|https://circleci.com/workflow-run/4db87bd0-acc6-48e8-afcc-d259841af5c2]
[Java11|https://circleci.com/workflow-run/f2969704-8d9b-4d9e-9af4-0f59186b08d5]
> Internode messaging connection setup between 4.0 and legacy SSL 3.0 fails if
> initial connection version incorrect
> -
>
> Key: CASSANDRA-15727
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15727
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-beta
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This was discovered while testing upgrading an SSL enabled cluster from 3.0
> to 4.0. The 3.0 cluster was configured to only listen on the ssl storage
> port. When the upgraded 4.0 node started it received a gossip messsage that
> triggered a shadow round before it had correctly set the messaging versions
> for the other endpoints.
> Sending the message created the connection, but because the endpoint
> defaulted to {code}VERSION_40{code} the initial connect attempt was to the
> regular {code}storage_port{code}. The 3.0 node was only listening on the
> {code}ssl_storage_port{code}, so the connection was refused and the
> {code}OutboundCOnnection.onFailure{code} handler was called. As the shadow
> gossip round had queued up a message, the {code}hasPending{code} branch was
> followed and the connection was rescheduled, however the port is never
> recalculated as the original settings are used so it always fails.
> Meanwhile, the node discovered information about peers through inbound
> connection and gossip updating the messaging version for the endpoint which
> could have been used to make a valid connection.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
