[jira] [Updated] (CASSANDRA-19344) Test Failure: org.apache.cassandra.distributed.test.TransientRangeMovementTest.testRemoveNode-_jdk17
[
https://issues.apache.org/jira/browse/CASSANDRA-19344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-19344:
Attachment: ci_summary.html
result_details.tar.gz
> Test Failure:
> org.apache.cassandra.distributed.test.TransientRangeMovementTest.testRemoveNode-_jdk17
>
>
> Key: CASSANDRA-19344
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19344
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 5.x
>
> Attachments: ci_summary.html, result_details.tar.gz
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The test can fail in two different ways:
> {code:java}
> junit.framework.AssertionFailedError: NOT IN CURRENT: 31 -- [(00,20),
> (31,50)] at
> org.apache.cassandra.distributed.test.TransientRangeMovementTest.assertAllContained(TransientRangeMovementTest.java:203)
> at
> org.apache.cassandra.distributed.test.TransientRangeMovementTest.testRemoveNode(TransientRangeMovementTest.java:183)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43){code}
> as in here -
> [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2639/workflows/32b92ce7-5e9d-4efb-8362-d200d2414597/jobs/55139/tests#failed-test-0]
> and
> {code:java}
> junit.framework.AssertionFailedError: nodetool command [removenode,
> 6d194555-f6eb-41d0-c000-0003, --force] was not successful stdout:
> stderr: error: Node /127.0.0.4:7012 is alive and owns this ID. Use
> decommission command to remove it from the ring -- StackTrace --
> java.lang.UnsupportedOperationException: Node /127.0.0.4:7012 is alive and
> owns this ID. Use decommission command to remove it from the ring at
> org.apache.cassandra.tcm.sequences.SingleNodeSequences.removeNode(SingleNodeSequences.java:110)
> at
> org.apache.cassandra.service.StorageService.removeNode(StorageService.java:3682)
> at org.apache.cassandra.tools.NodeProbe.removeNode(NodeProbe.java:1020) at
> org.apache.cassandra.tools.nodetool.RemoveNode.execute(RemoveNode.java:51) at
> org.apache.cassandra.tools.NodeTool$NodeToolCmd.runInternal(NodeTool.java:388)
> at org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:373) at
> org.apache.cassandra.tools.NodeTool.execute(NodeTool.java:272) at
> org.apache.cassandra.distributed.impl.Instance$DTestNodeTool.execute(Instance.java:1129)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$nodetoolResult$51(Instance.java:1038)
> at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61) at
> org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71) at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833) Notifications: Error:
> java.lang.UnsupportedOperationException: Node /127.0.0.4:7012 is alive and
> owns this ID. Use decommission command to remove it from the ring at
> org.apache.cassandra.tcm.sequences.SingleNodeSequences.removeNode(SingleNodeSequences.java:110)
> at
> org.apache.cassandra.service.StorageService.removeNode(StorageService.java:3682)
> at org.apache.cassandra.tools.NodeProbe.removeNode(NodeProbe.java:1020) at
> org.apache.cassandra.tools.nodetool.RemoveNode.execute(RemoveNode.java:51) at
> org.apache.cassandra.tools.NodeTool$NodeToolCmd.runInternal(NodeTool.java:388)
> at org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:373) at
> org.apache.cassandra.tools.NodeTool.execute(NodeTool.java:272) at
> org.apache.cassandra.distributed.impl.Instance$DTestNodeTool.execute(Instance.java:1129)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$nodetoolResult$51(Instance.java:1038)
> at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61) at
> org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71) at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833) at
>
[jira] [Updated] (CASSANDRA-19344) Test Failure: org.apache.cassandra.distributed.test.TransientRangeMovementTest.testRemoveNode-_jdk17
[
https://issues.apache.org/jira/browse/CASSANDRA-19344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-19344:
Authors: Marcus Eriksson, Sam Tunnicliffe (was: Sam
Tunnicliffe)
Test and Documentation Plan:
New and existing tests in attached CI results.
The few failures/errors in the attached results are known (CASSANDRA-19343) or
look to be infra-related.
Status: Patch Available (was: In Progress)
When an instance moves from a transient to a full replica for a given range, it
must begin acting as a full replica for writes before it does so for reads.
Otherwise, consistency can be violated as data streamed to the instance early
in the operation can be removed by cleanup if it occurs before the instance
assumes responsibility for full writes. Also, coordinators will route read
requests to instances which may not have received all preceding writes, causing
unnecessary read repair or potentially inconsistent results.
The root cause of the flaky test failure which originally produced this issue
was that in {{TransientRangeMovementTest::testRemoveNode}} cleanup happened to
run on one instance before it had enacted the final step of the removal
operation, leading it to remove more data than it should.
> Test Failure:
> org.apache.cassandra.distributed.test.TransientRangeMovementTest.testRemoveNode-_jdk17
>
>
> Key: CASSANDRA-19344
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19344
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The test can fail in two different ways:
> {code:java}
> junit.framework.AssertionFailedError: NOT IN CURRENT: 31 -- [(00,20),
> (31,50)] at
> org.apache.cassandra.distributed.test.TransientRangeMovementTest.assertAllContained(TransientRangeMovementTest.java:203)
> at
> org.apache.cassandra.distributed.test.TransientRangeMovementTest.testRemoveNode(TransientRangeMovementTest.java:183)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43){code}
> as in here -
> [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2639/workflows/32b92ce7-5e9d-4efb-8362-d200d2414597/jobs/55139/tests#failed-test-0]
> and
> {code:java}
> junit.framework.AssertionFailedError: nodetool command [removenode,
> 6d194555-f6eb-41d0-c000-0003, --force] was not successful stdout:
> stderr: error: Node /127.0.0.4:7012 is alive and owns this ID. Use
> decommission command to remove it from the ring -- StackTrace --
> java.lang.UnsupportedOperationException: Node /127.0.0.4:7012 is alive and
> owns this ID. Use decommission command to remove it from the ring at
> org.apache.cassandra.tcm.sequences.SingleNodeSequences.removeNode(SingleNodeSequences.java:110)
> at
> org.apache.cassandra.service.StorageService.removeNode(StorageService.java:3682)
> at org.apache.cassandra.tools.NodeProbe.removeNode(NodeProbe.java:1020) at
> org.apache.cassandra.tools.nodetool.RemoveNode.execute(RemoveNode.java:51) at
> org.apache.cassandra.tools.NodeTool$NodeToolCmd.runInternal(NodeTool.java:388)
> at org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:373) at
> org.apache.cassandra.tools.NodeTool.execute(NodeTool.java:272) at
> org.apache.cassandra.distributed.impl.Instance$DTestNodeTool.execute(Instance.java:1129)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$nodetoolResult$51(Instance.java:1038)
> at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61) at
> org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71) at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833) Notifications: Error:
> java.lang.UnsupportedOperationException: Node /127.0.0.4:7012 is alive and
> owns this ID. Use decommission command to remove it from the ring at
> org.apache.cassandra.tcm.sequences.SingleNodeSequences.removeNode(SingleNodeSequences.java:110)
> at
> org.apache.cassandra.service.StorageService.removeNode(StorageService.java:3682)
> at org.apache.cassandra.tools.NodePro
[jira] [Updated] (CASSANDRA-19482) Simplify metadata log implementation using custom partitioner
[
https://issues.apache.org/jira/browse/CASSANDRA-19482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-19482:
Change Category: Code Clarity
Complexity: Normal
Fix Version/s: 5.x
Status: Open (was: Triage Needed)
> Simplify metadata log implementation using custom partitioner
> -
>
> Key: CASSANDRA-19482
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19482
> Project: Cassandra
> Issue Type: Improvement
> Components: Transactional Cluster Metadata
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The distributed metadata log table can be simplified by leveraging the fact
> that replicas are all responsible for the entire token range. Given this
> assumption, we can then use {{ReversedLongLocalPartitioner}} introduced in
> CASSANDRA-19391 to make it much easier to append to/read from the tail of the
> log, effectively removing the need for the {{Period}} construct. This will
> also improve apply to the local metadata log used at startup.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19482) Simplify metadata log implementation using custom partitioner
[
https://issues.apache.org/jira/browse/CASSANDRA-19482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-19482:
Authors: Marcus Eriksson, Sam Tunnicliffe (was: Sam
Tunnicliffe)
Test and Documentation Plan:
New and existing tests in attached CI results.
The few failures/errors in the attached results are known (CASSANDRA-19343) or
look to be infra-related.
Status: Patch Available (was: Open)
> Simplify metadata log implementation using custom partitioner
> -
>
> Key: CASSANDRA-19482
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19482
> Project: Cassandra
> Issue Type: Improvement
> Components: Transactional Cluster Metadata
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The distributed metadata log table can be simplified by leveraging the fact
> that replicas are all responsible for the entire token range. Given this
> assumption, we can then use {{ReversedLongLocalPartitioner}} introduced in
> CASSANDRA-19391 to make it much easier to append to/read from the tail of the
> log, effectively removing the need for the {{Period}} construct. This will
> also improve apply to the local metadata log used at startup.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19477) Do not go to disk to get HintsStore.getTotalFileSize
[
https://issues.apache.org/jira/browse/CASSANDRA-19477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829791#comment-17829791
]
Stefan Miklosovic commented on CASSANDRA-19477:
---
[CASSANDRA-19477-4.1|https://github.com/instaclustr/cassandra/tree/CASSANDRA-19477-4.1]
{noformat}
java11_pre-commit_tests
✓ j11_build1m 30s
✓ j11_cqlsh_dtests_py3 5m 28s
✓ j11_cqlsh_dtests_py311 5m 47s
✓ j11_cqlsh_dtests_py311_vnode 6m 4s
✓ j11_cqlsh_dtests_py385m 53s
✓ j11_cqlsh_dtests_py38_vnode 6m 2s
✓ j11_cqlsh_dtests_py3_vnode 5m 53s
✓ j11_cqlshlib_cython_tests 9m 1s
✓ j11_cqlshlib_tests8m 6s
✓ j11_dtests 33m 38s
✓ j11_dtests_vnode 36m 1s
✓ j11_jvm_dtests 15m 44s
✓ j11_jvm_dtests_repeat 38m 12s
✓ j11_jvm_dtests_vnode12m 39s
✓ j11_jvm_dtests_vnode_repeat 38m 46s
✓ j11_unit_tests_repeat0m 32s
✕ j11_unit_tests 8m 33s
org.apache.cassandra.cql3.MemtableSizeTest testSize[skiplist]
{noformat}
[java11_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/4064/workflows/88c6d9da-308c-43a7-849b-a2b1a6b30307]
> Do not go to disk to get HintsStore.getTotalFileSize
>
>
> Key: CASSANDRA-19477
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19477
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Hints
>Reporter: Jon Haddad
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.1.x, 5.0-rc, 5.x
>
> Attachments: flamegraph.cpu.html
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> When testing a cluster with more requests than it could handle, I noticed
> significant CPU time (25%) spent in HintsStore.getTotalFileSize. Here's what
> I'm seeing from profiling:
> 10% of CPU time spent in HintsDescriptor.fileName which only does this:
>
> {noformat}
> return String.format("%s-%s-%s.hints", hostId, timestamp, version);{noformat}
> At a bare minimum here we should create this string up front with the host
> and version and eliminate 2 of the 3 substitutions, but I think it's probably
> faster to use a StringBuilder and avoid the underlying regular expression
> altogether.
> 12% of the time is spent in org.apache.cassandra.io.util.File.length. It
> looks like this is called once for each hint file on disk for each host we're
> hinting to. In the case of an overloaded cluster, this is significant. It
> would be better if we were to track the file size in memory for each hint
> file and reference that rather than go to the filesystem.
> These fairly small changes should make Cassandra more reliable when under
> load spikes.
> CPU Flame graph attached.
> I only tested this in 4.1 but it looks like this is present up to trunk.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19482) Simplify metadata log implementation using custom partitioner
[
https://issues.apache.org/jira/browse/CASSANDRA-19482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-19482:
Attachment: ci_summary.html
result_details.tar.gz
> Simplify metadata log implementation using custom partitioner
> -
>
> Key: CASSANDRA-19482
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19482
> Project: Cassandra
> Issue Type: Improvement
> Components: Transactional Cluster Metadata
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 5.x
>
> Attachments: ci_summary.html, result_details.tar.gz
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The distributed metadata log table can be simplified by leveraging the fact
> that replicas are all responsible for the entire token range. Given this
> assumption, we can then use {{ReversedLongLocalPartitioner}} introduced in
> CASSANDRA-19391 to make it much easier to append to/read from the tail of the
> log, effectively removing the need for the {{Period}} construct. This will
> also improve apply to the local metadata log used at startup.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19344) Range movements involving transient replicas must safely enact changes to read and write replica sets
[
https://issues.apache.org/jira/browse/CASSANDRA-19344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-19344:
Summary: Range movements involving transient replicas must safely enact
changes to read and write replica sets (was: Test Failure:
org.apache.cassandra.distributed.test.TransientRangeMovementTest.testRemoveNode-_jdk17)
> Range movements involving transient replicas must safely enact changes to
> read and write replica sets
> -
>
> Key: CASSANDRA-19344
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19344
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 5.x
>
> Attachments: ci_summary.html, result_details.tar.gz
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The test can fail in two different ways:
> {code:java}
> junit.framework.AssertionFailedError: NOT IN CURRENT: 31 -- [(00,20),
> (31,50)] at
> org.apache.cassandra.distributed.test.TransientRangeMovementTest.assertAllContained(TransientRangeMovementTest.java:203)
> at
> org.apache.cassandra.distributed.test.TransientRangeMovementTest.testRemoveNode(TransientRangeMovementTest.java:183)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43){code}
> as in here -
> [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2639/workflows/32b92ce7-5e9d-4efb-8362-d200d2414597/jobs/55139/tests#failed-test-0]
> and
> {code:java}
> junit.framework.AssertionFailedError: nodetool command [removenode,
> 6d194555-f6eb-41d0-c000-0003, --force] was not successful stdout:
> stderr: error: Node /127.0.0.4:7012 is alive and owns this ID. Use
> decommission command to remove it from the ring -- StackTrace --
> java.lang.UnsupportedOperationException: Node /127.0.0.4:7012 is alive and
> owns this ID. Use decommission command to remove it from the ring at
> org.apache.cassandra.tcm.sequences.SingleNodeSequences.removeNode(SingleNodeSequences.java:110)
> at
> org.apache.cassandra.service.StorageService.removeNode(StorageService.java:3682)
> at org.apache.cassandra.tools.NodeProbe.removeNode(NodeProbe.java:1020) at
> org.apache.cassandra.tools.nodetool.RemoveNode.execute(RemoveNode.java:51) at
> org.apache.cassandra.tools.NodeTool$NodeToolCmd.runInternal(NodeTool.java:388)
> at org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:373) at
> org.apache.cassandra.tools.NodeTool.execute(NodeTool.java:272) at
> org.apache.cassandra.distributed.impl.Instance$DTestNodeTool.execute(Instance.java:1129)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$nodetoolResult$51(Instance.java:1038)
> at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61) at
> org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71) at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833) Notifications: Error:
> java.lang.UnsupportedOperationException: Node /127.0.0.4:7012 is alive and
> owns this ID. Use decommission command to remove it from the ring at
> org.apache.cassandra.tcm.sequences.SingleNodeSequences.removeNode(SingleNodeSequences.java:110)
> at
> org.apache.cassandra.service.StorageService.removeNode(StorageService.java:3682)
> at org.apache.cassandra.tools.NodeProbe.removeNode(NodeProbe.java:1020) at
> org.apache.cassandra.tools.nodetool.RemoveNode.execute(RemoveNode.java:51) at
> org.apache.cassandra.tools.NodeTool$NodeToolCmd.runInternal(NodeTool.java:388)
> at org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:373) at
> org.apache.cassandra.tools.NodeTool.execute(NodeTool.java:272) at
> org.apache.cassandra.distributed.impl.Instance$DTestNodeTool.execute(Instance.java:1129)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$nodetoolResult$51(Instance.java:1038)
> at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61) at
> org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71) at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:63
[jira] [Updated] (CASSANDRA-19344) Range movements involving transient replicas must safely enact changes to read and write replica sets
[
https://issues.apache.org/jira/browse/CASSANDRA-19344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-19344:
Description:
(edit) This was originally opened due to a flaky test
{{org.apache.cassandra.distributed.test.TransientRangeMovementTest.testRemoveNode-_jdk17}}
The test can fail in two different ways:
{code:java}
junit.framework.AssertionFailedError: NOT IN CURRENT: 31 -- [(00,20), (31,50)]
at
org.apache.cassandra.distributed.test.TransientRangeMovementTest.assertAllContained(TransientRangeMovementTest.java:203)
at
org.apache.cassandra.distributed.test.TransientRangeMovementTest.testRemoveNode(TransientRangeMovementTest.java:183)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43){code}
as in here -
[https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2639/workflows/32b92ce7-5e9d-4efb-8362-d200d2414597/jobs/55139/tests#failed-test-0]
and
{code:java}
junit.framework.AssertionFailedError: nodetool command [removenode,
6d194555-f6eb-41d0-c000-0003, --force] was not successful stdout:
stderr: error: Node /127.0.0.4:7012 is alive and owns this ID. Use decommission
command to remove it from the ring -- StackTrace --
java.lang.UnsupportedOperationException: Node /127.0.0.4:7012 is alive and owns
this ID. Use decommission command to remove it from the ring at
org.apache.cassandra.tcm.sequences.SingleNodeSequences.removeNode(SingleNodeSequences.java:110)
at
org.apache.cassandra.service.StorageService.removeNode(StorageService.java:3682)
at org.apache.cassandra.tools.NodeProbe.removeNode(NodeProbe.java:1020) at
org.apache.cassandra.tools.nodetool.RemoveNode.execute(RemoveNode.java:51) at
org.apache.cassandra.tools.NodeTool$NodeToolCmd.runInternal(NodeTool.java:388)
at org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:373) at
org.apache.cassandra.tools.NodeTool.execute(NodeTool.java:272) at
org.apache.cassandra.distributed.impl.Instance$DTestNodeTool.execute(Instance.java:1129)
at
org.apache.cassandra.distributed.impl.Instance.lambda$nodetoolResult$51(Instance.java:1038)
at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61) at
org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71) at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:833) Notifications: Error:
java.lang.UnsupportedOperationException: Node /127.0.0.4:7012 is alive and owns
this ID. Use decommission command to remove it from the ring at
org.apache.cassandra.tcm.sequences.SingleNodeSequences.removeNode(SingleNodeSequences.java:110)
at
org.apache.cassandra.service.StorageService.removeNode(StorageService.java:3682)
at org.apache.cassandra.tools.NodeProbe.removeNode(NodeProbe.java:1020) at
org.apache.cassandra.tools.nodetool.RemoveNode.execute(RemoveNode.java:51) at
org.apache.cassandra.tools.NodeTool$NodeToolCmd.runInternal(NodeTool.java:388)
at org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:373) at
org.apache.cassandra.tools.NodeTool.execute(NodeTool.java:272) at
org.apache.cassandra.distributed.impl.Instance$DTestNodeTool.execute(Instance.java:1129)
at
org.apache.cassandra.distributed.impl.Instance.lambda$nodetoolResult$51(Instance.java:1038)
at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61) at
org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71) at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:833) at
org.apache.cassandra.distributed.api.NodeToolResult$Asserts.fail(NodeToolResult.java:214)
at
org.apache.cassandra.distributed.api.NodeToolResult$Asserts.success(NodeToolResult.java:97)
at
org.apache.cassandra.distributed.test.TransientRangeMovementTest.testRemoveNode(TransientRangeMovementTest.java:173)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43){code}
as in here -
[https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2634/wor
[jira] [Commented] (CASSANDRA-19344) Range movements involving transient replicas must safely enact changes to read and write replica sets
[
https://issues.apache.org/jira/browse/CASSANDRA-19344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829794#comment-17829794
]
Sam Tunnicliffe commented on CASSANDRA-19344:
-
The linked PR modifies a number existing tests to make the original failure
mode deterministic. It also adds support for transient replication to
{{PlacementSimulator}} {{MetadataChangeSimulationTest}} and the associated
{{{}TokenPlacementModel{}}}. Finally, it modifies the way the
{{PlacementTransitionPlan}} is prepared for operations involving range
movements to ensure that any transition from a transient to full replica
happens safely.
> Range movements involving transient replicas must safely enact changes to
> read and write replica sets
> -
>
> Key: CASSANDRA-19344
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19344
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 5.x
>
> Attachments: ci_summary.html, result_details.tar.gz
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> (edit) This was originally opened due to a flaky test
> {{org.apache.cassandra.distributed.test.TransientRangeMovementTest.testRemoveNode-_jdk17}}
> The test can fail in two different ways:
> {code:java}
> junit.framework.AssertionFailedError: NOT IN CURRENT: 31 -- [(00,20),
> (31,50)] at
> org.apache.cassandra.distributed.test.TransientRangeMovementTest.assertAllContained(TransientRangeMovementTest.java:203)
> at
> org.apache.cassandra.distributed.test.TransientRangeMovementTest.testRemoveNode(TransientRangeMovementTest.java:183)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43){code}
> as in here -
> [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2639/workflows/32b92ce7-5e9d-4efb-8362-d200d2414597/jobs/55139/tests#failed-test-0]
> and
> {code:java}
> junit.framework.AssertionFailedError: nodetool command [removenode,
> 6d194555-f6eb-41d0-c000-0003, --force] was not successful stdout:
> stderr: error: Node /127.0.0.4:7012 is alive and owns this ID. Use
> decommission command to remove it from the ring -- StackTrace --
> java.lang.UnsupportedOperationException: Node /127.0.0.4:7012 is alive and
> owns this ID. Use decommission command to remove it from the ring at
> org.apache.cassandra.tcm.sequences.SingleNodeSequences.removeNode(SingleNodeSequences.java:110)
> at
> org.apache.cassandra.service.StorageService.removeNode(StorageService.java:3682)
> at org.apache.cassandra.tools.NodeProbe.removeNode(NodeProbe.java:1020) at
> org.apache.cassandra.tools.nodetool.RemoveNode.execute(RemoveNode.java:51) at
> org.apache.cassandra.tools.NodeTool$NodeToolCmd.runInternal(NodeTool.java:388)
> at org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:373) at
> org.apache.cassandra.tools.NodeTool.execute(NodeTool.java:272) at
> org.apache.cassandra.distributed.impl.Instance$DTestNodeTool.execute(Instance.java:1129)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$nodetoolResult$51(Instance.java:1038)
> at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61) at
> org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71) at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833) Notifications: Error:
> java.lang.UnsupportedOperationException: Node /127.0.0.4:7012 is alive and
> owns this ID. Use decommission command to remove it from the ring at
> org.apache.cassandra.tcm.sequences.SingleNodeSequences.removeNode(SingleNodeSequences.java:110)
> at
> org.apache.cassandra.service.StorageService.removeNode(StorageService.java:3682)
> at org.apache.cassandra.tools.NodeProbe.removeNode(NodeProbe.java:1020) at
> org.apache.cassandra.tools.nodetool.RemoveNode.execute(RemoveNode.java:51) at
> org.apache.cassandra.tools.NodeTool$NodeToolCmd.runInternal(NodeTool.java:388)
> at org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:373) at
> org.apache.cassandra.tools.NodeTool.execute(NodeTool.java:272) at
> org.apache.cassandra.distributed.impl.Instance$DTestNodeTool.execute(Instance.java:1129)
> at
[jira] [Commented] (CASSANDRA-19193) Reimplement ClusterMetadata::writePlacementAllSettled
[ https://issues.apache.org/jira/browse/CASSANDRA-19193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829801#comment-17829801 ] Marcus Eriksson commented on CASSANDRA-19193: - pushed a test > Reimplement ClusterMetadata::writePlacementAllSettled > - > > Key: CASSANDRA-19193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19193 > Project: Cassandra > Issue Type: Improvement > Components: Transactional Cluster Metadata >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 5.1-alpha1 > > Attachments: ci_summary.html, result_details.tar.gz > > Time Spent: 10m > Remaining Estimate: 0h > > This should step through InProgressSequences to determine state when > finished, rather than emulating the logic inline. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19191) Optimisations to PlacementForRange, improve lookup on r/w path
[
https://issues.apache.org/jira/browse/CASSANDRA-19191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-19191:
Change Category: Performance
Complexity: Normal
Reviewers: Alex Petrov, Sam Tunnicliffe
Status: Open (was: Triage Needed)
> Optimisations to PlacementForRange, improve lookup on r/w path
> --
>
> Key: CASSANDRA-19191
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19191
> Project: Cassandra
> Issue Type: Improvement
> Components: Transactional Cluster Metadata
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 5.1-alpha1
>
>
> The lookup used when selecting the appropriate replica group for a range or
> token while peforming reads and writes is extremely simplistic and
> inefficient. There is plenty of scope to improve {{PlacementsForRange}} to by
> replacing the current naive iteration with use a more efficient lookup.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19191) Optimisations to PlacementForRange, improve lookup on r/w path
[
https://issues.apache.org/jira/browse/CASSANDRA-19191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-19191:
Test and Documentation Plan: ci run, existing tests
Status: Patch Available (was: Open)
> Optimisations to PlacementForRange, improve lookup on r/w path
> --
>
> Key: CASSANDRA-19191
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19191
> Project: Cassandra
> Issue Type: Improvement
> Components: Transactional Cluster Metadata
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 5.1-alpha1
>
> Attachments: ci_summary.html, result_details.tar.gz
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The lookup used when selecting the appropriate replica group for a range or
> token while peforming reads and writes is extremely simplistic and
> inefficient. There is plenty of scope to improve {{PlacementsForRange}} to by
> replacing the current naive iteration with use a more efficient lookup.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19191) Optimisations to PlacementForRange, improve lookup on r/w path
[
https://issues.apache.org/jira/browse/CASSANDRA-19191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-19191:
Attachment: ci_summary.html
result_details.tar.gz
> Optimisations to PlacementForRange, improve lookup on r/w path
> --
>
> Key: CASSANDRA-19191
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19191
> Project: Cassandra
> Issue Type: Improvement
> Components: Transactional Cluster Metadata
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 5.1-alpha1
>
> Attachments: ci_summary.html, result_details.tar.gz
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The lookup used when selecting the appropriate replica group for a range or
> token while peforming reads and writes is extremely simplistic and
> inefficient. There is plenty of scope to improve {{PlacementsForRange}} to by
> replacing the current naive iteration with use a more efficient lookup.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829805#comment-17829805
]
Branimir Lambov commented on CASSANDRA-19471:
-
They are only for the IAE, which is a a serious issue and IMHO a blocker for
5.0.
I have not investigated the commitlog being written with durable writes off
which is a much more benign issue. It is likely caused by the preparation of
the direct I/O segments writing and flushing the header and first sync marker
in advance of any use of the segment.
> Commitlog with direct io fails test_change_durable_writes
> -
>
> Key: CASSANDRA-19471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19471
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> With the commitlog_disk_access_mode set to direct, and the improved
> configuration_test.py::TestConfiguration::test_change_durable_writes from
> CASSANDRA-19465, this fails with either:
> {noformat}
> AssertionError: Commitlog was written with durable writes disabled
> {noformat}
> Or what appears to be the original exception reported in CASSANDRA-19465:
> {noformat}
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
> StorageService.java:631 - Stopping native transport
> node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465
> StorageProxy.java:1670 - Failed to apply mutation locally :
> java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
> at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
> at java.base/java.nio.Buffer.position(Buffer.java:316)
> at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
> at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
> at
> org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
> at
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833)
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
> StorageService.java:636 - Stopping gossiper
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19255) StorageService.getRangeToEndpointMap() MBean operation is running into NPE for LocalStrategy keysapces
[ https://issues.apache.org/jira/browse/CASSANDRA-19255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] n.v.harikrishna updated CASSANDRA-19255: Attachment: result_details.tar.gz > StorageService.getRangeToEndpointMap() MBean operation is running into NPE > for LocalStrategy keysapces > -- > > Key: CASSANDRA-19255 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19255 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Membership >Reporter: n.v.harikrishna >Assignee: n.v.harikrishna >Priority: Normal > Attachments: ci_summary.html, result_details.tar.gz > > Time Spent: 10m > Remaining Estimate: 0h > > When the StorageService's MBean operation getRangeToEndpointMap is called for > LocalStrategy keyspaces, then it is running into NPE. It is working in > earlier major version, but failing in trunk. It can be reproduced in local > using JConsole or using a tool like `jmxterm` (unfortunately these tools are > not giving full stacktrace). Observed the same behavior with > getRangeToEndpointWithPortMap operation too. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19255) StorageService.getRangeToEndpointMap() MBean operation is running into NPE for LocalStrategy keysapces
[ https://issues.apache.org/jira/browse/CASSANDRA-19255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] n.v.harikrishna updated CASSANDRA-19255: Attachment: ci_summary.html > StorageService.getRangeToEndpointMap() MBean operation is running into NPE > for LocalStrategy keysapces > -- > > Key: CASSANDRA-19255 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19255 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Membership >Reporter: n.v.harikrishna >Assignee: n.v.harikrishna >Priority: Normal > Attachments: ci_summary.html, result_details.tar.gz > > Time Spent: 10m > Remaining Estimate: 0h > > When the StorageService's MBean operation getRangeToEndpointMap is called for > LocalStrategy keyspaces, then it is running into NPE. It is working in > earlier major version, but failing in trunk. It can be reproduced in local > using JConsole or using a tool like `jmxterm` (unfortunately these tools are > not giving full stacktrace). Observed the same behavior with > getRangeToEndpointWithPortMap operation too. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19255) StorageService.getRangeToEndpointMap() MBean operation is running into NPE for LocalStrategy keysapces
[ https://issues.apache.org/jira/browse/CASSANDRA-19255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] n.v.harikrishna updated CASSANDRA-19255: Test and Documentation Plan: Verified in local and ran tests. Status: Patch Available (was: Open) Uploaded the test results. [PR|http://example.com|https://github.com/apache/cassandra/pull/3141] > StorageService.getRangeToEndpointMap() MBean operation is running into NPE > for LocalStrategy keysapces > -- > > Key: CASSANDRA-19255 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19255 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Membership >Reporter: n.v.harikrishna >Assignee: n.v.harikrishna >Priority: Normal > Attachments: ci_summary.html, result_details.tar.gz > > Time Spent: 10m > Remaining Estimate: 0h > > When the StorageService's MBean operation getRangeToEndpointMap is called for > LocalStrategy keyspaces, then it is running into NPE. It is working in > earlier major version, but failing in trunk. It can be reproduced in local > using JConsole or using a tool like `jmxterm` (unfortunately these tools are > not giving full stacktrace). Observed the same behavior with > getRangeToEndpointWithPortMap operation too. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19255) StorageService.getRangeToEndpointMap() MBean operation is running into NPE for LocalStrategy keysapces
[ https://issues.apache.org/jira/browse/CASSANDRA-19255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829814#comment-17829814 ] Marcus Eriksson commented on CASSANDRA-19255: - +1 > StorageService.getRangeToEndpointMap() MBean operation is running into NPE > for LocalStrategy keysapces > -- > > Key: CASSANDRA-19255 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19255 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Membership >Reporter: n.v.harikrishna >Assignee: n.v.harikrishna >Priority: Normal > Attachments: ci_summary.html, result_details.tar.gz > > Time Spent: 10m > Remaining Estimate: 0h > > When the StorageService's MBean operation getRangeToEndpointMap is called for > LocalStrategy keyspaces, then it is running into NPE. It is working in > earlier major version, but failing in trunk. It can be reproduced in local > using JConsole or using a tool like `jmxterm` (unfortunately these tools are > not giving full stacktrace). Observed the same behavior with > getRangeToEndpointWithPortMap operation too. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch trunk updated: Fix getRangeTo* operations of StorageService mbean for local strategy keyspaces
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new a69c8657d7 Fix getRangeTo* operations of StorageService mbean for
local strategy keyspaces
a69c8657d7 is described below
commit a69c8657d75de627fb1fe518bfe1d657add11740
Author: nvharikrishna
AuthorDate: Tue Feb 27 17:25:30 2024 +0530
Fix getRangeTo* operations of StorageService mbean for local strategy
keyspaces
Patch by Venkata Harikrishna Nukala; reviewed by Marcus Eriksson and Sam
Tunnicliffe for CASSANDRA-19255
---
CHANGES.txt| 1 +
.../apache/cassandra/service/StorageService.java | 21 --
.../test/jmx/StorageServiceJmxTest.java| 76 ++
3 files changed, 93 insertions(+), 5 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 98c4245a93..7a63df43cc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
5.1
+ * Fix StorageService::constructRangeToEndpointMap for non-distributed
keyspaces (CASSANDRA-19255)
* Group nodetool cms commands into single command group (CASSANDRA-19393)
* Register the measurements of the bootstrap process as Dropwizard metrics
(CASSANDRA-19447)
* Add LIST SUPERUSERS CQL statement (CASSANDRA-19417)
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 8a8e7812e3..36dabcd807 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -1967,13 +1967,24 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
{
ClusterMetadata metadata = ClusterMetadata.current();
KeyspaceMetadata keyspaceMetadata =
metadata.schema.getKeyspaces().getNullable(keyspace);
-TokenMap tokenMap = metadata.tokenMap;
-
Map, EndpointsForRange> rangeToEndpointMap = new
HashMap<>(ranges.size());
-for (Range range : ranges)
+if (null != keyspaceMetadata)
+{
+TokenMap tokenMap = metadata.tokenMap;
+
+for (Range range : ranges)
+{
+Token token = tokenMap.nextToken(tokenMap.tokens(),
range.right.getToken());
+rangeToEndpointMap.put(range,
metadata.placements.get(keyspaceMetadata.params.replication)
+ .reads.forRange(token).get());
+}
+}
+else
{
-Token token = tokenMap.nextToken(tokenMap.tokens(),
range.right.getToken());
-rangeToEndpointMap.put(range,
metadata.placements.get(keyspaceMetadata.params.replication).reads.forRange(token).get());
+// Handling the keyspaces which are not handled by CMS like system
keyspace which uses LocalStrategy.
+AbstractReplicationStrategy strategy =
Keyspace.open(keyspace).getReplicationStrategy();
+for (Range range : ranges)
+rangeToEndpointMap.put(range,
strategy.calculateNaturalReplicas(range.right, metadata));
}
return new EndpointsByRange(rangeToEndpointMap);
diff --git
a/test/distributed/org/apache/cassandra/distributed/test/jmx/StorageServiceJmxTest.java
b/test/distributed/org/apache/cassandra/distributed/test/jmx/StorageServiceJmxTest.java
new file mode 100644
index 00..a25260b713
--- /dev/null
+++
b/test/distributed/org/apache/cassandra/distributed/test/jmx/StorageServiceJmxTest.java
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.distributed.test.jmx;
+
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.List;
+import javax.management.InstanceNotFoundException;
+import javax.management.MBeanException;
+import javax.management.MBeanServerConnection;
+import javax.management.MalformedObjectNameException;
+import javax.management.ObjectName;
+import javax.management.ReflectionException;
+import javax.management.remote.JMXConnector;
+
+import org.junit.Test;
+
+import org.ap
[jira] [Updated] (CASSANDRA-19255) StorageService.getRangeToEndpointMap() MBean operation is running into NPE for LocalStrategy keysapces
[ https://issues.apache.org/jira/browse/CASSANDRA-19255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-19255: Status: Review In Progress (was: Needs Committer) > StorageService.getRangeToEndpointMap() MBean operation is running into NPE > for LocalStrategy keysapces > -- > > Key: CASSANDRA-19255 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19255 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Membership >Reporter: n.v.harikrishna >Assignee: n.v.harikrishna >Priority: Normal > Attachments: ci_summary.html, result_details.tar.gz > > Time Spent: 10m > Remaining Estimate: 0h > > When the StorageService's MBean operation getRangeToEndpointMap is called for > LocalStrategy keyspaces, then it is running into NPE. It is working in > earlier major version, but failing in trunk. It can be reproduced in local > using JConsole or using a tool like `jmxterm` (unfortunately these tools are > not giving full stacktrace). Observed the same behavior with > getRangeToEndpointWithPortMap operation too. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19255) StorageService.getRangeToEndpointMap() MBean operation is running into NPE for LocalStrategy keysapces
[ https://issues.apache.org/jira/browse/CASSANDRA-19255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-19255: Status: Ready to Commit (was: Review In Progress) > StorageService.getRangeToEndpointMap() MBean operation is running into NPE > for LocalStrategy keysapces > -- > > Key: CASSANDRA-19255 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19255 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Membership >Reporter: n.v.harikrishna >Assignee: n.v.harikrishna >Priority: Normal > Attachments: ci_summary.html, result_details.tar.gz > > Time Spent: 10m > Remaining Estimate: 0h > > When the StorageService's MBean operation getRangeToEndpointMap is called for > LocalStrategy keyspaces, then it is running into NPE. It is working in > earlier major version, but failing in trunk. It can be reproduced in local > using JConsole or using a tool like `jmxterm` (unfortunately these tools are > not giving full stacktrace). Observed the same behavior with > getRangeToEndpointWithPortMap operation too. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19255) StorageService.getRangeToEndpointMap() MBean operation is running into NPE for LocalStrategy keysapces
[
https://issues.apache.org/jira/browse/CASSANDRA-19255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-19255:
Fix Version/s: 5.x
Since Version: 5.x
Source Control Link:
https://github.com/apache/cassandra/commit/a69c8657d75de627fb1fe518bfe1d657add11740
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
LGTM too, committed as {{a69c8657d75de627fb1fe518bfe1d657add11740}} (with a
couple of extremely minor changes).
One thing to note is that this will need slight readjustment if CASSANDRA-19482
is committed as the way we handle the {{MetaStrategy}} keyspace is changed.
The attached diff will fix that if/when necessary.
> StorageService.getRangeToEndpointMap() MBean operation is running into NPE
> for LocalStrategy keysapces
> --
>
> Key: CASSANDRA-19255
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19255
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Membership
>Reporter: n.v.harikrishna
>Assignee: n.v.harikrishna
>Priority: Normal
> Fix For: 5.x
>
> Attachments: ci_summary.html, result_details.tar.gz
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When the StorageService's MBean operation getRangeToEndpointMap is called for
> LocalStrategy keyspaces, then it is running into NPE. It is working in
> earlier major version, but failing in trunk. It can be reproduced in local
> using JConsole or using a tool like `jmxterm` (unfortunately these tools are
> not giving full stacktrace). Observed the same behavior with
> getRangeToEndpointWithPortMap operation too.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19255) StorageService.getRangeToEndpointMap() MBean operation is running into NPE for LocalStrategy keysapces
[ https://issues.apache.org/jira/browse/CASSANDRA-19255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-19255: Status: Needs Committer (was: Patch Available) > StorageService.getRangeToEndpointMap() MBean operation is running into NPE > for LocalStrategy keysapces > -- > > Key: CASSANDRA-19255 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19255 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Membership >Reporter: n.v.harikrishna >Assignee: n.v.harikrishna >Priority: Normal > Attachments: ci_summary.html, result_details.tar.gz > > Time Spent: 10m > Remaining Estimate: 0h > > When the StorageService's MBean operation getRangeToEndpointMap is called for > LocalStrategy keyspaces, then it is running into NPE. It is working in > earlier major version, but failing in trunk. It can be reproduced in local > using JConsole or using a tool like `jmxterm` (unfortunately these tools are > not giving full stacktrace). Observed the same behavior with > getRangeToEndpointWithPortMap operation too. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19255) StorageService.getRangeToEndpointMap() MBean operation is running into NPE for LocalStrategy keysapces
[ https://issues.apache.org/jira/browse/CASSANDRA-19255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-19255: Attachment: 19482_patch_for_19255.diff > StorageService.getRangeToEndpointMap() MBean operation is running into NPE > for LocalStrategy keysapces > -- > > Key: CASSANDRA-19255 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19255 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Membership >Reporter: n.v.harikrishna >Assignee: n.v.harikrishna >Priority: Normal > Fix For: 5.x > > Attachments: 19482_patch_for_19255.diff, ci_summary.html, > result_details.tar.gz > > Time Spent: 10m > Remaining Estimate: 0h > > When the StorageService's MBean operation getRangeToEndpointMap is called for > LocalStrategy keyspaces, then it is running into NPE. It is working in > earlier major version, but failing in trunk. It can be reproduced in local > using JConsole or using a tool like `jmxterm` (unfortunately these tools are > not giving full stacktrace). Observed the same behavior with > getRangeToEndpointWithPortMap operation too. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19255) StorageService.getRangeToEndpointMap() MBean operation is running into NPE for LocalStrategy keysapces
[ https://issues.apache.org/jira/browse/CASSANDRA-19255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829829#comment-17829829 ] n.v.harikrishna commented on CASSANDRA-19255: - Thanks [~marcuse] & [~samt] ! > StorageService.getRangeToEndpointMap() MBean operation is running into NPE > for LocalStrategy keysapces > -- > > Key: CASSANDRA-19255 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19255 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Membership >Reporter: n.v.harikrishna >Assignee: n.v.harikrishna >Priority: Normal > Fix For: 5.x > > Attachments: 19482_patch_for_19255.diff, ci_summary.html, > result_details.tar.gz > > Time Spent: 10m > Remaining Estimate: 0h > > When the StorageService's MBean operation getRangeToEndpointMap is called for > LocalStrategy keyspaces, then it is running into NPE. It is working in > earlier major version, but failing in trunk. It can be reproduced in local > using JConsole or using a tool like `jmxterm` (unfortunately these tools are > not giving full stacktrace). Observed the same behavior with > getRangeToEndpointWithPortMap operation too. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19341) Relation and Restriction hierachies are too complex and error prone
[
https://issues.apache.org/jira/browse/CASSANDRA-19341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-19341:
---
Test and Documentation Plan: The patch adds new tests for some of the new
classes and relies on existing tests to validate the changes.
Status: Patch Available (was: Open)
CI results:
[J11|https://app.circleci.com/pipelines/github/blerer/cassandra/390/workflows/7c475c9c-9936-4def-9ec5-936859cfcc21],
[J17|https://app.circleci.com/pipelines/github/blerer/cassandra/390/workflows/2655f762-5fba-45ef-946e-5db7637873a4]
DTests changes: https://github.com/apache/cassandra-dtest/pull/259
> Relation and Restriction hierachies are too complex and error prone
> ---
>
> Key: CASSANDRA-19341
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19341
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL/Interpreter
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The {{Relation}} and {{Restriction}} hierarchy have been designed when C* was
> only supporting a limited amount of operators and columns expressions (single
> column, multi-column and token expressions). Over time they have grown in
> complexity making the code harder to understand and modify and error prone.
> Their design is also resulting in unnecessary limitations that could be
> easily lifted, like the ability to accept different predicates on the same
> column.
> Today adding a new operator requires the addition of a lot of glue code and
> surgical changes accross the CQL layer. Making patch for features such as
> CASSANDRA-18584 much complex than it should be.
> The goal of this ticket is to simplify the {{Relation}} and {{Restriction}}
> hierarchies and modify operator class so that adding new operators requires
> only changes to the {{Operator}} class and ANTLR file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18762) Repair triggers OOM with direct buffer memory
[
https://issues.apache.org/jira/browse/CASSANDRA-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829743#comment-17829743
]
Manish Khandelwal edited comment on CASSANDRA-18762 at 3/22/24 1:36 PM:
I think reason for getting OOM here is related to same reasoning as mentioned
in https://issues.apache.org/jira/browse/CASSANDRA-19336 I applied the patch
for https://issues.apache.org/jira/browse/CASSANDRA-19336 and all full repairs
with -pr on keyspace were successful.
As without this patch in one repair we can see almost 240 sessions triggered (
vnode:256, 11*11 cluster), resulting in 240*6 merkle tree requests for one
table. For a keywpace with 3 tables this number was astonishing 240*6*3
resulting in direct byte buffer within a minute of running.
After applying the patch repairs ran without issue also no memory pressue.
was (Author: manmagic3):
I think reason for getting OOM here is related to same reasoning as mentioned
in https://issues.apache.org/jira/browse/CASSANDRA-19336. I applied the patch
for https://issues.apache.org/jira/browse/CASSANDRA-19336 and all full repairs
with -pr on keyspace were successful.
As without this patch in one repair we can see almost 240 sessions triggered (
vnode:256, 11*11 cluster), resulting in 240*6 merkle tree requests for one
table. For a keywpace with 3 tables this number was astonishing 240*6*3
resulting in direct byte buffer within a minute of running.
After applying the patch repairs ran without issue also no memory pressue.
> Repair triggers OOM with direct buffer memory
> -
>
> Key: CASSANDRA-18762
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18762
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair
>Reporter: Brad Schoening
>Priority: Normal
> Labels: OutOfMemoryError
> Attachments: Cluster-dm-metrics-1.PNG,
> image-2023-12-06-15-28-05-459.png, image-2023-12-06-15-29-31-491.png,
> image-2023-12-06-15-58-55-007.png
>
>
> We are seeing repeated failures of nodes with 16GB of heap on a VM with 32GB
> of physical RAM due to direct memory. This seems to be related to
> CASSANDRA-15202 which moved Merkel trees off-heap in 4.0. Using Cassandra
> 4.0.6 with Java 11.
> {noformat}
> 2023-08-09 04:30:57,470 [INFO ] [AntiEntropyStage:1] cluster_id=101
> ip_address=169.0.0.1 RepairSession.java:202 - [repair
> #5e55a3b0-366d-11ee-a644-d91df26add5e] Received merkle tree for table_a from
> /169.102.200.241:7000
> 2023-08-09 04:30:57,567 [INFO ] [AntiEntropyStage:1] cluster_id=101
> ip_address=169.0.0.1 RepairSession.java:202 - [repair
> #5e0d2900-366d-11ee-a644-d91df26add5e] Received merkle tree for table_b from
> /169.93.192.29:7000
> 2023-08-09 04:30:57,568 [INFO ] [AntiEntropyStage:1] cluster_id=101
> ip_address=169.0.0.1 RepairSession.java:202 - [repair
> #5e1dcad0-366d-11ee-a644-d91df26add5e] Received merkle tree for table_c from
> /169.104.171.134:7000
> 2023-08-09 04:30:57,591 [INFO ] [AntiEntropyStage:1] cluster_id=101
> ip_address=169.0.0.1 RepairSession.java:202 - [repair
> #5e69a0e0-366d-11ee-a644-d91df26add5e] Received merkle tree for table_b from
> /169.79.232.67:7000
> 2023-08-09 04:30:57,876 [INFO ] [Service Thread] cluster_id=101
> ip_address=169.0.0.1 GCInspector.java:294 - G1 Old Generation GC in 282ms.
> Compressed Class Space: 8444560 -> 8372152; G1 Eden Space: 7809794048 -> 0;
> G1 Old Gen: 1453478400 -> 820942800; G1 Survivor Space: 419430400 -> 0;
> Metaspace: 80411136 -> 80176528
> 2023-08-09 04:30:58,387 [ERROR] [AntiEntropyStage:1] cluster_id=101
> ip_address=169.0.0.1 JVMStabilityInspector.java:102 - OutOfMemory error
> letting the JVM handle the error:
> java.lang.OutOfMemoryError: Direct buffer memory
> at java.base/java.nio.Bits.reserveMemory(Bits.java:175)
> at java.base/java.nio.DirectByteBuffer.(DirectByteBuffer.java:118)
> at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:318)
> at org.apache.cassandra.utils.MerkleTree.allocate(MerkleTree.java:742)
> at
> org.apache.cassandra.utils.MerkleTree.deserializeOffHeap(MerkleTree.java:780)
> at org.apache.cassandra.utils.MerkleTree.deserializeTree(MerkleTree.java:751)
> at org.apache.cassandra.utils.MerkleTree.deserialize(MerkleTree.java:720)
> at org.apache.cassandra.utils.MerkleTree.deserialize(MerkleTree.java:698)
> at
> org.apache.cassandra.utils.MerkleTrees$MerkleTreesSerializer.deserialize(MerkleTrees.java:416)
> at
> org.apache.cassandra.repair.messages.ValidationResponse$1.deserialize(ValidationResponse.java:100)
> at
> org.apache.cassandra.repair.messages.ValidationResponse$1.deserialize(ValidationResponse.java:84)
> at
> org.apache.cassandra.net.Message$Serializer.deserializePost40(Message.java:782)
> at org.apache.cassandra.n
[jira] [Commented] (CASSANDRA-19428) Clean up KeyRangeIterator classes
[ https://issues.apache.org/jira/browse/CASSANDRA-19428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829882#comment-17829882 ] Ekaterina Dimitrova commented on CASSANDRA-19428: - I forgot to mention I did not change the tests in SingleNodeQueryFailureTest to multiIndex for now(as per the title), but I think we should; I will leave my suggestion as a comment on Git Hub for confirmation. > Clean up KeyRangeIterator classes > - > > Key: CASSANDRA-19428 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19428 > Project: Cassandra > Issue Type: Improvement > Components: Feature/2i Index >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Low > Fix For: 5.0-rc, 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > Remove KeyRangeIterator.current and simplify -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-19130) Implement transactional table truncation
[
https://issues.apache.org/jira/browse/CASSANDRA-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov reassigned CASSANDRA-19130:
---
Assignee: Stefan Miklosovic
> Implement transactional table truncation
>
>
> Key: CASSANDRA-19130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19130
> Project: Cassandra
> Issue Type: New Feature
> Components: Consistency/Coordination
>Reporter: Marcus Eriksson
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TRUNCATE table should leverage cluster metadata to ensure consistent
> truncation timestamps across all replicas. The current implementation depends
> on all nodes being available, but this could be reimplemented as a
> {{Transformation}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19448) CommitlogArchiver only has granularity to seconds for restore_point_in_time
[ https://issues.apache.org/jira/browse/CASSANDRA-19448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829886#comment-17829886 ] Tiago L. Alves commented on CASSANDRA-19448: [~maxwellguo] I've looked into the code and it seems to be possible to implement millisecond and microsecond-level PIT restore. Supporting millisecond requires changes in `CommitLogArchiver` only while supporting microsecond requires additional changes in `CommitLogReplayer`. In both `CommitLogArchiver` and `CommitLogReplayer`, `restorePointInTime` and `restoreTarget` respectively, we're assuming long values in milliseconds. Supporting millisecond level PIT restore, can be achieved by just recognizing the milliseconds part specified in `restore_point_in_time` configuration. The existent code does not fail parsing if milliseconds are specified but it will ignore it. To support microsecond granularity, we need further changes in `CommitLogReplayer` to accept microseconds instead of forcing comparison to be in milliseconds. I have a patch that adds support for millisecond-level PIT restore and issues warnings on attempts to specify microseconds granularity: https://github.com/apache/cassandra/pull/3200 > CommitlogArchiver only has granularity to seconds for restore_point_in_time > --- > > Key: CASSANDRA-19448 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19448 > Project: Cassandra > Issue Type: Bug > Components: Local/Commit Log >Reporter: Jeremy Hanna >Assignee: Maxwell Guo >Priority: Normal > Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > Commitlog archiver allows users to backup commitlog files for the purpose of > doing point in time restores. The [configuration > file|https://github.com/apache/cassandra/blob/trunk/conf/commitlog_archiving.properties] > gives an example of down to the seconds granularity but then asks what > whether the timestamps are microseconds or milliseconds - defaulting to > microseconds. Because the [CommitLogArchiver uses a second based date > format|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java#L52], > if a user specifies to restore at something at a lower granularity like > milliseconds or microseconds, that means that the it will truncate everything > after the second and restore to that second. So say you specify a > restore_point_in_time like this: > restore_point_in_time=2024:01:18 17:01:01.623392 > it will silently truncate everything after the 01 seconds. So effectively to > the user, it is missing updates between 01 and 01.623392. > This appears to be a bug in the intent. We should allow users to specify > down to the millisecond or even microsecond level. If we allow them to > specify down to microseconds for the restore point in time, then it may > internally need to change from a long. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19130) Implement transactional table truncation
[
https://issues.apache.org/jira/browse/CASSANDRA-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829887#comment-17829887
]
Alex Petrov commented on CASSANDRA-19130:
-
Just had a very brief look at the patch. Several comments:
{code}
if (tableMetadata.isVirtual())
return executeLocally(state, options);
{code}
We probably should not truncate _before_ we commit, since there's nothing that
guarantees we _can_ actually commit.
Also, we should not have {{cfs.truncateBlocking();}} inside the {{execute}}
block of transformation, since this block has to be free of side effects.
> Implement transactional table truncation
>
>
> Key: CASSANDRA-19130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19130
> Project: Cassandra
> Issue Type: New Feature
> Components: Consistency/Coordination
>Reporter: Marcus Eriksson
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TRUNCATE table should leverage cluster metadata to ensure consistent
> truncation timestamps across all replicas. The current implementation depends
> on all nodes being available, but this could be reimplemented as a
> {{Transformation}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19130) Implement transactional table truncation
[
https://issues.apache.org/jira/browse/CASSANDRA-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829889#comment-17829889
]
Stefan Miklosovic commented on CASSANDRA-19130:
---
When it comes to
{code}
if (tableMetadata.isVirtual())
return executeLocally(state, options);
{code}
This will not work in TCM because virtual table is not distributed so there is
basically nothing to commit into TCM and I just return there instantly. The
truncation against vtables is done only against the implementations of
AbstractMutableVirtualTable-s and they are not covered by TCM, each vtable is
just a local one, so committing anything does not make sense, no?
I am not sure how to solve that blocking truncation ... Any ideas?
But when I step aside there for a while, is this really what we want to do?
Truncations committed to TCM? Currently, all nodes need to be up, as already
described in the description, so here we are moving away from that? I would
like to have that confirmed.
> Implement transactional table truncation
>
>
> Key: CASSANDRA-19130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19130
> Project: Cassandra
> Issue Type: New Feature
> Components: Consistency/Coordination
>Reporter: Marcus Eriksson
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TRUNCATE table should leverage cluster metadata to ensure consistent
> truncation timestamps across all replicas. The current implementation depends
> on all nodes being available, but this could be reimplemented as a
> {{Transformation}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19130) Implement transactional table truncation
[
https://issues.apache.org/jira/browse/CASSANDRA-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829889#comment-17829889
]
Stefan Miklosovic edited comment on CASSANDRA-19130 at 3/22/24 2:12 PM:
When it comes to
{code}
if (tableMetadata.isVirtual())
return executeLocally(state, options);
{code}
This will not work in TCM because virtual table is not distributed so there is
basically nothing to commit into TCM and I just return there instantly. The
truncation against vtables is done only against the implementations of
AbstractMutableVirtualTable-s and they are not covered by TCM, each vtable is
just a local one, so committing anything does not make sense, no?
I am not sure how to solve that blocking truncation ... Any ideas?
But when I step aside for a while, is this really what we want to do?
Truncations committed to TCM? Currently, all nodes need to be up, as already
described in the description, so here we are moving away from that? I would
like to have that confirmed.
was (Author: smiklosovic):
When it comes to
{code}
if (tableMetadata.isVirtual())
return executeLocally(state, options);
{code}
This will not work in TCM because virtual table is not distributed so there is
basically nothing to commit into TCM and I just return there instantly. The
truncation against vtables is done only against the implementations of
AbstractMutableVirtualTable-s and they are not covered by TCM, each vtable is
just a local one, so committing anything does not make sense, no?
I am not sure how to solve that blocking truncation ... Any ideas?
But when I step aside there for a while, is this really what we want to do?
Truncations committed to TCM? Currently, all nodes need to be up, as already
described in the description, so here we are moving away from that? I would
like to have that confirmed.
> Implement transactional table truncation
>
>
> Key: CASSANDRA-19130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19130
> Project: Cassandra
> Issue Type: New Feature
> Components: Consistency/Coordination
>Reporter: Marcus Eriksson
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TRUNCATE table should leverage cluster metadata to ensure consistent
> truncation timestamps across all replicas. The current implementation depends
> on all nodes being available, but this could be reimplemented as a
> {{Transformation}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19333) Fix decode in VectorCodec
[ https://issues.apache.org/jira/browse/CASSANDRA-19333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-19333: Reviewers: Alexandre Dutra, Bret McGuire (was: Bret McGuire) Status: Review In Progress (was: Needs Committer) > Fix decode in VectorCodec > - > > Key: CASSANDRA-19333 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19333 > Project: Cassandra > Issue Type: Bug > Components: Client/java-driver >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > > We made a server-side fix in CASSANDRA-19167 for VectorCodec. > This needs to be checked also in the driver [here > |https://github.com/apache/cassandra-java-driver/blob/8d5849cb38995b312f29314d18256c0c3e94cf07/core/src/main/java/com/datastax/oss/driver/internal/core/type/codec/VectorCodec.java#L130-L139] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19333) Fix decode in VectorCodec
[ https://issues.apache.org/jira/browse/CASSANDRA-19333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-19333: Status: Ready to Commit (was: Review In Progress) > Fix decode in VectorCodec > - > > Key: CASSANDRA-19333 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19333 > Project: Cassandra > Issue Type: Bug > Components: Client/java-driver >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > > We made a server-side fix in CASSANDRA-19167 for VectorCodec. > This needs to be checked also in the driver [here > |https://github.com/apache/cassandra-java-driver/blob/8d5849cb38995b312f29314d18256c0c3e94cf07/core/src/main/java/com/datastax/oss/driver/internal/core/type/codec/VectorCodec.java#L130-L139] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19130) Implement transactional table truncation
[
https://issues.apache.org/jira/browse/CASSANDRA-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829890#comment-17829890
]
Alex Petrov edited comment on CASSANDRA-19130 at 3/22/24 2:19 PM:
--
bq. This will not work in TCM because virtual table is not distributed so there
is basically nothing to commit into TCM and I just return there instantly.
I guess if the intention is to only truncate the table on the local machine,
this might be fine, I just don't think it's user-friendly semantics.
bq. But when I step aside for a while, is this really what we want to do?
Truncations committed to TCM? Currently, all nodes need to be up, as already
described in the description, so here we are moving away from that? I would
like to have that confirmed.
I think this is what we want to do, yes, since we'll be able to record
truncation timestamp in the log and have it replayed. And the limitation of the
nodes being up will be lifted, too.
was (Author: ifesdjeen):
bq. This will not work in TCM because virtual table is not distributed so there
is basically nothing to commit into TCM and I just return there instantly.
I guess if the intention is to only truncate the table on the local machine,
this might be fine, I just don't think it's user-friendly semantics.
> But when I step aside for a while, is this really what we want to do?
> Truncations committed to TCM? Currently, all nodes need to be up, as already
> described in the description, so here we are moving away from that? I would
> like to have that confirmed.
I think this is what we want to do, yes, since we'll be able to record
truncation timestamp in the log and have it replayed. And the limitation of the
nodes being up will be lifted, too.
> Implement transactional table truncation
>
>
> Key: CASSANDRA-19130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19130
> Project: Cassandra
> Issue Type: New Feature
> Components: Consistency/Coordination
>Reporter: Marcus Eriksson
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TRUNCATE table should leverage cluster metadata to ensure consistent
> truncation timestamps across all replicas. The current implementation depends
> on all nodes being available, but this could be reimplemented as a
> {{Transformation}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19130) Implement transactional table truncation
[
https://issues.apache.org/jira/browse/CASSANDRA-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829890#comment-17829890
]
Alex Petrov commented on CASSANDRA-19130:
-
bq. This will not work in TCM because virtual table is not distributed so there
is basically nothing to commit into TCM and I just return there instantly.
I guess if the intention is to only truncate the table on the local machine,
this might be fine, I just don't think it's user-friendly semantics.
> But when I step aside for a while, is this really what we want to do?
> Truncations committed to TCM? Currently, all nodes need to be up, as already
> described in the description, so here we are moving away from that? I would
> like to have that confirmed.
I think this is what we want to do, yes, since we'll be able to record
truncation timestamp in the log and have it replayed. And the limitation of the
nodes being up will be lifted, too.
> Implement transactional table truncation
>
>
> Key: CASSANDRA-19130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19130
> Project: Cassandra
> Issue Type: New Feature
> Components: Consistency/Coordination
>Reporter: Marcus Eriksson
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TRUNCATE table should leverage cluster metadata to ensure consistent
> truncation timestamps across all replicas. The current implementation depends
> on all nodes being available, but this could be reimplemented as a
> {{Transformation}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19130) Implement transactional table truncation
[
https://issues.apache.org/jira/browse/CASSANDRA-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829898#comment-17829898
]
Sam Tunnicliffe commented on CASSANDRA-19130:
-
The way truncation works is that it writes a timestamp into a system table on
each node, associated with the table being truncated (and a commitlog
position). Then, when local reads and writes are done against that table, any
cells with a timestamp earlier than the truncation is essentially discarded. If
any node misses that message and so doesn't write the timestamp it won't do
this filtering and so data can be resurrected. This is a strictly one time
operation and there's no way for a node which does miss such a message to catch
it up later, which is why {{TRUNCATE}} currently requires all nodes to be up.
With TCM, we can improve this by having an entry in the log which contains the
truncation timestamp. Then it can be distributed to peers the same way as any
other log entry, allowing them to catch up if they miss it. Replicas and
coordinators participating in a read already check that they're all up to date
with each other attempt to catch up if not.
We shouldn't have to change how truncation works on the local level, just have
{{TruncateStatement}} work by committing a new transform to the CMS. The
trickiest bit will be to make sure that the {{execute}} method itself is
side-effect free (i.e. it only produces a new ClusterMetadata). The way to do
that is with a {{ChangeListener}} which implements a post-commit event to do
the work of {{CFS::truncateBlocking}}
> Implement transactional table truncation
>
>
> Key: CASSANDRA-19130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19130
> Project: Cassandra
> Issue Type: New Feature
> Components: Consistency/Coordination
>Reporter: Marcus Eriksson
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TRUNCATE table should leverage cluster metadata to ensure consistent
> truncation timestamps across all replicas. The current implementation depends
> on all nodes being available, but this could be reimplemented as a
> {{Transformation}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13855) Implement Http Seed provider
[
https://issues.apache.org/jira/browse/CASSANDRA-13855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829900#comment-17829900
]
Maxim Muzafarov commented on CASSANDRA-13855:
-
As I said I have nothing against the HttpSeedProvider, I'll review the current
changes.
Regarding Cloud SeedProviders, I think it should be possible to get a node tag
from the pod metadata in json format as we do now, so we probably don't need to
add any extra dependencies. Once each the seed metadata info is initialized, we
can put that in the ClusterMetadata as we did it for the rack parameter, this
should work, however I haven't tried implementing it on my end.
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/locator/AbstractCloudMetadataServiceSnitch.java#L72
{code}
if (endpoint.equals(FBUtilities.getBroadcastAddressAndPort()))
return getLocalRack();
ClusterMetadata metadata = ClusterMetadata.current();
NodeId nodeId = metadata.directory.peerId(endpoint);
if (nodeId == null)
return DEFAULT_RACK;
return metadata.directory.location(nodeId).rack;
{code}
I'm going to create an issue so we don't mix it up here.
> Implement Http Seed provider
>
>
> Key: CASSANDRA-13855
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13855
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Coordination, Legacy/Core
>Reporter: Jon Haddad
>Assignee: Claude Warren
>Priority: Low
> Labels: lhf
> Fix For: 5.x
>
> Attachments: 0001-Add-URL-Seed-Provider-trunk.txt, signature.asc,
> signature.asc, signature.asc
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Seems like including a dead simple seed provider that can fetch from a URL, 1
> line per seed, would be useful.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19130) Implement transactional table truncation
[
https://issues.apache.org/jira/browse/CASSANDRA-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-19130:
--
Change Category: Operability
Complexity: Normal
Status: Open (was: Triage Needed)
> Implement transactional table truncation
>
>
> Key: CASSANDRA-19130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19130
> Project: Cassandra
> Issue Type: New Feature
> Components: Consistency/Coordination
>Reporter: Marcus Eriksson
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TRUNCATE table should leverage cluster metadata to ensure consistent
> truncation timestamps across all replicas. The current implementation depends
> on all nodes being available, but this could be reimplemented as a
> {{Transformation}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829919#comment-17829919
]
Brandon Williams commented on CASSANDRA-19471:
--
I believe the root cause is that direct mode initially allocates only the block
size on disk (4k in this case) while legacy allocates the full segment size
(1MB). This leaves legacy mode with enough headroom for mutations like schema
changes without changing the on-disk size, while direct mode will reflect them.
This is good news though, because it shows the test is faulty. First, it
shouldn't begin measuring the CL until _after_ the schema is created since
those mutations are always durable. Second, it needs to reset the batch mode
options when it recreates the cluster for the second run, otherwise the CL gets
an asynchronous flush that breaks the test again. I've made those changes
[here|https://github.com/driftx/cassandra-dtest/commit/2499a609b7a8bd750417c3fa56855ab1e9e800f1]
and
[here|https://app.circleci.com/pipelines/github/driftx/cassandra/1536/workflows/150c6753-e905-4503-9f58-f6c7e8357881]
is a circle run with repeats for latest/non-latest to make sure the test still
works, and a full latest gamut to make sure nothing else was broken by the
direct CL changes.
> Commitlog with direct io fails test_change_durable_writes
> -
>
> Key: CASSANDRA-19471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19471
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> With the commitlog_disk_access_mode set to direct, and the improved
> configuration_test.py::TestConfiguration::test_change_durable_writes from
> CASSANDRA-19465, this fails with either:
> {noformat}
> AssertionError: Commitlog was written with durable writes disabled
> {noformat}
> Or what appears to be the original exception reported in CASSANDRA-19465:
> {noformat}
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
> StorageService.java:631 - Stopping native transport
> node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465
> StorageProxy.java:1670 - Failed to apply mutation locally :
> java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
> at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
> at java.base/java.nio.Buffer.position(Buffer.java:316)
> at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
> at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
> at
> org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
> at
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833)
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
> StorageService.java:636 - Stopping gossiper
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19471:
-
Test and Documentation Plan: run CI
Status: Patch Available (was: Open)
> Commitlog with direct io fails test_change_durable_writes
> -
>
> Key: CASSANDRA-19471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19471
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> With the commitlog_disk_access_mode set to direct, and the improved
> configuration_test.py::TestConfiguration::test_change_durable_writes from
> CASSANDRA-19465, this fails with either:
> {noformat}
> AssertionError: Commitlog was written with durable writes disabled
> {noformat}
> Or what appears to be the original exception reported in CASSANDRA-19465:
> {noformat}
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
> StorageService.java:631 - Stopping native transport
> node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465
> StorageProxy.java:1670 - Failed to apply mutation locally :
> java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
> at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
> at java.base/java.nio.Buffer.position(Buffer.java:316)
> at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
> at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
> at
> org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
> at
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833)
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
> StorageService.java:636 - Stopping gossiper
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19488) Ensure snitches always defer to ClusterMetadata
Sam Tunnicliffe created CASSANDRA-19488:
---
Summary: Ensure snitches always defer to ClusterMetadata
Key: CASSANDRA-19488
URL: https://issues.apache.org/jira/browse/CASSANDRA-19488
Project: Cassandra
Issue Type: Improvement
Components: Cluster/Membership, Messaging/Internode, Transactional
Cluster Metadata
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
Internally, C* always uses {{ClusterMetadata}} as the source of topology
information when calculating data placements, replica plans etc and as such the
role of the snitch has been somewhat reduced.
Sorting and comparison functions as provided by specialisations like
{{DynamicEndpointSnitch}} are still used, but the snitch should only be
responsible for providing the DC and rack for a new node when it first joins a
cluster.
Aside from initial startup and registration, snitch implementations should
always defer to {{{}ClusterMetadata{}}}, for DC and rack otherwise there is a
risk that the snitch config drifts out of sync with TCM and output from tools
like {{nodetool ring}} and {{gossipinfo}} becomes incorrect.
A complication is that topology is used when opening connections to peers as
certain internode connection settings are variable at the DC level, so at the
time of connecting we want to check the location of the remote peer. Usually,
this is available from {{{}ClusterMetadata{}}}, but in the case of a brand new
node joining the cluster nothing is known a priori. The current implementation
assumes that the snitch will know the location of the new node ahead of time,
but in practice this is often not the case (though with variants of
{{PropertyFileSnitch}} it _should_ be), and the remote node is temporarily
assigned a default DC. This is problematic as it can cause the internode
connection settings which depend on DC to be incorrectly set. Internode
connections are long lived and any established while the DC is unknown
(potentially with incorrect config) will persist indefinitely. This particular
issue is not directly related to TCM and is present in earlier versions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19488) Ensure snitches always defer to ClusterMetadata
[
https://issues.apache.org/jira/browse/CASSANDRA-19488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-19488:
Change Category: Operability
Complexity: Normal
Fix Version/s: 5.x
Status: Open (was: Triage Needed)
> Ensure snitches always defer to ClusterMetadata
> ---
>
> Key: CASSANDRA-19488
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19488
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Membership, Messaging/Internode, Transactional
> Cluster Metadata
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 5.x
>
>
> Internally, C* always uses {{ClusterMetadata}} as the source of topology
> information when calculating data placements, replica plans etc and as such
> the role of the snitch has been somewhat reduced.
> Sorting and comparison functions as provided by specialisations like
> {{DynamicEndpointSnitch}} are still used, but the snitch should only be
> responsible for providing the DC and rack for a new node when it first joins
> a cluster.
> Aside from initial startup and registration, snitch implementations should
> always defer to {{{}ClusterMetadata{}}}, for DC and rack otherwise there is a
> risk that the snitch config drifts out of sync with TCM and output from tools
> like {{nodetool ring}} and {{gossipinfo}} becomes incorrect.
> A complication is that topology is used when opening connections to peers as
> certain internode connection settings are variable at the DC level, so at the
> time of connecting we want to check the location of the remote peer. Usually,
> this is available from {{{}ClusterMetadata{}}}, but in the case of a brand
> new node joining the cluster nothing is known a priori. The current
> implementation assumes that the snitch will know the location of the new node
> ahead of time, but in practice this is often not the case (though with
> variants of {{PropertyFileSnitch}} it _should_ be), and the remote node is
> temporarily assigned a default DC. This is problematic as it can cause the
> internode connection settings which depend on DC to be incorrectly set.
> Internode connections are long lived and any established while the DC is
> unknown (potentially with incorrect config) will persist indefinitely. This
> particular issue is not directly related to TCM and is present in earlier
> versions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 01/01: Merge branch 'cassandra-5.0' into trunk
This is an automated email from the ASF dual-hosted git repository. maedhroz pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit c50718fd3d96a26e74652616256746baebfb0813 Merge: a69c8657d7 46acaf22e6 Author: Caleb Rackliffe AuthorDate: Fri Mar 22 11:44:40 2024 -0500 Merge branch 'cassandra-5.0' into trunk * cassandra-5.0: Ensure SAI indexes empty byte buffers for types that allow them as a valid input CHANGES.txt| 1 + .../statements/schema/CreateIndexStatement.java| 2 +- .../index/sai/disk/v1/SSTableIndexWriter.java | 3 +- .../cassandra/index/sai/memory/MemtableIndex.java | 5 +- test/unit/accord/utils/Gens.java | 70 +--- .../index/sai/cql/AbstractSimpleEqTestBase.java| 92 + .../index/sai/cql/AllTypesSimpleEqTest.java| 95 ++ .../index/sai/cql/EmptyStringLifecycleTest.java| 79 ++ .../org/apache/cassandra/utils/Generators.java | 9 ++ 9 files changed, 343 insertions(+), 13 deletions(-) diff --cc CHANGES.txt index 7a63df43cc,c056c2785f..1e34cbe71e --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,30 -1,5 +1,31 @@@ -5.0-beta2 +5.1 + * Fix StorageService::constructRangeToEndpointMap for non-distributed keyspaces (CASSANDRA-19255) + * Group nodetool cms commands into single command group (CASSANDRA-19393) + * Register the measurements of the bootstrap process as Dropwizard metrics (CASSANDRA-19447) + * Add LIST SUPERUSERS CQL statement (CASSANDRA-19417) + * Modernize CQLSH datetime conversions (CASSANDRA-18879) + * Harry model and in-JVM tests for partition-restricted 2i queries (CASSANDRA-18275) + * Refactor cqlshmain global constants (CASSANDRA-19201) + * Remove native_transport_port_ssl (CASSANDRA-19397) + * Make nodetool reconfigurecms sync by default and add --cancel to be able to cancel ongoing reconfigurations (CASSANDRA-19216) + * Expose auth mode in system_views.clients, nodetool clientstats, metrics (CASSANDRA-19366) + * Remove sealed_periods and last_sealed_period tables (CASSANDRA-19189) + * Improve setup and initialisation of LocalLog/LogSpec (CASSANDRA-19271) + * Refactor structure of caching metrics and expose auth cache metrics via JMX (CASSANDRA-17062) + * Allow CQL client certificate authentication to work without sending an AUTHENTICATE request (CASSANDRA-18857) + * Extend nodetool tpstats and system_views.thread_pools with detailed pool parameters (CASSANDRA-19289) + * Remove dependency on Sigar in favor of OSHI (CASSANDRA-16565) + * Simplify the bind marker and Term logic (CASSANDRA-18813) + * Limit cassandra startup to supported JDKs, allow higher JDKs by setting CASSANDRA_JDK_UNSUPPORTED (CASSANDRA-18688) + * Standardize nodetool tablestats formatting of data units (CASSANDRA-19104) + * Make nodetool tablestats use number of significant digits for time and average values consistently (CASSANDRA-19015) + * Upgrade jackson to 2.15.3 and snakeyaml to 2.1 (CASSANDRA-18875) + * Transactional Cluster Metadata [CEP-21] (CASSANDRA-18330) + * Add ELAPSED command to cqlsh (CASSANDRA-18861) + * Add the ability to disable bulk loading of SSTables (CASSANDRA-18781) + * Clean up obsolete functions and simplify cql_version handling in cqlsh (CASSANDRA-18787) +Merged from 5.0: + * Ensure SAI indexes empty byte buffers for types that allow them as a valid input (CASSANDRA-19461) * Deprecate Python 3.7 and earlier, but allow cqlsh to run with Python 3.6-3.11 (CASSANDRA-19467) * Set uuid_sstable_identifiers_enabled to true in cassandra-latest.yaml (CASSANDRA-19460) * Revert switching to approxTime in Dispatcher (CASSANDRA-19454) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-5.0 updated: Ensure SAI indexes empty byte buffers for types that allow them as a valid input
This is an automated email from the ASF dual-hosted git repository.
maedhroz pushed a commit to branch cassandra-5.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-5.0 by this push:
new 46acaf22e6 Ensure SAI indexes empty byte buffers for types that allow
them as a valid input
46acaf22e6 is described below
commit 46acaf22e688e7a2e707ac61fd88c96ed33b60d7
Author: Caleb Rackliffe
AuthorDate: Fri Mar 15 17:29:01 2024 -0500
Ensure SAI indexes empty byte buffers for types that allow them as a valid
input
patch by Caleb Rackliffe; reviewed by David Capwell for CASSANDRA-19461
Co-authored-by: Caleb Rackliffe
Co-authored-by: David Capwell
---
CHANGES.txt| 1 +
.../statements/schema/CreateIndexStatement.java| 2 +-
.../index/sai/disk/v1/SSTableIndexWriter.java | 3 +-
.../cassandra/index/sai/memory/MemtableIndex.java | 5 +-
test/unit/accord/utils/Gens.java | 70 +---
.../index/sai/cql/AbstractSimpleEqTestBase.java| 92 +
.../index/sai/cql/AllTypesSimpleEqTest.java| 95 ++
.../index/sai/cql/EmptyStringLifecycleTest.java| 79 ++
.../org/apache/cassandra/utils/Generators.java | 9 ++
9 files changed, 343 insertions(+), 13 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 056c84a82b..c056c2785f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
5.0-beta2
+ * Ensure SAI indexes empty byte buffers for types that allow them as a valid
input (CASSANDRA-19461)
* Deprecate Python 3.7 and earlier, but allow cqlsh to run with Python
3.6-3.11 (CASSANDRA-19467)
* Set uuid_sstable_identifiers_enabled to true in cassandra-latest.yaml
(CASSANDRA-19460)
* Revert switching to approxTime in Dispatcher (CASSANDRA-19454)
diff --git
a/src/java/org/apache/cassandra/cql3/statements/schema/CreateIndexStatement.java
b/src/java/org/apache/cassandra/cql3/statements/schema/CreateIndexStatement.java
index b53e066f90..9f5bbfe058 100644
---
a/src/java/org/apache/cassandra/cql3/statements/schema/CreateIndexStatement.java
+++
b/src/java/org/apache/cassandra/cql3/statements/schema/CreateIndexStatement.java
@@ -242,7 +242,7 @@ public final class CreateIndexStatement extends
AlterSchemaStatement
throw ire(ONLY_PARTITION_KEY, column);
if (column.type.isFrozenCollection() && target.type != Type.FULL)
-throw ire(CREATE_ON_FROZEN_COLUMN, target.type, column, column);
+throw ire(CREATE_ON_FROZEN_COLUMN, target.type, column,
column.name.toCQLString());
if (!column.type.isFrozenCollection() && target.type == Type.FULL)
throw ire(FULL_ON_FROZEN_COLLECTIONS);
diff --git
a/src/java/org/apache/cassandra/index/sai/disk/v1/SSTableIndexWriter.java
b/src/java/org/apache/cassandra/index/sai/disk/v1/SSTableIndexWriter.java
index 3400f81b9d..06bb7d80fe 100644
--- a/src/java/org/apache/cassandra/index/sai/disk/v1/SSTableIndexWriter.java
+++ b/src/java/org/apache/cassandra/index/sai/disk/v1/SSTableIndexWriter.java
@@ -200,7 +200,8 @@ public class SSTableIndexWriter implements
PerColumnIndexWriter
currentBuilder = newSegmentBuilder();
}
-if (term.remaining() == 0) return;
+// Some types support empty byte buffers:
+if (term.remaining() == 0 &&
!index.termType().indexType().allowsEmpty()) return;
if (analyzer == null || !index.termType().isLiteral())
{
diff --git a/src/java/org/apache/cassandra/index/sai/memory/MemtableIndex.java
b/src/java/org/apache/cassandra/index/sai/memory/MemtableIndex.java
index d124af651e..f0f2ea36ad 100644
--- a/src/java/org/apache/cassandra/index/sai/memory/MemtableIndex.java
+++ b/src/java/org/apache/cassandra/index/sai/memory/MemtableIndex.java
@@ -28,6 +28,7 @@ import java.util.function.Function;
import org.apache.cassandra.db.Clustering;
import org.apache.cassandra.db.DecoratedKey;
import org.apache.cassandra.db.PartitionPosition;
+import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.dht.AbstractBounds;
import org.apache.cassandra.index.sai.QueryContext;
import org.apache.cassandra.index.sai.StorageAttachedIndex;
@@ -46,10 +47,12 @@ public class MemtableIndex implements MemtableOrdering
private final MemoryIndex memoryIndex;
private final LongAdder writeCount = new LongAdder();
private final LongAdder estimatedMemoryUsed = new LongAdder();
+private final AbstractType type;
public MemtableIndex(StorageAttachedIndex index)
{
this.memoryIndex = index.termType().isVector() ? new
VectorMemoryIndex(index) : new TrieMemoryIndex(index);
+this.type = index.termType().indexType();
}
public long writeCount()
@@ -79,7 +82,7 @@ public class MemtableIndex implements MemtableOrdering
public long index(Decorat
(cassandra) branch trunk updated (a69c8657d7 -> c50718fd3d)
This is an automated email from the ASF dual-hosted git repository. maedhroz pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from a69c8657d7 Fix getRangeTo* operations of StorageService mbean for local strategy keyspaces new 46acaf22e6 Ensure SAI indexes empty byte buffers for types that allow them as a valid input new c50718fd3d Merge branch 'cassandra-5.0' into trunk The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../statements/schema/CreateIndexStatement.java| 2 +- .../index/sai/disk/v1/SSTableIndexWriter.java | 3 +- .../cassandra/index/sai/memory/MemtableIndex.java | 5 +- test/unit/accord/utils/Gens.java | 70 +--- .../index/sai/cql/AbstractSimpleEqTestBase.java| 92 + .../index/sai/cql/AllTypesSimpleEqTest.java| 95 ++ .../index/sai/cql/EmptyStringLifecycleTest.java| 79 ++ .../org/apache/cassandra/utils/Generators.java | 9 ++ 9 files changed, 343 insertions(+), 13 deletions(-) create mode 100644 test/unit/org/apache/cassandra/index/sai/cql/AbstractSimpleEqTestBase.java create mode 100644 test/unit/org/apache/cassandra/index/sai/cql/AllTypesSimpleEqTest.java create mode 100644 test/unit/org/apache/cassandra/index/sai/cql/EmptyStringLifecycleTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19461) SAI does not index empty bytes even for types that allow empty bytes as a valid input
[
https://issues.apache.org/jira/browse/CASSANDRA-19461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-19461:
Fix Version/s: 5.0-beta2
5.1
(was: 5.x)
(was: 5.0-rc)
Since Version: 5.0-alpha1
Source Control Link:
https://github.com/apache/cassandra/commit/46acaf22e688e7a2e707ac61fd88c96ed33b60d7
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed as
https://github.com/apache/cassandra/commit/46acaf22e688e7a2e707ac61fd88c96ed33b60d7
> SAI does not index empty bytes even for types that allow empty bytes as a
> valid input
> -
>
> Key: CASSANDRA-19461
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19461
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0-beta2, 5.1
>
> Attachments: ci_summary.html, result_details.tar.gz
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> This is easy to reproduce with a test that looks something like this:
> {noformat}
> @Test
> public void testEmptyString()
> {
> createTable("CREATE TABLE %s (k TEXT PRIMARY KEY, v text)");
> createIndex(String.format(CREATE_INDEX_TEMPLATE, 'v'));
> execute("INSERT INTO %s (k, v) VALUES ('0', '')");
> execute("INSERT INTO %s (k) VALUES ('1')");
>
> // flush(); < there is not always a memtable index involved, a fix
> will have to pay attention to this
> List rows = executeNet("SELECT * FROM %s WHERE v = ''").all();
> assertEquals(1, rows.size()); <— FAILS! No matches...
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13855) Implement Http Seed provider
[ https://issues.apache.org/jira/browse/CASSANDRA-13855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829941#comment-17829941 ] Sam Tunnicliffe commented on CASSANDRA-13855: - Not completely relevant to the discussion here, but slightly adjacent... the roles of both snitches and seeds have changed slightly in trunk. See CASSANDRA-XXX which I finally remembered to file for more detail on snitches. Seeds are now really only necessary as initial contact points for joining nodes. Technically, they do still perform the same function regarding gossip convergence, but that it is way less important/relevant now as we don't rely on gossip state for correctness. Of course, this doesn't make the goal of this JIRA any less valid, I just thought I should mention it. > Implement Http Seed provider > > > Key: CASSANDRA-13855 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13855 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Coordination, Legacy/Core >Reporter: Jon Haddad >Assignee: Claude Warren >Priority: Low > Labels: lhf > Fix For: 5.x > > Attachments: 0001-Add-URL-Seed-Provider-trunk.txt, signature.asc, > signature.asc, signature.asc > > Time Spent: 0.5h > Remaining Estimate: 0h > > Seems like including a dead simple seed provider that can fetch from a URL, 1 > line per seed, would be useful. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13855) Implement Http Seed provider
[ https://issues.apache.org/jira/browse/CASSANDRA-13855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829941#comment-17829941 ] Sam Tunnicliffe edited comment on CASSANDRA-13855 at 3/22/24 5:27 PM: -- Not completely relevant to the discussion here, but slightly adjacent... the roles of both snitches and seeds have changed slightly in trunk. See CASSANDRA-19488 which I finally remembered to file for more detail on snitches. Seeds are now really only necessary as initial contact points for joining nodes. Technically, they do still perform the same function regarding gossip convergence, but that it is way less important/relevant now as we don't rely on gossip state for correctness. Of course, this doesn't make the goal of this JIRA any less valid, I just thought I should mention it. was (Author: beobal): Not completely relevant to the discussion here, but slightly adjacent... the roles of both snitches and seeds have changed slightly in trunk. See CASSANDRA-XXX which I finally remembered to file for more detail on snitches. Seeds are now really only necessary as initial contact points for joining nodes. Technically, they do still perform the same function regarding gossip convergence, but that it is way less important/relevant now as we don't rely on gossip state for correctness. Of course, this doesn't make the goal of this JIRA any less valid, I just thought I should mention it. > Implement Http Seed provider > > > Key: CASSANDRA-13855 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13855 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Coordination, Legacy/Core >Reporter: Jon Haddad >Assignee: Claude Warren >Priority: Low > Labels: lhf > Fix For: 5.x > > Attachments: 0001-Add-URL-Seed-Provider-trunk.txt, signature.asc, > signature.asc, signature.asc > > Time Spent: 0.5h > Remaining Estimate: 0h > > Seems like including a dead simple seed provider that can fetch from a URL, 1 > line per seed, would be useful. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19477) Do not go to disk to get HintsStore.getTotalFileSize
[
https://issues.apache.org/jira/browse/CASSANDRA-19477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829945#comment-17829945
]
Stefan Miklosovic commented on CASSANDRA-19477:
---
[CASSANDRA-19477-5.0|https://github.com/instaclustr/cassandra/tree/CASSANDRA-19477-5.0]
{noformat}
java17_pre-commit_tests
✓ j17_build3m 49s
✓ j17_cqlsh_dtests_py3116m 4s
✓ j17_cqlsh_dtests_py311_vnode 6m 13s
✓ j17_cqlsh_dtests_py38 6m 3s
✓ j17_cqlsh_dtests_py38_vnode 6m 20s
✓ j17_cqlshlib_cython_tests7m 25s
✓ j17_cqlshlib_tests 6m 27s
✓ j17_dtests 32m 3s
✓ j17_jvm_dtests 23m 21s
✓ j17_jvm_dtests_latest_vnode 14m 2s
✓ j17_jvm_dtests_latest_vnode_repeat 40m 42s
✓ j17_jvm_dtests_repeat 39m 52s
✓ j17_unit_tests 17m 44s
✓ j17_unit_tests_repeat2m 28s
✓ j17_utests_latest 15m 45s
✓ j17_utests_latest_repeat 2m 34s
✓ j17_utests_oa_repeat 0m 13s
✕ j17_dtests_latest 34m 34s
configuration_test.TestConfiguration test_change_durable_writes
✕ j17_dtests_vnode 32m 7s
✕ j17_utests_oa 15m 57s
org.apache.cassandra.net.ConnectionTest testTimeout
java17_separate_tests
java11_pre-commit_tests
✓ j11_build6m 59s
✓ j11_cqlsh_dtests_py311 9m 42s
✓ j11_cqlsh_dtests_py311_vnode 7m 41s
✓ j11_cqlsh_dtests_py388m 20s
✓ j11_cqlsh_dtests_py38_vnode 7m 59s
✓ j11_cqlshlib_cython_tests 11m 40s
✓ j11_cqlshlib_tests 9m 16s
✓ j11_dtests 38m 39s
✓ j11_dtests_vnode35m 20s
✓ j11_jvm_dtests 23m 29s
✓ j11_jvm_dtests_latest_vnode 14m 40s
✓ j11_jvm_dtests_latest_vnode_repeat 47m 40s
✓ j11_jvm_dtests_repeat 41m 59s
✓ j11_simulator_dtests 5m 54s
✓ j11_unit_tests 19m 38s
✓ j11_unit_tests_repeat3m 26s
✓ j11_utests_latest 21m 24s
✓ j11_utests_latest_repeat 3m 46s
✓ j11_utests_oa 21m 15s
✓ j11_utests_oa_repeat 8m 28s
✓ j11_utests_system_keyspace_directory16m 17s
✓ j11_utests_system_keyspace_directory_repeat 3m 58s
✓ j17_cqlsh_dtests_py311 5m 56s
✓ j17_cqlsh_dtests_py311_vnode 6m 46s
✓ j17_cqlsh_dtests_py38 6m 9s
✓ j17_cqlsh_dtests_py38_vnode 6m 55s
✓ j17_cqlshlib_cython_tests7m 32s
✓ j17_cqlshlib_tests 6m 27s
✓ j17_dtests 33m 37s
✓ j17_dtests_vnode32m 31s
✓ j17_jvm_dtests 23m 16s
✓ j17_jvm_dtests_latest_vnode 13m 28s
✓ j17_jvm_dtests_latest_vnode_repeat 40m 42s
✓ j17_jvm_dtests_repeat 41m 28s
✓ j17_unit_tests 14m 40s
✓ j17_unit_tests_repeat0m 16s
✓ j17_utests_latest_repeat 0m 14s
✓ j17_utests_oa 15m 51s
✓ j17_utests_oa_repeat 7m 58s
✕ j11_dtests_latest35m 2s
configuration_test.TestConfiguration test_change_durable_writes
✕ j17_dtests_latest 33m 55s
configuration_test.TestConfiguration test_change_durable_writes
✕ j17_utests_latest 16m 38s
org.apache.cassandra.cql3.validation.operations.SelectTest
testCreatingUDFWithSameNameAsBuiltin_PrefersCompatibleArgs
org.apache.cassandra.cql3.validation.operations.SelectTest
testCreatingUDFWithSameNameAsBuiltin_FullyQualifiedFunctionNameWorks
java11_separate_tests
{noformat}
[java17_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/4063/workflows/15b9eab1-70d5-4490-836a-49cc9169c2aa]
[java17_separate_tests
[jira] [Updated] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-19484:
---
Reviewers: Berenguer Blasi (was: Berenguer Blasi, Joshua McKenzie)
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19448) CommitlogArchiver only has granularity to seconds for restore_point_in_time
[ https://issues.apache.org/jira/browse/CASSANDRA-19448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829960#comment-17829960 ] Tiago L. Alves commented on CASSANDRA-19448: I have an even simpler (and more correct) patch for this using `DateTimeFormatter`. The first patch would parse both "2024:03:22 18:31:22.1" and "2024:03:22 18:31:22.1" as "2024:03:22 18:31:22.001" which is incorrect. I'll provide an updated patch with tests later. > CommitlogArchiver only has granularity to seconds for restore_point_in_time > --- > > Key: CASSANDRA-19448 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19448 > Project: Cassandra > Issue Type: Bug > Components: Local/Commit Log >Reporter: Jeremy Hanna >Assignee: Maxwell Guo >Priority: Normal > Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > Commitlog archiver allows users to backup commitlog files for the purpose of > doing point in time restores. The [configuration > file|https://github.com/apache/cassandra/blob/trunk/conf/commitlog_archiving.properties] > gives an example of down to the seconds granularity but then asks what > whether the timestamps are microseconds or milliseconds - defaulting to > microseconds. Because the [CommitLogArchiver uses a second based date > format|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java#L52], > if a user specifies to restore at something at a lower granularity like > milliseconds or microseconds, that means that the it will truncate everything > after the second and restore to that second. So say you specify a > restore_point_in_time like this: > restore_point_in_time=2024:01:18 17:01:01.623392 > it will silently truncate everything after the 01 seconds. So effectively to > the user, it is missing updates between 01 and 01.623392. > This appears to be a bug in the intent. We should allow users to specify > down to the millisecond or even microsecond level. If we allow them to > specify down to microseconds for the restore point in time, then it may > internally need to change from a long. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19333) Fix decode in VectorCodec
[ https://issues.apache.org/jira/browse/CASSANDRA-19333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829964#comment-17829964 ] Ekaterina Dimitrova commented on CASSANDRA-19333: - 2 approvals received in GH. CI was run by [~absurdfarce] I force-pushed the branch with updated commit message as per Slack conversation. Ready to commit. Thank you > Fix decode in VectorCodec > - > > Key: CASSANDRA-19333 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19333 > Project: Cassandra > Issue Type: Bug > Components: Client/java-driver >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > > We made a server-side fix in CASSANDRA-19167 for VectorCodec. > This needs to be checked also in the driver [here > |https://github.com/apache/cassandra-java-driver/blob/8d5849cb38995b312f29314d18256c0c3e94cf07/core/src/main/java/com/datastax/oss/driver/internal/core/type/codec/VectorCodec.java#L130-L139] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-java-driver) branch 4.x updated: Fix data corruption in VectorCodec when using heap buffers
This is an automated email from the ASF dual-hosted git repository.
absurdfarce pushed a commit to branch 4.x
in repository https://gitbox.apache.org/repos/asf/cassandra-java-driver.git
The following commit(s) were added to refs/heads/4.x by this push:
new 40a9a49d5 Fix data corruption in VectorCodec when using heap buffers
40a9a49d5 is described below
commit 40a9a49d50fac6abed2a5bb2cc2627e4085a399b
Author: Ekaterina Dimitrova
AuthorDate: Mon Jan 29 14:07:59 2024 -0500
Fix data corruption in VectorCodec when using heap buffers
patch by Ekaterina Dimitrova; reviewed by Alexandre Dutra and Bret McGuire
for CASSANDRA-19333
---
.../oss/driver/internal/core/type/codec/VectorCodec.java | 14 --
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git
a/core/src/main/java/com/datastax/oss/driver/internal/core/type/codec/VectorCodec.java
b/core/src/main/java/com/datastax/oss/driver/internal/core/type/codec/VectorCodec.java
index 1b663a29d..2c4d2200b 100644
---
a/core/src/main/java/com/datastax/oss/driver/internal/core/type/codec/VectorCodec.java
+++
b/core/src/main/java/com/datastax/oss/driver/internal/core/type/codec/VectorCodec.java
@@ -127,17 +127,19 @@ public class VectorCodec
implements TypeCodec rv = new ArrayList(cqlType.getDimensions());
for (int i = 0; i < cqlType.getDimensions(); ++i) {
- ByteBuffer slice = bytes.slice();
- slice.limit(elementSize);
+ // Set the limit for the current element
+ int originalPosition = slice.position();
+ slice.limit(originalPosition + elementSize);
rv.add(this.subtypeCodec.decode(slice, protocolVersion));
- bytes.position(bytes.position() + elementSize);
+ // Move to the start of the next element
+ slice.position(originalPosition + elementSize);
+ // Reset the limit to the end of the buffer
+ slice.limit(slice.capacity());
}
-/* Restore the input ByteBuffer to its original state */
-bytes.rewind();
-
return CqlVector.newInstance(rv);
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] Fix decode in VectorCodec [cassandra-java-driver]
absurdfarce merged PR #1909: URL: https://github.com/apache/cassandra-java-driver/pull/1909 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19333) Fix decode in VectorCodec
[ https://issues.apache.org/jira/browse/CASSANDRA-19333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829966#comment-17829966 ] Bret McGuire commented on CASSANDRA-19333: -- Committed as [40a9a49d50fac6abed2a5bb2cc2627e4085a399b|https://github.com/apache/cassandra-java-driver/commit/40a9a49d50fac6abed2a5bb2cc2627e4085a399b]. Huge thank you to [~e.dimitrova] for making this happen! > Fix decode in VectorCodec > - > > Key: CASSANDRA-19333 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19333 > Project: Cassandra > Issue Type: Bug > Components: Client/java-driver >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > > We made a server-side fix in CASSANDRA-19167 for VectorCodec. > This needs to be checked also in the driver [here > |https://github.com/apache/cassandra-java-driver/blob/8d5849cb38995b312f29314d18256c0c3e94cf07/core/src/main/java/com/datastax/oss/driver/internal/core/type/codec/VectorCodec.java#L130-L139] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19489) Guardrail to warn clients about possible transient incorrect responses for filtering queries against multiple mutable columns
[ https://issues.apache.org/jira/browse/CASSANDRA-19489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-19489: Fix Version/s: 5.0-rc 5.1 > Guardrail to warn clients about possible transient incorrect responses for > filtering queries against multiple mutable columns > - > > Key: CASSANDRA-19489 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19489 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Coordination, CQL/Semantics, Messaging/Client >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 5.0-rc, 5.1 > > > Given we may not have time to fully resolve CASSANDRA-19007 before we release > 5.0, it would still be helpful to have, at the very minimum, a client warning > for cases where a user filters on two or more mutable (static or regular) > columns at consistency levels that require coordinator reconciliation. We may > also want the option to fail these queries outright, although that need not > be the default. > The only art involved in this is deciding what we want to say in the > warning/error message. It's probably reasonable to mention there that this > only happens when we have unrepaired data. It's also worth noting that SAI > queries are no longer vulnerable to this after the resolution of > CASSANDRA-19018. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19489) Guardrail to warn clients about possible transient incorrect responses for filtering queries against multiple mutable columns
Caleb Rackliffe created CASSANDRA-19489: --- Summary: Guardrail to warn clients about possible transient incorrect responses for filtering queries against multiple mutable columns Key: CASSANDRA-19489 URL: https://issues.apache.org/jira/browse/CASSANDRA-19489 Project: Cassandra Issue Type: Improvement Components: Consistency/Coordination, CQL/Semantics, Messaging/Client Reporter: Caleb Rackliffe Assignee: Caleb Rackliffe Given we may not have time to fully resolve CASSANDRA-19007 before we release 5.0, it would still be helpful to have, at the very minimum, a client warning for cases where a user filters on two or more mutable (static or regular) columns at consistency levels that require coordinator reconciliation. We may also want the option to fail these queries outright, although that need not be the default. The only art involved in this is deciding what we want to say in the warning/error message. It's probably reasonable to mention there that this only happens when we have unrepaired data. It's also worth noting that SAI queries are no longer vulnerable to this after the resolution of CASSANDRA-19018. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19007) Queries with multi-column replica-side filtering can miss rows
[
https://issues.apache.org/jira/browse/CASSANDRA-19007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829967#comment-17829967
]
Caleb Rackliffe commented on CASSANDRA-19007:
-
Just created CASSANDRA-19489 to track this.
> Queries with multi-column replica-side filtering can miss rows
> --
>
> Key: CASSANDRA-19007
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19007
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Andres de la Peña
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{SELECT}} queries with multi-column replica-side filtering can miss rows if
> the filtered columns are spread across out-of-sync replicas. This dtest
> reproduces the issue:
> {code:java}
> @Test
> public void testMultiColumnReplicaSideFiltering() throws IOException
> {
> try (Cluster cluster = init(Cluster.build().withNodes(2).start()))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (k int PRIMARY
> KEY, a int, b int)"));
> // insert a split row
> cluster.get(1).executeInternal(withKeyspace("INSERT INTO %s.t(k, a)
> VALUES (0, 1)"));
> cluster.get(2).executeInternal(withKeyspace("INSERT INTO %s.t(k, b)
> VALUES (0, 2)"));
> String select = withKeyspace("SELECT * FROM %s.t WHERE a = 1 AND b =
> 2 ALLOW FILTERING");
> Object[][] initialRows = cluster.coordinator(1).execute(select, ALL);
> assertRows(initialRows, row(0, 1, 2)); // not found!!
> }
> }
> {code}
> This edge case affects queries using {{ALLOW FILTERING}} or any index
> implementation.
> It affects all branches since multi-column replica-side filtering queries
> were introduced, long before 3.0.
> The protection mechanism added by CASSANDRA-8272/8273 won't deal with this
> case, since it only solves single-column conflicts where stale rows could
> resurrect. This bug however doesn't resurrect data, it can only miss rows
> while the replicas are out-of-sync.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19489) Guardrail to warn clients about possible transient incorrect responses for filtering queries against multiple mutable columns
[ https://issues.apache.org/jira/browse/CASSANDRA-19489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-19489: Change Category: Operability Complexity: Low Hanging Fruit Status: Open (was: Triage Needed) > Guardrail to warn clients about possible transient incorrect responses for > filtering queries against multiple mutable columns > - > > Key: CASSANDRA-19489 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19489 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Coordination, CQL/Semantics, Messaging/Client >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 5.0-rc, 5.1 > > > Given we may not have time to fully resolve CASSANDRA-19007 before we release > 5.0, it would still be helpful to have, at the very minimum, a client warning > for cases where a user filters on two or more mutable (static or regular) > columns at consistency levels that require coordinator reconciliation. We may > also want the option to fail these queries outright, although that need not > be the default. > The only art involved in this is deciding what we want to say in the > warning/error message. It's probably reasonable to mention there that this > only happens when we have unrepaired data. It's also worth noting that SAI > queries are no longer vulnerable to this after the resolution of > CASSANDRA-19018. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19490) Add foundation for independent parsing of junit based output for CI reporting to cassandra-builds
Josh McKenzie created CASSANDRA-19490: - Summary: Add foundation for independent parsing of junit based output for CI reporting to cassandra-builds Key: CASSANDRA-19490 URL: https://issues.apache.org/jira/browse/CASSANDRA-19490 Project: Cassandra Issue Type: New Feature Components: CI Reporter: Josh McKenzie Assignee: Josh McKenzie PR attached. For doing CI ourselves, it's useful to have a single pane of glass report where you have a summary of results for all your suites as well as inlined failures. This should be agnostic to any xunit based output; so long as we co-locate the xunit data in directories adjacent to one another, the script in the PR will generate an in-memory representation of the xunit results as well as inline failures to an existing .html file. The contents will need to be tweaked a bit to generate the top level branch + sha + checkstyle + summaries information, but the vast majority of that is already parsed and easily available within the script and can be extended pretty trivially. Opening up a pr to pull this into [cassandra-builds](https://github.com/apache/cassandra-builds) since [~mck] is actively working on that and needs these primitives. I'd expect the contents in ci_parser to be massaged to become a more finalized, full solution before we start to use it but no harm in the incremental merge. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19491) decommissioned nodes show as UNREACHABLE in describecluster afterward
[ https://issues.apache.org/jira/browse/CASSANDRA-19491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19491: - Bug Category: Parent values: Correctness(12982) Complexity: Normal Component/s: Cluster/Membership Discovered By: User Report Fix Version/s: 4.1.x 5.0-rc Severity: Normal Status: Open (was: Triage Needed) Eric's cluster is on 4.1, I have repro'd against 5.0 > decommissioned nodes show as UNREACHABLE in describecluster afterward > - > > Key: CASSANDRA-19491 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19491 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Membership >Reporter: Brandon Williams >Priority: Normal > Fix For: 4.1.x, 5.0-rc > > > [~urandom] noticed this happening in his cluster and I have been able to > reproduce with a modified dtest. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19491) decommissioned nodes show as UNREACHABLE in describecluster afterward
Brandon Williams created CASSANDRA-19491: Summary: decommissioned nodes show as UNREACHABLE in describecluster afterward Key: CASSANDRA-19491 URL: https://issues.apache.org/jira/browse/CASSANDRA-19491 Project: Cassandra Issue Type: Bug Reporter: Brandon Williams [~urandom] noticed this happening in his cluster and I have been able to reproduce with a modified dtest. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19491) decommissioned nodes show as UNREACHABLE in describecluster afterward
[ https://issues.apache.org/jira/browse/CASSANDRA-19491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829976#comment-17829976 ] Brandon Williams commented on CASSANDRA-19491: -- My dtest branch is [here|https://github.com/driftx/cassandra-dtest/tree/CASSANDRA-19491] that reproduces. > decommissioned nodes show as UNREACHABLE in describecluster afterward > - > > Key: CASSANDRA-19491 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19491 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Membership >Reporter: Brandon Williams >Priority: Normal > Fix For: 4.1.x, 5.0-rc > > > [~urandom] noticed this happening in his cluster and I have been able to > reproduce with a modified dtest. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19491) decommissioned nodes show as UNREACHABLE in describecluster afterward
[ https://issues.apache.org/jira/browse/CASSANDRA-19491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19491: - Fix Version/s: 5.0.x (was: 5.0-rc) > decommissioned nodes show as UNREACHABLE in describecluster afterward > - > > Key: CASSANDRA-19491 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19491 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Membership >Reporter: Brandon Williams >Priority: Normal > Fix For: 4.1.x, 5.0.x > > > [~urandom] noticed this happening in his cluster and I have been able to > reproduce with a modified dtest. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19491) decommissioned nodes show as UNREACHABLE in describecluster afterward
[ https://issues.apache.org/jira/browse/CASSANDRA-19491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19491: - Fix Version/s: 4.0.x > decommissioned nodes show as UNREACHABLE in describecluster afterward > - > > Key: CASSANDRA-19491 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19491 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Membership >Reporter: Brandon Williams >Priority: Normal > Fix For: 4.0.x, 4.1.x, 5.0.x > > > [~urandom] noticed this happening in his cluster and I have been able to > reproduce with a modified dtest. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch trunk updated (c50718fd3d -> ee75a0d95e)
This is an automated email from the ASF dual-hosted git repository. aweisberg pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from c50718fd3d Merge branch 'cassandra-5.0' into trunk add 38eb339557 Add support for providing nvdDatafeedUrl to OWASP add de2a965a18 Merge branch 'cassandra-3.0' into cassandra-3.11 add 5dd9213149 Merge branch 'cassandra-3.11' into cassandra-4.0 add 96692d7206 Merge branch 'cassandra-4.0' into cassandra-4.1 add fdfc0019de Merge branch 'cassandra-4.1' into cassandra-5.0 new ee75a0d95e Merge branch 'cassandra-5.0' into trunk The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .build/build-owasp.xml | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-3.0 updated (98eea87aa5 -> 38eb339557)
This is an automated email from the ASF dual-hosted git repository. aweisberg pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 98eea87aa5 Fix SCM URL links add 38eb339557 Add support for providing nvdDatafeedUrl to OWASP No new revisions were added by this update. Summary of changes: .build/build-owasp.xml | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-4.0 updated (a2fbb17867 -> 5dd9213149)
This is an automated email from the ASF dual-hosted git repository. aweisberg pushed a change to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from a2fbb17867 Push LocalSessions info logs to debug add 38eb339557 Add support for providing nvdDatafeedUrl to OWASP add de2a965a18 Merge branch 'cassandra-3.0' into cassandra-3.11 add 5dd9213149 Merge branch 'cassandra-3.11' into cassandra-4.0 No new revisions were added by this update. Summary of changes: .build/build-owasp.xml | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-3.11 updated (099fdf2673 -> de2a965a18)
This is an automated email from the ASF dual-hosted git repository. aweisberg pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 099fdf2673 Move ClientWarn.State#warnings to a thread-safe list add 38eb339557 Add support for providing nvdDatafeedUrl to OWASP add de2a965a18 Merge branch 'cassandra-3.0' into cassandra-3.11 No new revisions were added by this update. Summary of changes: .build/build-owasp.xml | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-19484:
---
Source Control Link:
https://github.com/apache/cassandra/commit/c8fbb97ab04142f9b49fe86017b808ff3e35c10a
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-19484:
---
Status: Ready to Commit (was: Review In Progress)
It's clean on all the branches.
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 01/01: Merge branch 'cassandra-5.0' into trunk
This is an automated email from the ASF dual-hosted git repository. aweisberg pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit ee75a0d95e6d7103a8d1874b5a1134f6e2816d56 Merge: c50718fd3d fdfc0019de Author: Ariel Weisberg AuthorDate: Fri Mar 22 16:50:40 2024 -0400 Merge branch 'cassandra-5.0' into trunk .build/build-owasp.xml | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-5.0 updated (46acaf22e6 -> fdfc0019de)
This is an automated email from the ASF dual-hosted git repository. aweisberg pushed a change to branch cassandra-5.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 46acaf22e6 Ensure SAI indexes empty byte buffers for types that allow them as a valid input add 38eb339557 Add support for providing nvdDatafeedUrl to OWASP add de2a965a18 Merge branch 'cassandra-3.0' into cassandra-3.11 add 5dd9213149 Merge branch 'cassandra-3.11' into cassandra-4.0 add 96692d7206 Merge branch 'cassandra-4.0' into cassandra-4.1 add fdfc0019de Merge branch 'cassandra-4.1' into cassandra-5.0 No new revisions were added by this update. Summary of changes: .build/build-owasp.xml | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-4.1 updated (e4ae1f3a4f -> 96692d7206)
This is an automated email from the ASF dual-hosted git repository. aweisberg pushed a change to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git from e4ae1f3a4f Fix system_views.settings to handle array types add 38eb339557 Add support for providing nvdDatafeedUrl to OWASP add de2a965a18 Merge branch 'cassandra-3.0' into cassandra-3.11 add 5dd9213149 Merge branch 'cassandra-3.11' into cassandra-4.0 add 96692d7206 Merge branch 'cassandra-4.0' into cassandra-4.1 No new revisions were added by this update. Summary of changes: .build/build-owasp.xml | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-19484:
---
Status: Review In Progress (was: Patch Available)
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-19484:
---
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-19484:
---
Fix Version/s: 3.0.30
3.11.17
4.0.13
4.1.5
5.0
5.1
(was: 3.0.x)
(was: 3.11.x)
(was: 5.x)
(was: 4.0.x)
(was: 4.1.x)
(was: 5.0.x)
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.30, 3.11.17, 4.0.13, 4.1.5, 5.0, 5.1
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19491) decommissioned nodes show as UNREACHABLE in describecluster afterward
[ https://issues.apache.org/jira/browse/CASSANDRA-19491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19491: - Resolution: Invalid Status: Resolved (was: Open) I think this is normal until a state is evicted, since this happens on 3.0/3.11 too. > decommissioned nodes show as UNREACHABLE in describecluster afterward > - > > Key: CASSANDRA-19491 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19491 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Membership >Reporter: Brandon Williams >Priority: Normal > Fix For: 4.0.x, 4.1.x, 5.0.x > > > [~urandom] noticed this happening in his cluster and I have been able to > reproduce with a modified dtest. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19477) Do not go to disk to get HintsStore.getTotalFileSize
[
https://issues.apache.org/jira/browse/CASSANDRA-19477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829991#comment-17829991
]
Jon Haddad commented on CASSANDRA-19477:
Awesome. I'll fire up a test with the 4.1 branch, since that's what I tested
before, and post my findings.
> Do not go to disk to get HintsStore.getTotalFileSize
>
>
> Key: CASSANDRA-19477
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19477
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Hints
>Reporter: Jon Haddad
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.1.x, 5.0-rc, 5.x
>
> Attachments: flamegraph.cpu.html
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> When testing a cluster with more requests than it could handle, I noticed
> significant CPU time (25%) spent in HintsStore.getTotalFileSize. Here's what
> I'm seeing from profiling:
> 10% of CPU time spent in HintsDescriptor.fileName which only does this:
>
> {noformat}
> return String.format("%s-%s-%s.hints", hostId, timestamp, version);{noformat}
> At a bare minimum here we should create this string up front with the host
> and version and eliminate 2 of the 3 substitutions, but I think it's probably
> faster to use a StringBuilder and avoid the underlying regular expression
> altogether.
> 12% of the time is spent in org.apache.cassandra.io.util.File.length. It
> looks like this is called once for each hint file on disk for each host we're
> hinting to. In the case of an overloaded cluster, this is significant. It
> would be better if we were to track the file size in memory for each hint
> file and reference that rather than go to the filesystem.
> These fairly small changes should make Cassandra more reliable when under
> load spikes.
> CPU Flame graph attached.
> I only tested this in 4.1 but it looks like this is present up to trunk.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19418) [Analytics] Report additional bulk analytics job stats for instrumentation
[ https://issues.apache.org/jira/browse/CASSANDRA-19418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829992#comment-17829992 ] Yifan Cai commented on CASSANDRA-19418: --- +1 > [Analytics] Report additional bulk analytics job stats for instrumentation > -- > > Key: CASSANDRA-19418 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19418 > Project: Cassandra > Issue Type: Task > Components: Analytics Library >Reporter: Arjun Ashok >Assignee: Arjun Ashok >Priority: Normal > Time Spent: 3h > Remaining Estimate: 0h > > Currently, the Cassandra bulk analytics library supports a "dialHome" API to > publish some initial job metadata, which in its current form, is redirected > to a log. The intention behind this is to allow custom implementations that > can utilize these summarized stats for instrumentation or reporting of client > behavior. > This task is meant to enhance this API to allow for additional job metadata > to be published both at the Spark executor level and at the task levels to > gather stats such as "success/failure", "number of rows written/read", > "failure reason" etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19448) CommitlogArchiver only has granularity to seconds for restore_point_in_time
[ https://issues.apache.org/jira/browse/CASSANDRA-19448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1782#comment-1782 ] Tiago L. Alves commented on CASSANDRA-19448: I've simplified my original patched (fixing millisecond parsing), removed the warnings logic that was wrong, and added a test. I can prepare a patch for previous Cassandra versions if this change is accepted. > CommitlogArchiver only has granularity to seconds for restore_point_in_time > --- > > Key: CASSANDRA-19448 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19448 > Project: Cassandra > Issue Type: Bug > Components: Local/Commit Log >Reporter: Jeremy Hanna >Assignee: Maxwell Guo >Priority: Normal > Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > Commitlog archiver allows users to backup commitlog files for the purpose of > doing point in time restores. The [configuration > file|https://github.com/apache/cassandra/blob/trunk/conf/commitlog_archiving.properties] > gives an example of down to the seconds granularity but then asks what > whether the timestamps are microseconds or milliseconds - defaulting to > microseconds. Because the [CommitLogArchiver uses a second based date > format|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java#L52], > if a user specifies to restore at something at a lower granularity like > milliseconds or microseconds, that means that the it will truncate everything > after the second and restore to that second. So say you specify a > restore_point_in_time like this: > restore_point_in_time=2024:01:18 17:01:01.623392 > it will silently truncate everything after the 01 seconds. So effectively to > the user, it is missing updates between 01 and 01.623392. > This appears to be a bug in the intent. We should allow users to specify > down to the millisecond or even microsecond level. If we allow them to > specify down to microseconds for the restore point in time, then it may > internally need to change from a long. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-analytics) branch trunk updated: CASSANDRA-19418 - Changes to report additional bulk analytics job stats for instrumentation (#41)
This is an automated email from the ASF dual-hosted git repository.
frankgh pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-analytics.git
The following commit(s) were added to refs/heads/trunk by this push:
new 164243e CASSANDRA-19418 - Changes to report additional bulk
analytics job stats for instrumentation (#41)
164243e is described below
commit 164243e78f1557a34bc699ebc716b532781d6422
Author: Arjun Ashok
AuthorDate: Fri Mar 22 16:22:44 2024 -0700
CASSANDRA-19418 - Changes to report additional bulk analytics job stats
for instrumentation (#41)
Patch by Arjun Ashok; Reviewed by Doug Rohrer, Yifan Cai, Francisco
Guerrero for CASSANDRA-19418
---
CHANGES.txt| 1 +
.../spark/bulkwriter/BulkWriterContext.java| 4 ++
.../bulkwriter/CassandraBulkSourceRelation.java| 84 +++---
.../bulkwriter/CassandraBulkWriterContext.java | 45
.../cassandra/spark/bulkwriter/RecordWriter.java | 13 ++--
.../cassandra/spark/bulkwriter/RingInstance.java | 28
.../cassandra/spark/bulkwriter/SSTableWriter.java | 11 +++
.../cassandra/spark/bulkwriter/StreamResult.java | 20 --
.../cassandra/spark/bulkwriter/StreamSession.java | 14 +++-
.../cassandra/spark/bulkwriter/WriteResult.java| 62
.../stats/JobStatsPublisher.java} | 26 ---
.../stats/LogStatsPublisher.java} | 29
.../spark/bulkwriter/MockBulkWriterContext.java| 15 +++-
.../spark/bulkwriter/RecordWriterTest.java | 6 +-
.../bulkwriter/RingInstanceSerializationTest.java | 22 +-
15 files changed, 295 insertions(+), 85 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 6004ee3..2c6d0ae 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
1.0.0
+ * Report additional bulk analytics job stats for instrumentation
(CASSANDRA-19418)
* Add certificate expiry check to start up validations done in Cassandra
Analytics library (CASSANDRA-19424)
* Use constant reference time during bulk read process (CASSANDRA-19452)
* Update access of ClearSnapshotStrategy (CASSANDRA-19442)
diff --git
a/cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/BulkWriterContext.java
b/cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/BulkWriterContext.java
index e3b2421..10928d4 100644
---
a/cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/BulkWriterContext.java
+++
b/cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/BulkWriterContext.java
@@ -21,12 +21,16 @@ package org.apache.cassandra.spark.bulkwriter;
import java.io.Serializable;
+import org.apache.cassandra.spark.common.stats.JobStatsPublisher;
+
public interface BulkWriterContext extends Serializable
{
ClusterInfo cluster();
JobInfo job();
+JobStatsPublisher jobStats();
+
SchemaInfo schema();
DataTransferApi transfer();
diff --git
a/cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/CassandraBulkSourceRelation.java
b/cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/CassandraBulkSourceRelation.java
index 520fa2c..bba5a31 100644
---
a/cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/CassandraBulkSourceRelation.java
+++
b/cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/CassandraBulkSourceRelation.java
@@ -19,14 +19,20 @@
package org.apache.cassandra.spark.bulkwriter;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
import java.util.Iterator;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.spark.api.java.JavaPairRDD;
import org.apache.spark.api.java.JavaSparkContext;
-import org.apache.spark.api.java.function.VoidFunction;
+import org.apache.spark.api.java.function.FlatMapFunction;
import org.apache.spark.broadcast.Broadcast;
import org.apache.spark.sql.Dataset;
import org.apache.spark.sql.Row;
@@ -46,6 +52,7 @@ public class CassandraBulkSourceRelation extends BaseRelation
implements Inserta
private final SQLContext sqlContext;
private final JavaSparkContext sparkContext;
private final Broadcast broadcastContext;
+private long startTimeNanos;
@SuppressWarnings("RedundantTypeArguments")
public CassandraBulkSourceRelation(BulkWriterContext writerContext,
SQLContext sqlContext)
@@ -87,6 +94,7 @@ public class CassandraBulkSourceRelation extends BaseRelation
implements Inserta
@Override
public void insert(@NotNull Dataset data, boolean overwrite)
{
+this.startTimeNanos = System.nanoTime();
if (overwrite)
{
throw new LoadNotSupportedException("Overwr
Re: [PR] CASSANDRA-19418 - Changes to report additional bulk analytics job stats for instrumentation [cassandra-analytics]
frankgh merged PR #41: URL: https://github.com/apache/cassandra-analytics/pull/41 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19418) [Analytics] Report additional bulk analytics job stats for instrumentation
[ https://issues.apache.org/jira/browse/CASSANDRA-19418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francisco Guerrero updated CASSANDRA-19418: --- Fix Version/s: NA Source Control Link: https://github.com/apache/cassandra-analytics/commit/164243e78f1557a34bc699ebc716b532781d6422 Resolution: Fixed Status: Resolved (was: Ready to Commit) > [Analytics] Report additional bulk analytics job stats for instrumentation > -- > > Key: CASSANDRA-19418 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19418 > Project: Cassandra > Issue Type: Task > Components: Analytics Library >Reporter: Arjun Ashok >Assignee: Arjun Ashok >Priority: Normal > Fix For: NA > > Time Spent: 3h 10m > Remaining Estimate: 0h > > Currently, the Cassandra bulk analytics library supports a "dialHome" API to > publish some initial job metadata, which in its current form, is redirected > to a log. The intention behind this is to allow custom implementations that > can utilize these summarized stats for instrumentation or reporting of client > behavior. > This task is meant to enhance this API to allow for additional job metadata > to be published both at the Spark executor level and at the task levels to > gather stats such as "success/failure", "number of rows written/read", > "failure reason" etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19418) [Analytics] Report additional bulk analytics job stats for instrumentation
[ https://issues.apache.org/jira/browse/CASSANDRA-19418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17830003#comment-17830003 ] Francisco Guerrero commented on CASSANDRA-19418: Latest CI for reference: https://app.circleci.com/pipelines/github/arjunashok/cassandra-analytics/82/workflows/e4bd195d-042c-4b94-b6d6-67a40c371aee > [Analytics] Report additional bulk analytics job stats for instrumentation > -- > > Key: CASSANDRA-19418 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19418 > Project: Cassandra > Issue Type: Task > Components: Analytics Library >Reporter: Arjun Ashok >Assignee: Arjun Ashok >Priority: Normal > Fix For: NA > > Time Spent: 3h 10m > Remaining Estimate: 0h > > Currently, the Cassandra bulk analytics library supports a "dialHome" API to > publish some initial job metadata, which in its current form, is redirected > to a log. The intention behind this is to allow custom implementations that > can utilize these summarized stats for instrumentation or reporting of client > behavior. > This task is meant to enhance this API to allow for additional job metadata > to be published both at the Spark executor level and at the task levels to > gather stats such as "success/failure", "number of rows written/read", > "failure reason" etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19418) [Analytics] Report additional bulk analytics job stats for instrumentation
[ https://issues.apache.org/jira/browse/CASSANDRA-19418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francisco Guerrero updated CASSANDRA-19418: --- Status: Ready to Commit (was: Review In Progress) > [Analytics] Report additional bulk analytics job stats for instrumentation > -- > > Key: CASSANDRA-19418 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19418 > Project: Cassandra > Issue Type: Task > Components: Analytics Library >Reporter: Arjun Ashok >Assignee: Arjun Ashok >Priority: Normal > Time Spent: 3h 10m > Remaining Estimate: 0h > > Currently, the Cassandra bulk analytics library supports a "dialHome" API to > publish some initial job metadata, which in its current form, is redirected > to a log. The intention behind this is to allow custom implementations that > can utilize these summarized stats for instrumentation or reporting of client > behavior. > This task is meant to enhance this API to allow for additional job metadata > to be published both at the Spark executor level and at the task levels to > gather stats such as "success/failure", "number of rows written/read", > "failure reason" etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-111) Improve observability in Sidecar
[ https://issues.apache.org/jira/browse/CASSANDRASC-111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francisco Guerrero updated CASSANDRASC-111: --- Status: Ready to Commit (was: Review In Progress) > Improve observability in Sidecar > - > > Key: CASSANDRASC-111 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-111 > Project: Sidecar for Apache Cassandra > Issue Type: Task > Components: Rest API >Reporter: Saranya Krishnakumar >Assignee: Saranya Krishnakumar >Priority: Normal > Labels: pull-request-available > > Currently we capture minimal metrics in Sidecar, through this patch we want > to improve observability in Sidecar. > For this we want to record more metrics in Sidecar with Dropwizard. Introduce > setup changes needed for capturing sidecar metrics through Dropwizard. > Some of the metric kinds we will capture through this patch are, HTTP metrics > like number of requests created per route, response time taken endpoint wise. > Capture metrics per Cassandra instance maintained by Sidecar, such as, > instance specific restore metrics, instance specific stream and upload > metrics. We will also turn on registering, server route metrics emitted by > Vert.x. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-111) Improve observability in Sidecar
[ https://issues.apache.org/jira/browse/CASSANDRASC-111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francisco Guerrero updated CASSANDRASC-111: --- Status: Review In Progress (was: Patch Available) > Improve observability in Sidecar > - > > Key: CASSANDRASC-111 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-111 > Project: Sidecar for Apache Cassandra > Issue Type: Task > Components: Rest API >Reporter: Saranya Krishnakumar >Assignee: Saranya Krishnakumar >Priority: Normal > Labels: pull-request-available > > Currently we capture minimal metrics in Sidecar, through this patch we want > to improve observability in Sidecar. > For this we want to record more metrics in Sidecar with Dropwizard. Introduce > setup changes needed for capturing sidecar metrics through Dropwizard. > Some of the metric kinds we will capture through this patch are, HTTP metrics > like number of requests created per route, response time taken endpoint wise. > Capture metrics per Cassandra instance maintained by Sidecar, such as, > instance specific restore metrics, instance specific stream and upload > metrics. We will also turn on registering, server route metrics emitted by > Vert.x. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-111) Improve observability in Sidecar
[ https://issues.apache.org/jira/browse/CASSANDRASC-111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17830004#comment-17830004 ] Yifan Cai commented on CASSANDRASC-111: --- +1 > Improve observability in Sidecar > - > > Key: CASSANDRASC-111 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-111 > Project: Sidecar for Apache Cassandra > Issue Type: Task > Components: Rest API >Reporter: Saranya Krishnakumar >Assignee: Saranya Krishnakumar >Priority: Normal > Labels: pull-request-available > > Currently we capture minimal metrics in Sidecar, through this patch we want > to improve observability in Sidecar. > For this we want to record more metrics in Sidecar with Dropwizard. Introduce > setup changes needed for capturing sidecar metrics through Dropwizard. > Some of the metric kinds we will capture through this patch are, HTTP metrics > like number of requests created per route, response time taken endpoint wise. > Capture metrics per Cassandra instance maintained by Sidecar, such as, > instance specific restore metrics, instance specific stream and upload > metrics. We will also turn on registering, server route metrics emitted by > Vert.x. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-111) Improve observability in Sidecar
[ https://issues.apache.org/jira/browse/CASSANDRASC-111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17830006#comment-17830006 ] Francisco Guerrero commented on CASSANDRASC-111: +1 Great contribution? > Improve observability in Sidecar > - > > Key: CASSANDRASC-111 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-111 > Project: Sidecar for Apache Cassandra > Issue Type: Task > Components: Rest API >Reporter: Saranya Krishnakumar >Assignee: Saranya Krishnakumar >Priority: Normal > Labels: pull-request-available > > Currently we capture minimal metrics in Sidecar, through this patch we want > to improve observability in Sidecar. > For this we want to record more metrics in Sidecar with Dropwizard. Introduce > setup changes needed for capturing sidecar metrics through Dropwizard. > Some of the metric kinds we will capture through this patch are, HTTP metrics > like number of requests created per route, response time taken endpoint wise. > Capture metrics per Cassandra instance maintained by Sidecar, such as, > instance specific restore metrics, instance specific stream and upload > metrics. We will also turn on registering, server route metrics emitted by > Vert.x. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRASC-111) Improve observability in Sidecar
[ https://issues.apache.org/jira/browse/CASSANDRASC-111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17830006#comment-17830006 ] Francisco Guerrero edited comment on CASSANDRASC-111 at 3/22/24 11:34 PM: -- +1 Great contribution! was (Author: frankgh): +1 Great contribution? > Improve observability in Sidecar > - > > Key: CASSANDRASC-111 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-111 > Project: Sidecar for Apache Cassandra > Issue Type: Task > Components: Rest API >Reporter: Saranya Krishnakumar >Assignee: Saranya Krishnakumar >Priority: Normal > Labels: pull-request-available > > Currently we capture minimal metrics in Sidecar, through this patch we want > to improve observability in Sidecar. > For this we want to record more metrics in Sidecar with Dropwizard. Introduce > setup changes needed for capturing sidecar metrics through Dropwizard. > Some of the metric kinds we will capture through this patch are, HTTP metrics > like number of requests created per route, response time taken endpoint wise. > Capture metrics per Cassandra instance maintained by Sidecar, such as, > instance specific restore metrics, instance specific stream and upload > metrics. We will also turn on registering, server route metrics emitted by > Vert.x. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-sidecar) branch trunk updated: CASSANDRASC-111 Improve observability in Sidecar with Dropwizard metrics (#102)
This is an automated email from the ASF dual-hosted git repository. frankgh pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-sidecar.git The following commit(s) were added to refs/heads/trunk by this push: new 056faad8 CASSANDRASC-111 Improve observability in Sidecar with Dropwizard metrics (#102) 056faad8 is described below commit 056faad8278365a77a36dd336e002136c269165a Author: Saranya Krishnakumar AuthorDate: Fri Mar 22 17:46:06 2024 -0700 CASSANDRASC-111 Improve observability in Sidecar with Dropwizard metrics (#102) Patch by Saranya Krishnakumar; Reviewed by Yifan Cai, Francisco Guerrero for CASSANDRASC-111 --- CHANGES.txt| 1 + src/main/dist/conf/sidecar.yaml| 9 ++ .../sidecar/cluster/instance/InstanceMetadata.java | 8 +- .../cluster/instance/InstanceMetadataImpl.java | 42 - .../MetricsConfiguration.java} | 40 + .../sidecar/config/SidecarConfiguration.java | 5 + .../VertxMetricsConfiguration.java}| 36 ++--- .../config/yaml/MetricsConfigurationImpl.java | 67 .../config/yaml/SidecarConfigurationImpl.java | 27 .../config/yaml/VertxMetricsConfigurationImpl.java | 100 .../cassandra/sidecar/metrics/NamedMetric.java | 169 + .../cassandra/sidecar/metrics/ServerMetrics.java | 55 +++ .../instance/InstanceMetrics.java} | 39 ++--- .../metrics/instance/InstanceMetricsImpl.java | 61 .../metrics/instance/InstanceResourceMetrics.java | 49 ++ .../metrics/instance/StreamSSTableMetrics.java | 88 +++ .../metrics/instance/UploadSSTableMetrics.java | 88 +++ .../sidecar/routes/DiskSpaceProtectionHandler.java | 1 + .../sidecar/routes/FileStreamHandler.java | 7 +- .../sstableuploads/SSTableUploadHandler.java | 15 +- .../cassandra/sidecar/server/MainModule.java | 49 -- .../apache/cassandra/sidecar/server/Server.java| 9 +- .../sidecar/tasks/HealthCheckPeriodicTask.java | 14 +- .../cassandra/sidecar/utils/FileStreamer.java | 52 +-- .../cassandra/sidecar/utils/MetricUtils.java | 42 + .../testing/CassandraSidecarTestContext.java | 1 + .../org/apache/cassandra/sidecar/TestModule.java | 3 + .../org/apache/cassandra/sidecar/ThrottleTest.java | 17 ++- .../sidecar/config/SidecarConfigurationTest.java | 49 ++ .../cassandra/sidecar/metrics/NamedMetricTest.java | 103 + .../sidecar/restore/RestoreJobManagerTest.java | 4 +- .../sstableuploads/BaseUploadsHandlerTest.java | 3 + .../sstableuploads/SSTableUploadHandlerTest.java | 13 ++ .../cassandra/sidecar/server/ServerSSLTest.java| 23 ++- .../cassandra/sidecar/snapshots/SnapshotUtils.java | 2 + .../sidecar/tasks/HealthCheckPeriodicTaskTest.java | 50 +- .../cassandra/sidecar/utils/TestMetricUtils.java | 39 + src/test/resources/config/sidecar_metrics.yaml | 69 + 38 files changed, 1310 insertions(+), 139 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index 638ee316..3e79c80f 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,5 +1,6 @@ 1.0.0 - + * Improve observability in Sidecar (CASSANDRASC-111) * Improve logging for slice restore task (CASSANDRASC-107) * Add restore task watcher to report long running tasks (CASSANDRASC-106) * RestoreSliceTask could be stuck due to missing exception handling (CASSANDRASC-105) diff --git a/src/main/dist/conf/sidecar.yaml b/src/main/dist/conf/sidecar.yaml index 82e4b1b2..2369f252 100644 --- a/src/main/dist/conf/sidecar.yaml +++ b/src/main/dist/conf/sidecar.yaml @@ -142,6 +142,15 @@ healthcheck: initial_delay_millis: 0 poll_freq_millis: 3 +metrics: + registry_name: cassandra_sidecar + vertx: +enabled: true +expose_via_jmx: false +jmx_domain_name: sidecar.vertx.jmx_domain +monitored_server_route_regexes:# regex list to match server routes + - /api/v1/.* + cassandra_input_validation: forbidden_keyspaces: - system_schema diff --git a/src/main/java/org/apache/cassandra/sidecar/cluster/instance/InstanceMetadata.java b/src/main/java/org/apache/cassandra/sidecar/cluster/instance/InstanceMetadata.java index 26da25e7..8862c8a1 100644 --- a/src/main/java/org/apache/cassandra/sidecar/cluster/instance/InstanceMetadata.java +++ b/src/main/java/org/apache/cassandra/sidecar/cluster/instance/InstanceMetadata.java @@ -21,9 +21,10 @@ package org.apache.cassandra.sidecar.cluster.instance; import java.util.List; import org.apache.cassandra.sidecar.cluster.CassandraAdapterDelegate; +import org.apache.cassandra.sidecar.metrics.instance.InstanceMetrics; +import org.jetbrains.annotations.NotNull; import org.jetbrains.annotations.Nullable; - /** * Metadata of an instance */ @@ -58,4 +59,9 @@ pub
[jira] [Updated] (CASSANDRASC-111) Improve observability in Sidecar
[ https://issues.apache.org/jira/browse/CASSANDRASC-111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francisco Guerrero updated CASSANDRASC-111: --- Fix Version/s: 1.0 Source Control Link: https://github.com/apache/cassandra-sidecar/commit/056faad8278365a77a36dd336e002136c269165a Resolution: Fixed Status: Resolved (was: Ready to Commit) > Improve observability in Sidecar > - > > Key: CASSANDRASC-111 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-111 > Project: Sidecar for Apache Cassandra > Issue Type: Task > Components: Rest API >Reporter: Saranya Krishnakumar >Assignee: Saranya Krishnakumar >Priority: Normal > Labels: pull-request-available > Fix For: 1.0 > > > Currently we capture minimal metrics in Sidecar, through this patch we want > to improve observability in Sidecar. > For this we want to record more metrics in Sidecar with Dropwizard. Introduce > setup changes needed for capturing sidecar metrics through Dropwizard. > Some of the metric kinds we will capture through this patch are, HTTP metrics > like number of requests created per route, response time taken endpoint wise. > Capture metrics per Cassandra instance maintained by Sidecar, such as, > instance specific restore metrics, instance specific stream and upload > metrics. We will also turn on registering, server route metrics emitted by > Vert.x. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-website) branch asf-staging updated (b8c4937c9 -> 4b2dd127f)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard b8c4937c9 generate docs for fd550e9c new 4b2dd127f generate docs for fd550e9c This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (b8c4937c9) \ N -- N -- N refs/heads/asf-staging (4b2dd127f) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: content/search-index.js | 2 +- site-ui/build/ui-bundle.zip | Bin 4883646 -> 4883646 bytes 2 files changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19150) Align values in rows in CQLSH right for numbers, left for text
[
https://issues.apache.org/jira/browse/CASSANDRA-19150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17830032#comment-17830032
]
Arun Ganesh commented on CASSANDRA-19150:
-
Ping [~bschoeni] [~smiklosovic]
The patch is almost ready. Just a quick question. The patch will break almost
every test that depends on the cqlsh output. How should I go about fixing them?
> Align values in rows in CQLSH right for numbers, left for text
> --
>
> Key: CASSANDRA-19150
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19150
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL/Interpreter
>Reporter: Stefan Miklosovic
>Assignee: Arun Ganesh
>Priority: Low
> Fix For: 5.x
>
> Attachments: Screenshot 2023-12-04 at 00.38.16.png, Screenshot
> 2023-12-09 at 16.58.25.png, signature.asc
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> *Updated* Jan 17 2024 after dev discussion
> Change CQLSH to left-align text while continue to right-align numbers. This
> will match how Postgres shell and Excel treat alignment of text and number.
> -
> *Original*
> We need to make this
> [https://github.com/apache/cassandra/blob/trunk/pylib/cqlshlib/cqlshmain.py#L1101]
> configurable so values in columns are either all on left or on right side of
> the column (basically change col.rjust to col.ljust).
> By default, it would be like it is now but there would be configuration
> property in cqlsh for that as well as a corresponding CQLSH command
> (optional), something like
> {code:java}
> ALIGNMENT LEFT|RIGHT
> {code}
> cc [~bschoeni]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
