(cassandra-java-driver) branch 4.x updated: ninja-fix CASSANDRA-19568: fixing mangled Groovy
This is an automated email from the ASF dual-hosted git repository.
absurdfarce pushed a commit to branch 4.x
in repository https://gitbox.apache.org/repos/asf/cassandra-java-driver.git
The following commit(s) were added to refs/heads/4.x by this push:
new 4bc346885 ninja-fix CASSANDRA-19568: fixing mangled Groovy
4bc346885 is described below
commit 4bc346885fd373906ed6106b76df6d494cb51b67
Author: absurdfarce
AuthorDate: Wed May 8 21:37:02 2024 -0500
ninja-fix CASSANDRA-19568: fixing mangled Groovy
---
Jenkinsfile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Jenkinsfile b/Jenkinsfile
index 69ee0a294..d38b7c638 100644
--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -103,7 +103,7 @@ def buildDriver(jabbaVersion) {
. ${JABBA_SHELL}
jabba use '''+jabbaVersion+'''
-echo "Building with Java version '''+jabbaVersion+'''
+echo "Building with Java version '''+jabbaVersion+'''"
mvn -B -V install -DskipTests -Dmaven.javadoc.skip=true
'''
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-java-driver) branch 4.x updated: CASSANDRA-19568: Use Jabba to specify Java 1.8 for building the driver
This is an automated email from the ASF dual-hosted git repository.
absurdfarce pushed a commit to branch 4.x
in repository https://gitbox.apache.org/repos/asf/cassandra-java-driver.git
The following commit(s) were added to refs/heads/4.x by this push:
new 3a6873774 CASSANDRA-19568: Use Jabba to specify Java 1.8 for building
the driver
3a6873774 is described below
commit 3a687377449f736ba1ed28bfcff824982b3138c4
Author: janehe
AuthorDate: Wed Apr 17 14:59:59 2024 -0700
CASSANDRA-19568: Use Jabba to specify Java 1.8 for building the driver
patch by Jane He and Bret McGuire; reviewed by Bret McGuire for
CASSANDRA-19568
---
Jenkinsfile | 21 +++--
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/Jenkinsfile b/Jenkinsfile
index 0bfa4ca7f..69ee0a294 100644
--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -98,14 +98,16 @@ ENVIRONMENT_EOF
}
def buildDriver(jabbaVersion) {
- withEnv(["BUILD_JABBA_VERSION=${jabbaVersion}"]) {
-sh label: 'Build driver', script: '''#!/bin/bash -le
- . ${JABBA_SHELL}
- jabba use ${BUILD_JABBA_VERSION}
+ def buildDriverScript = '''#!/bin/bash -le
- mvn -B -V install -DskipTests -Dmaven.javadoc.skip=true
-'''
- }
+. ${JABBA_SHELL}
+jabba use '''+jabbaVersion+'''
+
+echo "Building with Java version '''+jabbaVersion+'''
+
+mvn -B -V install -DskipTests -Dmaven.javadoc.skip=true
+ '''
+ sh label: 'Build driver', script: buildDriverScript
}
def executeTests() {
@@ -484,7 +486,7 @@ pipeline {
}
stage('Build-Driver') {
steps {
- buildDriver('default')
+ buildDriver('1.8')
}
}
stage('Execute-Tests') {
@@ -600,8 +602,7 @@ pipeline {
}
stage('Build-Driver') {
steps {
- // Jabba default should be a JDK8 for now
- buildDriver('default')
+ buildDriver('1.8')
}
}
stage('Execute-Tests') {
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] CASSANDRA-19568: Use Jabba to specify Java 1.8 for building the driver [cassandra-java-driver]
absurdfarce merged PR #1932: URL: https://github.com/apache/cassandra-java-driver/pull/1932 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18108) Data loss after a system restart/upgrade (3.11.17)
[
https://issues.apache.org/jira/browse/CASSANDRA-18108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ke Han updated CASSANDRA-18108:
---
Description:
When we upgrade Cassandra from 3.11.17 to 4.0.12, we found a data loss during
the upgrade process. This bug can also be triggered if simply performing a
system restart for 3.11.17, 4.0.12 or 4.1.4.
h1. Steps to reproduce
Start a 3.11.15 or 4.1.4 Cassandra node using default configurations. Execute
the following cqlsh commands.
{code:java}
CREATE KEYSPACE ks WITH REPLICATION = { 'class' : 'SimpleStrategy',
'replication_factor' : 1 };
CREATE TABLE IF NOT EXISTS ks.tb (c1 INT,c3 INT,c2 TEXT, PRIMARY KEY (c1 ))
WITH speculative_retry = 'ALWAYS';
INSERT INTO ks.tb (c1, c2) VALUES (2,'val');
ALTER TABLE ks.tb DROP c2 ;
ALTER TABLE ks.tb RENAME c1 TO c2; {code}
Then execute a SELECT command, we get the correct data
{code:java}
cqlsh> SELECT * FROM ks.tb;
c2 | c3
+--
2 | null
(1 rows){code}
Flush and stop the Cassandra daemon.
{code:java}
bin/nodetool flush
bin/nodetool stopdaemon{code}
Then restart the node.
{code:java}
bin/cassandra{code}
Start cqlsh, and execute the same SELECT command. The data in ks.tb is lost.
{code:java}
cqlsh> SELECT * FROM ks.tb;
c2 | c3
+
(0 rows){code}
During the node restart, we found an error log about initializing the table,
but it didn't prevent the system from starting up.
{code:java}
INFO [main] 2022-12-09 21:37:54,234 ColumnFamilyStore.java:432 - Initializing
ks.tb
ERROR [SSTableBatchOpen:1] 2022-12-09 21:37:54,237 CassandraDaemon.java:244 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError: null
at
org.apache.cassandra.db.PartitionColumns$Builder.add(PartitionColumns.java:161)
at
org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:340)
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:522)
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:385)
at
org.apache.cassandra.io.sstable.format.SSTableReader$3.run(SSTableReader.java:570)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84)
at java.lang.Thread.run(Thread.java:750) {code}
This bug can also be triggered if we perform an upgrade from 3.11.15 to 4.0.12
or 4.1.4 and execute the SELECT command in the new version.
was:
When we upgrade Cassandra from 3.11.15 to 4.0.7, we found a data loss during
the upgrade process. This bug can also be triggered if simply performing a
system restart.
h1. Steps to reproduce
Start a 3.11.15 or 4.0.7 Cassandra node using default configurations. Execute
the following cqlsh commands.
{code:java}
CREATE KEYSPACE ks WITH REPLICATION = { 'class' : 'SimpleStrategy',
'replication_factor' : 1 };
CREATE TABLE IF NOT EXISTS ks.tb (c1 INT,c3 INT,c2 TEXT, PRIMARY KEY (c1 ))
WITH speculative_retry = 'ALWAYS';
INSERT INTO ks.tb (c1, c2) VALUES (2,'val');
ALTER TABLE ks.tb DROP c2 ;
ALTER TABLE ks.tb RENAME c1 TO c2; {code}
Then execute a SELECT command, we get the correct data
{code:java}
cqlsh> SELECT * FROM ks.tb;
c2 | c3
+--
2 | null
(1 rows){code}
Flush and stop the Cassandra daemon.
{code:java}
bin/nodetool flush
bin/nodetool stopdaemon{code}
Then restart the node.
{code:java}
bin/cassandra{code}
Start cqlsh, and execute the same SELECT command. The data in ks.tb is lost.
{code:java}
cqlsh> SELECT * FROM ks.tb;
c2 | c3
+
(0 rows){code}
During the node restart, we found an error log about initializing the table,
but it didn't prevent the system from starting up.
{code:java}
INFO [main] 2022-12-09 21:37:54,234 ColumnFamilyStore.java:432 - Initializing
ks.tb
ERROR [SSTableBatchOpen:1] 2022-12-09 21:37:54,237 CassandraDaemon.java:244 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError: null
at
org.apache.cassandra.db.PartitionColumns$Builder.add(PartitionColumns.java:161)
at
org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:340)
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:522)
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:385)
at
org.apache.cassandra.io.sstable.format.SSTableReader$3.run(SSTableReader.java:570)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
[jira] [Updated] (CASSANDRA-18108) Data loss after a system restart/upgrade (3.11.17)
[
https://issues.apache.org/jira/browse/CASSANDRA-18108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ke Han updated CASSANDRA-18108:
---
Description:
When we upgrade Cassandra from 3.11.17 to 4.0.12, we found a data loss during
the upgrade process. This bug can also be triggered if simply performing a
system restart for 3.11.17, 4.0.12 or 4.1.4.
h1. Steps to reproduce
Start a 3.11.15 or 4.1.4 Cassandra node using default configurations. Execute
the following cqlsh commands.
{code:java}
CREATE KEYSPACE ks WITH REPLICATION = { 'class' : 'SimpleStrategy',
'replication_factor' : 1 };
CREATE TABLE IF NOT EXISTS ks.tb (c1 INT,c3 INT,c2 TEXT, PRIMARY KEY (c1 ))
WITH speculative_retry = 'ALWAYS';
INSERT INTO ks.tb (c1, c2) VALUES (2,'val');
ALTER TABLE ks.tb DROP c2 ;
ALTER TABLE ks.tb RENAME c1 TO c2; {code}
Then execute a SELECT command, we get the correct data
{code:java}
cqlsh> SELECT * FROM ks.tb;
c2 | c3
+--
2 | null
(1 rows){code}
Flush and stop the Cassandra daemon.
{code:java}
bin/nodetool flush
bin/nodetool stopdaemon{code}
Then restart the node.
{code:java}
bin/cassandra{code}
Start cqlsh, and execute the same SELECT command. The data in ks.tb is lost.
{code:java}
cqlsh> SELECT * FROM ks.tb;
c2 | c3
+
(0 rows){code}
During the node restart, we found an error log about initializing the table,
but it didn't prevent the system from starting up.
{code:java}
INFO [main] 2022-12-09 21:37:54,234 ColumnFamilyStore.java:432 - Initializing
ks.tb
ERROR [SSTableBatchOpen:1] 2022-12-09 21:37:54,237 CassandraDaemon.java:244 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError: null
at
org.apache.cassandra.db.PartitionColumns$Builder.add(PartitionColumns.java:161)
at
org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:340)
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:522)
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:385)
at
org.apache.cassandra.io.sstable.format.SSTableReader$3.run(SSTableReader.java:570)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84)
at java.lang.Thread.run(Thread.java:750) {code}
This bug can also be triggered if we perform an upgrade (with drain) from
3.11.17 to 4.0.12 or 4.1.4 and execute the SELECT command in the new version.
was:
When we upgrade Cassandra from 3.11.17 to 4.0.12, we found a data loss during
the upgrade process. This bug can also be triggered if simply performing a
system restart for 3.11.17, 4.0.12 or 4.1.4.
h1. Steps to reproduce
Start a 3.11.15 or 4.1.4 Cassandra node using default configurations. Execute
the following cqlsh commands.
{code:java}
CREATE KEYSPACE ks WITH REPLICATION = { 'class' : 'SimpleStrategy',
'replication_factor' : 1 };
CREATE TABLE IF NOT EXISTS ks.tb (c1 INT,c3 INT,c2 TEXT, PRIMARY KEY (c1 ))
WITH speculative_retry = 'ALWAYS';
INSERT INTO ks.tb (c1, c2) VALUES (2,'val');
ALTER TABLE ks.tb DROP c2 ;
ALTER TABLE ks.tb RENAME c1 TO c2; {code}
Then execute a SELECT command, we get the correct data
{code:java}
cqlsh> SELECT * FROM ks.tb;
c2 | c3
+--
2 | null
(1 rows){code}
Flush and stop the Cassandra daemon.
{code:java}
bin/nodetool flush
bin/nodetool stopdaemon{code}
Then restart the node.
{code:java}
bin/cassandra{code}
Start cqlsh, and execute the same SELECT command. The data in ks.tb is lost.
{code:java}
cqlsh> SELECT * FROM ks.tb;
c2 | c3
+
(0 rows){code}
During the node restart, we found an error log about initializing the table,
but it didn't prevent the system from starting up.
{code:java}
INFO [main] 2022-12-09 21:37:54,234 ColumnFamilyStore.java:432 - Initializing
ks.tb
ERROR [SSTableBatchOpen:1] 2022-12-09 21:37:54,237 CassandraDaemon.java:244 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError: null
at
org.apache.cassandra.db.PartitionColumns$Builder.add(PartitionColumns.java:161)
at
org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:340)
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:522)
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:385)
at
org.apache.cassandra.io.sstable.format.SSTableReader$3.run(SSTableReader.java:570)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureT
[jira] [Updated] (CASSANDRA-18108) Data loss after a system restart/upgrade (3.11.17)
[
https://issues.apache.org/jira/browse/CASSANDRA-18108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Klay updated CASSANDRA-18108:
-
Summary: Data loss after a system restart/upgrade (3.11.17) (was: Data
loss after a system restart/upgrade (3.11.15))
> Data loss after a system restart/upgrade (3.11.17)
> --
>
> Key: CASSANDRA-18108
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18108
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ke Han
>Priority: Normal
>
> When we upgrade Cassandra from 3.11.15 to 4.0.7, we found a data loss during
> the upgrade process. This bug can also be triggered if simply performing a
> system restart.
> h1. Steps to reproduce
> Start a 3.11.15 or 4.0.7 Cassandra node using default configurations. Execute
> the following cqlsh commands.
> {code:java}
> CREATE KEYSPACE ks WITH REPLICATION = { 'class' : 'SimpleStrategy',
> 'replication_factor' : 1 };
> CREATE TABLE IF NOT EXISTS ks.tb (c1 INT,c3 INT,c2 TEXT, PRIMARY KEY (c1 ))
> WITH speculative_retry = 'ALWAYS';
> INSERT INTO ks.tb (c1, c2) VALUES (2,'val');
> ALTER TABLE ks.tb DROP c2 ;
> ALTER TABLE ks.tb RENAME c1 TO c2; {code}
> Then execute a SELECT command, we get the correct data
> {code:java}
> cqlsh> SELECT * FROM ks.tb;
> c2 | c3
> +--
> 2 | null
> (1 rows){code}
> Flush and stop the Cassandra daemon.
> {code:java}
> bin/nodetool flush
> bin/nodetool stopdaemon{code}
> Then restart the node.
> {code:java}
> bin/cassandra{code}
> Start cqlsh, and execute the same SELECT command. The data in ks.tb is lost.
> {code:java}
> cqlsh> SELECT * FROM ks.tb;
> c2 | c3
> +
> (0 rows){code}
>
> During the node restart, we found an error log about initializing the table,
> but it didn't prevent the system from starting up.
> {code:java}
> INFO [main] 2022-12-09 21:37:54,234 ColumnFamilyStore.java:432 -
> Initializing ks.tb
> ERROR [SSTableBatchOpen:1] 2022-12-09 21:37:54,237 CassandraDaemon.java:244 -
> Exception in thread Thread[SSTableBatchOpen:1,5,main]
> java.lang.AssertionError: null
> at
> org.apache.cassandra.db.PartitionColumns$Builder.add(PartitionColumns.java:161)
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:340)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:522)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:385)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$3.run(SSTableReader.java:570)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84)
> at java.lang.Thread.run(Thread.java:750) {code}
>
> This bug can also be triggered if we perform an upgrade from 3.11.15 to 4.0.7
> and execute the SELECT command in the new version. (*The token_num
> configuration in 4.0.7 is modified to 16 for upgrade compatibility purposes,
> all the other configurations are using default values)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15452) Improve disk access patterns during compaction and streaming
[
https://issues.apache.org/jira/browse/CASSANDRA-15452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844835#comment-17844835
]
Jon Haddad edited comment on CASSANDRA-15452 at 5/9/24 1:43 AM:
I should share some additional information and point out some things that might
not be immediately obvious in the graphs about the EBS test.
* The GP3 EBS volume was configured at 3K IOPs with 256MB throughput.
* All tests were run with an empty page cache by doing {{echo 3 | sudo tee
/proc/sys/vm/drop_caches}} then a nodetool compact.
* Compaction was not throttled.
* Read ahead was set to 4KB.
* The EBS volume was, as expected, completely limited by IOPS, as it peaks
right at 3K, with read throughput on the drive around 18-20MB/s.
* The spikes in writes, presumably from fsync, cause the dip in read perf.
* The patched version had a significant reduction in IOPS usage, using only
~500 IOPS to achieve around 65MB/s, *more than a 3x* improvement in throughput
and a {*}6x improvement in IOPS usage{*}. In a production environment this
would lead to more predictable read latencies, as compaction I/O competes with
reads for disk resources.
* Multiple compaction threads should be able to achieve an even more
impressive improvement since we haven't yet reached the resource limit of the
underlying device.
* This patch should significantly improve node density when using EBS as
compaction is a significant bottleneck.
Some other general information:
* The patch does not yet take advantage of the fadvise call, which should help
a production environment significantly by preventing compaction from polluting
the page cache with pages that are about to be removed from the filesystem.
Overall I am very excited by the results we're seeing here and will follow up
with additional testing.
was (Author: rustyrazorblade):
I should share some additional information and point out some things that might
not be immediately obvious in the graphs about the EBS test.
* The GP3 EBS volume was configured at 3K IOPs with 256MB throughput.
* All tests were run with an empty page cache by doing {{
echo 3 | sudo tee /proc/sys/vm/drop_caches}}then a nodetool compact.
* Compaction was not throttled.
* Read ahead was set to 4KB.
* The EBS volume was, as expected, completely limited by IOPS, as it peaks
right at 3K, with read throughput on the drive around 18-20MB/s.
* The spikes in writes, presumably from fsync, cause the dip in read perf.
* The patched version had a significant reduction in IOPS usage, using only
~500 IOPS to achieve around 65MB/s, *more than a 3x* improvement in throughput
and a {*}6x improvement in IOPS usage{*}. In a production environment this
would lead to more predictable read latencies, as compaction I/O competes with
reads for disk resources.
* Multiple compaction threads should be able to achieve an even more
impressive improvement since we haven't yet reached the resource limit of the
underlying device.
* This patch should significantly improve node density when using EBS as
compaction is a significant bottleneck.
Some other general information:
* The patch does not yet take advantage of the fadvise call, which should help
a production environment significantly by preventing compaction from polluting
the page cache with pages that are about to be removed from the filesystem.
Overall I am very excited by the results we're seeing here and will follow up
with additional testing.
> Improve disk access patterns during compaction and streaming
>
>
> Key: CASSANDRA-15452
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15452
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Local Write-Read Paths, Local/Compaction
>Reporter: Jon Haddad
>Assignee: Jordan West
>Priority: Normal
> Attachments: everyfs.txt, iostat-5.0-head.output,
> iostat-5.0-patched.output, iostat-ebs-15452.png, iostat-ebs-head.png,
> iostat-instance-15452.png, iostat-instance-head.png, results.txt,
> sequential.fio
>
>
> On read heavy workloads Cassandra performs much better when using a low read
> ahead setting. In my tests I've seen an 5x improvement in throughput and
> more than a 50% reduction in latency. However, I've also observed that it
> can have a negative impact on compaction and streaming throughput. It
> especially negatively impacts cloud environments where small reads incur high
> costs in IOPS due to tiny requests.
> # We should investigate using POSIX_FADV_DONTNEED on files we're compacting
> to see if we can improve performance and reduce page faults.
> # This should be combined with an internal read ahead style buffer that
> Cassandra manages, similar to a BufferedInput
[jira] [Comment Edited] (CASSANDRA-15452) Improve disk access patterns during compaction and streaming
[
https://issues.apache.org/jira/browse/CASSANDRA-15452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844835#comment-17844835
]
Jon Haddad edited comment on CASSANDRA-15452 at 5/9/24 1:42 AM:
I should share some additional information and point out some things that might
not be immediately obvious in the graphs about the EBS test.
* The GP3 EBS volume was configured at 3K IOPs with 256MB throughput.
* All tests were run with an empty page cache by doing {{
echo 3 | sudo tee /proc/sys/vm/drop_caches}}then a nodetool compact.
* Compaction was not throttled.
* Read ahead was set to 4KB.
* The EBS volume was, as expected, completely limited by IOPS, as it peaks
right at 3K, with read throughput on the drive around 18-20MB/s.
* The spikes in writes, presumably from fsync, cause the dip in read perf.
* The patched version had a significant reduction in IOPS usage, using only
~500 IOPS to achieve around 65MB/s, *more than a 3x* improvement in throughput
and a {*}6x improvement in IOPS usage{*}. In a production environment this
would lead to more predictable read latencies, as compaction I/O competes with
reads for disk resources.
* Multiple compaction threads should be able to achieve an even more
impressive improvement since we haven't yet reached the resource limit of the
underlying device.
* This patch should significantly improve node density when using EBS as
compaction is a significant bottleneck.
Some other general information:
* The patch does not yet take advantage of the fadvise call, which should help
a production environment significantly by preventing compaction from polluting
the page cache with pages that are about to be removed from the filesystem.
Overall I am very excited by the results we're seeing here and will follow up
with additional testing.
was (Author: rustyrazorblade):
I should share some additional information and point out some things that might
not be immediately obvious in the graphs about the EBS test.
* The GP3 EBS volume was configured at 3K IOPs with 256MB throughput.
* All tests were run with an empty page cache by doing the following, then a
nodetool compact.
* Compaction was not throttled.
* Read ahead was set to 4KB.
* The EBS volume was, as expected, completely limited by IOPS, as it peaks
right at 3K, with read throughput on the drive around 18-20MB/s.
* The spikes in writes, presumably from fsync, cause the dip in read perf.
* The patched version had a significant reduction in IOPS usage, using only
~500 IOPS to achieve around 65MB/s, *more than a 3x* improvement in throughput
and a {*}6x improvement in IOPS usage{*}. In a production environment this
would lead to more predictable read latencies, as compaction I/O competes with
reads for disk resources.
* Multiple compaction threads should be able to achieve an even more
impressive improvement since we haven't yet reached the resource limit of the
underlying device.
* This patch should significantly improve node density when using EBS as
compaction is a significant bottleneck.
Some other general information:
* The patch does not yet take advantage of the fadvise call, which should help
a production environment significantly by preventing compaction from polluting
the page cache with pages that are about to be removed from the filesystem.
Overall I am very excited by the results we're seeing here and will follow up
with additional testing.
> Improve disk access patterns during compaction and streaming
>
>
> Key: CASSANDRA-15452
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15452
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Local Write-Read Paths, Local/Compaction
>Reporter: Jon Haddad
>Assignee: Jordan West
>Priority: Normal
> Attachments: everyfs.txt, iostat-5.0-head.output,
> iostat-5.0-patched.output, iostat-ebs-15452.png, iostat-ebs-head.png,
> iostat-instance-15452.png, iostat-instance-head.png, results.txt,
> sequential.fio
>
>
> On read heavy workloads Cassandra performs much better when using a low read
> ahead setting. In my tests I've seen an 5x improvement in throughput and
> more than a 50% reduction in latency. However, I've also observed that it
> can have a negative impact on compaction and streaming throughput. It
> especially negatively impacts cloud environments where small reads incur high
> costs in IOPS due to tiny requests.
> # We should investigate using POSIX_FADV_DONTNEED on files we're compacting
> to see if we can improve performance and reduce page faults.
> # This should be combined with an internal read ahead style buffer that
> Cassandra manages, similar to a BufferedInputStream but with our own
> machi
[jira] [Commented] (CASSANDRA-15452) Improve disk access patterns during compaction and streaming
[
https://issues.apache.org/jira/browse/CASSANDRA-15452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844835#comment-17844835
]
Jon Haddad commented on CASSANDRA-15452:
I should share some additional information and point out some things that might
not be immediately obvious in the graphs about the EBS test.
* The GP3 EBS volume was configured at 3K IOPs with 256MB throughput.
* All tests were run with an empty page cache by doing the following, then a
nodetool compact.
* Compaction was not throttled.
* Read ahead was set to 4KB.
* The EBS volume was, as expected, completely limited by IOPS, as it peaks
right at 3K, with read throughput on the drive around 18-20MB/s.
* The spikes in writes, presumably from fsync, cause the dip in read perf.
* The patched version had a significant reduction in IOPS usage, using only
~500 IOPS to achieve around 65MB/s, *more than a 3x* improvement in throughput
and a {*}6x improvement in IOPS usage{*}. In a production environment this
would lead to more predictable read latencies, as compaction I/O competes with
reads for disk resources.
* Multiple compaction threads should be able to achieve an even more
impressive improvement since we haven't yet reached the resource limit of the
underlying device.
* This patch should significantly improve node density when using EBS as
compaction is a significant bottleneck.
Some other general information:
* The patch does not yet take advantage of the fadvise call, which should help
a production environment significantly by preventing compaction from polluting
the page cache with pages that are about to be removed from the filesystem.
Overall I am very excited by the results we're seeing here and will follow up
with additional testing.
> Improve disk access patterns during compaction and streaming
>
>
> Key: CASSANDRA-15452
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15452
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Local Write-Read Paths, Local/Compaction
>Reporter: Jon Haddad
>Assignee: Jordan West
>Priority: Normal
> Attachments: everyfs.txt, iostat-5.0-head.output,
> iostat-5.0-patched.output, iostat-ebs-15452.png, iostat-ebs-head.png,
> iostat-instance-15452.png, iostat-instance-head.png, results.txt,
> sequential.fio
>
>
> On read heavy workloads Cassandra performs much better when using a low read
> ahead setting. In my tests I've seen an 5x improvement in throughput and
> more than a 50% reduction in latency. However, I've also observed that it
> can have a negative impact on compaction and streaming throughput. It
> especially negatively impacts cloud environments where small reads incur high
> costs in IOPS due to tiny requests.
> # We should investigate using POSIX_FADV_DONTNEED on files we're compacting
> to see if we can improve performance and reduce page faults.
> # This should be combined with an internal read ahead style buffer that
> Cassandra manages, similar to a BufferedInputStream but with our own
> machinery. This buffer should read fairly large blocks of data off disk at
> at time. EBS, for example, allows 1 IOP to be up to 256KB. A considerable
> amount of time is spent in blocking I/O during compaction and streaming.
> Reducing the frequency we read from disk should speed up all sequential I/O
> operations.
> # We can reduce system calls by buffering writes as well, but I think it
> will have less of an impact than the reads
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19626) [Analytics] NullPointerException when reading static column with null values
[ https://issues.apache.org/jira/browse/CASSANDRA-19626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844830#comment-17844830 ] Francisco Guerrero commented on CASSANDRA-19626: CI: https://app.circleci.com/pipelines/github/frankgh/cassandra-analytics?branch=CASSANDRA-19626 > [Analytics] NullPointerException when reading static column with null values > > > Key: CASSANDRA-19626 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19626 > Project: Cassandra > Issue Type: Bug > Components: Analytics Library >Reporter: Francisco Guerrero >Assignee: Francisco Guerrero >Priority: Normal > Fix For: NA > > Time Spent: 10m > Remaining Estimate: 0h > > The analytics library fails to read static columns that has null values > depending on the type. For example, this problem is exhibited in the > Timestamp type, where the deserialized object is null, and the type will > attempt to get the time from the field. When that occurs it triggers the > NullPointerException. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-129) Remove tableId from list snapshot response
[
https://issues.apache.org/jira/browse/CASSANDRASC-129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francisco Guerrero updated CASSANDRASC-129:
---
Source Control Link:
https://github.com/apache/cassandra-sidecar/commit/e95786a077e1137dcaae206854986987edc6a71e
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Remove tableId from list snapshot response
> --
>
> Key: CASSANDRASC-129
> URL: https://issues.apache.org/jira/browse/CASSANDRASC-129
> Project: Sidecar for Apache Cassandra
> Issue Type: Bug
> Components: Rest API
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Labels: pull-request-available
> Fix For: 1.0
>
>
> In CASSANDRASC-94, a new field {{tableId}} was added to the payload of the
> list snapshots endpoint. Old Sidecar clients are compatible with new fields
> added from the server side. However, other clients can potentially break and
> for that reason we should remove the {{tableId}} field from the response and
> make any potential breaking changes with an API version change instead. As an
> alternative, we can add the {{tableId}} as part of the {{tableName}} field
> instead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-129) Remove tableId from list snapshot response
[
https://issues.apache.org/jira/browse/CASSANDRASC-129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844828#comment-17844828
]
ASF subversion and git services commented on CASSANDRASC-129:
-
Commit e95786a077e1137dcaae206854986987edc6a71e in cassandra-sidecar's branch
refs/heads/trunk from Francisco Guerrero
[ https://gitbox.apache.org/repos/asf?p=cassandra-sidecar.git;h=e95786a ]
CASSANDRASC-129: Remove tableId from list snapshot response (#120)
Patch by Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRASC-129
> Remove tableId from list snapshot response
> --
>
> Key: CASSANDRASC-129
> URL: https://issues.apache.org/jira/browse/CASSANDRASC-129
> Project: Sidecar for Apache Cassandra
> Issue Type: Bug
> Components: Rest API
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Labels: pull-request-available
> Fix For: 1.0
>
>
> In CASSANDRASC-94, a new field {{tableId}} was added to the payload of the
> list snapshots endpoint. Old Sidecar clients are compatible with new fields
> added from the server side. However, other clients can potentially break and
> for that reason we should remove the {{tableId}} field from the response and
> make any potential breaking changes with an API version change instead. As an
> alternative, we can add the {{tableId}} as part of the {{tableName}} field
> instead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-129) Remove tableId from list snapshot response
[
https://issues.apache.org/jira/browse/CASSANDRASC-129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francisco Guerrero updated CASSANDRASC-129:
---
Status: Ready to Commit (was: Review In Progress)
> Remove tableId from list snapshot response
> --
>
> Key: CASSANDRASC-129
> URL: https://issues.apache.org/jira/browse/CASSANDRASC-129
> Project: Sidecar for Apache Cassandra
> Issue Type: Bug
> Components: Rest API
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Labels: pull-request-available
> Fix For: 1.0
>
>
> In CASSANDRASC-94, a new field {{tableId}} was added to the payload of the
> list snapshots endpoint. Old Sidecar clients are compatible with new fields
> added from the server side. However, other clients can potentially break and
> for that reason we should remove the {{tableId}} field from the response and
> make any potential breaking changes with an API version change instead. As an
> alternative, we can add the {{tableId}} as part of the {{tableName}} field
> instead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-129) Remove tableId from list snapshot response
[
https://issues.apache.org/jira/browse/CASSANDRASC-129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844827#comment-17844827
]
Francisco Guerrero commented on CASSANDRASC-129:
CI is green
> Remove tableId from list snapshot response
> --
>
> Key: CASSANDRASC-129
> URL: https://issues.apache.org/jira/browse/CASSANDRASC-129
> Project: Sidecar for Apache Cassandra
> Issue Type: Bug
> Components: Rest API
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Labels: pull-request-available
> Fix For: 1.0
>
>
> In CASSANDRASC-94, a new field {{tableId}} was added to the payload of the
> list snapshots endpoint. Old Sidecar clients are compatible with new fields
> added from the server side. However, other clients can potentially break and
> for that reason we should remove the {{tableId}} field from the response and
> make any potential breaking changes with an API version change instead. As an
> alternative, we can add the {{tableId}} as part of the {{tableName}} field
> instead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-sidecar) branch trunk updated: CASSANDRASC-129: Remove tableId from list snapshot response (#120)
This is an automated email from the ASF dual-hosted git repository.
frankgh pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-sidecar.git
The following commit(s) were added to refs/heads/trunk by this push:
new e95786a CASSANDRASC-129: Remove tableId from list snapshot response
(#120)
e95786a is described below
commit e95786a077e1137dcaae206854986987edc6a71e
Author: Francisco Guerrero
AuthorDate: Wed May 8 18:04:05 2024 -0700
CASSANDRASC-129: Remove tableId from list snapshot response (#120)
Patch by Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRASC-129
---
.../common/response/ListSnapshotFilesResponse.java | 7 ---
.../sidecar/client/SidecarClientTest.java | 56 ++
src/main/dist/conf/sidecar.yaml| 15 +++---
.../cluster/instance/InstanceMetadataImpl.java | 8 +++-
.../routes/snapshots/ListSnapshotHandler.java | 6 ++-
.../apache/cassandra/sidecar/utils/FileUtils.java | 38 +++
.../routes/snapshots/ListSnapshotHandlerTest.java | 15 ++
.../cassandra/sidecar/utils/FileUtilsTest.java | 40
8 files changed, 147 insertions(+), 38 deletions(-)
diff --git
a/client-common/src/main/java/org/apache/cassandra/sidecar/common/response/ListSnapshotFilesResponse.java
b/client-common/src/main/java/org/apache/cassandra/sidecar/common/response/ListSnapshotFilesResponse.java
index b418a08..3700d93 100644
---
a/client-common/src/main/java/org/apache/cassandra/sidecar/common/response/ListSnapshotFilesResponse.java
+++
b/client-common/src/main/java/org/apache/cassandra/sidecar/common/response/ListSnapshotFilesResponse.java
@@ -64,7 +64,6 @@ public class ListSnapshotFilesResponse
public final String snapshotName;
public final String keySpaceName;
public final String tableName;
-public final String tableId;
public final String fileName;
private String componentDownloadUrl;
@@ -75,7 +74,6 @@ public class ListSnapshotFilesResponse
@JsonProperty("snapshotName") String snapshotName,
@JsonProperty("keySpaceName") String keySpaceName,
@JsonProperty("tableName") String tableName,
-@JsonProperty("tableId") String tableId,
@JsonProperty("fileName") String fileName)
{
this.size = size;
@@ -85,7 +83,6 @@ public class ListSnapshotFilesResponse
this.snapshotName = snapshotName;
this.keySpaceName = keySpaceName;
this.tableName = tableName;
-this.tableId = tableId;
this.fileName = fileName;
}
@@ -93,10 +90,6 @@ public class ListSnapshotFilesResponse
{
if (componentDownloadUrl == null)
{
-String tableName = this.tableId != null
- ? this.tableName + "-" + this.tableId
- : this.tableName;
-
componentDownloadUrl = ApiEndpointsV1.COMPONENTS_ROUTE
.replaceAll(ApiEndpointsV1.KEYSPACE_PATH_PARAM, keySpaceName)
.replaceAll(ApiEndpointsV1.TABLE_PATH_PARAM, tableName)
diff --git
a/client/src/testFixtures/java/org/apache/cassandra/sidecar/client/SidecarClientTest.java
b/client/src/testFixtures/java/org/apache/cassandra/sidecar/client/SidecarClientTest.java
index 84a5646..69459fe 100644
---
a/client/src/testFixtures/java/org/apache/cassandra/sidecar/client/SidecarClientTest.java
+++
b/client/src/testFixtures/java/org/apache/cassandra/sidecar/client/SidecarClientTest.java
@@ -498,6 +498,50 @@ abstract class SidecarClientTest
+
"?includeSecondaryIndexFiles=true");
}
+/**
+ * CASSANDRASC-94 introduced a new field ({@code tableId}) to the payload
when listing snapshots. We
+ * need to make sure the client is able to handle the payload with the
additional field (and ignore it).
+ *
+ * @throws Exception when the test fails
+ */
+@Test
+public void testListSnapshotFilesPayloadWithTableId() throws Exception
+{
+String responseAsString =
"{\"snapshotFilesInfo\":[{\"size\":15,\"host\":\"localhost1\",\"port\":2020," +
+
"\"dataDirIndex\":1,\"snapshotName\":\"2023.04.11\",\"keySpaceName\":\"cycling\","
+
+
"\"tableName\":\"cyclist_name\",\"tableId\":\"1234\",\"fileName\":" +
+ "\"nb-203-big-TOC.txt\"}]}";
+MockResponse response = new
MockResponse().setResponseCode(OK.code()).setBody(responseAsString);
+SidecarInstanceImpl sidecarInstance = instances.get(2);
+MockWebServer mockWebServer = servers.get(2);
+mockWebServer.enqueue(response);
+
+ListSnapshotFile
[jira] [Updated] (CASSANDRASC-129) Remove tableId from list snapshot response
[
https://issues.apache.org/jira/browse/CASSANDRASC-129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francisco Guerrero updated CASSANDRASC-129:
---
Reviewers: Yifan Cai, Francisco Guerrero (was: Yifan Cai)
Yifan Cai, Francisco Guerrero (was: Francisco Guerrero, Yifan
Cai)
Status: Review In Progress (was: Patch Available)
> Remove tableId from list snapshot response
> --
>
> Key: CASSANDRASC-129
> URL: https://issues.apache.org/jira/browse/CASSANDRASC-129
> Project: Sidecar for Apache Cassandra
> Issue Type: Bug
> Components: Rest API
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Labels: pull-request-available
> Fix For: 1.0
>
>
> In CASSANDRASC-94, a new field {{tableId}} was added to the payload of the
> list snapshots endpoint. Old Sidecar clients are compatible with new fields
> added from the server side. However, other clients can potentially break and
> for that reason we should remove the {{tableId}} field from the response and
> make any potential breaking changes with an API version change instead. As an
> alternative, we can add the {{tableId}} as part of the {{tableName}} field
> instead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15452) Improve disk access patterns during compaction and streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-15452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844826#comment-17844826 ] Jordan West commented on CASSANDRA-15452: - I wrote a [prototype|https://github.com/jrwest/cassandra/tree/jwest/15452-5.0] that uses a 256KB off-heap buffer for non-mmap'd, compressed data files only and tested it with [~rustyrazorblade]. The test uses "big" format. We have yet to validate with BTI format. *To summarize our findings when running a single compaction:* * *on NVMe: equal throughput with an approx 5x reduction in IOPS* * *on EBS: 3x throughput with an approx 5x reduction in IOPS* Attached are graphs of the iostat data and the raw iostat output from the EBS tests. *EBS (patched):* !iostat-ebs-15452.png! *EBS (5.0-HEAD):* !iostat-ebs-head.png! *NVMe (patched):* !iostat-instance-15452.png! *NVMe (5.0-HEAD):* !iostat-instance-head.png! [^iostat-5.0-head.output] [^iostat-5.0-patched.output] > Improve disk access patterns during compaction and streaming > > > Key: CASSANDRA-15452 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15452 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Local Write-Read Paths, Local/Compaction >Reporter: Jon Haddad >Assignee: Jordan West >Priority: Normal > Attachments: everyfs.txt, iostat-5.0-head.output, > iostat-5.0-patched.output, iostat-ebs-15452.png, iostat-ebs-head.png, > iostat-instance-15452.png, iostat-instance-head.png, results.txt, > sequential.fio > > > On read heavy workloads Cassandra performs much better when using a low read > ahead setting. In my tests I've seen an 5x improvement in throughput and > more than a 50% reduction in latency. However, I've also observed that it > can have a negative impact on compaction and streaming throughput. It > especially negatively impacts cloud environments where small reads incur high > costs in IOPS due to tiny requests. > # We should investigate using POSIX_FADV_DONTNEED on files we're compacting > to see if we can improve performance and reduce page faults. > # This should be combined with an internal read ahead style buffer that > Cassandra manages, similar to a BufferedInputStream but with our own > machinery. This buffer should read fairly large blocks of data off disk at > at time. EBS, for example, allows 1 IOP to be up to 256KB. A considerable > amount of time is spent in blocking I/O during compaction and streaming. > Reducing the frequency we read from disk should speed up all sequential I/O > operations. > # We can reduce system calls by buffering writes as well, but I think it > will have less of an impact than the reads -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15452) Improve disk access patterns during compaction and streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-15452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-15452: Attachment: iostat-5.0-head.output iostat-5.0-patched.output > Improve disk access patterns during compaction and streaming > > > Key: CASSANDRA-15452 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15452 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Local Write-Read Paths, Local/Compaction >Reporter: Jon Haddad >Assignee: Jordan West >Priority: Normal > Attachments: everyfs.txt, iostat-5.0-head.output, > iostat-5.0-patched.output, iostat-ebs-15452.png, iostat-ebs-head.png, > iostat-instance-15452.png, iostat-instance-head.png, results.txt, > sequential.fio > > > On read heavy workloads Cassandra performs much better when using a low read > ahead setting. In my tests I've seen an 5x improvement in throughput and > more than a 50% reduction in latency. However, I've also observed that it > can have a negative impact on compaction and streaming throughput. It > especially negatively impacts cloud environments where small reads incur high > costs in IOPS due to tiny requests. > # We should investigate using POSIX_FADV_DONTNEED on files we're compacting > to see if we can improve performance and reduce page faults. > # This should be combined with an internal read ahead style buffer that > Cassandra manages, similar to a BufferedInputStream but with our own > machinery. This buffer should read fairly large blocks of data off disk at > at time. EBS, for example, allows 1 IOP to be up to 256KB. A considerable > amount of time is spent in blocking I/O during compaction and streaming. > Reducing the frequency we read from disk should speed up all sequential I/O > operations. > # We can reduce system calls by buffering writes as well, but I think it > will have less of an impact than the reads -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15452) Improve disk access patterns during compaction and streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-15452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-15452: Attachment: iostat-ebs-15452.png iostat-ebs-head.png iostat-instance-15452.png iostat-instance-head.png > Improve disk access patterns during compaction and streaming > > > Key: CASSANDRA-15452 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15452 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Local Write-Read Paths, Local/Compaction >Reporter: Jon Haddad >Assignee: Jordan West >Priority: Normal > Attachments: everyfs.txt, iostat-ebs-15452.png, iostat-ebs-head.png, > iostat-instance-15452.png, iostat-instance-head.png, results.txt, > sequential.fio > > > On read heavy workloads Cassandra performs much better when using a low read > ahead setting. In my tests I've seen an 5x improvement in throughput and > more than a 50% reduction in latency. However, I've also observed that it > can have a negative impact on compaction and streaming throughput. It > especially negatively impacts cloud environments where small reads incur high > costs in IOPS due to tiny requests. > # We should investigate using POSIX_FADV_DONTNEED on files we're compacting > to see if we can improve performance and reduce page faults. > # This should be combined with an internal read ahead style buffer that > Cassandra manages, similar to a BufferedInputStream but with our own > machinery. This buffer should read fairly large blocks of data off disk at > at time. EBS, for example, allows 1 IOP to be up to 256KB. A considerable > amount of time is spent in blocking I/O during compaction and streaming. > Reducing the frequency we read from disk should speed up all sequential I/O > operations. > # We can reduce system calls by buffering writes as well, but I think it > will have less of an impact than the reads -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-129) Remove tableId from list snapshot response
[
https://issues.apache.org/jira/browse/CASSANDRASC-129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844821#comment-17844821
]
Francisco Guerrero commented on CASSANDRASC-129:
Rerunning:
https://app.circleci.com/pipelines/github/frankgh/cassandra-sidecar?branch=CASSANDRASC-129
> Remove tableId from list snapshot response
> --
>
> Key: CASSANDRASC-129
> URL: https://issues.apache.org/jira/browse/CASSANDRASC-129
> Project: Sidecar for Apache Cassandra
> Issue Type: Bug
> Components: Rest API
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Labels: pull-request-available
> Fix For: 1.0
>
>
> In CASSANDRASC-94, a new field {{tableId}} was added to the payload of the
> list snapshots endpoint. Old Sidecar clients are compatible with new fields
> added from the server side. However, other clients can potentially break and
> for that reason we should remove the {{tableId}} field from the response and
> make any potential breaking changes with an API version change instead. As an
> alternative, we can add the {{tableId}} as part of the {{tableName}} field
> instead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19627) Host replacement cannot start when nodes having different ports
[
https://issues.apache.org/jira/browse/CASSANDRA-19627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifan Cai updated CASSANDRA-19627:
--
Bug Category: Parent values: Availability(12983)Level 1 values: Process
Crash(12992)
Complexity: Normal
Discovered By: Unit Test
Severity: Normal
Status: Open (was: Triage Needed)
> Host replacement cannot start when nodes having different ports
> ---
>
> Key: CASSANDRA-19627
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19627
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Bootstrap and Decommission
>Reporter: Yifan Cai
>Priority: Normal
>
> CASSANDRA-7544 introduces configurable storage port per node. It means
> operator can pick different ports for nodes.
> In the case of host replacement, it cannot start if the ports are not the
> same of the replacing and the replacement nodes. The following is the test
> (modified from HostReplacementTest#replaceDownedHost) to prove and the
> failure stack trace.
> {code:java}
> @Test
> public void replaceDownedHost() throws IOException
> {
> // start with 2 nodes, stop both nodes, start the seed, host replace the
> down node)
> TokenSupplier even = TokenSupplier.evenlyDistributedTokens(2);
> try (Cluster cluster = Cluster.build(2)
> .withDynamicPortAllocation(true) // use a
> different storage port for each new node
> .withConfig(c -> c.with(Feature.GOSSIP,
> Feature.NETWORK))
> .withTokenSupplier(node -> even.token(node
> == 3 ? 2 : node))
> .start())
> {
> IInvokableInstance seed = cluster.get(1);
> IInvokableInstance nodeToRemove = cluster.get(2);
> setupCluster(cluster);
> // collect rows to detect issues later on if the state doesn't match
> SimpleQueryResult expectedState =
> nodeToRemove.coordinator().executeWithResult("SELECT * FROM " + KEYSPACE +
> ".tbl", ConsistencyLevel.ALL);
> stopUnchecked(nodeToRemove);
> // now create a new node to replace the other node
> IInvokableInstance replacingNode = replaceHostAndStart(cluster,
> nodeToRemove, props -> {
> // since we have a downed host there might be a schema version
> which is old show up but
> // can't be fetched since the host is down...
> props.set(BOOTSTRAP_SKIP_SCHEMA_CHECK, true);
> InetSocketAddress removedNodeAddress =
> nodeToRemove.config().broadcastAddress();
> String removedNode =
> removedNodeAddress.getAddress().getHostAddress() + ":" +
> removedNodeAddress.getPort();
> props.setProperty("cassandra.replace_address_first_boot",
> removedNode);
> });
> // wait till the replacing node is in the ring
> awaitRingJoin(seed, replacingNode);
> awaitRingJoin(replacingNode, seed);
> // make sure all nodes are healthy
> awaitRingHealthy(seed);
> assertRingIs(seed, seed, replacingNode);
> logger.info("Current ring is {}", assertRingIs(replacingNode, seed,
> replacingNode));
> validateRows(seed.coordinator(), expectedState);
> validateRows(replacingNode.coordinator(), expectedState);
> }
> }
> java.lang.RuntimeException: Node /127.0.0.3:58530 is already replacing
> /127.0.0.2:58495 but is trying to replace /127.0.0.2:58530.
> at
> org.apache.cassandra.service.StorageService.handleStateBootreplacing(StorageService.java:2929)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2597)
> at
> org.apache.cassandra.gms.Gossiper.doOnChangeNotifications(Gossiper.java:1711)
> at
> org.apache.cassandra.gms.Gossiper.addLocalApplicationStateInternal(Gossiper.java:2109)
> at
> org.apache.cassandra.gms.Gossiper.addLocalApplicationStates(Gossiper.java:2124)
> at
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:2005)
> at
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:1185)
> at
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:1145)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:936)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:854)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$startup$12(Instance.java:701)
> at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
> at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
> at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
> at
>
[jira] [Created] (CASSANDRA-19627) Host replacement cannot start when nodes having different ports
Yifan Cai created CASSANDRA-19627:
-
Summary: Host replacement cannot start when nodes having different
ports
Key: CASSANDRA-19627
URL: https://issues.apache.org/jira/browse/CASSANDRA-19627
Project: Cassandra
Issue Type: Bug
Components: Consistency/Bootstrap and Decommission
Reporter: Yifan Cai
CASSANDRA-7544 introduces configurable storage port per node. It means operator
can pick different ports for nodes.
In the case of host replacement, it cannot start if the ports are not the same
of the replacing and the replacement nodes. The following is the test (modified
from HostReplacementTest#replaceDownedHost) to prove and the failure stack
trace.
{code:java}
@Test
public void replaceDownedHost() throws IOException
{
// start with 2 nodes, stop both nodes, start the seed, host replace the
down node)
TokenSupplier even = TokenSupplier.evenlyDistributedTokens(2);
try (Cluster cluster = Cluster.build(2)
.withDynamicPortAllocation(true) // use a
different storage port for each new node
.withConfig(c -> c.with(Feature.GOSSIP,
Feature.NETWORK))
.withTokenSupplier(node -> even.token(node ==
3 ? 2 : node))
.start())
{
IInvokableInstance seed = cluster.get(1);
IInvokableInstance nodeToRemove = cluster.get(2);
setupCluster(cluster);
// collect rows to detect issues later on if the state doesn't match
SimpleQueryResult expectedState =
nodeToRemove.coordinator().executeWithResult("SELECT * FROM " + KEYSPACE +
".tbl", ConsistencyLevel.ALL);
stopUnchecked(nodeToRemove);
// now create a new node to replace the other node
IInvokableInstance replacingNode = replaceHostAndStart(cluster,
nodeToRemove, props -> {
// since we have a downed host there might be a schema version
which is old show up but
// can't be fetched since the host is down...
props.set(BOOTSTRAP_SKIP_SCHEMA_CHECK, true);
InetSocketAddress removedNodeAddress =
nodeToRemove.config().broadcastAddress();
String removedNode =
removedNodeAddress.getAddress().getHostAddress() + ":" +
removedNodeAddress.getPort();
props.setProperty("cassandra.replace_address_first_boot",
removedNode);
});
// wait till the replacing node is in the ring
awaitRingJoin(seed, replacingNode);
awaitRingJoin(replacingNode, seed);
// make sure all nodes are healthy
awaitRingHealthy(seed);
assertRingIs(seed, seed, replacingNode);
logger.info("Current ring is {}", assertRingIs(replacingNode, seed,
replacingNode));
validateRows(seed.coordinator(), expectedState);
validateRows(replacingNode.coordinator(), expectedState);
}
}
java.lang.RuntimeException: Node /127.0.0.3:58530 is already replacing
/127.0.0.2:58495 but is trying to replace /127.0.0.2:58530.
at
org.apache.cassandra.service.StorageService.handleStateBootreplacing(StorageService.java:2929)
at
org.apache.cassandra.service.StorageService.onChange(StorageService.java:2597)
at
org.apache.cassandra.gms.Gossiper.doOnChangeNotifications(Gossiper.java:1711)
at
org.apache.cassandra.gms.Gossiper.addLocalApplicationStateInternal(Gossiper.java:2109)
at
org.apache.cassandra.gms.Gossiper.addLocalApplicationStates(Gossiper.java:2124)
at
org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:2005)
at
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:1185)
at
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:1145)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:936)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:854)
at
org.apache.cassandra.distributed.impl.Instance.lambda$startup$12(Instance.java:701)
at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For a
[jira] [Commented] (CASSANDRASC-129) Remove tableId from list snapshot response
[
https://issues.apache.org/jira/browse/CASSANDRASC-129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844818#comment-17844818
]
Francisco Guerrero commented on CASSANDRASC-129:
CI :
https://app.circleci.com/pipelines/github/frankgh/cassandra-sidecar/487/workflows/c16708f8-0385-4928-b15a-628a0ae4de97
> Remove tableId from list snapshot response
> --
>
> Key: CASSANDRASC-129
> URL: https://issues.apache.org/jira/browse/CASSANDRASC-129
> Project: Sidecar for Apache Cassandra
> Issue Type: Bug
> Components: Rest API
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Labels: pull-request-available
> Fix For: 1.0
>
>
> In CASSANDRASC-94, a new field {{tableId}} was added to the payload of the
> list snapshots endpoint. Old Sidecar clients are compatible with new fields
> added from the server side. However, other clients can potentially break and
> for that reason we should remove the {{tableId}} field from the response and
> make any potential breaking changes with an API version change instead. As an
> alternative, we can add the {{tableId}} as part of the {{tableName}} field
> instead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-129) Remove tableId from list snapshot response
[
https://issues.apache.org/jira/browse/CASSANDRASC-129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844815#comment-17844815
]
Yifan Cai commented on CASSANDRASC-129:
---
+1, assume CI is green
> Remove tableId from list snapshot response
> --
>
> Key: CASSANDRASC-129
> URL: https://issues.apache.org/jira/browse/CASSANDRASC-129
> Project: Sidecar for Apache Cassandra
> Issue Type: Bug
> Components: Rest API
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Labels: pull-request-available
> Fix For: 1.0
>
>
> In CASSANDRASC-94, a new field {{tableId}} was added to the payload of the
> list snapshots endpoint. Old Sidecar clients are compatible with new fields
> added from the server side. However, other clients can potentially break and
> for that reason we should remove the {{tableId}} field from the response and
> make any potential breaking changes with an API version change instead. As an
> alternative, we can add the {{tableId}} as part of the {{tableName}} field
> instead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] JAVA-3142: Ability to optionally specify remote dcs for deterministic failovers when remote dcs are used in query plan [cassandra-java-driver]
nitinitt commented on PR #1896: URL: https://github.com/apache/cassandra-java-driver/pull/1896#issuecomment-2101687257 Thanks @absurdfarce! I have addressed the 2 minor documentation comments as part of: https://github.com/apache/cassandra-java-driver/pull/1933/files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19626) [Analytics] NullPointerException when reading static column with null values
[ https://issues.apache.org/jira/browse/CASSANDRA-19626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francisco Guerrero updated CASSANDRA-19626: --- Bug Category: Parent values: Correctness(12982)Level 1 values: Unrecoverable Corruption / Loss(13161) Complexity: Normal Discovered By: User Report Fix Version/s: NA Severity: Normal Status: Open (was: Triage Needed) > [Analytics] NullPointerException when reading static column with null values > > > Key: CASSANDRA-19626 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19626 > Project: Cassandra > Issue Type: Bug > Components: Analytics Library >Reporter: Francisco Guerrero >Assignee: Francisco Guerrero >Priority: Normal > Fix For: NA > > Time Spent: 10m > Remaining Estimate: 0h > > The analytics library fails to read static columns that has null values > depending on the type. For example, this problem is exhibited in the > Timestamp type, where the deserialized object is null, and the type will > attempt to get the time from the field. When that occurs it triggers the > NullPointerException. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19626) [Analytics] NullPointerException when reading static column with null values
[ https://issues.apache.org/jira/browse/CASSANDRA-19626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francisco Guerrero updated CASSANDRA-19626: --- Test and Documentation Plan: Added tests validating the fix Status: Patch Available (was: In Progress) > [Analytics] NullPointerException when reading static column with null values > > > Key: CASSANDRA-19626 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19626 > Project: Cassandra > Issue Type: Bug > Components: Analytics Library >Reporter: Francisco Guerrero >Assignee: Francisco Guerrero >Priority: Normal > Fix For: NA > > Time Spent: 10m > Remaining Estimate: 0h > > The analytics library fails to read static columns that has null values > depending on the type. For example, this problem is exhibited in the > Timestamp type, where the deserialized object is null, and the type will > attempt to get the time from the field. When that occurs it triggers the > NullPointerException. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[PR] JAVA-3142: Improving the documentation for remote local dc's feature [cassandra-java-driver]
nitinitt opened a new pull request, #1933: URL: https://github.com/apache/cassandra-java-driver/pull/1933 This PR is to address code review comment in merged [PR](https://github.com/apache/cassandra-java-driver/pull/1896) - https://github.com/apache/cassandra-java-driver/pull/1896#discussion_r1585228162 - https://github.com/apache/cassandra-java-driver/pull/1896#discussion_r1585228162 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[PR] CASSANDRA-19626 Fix NullPointerException when reading static column w… [cassandra-analytics]
frankgh opened a new pull request, #58: URL: https://github.com/apache/cassandra-analytics/pull/58 …ith null values Patch by Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRA-19626 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19626) [Analytics] NullPointerException when reading static column with null values
Francisco Guerrero created CASSANDRA-19626: -- Summary: [Analytics] NullPointerException when reading static column with null values Key: CASSANDRA-19626 URL: https://issues.apache.org/jira/browse/CASSANDRA-19626 Project: Cassandra Issue Type: Bug Components: Analytics Library Reporter: Francisco Guerrero Assignee: Francisco Guerrero The analytics library fails to read static columns that has null values depending on the type. For example, this problem is exhibited in the Timestamp type, where the deserialized object is null, and the type will attempt to get the time from the field. When that occurs it triggers the NullPointerException. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-129) Remove tableId from list snapshot response
[
https://issues.apache.org/jira/browse/CASSANDRASC-129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francisco Guerrero updated CASSANDRASC-129:
---
Authors: Francisco Guerrero
Test and Documentation Plan: Added tests for client compatibility. Updated
existing tests, and tested new utils methods
Status: Patch Available (was: In Progress)
> Remove tableId from list snapshot response
> --
>
> Key: CASSANDRASC-129
> URL: https://issues.apache.org/jira/browse/CASSANDRASC-129
> Project: Sidecar for Apache Cassandra
> Issue Type: Bug
> Components: Rest API
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Labels: pull-request-available
> Fix For: 1.0
>
>
> In CASSANDRASC-94, a new field {{tableId}} was added to the payload of the
> list snapshots endpoint. Old Sidecar clients are compatible with new fields
> added from the server side. However, other clients can potentially break and
> for that reason we should remove the {{tableId}} field from the response and
> make any potential breaking changes with an API version change instead. As an
> alternative, we can add the {{tableId}} as part of the {{tableName}} field
> instead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-129) Remove tableId from list snapshot response
[
https://issues.apache.org/jira/browse/CASSANDRASC-129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francisco Guerrero updated CASSANDRASC-129:
---
Bug Category: Parent values: Correctness(12982)Level 1 values:
Unrecoverable Corruption / Loss(13161)
Complexity: Low Hanging Fruit
Discovered By: User Report
Fix Version/s: 1.0
Reviewers: Yifan Cai
Severity: Normal
Status: Open (was: Triage Needed)
> Remove tableId from list snapshot response
> --
>
> Key: CASSANDRASC-129
> URL: https://issues.apache.org/jira/browse/CASSANDRASC-129
> Project: Sidecar for Apache Cassandra
> Issue Type: Bug
> Components: Rest API
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Labels: pull-request-available
> Fix For: 1.0
>
>
> In CASSANDRASC-94, a new field {{tableId}} was added to the payload of the
> list snapshots endpoint. Old Sidecar clients are compatible with new fields
> added from the server side. However, other clients can potentially break and
> for that reason we should remove the {{tableId}} field from the response and
> make any potential breaking changes with an API version change instead. As an
> alternative, we can add the {{tableId}} as part of the {{tableName}} field
> instead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-129) Remove tableId from list snapshot response
[
https://issues.apache.org/jira/browse/CASSANDRASC-129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CASSANDRASC-129:
---
Labels: pull-request-available (was: )
> Remove tableId from list snapshot response
> --
>
> Key: CASSANDRASC-129
> URL: https://issues.apache.org/jira/browse/CASSANDRASC-129
> Project: Sidecar for Apache Cassandra
> Issue Type: Bug
> Components: Rest API
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Labels: pull-request-available
>
> In CASSANDRASC-94, a new field {{tableId}} was added to the payload of the
> list snapshots endpoint. Old Sidecar clients are compatible with new fields
> added from the server side. However, other clients can potentially break and
> for that reason we should remove the {{tableId}} field from the response and
> make any potential breaking changes with an API version change instead. As an
> alternative, we can add the {{tableId}} as part of the {{tableName}} field
> instead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRASC-129) Remove tableId from list snapshot response
Francisco Guerrero created CASSANDRASC-129:
--
Summary: Remove tableId from list snapshot response
Key: CASSANDRASC-129
URL: https://issues.apache.org/jira/browse/CASSANDRASC-129
Project: Sidecar for Apache Cassandra
Issue Type: Bug
Components: Rest API
Reporter: Francisco Guerrero
Assignee: Francisco Guerrero
In CASSANDRASC-94, a new field {{tableId}} was added to the payload of the list
snapshots endpoint. Old Sidecar clients are compatible with new fields added
from the server side. However, other clients can potentially break and for that
reason we should remove the {{tableId}} field from the response and make any
potential breaking changes with an API version change instead. As an
alternative, we can add the {{tableId}} as part of the {{tableName}} field
instead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] Add ExecutionInfo to RequestTracker methods [cassandra-java-driver]
ajweave commented on PR #1640: URL: https://github.com/apache/cassandra-java-driver/pull/1640#issuecomment-2101654238 Sorry for the delay. I will get this done tomorrow. I appreciate the attention here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19556) Guardrail to block DDL/DCL queries
[ https://issues.apache.org/jira/browse/CASSANDRA-19556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844801#comment-17844801 ] Yuqi Yan commented on CASSANDRA-19556: -- [~smiklosovic] thanks for posting this in ML and the follow-ups here. Yes that sg to me. > Guardrail to block DDL/DCL queries > -- > > Key: CASSANDRA-19556 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19556 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Guardrails >Reporter: Yuqi Yan >Assignee: Yuqi Yan >Priority: Normal > Fix For: 5.x > > Time Spent: 1.5h > Remaining Estimate: 0h > > Sometimes we want to block DDL/DCL queries to stop new schemas being created > or roles created. (e.g. when doing live-upgrade) > For DDL guardrail current implementation won't block the query if it's no-op > (e.g. CREATE TABLE...IF NOT EXISTS, but table already exists, etc. The > guardrail check is added in apply() right after all the existence check) > I don't have preference on either block every DDL query or check whether if > it's no-op here. Just we have some users always run CREATE..IF NOT EXISTS.. > at startup, which is no-op but will be blocked by this guardrail and failed > to start. > > 4.1 PR: [https://github.com/apache/cassandra/pull/3248] > trunk PR: [https://github.com/apache/cassandra/pull/3275] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] Add ExecutionInfo to RequestTracker methods [cassandra-java-driver]
SiyaoIsHiding commented on PR #1640: URL: https://github.com/apache/cassandra-java-driver/pull/1640#issuecomment-2101624573 May you please merge apache/4.x to your branch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] Add ExecutionInfo to RequestTracker methods [cassandra-java-driver]
SiyaoIsHiding commented on code in PR #1640:
URL:
https://github.com/apache/cassandra-java-driver/pull/1640#discussion_r1594781483
##
core/src/main/java/com/datastax/dse/driver/internal/core/cql/continuous/ContinuousRequestHandlerBase.java:
##
@@ -1590,18 +1597,20 @@ private void completeResultSetFuture(
private ExecutionInfo createExecutionInfo(@NonNull Result result,
@Nullable Frame response) {
ByteBuffer pagingState =
result instanceof Rows ? ((Rows) result).getMetadata().pagingState :
null;
- return new DefaultExecutionInfo(
- statement,
- node,
- startedSpeculativeExecutionsCount.get(),
- executionIndex,
- errors,
- pagingState,
- response,
- true,
- session,
- context,
- executionProfile);
+ this.executionInfo =
Review Comment:
It's a bit of work to set up. I ran all the tests against this branch, and
it passed all the tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19625) Integrate Cassandra Analytics with SonarQube
[ https://issues.apache.org/jira/browse/CASSANDRA-19625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuriy Semchyshyn updated CASSANDRA-19625: - Source Control Link: https://github.com/apache/cassandra-analytics/pull/57 > Integrate Cassandra Analytics with SonarQube > > > Key: CASSANDRA-19625 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19625 > Project: Cassandra > Issue Type: Improvement > Components: Analytics Library >Reporter: Yuriy Semchyshyn >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > > Introduce the initial Gradle configuration for integration of Cassandra > Analytics with SonarCube for code analysis -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-128) Integrate Cassandra Sidecar with SonarQube
[ https://issues.apache.org/jira/browse/CASSANDRASC-128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuriy Semchyshyn updated CASSANDRASC-128: - Source Control Link: https://github.com/apache/cassandra-sidecar/pull/119 > Integrate Cassandra Sidecar with SonarQube > -- > > Key: CASSANDRASC-128 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-128 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Configuration >Reporter: Yuriy Semchyshyn >Priority: Normal > Labels: pull-request-available > > Introduce the initial Gradle configuration for integration of Cassandra > Sidecar with SonarCube for code analysis -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-128) Integrate Cassandra Sidecar with SonarQube
[ https://issues.apache.org/jira/browse/CASSANDRASC-128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CASSANDRASC-128: --- Labels: pull-request-available (was: ) > Integrate Cassandra Sidecar with SonarQube > -- > > Key: CASSANDRASC-128 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-128 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Configuration >Reporter: Yuriy Semchyshyn >Priority: Normal > Labels: pull-request-available > > Introduce the initial Gradle configuration for integration of Cassandra > Sidecar with SonarCube for code analysis -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[PR] [CASSANDRA-19625] Initial Configuration for SonarCube Analysis [cassandra-analytics]
5 opened a new pull request, #57: URL: https://github.com/apache/cassandra-analytics/pull/57 Introduce the initial Gradle configuration for integration of Cassandra Analytics with SonarCube for code analysis -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19625) Integrate Cassandra Analytics with SonarQube
Yuriy Semchyshyn created CASSANDRA-19625: Summary: Integrate Cassandra Analytics with SonarQube Key: CASSANDRA-19625 URL: https://issues.apache.org/jira/browse/CASSANDRA-19625 Project: Cassandra Issue Type: Improvement Components: Analytics Library Reporter: Yuriy Semchyshyn Introduce the initial Gradle configuration for integration of Cassandra Analytics with SonarCube for code analysis -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRASC-128) Integrate Cassandra Sidecar with SonarQube
Yuriy Semchyshyn created CASSANDRASC-128: Summary: Integrate Cassandra Sidecar with SonarQube Key: CASSANDRASC-128 URL: https://issues.apache.org/jira/browse/CASSANDRASC-128 Project: Sidecar for Apache Cassandra Issue Type: Improvement Components: Configuration Reporter: Yuriy Semchyshyn Introduce the initial Gradle configuration for integration of Cassandra Sidecar with SonarCube for code analysis -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19556) Guardrail to block DDL/DCL queries
[ https://issues.apache.org/jira/browse/CASSANDRA-19556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844790#comment-17844790 ] Stefan Miklosovic commented on CASSANDRA-19556: --- [~yukei] I have created a PR against your branch which removes CASSANDRA-17495 and I refactored the tests little bit. To speed up things a little bit, I have created 5.0 PR here. Based on the ML link above, I think we need to remove CASSANDRA-17495 and incorporate these two guardrails into that. I think that we will work on 5.0 patch for now, I will take it from here if you do not mind. Does this sound OK for you? (1) https://github.com/Yukei7/cassandra/pull/1/files (2) https://github.com/apache/cassandra/pull/3298 > Guardrail to block DDL/DCL queries > -- > > Key: CASSANDRA-19556 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19556 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Guardrails >Reporter: Yuqi Yan >Assignee: Yuqi Yan >Priority: Normal > Fix For: 5.x > > Time Spent: 1.5h > Remaining Estimate: 0h > > Sometimes we want to block DDL/DCL queries to stop new schemas being created > or roles created. (e.g. when doing live-upgrade) > For DDL guardrail current implementation won't block the query if it's no-op > (e.g. CREATE TABLE...IF NOT EXISTS, but table already exists, etc. The > guardrail check is added in apply() right after all the existence check) > I don't have preference on either block every DDL query or check whether if > it's no-op here. Just we have some users always run CREATE..IF NOT EXISTS.. > at startup, which is no-op but will be blocked by this guardrail and failed > to start. > > 4.1 PR: [https://github.com/apache/cassandra/pull/3248] > trunk PR: [https://github.com/apache/cassandra/pull/3275] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[PR] An experiment [cassandra-java-driver]
absurdfarce opened a new pull request, #1932: URL: https://github.com/apache/cassandra-java-driver/pull/1932 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15439) Token metadata for bootstrapping nodes is lost under temporary failures
[
https://issues.apache.org/jira/browse/CASSANDRA-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844744#comment-17844744
]
Raymond Huffman commented on CASSANDRA-15439:
-
Thank you!
> Token metadata for bootstrapping nodes is lost under temporary failures
> ---
>
> Key: CASSANDRA-15439
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15439
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Membership
>Reporter: Josh Snyder
>Assignee: Raymond Huffman
>Priority: Normal
> Fix For: 4.0.14, 4.1.6, 5.0-beta2, 5.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> In CASSANDRA-8838, [~pauloricardomg] asked "hints will not be stored to the
> bootstrapping node after RING_DELAY, since it will evicted from the TMD
> pending ranges. Should we create a ticket to address this?"
> CASSANDRA-15264 relates to the most likely cause of such situations, where
> the Cassandra daemon on the bootstrapping node completely crashes. Based on
> testing with {{kill -STOP}} on a bootstrapping Cassandra JVM, I believe it
> also is possible to remove token metadata (and thus pending ranges, and thus
> hints) for a bootstrapping node, simply by affecting its status in the
> failure detector.
> A node in the cluster sees the bootstrapping node this way:
> {noformat}
> INFO [GossipStage:1] 2019-11-27 20:41:41,101 Gossiper.java: - Node
> /PUBLIC-IP is now part of the cluster
> INFO [GossipStage:1] 2019-11-27 20:41:41,199 Gossiper.java:1073 -
> InetAddress /PUBLIC-IP is now UP
> INFO [HANDSHAKE-/PRIVATE-IP] 2019-11-27 20:41:41,412
> OutboundTcpConnection.java:565 - Handshaking version with /PRIVATE-IP
> INFO [STREAM-INIT-/PRIVATE-IP:21233] 2019-11-27 20:42:10,019
> StreamResultFuture.java:112 - [Stream #6219a950-1156-11ea-b45d-4d30364576c4
> ID#0] Creating new streaming plan for Bootstrap
> INFO [STREAM-INIT-/PRIVATE-IP:21233] 2019-11-27 20:42:10,020
> StreamResultFuture.java:119 - [Stream #6219a950-1156-11ea-b45d-4d30364576c4,
> ID#0] Received streaming plan for Bootstrap
> INFO [STREAM-INIT-/PRIVATE-IP:56003] 2019-11-27 20:42:10,112
> StreamResultFuture.java:119 - [Stream #6219a950-1156-11ea-b45d-4d30364576c4,
> ID#0] Received streaming plan for Bootstrap
> INFO [STREAM-IN-/PUBLIC-IP] 2019-11-27 20:42:10,179
> StreamResultFuture.java:169 - [Stream #6219a950-1156-11ea-b45d-4d30364576c4
> ID#0] Prepare completed. Receiving 0 files(0 bytes), sending 833
> files(139744616815 bytes)
> INFO [GossipStage:1] 2019-11-27 20:54:47,547 Gossiper.java:1089 -
> InetAddress /PUBLIC-IP is now DOWN
> INFO [GossipTasks:1] 2019-11-27 20:54:57,551 Gossiper.java:849 - FatClient
> /PUBLIC-IP has been silent for 3ms, removing from gossip
> {noformat}
> Since the bootstrapping node has no tokens, it is treated like a fat client,
> and it is removed from the ring. For correctness purposes, I believe we must
> keep storing hints for the downed bootstrapping node until it is either
> assassinated or until a replacement attempts to bootstrap for the same token.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19619) Enforce contract for internal metrics naming
[
https://issues.apache.org/jira/browse/CASSANDRA-19619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated CASSANDRA-19619:
Fix Version/s: 5.0-beta2
5.0
5.1
(was: 5.x)
(was: 5.0.x)
(was: 5.0-rc)
Source Control Link:
https://github.com/apache/cassandra/commit/ccdeb12419a70afb00e9aa3958fc05eb4617ee44
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
[~maedhroz] thank you for the review and running CI.
[~mck] thank you for the review either.
Committed to 5.0 and the trunk.
> Enforce contract for internal metrics naming
>
>
> Key: CASSANDRA-19619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19619
> Project: Cassandra
> Issue Type: Task
> Components: Observability/Metrics
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.0-beta2, 5.0, 5.1
>
> Attachments: ci_summary-1.html, ci_summary.html
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Metrics have their internal representation which uniquely identifies a
> particular metric name. The name is formed as {{. type>..}}. For some metrics, the scope includes a
> metric name that is not necessary for uniqueness, this can be simplified.
> For example,
> {code}
> // AS IS
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics.BalancedTreeIntersectionEarlyExits
> // TO BE
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics
> {code}
> The metrics are filtered based on knowledge of how they are formatted, so
> having a metric scope without a built-in metric name also simplifies the way
> we filter metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 01/01: Merge branch 'cassandra-5.0' into trunk
This is an automated email from the ASF dual-hosted git repository. mmuzaf pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 6bae4f76fb043b4c3a3886178b5650b280e9a50b Merge: 7f4a0f0c00 ccdeb12419 Author: Maxim Muzafarov AuthorDate: Wed May 8 19:49:54 2024 +0200 Merge branch 'cassandra-5.0' into trunk * cassandra-5.0: Enforce metric naming contract if scope is used in a metric name CHANGES.txt | 1 + src/java/org/apache/cassandra/index/sai/metrics/AbstractMetrics.java | 2 +- src/java/org/apache/cassandra/metrics/CassandraMetricsRegistry.java | 5 + 3 files changed, 7 insertions(+), 1 deletion(-) diff --cc CHANGES.txt index 6b391cc980,d3400ec57a..7ddd707989 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,45 -1,5 +1,46 @@@ -5.0-beta2 +5.1 + * Refactor Relation and Restriction hierarchies (CASSANDRA-19341) + * Raise priority of TCM internode messages during critical operations (CASSANDRA-19517) + * Add nodetool command to unregister LEFT nodes (CASSANDRA-19581) + * Add cluster metadata id to gossip syn messages (CASSANDRA-19613) + * Reduce heap usage occupied by the metrics (CASSANDRA-19567) + * Improve handling of transient replicas during range movements (CASSANDRA-19344) + * Enable debounced internode log requests to be cancelled at shutdown (CASSANDRA-19514) + * Correctly set last modified epoch when combining multistep operations into a single step (CASSANDRA-19538) + * Add new TriggersPolicy configuration to allow operators to disable triggers (CASSANDRA-19532) + * Use Transformation.Kind.id in local and distributed log tables (CASSANDRA-19516) + * Remove period field from ClusterMetadata and metadata log tables (CASSANDRA-19482) + * Enrich system_views.pending_hints vtable with hints sizes (CASSANDRA-19486) + * Expose all dropwizard metrics in virtual tables (CASSANDRA-14572) + * Ensured that PropertyFileSnitchTest do not overwrite cassandra-toploogy.properties (CASSANDRA-19502) + * Add option for MutualTlsAuthenticator to restrict the certificate validity period (CASSANDRA-18951) + * Fix StorageService::constructRangeToEndpointMap for non-distributed keyspaces (CASSANDRA-19255) + * Group nodetool cms commands into single command group (CASSANDRA-19393) + * Register the measurements of the bootstrap process as Dropwizard metrics (CASSANDRA-19447) + * Add LIST SUPERUSERS CQL statement (CASSANDRA-19417) + * Modernize CQLSH datetime conversions (CASSANDRA-18879) + * Harry model and in-JVM tests for partition-restricted 2i queries (CASSANDRA-18275) + * Refactor cqlshmain global constants (CASSANDRA-19201) + * Remove native_transport_port_ssl (CASSANDRA-19397) + * Make nodetool reconfigurecms sync by default and add --cancel to be able to cancel ongoing reconfigurations (CASSANDRA-19216) + * Expose auth mode in system_views.clients, nodetool clientstats, metrics (CASSANDRA-19366) + * Remove sealed_periods and last_sealed_period tables (CASSANDRA-19189) + * Improve setup and initialisation of LocalLog/LogSpec (CASSANDRA-19271) + * Refactor structure of caching metrics and expose auth cache metrics via JMX (CASSANDRA-17062) + * Allow CQL client certificate authentication to work without sending an AUTHENTICATE request (CASSANDRA-18857) + * Extend nodetool tpstats and system_views.thread_pools with detailed pool parameters (CASSANDRA-19289) + * Remove dependency on Sigar in favor of OSHI (CASSANDRA-16565) + * Simplify the bind marker and Term logic (CASSANDRA-18813) + * Limit cassandra startup to supported JDKs, allow higher JDKs by setting CASSANDRA_JDK_UNSUPPORTED (CASSANDRA-18688) + * Standardize nodetool tablestats formatting of data units (CASSANDRA-19104) + * Make nodetool tablestats use number of significant digits for time and average values consistently (CASSANDRA-19015) + * Upgrade jackson to 2.15.3 and snakeyaml to 2.1 (CASSANDRA-18875) + * Transactional Cluster Metadata [CEP-21] (CASSANDRA-18330) + * Add ELAPSED command to cqlsh (CASSANDRA-18861) + * Add the ability to disable bulk loading of SSTables (CASSANDRA-18781) + * Clean up obsolete functions and simplify cql_version handling in cqlsh (CASSANDRA-18787) +Merged from 5.0: + * Enforce metric naming contract if scope is used in a metric name (CASSANDRA-19619) * Avoid reading of the same IndexInfo from disk many times for a large partition (CASSANDRA-19557) * Resolve the oldest hints just from descriptors and current writer if available (CASSANDRA-19600) * Optionally fail writes when SAI refuses to index a term value exceeding configured term max size (CASSANDRA-19493) diff --cc src/java/org/apache/cassandra/index/sai/metrics/AbstractMetrics.java index 17fd99291c,1bb0a7666a..642cfd24fc --- a/src/java/org/apache/cassandra/index/sai/metrics/AbstractMetrics.java +++ b/src/java/org/apache/cassandra/index/sai/metrics/AbstractMetrics.java @@@
(cassandra) branch trunk updated (7f4a0f0c00 -> 6bae4f76fb)
This is an automated email from the ASF dual-hosted git repository. mmuzaf pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from 7f4a0f0c00 Merge branch 'cassandra-5.0' into trunk add ccdeb12419 Enforce metric naming contract if scope is used in a metric name new 6bae4f76fb Merge branch 'cassandra-5.0' into trunk The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 1 + src/java/org/apache/cassandra/index/sai/metrics/AbstractMetrics.java | 2 +- src/java/org/apache/cassandra/metrics/CassandraMetricsRegistry.java | 5 + 3 files changed, 7 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-5.0 updated (3b8c48024e -> ccdeb12419)
This is an automated email from the ASF dual-hosted git repository. mmuzaf pushed a change to branch cassandra-5.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 3b8c48024e Merge branch 'cassandra-4.1' into cassandra-5.0 add ccdeb12419 Enforce metric naming contract if scope is used in a metric name No new revisions were added by this update. Summary of changes: CHANGES.txt | 1 + src/java/org/apache/cassandra/index/sai/metrics/AbstractMetrics.java | 2 +- src/java/org/apache/cassandra/metrics/CassandraMetricsRegistry.java | 5 + 3 files changed, 7 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-java-driver) branch 4.x updated: JAVA-3142: Ability to specify ordering of remote local dc's via new configuration for graceful automatic failovers
This is an automated email from the ASF dual-hosted git repository.
absurdfarce pushed a commit to branch 4.x
in repository https://gitbox.apache.org/repos/asf/cassandra-java-driver.git
The following commit(s) were added to refs/heads/4.x by this push:
new b9760b473 JAVA-3142: Ability to specify ordering of remote local dc's
via new configuration for graceful automatic failovers
b9760b473 is described below
commit b9760b473b6e6e30f5da5f743e37e02150e13e39
Author: Nitin Chhabra
AuthorDate: Thu Nov 30 12:38:23 2023 -0800
JAVA-3142: Ability to specify ordering of remote local dc's via new
configuration for graceful automatic failovers
patch by Nitin Chhabra; reviewed by Alexandre Dutra, Andy Tolbert, and Bret
McGuire for JAVA-3142
---
.../api/core/config/DefaultDriverOption.java | 8 +-
.../oss/driver/api/core/config/OptionsMap.java | 2 +
.../driver/api/core/config/TypedDriverOption.java | 10 ++
.../loadbalancing/BasicLoadBalancingPolicy.java| 84 +++---
core/src/main/resources/reference.conf | 5 +
...cLoadBalancingPolicyPreferredRemoteDcsTest.java | 184 +
6 files changed, 271 insertions(+), 22 deletions(-)
diff --git
a/core/src/main/java/com/datastax/oss/driver/api/core/config/DefaultDriverOption.java
b/core/src/main/java/com/datastax/oss/driver/api/core/config/DefaultDriverOption.java
index afe16e968..11f2702c3 100644
---
a/core/src/main/java/com/datastax/oss/driver/api/core/config/DefaultDriverOption.java
+++
b/core/src/main/java/com/datastax/oss/driver/api/core/config/DefaultDriverOption.java
@@ -982,7 +982,13 @@ public enum DefaultDriverOption implements DriverOption {
* Value-type: {@link java.time.Duration}
*/
SSL_KEYSTORE_RELOAD_INTERVAL("advanced.ssl-engine-factory.keystore-reload-interval"),
- ;
+ /**
+ * Ordered preference list of remote dcs optionally supplied for automatic
failover.
+ *
+ * Value type: {@link java.util.List List}<{@link String}>
+ */
+ LOAD_BALANCING_DC_FAILOVER_PREFERRED_REMOTE_DCS(
+ "advanced.load-balancing-policy.dc-failover.preferred-remote-dcs");
private final String path;
diff --git
a/core/src/main/java/com/datastax/oss/driver/api/core/config/OptionsMap.java
b/core/src/main/java/com/datastax/oss/driver/api/core/config/OptionsMap.java
index 8906e1dd3..98faf3e59 100644
--- a/core/src/main/java/com/datastax/oss/driver/api/core/config/OptionsMap.java
+++ b/core/src/main/java/com/datastax/oss/driver/api/core/config/OptionsMap.java
@@ -381,6 +381,8 @@ public class OptionsMap implements Serializable {
map.put(TypedDriverOption.LOAD_BALANCING_DC_FAILOVER_MAX_NODES_PER_REMOTE_DC,
0);
map.put(TypedDriverOption.LOAD_BALANCING_DC_FAILOVER_ALLOW_FOR_LOCAL_CONSISTENCY_LEVELS,
false);
map.put(TypedDriverOption.METRICS_GENERATE_AGGREGABLE_HISTOGRAMS, true);
+map.put(
+TypedDriverOption.LOAD_BALANCING_DC_FAILOVER_PREFERRED_REMOTE_DCS,
ImmutableList.of(""));
}
@Immutable
diff --git
a/core/src/main/java/com/datastax/oss/driver/api/core/config/TypedDriverOption.java
b/core/src/main/java/com/datastax/oss/driver/api/core/config/TypedDriverOption.java
index 88c012fa3..ca60b67f0 100644
---
a/core/src/main/java/com/datastax/oss/driver/api/core/config/TypedDriverOption.java
+++
b/core/src/main/java/com/datastax/oss/driver/api/core/config/TypedDriverOption.java
@@ -892,6 +892,16 @@ public class TypedDriverOption {
DefaultDriverOption.LOAD_BALANCING_DC_FAILOVER_ALLOW_FOR_LOCAL_CONSISTENCY_LEVELS,
GenericType.BOOLEAN);
+ /**
+ * Ordered preference list of remote dcs optionally supplied for automatic
failover and included
+ * in query plan. This feature is enabled only when max-nodes-per-remote-dc
is greater than 0.
+ */
+ public static final TypedDriverOption>
+ LOAD_BALANCING_DC_FAILOVER_PREFERRED_REMOTE_DCS =
+ new TypedDriverOption<>(
+
DefaultDriverOption.LOAD_BALANCING_DC_FAILOVER_PREFERRED_REMOTE_DCS,
+ GenericType.listOf(String.class));
+
private static Iterable> introspectBuiltInValues() {
try {
ImmutableList.Builder> result =
ImmutableList.builder();
diff --git
a/core/src/main/java/com/datastax/oss/driver/internal/core/loadbalancing/BasicLoadBalancingPolicy.java
b/core/src/main/java/com/datastax/oss/driver/internal/core/loadbalancing/BasicLoadBalancingPolicy.java
index b1adec3f1..587ef4183 100644
---
a/core/src/main/java/com/datastax/oss/driver/internal/core/loadbalancing/BasicLoadBalancingPolicy.java
+++
b/core/src/main/java/com/datastax/oss/driver/internal/core/loadbalancing/BasicLoadBalancingPolicy.java
@@ -45,10 +45,14 @@ import
com.datastax.oss.driver.internal.core.util.collection.LazyQueryPlan;
import com.datastax.oss.driver.internal.core.util.collection.QueryPlan;
import com.datastax.oss.driver.internal.core.util.collection.SimpleQueryPlan;
import com.datastax.oss.driver.shaded.guava.common.base.Predica
Re: [PR] JAVA-3142: Ability to optionally specify remote dcs for deterministic failovers when remote dcs are used in query plan [cassandra-java-driver]
absurdfarce merged PR #1896: URL: https://github.com/apache/cassandra-java-driver/pull/1896 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] JAVA-3142: Ability to optionally specify remote dcs for deterministic failovers when remote dcs are used in query plan [cassandra-java-driver]
absurdfarce commented on PR #1896: URL: https://github.com/apache/cassandra-java-driver/pull/1896#issuecomment-2101007620 Thanks @nitinitt, this looks great! I concur that this commit looks like it contains all the fixes we've been discussing so we are ready to get this guy in! Big thank you for all your help in making this happen! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] JAVA-3142: Ability to optionally specify remote dcs for deterministic failovers when remote dcs are used in query plan [cassandra-java-driver]
absurdfarce commented on code in PR #1896:
URL:
https://github.com/apache/cassandra-java-driver/pull/1896#discussion_r1585228162
##
core/src/main/resources/reference.conf:
##
@@ -574,6 +574,11 @@ datastax-java-driver {
# Modifiable at runtime: no
# Overridable in a profile: yes
allow-for-local-consistency-levels = false
+ # Ordered preference list of remote dc's (in order) optionally supplied
for automatic failover. While building a query plan, the driver uses the DC's
supplied in order together with max-nodes-per-remote-dc
Review Comment:
Another nit: might be worth mentioning that users aren't required to specify
all DCs when listing preferences via this config.
##
core/src/main/resources/reference.conf:
##
@@ -574,6 +574,11 @@ datastax-java-driver {
# Modifiable at runtime: no
# Overridable in a profile: yes
allow-for-local-consistency-levels = false
+ # Ordered preference list of remote dc's (in order) optionally supplied
for automatic failover. While building a query plan, the driver uses the DC's
supplied in order together with max-nodes-per-remote-dc
Review Comment:
Nit: should be a space between this and allow-for-local-consistency-levels
in order to make it clear we're talking about two distinct configs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19623) Read fail with "illegal RT bounds sequence" after migrating data from 2.2.19 to 3.0.30
[
https://issues.apache.org/jira/browse/CASSANDRA-19623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Klay updated CASSANDRA-19623:
-
Description:
After migrating data from 2.2.19 to 3.0.30, reading the legacy data in 3.0.30
would fail with the following exception.
{code:java}
INFO [main] 2024-05-08 00:42:42,397 CassandraDaemon.java:653 - Startup complete
ERROR [SharedPool-Worker-2] 2024-05-08 00:42:49,975
AbstractLocalAwareExecutorService.java:166 - Uncaught exception on thread
Thread[SharedPool-Worker-2,10,main]
java.lang.RuntimeException: java.lang.IllegalStateException: SSTABLE
UnfilteredRowIterator for ks.tb has an illegal RT bounds sequence: unexpected
end bound or boundary Marker INCL_START_BOUND(1)@1715128933051779/1715128933
at
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2656)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.IllegalStateException: SSTABLE UnfilteredRowIterator for
ks.tb has an illegal RT bounds sequence: unexpected end bound or boundary
Marker INCL_START_BOUND(1)@1715128933051779/1715128933
at
org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.ise(RTBoundValidator.java:120)
at
org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.applyToMarker(RTBoundValidator.java:87)
at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:144)
at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:131)
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:77)
at
org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:297)
at
org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:187)
at
org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:180)
at
org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:176)
at
org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:76)
at
org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:347)
at
org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1914)
at
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2652)
... 5 common frames omitted {code}
h1. Reproduce
1. Start up cassandra-2.2.19, single node. Set column_index_size_in_kb in
cassandra.yaml to 1.
{code:java}
// cassandra.yaml
column_index_size_in_kb: 1 {code}
2. Execute the following commands (I masked out all data in the INSERT command
using a long string of 'A' to make it simpler)
{code:java}
CREATE KEYSPACE ks WITH REPLICATION = { 'class' : 'SimpleStrategy',
'replication_factor' : 1 };
CREATE TABLE ks.tb (c3 INT, c7 INT, c0 TEXT,c6 set, PRIMARY KEY (c3, c7,
c0));
DELETE FROM ks.tb WHERE c3 = 1 AND c7 = 1;
DELETE FROM ks.tb WHERE c3 = 1 AND c7 = 1 AND c0 = 'e';
INSERT INTO ks.tb (c7, c3, c6, c0) VALUES
(1,1,{'','','AAA'},'');
{code}
Perform a read in the old version, it works normally
{code:java}
cqlsh> SELECT c7, c0, c3 FROM ks.tb WHERE c3 = 1 AND c7 = 1 ORDER BY c7 DESC;
c7 | c0 | c3
+--+
1 | | 1
(1 rows) {code}
3. Drain and stop 2.2.19.
{code:java}
bin/nodetool -h :::127.0.0.1 drain
bin/nodetool -h
[jira] [Updated] (CASSANDRA-19623) Read fail with "illegal RT bounds sequence" after migrating data from 2.2.19 to 3.0.30
[
https://issues.apache.org/jira/browse/CASSANDRA-19623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Klay updated CASSANDRA-19623:
-
Description:
After migrating data from 2.2.19 to 3.0.30, reading the legacy data in 3.0.30
would fail with the following exception.
{code:java}
INFO [main] 2024-05-08 00:42:42,397 CassandraDaemon.java:653 - Startup complete
ERROR [SharedPool-Worker-2] 2024-05-08 00:42:49,975
AbstractLocalAwareExecutorService.java:166 - Uncaught exception on thread
Thread[SharedPool-Worker-2,10,main]
java.lang.RuntimeException: java.lang.IllegalStateException: SSTABLE
UnfilteredRowIterator for ks.tb has an illegal RT bounds sequence: unexpected
end bound or boundary Marker INCL_START_BOUND(1)@1715128933051779/1715128933
at
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2656)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.IllegalStateException: SSTABLE UnfilteredRowIterator for
ks.tb has an illegal RT bounds sequence: unexpected end bound or boundary
Marker INCL_START_BOUND(1)@1715128933051779/1715128933
at
org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.ise(RTBoundValidator.java:120)
at
org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.applyToMarker(RTBoundValidator.java:87)
at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:144)
at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:131)
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:77)
at
org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:297)
at
org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:187)
at
org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:180)
at
org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:176)
at
org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:76)
at
org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:347)
at
org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1914)
at
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2652)
... 5 common frames omitted {code}
h1. Reproduce
1. Start up cassandra-2.2.19, single node. Set column_index_size_in_kb in
cassandra.yaml to 1.
{code:java}
// cassandra.yaml
column_index_size_in_kb: 1 {code}
2. Execute the following commands (I masked out all data in the INSERT command
using a long string of 'A' to make it simpler)
{code:java}
CREATE KEYSPACE ks WITH REPLICATION = { 'class' : 'SimpleStrategy',
'replication_factor' : 1 };
CREATE TABLE ks.tb (c3 INT, c7 INT, c0 TEXT,c6 set, PRIMARY KEY (c3, c7,
c0));
DELETE FROM ks.tb WHERE c3 = 1 AND c7 = 1;
DELETE FROM ks.tb WHERE c3 = 1 AND c7 = 1 AND c0 = 'e';
INSERT INTO ks.tb (c7, c3, c6, c0) VALUES
(1,1,{'','','AAA'},'');
{code}
Perform a read in the old version, it works normally
{code:java}
cqlsh> SELECT c7, c0, c3 FROM ks.tb WHERE c3 = 1 AND c7 = 1 ORDER BY c7 DESC;
c7 | c0 | c3
+--+
1 | | 1
(1 rows) {code}
3. Drain and stop 2.2.19.
{code:java}
bin/nodetool -h :::127.0.0.1 drain
bin/nodetool -h
[jira] [Updated] (CASSANDRA-19623) Read fail with "illegal RT bounds sequence" after migrating data from 2.2.19 to 3.0.30
[
https://issues.apache.org/jira/browse/CASSANDRA-19623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Klay updated CASSANDRA-19623:
-
Description:
After migrating data from 2.2.19 to 3.0.30, reading the legacy data in 3.0.30
would fail with the following exception.
{code:java}
INFO [main] 2024-05-08 00:42:42,397 CassandraDaemon.java:653 - Startup complete
ERROR [SharedPool-Worker-2] 2024-05-08 00:42:49,975
AbstractLocalAwareExecutorService.java:166 - Uncaught exception on thread
Thread[SharedPool-Worker-2,10,main]
java.lang.RuntimeException: java.lang.IllegalStateException: SSTABLE
UnfilteredRowIterator for ks.tb has an illegal RT bounds sequence: unexpected
end bound or boundary Marker INCL_START_BOUND(1)@1715128933051779/1715128933
at
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2656)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.IllegalStateException: SSTABLE UnfilteredRowIterator for
ks.tb has an illegal RT bounds sequence: unexpected end bound or boundary
Marker INCL_START_BOUND(1)@1715128933051779/1715128933
at
org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.ise(RTBoundValidator.java:120)
at
org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.applyToMarker(RTBoundValidator.java:87)
at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:144)
at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:131)
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:77)
at
org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:297)
at
org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:187)
at
org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:180)
at
org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:176)
at
org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:76)
at
org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:347)
at
org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1914)
at
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2652)
... 5 common frames omitted {code}
h1. Reproduce
1. Start up cassandra-2.2.19, single node. Set column_index_size_in_kb in
cassandra.yaml to 1.
{code:java}
// cassandra.yaml
column_index_size_in_kb: 1 {code}
2. Execute the following commands (I masked out all data in the INSERT command
using a long string of 'A' to make it simpler)
{code:java}
CREATE KEYSPACE ks WITH REPLICATION = { 'class' : 'SimpleStrategy',
'replication_factor' : 1 };
CREATE TABLE ks.tb (c3 INT, c7 INT, c0 TEXT,c6 set, PRIMARY KEY (c3, c7,
c0));
DELETE FROM ks.tb WHERE c3 = 1 AND c7 = 1;
DELETE FROM ks.tb WHERE c3 = 1 AND c7 = 1 AND c0 = 'e';
INSERT INTO ks.tb (c7, c3, c6, c0) VALUES
(1,1,{'','','AAA'},'');
{code}
Perform a read in the old version, it works normally
{code:java}
cqlsh> SELECT c7, c0, c3 FROM ks.tb WHERE c3 = 1 AND c7 = 1 ORDER BY c7 DESC;
c7 | c0 | c3
+--+
1 | | 1(1 rows) {code}
3. Drain and stop 2.2.19.
{code:java}
bin/nodetool -h :::127.0.0.1 drain
bin/nodetool -h :
[jira] [Updated] (CASSANDRA-19624) ModificationStatement#casInternal leaks RowIterator
[ https://issues.apache.org/jira/browse/CASSANDRA-19624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19624: - Bug Category: Parent values: Degradation(12984)Level 1 values: Resource Management(12995) Complexity: Normal Component/s: Legacy/Core Discovered By: User Report Fix Version/s: 4.0.x 4.1.x 5.0.x 5.x Severity: Normal Status: Open (was: Triage Needed) > ModificationStatement#casInternal leaks RowIterator > --- > > Key: CASSANDRA-19624 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19624 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core >Reporter: Michael Marshall >Assignee: Michael Marshall >Priority: Normal > Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x > > Time Spent: 40m > Remaining Estimate: 0h > > In the `ModificationStatement` class, the `casInternal` method opens a row > iterator without closing it, causing the iterator to leak resources. Here is > a link to the relevant code in the `trunk` branch. > [https://github.com/apache/cassandra/blob/a77a2d10b1d247ed920b75df79f982a3b7c6a431/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java#L680-L684] > > It seems that `cassandra-3.0.30` has the bug, but `cassandra-2.2.19` does not > have it. Is it correct to target `cassandra-3.0.30`? > What is the best practice for testing this kind of bug fix? It seems like a > low complexity fix. This is my first contribution to the Cassandra community, > so any guidance is appreciated. Thanks! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19158) Reuse native transport-driven futures in Debounce
[ https://issues.apache.org/jira/browse/CASSANDRA-19158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-19158: Status: Changes Suggested (was: Review In Progress) Thanks, this is definitely an improvement. I've left a few comments on the PR > Reuse native transport-driven futures in Debounce > - > > Key: CASSANDRA-19158 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19158 > Project: Cassandra > Issue Type: Improvement > Components: Transactional Cluster Metadata >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > Attachments: ci_summary.html > > Time Spent: 20m > Remaining Estimate: 0h > > Currently, we create a future in Debounce, then create one more future in > RemoteProcessor#sendWithCallback. This is further exacerbated by chaining > calls, when we first attempt to catch up from peer, and then from CMS. > First of all, we should always only use a native transport timeout driven > futures returned from sendWithCallback, since they implement reasonable > retries under the hood, and are easy to bulk-configure (ie you can simply > change timeout in yaml and have all futures change their behaviour). > Second, we should _chain_ futures and use map or andThen for fallback > operations such as trying to catch up from CMS after an unsuccesful attemp to > catch up from peer. > This should significantly simplify the code and number of blocked/waiting > threads. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19621) Test failure: gossip_test.TestGossip::test_2dc_parallel_startup
[
https://issues.apache.org/jira/browse/CASSANDRA-19621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19621:
-
Test and Documentation Plan: run CI
Status: Patch Available (was: Open)
> Test failure: gossip_test.TestGossip::test_2dc_parallel_startup
> ---
>
> Key: CASSANDRA-19621
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19621
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0.x
>
>
> As seen at
> https://app.circleci.com/pipelines/github/driftx/cassandra/1625/workflows/d3849d2a-987e-49f8-9f4b-916c7ce85279/jobs/88756/tests:
> {quote}
> failed on teardown with "Unexpected error found in node logs (see stdout for
> full details). Errors: [[node4] 'ERROR [Messaging-EventLoop-3-3] 2024-05-07
> 20:10:54,261 NoSpamLogger.java:110 -
> /127.0.0.4:7004->/127.0.0.1:7001-URGENT_MESSAGES-[no-channel] failed to
> connect\njava.nio.channels.ClosedChannelException: null\n\tat
> org.apache.cassandra.net.OutboundConnectionInitiator$Handler.channelInactive(OutboundConnectionInitiator.java:322)\n\tat
>
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:305)\n\tat
>
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)\n\tat
>
> io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:274)\n\tat
>
> io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:411)\n\tat
>
> io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:376)\n\tat
>
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:305)\n\tat
>
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)\n\tat
>
> io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:274)\n\tat
>
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405)\n\tat
>
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:301)\n\tat
>
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)\n\tat
>
> io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901)\n\tat
>
> io.netty.channel.AbstractChannel$AbstractUnsafe$7.run(AbstractChannel.java:813)\n\tat
>
> io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)\n\tat
>
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)\n\tat
>
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)\n\tat
> io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:413)\n\tat
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)\n\tat
>
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)\n\tat
>
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat
> java.base/java.lang.Thread.run(Thread.java:833)']"
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19621) Test failure: gossip_test.TestGossip::test_2dc_parallel_startup
[
https://issues.apache.org/jira/browse/CASSANDRA-19621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844700#comment-17844700
]
Brandon Williams commented on CASSANDRA-19621:
--
Patch to ignore this
[here|https://github.com/driftx/cassandra-dtest/tree/CASSANDRA-19621]. I
haven't been able to reproduce but this should do the trick and
[here|https://app.circleci.com/pipelines/github/driftx/cassandra/1627/workflows/270dafe1-230c-4650-977b-a6c0bbaa370f/jobs/89041]
is multiplexed CI.
> Test failure: gossip_test.TestGossip::test_2dc_parallel_startup
> ---
>
> Key: CASSANDRA-19621
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19621
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0.x
>
>
> As seen at
> https://app.circleci.com/pipelines/github/driftx/cassandra/1625/workflows/d3849d2a-987e-49f8-9f4b-916c7ce85279/jobs/88756/tests:
> {quote}
> failed on teardown with "Unexpected error found in node logs (see stdout for
> full details). Errors: [[node4] 'ERROR [Messaging-EventLoop-3-3] 2024-05-07
> 20:10:54,261 NoSpamLogger.java:110 -
> /127.0.0.4:7004->/127.0.0.1:7001-URGENT_MESSAGES-[no-channel] failed to
> connect\njava.nio.channels.ClosedChannelException: null\n\tat
> org.apache.cassandra.net.OutboundConnectionInitiator$Handler.channelInactive(OutboundConnectionInitiator.java:322)\n\tat
>
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:305)\n\tat
>
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)\n\tat
>
> io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:274)\n\tat
>
> io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:411)\n\tat
>
> io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:376)\n\tat
>
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:305)\n\tat
>
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)\n\tat
>
> io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:274)\n\tat
>
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405)\n\tat
>
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:301)\n\tat
>
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)\n\tat
>
> io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901)\n\tat
>
> io.netty.channel.AbstractChannel$AbstractUnsafe$7.run(AbstractChannel.java:813)\n\tat
>
> io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)\n\tat
>
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)\n\tat
>
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)\n\tat
> io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:413)\n\tat
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)\n\tat
>
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)\n\tat
>
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat
> java.base/java.lang.Thread.run(Thread.java:833)']"
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15439) Token metadata for bootstrapping nodes is lost under temporary failures
[
https://issues.apache.org/jira/browse/CASSANDRA-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-15439:
-
Fix Version/s: 4.0.14
4.1.6
5.0-beta2
5.0
(was: 4.0.x)
(was: 4.1.x)
(was: 5.0.x)
Since Version: NA
Source Control Link:
https://github.com/apache/cassandra/commit/057d082e00f7d10b8e9b127cfabd9b8cd228da3d
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed, thanks all!
> Token metadata for bootstrapping nodes is lost under temporary failures
> ---
>
> Key: CASSANDRA-15439
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15439
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Membership
>Reporter: Josh Snyder
>Assignee: Raymond Huffman
>Priority: Normal
> Fix For: 4.0.14, 4.1.6, 5.0-beta2, 5.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> In CASSANDRA-8838, [~pauloricardomg] asked "hints will not be stored to the
> bootstrapping node after RING_DELAY, since it will evicted from the TMD
> pending ranges. Should we create a ticket to address this?"
> CASSANDRA-15264 relates to the most likely cause of such situations, where
> the Cassandra daemon on the bootstrapping node completely crashes. Based on
> testing with {{kill -STOP}} on a bootstrapping Cassandra JVM, I believe it
> also is possible to remove token metadata (and thus pending ranges, and thus
> hints) for a bootstrapping node, simply by affecting its status in the
> failure detector.
> A node in the cluster sees the bootstrapping node this way:
> {noformat}
> INFO [GossipStage:1] 2019-11-27 20:41:41,101 Gossiper.java: - Node
> /PUBLIC-IP is now part of the cluster
> INFO [GossipStage:1] 2019-11-27 20:41:41,199 Gossiper.java:1073 -
> InetAddress /PUBLIC-IP is now UP
> INFO [HANDSHAKE-/PRIVATE-IP] 2019-11-27 20:41:41,412
> OutboundTcpConnection.java:565 - Handshaking version with /PRIVATE-IP
> INFO [STREAM-INIT-/PRIVATE-IP:21233] 2019-11-27 20:42:10,019
> StreamResultFuture.java:112 - [Stream #6219a950-1156-11ea-b45d-4d30364576c4
> ID#0] Creating new streaming plan for Bootstrap
> INFO [STREAM-INIT-/PRIVATE-IP:21233] 2019-11-27 20:42:10,020
> StreamResultFuture.java:119 - [Stream #6219a950-1156-11ea-b45d-4d30364576c4,
> ID#0] Received streaming plan for Bootstrap
> INFO [STREAM-INIT-/PRIVATE-IP:56003] 2019-11-27 20:42:10,112
> StreamResultFuture.java:119 - [Stream #6219a950-1156-11ea-b45d-4d30364576c4,
> ID#0] Received streaming plan for Bootstrap
> INFO [STREAM-IN-/PUBLIC-IP] 2019-11-27 20:42:10,179
> StreamResultFuture.java:169 - [Stream #6219a950-1156-11ea-b45d-4d30364576c4
> ID#0] Prepare completed. Receiving 0 files(0 bytes), sending 833
> files(139744616815 bytes)
> INFO [GossipStage:1] 2019-11-27 20:54:47,547 Gossiper.java:1089 -
> InetAddress /PUBLIC-IP is now DOWN
> INFO [GossipTasks:1] 2019-11-27 20:54:57,551 Gossiper.java:849 - FatClient
> /PUBLIC-IP has been silent for 3ms, removing from gossip
> {noformat}
> Since the bootstrapping node has no tokens, it is treated like a fat client,
> and it is removed from the ring. For correctness purposes, I believe we must
> keep storing hints for the downed bootstrapping node until it is either
> assassinated or until a replacement attempts to bootstrap for the same token.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 01/01: Merge branch 'cassandra-4.0' into cassandra-4.1
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch cassandra-4.1
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit c6017aca778df72e1944b1855c902027b8abad7f
Merge: 103e9cca1a 057d082e00
Author: Brandon Williams
AuthorDate: Wed May 8 10:22:07 2024 -0500
Merge branch 'cassandra-4.0' into cassandra-4.1
CHANGES.txt| 1 +
conf/jvm-server.options| 4 ++
.../config/CassandraRelevantProperties.java| 2 +
src/java/org/apache/cassandra/gms/Gossiper.java| 46 --
.../org/apache/cassandra/gms/VersionedValue.java | 2 +
5 files changed, 52 insertions(+), 3 deletions(-)
diff --cc CHANGES.txt
index 0095e2d078,2d56a56c61..2f31f4a2e2
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,5 -1,5 +1,6 @@@
-4.0.14
+4.1.6
+Merged from 4.0:
+ * Add timeout specifically for bootstrapping nodes (CASSANDRA-15439)
* Bring Redhat packge dirs/ownership/perms in line with Debian package
(CASSANDRA-19565)
diff --cc src/java/org/apache/cassandra/gms/Gossiper.java
index 9be48f96bb,009e6b255f..ff1264d6fa
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@@ -272,9 -258,28 +276,28 @@@ public class Gossiper implements IFailu
return 259200 * 1000; // 3 days
}
+ private static long getFailedBootstrapTimeout()
+ {
+ String newtimeout =
CassandraRelevantProperties.FAILED_BOOTSTRAP_TIMEOUT.getString();
+ if (newtimeout != null)
+ {
+ long longValue = Long.parseLong(newtimeout);
+ if (longValue == -1)
+ {
+ longValue = Long.MAX_VALUE;
+ }
+ logger.info("Overriding FAILED_BOOTSTRAP_TIMEOUT to {}ms",
longValue);
+ return longValue;
+ }
+ else
+ {
+ return FAT_CLIENT_TIMEOUT * 2;
+ }
+ }
+
private static boolean isInGossipStage()
{
-return ((JMXEnabledSingleThreadExecutor)
Stage.GOSSIP.executor()).isExecutedBy(Thread.currentThread());
+return Stage.GOSSIP.executor().inExecutor();
}
private static void checkProperThreadForStateMutation()
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-5.0 updated (049160e70a -> 3b8c48024e)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch cassandra-5.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 049160e70a Avoid reading of the same IndexInfo from disk many times for a large partition when ShallowIndexedEntry is used new 057d082e00 Add option to override the FatClient timeout for Bootstrapping nodes new c6017aca77 Merge branch 'cassandra-4.0' into cassandra-4.1 new 3b8c48024e Merge branch 'cassandra-4.1' into cassandra-5.0 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + conf/jvm-server.options| 4 ++ .../config/CassandraRelevantProperties.java| 2 + src/java/org/apache/cassandra/gms/Gossiper.java| 46 -- .../org/apache/cassandra/gms/VersionedValue.java | 2 +- 5 files changed, 51 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 01/01: Merge branch 'cassandra-4.1' into cassandra-5.0
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch cassandra-5.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 3b8c48024ed07a30bb8ccd338f0eee0f83eb89dc
Merge: 049160e70a c6017aca77
Author: Brandon Williams
AuthorDate: Wed May 8 10:28:21 2024 -0500
Merge branch 'cassandra-4.1' into cassandra-5.0
CHANGES.txt| 1 +
conf/jvm-server.options| 4 ++
.../config/CassandraRelevantProperties.java| 2 +
src/java/org/apache/cassandra/gms/Gossiper.java| 46 --
.../org/apache/cassandra/gms/VersionedValue.java | 2 +-
5 files changed, 51 insertions(+), 4 deletions(-)
diff --cc CHANGES.txt
index 8af3d01ac9,2f31f4a2e2..370971fcea
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -40,10 -10,7 +40,11 @@@ Merged from 4.1
* Fix hints delivery for a node going down repeatedly (CASSANDRA-19495)
* Do not go to disk for reading hints file sizes (CASSANDRA-19477)
* Fix system_views.settings to handle array types (CASSANDRA-19475)
+ * Memoize Cassandra verion and add a backoff interval for failed schema
pulls (CASSANDRA-18902)
+ * Fix StackOverflowError on ALTER after many previous schema changes
(CASSANDRA-19166)
Merged from 4.0:
++ * Add timeout specifically for bootstrapping nodes (CASSANDRA-15439)
+ * Bring Redhat packge dirs/ownership/perms in line with Debian package
(CASSANDRA-19565)
* Make nodetool import congruent with the documentation by not relying on
the folder structure of the imported SSTable files (CASSANDRA-19401)
* IR may leak SSTables with pending repair when coming from streaming
(CASSANDRA-19182)
* Streaming exception race creates corrupt transaction log files that
prevent restart (CASSANDRA-18736)
diff --cc conf/jvm-server.options
index c50f3631f5,d529a2b9e2..c9a9336683
--- a/conf/jvm-server.options
+++ b/conf/jvm-server.options
@@@ -71,6 -74,10 +71,10 @@@
# before joining the ring.
#-Dcassandra.ring_delay_ms=ms
+ # Allows overriding the timeout after which an unresponsive bootstrapping
node is considered failed
-# and is removed from gossip state and bootstrapTokens. (Default:
cassandra.ring_delay * 2)
++# and is removed from gossip state and bootstrapTokens. (Default:
cassandra.ring_delay * 10)
+ #-Dcassandra.failed_bootstrap_timeout_ms=ms
+
# Set the SSL port for encrypted communication. (Default: 7001)
#-Dcassandra.ssl_storage_port=port
diff --cc src/java/org/apache/cassandra/config/CassandraRelevantProperties.java
index e0dfeb19ff,4ee11ec591..336f68838f
--- a/src/java/org/apache/cassandra/config/CassandraRelevantProperties.java
+++ b/src/java/org/apache/cassandra/config/CassandraRelevantProperties.java
@@@ -145,191 -107,92 +145,193 @@@ public enum CassandraRelevantPropertie
* if there is no port specified, if the port does not have the correct
numeric format,
* or if the specified name is empty or null.
*/
-COM_SUN_MANAGEMENT_JMXREMOTE_RMI_PORT
("com.sun.management.jmxremote.rmi.port", "0"),
-
-/** Cassandra jmx remote and local port */
-CASSANDRA_JMX_REMOTE_PORT("cassandra.jmx.remote.port"),
-CASSANDRA_JMX_LOCAL_PORT("cassandra.jmx.local.port"),
-
+
COM_SUN_MANAGEMENT_JMXREMOTE_RMI_PORT("com.sun.management.jmxremote.rmi.port",
"0"),
/** This property indicates whether SSL is enabled for monitoring
remotely. Default is set to false. */
-COM_SUN_MANAGEMENT_JMXREMOTE_SSL ("com.sun.management.jmxremote.ssl"),
-
+COM_SUN_MANAGEMENT_JMXREMOTE_SSL("com.sun.management.jmxremote.ssl"),
/**
- * This property indicates whether SSL client authentication is enabled -
com.sun.management.jmxremote.ssl.need.client.auth.
- * Default is set to false.
- */
-COM_SUN_MANAGEMENT_JMXREMOTE_SSL_NEED_CLIENT_AUTH
("com.sun.management.jmxremote.ssl.need.client.auth"),
-
-/**
- * This property indicates the location for the access file. If
com.sun.management.jmxremote.authenticate is false,
- * then this property and the password and access files, are ignored.
Otherwise, the access file must exist and
- * be in the valid format. If the access file is empty or nonexistent,
then no access is allowed.
+ * A comma-delimited list of SSL/TLS cipher suites to enable.
+ * Used in conjunction with com.sun.management.jmxremote.ssl -
com.sun.management.jmxremote.ssl.enabled.cipher.suites
*/
-COM_SUN_MANAGEMENT_JMXREMOTE_ACCESS_FILE
("com.sun.management.jmxremote.access.file"),
-
-/** This property indicates the path to the password file -
com.sun.management.jmxremote.password.file */
-COM_SUN_MANAGEMENT_JMXREMOTE_PASSWORD_FILE
("com.sun.management.jmxremote.password.file"),
-
-/** Port number to enable JMX RMI connections -
com.sun.management.jmxremote.port */
-COM_SUN_MANAGEMENT_JMXREMOTE_PORT ("com.sun.management.jmxremote.port"),
+
CO
(cassandra) branch cassandra-4.1 updated (103e9cca1a -> c6017aca77)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 103e9cca1a Merge branch 'cassandra-4.0' into cassandra-4.1 new 057d082e00 Add option to override the FatClient timeout for Bootstrapping nodes new c6017aca77 Merge branch 'cassandra-4.0' into cassandra-4.1 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + conf/jvm-server.options| 4 ++ .../config/CassandraRelevantProperties.java| 2 + src/java/org/apache/cassandra/gms/Gossiper.java| 46 -- .../org/apache/cassandra/gms/VersionedValue.java | 2 + 5 files changed, 52 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch trunk updated (534da538d2 -> 7f4a0f0c00)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from 534da538d2 Refactor Relation and Restrictions hierachies new 057d082e00 Add option to override the FatClient timeout for Bootstrapping nodes new c6017aca77 Merge branch 'cassandra-4.0' into cassandra-4.1 new 3b8c48024e Merge branch 'cassandra-4.1' into cassandra-5.0 new 7f4a0f0c00 Merge branch 'cassandra-5.0' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 01/01: Merge branch 'cassandra-5.0' into trunk
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 7f4a0f0c00e193018a590a8fc158ded51b411027 Merge: 534da538d2 3b8c48024e Author: Brandon Williams AuthorDate: Wed May 8 10:29:02 2024 -0500 Merge branch 'cassandra-5.0' into trunk - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-4.0 updated: Add option to override the FatClient timeout for Bootstrapping nodes
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch cassandra-4.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-4.0 by this push:
new 057d082e00 Add option to override the FatClient timeout for
Bootstrapping nodes
057d082e00 is described below
commit 057d082e00f7d10b8e9b127cfabd9b8cd228da3d
Author: Raymond Huffman
AuthorDate: Tue Apr 23 16:41:12 2024 -0400
Add option to override the FatClient timeout for Bootstrapping nodes
Patch by Raymond Huffman; reviewed by brandonwilliams and dcapwell for
CASSANDRA-15439
---
CHANGES.txt| 1 +
conf/jvm-server.options| 4 ++
.../config/CassandraRelevantProperties.java| 4 ++
src/java/org/apache/cassandra/gms/Gossiper.java| 46 --
.../org/apache/cassandra/gms/VersionedValue.java | 2 +
5 files changed, 54 insertions(+), 3 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index a506c9e9e0..2d56a56c61 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0.14
+ * Add timeout specifically for bootstrapping nodes (CASSANDRA-15439)
* Bring Redhat packge dirs/ownership/perms in line with Debian package
(CASSANDRA-19565)
diff --git a/conf/jvm-server.options b/conf/jvm-server.options
index e89cf7343a..d529a2b9e2 100644
--- a/conf/jvm-server.options
+++ b/conf/jvm-server.options
@@ -74,6 +74,10 @@
# before joining the ring.
#-Dcassandra.ring_delay_ms=ms
+# Allows overriding the timeout after which an unresponsive bootstrapping node
is considered failed
+# and is removed from gossip state and bootstrapTokens. (Default:
cassandra.ring_delay * 2)
+#-Dcassandra.failed_bootstrap_timeout_ms=ms
+
# Set the SSL port for encrypted communication. (Default: 7001)
#-Dcassandra.ssl_storage_port=port
diff --git
a/src/java/org/apache/cassandra/config/CassandraRelevantProperties.java
b/src/java/org/apache/cassandra/config/CassandraRelevantProperties.java
index 4de333e101..0377bc40e2 100644
--- a/src/java/org/apache/cassandra/config/CassandraRelevantProperties.java
+++ b/src/java/org/apache/cassandra/config/CassandraRelevantProperties.java
@@ -151,6 +151,10 @@ public enum CassandraRelevantProperties
/** mx4jport */
MX4JPORT ("mx4jport"),
+RING_DELAY("cassandra.ring_delay_ms"),
+
+FAILED_BOOTSTRAP_TIMEOUT("cassandra.failed_bootstrap_timeout_ms"),
+
/**
* When bootstraping we wait for all schema versions found in gossip to be
seen, and if not seen in time we fail
* the bootstrap; this property will avoid failing and allow bootstrap to
continue if set to true.
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java
b/src/java/org/apache/cassandra/gms/Gossiper.java
index 63ff5150a7..009e6b255f 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -107,6 +107,7 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
SILENT_SHUTDOWN_STATES.add(VersionedValue.STATUS_BOOTSTRAPPING);
SILENT_SHUTDOWN_STATES.add(VersionedValue.STATUS_BOOTSTRAPPING_REPLACE);
}
+
private static final List ADMINISTRATIVELY_INACTIVE_STATES =
Arrays.asList(VersionedValue.HIBERNATE,
VersionedValue.REMOVED_TOKEN,
VersionedValue.STATUS_LEFT);
@@ -126,7 +127,10 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
// Maximimum difference between generation value and local time we are
willing to accept about a peer
static final int MAX_GENERATION_DIFFERENCE = 86400 * 365;
-private final long fatClientTimeout;
+
+// half of QUARATINE_DELAY, to ensure justRemovedEndpoints has enough
leeway to prevent re-gossip
+private static final long FAT_CLIENT_TIMEOUT = (QUARANTINE_DELAY / 2);
+private static final long FAILED_BOOTSTRAP_TIMEOUT =
getFailedBootstrapTimeout();
private final Random random = new Random();
/* subscribers for interest in EndpointState change */
@@ -254,6 +258,25 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
return 259200 * 1000; // 3 days
}
+private static long getFailedBootstrapTimeout()
+{
+String newtimeout =
CassandraRelevantProperties.FAILED_BOOTSTRAP_TIMEOUT.getString();
+if (newtimeout != null)
+{
+long longValue = Long.parseLong(newtimeout);
+if (longValue == -1)
+{
+longValue = Long.MAX_VALUE;
+}
+logger.info("Overriding FAILED_BOOTSTRAP_TIMEOUT to {}ms",
longValue);
+return longValue;
+}
+else
+{
+ret
(cassandra-website) branch asf-staging updated (51600acf3 -> ea32cb4df)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git omit 51600acf3 generate docs for 96fc3baf new ea32cb4df generate docs for 96fc3baf This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (51600acf3) \ N -- N -- N refs/heads/asf-staging (ea32cb4df) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: site-ui/build/ui-bundle.zip | Bin 4883646 -> 4883646 bytes 1 file changed, 0 insertions(+), 0 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19624) ModificationStatement#casInternal leaks RowIterator
[ https://issues.apache.org/jira/browse/CASSANDRA-19624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844683#comment-17844683 ] Michael Marshall commented on CASSANDRA-19624: -- The bug is as early as 3.0, but I am targeting 4.0. Please let me know if we should change that. > ModificationStatement#casInternal leaks RowIterator > --- > > Key: CASSANDRA-19624 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19624 > Project: Cassandra > Issue Type: Bug >Reporter: Michael Marshall >Assignee: Michael Marshall >Priority: Normal > Time Spent: 40m > Remaining Estimate: 0h > > In the `ModificationStatement` class, the `casInternal` method opens a row > iterator without closing it, causing the iterator to leak resources. Here is > a link to the relevant code in the `trunk` branch. > [https://github.com/apache/cassandra/blob/a77a2d10b1d247ed920b75df79f982a3b7c6a431/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java#L680-L684] > > It seems that `cassandra-3.0.30` has the bug, but `cassandra-2.2.19` does not > have it. Is it correct to target `cassandra-3.0.30`? > What is the best practice for testing this kind of bug fix? It seems like a > low complexity fix. This is my first contribution to the Cassandra community, > so any guidance is appreciated. Thanks! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19624) ModificationStatement#casInternal leaks RowIterator
[ https://issues.apache.org/jira/browse/CASSANDRA-19624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Lewandowski updated CASSANDRA-19624: -- Reviewers: Jacek Lewandowski > ModificationStatement#casInternal leaks RowIterator > --- > > Key: CASSANDRA-19624 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19624 > Project: Cassandra > Issue Type: Bug >Reporter: Michael Marshall >Assignee: Michael Marshall >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > > In the `ModificationStatement` class, the `casInternal` method opens a row > iterator without closing it, causing the iterator to leak resources. Here is > a link to the relevant code in the `trunk` branch. > [https://github.com/apache/cassandra/blob/a77a2d10b1d247ed920b75df79f982a3b7c6a431/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java#L680-L684] > > It seems that `cassandra-3.0.30` has the bug, but `cassandra-2.2.19` does not > have it. Is it correct to target `cassandra-3.0.30`? > What is the best practice for testing this kind of bug fix? It seems like a > low complexity fix. This is my first contribution to the Cassandra community, > so any guidance is appreciated. Thanks! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19624) ModificationStatement#casInternal leaks RowIterator
Michael Marshall created CASSANDRA-19624: Summary: ModificationStatement#casInternal leaks RowIterator Key: CASSANDRA-19624 URL: https://issues.apache.org/jira/browse/CASSANDRA-19624 Project: Cassandra Issue Type: Bug Reporter: Michael Marshall Assignee: Michael Marshall In the `ModificationStatement` class, the `casInternal` method opens a row iterator without closing it, causing the iterator to leak resources. Here is a link to the relevant code in the `trunk` branch. [https://github.com/apache/cassandra/blob/a77a2d10b1d247ed920b75df79f982a3b7c6a431/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java#L680-L684] It seems that `cassandra-3.0.30` has the bug, but `cassandra-2.2.19` does not have it. Is it correct to target `cassandra-3.0.30`? What is the best practice for testing this kind of bug fix? It seems like a low complexity fix. This is my first contribution to the Cassandra community, so any guidance is appreciated. Thanks! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-website) branch asf-site updated (690a3a5d6 -> 51600acf3)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard 690a3a5d6 generate docs for cc1c7113 add 3b8cf04d9 fix: remove the paragraph tag from "ecosystem.adoc" add 96fc3bafa Add the link to the KEYS file on the downloads page, per ASF policy add 51600acf3 generate docs for 96fc3baf This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (690a3a5d6) \ N -- N -- N refs/heads/asf-site (51600acf3) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: content/_/download.html| 1 + content/_/ecosystem.html | 2 +- .../managing/tools/nodetool/bootstrap.html | 6 +- .../5.1/cassandra/managing/tools/nodetool/cms.html | 59 +++- .../managing/tools/nodetool/nodetool.html | 8 +-- .../managing/tools/nodetool/repair_admin.html | 62 ++--- .../managing/tools/nodetool/bootstrap.html | 6 +- .../cassandra/managing/tools/nodetool/cms.html | 59 +++- .../managing/tools/nodetool/nodetool.html | 8 +-- .../managing/tools/nodetool/repair_admin.html | 62 ++--- content/search-index.js| 2 +- .../source/modules/ROOT/pages/download.adoc| 2 + .../source/modules/ROOT/pages/ecosystem.adoc | 2 +- site-ui/build/ui-bundle.zip| Bin 4883646 -> 4883646 bytes 14 files changed, 146 insertions(+), 133 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19158) Reuse native transport-driven futures in Debounce
[ https://issues.apache.org/jira/browse/CASSANDRA-19158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-19158: Status: Review In Progress (was: Patch Available) > Reuse native transport-driven futures in Debounce > - > > Key: CASSANDRA-19158 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19158 > Project: Cassandra > Issue Type: Improvement > Components: Transactional Cluster Metadata >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > Attachments: ci_summary.html > > Time Spent: 10m > Remaining Estimate: 0h > > Currently, we create a future in Debounce, then create one more future in > RemoteProcessor#sendWithCallback. This is further exacerbated by chaining > calls, when we first attempt to catch up from peer, and then from CMS. > First of all, we should always only use a native transport timeout driven > futures returned from sendWithCallback, since they implement reasonable > retries under the hood, and are easy to bulk-configure (ie you can simply > change timeout in yaml and have all futures change their behaviour). > Second, we should _chain_ futures and use map or andThen for fallback > operations such as trying to catch up from CMS after an unsuccesful attemp to > catch up from peer. > This should significantly simplify the code and number of blocked/waiting > threads. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19158) Reuse native transport-driven futures in Debounce
[ https://issues.apache.org/jira/browse/CASSANDRA-19158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-19158: Test and Documentation Plan: Refactoring, CI tests Status: Patch Available (was: Open) > Reuse native transport-driven futures in Debounce > - > > Key: CASSANDRA-19158 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19158 > Project: Cassandra > Issue Type: Improvement > Components: Transactional Cluster Metadata >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > Attachments: ci_summary.html > > Time Spent: 10m > Remaining Estimate: 0h > > Currently, we create a future in Debounce, then create one more future in > RemoteProcessor#sendWithCallback. This is further exacerbated by chaining > calls, when we first attempt to catch up from peer, and then from CMS. > First of all, we should always only use a native transport timeout driven > futures returned from sendWithCallback, since they implement reasonable > retries under the hood, and are easy to bulk-configure (ie you can simply > change timeout in yaml and have all futures change their behaviour). > Second, we should _chain_ futures and use map or andThen for fallback > operations such as trying to catch up from CMS after an unsuccesful attemp to > catch up from peer. > This should significantly simplify the code and number of blocked/waiting > threads. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-website) branch asf-staging updated (fe1482063 -> 51600acf3)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard fe1482063 generate docs for 3b8cf04d add 96fc3bafa Add the link to the KEYS file on the downloads page, per ASF policy new 51600acf3 generate docs for 96fc3baf This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (fe1482063) \ N -- N -- N refs/heads/asf-staging (51600acf3) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: content/_/download.html| 1 + content/search-index.js| 2 +- .../source/modules/ROOT/pages/download.adoc| 2 ++ site-ui/build/ui-bundle.zip| Bin 4883646 -> 4883646 bytes 4 files changed, 4 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19158) Reuse native transport-driven futures in Debounce
[ https://issues.apache.org/jira/browse/CASSANDRA-19158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-19158: Change Category: Code Clarity Complexity: Normal Component/s: Transactional Cluster Metadata Reviewers: Sam Tunnicliffe Status: Open (was: Triage Needed) > Reuse native transport-driven futures in Debounce > - > > Key: CASSANDRA-19158 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19158 > Project: Cassandra > Issue Type: Improvement > Components: Transactional Cluster Metadata >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > Attachments: ci_summary.html > > Time Spent: 10m > Remaining Estimate: 0h > > Currently, we create a future in Debounce, then create one more future in > RemoteProcessor#sendWithCallback. This is further exacerbated by chaining > calls, when we first attempt to catch up from peer, and then from CMS. > First of all, we should always only use a native transport timeout driven > futures returned from sendWithCallback, since they implement reasonable > retries under the hood, and are easy to bulk-configure (ie you can simply > change timeout in yaml and have all futures change their behaviour). > Second, we should _chain_ futures and use map or andThen for fallback > operations such as trying to catch up from CMS after an unsuccesful attemp to > catch up from peer. > This should significantly simplify the code and number of blocked/waiting > threads. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] JAVA-3118: Add support for vector data type in Schema Builder, QueryBuilder [cassandra-java-driver]
michaelsembwever commented on PR #1931: URL: https://github.com/apache/cassandra-java-driver/pull/1931#issuecomment-2100603793 what's the CASSANDRA ticket for this ? i will test this downstream here: - https://github.com/spring-projects/spring-ai/blob/main/vector-stores/spring-ai-cassandra/src/main/java/org/springframework/ai/vectorstore/CassandraVectorStoreConfig.java#L551-L568 - https://github.com/spring-projects/spring-ai/blob/main/vector-stores/spring-ai-cassandra/src/main/java/org/springframework/ai/vectorstore/CassandraVectorStoreConfig.java#L502-L512 - https://github.com/spring-projects/spring-ai/blob/main/vector-stores/spring-ai-cassandra/src/main/java/org/springframework/ai/vectorstore/CassandraVectorStore.java#L342-L345 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-website) branch trunk updated: Add the link to the KEYS file on the downloads page, per ASF policy
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-website.git The following commit(s) were added to refs/heads/trunk by this push: new 96fc3bafa Add the link to the KEYS file on the downloads page, per ASF policy 96fc3bafa is described below commit 96fc3bafa2ee23a248e2904859a1a9554f0ffb01 Author: mck AuthorDate: Wed May 8 15:26:23 2024 +0200 Add the link to the KEYS file on the downloads page, per ASF policy https://infra.apache.org/release-download-pages.html --- site-content/source/modules/ROOT/pages/download.adoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/site-content/source/modules/ROOT/pages/download.adoc b/site-content/source/modules/ROOT/pages/download.adoc index 613d3f27e..79385f7c3 100644 --- a/site-content/source/modules/ROOT/pages/download.adoc +++ b/site-content/source/modules/ROOT/pages/download.adoc @@ -120,6 +120,8 @@ https://www.apache.org/dyn/closer.lua/cassandra/3.0.30/apache-cassandra-3.0.30-b Unmaintained older versions of Cassandra are archived https://archive.apache.org/dist/cassandra/[here,window=_blank]. [discrete] CVE fixes may be applied to unmaintained versions as decided on a case-by-case basis. +[discrete] + The PGP keys used to sign releases are available in this https://downloads.apache.org/cassandra/KEYS[KEYS,window=_blank] file. -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19621) Test failure: gossip_test.TestGossip::test_2dc_parallel_startup
[
https://issues.apache.org/jira/browse/CASSANDRA-19621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19621:
-
Description:
As seen at
https://app.circleci.com/pipelines/github/driftx/cassandra/1625/workflows/d3849d2a-987e-49f8-9f4b-916c7ce85279/jobs/88756/tests:
{quote}
failed on teardown with "Unexpected error found in node logs (see stdout for
full details). Errors: [[node4] 'ERROR [Messaging-EventLoop-3-3] 2024-05-07
20:10:54,261 NoSpamLogger.java:110 -
/127.0.0.4:7004->/127.0.0.1:7001-URGENT_MESSAGES-[no-channel] failed to
connect\njava.nio.channels.ClosedChannelException: null\n\tat
org.apache.cassandra.net.OutboundConnectionInitiator$Handler.channelInactive(OutboundConnectionInitiator.java:322)\n\tat
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:305)\n\tat
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)\n\tat
io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:274)\n\tat
io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:411)\n\tat
io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:376)\n\tat
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:305)\n\tat
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)\n\tat
io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:274)\n\tat
io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405)\n\tat
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:301)\n\tat
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)\n\tat
io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901)\n\tat
io.netty.channel.AbstractChannel$AbstractUnsafe$7.run(AbstractChannel.java:813)\n\tat
io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)\n\tat
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)\n\tat
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)\n\tat
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:413)\n\tat
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)\n\tat
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)\n\tat
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat
java.base/java.lang.Thread.run(Thread.java:833)']"
{quote}
was:
As seen at
https://app.circleci.com/pipelines/github/driftx/cassandra/1625/workflows/d3849d2a-987e-49f8-9f4b-916c7ce85279/jobs/88756/tests:
{noformat}
failed on teardown with "Unexpected error found in node logs (see stdout for
full details). Errors: [[node4] 'ERROR [Messaging-EventLoop-3-3] 2024-05-07
20:10:54,261 NoSpamLogger.java:110 -
/127.0.0.4:7004->/127.0.0.1:7001-URGENT_MESSAGES-[no-channel] failed to
connect\njava.nio.channels.ClosedChannelException: null\n\tat
org.apache.cassandra.net.OutboundConnectionInitiator$Handler.channelInactive(OutboundConnectionInitiator.java:322)\n\tat
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:305)\n\tat
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)\n\tat
io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:274)\n\tat
io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:411)\n\tat
io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:376)\n\tat
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:305)\n\tat
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)\n\tat
io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:274)\n\tat
io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405)\n\tat
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:301)\n\tat
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:281)\n\tat
io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901)\n\tat
io.netty.channel.AbstractChannel$AbstractUnsafe$7.run(AbstractCh
[jira] [Updated] (CASSANDRA-19623) Read fail with "illegal RT bounds sequence" after migrating data from 2.2.19 to 3.0.30
[
https://issues.apache.org/jira/browse/CASSANDRA-19623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19623:
-
Bug Category: Parent values: Availability(12983)Level 1 values:
Unavailable(12994)
Complexity: Normal
Discovered By: User Report
Fix Version/s: 3.0.x
Severity: Normal
Status: Open (was: Triage Needed)
> Read fail with "illegal RT bounds sequence" after migrating data from 2.2.19
> to 3.0.30
> --
>
> Key: CASSANDRA-19623
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19623
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Klay
>Priority: Normal
> Fix For: 3.0.x
>
> Attachments: cassandra.yaml, data.tar.gz, system.log
>
>
> After migrating data from 2.2.19 to 3.0.30, reading the legacy data in 3.0.30
> would fail with the following exception.
> {code:java}
> INFO [main] 2024-05-08 00:42:42,397 CassandraDaemon.java:653 - Startup
> complete
> ERROR [SharedPool-Worker-2] 2024-05-08 00:42:49,975
> AbstractLocalAwareExecutorService.java:166 - Uncaught exception on thread
> Thread[SharedPool-Worker-2,10,main]
> java.lang.RuntimeException: java.lang.IllegalStateException: SSTABLE
> UnfilteredRowIterator for ks.tb has an illegal RT bounds sequence: unexpected
> end bound or boundary Marker INCL_START_BOUND(1)@1715128933051779/1715128933
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2656)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109)
> at java.lang.Thread.run(Thread.java:750)
> Caused by: java.lang.IllegalStateException: SSTABLE UnfilteredRowIterator for
> ks.tb has an illegal RT bounds sequence: unexpected end bound or boundary
> Marker INCL_START_BOUND(1)@1715128933051779/1715128933
> at
> org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.ise(RTBoundValidator.java:120)
> at
> org.apache.cassandra.db.transform.RTBoundValidator$RowsTransformation.applyToMarker(RTBoundValidator.java:87)
> at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:144)
> at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:131)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:77)
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:297)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:187)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:180)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:176)
> at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:76)
> at
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:347)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1914)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2652)
> ... 5 common frames omitted {code}
> h1. Reproduce
> 1. Start up cassandra-2.2.19, single node. Set column_index_size_in_kb in
> cassandra.yaml to 1.
> {code:java}
> // cassandra.yaml
> column_index_size_in_kb: 1 {code}
> 2. Execute the following commands (I masked out all data in the INSERT
> command using a long string of 'A' to make it simpler)
> {code:java}
> CREATE KEYSPACE ks WITH REPLICATION = { 'class' : 'SimpleStrategy',
> 'replication_factor' : 1 };
> CREATE TABLE ks.tb (c3 INT, c7 INT, c0 TEXT,c6 set, PRIMARY KEY (c3,
> c7, c0));DELETE FROM ks.tb WHERE c3 = 1 AND c7 = 1;
> DELETE FROM ks.tb WHERE c3 = 1 AND c7 = 1 AND c0 = 'e';INSERT INTO ks.tb (c7,
> c3, c6, c0) VALUES
> (1,1,{'AA
[jira] [Updated] (CASSANDRA-19429) Remove lock contention generated by getCapacity function in SSTableReader
[
https://issues.apache.org/jira/browse/CASSANDRA-19429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-19429:
--
Status: Needs Committer (was: Patch Available)
> Remove lock contention generated by getCapacity function in SSTableReader
> -
>
> Key: CASSANDRA-19429
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19429
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dipietro Salvatore
>Assignee: Dipietro Salvatore
>Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
> Attachments: Screenshot 2024-02-26 at 10.27.10.png, Screenshot
> 2024-02-27 at 11.29.41.png, Screenshot 2024-03-19 at 15.22.50.png,
> asprof_cass4.1.3__lock_20240216052912lock.html,
> image-2024-03-08-15-51-30-439.png, image-2024-03-08-15-52-07-902.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Profiling Cassandra 4.1.3 on large AWS instances, a high number of lock
> acquires is measured in the `getCapacity` function from
> `org/apache/cassandra/cache/InstrumentingCache` (1.9M lock acquires per 60
> seconds). Based on our tests on r8g.24xlarge instances (using Ubuntu 22.04),
> this limits the CPU utilization of the system to under 50% when testing at
> full load and therefore limits the achieved throughput.
> Removing the lock contention from the SSTableReader.java file by replacing
> the call to `getCapacity` with `size` achieves up to 2.95x increase in
> throughput on r8g.24xlarge and 2x on r7i.24xlarge:
> |Instance type|Cass 4.1.3|Cass 4.1.3 patched|
> |r8g.24xlarge|168k ops|496k ops (2.95x)|
> |r7i.24xlarge|153k ops|304k ops (1.98x)|
>
> Instructions to reproduce:
> {code:java}
> ## Requirements for Ubuntu 22.04
> sudo apt install -y ant git openjdk-11-jdk
> ## Build and run
> CASSANDRA_USE_JDK11=true ant realclean && CASSANDRA_USE_JDK11=true ant jar &&
> CASSANDRA_USE_JDK11=true ant stress-build && rm -rf data && bin/cassandra -f
> -R
> # Run
> bin/cqlsh -e 'drop table if exists keyspace1.standard1;' && \
> bin/cqlsh -e 'drop keyspace if exists keyspace1;' && \
> bin/nodetool clearsnapshot --all && tools/bin/cassandra-stress write
> n=1000 cl=ONE -rate threads=384 -node 127.0.0.1 -log file=cload.log
> -graph file=cload.html && \
> bin/nodetool compact keyspace1 && sleep 30s && \
> tools/bin/cassandra-stress mixed ratio\(write=10,read=90\) duration=10m
> cl=ONE -rate threads=406 -node localhost -log file=result.log -graph
> file=graph.html
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17354) Bump java-driver dependency in Cassandra to latest 3.x series
[ https://issues.apache.org/jira/browse/CASSANDRA-17354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-17354: Resolution: Won't Fix Status: Resolved (was: Open) As per Abe's message > Bump java-driver dependency in Cassandra to latest 3.x series > -- > > Key: CASSANDRA-17354 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17354 > Project: Cassandra > Issue Type: Task > Components: Test/unit >Reporter: Alex Petrov >Priority: High > Fix For: 5.x > > > We depend on java-driver for testing, and developing/validating native > protocol changes. Unfortunately, the version of drvier that is included with > Cassandra is quite ancient: 3.0.1. We need to bump this dependency to latest > in 3.x series, without upgrading to 4.0 at least for now. Unfortunately, this > is not a trivial change in build.xml (otherwise I would’ve done it rather > than opening this ticket), and bumping version breaks a few tests in all > versions, so those need to be fixed, too. > This should be a prerequiste for the next minor version release, too. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
