[jira] [Updated] (CASSANDRA-15804) system_schema keyspace complain of schema mismatch during upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-15804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandeep Varupula updated CASSANDRA-15804:
-
Since Version: 3.11.3 (was: 3.11.4)
> system_schema keyspace complain of schema mismatch during upgrade
> -
>
> Key: CASSANDRA-15804
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15804
> Project: Cassandra
> Issue Type: Bug
>Reporter: Pedro Gordo
>Priority: Low
>
> When upgrading from 3.11.4 to 3.11.6, we got the following error:
> {code:Plain Text}
> ERROR [MessagingService-Incoming-/10.20.11.59] 2020-05-07 13:53:52,627
> CassandraDaemon.java:228 - Exception in thread
> Thread[MessagingService-Incoming-/10.20.11.59,5,main]
> java.lang.RuntimeException: Unknown column kind during deserialization
> at
> org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:464)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.db.SerializationHeader$Serializer.deserializeForMessaging(SerializationHeader.java:419)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.deserializeHeader(UnfilteredRowIteratorSerializer.java:195)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:851)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:839)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:425)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:434)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:675)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:658)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at org.apache.cassandra.net.MessageIn.read(MessageIn.java:123)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:192)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:180)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:94)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> {code}
> I've noticed that system_schema.dropped_columns has a new column called
> "kind".
> No issues arise from this error message, and the error disappeared after
> upgrading all nodes. But it still caused concerns due to the ERROR logging
> level, although "nodetool describecluster" reported only one schema version.
> It makes sense for the system keyspaces to not be included for the
> "describecluster" schema version check, but it seems to me that these
> internal schema mismatches should be ignored if the versions are different
> between the nodes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15804) system_schema keyspace complain of schema mismatch during upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-15804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17140191#comment-17140191
]
Sandeep Varupula commented on CASSANDRA-15804:
--
I faced same issue while upgrading from 3.11.3 to 3.11.6. I agree with above
description. We need to handle error better.
{code:java}
// DEBUG [NonPeriodicTasks:1] 2020-06-18 13:55:13,490 MigrationManager.java:142
- submitting migration task for /xx.xx.xx.xx, schema version mismatch:
local/real=efc0553c-f5af-3a7c-ae2b-59403c
3aa9a1, local/compatible=313561f1-3294-3395-ba8e-81b712ecf7e9,
remote=e14e388e-f3e6-34ba-9067-8379f6c4f012
DEBUG [NonPeriodicTasks:1] 2020-06-18 13:55:13,491 MigrationManager.java:142 -
submitting migration task for /xx.xx.xx.xx, schema version mismatch:
local/real=efc0553c-f5af-3a7c-ae2b-59403c
3aa9a1, local/compatible=313561f1-3294-3395-ba8e-81b712ecf7e9,
remote=e14e388e-f3e6-34ba-9067-8379f6c4f012
ERROR [MessagingService-Incoming-/xx.xx.xx.xx] 2020-06-18 13:55:13,502
CassandraDaemon.java:228 - Exception in thread
Thread[MessagingService-Incoming-/xx.xx.xx.xx,5,main]
java.lang.RuntimeException: Unknown column kind during deserialization
at
org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:452)
~[apache-cassandra-3.11.3.jar:3.11.3]
at
org.apache.cassandra.db.SerializationHeader$Serializer.deserializeForMessaging(SerializationHeader.java:412)
~[apache-cassandra-3.11.3.jar:3.11.3]
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.deserializeHeader(UnfilteredRowIteratorSerializer.java:195)
~[apache-cassandra-3.11.3.jar:3.11.3]
at
org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:851)
~[apache-cassandra-3.11.3.jar:3.11.3]
at
org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:839)
~[apache-cassandra-3.11.3.jar:3.11.3]
at
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:425)
~[apache-cassandra-3.11.3.jar:3.11.3]
at
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:434)
~[apache-cassandra-3.11.3.jar:3.11.3]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:669)
~[apache-cassandra-3.11.3.jar:3.11.3]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:652)
~[apache-cassandra-3.11.3.jar:3.11.3]
at org.apache.cassandra.net.MessageIn.read(MessageIn.java:123)
~[apache-cassandra-3.11.3.jar:3.11.3]
at
org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:192)
~[apache-cassandra-3.11.3.jar:3.11.3]
at
org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:180)
~[apache-cassandra-3.11.3.jar:3.11.3]
at
org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:94)
~[apache-cassandra-3.11.3.jar:3.11.3]
{code}
> system_schema keyspace complain of schema mismatch during upgrade
> -
>
> Key: CASSANDRA-15804
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15804
> Project: Cassandra
> Issue Type: Bug
>Reporter: Pedro Gordo
>Priority: Low
>
> When upgrading from 3.11.4 to 3.11.6, we got the following error:
> {code:Plain Text}
> ERROR [MessagingService-Incoming-/10.20.11.59] 2020-05-07 13:53:52,627
> CassandraDaemon.java:228 - Exception in thread
> Thread[MessagingService-Incoming-/10.20.11.59,5,main]
> java.lang.RuntimeException: Unknown column kind during deserialization
> at
> org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:464)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.db.SerializationHeader$Serializer.deserializeForMessaging(SerializationHeader.java:419)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.deserializeHeader(UnfilteredRowIteratorSerializer.java:195)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:851)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:839)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:425)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:434)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
>
[jira] [Commented] (CASSANDRA-15884) Add a startup check to detect if LZ4 uses java rather than native implementation
[ https://issues.apache.org/jira/browse/CASSANDRA-15884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17140079#comment-17140079 ] David Capwell commented on CASSANDRA-15884: --- testing was down manually by making sure the glibc on the host was older than the one compiled against lz4-java. When profiled saw that the Unsafe java implementation was used, but when the host glibc is new enough then Jni is used. > Add a startup check to detect if LZ4 uses java rather than native > implementation > > > Key: CASSANDRA-15884 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15884 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > > In some environments the native implementation can’t link with glibc and > silently falls back to java; this hurts performance. To help warn the > operators, we should add a startup check to detect this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15884) Add a startup check to detect if LZ4 uses java rather than native implementation
[ https://issues.apache.org/jira/browse/CASSANDRA-15884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-15884: -- Test and Documentation Plan: manual testing Status: Patch Available (was: Open) > Add a startup check to detect if LZ4 uses java rather than native > implementation > > > Key: CASSANDRA-15884 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15884 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > > In some environments the native implementation can’t link with glibc and > silently falls back to java; this hurts performance. To help warn the > operators, we should add a startup check to detect this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15884) Add a startup check to detect if LZ4 uses java rather than native implementation
[ https://issues.apache.org/jira/browse/CASSANDRA-15884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-15884: -- Change Category: Performance Complexity: Low Hanging Fruit Status: Open (was: Triage Needed) > Add a startup check to detect if LZ4 uses java rather than native > implementation > > > Key: CASSANDRA-15884 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15884 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > > In some environments the native implementation can’t link with glibc and > silently falls back to java; this hurts performance. To help warn the > operators, we should add a startup check to detect this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15884) Add a startup check to detect if LZ4 uses java rather than native implementation
David Capwell created CASSANDRA-15884: - Summary: Add a startup check to detect if LZ4 uses java rather than native implementation Key: CASSANDRA-15884 URL: https://issues.apache.org/jira/browse/CASSANDRA-15884 Project: Cassandra Issue Type: Improvement Components: Dependencies Reporter: David Capwell Assignee: David Capwell In some environments the native implementation can’t link with glibc and silently falls back to java; this hurts performance. To help warn the operators, we should add a startup check to detect this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15877) Followup on CASSANDRA-15600
[ https://issues.apache.org/jira/browse/CASSANDRA-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15877: Test and Documentation Plan: https://jira.apache.org/jira/browse/CASSANDRA-15877?focusedCommentId=17140064=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17140064 (was: https://jira.apache.org/jira/browse/CASSANDRA-15877?focusedCommentId=17136175=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17136175) > Followup on CASSANDRA-15600 > --- > > Key: CASSANDRA-15877 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15877 > Project: Cassandra > Issue Type: Bug > Components: Feature/Virtual Nodes >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-alpha > > Attachments: Screen Shot 2020-06-12 at 3.21.18 PM.png > > > As part of CASSANDRA-15600 generateSplits method replaced the > generateRandomTokens for NoReplicationAwareTokenAllocator. generateSplits > should be used also in ReplicationAwareTokenAllocator. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15877) Followup on CASSANDRA-15600
[ https://issues.apache.org/jira/browse/CASSANDRA-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17140064#comment-17140064 ] Ekaterina Dimitrova commented on CASSANDRA-15877: - [ [branch |https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15600-followup]] [ [Java8 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/222/workflows/46515d14-9be4-4edb-8db4-5930312d2bfb]] [ [Java11 CI |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/222/workflows/11202c7e-6c94-4d4e-bbbf-9e2fa9791ad0]] [ [pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/44]] Failing dtests not related to this patch but I didn't find tickets so probably good to follow up on them: JAVA 11: test_resumable_rebuild - rebuild_test.TestRebuild - not related but I didn't find a ticket so good to follow up on it probably test_short_read - consistency_test.TestConsistency - not related but I didn't find a ticket so good to follow up on it probably JAVA 8: test_multiple_repair - repair_tests.incremental_repair_test.TestIncRepair - not related but I didn't find a ticket so good to follow up on it probably The rest of the dtests failures are handled under [CASSANDRA-15677|https://issues.apache.org/jira/browse/CASSANDRA-15677] Screenshot added to prove the fixed unit test is not flaky anymore. *The first commit is the actual patch. The second commit applies our custom CircleCI config and shouldn't be committed! Kept for test purposes.* *The third commit is to add event for new split generations plus removal of unused code.* > Followup on CASSANDRA-15600 > --- > > Key: CASSANDRA-15877 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15877 > Project: Cassandra > Issue Type: Bug > Components: Feature/Virtual Nodes >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-alpha > > Attachments: Screen Shot 2020-06-12 at 3.21.18 PM.png > > > As part of CASSANDRA-15600 generateSplits method replaced the > generateRandomTokens for NoReplicationAwareTokenAllocator. generateSplits > should be used also in ReplicationAwareTokenAllocator. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15766) NoSpamLogger arguments building objects on hot paths
[ https://issues.apache.org/jira/browse/CASSANDRA-15766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17140037#comment-17140037 ] Jon Meredith commented on CASSANDRA-15766: -- Oh yeah, forgot that's how it went down - thanks for the reminder [~dcapwell]. This was the Jira I filed after it was found. > NoSpamLogger arguments building objects on hot paths > > > Key: CASSANDRA-15766 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15766 > Project: Cassandra > Issue Type: Bug > Components: Observability/Logging >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > Fix For: 4.0-beta > > > NoSpamLogger is used in hot logging paths to prevent logs being overrun. For > that to be most effective the arguments to the logger need to be cheap to > construct. During the internode messaging refactor CASSANDRA-15066, > performance changes to BufferPool for CASSANDRA-14416 > were accidentally reverted in the merge up from 3.11. > Reviewing other uses since, it looks like there are a few places where the > arguments require some form of String building. > org.apache.cassandra.net.InboundSink#accept > org.apache.cassandra.net.InboundMessageHandler#processCorruptFrame > org.apache.cassandra.net.InboundMessageHandler.LargeMessage#deserialize > org.apache.cassandra.net.OutboundConnection#onOverloaded > org.apache.cassandra.utils.memory.BufferPool.GlobalPool#allocateMoreChunks > Formatting arguments should either be precomputed, or if expensive they > should be computed after the decision on whether to noSpamLog has been made. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15861) Mutating sstable component may race with entire-sstable-streaming(ZCS) causing checksum validation failure
[
https://issues.apache.org/jira/browse/CASSANDRA-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17140036#comment-17140036
]
David Capwell commented on CASSANDRA-15861:
---
I took a quick look and have a few comments
*
https://github.com/apache/cassandra/pull/642/files#diff-de503ddc819368b078a86f6d9e3921aeR211-R228.
isn't the write async; we are just storing on a buffer to be written later?
Looking at
org.apache.cassandra.net.AsyncStreamingOutputPlus#writeFileToChannelZeroCopy it
calls channel.writeAndFlush but doesn't look at the future, so the write is
async.
* have you done any longevity testing of this? My fear is that compaction will
get blocked while streaming is running which could cause slowness or stability
issues.
> Mutating sstable component may race with entire-sstable-streaming(ZCS)
> causing checksum validation failure
> --
>
> Key: CASSANDRA-15861
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15861
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair, Consistency/Streaming,
> Local/Compaction
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Flaky dtest: [test_dead_sync_initiator -
> repair_tests.repair_test.TestRepair|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/143/testReport/junit/dtest.repair_tests.repair_test/TestRepair/test_dead_sync_initiator/]
> {code:java|title=stacktrace}
> Unexpected error found in node logs (see stdout for full details). Errors:
> [ERROR [Stream-Deserializer-127.0.0.1:7000-570871f3] 2020-06-03 04:05:19,081
> CassandraEntireSSTableStreamReader.java:145 - [Stream
> 6f1c3360-a54f-11ea-a808-2f23710fdc90] Error while reading sstable from stream
> for table = keyspace1.standard1
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted:
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.maybeValidateChecksum(MetadataSerializer.java:219)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:198)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:129)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.mutate(MetadataSerializer.java:226)
> at
> org.apache.cassandra.db.streaming.CassandraEntireSSTableStreamReader.read(CassandraEntireSSTableStreamReader.java:140)
> at
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:78)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)
> at
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:181)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Checksums do not match for
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
> {code}
>
> In the above test, it executes "nodetool repair" on node1 and kills node2
> during repair. At the end, node3 reports checksum validation failure on
> sstable transferred from node1.
> {code:java|title=what happened}
> 1. When repair started on node1, it performs anti-compaction which modifies
> sstable's repairAt to 0 and pending repair id to session-id.
> 2. Then node1 creates {{ComponentManifest}} which contains file lengths to be
> transferred to node3.
> 3. Before node1 actually sends the files to node3, node2 is killed and node1
> starts to broadcast repair-failure-message to all participants in
> {{CoordinatorSession#fail}}
> 4. Node1 receives its own repair-failure-message and fails its local repair
> sessions at {{LocalSessions#failSession}} which triggers async background
> compaction.
> 5. Node1's background compaction will mutate sstable's repairAt to 0 and
> pending repair id to null via
> {{PendingRepairManager#getNextRepairFinishedTask}}, as there is no more
> in-progress repair.
> 6. Node1 actually sends the sstable to node3 where the sstable's STATS
> component size is different from the original size recorded
[jira] [Commented] (CASSANDRA-15766) NoSpamLogger arguments building objects on hot paths
[ https://issues.apache.org/jira/browse/CASSANDRA-15766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17140024#comment-17140024 ] David Capwell commented on CASSANDRA-15766: --- [~jmeredithco], [~jmckenzie], [~djoshi] found a regression in networking that was fixed by Jeff in 3.x; checking latest trunk I don't see it fixed... [~djoshi] did you file? prettyPrintMemory gets called in the hot path > NoSpamLogger arguments building objects on hot paths > > > Key: CASSANDRA-15766 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15766 > Project: Cassandra > Issue Type: Bug > Components: Observability/Logging >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > Fix For: 4.0-beta > > > NoSpamLogger is used in hot logging paths to prevent logs being overrun. For > that to be most effective the arguments to the logger need to be cheap to > construct. During the internode messaging refactor CASSANDRA-15066, > performance changes to BufferPool for CASSANDRA-14416 > were accidentally reverted in the merge up from 3.11. > Reviewing other uses since, it looks like there are a few places where the > arguments require some form of String building. > org.apache.cassandra.net.InboundSink#accept > org.apache.cassandra.net.InboundMessageHandler#processCorruptFrame > org.apache.cassandra.net.InboundMessageHandler.LargeMessage#deserialize > org.apache.cassandra.net.OutboundConnection#onOverloaded > org.apache.cassandra.utils.memory.BufferPool.GlobalPool#allocateMoreChunks > Formatting arguments should either be precomputed, or if expensive they > should be computed after the decision on whether to noSpamLog has been made. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15766) NoSpamLogger arguments building objects on hot paths
[ https://issues.apache.org/jira/browse/CASSANDRA-15766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Meredith updated CASSANDRA-15766: - Fix Version/s: (was: 4.0-rc) 4.0-beta > NoSpamLogger arguments building objects on hot paths > > > Key: CASSANDRA-15766 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15766 > Project: Cassandra > Issue Type: Bug > Components: Observability/Logging >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > Fix For: 4.0-beta > > > NoSpamLogger is used in hot logging paths to prevent logs being overrun. For > that to be most effective the arguments to the logger need to be cheap to > construct. During the internode messaging refactor CASSANDRA-15066, > performance changes to BufferPool for CASSANDRA-14416 > were accidentally reverted in the merge up from 3.11. > Reviewing other uses since, it looks like there are a few places where the > arguments require some form of String building. > org.apache.cassandra.net.InboundSink#accept > org.apache.cassandra.net.InboundMessageHandler#processCorruptFrame > org.apache.cassandra.net.InboundMessageHandler.LargeMessage#deserialize > org.apache.cassandra.net.OutboundConnection#onOverloaded > org.apache.cassandra.utils.memory.BufferPool.GlobalPool#allocateMoreChunks > Formatting arguments should either be precomputed, or if expensive they > should be computed after the decision on whether to noSpamLog has been made. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15766) NoSpamLogger arguments building objects on hot paths
[ https://issues.apache.org/jira/browse/CASSANDRA-15766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17140005#comment-17140005 ] Jon Meredith commented on CASSANDRA-15766: -- We don't currently. The original regression was spotted by code inspection rather than profiling, as were the other candidates mentioned in the original description. I think it's a fair observation that we should have a quantitive measure of the improvement. I've also noticed this was tagged as 4.0-rc and moved this back to the 4.0-beta milestone for now. It may be possible to slip this, though the original changes were made due to operational issues that made the change important. > NoSpamLogger arguments building objects on hot paths > > > Key: CASSANDRA-15766 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15766 > Project: Cassandra > Issue Type: Bug > Components: Observability/Logging >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > Fix For: 4.0-rc > > > NoSpamLogger is used in hot logging paths to prevent logs being overrun. For > that to be most effective the arguments to the logger need to be cheap to > construct. During the internode messaging refactor CASSANDRA-15066, > performance changes to BufferPool for CASSANDRA-14416 > were accidentally reverted in the merge up from 3.11. > Reviewing other uses since, it looks like there are a few places where the > arguments require some form of String building. > org.apache.cassandra.net.InboundSink#accept > org.apache.cassandra.net.InboundMessageHandler#processCorruptFrame > org.apache.cassandra.net.InboundMessageHandler.LargeMessage#deserialize > org.apache.cassandra.net.OutboundConnection#onOverloaded > org.apache.cassandra.utils.memory.BufferPool.GlobalPool#allocateMoreChunks > Formatting arguments should either be precomputed, or if expensive they > should be computed after the decision on whether to noSpamLog has been made. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139987#comment-17139987
]
Benedict Elliott Smith commented on CASSANDRA-15299:
I think it makes sense to dub v5 protocol alpha, or unavailable, or stub it
out, since this will change the protocol more than we would permit in a beta.
But otherwise I think everybody expressed a belief it was OK to move to beta
without this, so long as it is delivered during beta.
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-alpha
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it
> introduced checksumming/CRC32 to request and response bodies. It’s an
> important step forward, but it doesn’t cover the entire stream. In
> particular, the message header is not covered by a checksum or a crc, which
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of
> computing the CRC32 on the bytes written on the wire - losing the properties
> of the CRC32. In some cases, due to this sequencing, attempting to decompress
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order,
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression
> operate on individual message bodies rather than frames of multiple complete
> messages. In reality, this has several important additional downsides. To
> name a couple:
> # For compression, we are getting poor compression ratios for smaller
> messages - when operating on tiny sequences of bytes. In reality, for most
> small requests and responses we are discarding the compressed value as it’d
> be smaller than the uncompressed one - incurring both redundant allocations
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages.
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed,
> and that we do it before native protocol v5 graduates from beta. See
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
> and
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderLZ4.java.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139984#comment-17139984
]
Josh McKenzie commented on CASSANDRA-15299:
---
Where did we land from the ML? I think it was that we're going to dub v5
protocol beta until this gets in and not block rel of beta1 on this - did I get
that right?
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-alpha
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it
> introduced checksumming/CRC32 to request and response bodies. It’s an
> important step forward, but it doesn’t cover the entire stream. In
> particular, the message header is not covered by a checksum or a crc, which
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of
> computing the CRC32 on the bytes written on the wire - losing the properties
> of the CRC32. In some cases, due to this sequencing, attempting to decompress
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order,
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression
> operate on individual message bodies rather than frames of multiple complete
> messages. In reality, this has several important additional downsides. To
> name a couple:
> # For compression, we are getting poor compression ratios for smaller
> messages - when operating on tiny sequences of bytes. In reality, for most
> small requests and responses we are discarding the compressed value as it’d
> be smaller than the uncompressed one - incurring both redundant allocations
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages.
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed,
> and that we do it before native protocol v5 graduates from beta. See
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
> and
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderLZ4.java.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15883) Delete and save with same keys
[ https://issues.apache.org/jira/browse/CASSANDRA-15883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yazal Ulloa updated CASSANDRA-15883: Description: I have a table like this: key1, key2, key3, column1, column2. When I delete a row with keys 1, 2, 3. and then insert a new row almost immediately with the same keys, cassandra does not store the new row even though the query executes successfully. How can I force Cassandra to store the new row, or do I have to change my data model? was: I have a table like this: key1, key2, key3, column1, column2. When I delete a row with keys 1, 2, 3. and then insert a new row with the same keys, cassandra does not store the new row even though the query executes successfully. How can I force Cassandra to store the new row, or do I have to change my data model? > Delete and save with same keys > -- > > Key: CASSANDRA-15883 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15883 > Project: Cassandra > Issue Type: Bug >Reporter: Yazal Ulloa >Priority: Normal > > I have a table like this: key1, key2, key3, column1, column2. > > When I delete a row with keys 1, 2, 3. and then insert a new row almost > immediately with the same keys, cassandra does not store the new row even > though the query executes successfully. > > How can I force Cassandra to store the new row, or do I have to change my > data model? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15883) Delete and save with same keys
[ https://issues.apache.org/jira/browse/CASSANDRA-15883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yazal Ulloa updated CASSANDRA-15883: Description: I have a table like this: key1, key2, key3, column1, column2. When I delete a row with keys 1, 2, 3. and then insert a new row with the same keys, cassandra does not store the new row even though the query executes successfully. How can I force Cassandra to store the new row, or do I have to change my data model? was: I have a table like this: key1, key2, key3, column1, column2. When I delete a row with keys 1, 2, 3. and then insert a new row with the same keys, cassandra does not store the new row even though the query executes successfully > Delete and save with same keys > -- > > Key: CASSANDRA-15883 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15883 > Project: Cassandra > Issue Type: Bug >Reporter: Yazal Ulloa >Priority: Normal > > I have a table like this: key1, key2, key3, column1, column2. > > When I delete a row with keys 1, 2, 3. and then insert a new row with the > same keys, cassandra does not store the new row even though the query > executes successfully. > > How can I force Cassandra to store the new row, or do I have to change my > data model? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15883) Delete and save with same keys
[ https://issues.apache.org/jira/browse/CASSANDRA-15883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yazal Ulloa updated CASSANDRA-15883: Description: I have a table like this: key1, key2, key3, column1, column2. When I delete a row with keys 1, 2, 3. and then insert a new row with the same keys, cassandra does not store the new row even though the query executes successfully was:I have a table like this: key1, key2, key3, column1, column2. > Delete and save with same keys > -- > > Key: CASSANDRA-15883 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15883 > Project: Cassandra > Issue Type: Bug >Reporter: Yazal Ulloa >Priority: Normal > > I have a table like this: key1, key2, key3, column1, column2. > > When I delete a row with keys 1, 2, 3. and then insert a new row with the > same keys, cassandra does not store the new row even though the query > executes successfully -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15883) Delete and save with same keys
[ https://issues.apache.org/jira/browse/CASSANDRA-15883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yazal Ulloa updated CASSANDRA-15883: Description: I have a table like this: key1, key2, key3, column1, column2. (was: I have a table likes this:) > Delete and save with same keys > -- > > Key: CASSANDRA-15883 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15883 > Project: Cassandra > Issue Type: Bug >Reporter: Yazal Ulloa >Priority: Normal > > I have a table like this: key1, key2, key3, column1, column2. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15883) Delete and save with same keys
Yazal Ulloa created CASSANDRA-15883: --- Summary: Delete and save with same keys Key: CASSANDRA-15883 URL: https://issues.apache.org/jira/browse/CASSANDRA-15883 Project: Cassandra Issue Type: Bug Reporter: Yazal Ulloa I have a table likes this: -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[ https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139920#comment-17139920 ] Sam Tunnicliffe edited comment on CASSANDRA-15299 at 6/18/20, 6:58 PM: --- Pushed an updated branch where the protocol changes are pretty stable. I've been testing this with the [java2772|https://github.com/datastax/java-driver/tree/java2772] branch of the driver and with debug-cql and everything is working pretty much as expected. Due to current lack of support in the python driver, I've had to use a modified [dtest branch|https://github.com/beobal/cassandra-dtest/tree/15299] and make a temporary hack to cqlsh, but the tests that are running are pretty much green. Obviously, those are not covering anything from v5 now, but my primary concern was to make sure there's no regressions for v4 clients. Although I think this is ready for some more eyes on it, there's still a non-trivial amount of work to be done. Items outstanding include: * Comprehensive unit and in-jvm tests - in progress * Metrics * Python driver support (doesn't have to be fully implemented, but a basic level is needed for pytests and cqlsh) * Documentation * Renaming existing classes. There are a number of slightly confusing conflicts in naming now. These should be simple to resolve, just automated renaming mostly, but I've held off doing them for now because they'll make the patch much bigger and probably harder to read. The patch is also not quite a massive at it might appear at first. A large proportion is the revert of CASSANDRA-13304, and the rest is largely additive. I've tried hard not touch code on the v4 path, even where it could clearly be refactored, to minimize the delta & the risk there. So the patch largely consists of some v5 specific additions, moving a few existing classes/methods around, and changing modifiers on previously private/package-private things. I'm still actively working on the tests, but I don't think that (nor the renaming) need hold up review any more. |branch|[15299-trunk|https://github.com/beobal/cassandra/tree/15299-trunk]| |dtests|[15299|https://github.com/beobal/cassandra-dtest/tree/15299]| |ci|[circle|https://app.circleci.com/pipelines/github/beobal/cassandra?branch=15299-trunk]| was (Author: beobal): Pushed an updated branch where the protocol changes are pretty stable. I've been testing this with the java2772 branch of the driver and with debug-cql and everything is working pretty much as expected. Due to current lack of support in the python driver, I've had to use a modified [dtest branch|https://github.com/beobal/cassandra-dtest/tree/15299] and make a temporary hack to cqlsh, but the tests that are running are pretty much green. Obviously, those are not covering anything from v5 now, but my primary concern was to make sure there's no regressions for v4 clients. Although I think this is ready for some more eyes on it, there's still a non-trivial amount of work to be done. Items outstanding include: * Comprehensive unit and in-jvm tests - in progress * Metrics * Python driver support (doesn't have to be fully implemented, but a basic level is needed for pytests and cqlsh) * Documentation * Renaming existing classes. There are a number of slightly confusing conflicts in naming now. These should be simple to resolve, just automated renaming mostly, but I've held off doing them for now because they'll make the patch much bigger and probably harder to read. The patch is also not quite a massive at it might appear at first. A large proportion is the revert of CASSANDRA-13304, and the rest is largely additive. I've tried hard not touch code on the v4 path, even where it could clearly be refactored, to minimize the delta & the risk there. So the patch largely consists of some v5 specific additions, moving a few existing classes/methods around, and changing modifiers on previously private/package-private things. I'm still actively working on the tests, but I don't think that (nor the renaming) need hold up review any more. |branch|[15299-trunk|https://github.com/beobal/cassandra/tree/15299-trunk]| |dtests|[15299|https://github.com/beobal/cassandra-dtest/tree/15299]| |ci|[circle|https://app.circleci.com/pipelines/github/beobal/cassandra?branch=15299-trunk]| > CASSANDRA-13304 follow-up: improve checksumming and compression in protocol > v5-beta > --- > > Key: CASSANDRA-15299 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15299 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client >Reporter: Aleksey Yeschenko >Assignee: Sam Tunnicliffe >Priority: Normal > Labels: protocolv5 > Fix For: 4.0-alpha >
[jira] [Updated] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-15299:
Reviewers: Alex Petrov, ZhaoYang (was: ZhaoYang)
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-alpha
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it
> introduced checksumming/CRC32 to request and response bodies. It’s an
> important step forward, but it doesn’t cover the entire stream. In
> particular, the message header is not covered by a checksum or a crc, which
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of
> computing the CRC32 on the bytes written on the wire - losing the properties
> of the CRC32. In some cases, due to this sequencing, attempting to decompress
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order,
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression
> operate on individual message bodies rather than frames of multiple complete
> messages. In reality, this has several important additional downsides. To
> name a couple:
> # For compression, we are getting poor compression ratios for smaller
> messages - when operating on tiny sequences of bytes. In reality, for most
> small requests and responses we are discarding the compressed value as it’d
> be smaller than the uncompressed one - incurring both redundant allocations
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages.
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed,
> and that we do it before native protocol v5 graduates from beta. See
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
> and
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderLZ4.java.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-15299:
Test and Documentation Plan:
* Validate with java-driver test suites
* Improve coverage of client/server interaction in unit/in-jvm dtests
* Add burn tests
* Update v5 protocol spec
Status: Patch Available (was: In Progress)
Pushed an updated branch where the protocol changes are pretty stable. I've
been testing this with the java2772 branch of the driver and with debug-cql and
everything is working pretty much as expected. Due to current lack of support
in the python driver, I've had to use a modified [dtest
branch|https://github.com/beobal/cassandra-dtest/tree/15299] and make a
temporary hack to cqlsh, but the tests that are running are pretty much green.
Obviously, those are not covering anything from v5 now, but my primary concern
was to make sure there's no regressions for v4 clients.
Although I think this is ready for some more eyes on it, there's still a
non-trivial amount of work to be done. Items outstanding include:
* Comprehensive unit and in-jvm tests - in progress
* Metrics
* Python driver support (doesn't have to be fully implemented, but a basic
level is needed for pytests and cqlsh)
* Documentation
* Renaming existing classes. There are a number of slightly confusing conflicts
in naming now. These should be simple to resolve, just automated renaming
mostly, but I've held off doing them for now because they'll make the patch
much bigger and probably harder to read.
The patch is also not quite a massive at it might appear at first. A large
proportion is the revert of CASSANDRA-13304, and the rest is largely additive.
I've tried hard not touch code on the v4 path, even where it could clearly be
refactored, to minimize the delta & the risk there. So the patch largely
consists of some v5 specific additions, moving a few existing classes/methods
around, and changing modifiers on previously private/package-private things.
I'm still actively working on the tests, but I don't think that (nor the
renaming) need hold up review any more.
|branch|[15299-trunk|https://github.com/beobal/cassandra/tree/15299-trunk]|
|dtests|[15299|https://github.com/beobal/cassandra-dtest/tree/15299]|
|ci|[circle|https://app.circleci.com/pipelines/github/beobal/cassandra?branch=15299-trunk]|
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-alpha
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it
> introduced checksumming/CRC32 to request and response bodies. It’s an
> important step forward, but it doesn’t cover the entire stream. In
> particular, the message header is not covered by a checksum or a crc, which
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of
> computing the CRC32 on the bytes written on the wire - losing the properties
> of the CRC32. In some cases, due to this sequencing, attempting to decompress
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order,
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression
> operate on individual message bodies rather than frames of multiple complete
> messages. In reality, this has several important additional downsides. To
> name a couple:
> # For compression, we are getting poor compression ratios for smaller
> messages - when operating on tiny sequences of bytes. In reality, for most
> small requests and responses we are discarding the compressed value as it’d
> be smaller than the uncompressed one - incurring both redundant allocations
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages.
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we
[jira] [Commented] (CASSANDRA-15766) NoSpamLogger arguments building objects on hot paths
[ https://issues.apache.org/jira/browse/CASSANDRA-15766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139911#comment-17139911 ] Josh McKenzie commented on CASSANDRA-15766: --- Do we know how heavy / prevalent these calls are on a flame graph? > NoSpamLogger arguments building objects on hot paths > > > Key: CASSANDRA-15766 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15766 > Project: Cassandra > Issue Type: Bug > Components: Observability/Logging >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > Fix For: 4.0-rc > > > NoSpamLogger is used in hot logging paths to prevent logs being overrun. For > that to be most effective the arguments to the logger need to be cheap to > construct. During the internode messaging refactor CASSANDRA-15066, > performance changes to BufferPool for CASSANDRA-14416 > were accidentally reverted in the merge up from 3.11. > Reviewing other uses since, it looks like there are a few places where the > arguments require some form of String building. > org.apache.cassandra.net.InboundSink#accept > org.apache.cassandra.net.InboundMessageHandler#processCorruptFrame > org.apache.cassandra.net.InboundMessageHandler.LargeMessage#deserialize > org.apache.cassandra.net.OutboundConnection#onOverloaded > org.apache.cassandra.utils.memory.BufferPool.GlobalPool#allocateMoreChunks > Formatting arguments should either be precomputed, or if expensive they > should be computed after the decision on whether to noSpamLog has been made. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15809) ASF CI builds for JDK11
[ https://issues.apache.org/jira/browse/CASSANDRA-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139900#comment-17139900 ] shylaja kokoori commented on CASSANDRA-15809: - I have a patch ready. Will upload soon > ASF CI builds for JDK11 > --- > > Key: CASSANDRA-15809 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15809 > Project: Cassandra > Issue Type: Task > Components: Build, CI >Reporter: Michael Semb Wever >Assignee: shylaja kokoori >Priority: Normal > Fix For: 4.0-beta > > Attachments: Screenshot 2020-05-13 at 09.39.56.png > > > ASF CI builds today only run on JDK1.8 > On the Jenkins cluster JDKs from 1.4 through to 15 are available. See > attached screenshot for naming specifics. > This ticket is to add JDK11 compile and test targets on Cassandra-trunk, for > parity to CircleCI's workflows. (There is also the question about > testing/running on Cassandra-2.2 which needs to support JDK1.7, though > Cassandra-2.2 is nearing EOL.) > > The JDK is specified in the groovy DSL: > [https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L11] > > > Some examples: > * CircleCI JDK1.8 workflow example. This builds with JDK1.8 and tests with > both JDK1.8 and JDK11. > ** > [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/bce6fbf9-a4a3-4bfd-be7d-3e7961b440d8] > * CircleCI JDK11 workflow example. This builds and tests only with JDK11. > ** > [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/38e7d77e-1d5e-47f7-a462-277665428306] > * Jenkins Cqlshlib tests showing matrix builds: > ** > [https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/] > > Background > thread:[https://lists.apache.org/thread.html/rbaeb960901fa53b50227b37d64e5a456b68f749f4229c6e3e086ff85%40%3Cdev.cassandra.apache.org%3E] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15809) ASF CI builds for JDK11
[ https://issues.apache.org/jira/browse/CASSANDRA-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139895#comment-17139895 ] Josh McKenzie commented on CASSANDRA-15809: --- [~shylaja.koko...@intel.com] - are you active on this ticket or should someone else take a look at this? [~mck2]- I know you were looking into the jenkins situation quite a bit. Any relevant updates here? > ASF CI builds for JDK11 > --- > > Key: CASSANDRA-15809 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15809 > Project: Cassandra > Issue Type: Task > Components: Build, CI >Reporter: Michael Semb Wever >Assignee: shylaja kokoori >Priority: Normal > Fix For: 4.0-beta > > Attachments: Screenshot 2020-05-13 at 09.39.56.png > > > ASF CI builds today only run on JDK1.8 > On the Jenkins cluster JDKs from 1.4 through to 15 are available. See > attached screenshot for naming specifics. > This ticket is to add JDK11 compile and test targets on Cassandra-trunk, for > parity to CircleCI's workflows. (There is also the question about > testing/running on Cassandra-2.2 which needs to support JDK1.7, though > Cassandra-2.2 is nearing EOL.) > > The JDK is specified in the groovy DSL: > [https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L11] > > > Some examples: > * CircleCI JDK1.8 workflow example. This builds with JDK1.8 and tests with > both JDK1.8 and JDK11. > ** > [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/bce6fbf9-a4a3-4bfd-be7d-3e7961b440d8] > * CircleCI JDK11 workflow example. This builds and tests only with JDK11. > ** > [https://app.circleci.com/pipelines/github/dcapwell/cassandra/259/workflows/38e7d77e-1d5e-47f7-a462-277665428306] > * Jenkins Cqlshlib tests showing matrix builds: > ** > [https://ci-cassandra.apache.org/view/Cassandra%204.0/job/Cassandra-trunk-cqlsh-tests/] > > Background > thread:[https://lists.apache.org/thread.html/rbaeb960901fa53b50227b37d64e5a456b68f749f4229c6e3e086ff85%40%3Cdev.cassandra.apache.org%3E] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15663) DESCRIBE KEYSPACE does not properly quote table names
[
https://issues.apache.org/jira/browse/CASSANDRA-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139873#comment-17139873
]
Alan Boudreault commented on CASSANDRA-15663:
-
[~Ge] We just released cassandra driver 3.24. geomet is not required
anymore at runtime.
> DESCRIBE KEYSPACE does not properly quote table names
> -
>
> Key: CASSANDRA-15663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15663
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Syntax
>Reporter: Oskar Liljeblad
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
> Labels: pull-request-available
> Fix For: 3.11.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> How to reproduce (3.11.6) - cqlsh:
> {code}
> CREATE KEYSPACE test1 WITH replication = \{'class': 'SimpleStrategy',
> 'replication_factor': '1'} AND durable_writes = true;
> CREATE TABLE test1."default" (id text PRIMARY KEY, data text, etag text);
> DESCRIBE KEYSPACE test1;
> {code}
> Output will be:
> {code}
> CREATE TABLE test1.default (
> id text PRIMARY KEY,
> data text,
> etag text
> ) WITH [..]
> {code}
> Output should be:
> {code}
> CREATE TABLE test1."default" (
> id text PRIMARY KEY,
> data text,
> etag text
> ) WITH [..]
> {code}
> If you try to run {{CREATE TABLE test1.default [..]}} you will get an error
> SyntaxException: line 1:19 no viable alternative at input 'default' (CREATE
> TABLE test1.[default]...)
> Oskar Liljeblad
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15872) Flaky test: testNoTreesRetainedAfterDifference - org.apache.cassandra.repair.RepairJobTest
[ https://issues.apache.org/jira/browse/CASSANDRA-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhaoYang updated CASSANDRA-15872: - Test and Documentation Plan: https://circleci.com/workflow-run/06e85371-980f-4cf6-b5fa-e52f73e8c158 (was: https://circleci.com/workflow-run/fe9a153f-1a04-4993-b582-f0f1bbd9ca77) > Flaky test: testNoTreesRetainedAfterDifference - > org.apache.cassandra.repair.RepairJobTest > -- > > Key: CASSANDRA-15872 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15872 > Project: Cassandra > Issue Type: Task > Components: Consistency/Repair >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Normal > Fix For: 4.0-rc > > > link: [https://circleci.com/gh/maedhroz/cassandra/21#tests/containers/95] > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15872) Flaky test: testNoTreesRetainedAfterDifference - org.apache.cassandra.repair.RepairJobTest
[ https://issues.apache.org/jira/browse/CASSANDRA-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhaoYang updated CASSANDRA-15872: - Status: Patch Available (was: In Progress) > Flaky test: testNoTreesRetainedAfterDifference - > org.apache.cassandra.repair.RepairJobTest > -- > > Key: CASSANDRA-15872 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15872 > Project: Cassandra > Issue Type: Task > Components: Consistency/Repair >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Normal > Fix For: 4.0-rc > > > link: [https://circleci.com/gh/maedhroz/cassandra/21#tests/containers/95] > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15872) Flaky test: testNoTreesRetainedAfterDifference - org.apache.cassandra.repair.RepairJobTest
[ https://issues.apache.org/jira/browse/CASSANDRA-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhaoYang updated CASSANDRA-15872: - Reviewers: Caleb Rackliffe, ZhaoYang (was: Caleb Rackliffe) Status: Review In Progress (was: Patch Available) > Flaky test: testNoTreesRetainedAfterDifference - > org.apache.cassandra.repair.RepairJobTest > -- > > Key: CASSANDRA-15872 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15872 > Project: Cassandra > Issue Type: Task > Components: Consistency/Repair >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Normal > Fix For: 4.0-rc > > > link: [https://circleci.com/gh/maedhroz/cassandra/21#tests/containers/95] > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139545#comment-17139545
]
Ekaterina Dimitrova edited comment on CASSANDRA-15234 at 6/18/20, 4:55 PM:
---
[~dcapwell], I am returning this to "in progress" as things have changed and to
prevent people from confusion. Your concerns are valid and addressed. Change
has happened because of some misunderstanding on my end but it was corrected
already. New submission to follow soon:
* custom types (as per the initial agreement)
* annotations as suggested by you for easier maintenance
* backward compatibility will be on property level instead of yaml file level
New branch will be shared soon. Thank you!
was (Author: e.dimitrova):
[~dcapwell], I am returning this to "in progress" as things have changed and to
prevent people from confusion. Your concerns are valid and addressed. Change
has happened because of some misunderstanding on my end but it was corrected.
New submission to follow soon:
* custom types (as per the initial agreement)
* annotations as suggested by you for easier maintenance
* backward compatibility will be on property level instead of yaml file level
New branch will be shared soon. Thank you!
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15234:
Status: In Progress (was: Patch Available)
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139545#comment-17139545
]
Ekaterina Dimitrova edited comment on CASSANDRA-15234 at 6/18/20, 4:54 PM:
---
[~dcapwell], I am returning this to "in progress" as things have changed and to
prevent people from confusion. Your concerns are valid and addressed. Change
has happened because of some misunderstanding on my end but it was corrected.
New submission to follow soon:
* custom types (as per the initial agreement)
* annotations as suggested by you for easier maintenance
* backward compatibility will be on property level instead of yaml file level
New branch will be shared soon. Thank you!
was (Author: e.dimitrova):
[~dcapwell], I am returning this to "in progress" as things have changed. Your
concerns are valid and addressed. Change has happened because of some
misunderstanding on my end but it was corrected. New submission to follow soon:
* custom types (as per the initial agreement)
* annotations as suggested by you for easier maintenance
* backward compatibility will be on property level instead of yaml file level
New branch will be shared soon. Thank you!
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139545#comment-17139545
]
Ekaterina Dimitrova edited comment on CASSANDRA-15234 at 6/18/20, 4:53 PM:
---
[~dcapwell], I am returning this to "in progress" as things have changed. Your
concerns are valid and addressed. Change has happened because of some
misunderstanding on my end but it was corrected. New submission to follow soon:
* custom types (as per the initial agreement)
* annotations as suggested by you for easier maintenance
* backward compatibility will be on property level instead of yaml file level
New branch will be shared soon. Thank you!
was (Author: e.dimitrova):
Still working on a couple of changes after review and discussions happened
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15234:
Status: Patch Available (was: Review In Progress)
Still working on a couple of changes after review and discussions happened
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14608) Confirm correctness of windows scripts post-9608
[
https://issues.apache.org/jira/browse/CASSANDRA-14608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139542#comment-17139542
]
João Reis commented on CASSANDRA-14608:
---
Running cassandra 4.0-alpha4 with ccm on windows returns this output but
cassandra starts just fine:
{code:java}
12:52:01,943 ccm INFO Starting node1 with JAVA_HOME=C:\Program
Files\Java\jdk1.8.0 java_version=8 cassandra_version=4.0-alpha4,
install_dir=C:\Users\appveyor\.ccm\repository\4.0-alpha4
[node1 ERROR] Get-Content : Cannot find path
'C:\Users\appveyor\.ccm\test_f4cd10b2782a\node1\conf\jvm.options' because it
does not
exist.
At C:\Users\appveyor\.ccm\test_f4cd10b2782a\node1\conf\cassandra-env.ps1:305
char:16{code}
I also agree with supporting Windows for dev environments but our
recommendation should be to use WSL2 or docker before attempting to run it on
native Windows.
> Confirm correctness of windows scripts post-9608
>
>
> Key: CASSANDRA-14608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14608
> Project: Cassandra
> Issue Type: Task
> Environment: Windows
>Reporter: Jason Brown
>Priority: Urgent
>
> In CASSANDRA-9608, we chose to defer making all the changes to Windows
> scripts. This ticket is to ensure that we do that work.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] 01/01: Fix describe tests
This is an automated email from the ASF dual-hosted git repository.
blerer pushed a commit to branch CASSANDRA-14825
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
commit 73be52283dae21141773c05e2cb6e2d5d2aaa2f4
Author: Benjamin Lerer
AuthorDate: Thu Jun 18 17:34:37 2020 +0200
Fix describe tests
---
cqlsh_tests/test_cqlsh.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/cqlsh_tests/test_cqlsh.py b/cqlsh_tests/test_cqlsh.py
index 262df8a..cb0236d 100644
--- a/cqlsh_tests/test_cqlsh.py
+++ b/cqlsh_tests/test_cqlsh.py
@@ -1014,6 +1014,7 @@ CREATE TYPE test.address_type (
AND additional_write_policy = '99p'
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
+AND cdc = false
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '16', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
@@ -1205,6 +1206,7 @@ CREATE TYPE test.address_type (
AND additional_write_policy = '99p'
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
+AND cdc = false
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '16', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch CASSANDRA-14825 created (now 73be522)
This is an automated email from the ASF dual-hosted git repository. blerer pushed a change to branch CASSANDRA-14825 in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git. at 73be522 Fix describe tests This branch includes the following new commits: new 73be522 Fix describe tests The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15881) Flaky unit test: SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex
[
https://issues.apache.org/jira/browse/CASSANDRA-15881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139484#comment-17139484
]
Caleb Rackliffe commented on CASSANDRA-15881:
-
[~adelapena] For what it's worth, I saw both fail
[here|https://app.circleci.com/pipelines/github/maedhroz/cassandra/9/workflows/14c42f46-bb04-488f-a2d8-2718d600b848/jobs/90]
as well.
> Flaky unit test: SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex
> ---
>
> Key: CASSANDRA-15881
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15881
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Andres de la Peña
>Priority: Normal
> Fix For: 3.11.7
>
>
> The unit test {{SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex}}
> seems to be flaky in 3.11, as it can be seen in
> [cassandra-ci|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testInsertingIncorrectValuesIntoAgeIndex/]
> and also locally. Trunk doesn't seem to be affected.
> {{SASIIndexTest.testIndexMemtableSwitching}} is [also
> flaky|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testIndexMemtableSwitching/],
> although I haven't been able to reproduce it running it separately, but only
> when running the entire {{SASIIndexTest}}.
> These could have been introduced when merging up CASSANDRA-15778, since they
> failed for their [CircleCI
> run|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra/17/workflows/0442eebe-c764-41c5-ba06-6617bcb9fc5f/jobs/2213],
> and they didn't seem to fail before that, or at least I cannot reproduce
> them locally with the previous commit.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15879) Flaky unit test: BlacklistingCompactionsTest.testBlacklistingWithSizeTieredCompactionStrategy
[
https://issues.apache.org/jira/browse/CASSANDRA-15879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-15879:
Status: Ready to Commit (was: Review In Progress)
[~marcuse][~djoshi] Thanks for the review. Let me know if you need anything
else before committing.
> Flaky unit test:
> BlacklistingCompactionsTest.testBlacklistingWithSizeTieredCompactionStrategy
> -
>
> Key: CASSANDRA-15879
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15879
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> CASSANDRA-14238 addressed the failure in
> {{BlacklistingCompactionsTest.testBlacklistingWithSizeTieredCompactionStrategy}},
> but only on 2.2. While working on CASSANDRA-14888, we’ve reproduced [the
> failure|https://app.circleci.com/pipelines/github/dineshjoshi/cassandra/47/workflows/de5f7cdb-06b6-4869-9d19-81a145e79f3f/jobs/2516/tests]
> on trunk.
> It looks like this should be a clean merge forward.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15879) Flaky unit test: BlacklistingCompactionsTest.testBlacklistingWithSizeTieredCompactionStrategy
[
https://issues.apache.org/jira/browse/CASSANDRA-15879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17138694#comment-17138694
]
Caleb Rackliffe edited comment on CASSANDRA-15879 at 6/18/20, 2:52 PM:
---
{{SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex}} is already failing
in {{cassandra-3.11}}, as reported by CASSANDRA-15881.
was (Author: maedhroz):
{{SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex}} is already failing
in {{cassandra-3.11}} #justfyi
> Flaky unit test:
> BlacklistingCompactionsTest.testBlacklistingWithSizeTieredCompactionStrategy
> -
>
> Key: CASSANDRA-15879
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15879
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> CASSANDRA-14238 addressed the failure in
> {{BlacklistingCompactionsTest.testBlacklistingWithSizeTieredCompactionStrategy}},
> but only on 2.2. While working on CASSANDRA-14888, we’ve reproduced [the
> failure|https://app.circleci.com/pipelines/github/dineshjoshi/cassandra/47/workflows/de5f7cdb-06b6-4869-9d19-81a145e79f3f/jobs/2516/tests]
> on trunk.
> It looks like this should be a clean merge forward.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15873) Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues)
[ https://issues.apache.org/jira/browse/CASSANDRA-15873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139461#comment-17139461 ] Matt Davis commented on CASSANDRA-15873: [https://github.com/apache/cassandra/pull/634] > Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues) > --- > > Key: CASSANDRA-15873 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15873 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Matt Davis >Assignee: Matt Davis >Priority: Normal > Fix For: 3.11.x > > Attachments: dependency-check-report.html > > > See https://issues.apache.org/jira/browse/CASSANDRA-15868 for the same issue > on 4.0 / trunk. Attached is an OWASP dependency report for Netty 4.0.44, > which identifies 3 of the same vulnerabilities as above. > > Additionally, 4.1.50 contains aarch64 native libraries which can improve > performance on ARM processors. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15873) Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues)
[ https://issues.apache.org/jira/browse/CASSANDRA-15873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Davis updated CASSANDRA-15873: --- Test and Documentation Plan: TBD Status: Patch Available (was: Open) > Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues) > --- > > Key: CASSANDRA-15873 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15873 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Matt Davis >Assignee: Matt Davis >Priority: Normal > Fix For: 3.11.x > > Attachments: dependency-check-report.html > > > See https://issues.apache.org/jira/browse/CASSANDRA-15868 for the same issue > on 4.0 / trunk. Attached is an OWASP dependency report for Netty 4.0.44, > which identifies 3 of the same vulnerabilities as above. > > Additionally, 4.1.50 contains aarch64 native libraries which can improve > performance on ARM processors. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15873) Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues)
[ https://issues.apache.org/jira/browse/CASSANDRA-15873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Davis reassigned CASSANDRA-15873: -- Assignee: Matt Davis > Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues) > --- > > Key: CASSANDRA-15873 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15873 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Matt Davis >Assignee: Matt Davis >Priority: Normal > Fix For: 3.11.x > > Attachments: dependency-check-report.html > > > See https://issues.apache.org/jira/browse/CASSANDRA-15868 for the same issue > on 4.0 / trunk. Attached is an OWASP dependency report for Netty 4.0.44, > which identifies 3 of the same vulnerabilities as above. > > Additionally, 4.1.50 contains aarch64 native libraries which can improve > performance on ARM processors. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15873) Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues)
[ https://issues.apache.org/jira/browse/CASSANDRA-15873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Davis updated CASSANDRA-15873: --- Complexity: Low Hanging Fruit Description: See https://issues.apache.org/jira/browse/CASSANDRA-15868 for the same issue on 4.0 / trunk. Attached is an OWASP dependency report for Netty 4.0.44, which identifies 3 of the same vulnerabilities as above. Additionally, 4.1.50 contains aarch64 native libraries which can improve performance on ARM processors. was: See https://issues.apache.org/jira/browse/CASSANDRA-15868 for the same issue on 4.0 / trunk. Attached is an OWASP dependency report for Netty 4.0.44, which identifies 3 of the same vulnerabilities as above. Additionally, 4.1.50 contains aarch64 native libraries which can improve performance on ARM processors. (If the preference is to handle PRs for both versions/branches in a single issue, feel free to close this as a duplicate.) > Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues) > --- > > Key: CASSANDRA-15873 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15873 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Matt Davis >Priority: Normal > Fix For: 3.11.x > > Attachments: dependency-check-report.html > > > See https://issues.apache.org/jira/browse/CASSANDRA-15868 for the same issue > on 4.0 / trunk. Attached is an OWASP dependency report for Netty 4.0.44, > which identifies 3 of the same vulnerabilities as above. > > Additionally, 4.1.50 contains aarch64 native libraries which can improve > performance on ARM processors. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15873) Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues)
[ https://issues.apache.org/jira/browse/CASSANDRA-15873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139458#comment-17139458 ] Matt Davis commented on CASSANDRA-15873: I understand the hesitancy. What's the path forward here then? If we can determine what the bar would be to accept this change, I'd be glad to do what is necessary to meet it. My suggestion was to run unit tests and dtests and add the results here. (Running cassandra-stress I've so far seen no issues, we just need to add proof.) > Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues) > --- > > Key: CASSANDRA-15873 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15873 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Matt Davis >Priority: Normal > Fix For: 3.11.x > > Attachments: dependency-check-report.html > > > See https://issues.apache.org/jira/browse/CASSANDRA-15868 for the same issue > on 4.0 / trunk. Attached is an OWASP dependency report for Netty 4.0.44, > which identifies 3 of the same vulnerabilities as above. > > Additionally, 4.1.50 contains aarch64 native libraries which can improve > performance on ARM processors. > > (If the preference is to handle PRs for both versions/branches in a single > issue, feel free to close this as a duplicate.) > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15677) Topology events are not sent to clients if the nodes use the same network interface
[
https://issues.apache.org/jira/browse/CASSANDRA-15677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139261#comment-17139261
]
Bryn Cooke edited comment on CASSANDRA-15677 at 6/18/20, 2:05 PM:
--
-This comment seems to indicate that the existing behaviour was a workaround to
the drivers not distinguishing nodes by address and socket:
https://issues.apache.org/jira/browse/CASSANDRA-10052?focusedCommentId=14725534=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14725534-
-It looks like the Java driver no longer has this issue, at least I don't see
any calls to Host#getListenAddress or Host#getBroadcastAddress. Everything
seems to be using address and port.-
[~aboudreault] -how do things look like on the other drivers?-
-Edit:-
-Just had a look at the python driver and Endpoint was added to the Python
driver here:
[https://github.com/datastax/python-driver/blob/c88255f202a21bbbae35f16e603b0f10f2f2cf36/cassandra/connection.py#L116]-
-Node.js has Host support here:
[https://github.com/datastax/nodejs-driver/blob/master/lib/host.js#L41]-
-C# has
[https://github.com/datastax/csharp-driver/blob/master/src/Cassandra/Host.cs#L78]-
-CPP has [https://github.com/datastax/cpp-driver/blob/master/src/host.hpp#L92]-
-So I think all of the drivers can now distinguish between nodes based on node
and port. But it would probably be best for one of the driver devs to weigh in.-
Ignore this. I misread Tyler's comment.
was (Author: bryncooke):
This comment seems to indicate that the existing behaviour was a workaround to
the drivers not distinguishing nodes by address and socket:
https://issues.apache.org/jira/browse/CASSANDRA-10052?focusedCommentId=14725534=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14725534
It looks like the Java driver no longer has this issue, at least I don't see
any calls to Host#getListenAddress or Host#getBroadcastAddress. Everything
seems to be using address and port.
[~aboudreault] how do things look like on the other drivers?
Edit:
Just had a look at the python driver and Endpoint was added to the Python
driver here:
[https://github.com/datastax/python-driver/blob/c88255f202a21bbbae35f16e603b0f10f2f2cf36/cassandra/connection.py#L116]
Node.js has Host support here:
[https://github.com/datastax/nodejs-driver/blob/master/lib/host.js#L41]
C# has
[https://github.com/datastax/csharp-driver/blob/master/src/Cassandra/Host.cs#L78]
CPP has [https://github.com/datastax/cpp-driver/blob/master/src/host.hpp#L92]
So I think all of the drivers can now distinguish between nodes based on node
and port. But it would probably be best for one of the driver devs to weigh in.
> Topology events are not sent to clients if the nodes use the same network
> interface
> ---
>
> Key: CASSANDRA-15677
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15677
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client
>Reporter: Alan Boudreault
>Assignee: Bryn Cooke
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha5
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> *This bug only happens when the cassandra nodes are configured to use a
> single network interface (ip) but different ports. See CASSANDRA-7544.*
> Issue: The topology events aren't sent to clients. The problem is that the
> port is not taken into account when determining if we send it or not:
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/transport/Server.java#L624
> To reproduce:
> {code}
> # I think the cassandra-test branch is required to get the -S option
> (USE_SINGLE_INTERFACE)
> ccm create -n4 local40 -v 4.0-alpha2 -S
> {code}
>
> Then run this small python driver script:
> {code}
> import time
> from cassandra.cluster import Cluster
> cluster = Cluster()
> session = cluster.connect()
> while True:
> print(cluster.metadata.all_hosts())
> print([h.is_up for h in cluster.metadata.all_hosts()])
> time.sleep(5)
> {code}
> Then decommission a node:
> {code}
> ccm node2 nodetool disablebinary
> ccm node2 nodetool decommission
> {code}
>
> You should see that the node is never removed from the client cluster
> metadata and the reconnector started.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15868) Update Netty version to 4.1.50 because there are security issues in 4.1.37
[
https://issues.apache.org/jira/browse/CASSANDRA-15868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139442#comment-17139442
]
Matt Davis commented on CASSANDRA-15868:
{quote}I'm hesitant to make a 4.0 -> 4.1 upgrade here for the 3.11 branch, as
4.1 is known to not be fully compatible with 4.0, and we'd have to do quite a
bit of testing to safely upgrade the dependency.
{quote}
[~aleksey] Understandable - if we can determine what the bar is, I'd be glad to
help with testing.
> Update Netty version to 4.1.50 because there are security issues in 4.1.37
> --
>
> Key: CASSANDRA-15868
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15868
> Project: Cassandra
> Issue Type: Task
> Components: Dependencies
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: dependency-check-report.html
>
>
> Please see attached HTML report from OWASP dependency check.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15873) Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues)
[ https://issues.apache.org/jira/browse/CASSANDRA-15873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15873: -- Fix Version/s: 3.11.x > Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues) > --- > > Key: CASSANDRA-15873 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15873 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Matt Davis >Priority: Normal > Fix For: 3.11.x > > Attachments: dependency-check-report.html > > > See https://issues.apache.org/jira/browse/CASSANDRA-15868 for the same issue > on 4.0 / trunk. Attached is an OWASP dependency report for Netty 4.0.44, > which identifies 3 of the same vulnerabilities as above. > > Additionally, 4.1.50 contains aarch64 native libraries which can improve > performance on ARM processors. > > (If the preference is to handle PRs for both versions/branches in a single > issue, feel free to close this as a duplicate.) > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15873) Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues)
[ https://issues.apache.org/jira/browse/CASSANDRA-15873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15873: -- Status: Open (was: Resolved) Reopening this one, as doing a major Netty upgrade in a minor 3.11.x is a more questionable proposition than doing a minor Netty upgrade in a major 4.0 C*. > Update Netty 4.0.44 -> 4.1.50 (fix security/performance issues) > --- > > Key: CASSANDRA-15873 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15873 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Matt Davis >Priority: Normal > Attachments: dependency-check-report.html > > > See https://issues.apache.org/jira/browse/CASSANDRA-15868 for the same issue > on 4.0 / trunk. Attached is an OWASP dependency report for Netty 4.0.44, > which identifies 3 of the same vulnerabilities as above. > > Additionally, 4.1.50 contains aarch64 native libraries which can improve > performance on ARM processors. > > (If the preference is to handle PRs for both versions/branches in a single > issue, feel free to close this as a duplicate.) > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15868) Update Netty version to 4.1.50 because there are security issues in 4.1.37
[ https://issues.apache.org/jira/browse/CASSANDRA-15868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15868: -- Fix Version/s: (was: 3.11.7) Source Control Link: [d31ea25f42eed510a28f4c73355ac0ae7bf17595|https://github.com/apache/cassandra/commit/d31ea25f42eed510a28f4c73355ac0ae7bf17595] Resolution: Fixed Status: Resolved (was: Ready to Commit) > Update Netty version to 4.1.50 because there are security issues in 4.1.37 > -- > > Key: CASSANDRA-15868 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15868 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 4.0-beta > > Attachments: dependency-check-report.html > > > Please see attached HTML report from OWASP dependency check. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15868) Update Netty version to 4.1.50 because there are security issues in 4.1.37
[ https://issues.apache.org/jira/browse/CASSANDRA-15868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15868: -- Status: Ready to Commit (was: Review In Progress) > Update Netty version to 4.1.50 because there are security issues in 4.1.37 > -- > > Key: CASSANDRA-15868 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15868 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.11.7, 4.0-beta > > Attachments: dependency-check-report.html > > > Please see attached HTML report from OWASP dependency check. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15868) Update Netty version to 4.1.50 because there are security issues in 4.1.37
[ https://issues.apache.org/jira/browse/CASSANDRA-15868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139433#comment-17139433 ] Aleksey Yeschenko commented on CASSANDRA-15868: --- Committed to trunk as [d31ea25f42eed510a28f4c73355ac0ae7bf17595|https://github.com/apache/cassandra/commit/d31ea25f42eed510a28f4c73355ac0ae7bf17595]. > Update Netty version to 4.1.50 because there are security issues in 4.1.37 > -- > > Key: CASSANDRA-15868 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15868 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.11.7, 4.0-beta > > Attachments: dependency-check-report.html > > > Please see attached HTML report from OWASP dependency check. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15752) Range read concurrency factor didn't consider range merger
[
https://issues.apache.org/jira/browse/CASSANDRA-15752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139400#comment-17139400
]
ZhaoYang edited comment on CASSANDRA-15752 at 6/18/20, 1:44 PM:
backported to 3.0 and 3.11
|[trunk|https://github.com/apache/cassandra/pull/606/files]|
[circle|https://circleci.com/workflow-run/440126c4-058d-4511-8301-16df2bddf5db]|
|[3.11|https://github.com/apache/cassandra/pull/643/files]|
[circle|https://circleci.com/workflow-run/000e8712-3259-42c1-a2b3-20ff166fe5b4]
|
|[3.0|https://github.com/apache/cassandra/pull/644] |
[circle|https://circleci.com/workflow-run/3714cc8b-c0ef-4974-9b6a-d8ea815aaaea]
|
was (Author: jasonstack):
backported to 3.0 and 3.11
|[trunk|https://github.com/apache/cassandra/pull/606/files]|
[circle|https://circleci.com/workflow-run/440126c4-058d-4511-8301-16df2bddf5db]|
|[3.11|https://github.com/apache/cassandra/pull/643/files]| CI-pending |
|[3.0|https://github.com/apache/cassandra/pull/644] | CI-pending |
> Range read concurrency factor didn't consider range merger
> --
>
> Key: CASSANDRA-15752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15752
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Coordination
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> During range read, coordinator computes concurrency factor which is the
> number of vnode ranges to contact in parallel for the next batch.
> But in {{RangeCommandIterator}}, vnode ranges are merged by {{RangeMerger}}
> if vnode ranges share enough replicas to satisfy consistency level. eg. vnode
> range [a,b) has replica n1,n2,n3 and vnode range [b,c) has replica n2,n3,n4,
> so they can be merged as range [a,c) with replica n2, n3 for Quorum.
> Currently it counts number of merged ranges towards concurrency factor.
> Coordinator may fetch more ranges than needed.
>
> Another issue is that when executing range read on table with very small
> amount of data, concurrency factor can be bumped to {{size of total vnode
> ranges}}, eg. 10k, depending on the num of vnodes and cluster size. As a
> result, coordinator will send large number of concurrent range requests,
> potentially slowing down the cluster.. We should cap the max concurrency
> factor..
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Update Netty dependencies to latest
This is an automated email from the ASF dual-hosted git repository.
aleksey pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new d31ea25 Update Netty dependencies to latest
d31ea25 is described below
commit d31ea25f42eed510a28f4c73355ac0ae7bf17595
Author: Stefan Miklosovic
AuthorDate: Wed Jun 3 22:00:59 2020 +0200
Update Netty dependencies to latest
patch by Stefan Miklosovic; reviewed by Aleksey Yeschenko for
CASSANDRA-15868
---
build.xml | 4 ++--
.../{netty-4.1.37.txt => netty-4.1.50.txt} | 0
...native-2.0.25.txt => netty-tcnative-2.0.31.txt} | 0
...4.1.37.Final.jar => netty-all-4.1.50.Final.jar} | Bin 4024948 -> 4213710
bytes
...etty-tcnative-boringssl-static-2.0.25.Final.jar | Bin 3108312 -> 0 bytes
...etty-tcnative-boringssl-static-2.0.31.Final.jar | Bin 0 -> 3953120 bytes
6 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/build.xml b/build.xml
index 495d3d3..2d10819 100644
--- a/build.xml
+++ b/build.xml
@@ -586,8 +586,8 @@
-
-
+
+
diff --git a/lib/licenses/netty-4.1.37.txt b/lib/licenses/netty-4.1.50.txt
similarity index 100%
rename from lib/licenses/netty-4.1.37.txt
rename to lib/licenses/netty-4.1.50.txt
diff --git a/lib/licenses/netty-tcnative-2.0.25.txt
b/lib/licenses/netty-tcnative-2.0.31.txt
similarity index 100%
rename from lib/licenses/netty-tcnative-2.0.25.txt
rename to lib/licenses/netty-tcnative-2.0.31.txt
diff --git a/lib/netty-all-4.1.37.Final.jar b/lib/netty-all-4.1.50.Final.jar
similarity index 51%
rename from lib/netty-all-4.1.37.Final.jar
rename to lib/netty-all-4.1.50.Final.jar
index 93cff04..f8b1557 100644
Binary files a/lib/netty-all-4.1.37.Final.jar and
b/lib/netty-all-4.1.50.Final.jar differ
diff --git a/lib/netty-tcnative-boringssl-static-2.0.25.Final.jar
b/lib/netty-tcnative-boringssl-static-2.0.25.Final.jar
deleted file mode 100644
index 954627f..000
Binary files a/lib/netty-tcnative-boringssl-static-2.0.25.Final.jar and
/dev/null differ
diff --git a/lib/netty-tcnative-boringssl-static-2.0.31.Final.jar
b/lib/netty-tcnative-boringssl-static-2.0.31.Final.jar
new file mode 100644
index 000..582c582
Binary files /dev/null and
b/lib/netty-tcnative-boringssl-static-2.0.31.Final.jar differ
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15861) Mutating sstable component may race with entire-sstable-streaming(ZCS) causing checksum validation failure
[
https://issues.apache.org/jira/browse/CASSANDRA-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang updated CASSANDRA-15861:
-
Summary: Mutating sstable component may race with
entire-sstable-streaming(ZCS) causing checksum validation failure (was:
Mutating sstable STATS metadata may race with entire-sstable-streaming(ZCS)
causing checksum validation failure)
> Mutating sstable component may race with entire-sstable-streaming(ZCS)
> causing checksum validation failure
> --
>
> Key: CASSANDRA-15861
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15861
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair, Consistency/Streaming,
> Local/Compaction
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Flaky dtest: [test_dead_sync_initiator -
> repair_tests.repair_test.TestRepair|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/143/testReport/junit/dtest.repair_tests.repair_test/TestRepair/test_dead_sync_initiator/]
> {code:java|title=stacktrace}
> Unexpected error found in node logs (see stdout for full details). Errors:
> [ERROR [Stream-Deserializer-127.0.0.1:7000-570871f3] 2020-06-03 04:05:19,081
> CassandraEntireSSTableStreamReader.java:145 - [Stream
> 6f1c3360-a54f-11ea-a808-2f23710fdc90] Error while reading sstable from stream
> for table = keyspace1.standard1
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted:
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.maybeValidateChecksum(MetadataSerializer.java:219)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:198)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:129)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.mutate(MetadataSerializer.java:226)
> at
> org.apache.cassandra.db.streaming.CassandraEntireSSTableStreamReader.read(CassandraEntireSSTableStreamReader.java:140)
> at
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:78)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)
> at
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:181)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Checksums do not match for
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
> {code}
>
> In the above test, it executes "nodetool repair" on node1 and kills node2
> during repair. At the end, node3 reports checksum validation failure on
> sstable transferred from node1.
> {code:java|title=what happened}
> 1. When repair started on node1, it performs anti-compaction which modifies
> sstable's repairAt to 0 and pending repair id to session-id.
> 2. Then node1 creates {{ComponentManifest}} which contains file lengths to be
> transferred to node3.
> 3. Before node1 actually sends the files to node3, node2 is killed and node1
> starts to broadcast repair-failure-message to all participants in
> {{CoordinatorSession#fail}}
> 4. Node1 receives its own repair-failure-message and fails its local repair
> sessions at {{LocalSessions#failSession}} which triggers async background
> compaction.
> 5. Node1's background compaction will mutate sstable's repairAt to 0 and
> pending repair id to null via
> {{PendingRepairManager#getNextRepairFinishedTask}}, as there is no more
> in-progress repair.
> 6. Node1 actually sends the sstable to node3 where the sstable's STATS
> component size is different from the original size recorded in the manifest.
> 7. At the end, node3 reports checksum validation failure when it tries to
> mutate sstable level and "isTransient" attribute in
> {{CassandraEntireSSTableStreamReader#read}}.
> {code}
> This isn't a problem in legacy streaming as STATS file length didn't matter.
> Ideally it will be great to make sstable STATS metadata immutable, just like
> other sstable
[jira] [Commented] (CASSANDRA-15868) Update Netty version to 4.1.50 because there are security issues in 4.1.37
[
https://issues.apache.org/jira/browse/CASSANDRA-15868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139422#comment-17139422
]
Aleksey Yeschenko commented on CASSANDRA-15868:
---
Hey Matt. To answer your questions from Slack:
{quote}
Do you typically accept these type of dependency updates?
https://issues.apache.org/jira/browse/CASSANDRA-14473 suggests you do not,
though there are legitimate vulnerabilities in this case.
Is the minor version bump concerning to anyone? We have heard Netty may not
guarantee API compatibility across minor versions.
Would adding test results to the PR (unit tests, or running
https://github.com/apache/cassandra-dtest) help?
{quote}
I'm hesitant to make a 4.0 -> 4.1 upgrade here for the 3.11 branch, as 4.1 is
known to not be fully compatible with 4.0, and we'd have to do quite a bit of
testing to safely upgrade the dependency.
bq. Any idea what kind of ETA we might expect for this type of issue / PR?
I'll commit the 4.0 branch version right now, but hold off upgrade the 3.11
branch. The bar for that would be higher, though we've never explicitly defined
it.
> Update Netty version to 4.1.50 because there are security issues in 4.1.37
> --
>
> Key: CASSANDRA-15868
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15868
> Project: Cassandra
> Issue Type: Task
> Components: Dependencies
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.11.7, 4.0-beta
>
> Attachments: dependency-check-report.html
>
>
> Please see attached HTML report from OWASP dependency check.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15814) order by descending on frozen list not working
[
https://issues.apache.org/jira/browse/CASSANDRA-15814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña reassigned CASSANDRA-15814:
-
Assignee: Andres de la Peña
> order by descending on frozen list not working
> --
>
> Key: CASSANDRA-15814
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15814
> Project: Cassandra
> Issue Type: Bug
>Reporter: Felipe Perez
>Assignee: Andres de la Peña
>Priority: Normal
>
> By creating a table like the following:
> {code:java}
> CREATE TABLE IF NOT EXISTS software (
> name ascii,
> version frozen>,
> data ascii,
> PRIMARY KEY(name,version)
> )
> {code}
> It works and version is ordered in an ascending order. But when trying to
> order in descending order:
> {code:java}
> CREATE TABLE IF NOT EXISTS software (
> name ascii,
> version frozen>,
> data ascii,
> PRIMARY KEY(name,version)
> ) WITH CLUSTERING ORDER BY (version DESC);
> {code}
> The table is created normally, but when trying to insert a row:
> {code:java}
> insert into software(name, version) values ('t1', [2,10,30,40,50]);
> {code}
> Cassandra throws an error:
> {code:java}
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid
> list literal for version of type frozen>"
> {code}
> The goal here is that I would like to get the last version of a software.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15500) Slow query logging emits incomplete predicates
[ https://issues.apache.org/jira/browse/CASSANDRA-15500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139407#comment-17139407 ] Andres de la Peña commented on CASSANDRA-15500: --- Closing since the fix for CASSANDRA-15503 includes a fix for this, and it's new tests cover the described case. > Slow query logging emits incomplete predicates > -- > > Key: CASSANDRA-15500 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15500 > Project: Cassandra > Issue Type: Bug > Components: Observability/Logging >Reporter: onmstester >Priority: Normal > > Using Apache Cassandra 3.11.2, defined a table like this: > > _create table my_table(_ > __ _partition text,_ > __ _clustering1 int,_ > _clustering2 text,_ > _data set,_ > ** _*primary key (partition, clustering1, > clustering2))*_ > > and configured slow queries threshold to 1ms in yaml to see how queries > passed to cassandra. Query below: > > _select * from my_table where partition='a' and clustering1= 1 and > clustering2='b'_ > > would be like this in debug.log of cassandra: > > _select * from my_table where partition='a' LIMIT 100> (it means that the > two cluster key restriction did not push down to storage engine and the whole > partition been retrieved)_ > > but this query: > > _select * from my_table where partition='a' and clustering1= 1_ > > _would be_ > > _select * from my_table where partition='a' and_ _clustering1= 1_ _LIMIT 100> > (single cluster key been pushed down to storage engine)_ > > > _So it seems to me that, we could not restrict multiple clustering keys in > select because it would retrieve the whole partition ?!_ -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15500) Slow query logging emits incomplete predicates
[ https://issues.apache.org/jira/browse/CASSANDRA-15500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-15500: -- Resolution: Duplicate Status: Resolved (was: Open) > Slow query logging emits incomplete predicates > -- > > Key: CASSANDRA-15500 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15500 > Project: Cassandra > Issue Type: Bug > Components: Observability/Logging >Reporter: onmstester >Priority: Normal > > Using Apache Cassandra 3.11.2, defined a table like this: > > _create table my_table(_ > __ _partition text,_ > __ _clustering1 int,_ > _clustering2 text,_ > _data set,_ > ** _*primary key (partition, clustering1, > clustering2))*_ > > and configured slow queries threshold to 1ms in yaml to see how queries > passed to cassandra. Query below: > > _select * from my_table where partition='a' and clustering1= 1 and > clustering2='b'_ > > would be like this in debug.log of cassandra: > > _select * from my_table where partition='a' LIMIT 100> (it means that the > two cluster key restriction did not push down to storage engine and the whole > partition been retrieved)_ > > but this query: > > _select * from my_table where partition='a' and clustering1= 1_ > > _would be_ > > _select * from my_table where partition='a' and_ _clustering1= 1_ _LIMIT 100> > (single cluster key been pushed down to storage engine)_ > > > _So it seems to me that, we could not restrict multiple clustering keys in > select because it would retrieve the whole partition ?!_ -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15882) sstablemetadata should show tombstone timestamp not just the local drop time
[
https://issues.apache.org/jira/browse/CASSANDRA-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thanh updated CASSANDRA-15882:
--
Description:
issue/request:
sstablemetadata should show tombstone timestamp not just the local drop time
I'm not sure what all versions this exists in but the sstablemetadata
tombstone "drop time" only just shows the local delete time of the tombstones.
What's more important (what the tombstone expiration time depends on) is the
tombstone writetime (tombstone timestamp). It's possible to, and a surprising
number of users have done this, write a tombstone with a timetamp in the future
(by doing "DELETE...USING TIMESTAMP "). When this happens,
the tombstone hangs around in sstables for a lot longer than users expect
because
{code:java}
tombstone_expiration_time = gc_grace_seconds + tombstone_timestamp{code}
NOT
{code:java}
tombstone_expiration_time = gc_grace_seconds +
tombstone_local_delete_time{code}
and the sstablemetadata output doesn't help here because it only shows the
local delete time, not the actual tombstone timestamp that's used to calculate
tombstone expiration time.
was:
issue/request:
sstablemetadata should show tombstone timestamp not just the local drop time
I'm not sure what all versions this exists in but the sstablemetadata
tombstone "drop time" only just shows the local delete time of the tombstones.
What's more important (what the tombstone expiration time depends on) is the
tombstone writetime (tombstone timestamp). It's possible to, and a surprising
number of users have done this, write a tombstone with a timetamp in the future
(by doing "DELETE...USING TIMESTAMP "). When this happens,
the tombstone hangs around in sstables for a lot longer than users expect
because
{code:java}
tombstone_expiration_time = gc_grace_seconds + tombstone_timestamp{code}
NOT
{code:java}
tombstone_expiration_time = gc_grace_seconds +
tombstone_local_delete_time{code}
> sstablemetadata should show tombstone timestamp not just the local drop time
>
>
> Key: CASSANDRA-15882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15882
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Thanh
>Priority: Normal
>
> issue/request:
> sstablemetadata should show tombstone timestamp not just the local drop time
>
> I'm not sure what all versions this exists in but the sstablemetadata
> tombstone "drop time" only just shows the local delete time of the
> tombstones. What's more important (what the tombstone expiration time
> depends on) is the tombstone writetime (tombstone timestamp). It's possible
> to, and a surprising number of users have done this, write a tombstone with a
> timetamp in the future (by doing "DELETE...USING TIMESTAMP
> "). When this happens, the tombstone hangs around in
> sstables for a lot longer than users expect because
> {code:java}
> tombstone_expiration_time = gc_grace_seconds + tombstone_timestamp{code}
> NOT
> {code:java}
> tombstone_expiration_time = gc_grace_seconds +
> tombstone_local_delete_time{code}
> and the sstablemetadata output doesn't help here because it only shows the
> local delete time, not the actual tombstone timestamp that's used to
> calculate tombstone expiration time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15882) sstablemetadata should show tombstone timestamp not just the local drop time
Thanh created CASSANDRA-15882:
-
Summary: sstablemetadata should show tombstone timestamp not just
the local drop time
Key: CASSANDRA-15882
URL: https://issues.apache.org/jira/browse/CASSANDRA-15882
Project: Cassandra
Issue Type: Improvement
Reporter: Thanh
issue/request:
sstablemetadata should show tombstone timestamp not just the local drop time
I'm not sure what all versions this exists in but the sstablemetadata
tombstone "drop time" only just shows the local delete time of the tombstones.
What's more important (what the tombstone expiration time depends on) is the
tombstone writetime (tombstone timestamp). It's possible to, and a surprising
number of users have done this, write a tombstone with a timetamp in the future
(by doing "DELETE...USING TIMESTAMP "). When this happens,
the tombstone hangs around in sstables for a lot longer than users expect
because
{code:java}
tombstone_expiration_time = gc_grace_seconds + tombstone_timestamp{code}
NOT
{code:java}
tombstone_expiration_time = gc_grace_seconds +
tombstone_local_delete_time{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15881) Flaky unit test: SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex
[
https://issues.apache.org/jira/browse/CASSANDRA-15881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-15881:
--
Bug Category: Parent values: Correctness(12982)Level 1 values: Test
Failure(12990)
Discovered By: Unit Test
Since Version: 3.11.7
> Flaky unit test: SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex
> ---
>
> Key: CASSANDRA-15881
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15881
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Andres de la Peña
>Priority: Normal
> Fix For: 3.11.7
>
>
> The unit test {{SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex}}
> seems to be flaky in 3.11, as it can be seen in
> [cassandra-ci|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testInsertingIncorrectValuesIntoAgeIndex/]
> and also locally. Trunk doesn't seem to be affected.
> {{SASIIndexTest.testIndexMemtableSwitching}} is [also
> flaky|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testIndexMemtableSwitching/],
> although I haven't been able to reproduce it running it separately, but only
> when running the entire {{SASIIndexTest}}.
> These could have been introduced when merging up CASSANDRA-15778, since they
> failed for their [CircleCI
> run|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra/17/workflows/0442eebe-c764-41c5-ba06-6617bcb9fc5f/jobs/2213],
> and they didn't seem to fail before that, or at least I cannot reproduce
> them locally with the previous commit.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15881) Flaky unit test: SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex
[
https://issues.apache.org/jira/browse/CASSANDRA-15881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-15881:
--
Description:
The unit test {{SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex}} seems
to be flaky in 3.11, as it can be seen in
[cassandra-ci|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testInsertingIncorrectValuesIntoAgeIndex/]
and also locally. Trunk doesn't seem to be affected.
{{SASIIndexTest.testIndexMemtableSwitching}} is [also
flaky|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testIndexMemtableSwitching/],
although I haven't been able to reproduce it running it separately, but only
when running the entire {{SASIIndexTest}}.
These could have been introduced when merging up CASSANDRA-15778, since they
failed for their [CircleCI
run|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra/17/workflows/0442eebe-c764-41c5-ba06-6617bcb9fc5f/jobs/2213],
and they didn't seem to fail before that, or at least I cannot reproduce them
locally with the previous commit.
was:
The unit test {{SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex}} seems
to be flaky in 3.11, as it can be seen in
[cassandra-ci|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testInsertingIncorrectValuesIntoAgeIndex/]
and also locally. Trunk doesn't seem to be affected.
{{SASIIndexTest.testIndexMemtableSwitching}} is [also
flaky|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testIndexMemtableSwitching/],
although I haven't been to reproduce it running it separately, but only when
running the entire {{SASIIndexTest}}.
These could have been introduced when merging up CASSANDRA-15778, since they
failed for their [CircleCI
run|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra/17/workflows/0442eebe-c764-41c5-ba06-6617bcb9fc5f/jobs/2213],
and they didn't seem to fail before that, or at least I cannot reproduce them
locally with the previous commit.
> Flaky unit test: SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex
> ---
>
> Key: CASSANDRA-15881
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15881
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Andres de la Peña
>Priority: Normal
> Fix For: 3.11.7
>
>
> The unit test {{SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex}}
> seems to be flaky in 3.11, as it can be seen in
> [cassandra-ci|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testInsertingIncorrectValuesIntoAgeIndex/]
> and also locally. Trunk doesn't seem to be affected.
> {{SASIIndexTest.testIndexMemtableSwitching}} is [also
> flaky|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testIndexMemtableSwitching/],
> although I haven't been able to reproduce it running it separately, but only
> when running the entire {{SASIIndexTest}}.
> These could have been introduced when merging up CASSANDRA-15778, since they
> failed for their [CircleCI
> run|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra/17/workflows/0442eebe-c764-41c5-ba06-6617bcb9fc5f/jobs/2213],
> and they didn't seem to fail before that, or at least I cannot reproduce
> them locally with the previous commit.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15752) Range read concurrency factor didn't consider range merger
[
https://issues.apache.org/jira/browse/CASSANDRA-15752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139400#comment-17139400
]
ZhaoYang commented on CASSANDRA-15752:
--
backported to 3.0 and 3.11
|[trunk|https://github.com/apache/cassandra/pull/606/files]|
[circle|https://circleci.com/workflow-run/440126c4-058d-4511-8301-16df2bddf5db]|
|[3.11|https://github.com/apache/cassandra/pull/643/files]| CI-pending |
|[3.0|https://github.com/apache/cassandra/pull/644] | CI-pending |
> Range read concurrency factor didn't consider range merger
> --
>
> Key: CASSANDRA-15752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15752
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Coordination
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> During range read, coordinator computes concurrency factor which is the
> number of vnode ranges to contact in parallel for the next batch.
> But in {{RangeCommandIterator}}, vnode ranges are merged by {{RangeMerger}}
> if vnode ranges share enough replicas to satisfy consistency level. eg. vnode
> range [a,b) has replica n1,n2,n3 and vnode range [b,c) has replica n2,n3,n4,
> so they can be merged as range [a,c) with replica n2, n3 for Quorum.
> Currently it counts number of merged ranges towards concurrency factor.
> Coordinator may fetch more ranges than needed.
>
> Another issue is that when executing range read on table with very small
> amount of data, concurrency factor can be bumped to {{size of total vnode
> ranges}}, eg. 10k, depending on the num of vnodes and cluster size. As a
> result, coordinator will send large number of concurrent range requests,
> potentially slowing down the cluster.. We should cap the max concurrency
> factor..
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15881) Flaky unit test: SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex
[
https://issues.apache.org/jira/browse/CASSANDRA-15881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-15881:
--
Description:
The unit test {{SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex}} seems
to be flaky in 3.11, as it can be seen in
[cassandra-ci|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testInsertingIncorrectValuesIntoAgeIndex/]
and also locally. Trunk doesn't seem to be affected.
{{SASIIndexTest.testIndexMemtableSwitching}} is [also
flaky|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testIndexMemtableSwitching/],
although I haven't been to reproduce it running it separately, but only when
running the entire {{SASIIndexTest}}.
These could have been introduced when merging up CASSANDRA-15778, since they
failed for their [CircleCI
run|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra/17/workflows/0442eebe-c764-41c5-ba06-6617bcb9fc5f/jobs/2213],
and they didn't seem to fail before that, or at least I cannot reproduce them
locally with the previous commit.
was:
The unit test {{SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex}} seems
to be flaky in 3.11, as it can be seen in
[cassandra-ci|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testInsertingIncorrectValuesIntoAgeIndex/]
and also locally.
{{SASIIndexTest.testIndexMemtableSwitching}} is [also
flaky|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testIndexMemtableSwitching/],
although I haven't been to reproduce it running it separately, but only when
running the entire {{SASIIndexTest}}.
These could have been introduced when merging up CASSANDRA-15778, since they
failed for their [CircleCI
run|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra/17/workflows/0442eebe-c764-41c5-ba06-6617bcb9fc5f/jobs/2213],
and they didn't seem to fail before that, or at least I cannot reproduce them
locally with the previous commit.
> Flaky unit test: SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex
> ---
>
> Key: CASSANDRA-15881
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15881
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Andres de la Peña
>Priority: Normal
> Fix For: 3.11.7
>
>
> The unit test {{SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex}}
> seems to be flaky in 3.11, as it can be seen in
> [cassandra-ci|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testInsertingIncorrectValuesIntoAgeIndex/]
> and also locally. Trunk doesn't seem to be affected.
> {{SASIIndexTest.testIndexMemtableSwitching}} is [also
> flaky|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testIndexMemtableSwitching/],
> although I haven't been to reproduce it running it separately, but only when
> running the entire {{SASIIndexTest}}.
> These could have been introduced when merging up CASSANDRA-15778, since they
> failed for their [CircleCI
> run|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra/17/workflows/0442eebe-c764-41c5-ba06-6617bcb9fc5f/jobs/2213],
> and they didn't seem to fail before that, or at least I cannot reproduce
> them locally with the previous commit.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15881) Flaky unit test: SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex
Andres de la Peña created CASSANDRA-15881:
-
Summary: Flaky unit test:
SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex
Key: CASSANDRA-15881
URL: https://issues.apache.org/jira/browse/CASSANDRA-15881
Project: Cassandra
Issue Type: Bug
Components: Test/unit
Reporter: Andres de la Peña
The unit test {{SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex}} seems
to be flaky in 3.11, as it can be seen in
[cassandra-ci|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testInsertingIncorrectValuesIntoAgeIndex/]
and also locally.
{{SASIIndexTest.testIndexMemtableSwitching}} is [also
flaky|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testIndexMemtableSwitching/],
although I haven't been to reproduce it running it separately, but only when
running the entire {{SASIIndexTest}}.
These could have been introduced when merging up CASSANDRA-15778, since they
failed for their [CircleCI
run|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra/17/workflows/0442eebe-c764-41c5-ba06-6617bcb9fc5f/jobs/2213],
and they didn't seem to fail before that, or at least I cannot reproduce them
locally with the previous commit.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15881) Flaky unit test: SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex
[
https://issues.apache.org/jira/browse/CASSANDRA-15881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-15881:
--
Fix Version/s: 3.11.7
> Flaky unit test: SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex
> ---
>
> Key: CASSANDRA-15881
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15881
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Andres de la Peña
>Priority: Normal
> Fix For: 3.11.7
>
>
> The unit test {{SASIIndexTest.testInsertingIncorrectValuesIntoAgeIndex}}
> seems to be flaky in 3.11, as it can be seen in
> [cassandra-ci|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testInsertingIncorrectValuesIntoAgeIndex/]
> and also locally.
> {{SASIIndexTest.testIndexMemtableSwitching}} is [also
> flaky|https://ci-cassandra.apache.org/view/Cassandra%203.11/job/Cassandra-3.11-test/42/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/testIndexMemtableSwitching/],
> although I haven't been to reproduce it running it separately, but only when
> running the entire {{SASIIndexTest}}.
> These could have been introduced when merging up CASSANDRA-15778, since they
> failed for their [CircleCI
> run|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra/17/workflows/0442eebe-c764-41c5-ba06-6617bcb9fc5f/jobs/2213],
> and they didn't seem to fail before that, or at least I cannot reproduce
> them locally with the previous commit.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15677) Topology events are not sent to clients if the nodes use the same network interface
[
https://issues.apache.org/jira/browse/CASSANDRA-15677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139261#comment-17139261
]
Bryn Cooke edited comment on CASSANDRA-15677 at 6/18/20, 12:01 PM:
---
This comment seems to indicate that the existing behaviour was a workaround to
the drivers not distinguishing nodes by address and socket:
https://issues.apache.org/jira/browse/CASSANDRA-10052?focusedCommentId=14725534=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14725534
It looks like the Java driver no longer has this issue, at least I don't see
any calls to Host#getListenAddress or Host#getBroadcastAddress. Everything
seems to be using address and port.
[~aboudreault] how do things look like on the other drivers?
Edit:
Just had a look at the python driver and Endpoint was added to the Python
driver here:
[https://github.com/datastax/python-driver/blob/c88255f202a21bbbae35f16e603b0f10f2f2cf36/cassandra/connection.py#L116]
Node.js has Host support here:
[https://github.com/datastax/nodejs-driver/blob/master/lib/host.js#L41]
C# has
[https://github.com/datastax/csharp-driver/blob/master/src/Cassandra/Host.cs#L78]
CPP has [https://github.com/datastax/cpp-driver/blob/master/src/host.hpp#L92]
So I think all of the drivers can now distinguish between nodes based on node
and port. But it would probably be best for one of the driver devs to weigh in.
was (Author: bryncooke):
This comment seems to indicate that the existing behaviour was a workaround to
the drivers not distinguishing nodes by address and socket.
https://issues.apache.org/jira/browse/CASSANDRA-10052?focusedCommentId=14725534=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14725534
It looks like the Java driver no longer has this issue, at least I don't see
any calls to Host#getListenAddress or Host#getBroadcastAddress. Everything
seems to be using address and port.
[~aboudreault] how do things look like on the other drivers?
Edit:
Just had a look at the python driver and Endpoint was added to the Python
driver here:
[https://github.com/datastax/python-driver/blob/c88255f202a21bbbae35f16e603b0f10f2f2cf36/cassandra/connection.py#L116]
Node.js has Host support here:
[https://github.com/datastax/nodejs-driver/blob/master/lib/host.js#L41]
C# has
[https://github.com/datastax/csharp-driver/blob/master/src/Cassandra/Host.cs#L78]
CPP has [https://github.com/datastax/cpp-driver/blob/master/src/host.hpp#L92]
So I think all of the drivers can now distinguish between nodes based on node
and port. But it would probably be best for one of the driver devs to weigh in.
> Topology events are not sent to clients if the nodes use the same network
> interface
> ---
>
> Key: CASSANDRA-15677
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15677
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client
>Reporter: Alan Boudreault
>Assignee: Bryn Cooke
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha5
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> *This bug only happens when the cassandra nodes are configured to use a
> single network interface (ip) but different ports. See CASSANDRA-7544.*
> Issue: The topology events aren't sent to clients. The problem is that the
> port is not taken into account when determining if we send it or not:
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/transport/Server.java#L624
> To reproduce:
> {code}
> # I think the cassandra-test branch is required to get the -S option
> (USE_SINGLE_INTERFACE)
> ccm create -n4 local40 -v 4.0-alpha2 -S
> {code}
>
> Then run this small python driver script:
> {code}
> import time
> from cassandra.cluster import Cluster
> cluster = Cluster()
> session = cluster.connect()
> while True:
> print(cluster.metadata.all_hosts())
> print([h.is_up for h in cluster.metadata.all_hosts()])
> time.sleep(5)
> {code}
> Then decommission a node:
> {code}
> ccm node2 nodetool disablebinary
> ccm node2 nodetool decommission
> {code}
>
> You should see that the node is never removed from the client cluster
> metadata and the reconnector started.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15868) Update Netty version to 4.1.50 because there are security issues in 4.1.37
[ https://issues.apache.org/jira/browse/CASSANDRA-15868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15868: -- Reviewers: Aleksey Yeschenko, Aleksey Yeschenko (was: Aleksey Yeschenko) Aleksey Yeschenko, Aleksey Yeschenko (was: Aleksey Yeschenko) Status: Review In Progress (was: Patch Available) > Update Netty version to 4.1.50 because there are security issues in 4.1.37 > -- > > Key: CASSANDRA-15868 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15868 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.11.7, 4.0-beta > > Attachments: dependency-check-report.html > > > Please see attached HTML report from OWASP dependency check. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15752) Range read concurrency factor didn't consider range merger
[
https://issues.apache.org/jira/browse/CASSANDRA-15752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139340#comment-17139340
]
Andres de la Peña commented on CASSANDRA-15752:
---
Running cassandra-ci:
||[utest|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-test/130/]||[dtest|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/167/]||[jvm-dtest|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-jvm-dtest/106/]||
> Range read concurrency factor didn't consider range merger
> --
>
> Key: CASSANDRA-15752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15752
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Coordination
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> During range read, coordinator computes concurrency factor which is the
> number of vnode ranges to contact in parallel for the next batch.
> But in {{RangeCommandIterator}}, vnode ranges are merged by {{RangeMerger}}
> if vnode ranges share enough replicas to satisfy consistency level. eg. vnode
> range [a,b) has replica n1,n2,n3 and vnode range [b,c) has replica n2,n3,n4,
> so they can be merged as range [a,c) with replica n2, n3 for Quorum.
> Currently it counts number of merged ranges towards concurrency factor.
> Coordinator may fetch more ranges than needed.
>
> Another issue is that when executing range read on table with very small
> amount of data, concurrency factor can be bumped to {{size of total vnode
> ranges}}, eg. 10k, depending on the num of vnodes and cluster size. As a
> result, coordinator will send large number of concurrent range requests,
> potentially slowing down the cluster.. We should cap the max concurrency
> factor..
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15861) Mutating sstable STATS metadata may race with entire-sstable-streaming(ZCS) causing checksum validation failure
[
https://issues.apache.org/jira/browse/CASSANDRA-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang updated CASSANDRA-15861:
-
Source Control Link: https://github.com/apache/cassandra/pull/642
Test and Documentation Plan:
https://circleci.com/workflow-run/3a2fed2c-c469-4f3f-a620-07079f0dc0db
> Mutating sstable STATS metadata may race with entire-sstable-streaming(ZCS)
> causing checksum validation failure
> ---
>
> Key: CASSANDRA-15861
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15861
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair, Consistency/Streaming,
> Local/Compaction
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Flaky dtest: [test_dead_sync_initiator -
> repair_tests.repair_test.TestRepair|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/143/testReport/junit/dtest.repair_tests.repair_test/TestRepair/test_dead_sync_initiator/]
> {code:java|title=stacktrace}
> Unexpected error found in node logs (see stdout for full details). Errors:
> [ERROR [Stream-Deserializer-127.0.0.1:7000-570871f3] 2020-06-03 04:05:19,081
> CassandraEntireSSTableStreamReader.java:145 - [Stream
> 6f1c3360-a54f-11ea-a808-2f23710fdc90] Error while reading sstable from stream
> for table = keyspace1.standard1
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted:
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.maybeValidateChecksum(MetadataSerializer.java:219)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:198)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:129)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.mutate(MetadataSerializer.java:226)
> at
> org.apache.cassandra.db.streaming.CassandraEntireSSTableStreamReader.read(CassandraEntireSSTableStreamReader.java:140)
> at
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:78)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)
> at
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:181)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Checksums do not match for
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
> {code}
>
> In the above test, it executes "nodetool repair" on node1 and kills node2
> during repair. At the end, node3 reports checksum validation failure on
> sstable transferred from node1.
> {code:java|title=what happened}
> 1. When repair started on node1, it performs anti-compaction which modifies
> sstable's repairAt to 0 and pending repair id to session-id.
> 2. Then node1 creates {{ComponentManifest}} which contains file lengths to be
> transferred to node3.
> 3. Before node1 actually sends the files to node3, node2 is killed and node1
> starts to broadcast repair-failure-message to all participants in
> {{CoordinatorSession#fail}}
> 4. Node1 receives its own repair-failure-message and fails its local repair
> sessions at {{LocalSessions#failSession}} which triggers async background
> compaction.
> 5. Node1's background compaction will mutate sstable's repairAt to 0 and
> pending repair id to null via
> {{PendingRepairManager#getNextRepairFinishedTask}}, as there is no more
> in-progress repair.
> 6. Node1 actually sends the sstable to node3 where the sstable's STATS
> component size is different from the original size recorded in the manifest.
> 7. At the end, node3 reports checksum validation failure when it tries to
> mutate sstable level and "isTransient" attribute in
> {{CassandraEntireSSTableStreamReader#read}}.
> {code}
> This isn't a problem in legacy streaming as STATS file length didn't matter.
> Ideally it will be great to make sstable STATS metadata immutable, just like
> other sstable components, so we don't have to worry this special
[jira] [Comment Edited] (CASSANDRA-15677) Topology events are not sent to clients if the nodes use the same network interface
[
https://issues.apache.org/jira/browse/CASSANDRA-15677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139261#comment-17139261
]
Bryn Cooke edited comment on CASSANDRA-15677 at 6/18/20, 9:56 AM:
--
This comment seems to indicate that the existing behaviour was a workaround to
the drivers not distinguishing nodes by address and socket.
https://issues.apache.org/jira/browse/CASSANDRA-10052?focusedCommentId=14725534=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14725534
It looks like the Java driver no longer has this issue, at least I don't see
any calls to Host#getListenAddress or Host#getBroadcastAddress. Everything
seems to be using address and port.
[~aboudreault] how do things look like on the other drivers?
Edit:
Just had a look at the python driver and Endpoint was added to the Python
driver here:
[https://github.com/datastax/python-driver/blob/c88255f202a21bbbae35f16e603b0f10f2f2cf36/cassandra/connection.py#L116]
Node.js has Host support here:
[https://github.com/datastax/nodejs-driver/blob/master/lib/host.js#L41]
C# has
[https://github.com/datastax/csharp-driver/blob/master/src/Cassandra/Host.cs#L78]
CPP has [https://github.com/datastax/cpp-driver/blob/master/src/host.hpp#L92]
So I *think* all of the drivers can now distinguish between nodes based on node
and port. But it would probably be best for one of the driver devs to weigh in.
was (Author: bryncooke):
This comment seems to indicate that the existing behaviour was a workaround to
the drivers not distinguishing nodes by address and socket.
https://issues.apache.org/jira/browse/CASSANDRA-10052?focusedCommentId=14725534=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14725534
It looks like the Java driver no longer has this issue, at least I don't see
any calls to Host#getListenAddress or Host#getBroadcastAddress. Everything
seems to be using address and port.
[~aboudreault] how do things look like on the other drivers?
> Topology events are not sent to clients if the nodes use the same network
> interface
> ---
>
> Key: CASSANDRA-15677
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15677
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client
>Reporter: Alan Boudreault
>Assignee: Bryn Cooke
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha5
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> *This bug only happens when the cassandra nodes are configured to use a
> single network interface (ip) but different ports. See CASSANDRA-7544.*
> Issue: The topology events aren't sent to clients. The problem is that the
> port is not taken into account when determining if we send it or not:
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/transport/Server.java#L624
> To reproduce:
> {code}
> # I think the cassandra-test branch is required to get the -S option
> (USE_SINGLE_INTERFACE)
> ccm create -n4 local40 -v 4.0-alpha2 -S
> {code}
>
> Then run this small python driver script:
> {code}
> import time
> from cassandra.cluster import Cluster
> cluster = Cluster()
> session = cluster.connect()
> while True:
> print(cluster.metadata.all_hosts())
> print([h.is_up for h in cluster.metadata.all_hosts()])
> time.sleep(5)
> {code}
> Then decommission a node:
> {code}
> ccm node2 nodetool disablebinary
> ccm node2 nodetool decommission
> {code}
>
> You should see that the node is never removed from the client cluster
> metadata and the reconnector started.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15677) Topology events are not sent to clients if the nodes use the same network interface
[
https://issues.apache.org/jira/browse/CASSANDRA-15677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139261#comment-17139261
]
Bryn Cooke edited comment on CASSANDRA-15677 at 6/18/20, 9:56 AM:
--
This comment seems to indicate that the existing behaviour was a workaround to
the drivers not distinguishing nodes by address and socket.
https://issues.apache.org/jira/browse/CASSANDRA-10052?focusedCommentId=14725534=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14725534
It looks like the Java driver no longer has this issue, at least I don't see
any calls to Host#getListenAddress or Host#getBroadcastAddress. Everything
seems to be using address and port.
[~aboudreault] how do things look like on the other drivers?
Edit:
Just had a look at the python driver and Endpoint was added to the Python
driver here:
[https://github.com/datastax/python-driver/blob/c88255f202a21bbbae35f16e603b0f10f2f2cf36/cassandra/connection.py#L116]
Node.js has Host support here:
[https://github.com/datastax/nodejs-driver/blob/master/lib/host.js#L41]
C# has
[https://github.com/datastax/csharp-driver/blob/master/src/Cassandra/Host.cs#L78]
CPP has [https://github.com/datastax/cpp-driver/blob/master/src/host.hpp#L92]
So I think all of the drivers can now distinguish between nodes based on node
and port. But it would probably be best for one of the driver devs to weigh in.
was (Author: bryncooke):
This comment seems to indicate that the existing behaviour was a workaround to
the drivers not distinguishing nodes by address and socket.
https://issues.apache.org/jira/browse/CASSANDRA-10052?focusedCommentId=14725534=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14725534
It looks like the Java driver no longer has this issue, at least I don't see
any calls to Host#getListenAddress or Host#getBroadcastAddress. Everything
seems to be using address and port.
[~aboudreault] how do things look like on the other drivers?
Edit:
Just had a look at the python driver and Endpoint was added to the Python
driver here:
[https://github.com/datastax/python-driver/blob/c88255f202a21bbbae35f16e603b0f10f2f2cf36/cassandra/connection.py#L116]
Node.js has Host support here:
[https://github.com/datastax/nodejs-driver/blob/master/lib/host.js#L41]
C# has
[https://github.com/datastax/csharp-driver/blob/master/src/Cassandra/Host.cs#L78]
CPP has [https://github.com/datastax/cpp-driver/blob/master/src/host.hpp#L92]
So I *think* all of the drivers can now distinguish between nodes based on node
and port. But it would probably be best for one of the driver devs to weigh in.
> Topology events are not sent to clients if the nodes use the same network
> interface
> ---
>
> Key: CASSANDRA-15677
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15677
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client
>Reporter: Alan Boudreault
>Assignee: Bryn Cooke
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha5
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> *This bug only happens when the cassandra nodes are configured to use a
> single network interface (ip) but different ports. See CASSANDRA-7544.*
> Issue: The topology events aren't sent to clients. The problem is that the
> port is not taken into account when determining if we send it or not:
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/transport/Server.java#L624
> To reproduce:
> {code}
> # I think the cassandra-test branch is required to get the -S option
> (USE_SINGLE_INTERFACE)
> ccm create -n4 local40 -v 4.0-alpha2 -S
> {code}
>
> Then run this small python driver script:
> {code}
> import time
> from cassandra.cluster import Cluster
> cluster = Cluster()
> session = cluster.connect()
> while True:
> print(cluster.metadata.all_hosts())
> print([h.is_up for h in cluster.metadata.all_hosts()])
> time.sleep(5)
> {code}
> Then decommission a node:
> {code}
> ccm node2 nodetool disablebinary
> ccm node2 nodetool decommission
> {code}
>
> You should see that the node is never removed from the client cluster
> metadata and the reconnector started.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15677) Topology events are not sent to clients if the nodes use the same network interface
[
https://issues.apache.org/jira/browse/CASSANDRA-15677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139261#comment-17139261
]
Bryn Cooke commented on CASSANDRA-15677:
This comment seems to indicate that the existing behaviour was a workaround to
the drivers not distinguishing nodes by address and socket.
https://issues.apache.org/jira/browse/CASSANDRA-10052?focusedCommentId=14725534=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14725534
It looks like the Java driver no longer has this issue, at least I don't see
any calls to Host#getListenAddress or Host#getBroadcastAddress. Everything
seems to be using address and port.
[~aboudreault] how do things look like on the other drivers?
> Topology events are not sent to clients if the nodes use the same network
> interface
> ---
>
> Key: CASSANDRA-15677
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15677
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client
>Reporter: Alan Boudreault
>Assignee: Bryn Cooke
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha5
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> *This bug only happens when the cassandra nodes are configured to use a
> single network interface (ip) but different ports. See CASSANDRA-7544.*
> Issue: The topology events aren't sent to clients. The problem is that the
> port is not taken into account when determining if we send it or not:
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/transport/Server.java#L624
> To reproduce:
> {code}
> # I think the cassandra-test branch is required to get the -S option
> (USE_SINGLE_INTERFACE)
> ccm create -n4 local40 -v 4.0-alpha2 -S
> {code}
>
> Then run this small python driver script:
> {code}
> import time
> from cassandra.cluster import Cluster
> cluster = Cluster()
> session = cluster.connect()
> while True:
> print(cluster.metadata.all_hosts())
> print([h.is_up for h in cluster.metadata.all_hosts()])
> time.sleep(5)
> {code}
> Then decommission a node:
> {code}
> ccm node2 nodetool disablebinary
> ccm node2 nodetool decommission
> {code}
>
> You should see that the node is never removed from the client cluster
> metadata and the reconnector started.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15875) Jenkins pipeline small fixes: slack notification body, StabilityTestDataPublisher, archive cassandra-test-report.txt
[ https://issues.apache.org/jira/browse/CASSANDRA-15875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139259#comment-17139259 ] Michael Semb Wever commented on CASSANDRA-15875: Thanks for the testing [~Bereng], very much appreciated. Yes, I have run all the branches. The slack plugin is not normally locally, as we only want slack and email notifications from the official builds at ci-cassandra.apache.org. But, adding the slack plugin by itself isn't enough to fire off the notifications, the credentials to the ASF slack needed to be added as well. So it is safe to add the plugin I will add the doc changes, thanks. > Jenkins pipeline small fixes: slack notification body, > StabilityTestDataPublisher, archive cassandra-test-report.txt > > > Key: CASSANDRA-15875 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15875 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 10m > Remaining Estimate: 0h > > 1) The slack notification bodies contain an array of SHA commit messages > involved in the build. The array needs to be formatting. > See https://the-asf.slack.com/archives/CS6CA748J/p1592029643136500 > 2) Remove cassandra-builds directory in Summary stage, so 'Restart from > Stage' builds work. If the pipeline gets into the final Summary stage it will > clone the cassandra-builds directory. Once this happens it is not possible to > restart the pipeline. Restarting from a given stage is useful if one of the > test stages failed for unknown/infrastructure reasons. > 3) Archive the cassandra-test-report.txt artifact. This plaintext report was > added under CASSANDRA-15729. It should also be made easily available as a > downloadable artefact on the build. > 4) Add StabilityTestDataPublisher to the pipeline junit results. This > provides the flakey info next to each test on the test results page. The > individual test stages had this enabled, but not the pipeline. For example > here: > https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L252 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15880) Memory leak in CompressedChunkReader
[ https://issues.apache.org/jira/browse/CASSANDRA-15880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaroslaw Grabowski updated CASSANDRA-15880: --- Since Version: 3.6 > Memory leak in CompressedChunkReader > > > Key: CASSANDRA-15880 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15880 > Project: Cassandra > Issue Type: Bug > Components: Feature/Compression >Reporter: Jaroslaw Grabowski >Priority: Normal > > CompressedChunkReader uses java.lang.ThreadLocal to reuse ByteBuffer for > compressed data. ByteBuffers leak due to peculiar ThreadLocal quality. > ThreadLocals are stored in a map, where the key is a weak reference to a > ThreadLocal and the value is the user's object (ByteBuffer in this case). > When a last strong reference to a ThreadLocal is lost, weak reference to > ThreadLocal (key) is removed but the value (ByteBuffer) is kept until cleaned > by ThreadLocal heuristic expunge mechanism. See ThreadLocal's "stale entries" > for details. > When a number of long-living threads is high enough this results in thousands > of ByteBuffers stored as stale entries in ThreadLocals. In a not-so-lucky > scenario we get OutOfMemoryException. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15880) Memory leak in CompressedChunkReader
Jaroslaw Grabowski created CASSANDRA-15880: -- Summary: Memory leak in CompressedChunkReader Key: CASSANDRA-15880 URL: https://issues.apache.org/jira/browse/CASSANDRA-15880 Project: Cassandra Issue Type: Bug Components: Feature/Compression Reporter: Jaroslaw Grabowski CompressedChunkReader uses java.lang.ThreadLocal to reuse ByteBuffer for compressed data. ByteBuffers leak due to peculiar ThreadLocal quality. ThreadLocals are stored in a map, where the key is a weak reference to a ThreadLocal and the value is the user's object (ByteBuffer in this case). When a last strong reference to a ThreadLocal is lost, weak reference to ThreadLocal (key) is removed but the value (ByteBuffer) is kept until cleaned by ThreadLocal heuristic expunge mechanism. See ThreadLocal's "stale entries" for details. When a number of long-living threads is high enough this results in thousands of ByteBuffers stored as stale entries in ThreadLocals. In a not-so-lucky scenario we get OutOfMemoryException. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15875) Jenkins pipeline small fixes: slack notification body, StabilityTestDataPublisher, archive cassandra-test-report.txt
[ https://issues.apache.org/jira/browse/CASSANDRA-15875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-15875: Reviewers: Berenguer Blasi, Berenguer Blasi (was: Berenguer Blasi) Berenguer Blasi, Berenguer Blasi (was: Berenguer Blasi) Status: Review In Progress (was: Patch Available) > Jenkins pipeline small fixes: slack notification body, > StabilityTestDataPublisher, archive cassandra-test-report.txt > > > Key: CASSANDRA-15875 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15875 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 10m > Remaining Estimate: 0h > > 1) The slack notification bodies contain an array of SHA commit messages > involved in the build. The array needs to be formatting. > See https://the-asf.slack.com/archives/CS6CA748J/p1592029643136500 > 2) Remove cassandra-builds directory in Summary stage, so 'Restart from > Stage' builds work. If the pipeline gets into the final Summary stage it will > clone the cassandra-builds directory. Once this happens it is not possible to > restart the pipeline. Restarting from a given stage is useful if one of the > test stages failed for unknown/infrastructure reasons. > 3) Archive the cassandra-test-report.txt artifact. This plaintext report was > added under CASSANDRA-15729. It should also be made easily available as a > downloadable artefact on the build. > 4) Add StabilityTestDataPublisher to the pipeline junit results. This > provides the flakey info next to each test on the test results page. The > individual test stages had this enabled, but not the pipeline. For example > here: > https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L252 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15875) Jenkins pipeline small fixes: slack notification body, StabilityTestDataPublisher, archive cassandra-test-report.txt
[ https://issues.apache.org/jira/browse/CASSANDRA-15875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139233#comment-17139233 ] Berenguer Blasi commented on CASSANDRA-15875: - [~mck] I would suggest [these|https://github.com/apache/cassandra/pull/641] doc changes to be committed as well > Jenkins pipeline small fixes: slack notification body, > StabilityTestDataPublisher, archive cassandra-test-report.txt > > > Key: CASSANDRA-15875 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15875 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 10m > Remaining Estimate: 0h > > 1) The slack notification bodies contain an array of SHA commit messages > involved in the build. The array needs to be formatting. > See https://the-asf.slack.com/archives/CS6CA748J/p1592029643136500 > 2) Remove cassandra-builds directory in Summary stage, so 'Restart from > Stage' builds work. If the pipeline gets into the final Summary stage it will > clone the cassandra-builds directory. Once this happens it is not possible to > restart the pipeline. Restarting from a given stage is useful if one of the > test stages failed for unknown/infrastructure reasons. > 3) Archive the cassandra-test-report.txt artifact. This plaintext report was > added under CASSANDRA-15729. It should also be made easily available as a > downloadable artefact on the build. > 4) Add StabilityTestDataPublisher to the pipeline junit results. This > provides the flakey info next to each test on the test results page. The > individual test stages had this enabled, but not the pipeline. For example > here: > https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L252 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15875) Jenkins pipeline small fixes: slack notification body, StabilityTestDataPublisher, archive cassandra-test-report.txt
[ https://issues.apache.org/jira/browse/CASSANDRA-15875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139223#comment-17139223 ] Berenguer Blasi commented on CASSANDRA-15875: - [~mck] I installed a jenkins and did a run for the 4.0 branch. Everything worked as expected except for the Slack message as that plugin was missing, will have to update docs. It takes a long time to run these and it downloads half the internet for each branch upon building. If you confirm you have ran all the branches yourself already, changes being identical across the, I'm happy to +1. wdyt? > Jenkins pipeline small fixes: slack notification body, > StabilityTestDataPublisher, archive cassandra-test-report.txt > > > Key: CASSANDRA-15875 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15875 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 4.0-beta > > > 1) The slack notification bodies contain an array of SHA commit messages > involved in the build. The array needs to be formatting. > See https://the-asf.slack.com/archives/CS6CA748J/p1592029643136500 > 2) Remove cassandra-builds directory in Summary stage, so 'Restart from > Stage' builds work. If the pipeline gets into the final Summary stage it will > clone the cassandra-builds directory. Once this happens it is not possible to > restart the pipeline. Restarting from a given stage is useful if one of the > test stages failed for unknown/infrastructure reasons. > 3) Archive the cassandra-test-report.txt artifact. This plaintext report was > added under CASSANDRA-15729. It should also be made easily available as a > downloadable artefact on the build. > 4) Add StabilityTestDataPublisher to the pipeline junit results. This > provides the flakey info next to each test on the test results page. The > individual test stages had this enabled, but not the pipeline. For example > here: > https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L252 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15879) Flaky unit test: BlacklistingCompactionsTest.testBlacklistingWithSizeTieredCompactionStrategy
[
https://issues.apache.org/jira/browse/CASSANDRA-15879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139202#comment-17139202
]
Marcus Eriksson commented on CASSANDRA-15879:
-
+1
> Flaky unit test:
> BlacklistingCompactionsTest.testBlacklistingWithSizeTieredCompactionStrategy
> -
>
> Key: CASSANDRA-15879
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15879
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> CASSANDRA-14238 addressed the failure in
> {{BlacklistingCompactionsTest.testBlacklistingWithSizeTieredCompactionStrategy}},
> but only on 2.2. While working on CASSANDRA-14888, we’ve reproduced [the
> failure|https://app.circleci.com/pipelines/github/dineshjoshi/cassandra/47/workflows/de5f7cdb-06b6-4869-9d19-81a145e79f3f/jobs/2516/tests]
> on trunk.
> It looks like this should be a clean merge forward.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15863) Bootstrap resume and TestReplaceAddress fixes
[ https://issues.apache.org/jira/browse/CASSANDRA-15863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139173#comment-17139173 ] Berenguer Blasi commented on CASSANDRA-15863: - Ok so after some investigation by a git master friend of mine... That is a merge commit. When you click on GH on the commit if shows you the _diff_ with the topmost parent, hence we get the right code. When you append '.patch' you get the patch for that _commit_, not for the _diff_ as you were seeing before. So... not nice GH, not nice... > Bootstrap resume and TestReplaceAddress fixes > - > > Key: CASSANDRA-15863 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15863 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Bootstrap and Decommission, Test/dtest >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-alpha > > Time Spent: 10m > Remaining Estimate: 0h > > This has been > [broken|https://ci-cassandra.apache.org/job/Cassandra-trunk/159/testReport/dtest-large.replace_address_test/TestReplaceAddress/test_restart_failed_replace/history/] > for ages -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15872) Flaky test: testNoTreesRetainedAfterDifference - org.apache.cassandra.repair.RepairJobTest
[ https://issues.apache.org/jira/browse/CASSANDRA-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhaoYang updated CASSANDRA-15872: - Test and Documentation Plan: https://circleci.com/workflow-run/fe9a153f-1a04-4993-b582-f0f1bbd9ca77 (was: https://circleci.com/workflow-run/d899f088-c1ce-4e25-8639-2910b3f0362b) > Flaky test: testNoTreesRetainedAfterDifference - > org.apache.cassandra.repair.RepairJobTest > -- > > Key: CASSANDRA-15872 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15872 > Project: Cassandra > Issue Type: Task > Components: Consistency/Repair >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Normal > Fix For: 4.0-rc > > > link: [https://circleci.com/gh/maedhroz/cassandra/21#tests/containers/95] > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15872) Flaky test: testNoTreesRetainedAfterDifference - org.apache.cassandra.repair.RepairJobTest
[ https://issues.apache.org/jira/browse/CASSANDRA-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhaoYang updated CASSANDRA-15872: - Source Control Link: https://github.com/apache/cassandra/pull/630 Test and Documentation Plan: https://circleci.com/workflow-run/d899f088-c1ce-4e25-8639-2910b3f0362b > Flaky test: testNoTreesRetainedAfterDifference - > org.apache.cassandra.repair.RepairJobTest > -- > > Key: CASSANDRA-15872 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15872 > Project: Cassandra > Issue Type: Task > Components: Consistency/Repair >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Normal > Fix For: 4.0-rc > > > link: [https://circleci.com/gh/maedhroz/cassandra/21#tests/containers/95] > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15872) Flaky test: testNoTreesRetainedAfterDifference - org.apache.cassandra.repair.RepairJobTest
[ https://issues.apache.org/jira/browse/CASSANDRA-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17139141#comment-17139141 ] ZhaoYang commented on CASSANDRA-15872: -- bq. One alternative is that we could take MeasureableRepairSession, provide it a callback (perhaps allow it to be set int the test body), override syncComplete(), and hit the callback from that before we call super.syncComplete() good idea. updated the patch to put a latch on the "MeasureableRepairSession" callback. > Flaky test: testNoTreesRetainedAfterDifference - > org.apache.cassandra.repair.RepairJobTest > -- > > Key: CASSANDRA-15872 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15872 > Project: Cassandra > Issue Type: Task > Components: Consistency/Repair >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Normal > Fix For: 4.0-rc > > > link: [https://circleci.com/gh/maedhroz/cassandra/21#tests/containers/95] > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
