[jira] [Commented] (CASSANDRA-12126) CAS Reads Inconsistencies
[ https://issues.apache.org/jira/browse/CASSANDRA-12126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241218#comment-17241218 ] Hiroyuki Yamada commented on CASSANDRA-12126: - > 3) Issue another CAS Read and it goes to A and B. Now we will discover that > there is something inflight from A and will propose and commit it with the > current ballot. Now we can read the value written in step 1 as part of this > CAS read. Sorry, I'm not fully sure about the current implementation and how realistic my proposal is but, can we read all the replicas to do the read recovery in step 3 to solve the issue? It only reads A and B but if we read C as well, we know that the proposal is not accepted by the majority. > CAS Reads Inconsistencies > -- > > Key: CASSANDRA-12126 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12126 > Project: Cassandra > Issue Type: Bug > Components: Feature/Lightweight Transactions, Legacy/Coordination >Reporter: Sankalp Kohli >Assignee: Sylvain Lebresne >Priority: Normal > Labels: LWT, pull-request-available > Fix For: 4.0-beta4, 3.0.24, 3.11.10 > > Time Spent: 10m > Remaining Estimate: 0h > > While looking at the CAS code in Cassandra, I found a potential issue with > CAS Reads. Here is how it can happen with RF=3 > 1) You issue a CAS Write and it fails in the propose phase. A machine replies > true to a propose and saves the commit in accepted filed. The other two > machines B and C does not get to the accept phase. > Current state is that machine A has this commit in paxos table as accepted > but not committed and B and C does not. > 2) Issue a CAS Read and it goes to only B and C. You wont be able to read the > value written in step 1. This step is as if nothing is inflight. > 3) Issue another CAS Read and it goes to A and B. Now we will discover that > there is something inflight from A and will propose and commit it with the > current ballot. Now we can read the value written in step 1 as part of this > CAS read. > If we skip step 3 and instead run step 4, we will never learn about value > written in step 1. > 4. Issue a CAS Write and it involves only B and C. This will succeed and > commit a different value than step 1. Step 1 value will never be seen again > and was never seen before. > If you read the Lamport “paxos made simple” paper and read section 2.3. It > talks about this issue which is how learners can find out if majority of the > acceptors have accepted the proposal. > In step 3, it is correct that we propose the value again since we dont know > if it was accepted by majority of acceptors. When we ask majority of > acceptors, and more than one acceptors but not majority has something in > flight, we have no way of knowing if it is accepted by majority of acceptors. > So this behavior is correct. > However we need to fix step 2, since it caused reads to not be linearizable > with respect to writes and other reads. In this case, we know that majority > of acceptors have no inflight commit which means we have majority that > nothing was accepted by majority. I think we should run a propose step here > with empty commit and that will cause write written in step 1 to not be > visible ever after. > With this fix, we will either see data written in step 1 on next serial read > or will never see it which is what we want. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16225) Followup CASSANDRA-14554
[ https://issues.apache.org/jira/browse/CASSANDRA-16225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241197#comment-17241197 ] Ekaterina Dimitrova commented on CASSANDRA-16225: - Thanks [~psy...@t-online.de], do I understand correctly that no full CI run is expected and I can ask [~brandon.williams] Brandon for this ninja fix commit, if he doesn't have any concerns? > Followup CASSANDRA-14554 > > > Key: CASSANDRA-16225 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16225 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Internode >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-beta, 4.0-beta4, 3.0.24, 3.11.10 > > > As per [~stefania]'s advice, additional synchronization should be added to > LogTransaction.java. Without synchronization, we could have corrupted txn log > files with JBOD. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16307) GROUP BY queries with paging can return deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-16307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241159#comment-17241159
]
Andres de la Peña edited comment on CASSANDRA-16307 at 12/1/20, 12:40 AM:
--
[~dcapwell] Yes, it affects 3.11 and trunk, but not 3.0 because it doesn't have
group-by queries.
was (Author: adelapena):
[~dcapwell] Yes, it affects 3.11 and trunk.
> GROUP BY queries with paging can return deleted data
>
>
> Key: CASSANDRA-16307
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16307
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Andres de la Peña
>Priority: Normal
>
> {{GROUP BY}} queries using paging and CL>ONE/LOCAL_ONE. This dtest reproduces
> the problem:
> {code:java}
> try (Cluster cluster = init(Cluster.create(2)))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (pk int, ck int,
> PRIMARY KEY (pk, ck))"));
> ICoordinator coordinator = cluster.coordinator(1);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (0,
> 0)"), ConsistencyLevel.ALL);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (1,
> 1)"), ConsistencyLevel.ALL);
>
> cluster.get(1).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=0
> AND ck=0"));
> cluster.get(2).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=1
> AND ck=1"));
> String query = withKeyspace("SELECT * FROM %s.t GROUP BY pk");
> Iterator rows = coordinator.executeWithPaging(query,
> ConsistencyLevel.ALL, 1);
> assertRows(Iterators.toArray(rows, Object[].class));
> }
> {code}
> Using a 2-node cluster and RF=2, the test inserts two partitions in both
> nodes. Then it locally deletes each row in a separate node, so each node sees
> a different partition alive, but reconciliation should produce no alive
> partitions. However, a {{GROUP BY}} query using a page size of 1 wrongly
> returns one of the rows.
> This has been detected during CASSANDRA-16180, and it is probably related to
> CASSANDRA-15459, which solved a similar problem for group-by queries with
> limit, instead of paging.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16307) GROUP BY queries with paging can return deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-16307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241159#comment-17241159
]
Andres de la Peña commented on CASSANDRA-16307:
---
[~dcapwell] Yes, it affects 3.11 and trunk.
> GROUP BY queries with paging can return deleted data
>
>
> Key: CASSANDRA-16307
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16307
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Andres de la Peña
>Priority: Normal
>
> {{GROUP BY}} queries using paging and CL>ONE/LOCAL_ONE. This dtest reproduces
> the problem:
> {code:java}
> try (Cluster cluster = init(Cluster.create(2)))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (pk int, ck int,
> PRIMARY KEY (pk, ck))"));
> ICoordinator coordinator = cluster.coordinator(1);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (0,
> 0)"), ConsistencyLevel.ALL);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (1,
> 1)"), ConsistencyLevel.ALL);
>
> cluster.get(1).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=0
> AND ck=0"));
> cluster.get(2).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=1
> AND ck=1"));
> String query = withKeyspace("SELECT * FROM %s.t GROUP BY pk");
> Iterator rows = coordinator.executeWithPaging(query,
> ConsistencyLevel.ALL, 1);
> assertRows(Iterators.toArray(rows, Object[].class));
> }
> {code}
> Using a 2-node cluster and RF=2, the test inserts two partitions in both
> nodes. Then it locally deletes each row in a separate node, so each node sees
> a different partition alive, but reconciliation should produce no alive
> partitions. However, a {{GROUP BY}} query using a page size of 1 wrongly
> returns one of the rows.
> This has been detected during CASSANDRA-16180, and it is probably related to
> CASSANDRA-15459, which solved a similar problem for group-by queries with
> limit, instead of paging.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16307) GROUP BY queries with paging can return deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-16307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-16307:
--
Since Version: 3.10
> GROUP BY queries with paging can return deleted data
>
>
> Key: CASSANDRA-16307
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16307
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Andres de la Peña
>Priority: Normal
>
> {{GROUP BY}} queries using paging and CL>ONE/LOCAL_ONE. This dtest reproduces
> the problem:
> {code:java}
> try (Cluster cluster = init(Cluster.create(2)))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (pk int, ck int,
> PRIMARY KEY (pk, ck))"));
> ICoordinator coordinator = cluster.coordinator(1);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (0,
> 0)"), ConsistencyLevel.ALL);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (1,
> 1)"), ConsistencyLevel.ALL);
>
> cluster.get(1).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=0
> AND ck=0"));
> cluster.get(2).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=1
> AND ck=1"));
> String query = withKeyspace("SELECT * FROM %s.t GROUP BY pk");
> Iterator rows = coordinator.executeWithPaging(query,
> ConsistencyLevel.ALL, 1);
> assertRows(Iterators.toArray(rows, Object[].class));
> }
> {code}
> Using a 2-node cluster and RF=2, the test inserts two partitions in both
> nodes. Then it locally deletes each row in a separate node, so each node sees
> a different partition alive, but reconciliation should produce no alive
> partitions. However, a {{GROUP BY}} query using a page size of 1 wrongly
> returns one of the rows.
> This has been detected during CASSANDRA-16180, and it is probably related to
> CASSANDRA-15459, which solved a similar problem for group-by queries with
> limit, instead of paging.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16307) GROUP BY queries with paging can return deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-16307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-16307:
--
Bug Category: Parent values: Correctness(12982)Level 1 values:
Consistency(12989)
Complexity: Normal
Discovered By: Unit Test
Severity: Normal
Status: Open (was: Triage Needed)
> GROUP BY queries with paging can return deleted data
>
>
> Key: CASSANDRA-16307
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16307
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Andres de la Peña
>Priority: Normal
>
> {{GROUP BY}} queries using paging and CL>ONE/LOCAL_ONE. This dtest reproduces
> the problem:
> {code:java}
> try (Cluster cluster = init(Cluster.create(2)))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (pk int, ck int,
> PRIMARY KEY (pk, ck))"));
> ICoordinator coordinator = cluster.coordinator(1);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (0,
> 0)"), ConsistencyLevel.ALL);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (1,
> 1)"), ConsistencyLevel.ALL);
>
> cluster.get(1).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=0
> AND ck=0"));
> cluster.get(2).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=1
> AND ck=1"));
> String query = withKeyspace("SELECT * FROM %s.t GROUP BY pk");
> Iterator rows = coordinator.executeWithPaging(query,
> ConsistencyLevel.ALL, 1);
> assertRows(Iterators.toArray(rows, Object[].class));
> }
> {code}
> Using a 2-node cluster and RF=2, the test inserts two partitions in both
> nodes. Then it locally deletes each row in a separate node, so each node sees
> a different partition alive, but reconciliation should produce no alive
> partitions. However, a {{GROUP BY}} query using a page size of 1 wrongly
> returns one of the rows.
> This has been detected during CASSANDRA-16180, and it is probably related to
> CASSANDRA-15459, which solved a similar problem for group-by queries with
> limit, instead of paging.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16307) GROUP BY queries with paging can return deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-16307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241147#comment-17241147
]
David Capwell commented on CASSANDRA-16307:
---
Any idea if this impacts 3.x as well?
> GROUP BY queries with paging can return deleted data
>
>
> Key: CASSANDRA-16307
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16307
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Andres de la Peña
>Priority: Normal
>
> {{GROUP BY}} queries using paging and CL>ONE/LOCAL_ONE. This dtest reproduces
> the problem:
> {code:java}
> try (Cluster cluster = init(Cluster.create(2)))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (pk int, ck int,
> PRIMARY KEY (pk, ck))"));
> ICoordinator coordinator = cluster.coordinator(1);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (0,
> 0)"), ConsistencyLevel.ALL);
> coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (1,
> 1)"), ConsistencyLevel.ALL);
>
> cluster.get(1).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=0
> AND ck=0"));
> cluster.get(2).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=1
> AND ck=1"));
> String query = withKeyspace("SELECT * FROM %s.t GROUP BY pk");
> Iterator rows = coordinator.executeWithPaging(query,
> ConsistencyLevel.ALL, 1);
> assertRows(Iterators.toArray(rows, Object[].class));
> }
> {code}
> Using a 2-node cluster and RF=2, the test inserts two partitions in both
> nodes. Then it locally deletes each row in a separate node, so each node sees
> a different partition alive, but reconciliation should produce no alive
> partitions. However, a {{GROUP BY}} query using a page size of 1 wrongly
> returns one of the rows.
> This has been detected during CASSANDRA-16180, and it is probably related to
> CASSANDRA-15459, which solved a similar problem for group-by queries with
> limit, instead of paging.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16217) Minimal 4.0 COMPACT STORAGE backport
[ https://issues.apache.org/jira/browse/CASSANDRA-16217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241145#comment-17241145 ] David Capwell commented on CASSANDRA-16217: --- As long as the author's of the tests modified (Caleb in this case) are ok then I am ok. > Minimal 4.0 COMPACT STORAGE backport > > > Key: CASSANDRA-16217 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16217 > Project: Cassandra > Issue Type: Bug > Components: Legacy/CQL >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > Fix For: 4.0-beta4 > > Time Spent: 20m > Remaining Estimate: 0h > > There are several behavioural changes related to compact storage, and these > differences are larger than most of us have anticipated: we first thought > there’ll be that “appearing column”, but there’s implicit nulls in > clusterings thing, and row vs column deletion. > Some of the recent issues on the subject are: CASSANDRA-16048, which allows > to ignore these differences. The other one was trying to improve user > experience of anyone still using compact storage: CASSANDRA-15811. > Easily reproducible differernces are: > (1) hidden columns show up, which breaks SELECT * queries > (2) DELETE v and UPDATE v WITH TTL would result into row removals in > non-dense compact tables (CASSANDRA-16069) > (3) INSERT allows skipping clusterings, which are filled with nulls by > default. > Some of these are tricky to support, as 15811 has shown. Anyone on OSS side > who might want to upgrade to 4.0 while still using compact storage might be > affected by being forced into one of these behaviours. > Possible solutions are to document these behaviours, or to bring back a > minimal set of COMPACT STORAGE to keep supporting these. > It looks like it is possible to leave some of the functionality related to > DENSE flag and allow it to be present in 4.0, but only for these three (and > potential related, however not direrclty visible) cases. > [~e.dimitrova] since you were working on removal on compact storage, wanted > to reassure that this is not a revert of your patch. On contrary: your patch > was instrumental in identifying the right places. > cc [~slebresne] [~aleksey] [~benedict] [~marcuse] > |[patch|https://github.com/apache/cassandra/pull/785]|[ci|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=13994-followup]| -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16083) Missing JMX objects and attributes upgrading from 3.0 to 4.0
[
https://issues.apache.org/jira/browse/CASSANDRA-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241141#comment-17241141
]
David Capwell commented on CASSANDRA-16083:
---
bq. The Stage threadpools are initialized lazily. By consequence, their metrics
will not appear until the stage has been used at least once.
Would it explain the missing metrics?
Since this was seen from scraping metrics it could, but that would imply that
3.x was not lazy and 4.x is? If we validate that they are present on a running
cluster than it can be safe to ride off as tooling issue.
bq. To me it looks like the rest requires update of the documentation?
DroppedMessage looks like we could add an alias layer to avoid breaking, as
that is a good metric to monitor to make sure the cluster is healthy.
ConditionNotMet looks like a rename regression, we should add an alias to fix
it (like we have for other metrics).
> Missing JMX objects and attributes upgrading from 3.0 to 4.0
>
>
> Key: CASSANDRA-16083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16083
> Project: Cassandra
> Issue Type: Task
> Components: Observability/Metrics
>Reporter: David Capwell
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Using the tools added in CASSANDRA-16082, below are the list of metrics
> missing in 4.0 but present in 3.0. The work here is to make sure we had
> proper deprecation for each metric, and if not to add it back.
> {code}
> $ tools/bin/jmxtool diff -f yaml cassandra-3.0-jmx.yaml trunk-jmx.yaml
> --ignore-missing-on-left
> Objects not in right:
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=schema_columnfamilies,name=CasPrepareLatency
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=EstimatedPartitionSizeHistogram
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=hints,name=BloomFilterFalseRatio
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=ReplicaFilteringProtectionRequests
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=batchlog,name=RowCacheHitOutOfRange
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=CasPrepareLatency
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=CounterMutationStage,name=MaxPoolSize
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=ColUpdateTimeDeltaHistogram
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=batchlog,name=TombstoneScannedHistogram
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadStage,name=ActiveTasks
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=hints,name=WaitingOnFreeMemtableSpace
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=schema_columnfamilies,name=CasCommitTotalLatency
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=hints,name=MemtableOnHeapSize
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=schema_aggregates,name=CasProposeLatency
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=batchlog,name=AllMemtablesLiveDataSize
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=hints,name=ViewReadTime
> org.apache.cassandra.db:type=HintedHandoffManager
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=BloomFilterDiskSpaceUsed
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=RequestResponseStage,name=PendingTasks
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=MemtableSwitchCount
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=hints,name=MemtableOnHeapSize
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=range_xfers,name=ReplicaFilteringProtectionRowsCachedPerQuery
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=SnapshotsSize
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=RecentBloomFilterFalsePositives
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=ColUpdateTimeDeltaHistogram
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=range_xfers,name=SpeculativeRetries
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=LiveDiskSpaceUsed
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=ViewReadTime
> org.apache.cassandra.metrics:type=ThreadPools,path=internal,scope=MigrationStage,name=CompletedTasks
>
[jira] [Commented] (CASSANDRA-16192) Add more tests to cover compaction metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241097#comment-17241097 ] Adam Holmberg commented on CASSANDRA-16192: --- Another PR ready for review: https://github.com/aholmberg/cassandra/pull/18 ci: https://app.circleci.com/pipelines/github/aholmberg/cassandra?branch=CASSANDRA-16192 Removed myself as Reviewer since I revised the tests. > Add more tests to cover compaction metrics > -- > > Key: CASSANDRA-16192 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16192 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Assignee: Adam Holmberg >Priority: Normal > Fix For: 4.0-beta > > > Some compaction metrics do not seems to be tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16192) Add more tests to cover compaction metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg reassigned CASSANDRA-16192: - Assignee: Adam Holmberg > Add more tests to cover compaction metrics > -- > > Key: CASSANDRA-16192 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16192 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Assignee: Adam Holmberg >Priority: Normal > Fix For: 4.0-beta > > > Some compaction metrics do not seems to be tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16192) Add more tests to cover compaction metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-16192: -- Reviewers: (was: Adam Holmberg) > Add more tests to cover compaction metrics > -- > > Key: CASSANDRA-16192 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16192 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Assignee: Adam Holmberg >Priority: Normal > Fix For: 4.0-beta > > > Some compaction metrics do not seems to be tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16192) Add more tests to cover compaction metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Holmberg updated CASSANDRA-16192: -- Authors: Adam Holmberg, Mohamed Zafraan Test and Documentation Plan: Added new unit tests. No Docs Status: Patch Available (was: Open) > Add more tests to cover compaction metrics > -- > > Key: CASSANDRA-16192 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16192 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Priority: Normal > Fix For: 4.0-beta > > > Some compaction metrics do not seems to be tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13325) Bring back the accepted encryption protocols list as configurable option
[
https://issues.apache.org/jira/browse/CASSANDRA-13325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241090#comment-17241090
]
Jon Meredith commented on CASSANDRA-13325:
--
[~dcapwell] - thanks for the review - pushed up nits.
> Bring back the accepted encryption protocols list as configurable option
>
>
> Key: CASSANDRA-13325
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13325
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Nachiket Patil
>Assignee: Jon Meredith
>Priority: Low
> Fix For: 4.0-beta
>
> Attachments: trunk.diff
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> With CASSANDRA-10508, the hard coded list of accepted encryption protocols
> was eliminated. For some use cases, it is necessary to restrict the
> encryption protocols used for communication between client and server.
> Default JVM way of negotiations allows the best encryption protocol that
> client can use.
> e.g. I have set Cassandra to use encryption. Ideally client and server
> negotiate to use best protocol (TLSv1.2). But a malicious client might force
> TLSv1.0 which is susceptible to POODLE attacks.
> At the moment only way to restrict the encryption protocol is using the
> {{jdk.tls.client.protocols}} systems property. If I dont have enough access
> to modify this property, I dont have any way of restricting the encryption
> protocols.
> I am proposing bring back the accepted_protocols property but make it
> configurable. If not specified, let the JVM take care of the TLS negotiations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16213) Cannot replace_address /X because it doesn't exist in gossip
[ https://issues.apache.org/jira/browse/CASSANDRA-16213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241059#comment-17241059 ] David Capwell commented on CASSANDRA-16213: --- I left comments on the commit https://github.com/beobal/cassandra/commit/b33d3aaf01326e49bacda2410c0fa6fc17677ae5 To summarize I think these are the following changes 1) size of the patch 2) avoid injecting the host_id + token on startup, but keep the "empty" node in gossip as is. On shadow round, query table to repopulate this in order to reply 3) removed the allowed state logic For #1 it looks like this is only done by removing the explicit "empty" flags and replaced access with logic which tries to detect it based off symptoms (generation and version == 0, absents from gossip, etc.). I personally feel this logic is justifiable as it makes it easy to see how/where we handle the logic, by scattering the handling it gets harder to see how its handled. For #2 I am not sure why, would love to hear from you on what you are thinking. I do love the extra comments but it isn't clear the motivation between the two versions. [~samt] thanks for the review. If it helps we can chat on slack tomorrow (or when you are free) and can summaries here after. > Cannot replace_address /X because it doesn't exist in gossip > > > Key: CASSANDRA-16213 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16213 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip, Cluster/Membership >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > We see this exception around nodes crashing and trying to do a host > replacement; this error appears to be correlated around multiple node > failures. > A simplified case to trigger this is the following > *) Have a N node cluster > *) Shutdown all N nodes > *) Bring up N-1 nodes (at least 1 seed, else replace seed) > *) Host replace the N-1th node -> this will fail with the above > The reason this happens is that the N-1th node isn’t gossiping anymore, and > the existing nodes do not have its details in gossip (but have the details in > the peers table), so the host replacement fails as the node isn’t known in > gossip. > This affects all versions (tested 3.0 and trunk, assume 2.2 as well) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16192) Add more tests to cover compaction metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohamed Zafraan reassigned CASSANDRA-16192: --- Assignee: (was: Mohamed Zafraan) > Add more tests to cover compaction metrics > -- > > Key: CASSANDRA-16192 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16192 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Priority: Normal > Fix For: 4.0-beta > > > Some compaction metrics do not seems to be tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16192) Add more tests to cover compaction metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohamed Zafraan updated CASSANDRA-16192: Status: Patch Available (was: Review In Progress) > Add more tests to cover compaction metrics > -- > > Key: CASSANDRA-16192 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16192 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Assignee: Mohamed Zafraan >Priority: Normal > Fix For: 4.0-beta > > > Some compaction metrics do not seems to be tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16192) Add more tests to cover compaction metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohamed Zafraan updated CASSANDRA-16192: Attachment: (was: 0001-added-unit-tests-to-cover-compaction-metrics.patch) > Add more tests to cover compaction metrics > -- > > Key: CASSANDRA-16192 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16192 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Assignee: Mohamed Zafraan >Priority: Normal > Fix For: 4.0-beta > > > Some compaction metrics do not seems to be tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16192) Add more tests to cover compaction metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohamed Zafraan updated CASSANDRA-16192: Status: Open (was: Patch Available) > Add more tests to cover compaction metrics > -- > > Key: CASSANDRA-16192 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16192 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Assignee: Mohamed Zafraan >Priority: Normal > Fix For: 4.0-beta > > > Some compaction metrics do not seems to be tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16192) Add more tests to cover compaction metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohamed Zafraan updated CASSANDRA-16192: Test and Documentation Plan: (was: Compaction Metrics were divided into three categories for testing: 1. testSimpleCompactionMetricsForCompletedTasks: (totalCompactionsCompleted, completedTasks, bytesCompacted) 2. testCompactionMetricsForPendingTasks: (pendingTasks, pendingTasksByTableName) 3. testCompactionMetricsForFailedTasks: (compactionsAborted, compactionsReduced, sstablesDropppedFromCompactions) Test 2 and 3 make use of TestCompactionTask written in the same class to trigger metrics for workflows for pending tasks and aborted sstables during compaction. The class was written so we did not have to simulate scenarios were compactions requests were stalled or disk was full.) > Add more tests to cover compaction metrics > -- > > Key: CASSANDRA-16192 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16192 > Project: Cassandra > Issue Type: Improvement > Components: Test/unit >Reporter: Benjamin Lerer >Assignee: Mohamed Zafraan >Priority: Normal > Fix For: 4.0-beta > > Attachments: 0001-added-unit-tests-to-cover-compaction-metrics.patch > > > Some compaction metrics do not seems to be tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15969) jvm dtest execute APIs do not support collections
[ https://issues.apache.org/jira/browse/CASSANDRA-15969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohamed Zafraan reassigned CASSANDRA-15969: --- Assignee: (was: Mohamed Zafraan) > jvm dtest execute APIs do not support collections > - > > Key: CASSANDRA-15969 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15969 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: David Capwell >Priority: Normal > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0.x > > Attachments: Vote for this issue.url > > > If you use a collection type they will be transferred to the instance and we > call org.apache.cassandra.utils.ByteBufferUtil#objectToBytes to convert to > ByteBuffers; this doesn’t support collections. If you try to work around > this by converting before sending, it will fail since that method doesn’t > support ByteBuffer as input -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16275) Update python driver used by cassandra-dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-16275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16275:
Reviewers: Adam Holmberg, Michael Semb Wever (was: Adam Holmberg)
> Update python driver used by cassandra-dtest
>
>
> Key: CASSANDRA-16275
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16275
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/python
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: NA
>
>
> In order to commit CASSANDRA-15299, the python driver used by the dtests
> needs to include PYTHON-1258, support for V5 framing.
> Updating the python driver's cassandra-test branch to latest trunk causes 1
> additional dtest failure in
> {{auth_test.py::TestAuth::test_handle_corrupt_role_data}} because the
> {{ServerError}} response is now subject to the configured {{retry_policy}}.
> This means the error ultimately returned from the driver is
> {{NoHostAvailable}}, rather than {{ServerError}}.
> I'll open a dtest pr to change the expectation in the test and we can commit
> that when the cassandra-test branch is updated.
> cc [~aholmber] [~aboudreault]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16275) Update python driver used by cassandra-dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-16275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16275:
Status: Patch Available (was: Open)
I've published new Docker images
{{beobal/cassandra-testing-ubuntu1910-java11:20201130}} &
{{beobal/cassandra-testing-ubuntu1910-java11-w-dependencies:20201130}}, the
only differences being that they've been rebuilt with the latest driver
installed in the base image. In [this C*
branch|https://github.com/beobal/cassandra/tree/16275-trunk] I've updated
Circle config to pull those updated images.
[Here's|https://app.circleci.com/pipelines/github/beobal/cassandra?branch=16275-trunk]
the Circle runs from that branch. Also, an [Apache
CI|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/229/]
run against trunk using the updated image. Those are still currently running,
but the python dtest failures are due to new V5 wire format now being supported
in the driver, but not yet the server (until CASSANDRA-15299 is committed).
Something that I overlooked until just now is that all C* branches use the same
cassandra-dtest branch, and by extension the same version of the driver, but
2.2, 3.0 and 3.11 use a different docker image to trunk. So I think we'll also
have to update the Circle config for those branches too but first I need to
audit the differences.
> Update python driver used by cassandra-dtest
>
>
> Key: CASSANDRA-16275
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16275
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/python
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: NA
>
>
> In order to commit CASSANDRA-15299, the python driver used by the dtests
> needs to include PYTHON-1258, support for V5 framing.
> Updating the python driver's cassandra-test branch to latest trunk causes 1
> additional dtest failure in
> {{auth_test.py::TestAuth::test_handle_corrupt_role_data}} because the
> {{ServerError}} response is now subject to the configured {{retry_policy}}.
> This means the error ultimately returned from the driver is
> {{NoHostAvailable}}, rather than {{ServerError}}.
> I'll open a dtest pr to change the expectation in the test and we can commit
> that when the cassandra-test branch is updated.
> cc [~aholmber] [~aboudreault]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16213) Cannot replace_address /X because it doesn't exist in gossip
[ https://issues.apache.org/jira/browse/CASSANDRA-16213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240979#comment-17240979 ] David Capwell commented on CASSANDRA-16213: --- Thanks sam for the review, ill try to look closer at your changes as the original logic is no longer fresh in my head, will try to replay later today, if not then tomorrow. > Cannot replace_address /X because it doesn't exist in gossip > > > Key: CASSANDRA-16213 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16213 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip, Cluster/Membership >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > We see this exception around nodes crashing and trying to do a host > replacement; this error appears to be correlated around multiple node > failures. > A simplified case to trigger this is the following > *) Have a N node cluster > *) Shutdown all N nodes > *) Bring up N-1 nodes (at least 1 seed, else replace seed) > *) Host replace the N-1th node -> this will fail with the above > The reason this happens is that the N-1th node isn’t gossiping anymore, and > the existing nodes do not have its details in gossip (but have the details in > the peers table), so the host replacement fails as the node isn’t known in > gossip. > This affects all versions (tested 3.0 and trunk, assume 2.2 as well) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16275) Update python driver used by cassandra-dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-16275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16275:
Resolution: (was: Fixed)
Status: Open (was: Resolved)
> Update python driver used by cassandra-dtest
>
>
> Key: CASSANDRA-16275
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16275
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/python
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: NA
>
>
> In order to commit CASSANDRA-15299, the python driver used by the dtests
> needs to include PYTHON-1258, support for V5 framing.
> Updating the python driver's cassandra-test branch to latest trunk causes 1
> additional dtest failure in
> {{auth_test.py::TestAuth::test_handle_corrupt_role_data}} because the
> {{ServerError}} response is now subject to the configured {{retry_policy}}.
> This means the error ultimately returned from the driver is
> {{NoHostAvailable}}, rather than {{ServerError}}.
> I'll open a dtest pr to change the expectation in the test and we can commit
> that when the cassandra-test branch is updated.
> cc [~aholmber] [~aboudreault]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16275) Update python driver used by cassandra-dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-16275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16275:
Fix Version/s: NA
Source Control Link:
https://github.com/apache/cassandra-dtest/commit/192b70607a0c4a96f524bd5dc0c19c3a28f938f0
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Thanks, [~aholmber]. Adam has updated the {{cassandra-test}} branch of the
python driver, and I've committed this test fix. Next step is republishing CI
docker images, which is underway now.
> Update python driver used by cassandra-dtest
>
>
> Key: CASSANDRA-16275
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16275
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/python
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: NA
>
>
> In order to commit CASSANDRA-15299, the python driver used by the dtests
> needs to include PYTHON-1258, support for V5 framing.
> Updating the python driver's cassandra-test branch to latest trunk causes 1
> additional dtest failure in
> {{auth_test.py::TestAuth::test_handle_corrupt_role_data}} because the
> {{ServerError}} response is now subject to the configured {{retry_policy}}.
> This means the error ultimately returned from the driver is
> {{NoHostAvailable}}, rather than {{ServerError}}.
> I'll open a dtest pr to change the expectation in the test and we can commit
> that when the cassandra-test branch is updated.
> cc [~aholmber] [~aboudreault]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16275) Update python driver used by cassandra-dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-16275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16275:
Status: Ready to Commit (was: Review In Progress)
> Update python driver used by cassandra-dtest
>
>
> Key: CASSANDRA-16275
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16275
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/python
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
>
> In order to commit CASSANDRA-15299, the python driver used by the dtests
> needs to include PYTHON-1258, support for V5 framing.
> Updating the python driver's cassandra-test branch to latest trunk causes 1
> additional dtest failure in
> {{auth_test.py::TestAuth::test_handle_corrupt_role_data}} because the
> {{ServerError}} response is now subject to the configured {{retry_policy}}.
> This means the error ultimately returned from the driver is
> {{NoHostAvailable}}, rather than {{ServerError}}.
> I'll open a dtest pr to change the expectation in the test and we can commit
> that when the cassandra-test branch is updated.
> cc [~aholmber] [~aboudreault]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16275) Update python driver used by cassandra-dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-16275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16275:
Status: Review In Progress (was: Patch Available)
> Update python driver used by cassandra-dtest
>
>
> Key: CASSANDRA-16275
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16275
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/python
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
>
> In order to commit CASSANDRA-15299, the python driver used by the dtests
> needs to include PYTHON-1258, support for V5 framing.
> Updating the python driver's cassandra-test branch to latest trunk causes 1
> additional dtest failure in
> {{auth_test.py::TestAuth::test_handle_corrupt_role_data}} because the
> {{ServerError}} response is now subject to the configured {{retry_policy}}.
> This means the error ultimately returned from the driver is
> {{NoHostAvailable}}, rather than {{ServerError}}.
> I'll open a dtest pr to change the expectation in the test and we can commit
> that when the cassandra-test branch is updated.
> cc [~aholmber] [~aboudreault]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch trunk updated: Change expected response when role data is corrupt
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git The following commit(s) were added to refs/heads/trunk by this push: new 192b706 Change expected response when role data is corrupt 192b706 is described below commit 192b70607a0c4a96f524bd5dc0c19c3a28f938f0 Author: Sam Tunnicliffe AuthorDate: Fri Nov 13 17:36:22 2020 + Change expected response when role data is corrupt Patch by Sam Tunnicliffe; reviewed by Adam Holmberg for CASSANDRA-16275 --- auth_test.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/auth_test.py b/auth_test.py index 1061b17..6d92ff5 100644 --- a/auth_test.py +++ b/auth_test.py @@ -216,7 +216,7 @@ class TestAuth(Tester): self.fixture_dtest_setup.ignore_log_patterns = list(self.fixture_dtest_setup.ignore_log_patterns) + [ r'Invalid metadata has been detected for role bob'] -assert_exception(session, "LIST USERS", "Invalid metadata has been detected for role", expected=(ServerError)) +assert_exception(session, "LIST USERS", "Invalid metadata has been detected for role", expected=(NoHostAvailable)) try: self.get_session(user='bob', password='12345') except NoHostAvailable as e: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16180) 4.0 Quality: Coordination Test Audit
[
https://issues.apache.org/jira/browse/CASSANDRA-16180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240952#comment-17240952
]
Andres de la Peña commented on CASSANDRA-16180:
---
[~maedhroz] I have ported the Python dtests for short read protection to in-JVM
([this
commit|https://github.com/apache/cassandra/pull/801/commits/af71097f75ec17cc9dbc98176c8c012196cddefd]).
I have extended them with some extra cases and added parametrization to toggle
paging, flush, coordinator, etc., similarly to what we did for read repair in
CASSANDRA-15977. The PR for dtests marks the ported tests with the new
{{@ported_to_in_jvm}} annotation.
While writing these tests I have found what seems to be a bug in group-by
queries using paging. I have created CASSANDRA-16307 to deal with that
particular issue, trying to limit the scope of this ticket. The new tests
hitting that bug are just temporarily
[ignored|https://github.com/apache/cassandra/blob/af71097f75ec17cc9dbc98176c8c012196cddefd/test/distributed/org/apache/cassandra/distributed/test/ShortReadProtectionTest.java#L334]
in the PR, so we can enable them once we fix the bug.
Next thing is working on the upgrade tests mentioned in the previous comment.
> 4.0 Quality: Coordination Test Audit
>
>
> Key: CASSANDRA-16180
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16180
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.0
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> This is a subtask of CASSANDRA-15579 focusing on coordination.
> I think that the main reference dtest for this is
> [consistency_test.py|https://github.com/apache/cassandra-dtest/blob/master/consistency_test.py].
> We should identify which other tests cover this and identify what should be
> extended, similarly to what has been done with CASSANDRA-15977.
> [This
> document|https://docs.google.com/document/d/1BBeGKla9tIPflZ2awkyfl5Hc_FTs3QYw98LWUnb1fTQ/edit?usp=sharing]
> aims to list and describe the existing functional tests for coordination, so
> we can have a broad view of what is currently covered.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16217) Minimal 4.0 COMPACT STORAGE backport
[ https://issues.apache.org/jira/browse/CASSANDRA-16217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240903#comment-17240903 ] Ekaterina Dimitrova commented on CASSANDRA-16217: - As far as I can see from the thread, all concerns were already addressed? Is this ready for commit or I am missing something? :) > Minimal 4.0 COMPACT STORAGE backport > > > Key: CASSANDRA-16217 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16217 > Project: Cassandra > Issue Type: Bug > Components: Legacy/CQL >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > Fix For: 4.0-beta4 > > Time Spent: 20m > Remaining Estimate: 0h > > There are several behavioural changes related to compact storage, and these > differences are larger than most of us have anticipated: we first thought > there’ll be that “appearing column”, but there’s implicit nulls in > clusterings thing, and row vs column deletion. > Some of the recent issues on the subject are: CASSANDRA-16048, which allows > to ignore these differences. The other one was trying to improve user > experience of anyone still using compact storage: CASSANDRA-15811. > Easily reproducible differernces are: > (1) hidden columns show up, which breaks SELECT * queries > (2) DELETE v and UPDATE v WITH TTL would result into row removals in > non-dense compact tables (CASSANDRA-16069) > (3) INSERT allows skipping clusterings, which are filled with nulls by > default. > Some of these are tricky to support, as 15811 has shown. Anyone on OSS side > who might want to upgrade to 4.0 while still using compact storage might be > affected by being forced into one of these behaviours. > Possible solutions are to document these behaviours, or to bring back a > minimal set of COMPACT STORAGE to keep supporting these. > It looks like it is possible to leave some of the functionality related to > DENSE flag and allow it to be present in 4.0, but only for these three (and > potential related, however not direrclty visible) cases. > [~e.dimitrova] since you were working on removal on compact storage, wanted > to reassure that this is not a revert of your patch. On contrary: your patch > was instrumental in identifying the right places. > cc [~slebresne] [~aleksey] [~benedict] [~marcuse] > |[patch|https://github.com/apache/cassandra/pull/785]|[ci|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=13994-followup]| -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16307) GROUP BY queries with paging can return deleted data
Andres de la Peña created CASSANDRA-16307:
-
Summary: GROUP BY queries with paging can return deleted data
Key: CASSANDRA-16307
URL: https://issues.apache.org/jira/browse/CASSANDRA-16307
Project: Cassandra
Issue Type: Bug
Components: Consistency/Coordination
Reporter: Andres de la Peña
{{GROUP BY}} queries using paging and CL>ONE/LOCAL_ONE. This dtest reproduces
the problem:
{code:java}
try (Cluster cluster = init(Cluster.create(2)))
{
cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (pk int, ck int,
PRIMARY KEY (pk, ck))"));
ICoordinator coordinator = cluster.coordinator(1);
coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (0,
0)"), ConsistencyLevel.ALL);
coordinator.execute(withKeyspace("INSERT INTO %s.t (pk, ck) VALUES (1,
1)"), ConsistencyLevel.ALL);
cluster.get(1).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=0
AND ck=0"));
cluster.get(2).executeInternal(withKeyspace("DELETE FROM %s.t WHERE pk=1
AND ck=1"));
String query = withKeyspace("SELECT * FROM %s.t GROUP BY pk");
Iterator rows = coordinator.executeWithPaging(query,
ConsistencyLevel.ALL, 1);
assertRows(Iterators.toArray(rows, Object[].class));
}
{code}
Using a 2-node cluster and RF=2, the test inserts two partitions in both nodes.
Then it locally deletes each row in a separate node, so each node sees a
different partition alive, but reconciliation should produce no alive
partitions. However, a {{GROUP BY}} query using a page size of 1 wrongly
returns one of the rows.
This has been detected during CASSANDRA-16180, and it is probably related to
CASSANDRA-15459, which solved a similar problem for group-by queries with
limit, instead of paging.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16171) Remove Windows scripts
[ https://issues.apache.org/jira/browse/CASSANDRA-16171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240861#comment-17240861 ] Berenguer Blasi commented on CASSANDRA-16171: - [~yukim] are you ok this gets merged? we were just talking about it with [~adelapena] and wondering about it. Thx. > Remove Windows scripts > -- > > Key: CASSANDRA-16171 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16171 > Project: Cassandra > Issue Type: Task > Components: Packaging >Reporter: Yuki Morishita >Assignee: Yuki Morishita >Priority: Normal > Fix For: 4.0-rc > > Time Spent: 10m > Remaining Estimate: 0h > > As per the email thread in cassandra-dev mailing list[1], remove windows > scripts from Cassandra 4.0 onwards, due to the lack of maintenance and tests. > 1: https://www.mail-archive.com/dev@cassandra.apache.org/msg15583.html -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16306) Using limit in TWCS sorted table doesn't work
[
https://issues.apache.org/jira/browse/CASSANDRA-16306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuval Feilchenfeld updated CASSANDRA-16306:
---
Description:
Cassandra version: 3.11.6, (checked on 3.11.9 as well). A 7 nodes cluster
running on Kubernetes.
We have a table that uses to sort sessions by username and app_id. The table
uses TWCS and saves the data for 60 days as follows:
{code:java}
@cqlsh> DESCRIBE sorted_sessions_by_username;
CREATE TABLE securedtouch.sorted_sessions_by_username (
app_id text,
username text,
start_timestamp bigint,
session_id text,
PRIMARY KEY ((app_id, username), start_timestamp, session_id)
) WITH CLUSTERING ORDER BY (start_timestamp DESC, session_id ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy',
'compaction_window_size': '4', 'compaction_window_unit': 'DAYS',
'max_threshold': '32', 'min_threshold': '2'}
AND compression = {'chunk_length_in_kb': '4', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 5184000
AND gc_grace_seconds = 10800
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
{code}
With tracing on, I'm going to insert a new app_id and username into the table:
{code:java}
@cqlsh> INSERT INTO sorted_sessions_by_username (app_id, username,
start_timestamp, session_id) values ('bug_test_app_id', 'bug_test_username',
123456, 'bug_test_session_id');{code}
And now I'll query for it like that:
{code:java}
@cqlsh> SELECT * FROM sorted_sessions_by_username WHERE app_id =
'bug_test_app_id' and username = 'bug_test_username' LIMIT 1;
app_id | username | start_timestamp | session_id
-+---+-+-
bug_test_app_id | bug_test_username | 123456 | bug_test_session_id
(1 rows)
Tracing session: 4be251c0-3322-11eb-ba1d-cdd072748f27
activity | timestamp | source | source_elapsed | client
---++++---
Execute CQL3 query | 2020-11-30 15:39:55.996000 | 10.0.2.30 | 0 | 127.0.0.1
Parsing SELECT * FROM sorted_sessions_by_username WHERE app_id =
'bug_test_app_id' and username = 'bug_test_username' LIMIT 1;
[Native-Transport-Requests-7] | 2020-11-30 15:39:55.997000 | 10.0.2.30 | 678 |
127.0.0.1
Preparing statement [Native-Transport-Requests-7] | 2020-11-30 15:39:55.997000
| 10.0.2.30 | 852 | 127.0.0.1
reading data from /10.0.0.103 [Native-Transport-Requests-7] | 2020-11-30
15:39:55.997000 | 10.0.2.30 | 1126 | 127.0.0.1
speculating read retry on /10.2.42.36 [Native-Transport-Requests-7] |
2020-11-30 15:39:55.997000 | 10.0.2.30 | 1272 | 127.0.0.1
Sending READ message to
cassandra-1.cassandra.default.svc.cluster.local/10.0.0.103
[MessagingService-Outgoing-cassandra-1.cassandra.default.svc.cluster.local/10.0.0.103-Small]
| 2020-11-30 15:39:55.997000 | 10.0.2.30 | 1322 | 127.0.0.1
Sending READ message to
cassandra-4.cassandra.default.svc.cluster.local/10.2.42.36
[MessagingService-Outgoing-cassandra-4.cassandra.default.svc.cluster.local/10.2.42.36-Small]
| 2020-11-30 15:39:55.997000 | 10.0.2.30 | 1418 | 127.0.0.1
READ message received from /10.0.2.30 [MessagingService-Incoming-/10.0.2.30] |
2020-11-30 15:39:55.998000 | 10.0.0.103 | 75 | 127.0.0.1
READ message received from /10.0.2.30 [MessagingService-Incoming-/10.0.2.30] |
2020-11-30 15:39:55.998000 | 10.2.42.36 | 52 | 127.0.0.1
Executing single-partition query on sorted_sessions_by_username [ReadStage-6]
| 2020-11-30 15:39:55.998000 | 10.0.0.103 | 399 | 127.0.0.1
Executing single-partition query on sorted_sessions_by_username [ReadStage-4]
| 2020-11-30 15:39:55.998000 | 10.2.42.36 | 290 | 127.0.0.1
Acquiring sstable references [ReadStage-6] | 2020-11-30 15:39:55.998000 |
10.0.0.103 | 487 | 127.0.0.1
Acquiring sstable references [ReadStage-4] | 2020-11-30 15:39:55.998000 |
10.2.42.36 | 399 | 127.0.0.1
Bloom filter allows skipping sstable 16267 [ReadStage-6] | 2020-11-30
15:39:55.998000 | 10.0.0.103 | 584 | 127.0.0.1
Bloom filter allows skipping sstable 12935 [ReadStage-4] | 2020-11-30
15:39:55.998000 | 10.2.42.36 | 502 | 127.0.0.1
Bloom filter allows skipping sstable 16207 [ReadStage-6] | 2020-11-30
15:39:55.998000 | 10.0.0.103 | 617 | 127.0.0.1
Bloom filter allows skipping sstable 12885 [ReadStage-4] | 2020-11-30
15:39:55.998000 | 10.2.42.36 | 545 | 127.0.0.1
Bloom filter allows skipping sstable 15659
[jira] [Created] (CASSANDRA-16306) Using limit in TWCS sorted table doesn't work
Yuval Feilchenfeld created CASSANDRA-16306:
--
Summary: Using limit in TWCS sorted table doesn't work
Key: CASSANDRA-16306
URL: https://issues.apache.org/jira/browse/CASSANDRA-16306
Project: Cassandra
Issue Type: Bug
Reporter: Yuval Feilchenfeld
Cassandra version: 3.11.6, (checked on 3.11.9 as well). A 7 nodes cluster
running on Kubernetes.
We have a table that uses to sort sessions by username and app_id. The table
uses TWCS and saves the data for 60 days as follows:
{code:java}
@cqlsh> DESCRIBE sorted_sessions_by_username;
CREATE TABLE securedtouch.sorted_sessions_by_username (
app_id text,
username text,
start_timestamp bigint,
session_id text,
PRIMARY KEY ((app_id, username), start_timestamp, session_id)
) WITH CLUSTERING ORDER BY (start_timestamp DESC, session_id ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy',
'compaction_window_size': '4', 'compaction_window_unit': 'DAYS',
'max_threshold': '32', 'min_threshold': '2'}
AND compression = {'chunk_length_in_kb': '4', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 5184000
AND gc_grace_seconds = 10800
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
{code}
With tracing on, I'm going to insert a new app_id and username into the table:
{code:java}
@cqlsh> INSERT INTO sorted_sessions_by_username (app_id, username,
start_timestamp, session_id) values ('bug_test_app_id', 'bug_test_username',
123456, 'bug_test_session_id');{code}
And now I'll query for it like that:
{code:java}
sys_admin@cqlsh:securedtouch> SELECT * FROM sorted_sessions_by_username WHERE
app_id = 'bug_test_app_id' and username = 'bug_test_username' LIMIT 1;
app_id | username | start_timestamp | session_id
-+---+-+-
bug_test_app_id | bug_test_username | 123456 | bug_test_session_id
(1 rows)
Tracing session: 4be251c0-3322-11eb-ba1d-cdd072748f27
activity | timestamp | source | source_elapsed | client
---++++---
Execute CQL3 query | 2020-11-30 15:39:55.996000 | 10.0.2.30 | 0 | 127.0.0.1
Parsing SELECT * FROM sorted_sessions_by_username WHERE app_id =
'bug_test_app_id' and username = 'bug_test_username' LIMIT 1;
[Native-Transport-Requests-7] | 2020-11-30 15:39:55.997000 | 10.0.2.30 | 678 |
127.0.0.1
Preparing statement [Native-Transport-Requests-7] | 2020-11-30 15:39:55.997000
| 10.0.2.30 | 852 | 127.0.0.1
reading data from /10.0.0.103 [Native-Transport-Requests-7] | 2020-11-30
15:39:55.997000 | 10.0.2.30 | 1126 | 127.0.0.1
speculating read retry on /10.2.42.36 [Native-Transport-Requests-7] |
2020-11-30 15:39:55.997000 | 10.0.2.30 | 1272 | 127.0.0.1
Sending READ message to
cassandra-1.cassandra.default.svc.cluster.local/10.0.0.103
[MessagingService-Outgoing-cassandra-1.cassandra.default.svc.cluster.local/10.0.0.103-Small]
| 2020-11-30 15:39:55.997000 | 10.0.2.30 | 1322 | 127.0.0.1
Sending READ message to
cassandra-4.cassandra.default.svc.cluster.local/10.2.42.36
[MessagingService-Outgoing-cassandra-4.cassandra.default.svc.cluster.local/10.2.42.36-Small]
| 2020-11-30 15:39:55.997000 | 10.0.2.30 | 1418 | 127.0.0.1
READ message received from /10.0.2.30 [MessagingService-Incoming-/10.0.2.30] |
2020-11-30 15:39:55.998000 | 10.0.0.103 | 75 | 127.0.0.1
READ message received from /10.0.2.30 [MessagingService-Incoming-/10.0.2.30] |
2020-11-30 15:39:55.998000 | 10.2.42.36 | 52 | 127.0.0.1
Executing single-partition query on sorted_sessions_by_username [ReadStage-6]
| 2020-11-30 15:39:55.998000 | 10.0.0.103 | 399 | 127.0.0.1
Executing single-partition query on sorted_sessions_by_username [ReadStage-4]
| 2020-11-30 15:39:55.998000 | 10.2.42.36 | 290 | 127.0.0.1
Acquiring sstable references [ReadStage-6] | 2020-11-30 15:39:55.998000 |
10.0.0.103 | 487 | 127.0.0.1
Acquiring sstable references [ReadStage-4] | 2020-11-30 15:39:55.998000 |
10.2.42.36 | 399 | 127.0.0.1
Bloom filter allows skipping sstable 16267 [ReadStage-6] | 2020-11-30
15:39:55.998000 | 10.0.0.103 | 584 | 127.0.0.1
Bloom filter allows skipping sstable 12935 [ReadStage-4] | 2020-11-30
15:39:55.998000 | 10.2.42.36 | 502 | 127.0.0.1
Bloom filter allows skipping sstable 16207 [ReadStage-6] | 2020-11-30
15:39:55.998000 | 10.0.0.103 | 617 | 127.0.0.1
Bloom filter
[jira] [Updated] (CASSANDRA-16205) Offline token allocation strategy generator tool
[
https://issues.apache.org/jira/browse/CASSANDRA-16205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-16205:
---
Status: Review In Progress (was: Changes Suggested)
> Offline token allocation strategy generator tool
>
>
> Key: CASSANDRA-16205
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16205
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config, Local/Scripts
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>
> A command line tool to generate tokens (using the
> allocate_tokens_for_local_replication_factor algorithm) for pre-configuration
> of {{initial_tokens}} in cassandra.yaml.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16274) Improve performance when calculating StreamTasks with optimised streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-16274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16274: Status: Ready to Commit (was: Review In Progress) bq. so in a perfect setting we would stream range1 and range2 from addr1 - but if addr2 sorts before addr1 we will stream range1 from addr2 - I added a todo on the method that this can be improved Cool, makes sense. +1 > Improve performance when calculating StreamTasks with optimised streaming > - > > Key: CASSANDRA-16274 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16274 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Repair >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0-beta4 > > > The way stream tasks are calculated currently is quite inefficient, improve > that. > Also, we currently try to distribute the streaming nodes evenly, this creates > many more sstables than necessary - instead we should try to stream > everything from a single peer, this should reduce the number of sstables > created on the out-of-sync node. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16274) Improve performance when calculating StreamTasks with optimised streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-16274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16274: Status: Review In Progress (was: Patch Available) > Improve performance when calculating StreamTasks with optimised streaming > - > > Key: CASSANDRA-16274 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16274 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Repair >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0-beta4 > > > The way stream tasks are calculated currently is quite inefficient, improve > that. > Also, we currently try to distribute the streaming nodes evenly, this creates > many more sstables than necessary - instead we should try to stream > everything from a single peer, this should reduce the number of sstables > created on the out-of-sync node. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16274) Improve performance when calculating StreamTasks with optimised streaming
[
https://issues.apache.org/jira/browse/CASSANDRA-16274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-16274:
Status: Patch Available (was: In Progress)
pushed 2 commits to the branch above - one removes the duplicated
{{AsymmetricSyncRequest}} class in favour of a flag on {{SyncRequest}}, and the
second commit fixes the review comments
bq. shouldn't that same test verify that ReduceHelper::pickConsistent makes the
selections deterministic?
in the test we have:
{code}
addr3 will stream range1 from addr1 or addr2
range2 from addr1
{code}
so in a perfect setting we would stream range1 and range2 from addr1 - but if
addr2 sorts before addr1 we will stream range1 from addr2 - I added a todo on
the method that this can be improved
> Improve performance when calculating StreamTasks with optimised streaming
> -
>
> Key: CASSANDRA-16274
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16274
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Repair
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 4.0-beta4
>
>
> The way stream tasks are calculated currently is quite inefficient, improve
> that.
> Also, we currently try to distribute the streaming nodes evenly, this creates
> many more sstables than necessary - instead we should try to stream
> everything from a single peer, this should reduce the number of sstables
> created on the out-of-sync node.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16274) Improve performance when calculating StreamTasks with optimised streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-16274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-16274: Status: In Progress (was: Changes Suggested) > Improve performance when calculating StreamTasks with optimised streaming > - > > Key: CASSANDRA-16274 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16274 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Repair >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0-beta4 > > > The way stream tasks are calculated currently is quite inefficient, improve > that. > Also, we currently try to distribute the streaming nodes evenly, this creates > many more sstables than necessary - instead we should try to stream > everything from a single peer, this should reduce the number of sstables > created on the out-of-sync node. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16305) Remove duplicate code used by CQLTester and SchemaLoader
[
https://issues.apache.org/jira/browse/CASSANDRA-16305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-16305:
Reviewers: Berenguer Blasi
> Remove duplicate code used by CQLTester and SchemaLoader
>
>
> Key: CASSANDRA-16305
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16305
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 4.x
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> While digging into the cause behind CASSANDRA-16302, we discovered that there
> is some duplicate code in {{SchemaLoader}} and {{CQLTester}}. Over time that
> code has evolved slightly differently in both classes leading to some
> different behavior.
> Having the same code being used by both class will ensure that if the code
> needs to be change the behavior will be correct same one for both classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16305) Remove duplicate code used by CQLTester and SchemaLoader
[
https://issues.apache.org/jira/browse/CASSANDRA-16305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-16305:
---
Test and Documentation Plan: The refactoring did not break any tests.
Status: Patch Available (was: Open)
[PR|https://github.com/apache/cassandra/pull/840] CI results
[j8|https://app.circleci.com/pipelines/github/blerer/cassandra/77/workflows/704547fd-e4c0-44ed-94ef-92b0285849bb],
[j11|https://app.circleci.com/pipelines/github/blerer/cassandra/77/workflows/559f3fe8-ed34-45de-b801-092d7b9f4352]
> Remove duplicate code used by CQLTester and SchemaLoader
>
>
> Key: CASSANDRA-16305
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16305
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 4.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While digging into the cause behind CASSANDRA-16302, we discovered that there
> is some duplicate code in {{SchemaLoader}} and {{CQLTester}}. Over time that
> code has evolved slightly differently in both classes leading to some
> different behavior.
> Having the same code being used by both class will ensure that if the code
> needs to be change the behavior will be correct same one for both classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16305) Remove duplicate code used by CQLTester and SchemaLoader
[
https://issues.apache.org/jira/browse/CASSANDRA-16305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-16305:
---
Change Category: Quality Assurance
Complexity: Low Hanging Fruit
Fix Version/s: 4.x
Status: Open (was: Triage Needed)
> Remove duplicate code used by CQLTester and SchemaLoader
>
>
> Key: CASSANDRA-16305
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16305
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 4.x
>
>
> While digging into the cause behind CASSANDRA-16302, we discovered that there
> is some duplicate code in {{SchemaLoader}} and {{CQLTester}}. Over time that
> code has evolved slightly differently in both classes leading to some
> different behavior.
> Having the same code being used by both class will ensure that if the code
> needs to be change the behavior will be correct same one for both classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16250) LongSharedExecutorPoolTest.testPromptnessOfExecution burn test is flaky
[
https://issues.apache.org/jira/browse/CASSANDRA-16250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-16250:
---
Resolution: (was: Duplicate)
Status: Open (was: Resolved)
> LongSharedExecutorPoolTest.testPromptnessOfExecution burn test is flaky
> ---
>
> Key: CASSANDRA-16250
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16250
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/burn
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
>
> Within the burn tests
> {{LongSharedExecutorPoolTest.testPromptnessOfExecution}} fail regularily with
> the following stacktrace:
> {code}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.concurrent.LongSharedExecutorPoolTest.testPromptnessOfExecution(LongSharedExecutorPoolTest.java:213)
> at
> org.apache.cassandra.concurrent.LongSharedExecutorPoolTest.testPromptnessOfExecution(LongSharedExecutorPoolTest.java:102)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16250) LongSharedExecutorPoolTest.testPromptnessOfExecution burn test is flaky
[
https://issues.apache.org/jira/browse/CASSANDRA-16250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-16250:
---
Resolution: Duplicate
Status: Resolved (was: Open)
> LongSharedExecutorPoolTest.testPromptnessOfExecution burn test is flaky
> ---
>
> Key: CASSANDRA-16250
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16250
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/burn
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
>
> Within the burn tests
> {{LongSharedExecutorPoolTest.testPromptnessOfExecution}} fail regularily with
> the following stacktrace:
> {code}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.concurrent.LongSharedExecutorPoolTest.testPromptnessOfExecution(LongSharedExecutorPoolTest.java:213)
> at
> org.apache.cassandra.concurrent.LongSharedExecutorPoolTest.testPromptnessOfExecution(LongSharedExecutorPoolTest.java:102)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16205) Offline token allocation strategy generator tool
[
https://issues.apache.org/jira/browse/CASSANDRA-16205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta updated CASSANDRA-16205:
Status: Changes Suggested (was: Review In Progress)
> Offline token allocation strategy generator tool
>
>
> Key: CASSANDRA-16205
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16205
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config, Local/Scripts
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>
> A command line tool to generate tokens (using the
> allocate_tokens_for_local_replication_factor algorithm) for pre-configuration
> of {{initial_tokens}} in cassandra.yaml.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16205) Offline token allocation strategy generator tool
[
https://issues.apache.org/jira/browse/CASSANDRA-16205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta updated CASSANDRA-16205:
Status: Review In Progress (was: Patch Available)
> Offline token allocation strategy generator tool
>
>
> Key: CASSANDRA-16205
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16205
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config, Local/Scripts
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>
> A command line tool to generate tokens (using the
> allocate_tokens_for_local_replication_factor algorithm) for pre-configuration
> of {{initial_tokens}} in cassandra.yaml.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16271) Writes timeout instead of failing on cluster with CL-1 replicas available during replace
[
https://issues.apache.org/jira/browse/CASSANDRA-16271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta updated CASSANDRA-16271:
Status: Changes Suggested (was: Review In Progress)
> Writes timeout instead of failing on cluster with CL-1 replicas available
> during replace
>
>
> Key: CASSANDRA-16271
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16271
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Krishna Vadali
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Attachments: sleep_before_replace.diff
>
>
> Writes timeout instead of failing on cluster with CL-1 replicas available
> during replace node operation.
> With Consistency Level ALL, we are observing Timeout exceptions during writes
> when (RF - 1) nodes are available in the cluster with one replace-node
> operation running. The coordinator is expecting RF + 1 responses, while there
> are only RF nodes (RF-1 nodes in UN and 1 node in UJ) are available in the
> cluster, hence timing out.
> The same problem happens on a keyspace with RF=1, CL=ONE and one replica
> being replaced. Also RF=3, CL=QUORUM, one replica down and another being
> replaced.
> I believe the expected behavior is that the write should fail with
> UnavailableException since there are not enough NORMAL replicas to fulfill
> the request.
> h4. *Steps to reproduce:*
> Run a 3 node test cluster (call the nodes node1 (127.0.0.1), node2
> (127.0.0.2), node3 (127.0.0.3)):
> {code:java}
> ccm create test -v 3.11.3 -n 3 -s
> {code}
> Create test keyspaces with RF = 3 and RF = 1 respectively:
> {code:java}
> create keyspace rf3 with replication = \{'class': 'SimpleStrategy',
> 'replication_factor': 3};
> create keyspace rf1 with replication = \{'class': 'SimpleStrategy',
> 'replication_factor': 1};
> {code}
> Create a table test in both the keyspaces:
> {code:java}
> create table rf3.test ( pk int primary KEY, value int);
> create table rf1.test ( pk int primary KEY, value int);
> {code}
> Stop node node2:
> {code:java}
> ccm node2 stop
> {code}
> Create node node4:
> {code:java}
> ccm add node4 -i 127.0.0.4
> {code}
> Enable auto_bootstrap
> {code:java}
> ccm node4 updateconf 'auto_bootstrap: true'
> {code}
> Ensure node4 does not have itself in its seeds list.
> Run a replace node to replace node2 (address 127.0.0.2 corresponds to node
> node2)
> {code:java}
> ccm node4 start --jvm_arg="-Dcassandra.replace_address=127.0.0.2"
> {code}
> When the replace node is running, perform write/reads with CONSISTENCY ALL,
> we observed TimeoutException.
> {code:java}
> SET CONSISTENCY ALL:SET CONSISTENCY ALL:
> cqlsh> insert into rf3.test (pk, value) values (16, 7);
> WriteTimeout: Error from server: code=1100 [Coordinator node timed out
> waiting for replica nodes' responses] message="Operation timed out - received
> only 3 responses." info=\{'received_responses': 3, 'required_responses': 4,
> 'consistency': 'ALL'}{code}
> {code:java}
> cqlsh> CONSISTENCY ONE;
> cqlsh> insert into rf1.test (pk, value) VALUES(5, 1);
> WriteTimeout: Error from server: code=1100 [Coordinator node timed out
> waiting for replica nodes' responses] message="Operation timed out - received
> only 1 responses." info=\{'received_responses': 1, 'required_responses': 2,
> 'consistency': 'ONE'}
> {code}
> Cluster State:
> {code:java}
> Datacenter: datacenter1
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- AddressLoad Tokens Owns (effective) Host ID
> Rack
> UN 127.0.0.1 70.45 KiB 1100.0%
> 4f652b22-045b-493b-8722-fb5f7e1723ce rack1
> UN 127.0.0.3 70.43 KiB 1100.0%
> a0dcd677-bdb3-4947-b9a7-14f3686a709f rack1
> UJ 127.0.0.4 137.47 KiB 1?
> e3d794f1-081e-4aba-94f2-31950c713846 rack1
> {code}
> Note:
> We introduced sleep during replace operation in order to simulate do our
> experiments. We attached code diff that does it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16271) Writes timeout instead of failing on cluster with CL-1 replicas available during replace
[
https://issues.apache.org/jira/browse/CASSANDRA-16271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240697#comment-17240697
]
Paulo Motta commented on CASSANDRA-16271:
-
Thanks for the patch Sam!
I think we don't need to check pending replicas on
{{assureSufficientLiveReplicas}}, they are only needed to ensure one extra
replica is acknowledged on {{CL.blockForWrite}} but not to throw
{{UnavailableException}} when not enough natural replicas are available. I did
this approach for trunk [on this
commit|https://github.com/pauloricardomg/cassandra/commit/4027b8614ffd898b83e547fa0e49352a8aa8a739]
and the tests are passing, please let me know what do you think.
I think this should simplify the 3.X patches as we don't need to filter live
replicas on the writehandlers.
I really like the unit tests, but I personally prefer having them as individual
unit tests rather a single one testing many cases (or maybe grouped in smaller
units), since it makes it easier to spot which test fails on CI. Feel free to
keep it if you prefer it this way. Also, can you add a regression dtest for the
scenarios in the ticket description?
> Writes timeout instead of failing on cluster with CL-1 replicas available
> during replace
>
>
> Key: CASSANDRA-16271
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16271
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Krishna Vadali
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Attachments: sleep_before_replace.diff
>
>
> Writes timeout instead of failing on cluster with CL-1 replicas available
> during replace node operation.
> With Consistency Level ALL, we are observing Timeout exceptions during writes
> when (RF - 1) nodes are available in the cluster with one replace-node
> operation running. The coordinator is expecting RF + 1 responses, while there
> are only RF nodes (RF-1 nodes in UN and 1 node in UJ) are available in the
> cluster, hence timing out.
> The same problem happens on a keyspace with RF=1, CL=ONE and one replica
> being replaced. Also RF=3, CL=QUORUM, one replica down and another being
> replaced.
> I believe the expected behavior is that the write should fail with
> UnavailableException since there are not enough NORMAL replicas to fulfill
> the request.
> h4. *Steps to reproduce:*
> Run a 3 node test cluster (call the nodes node1 (127.0.0.1), node2
> (127.0.0.2), node3 (127.0.0.3)):
> {code:java}
> ccm create test -v 3.11.3 -n 3 -s
> {code}
> Create test keyspaces with RF = 3 and RF = 1 respectively:
> {code:java}
> create keyspace rf3 with replication = \{'class': 'SimpleStrategy',
> 'replication_factor': 3};
> create keyspace rf1 with replication = \{'class': 'SimpleStrategy',
> 'replication_factor': 1};
> {code}
> Create a table test in both the keyspaces:
> {code:java}
> create table rf3.test ( pk int primary KEY, value int);
> create table rf1.test ( pk int primary KEY, value int);
> {code}
> Stop node node2:
> {code:java}
> ccm node2 stop
> {code}
> Create node node4:
> {code:java}
> ccm add node4 -i 127.0.0.4
> {code}
> Enable auto_bootstrap
> {code:java}
> ccm node4 updateconf 'auto_bootstrap: true'
> {code}
> Ensure node4 does not have itself in its seeds list.
> Run a replace node to replace node2 (address 127.0.0.2 corresponds to node
> node2)
> {code:java}
> ccm node4 start --jvm_arg="-Dcassandra.replace_address=127.0.0.2"
> {code}
> When the replace node is running, perform write/reads with CONSISTENCY ALL,
> we observed TimeoutException.
> {code:java}
> SET CONSISTENCY ALL:SET CONSISTENCY ALL:
> cqlsh> insert into rf3.test (pk, value) values (16, 7);
> WriteTimeout: Error from server: code=1100 [Coordinator node timed out
> waiting for replica nodes' responses] message="Operation timed out - received
> only 3 responses." info=\{'received_responses': 3, 'required_responses': 4,
> 'consistency': 'ALL'}{code}
> {code:java}
> cqlsh> CONSISTENCY ONE;
> cqlsh> insert into rf1.test (pk, value) VALUES(5, 1);
> WriteTimeout: Error from server: code=1100 [Coordinator node timed out
> waiting for replica nodes' responses] message="Operation timed out - received
> only 1 responses." info=\{'received_responses': 1, 'required_responses': 2,
> 'consistency': 'ONE'}
> {code}
> Cluster State:
> {code:java}
> Datacenter: datacenter1
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- AddressLoad Tokens Owns (effective) Host ID
> Rack
> UN 127.0.0.1 70.45 KiB 1100.0%
> 4f652b22-045b-493b-8722-fb5f7e1723ce rack1
> UN 127.0.0.3 70.43 KiB 1100.0%
>
[jira] [Updated] (CASSANDRA-16271) Writes timeout instead of failing on cluster with CL-1 replicas available during replace
[
https://issues.apache.org/jira/browse/CASSANDRA-16271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta updated CASSANDRA-16271:
Reviewers: Krishna Vadali, Paulo Motta, Paulo Motta (was: Krishna Vadali,
Paulo Motta)
Krishna Vadali, Paulo Motta, Paulo Motta (was: Krishna Vadali,
Paulo Motta)
Status: Review In Progress (was: Patch Available)
> Writes timeout instead of failing on cluster with CL-1 replicas available
> during replace
>
>
> Key: CASSANDRA-16271
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16271
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Krishna Vadali
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Attachments: sleep_before_replace.diff
>
>
> Writes timeout instead of failing on cluster with CL-1 replicas available
> during replace node operation.
> With Consistency Level ALL, we are observing Timeout exceptions during writes
> when (RF - 1) nodes are available in the cluster with one replace-node
> operation running. The coordinator is expecting RF + 1 responses, while there
> are only RF nodes (RF-1 nodes in UN and 1 node in UJ) are available in the
> cluster, hence timing out.
> The same problem happens on a keyspace with RF=1, CL=ONE and one replica
> being replaced. Also RF=3, CL=QUORUM, one replica down and another being
> replaced.
> I believe the expected behavior is that the write should fail with
> UnavailableException since there are not enough NORMAL replicas to fulfill
> the request.
> h4. *Steps to reproduce:*
> Run a 3 node test cluster (call the nodes node1 (127.0.0.1), node2
> (127.0.0.2), node3 (127.0.0.3)):
> {code:java}
> ccm create test -v 3.11.3 -n 3 -s
> {code}
> Create test keyspaces with RF = 3 and RF = 1 respectively:
> {code:java}
> create keyspace rf3 with replication = \{'class': 'SimpleStrategy',
> 'replication_factor': 3};
> create keyspace rf1 with replication = \{'class': 'SimpleStrategy',
> 'replication_factor': 1};
> {code}
> Create a table test in both the keyspaces:
> {code:java}
> create table rf3.test ( pk int primary KEY, value int);
> create table rf1.test ( pk int primary KEY, value int);
> {code}
> Stop node node2:
> {code:java}
> ccm node2 stop
> {code}
> Create node node4:
> {code:java}
> ccm add node4 -i 127.0.0.4
> {code}
> Enable auto_bootstrap
> {code:java}
> ccm node4 updateconf 'auto_bootstrap: true'
> {code}
> Ensure node4 does not have itself in its seeds list.
> Run a replace node to replace node2 (address 127.0.0.2 corresponds to node
> node2)
> {code:java}
> ccm node4 start --jvm_arg="-Dcassandra.replace_address=127.0.0.2"
> {code}
> When the replace node is running, perform write/reads with CONSISTENCY ALL,
> we observed TimeoutException.
> {code:java}
> SET CONSISTENCY ALL:SET CONSISTENCY ALL:
> cqlsh> insert into rf3.test (pk, value) values (16, 7);
> WriteTimeout: Error from server: code=1100 [Coordinator node timed out
> waiting for replica nodes' responses] message="Operation timed out - received
> only 3 responses." info=\{'received_responses': 3, 'required_responses': 4,
> 'consistency': 'ALL'}{code}
> {code:java}
> cqlsh> CONSISTENCY ONE;
> cqlsh> insert into rf1.test (pk, value) VALUES(5, 1);
> WriteTimeout: Error from server: code=1100 [Coordinator node timed out
> waiting for replica nodes' responses] message="Operation timed out - received
> only 1 responses." info=\{'received_responses': 1, 'required_responses': 2,
> 'consistency': 'ONE'}
> {code}
> Cluster State:
> {code:java}
> Datacenter: datacenter1
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- AddressLoad Tokens Owns (effective) Host ID
> Rack
> UN 127.0.0.1 70.45 KiB 1100.0%
> 4f652b22-045b-493b-8722-fb5f7e1723ce rack1
> UN 127.0.0.3 70.43 KiB 1100.0%
> a0dcd677-bdb3-4947-b9a7-14f3686a709f rack1
> UJ 127.0.0.4 137.47 KiB 1?
> e3d794f1-081e-4aba-94f2-31950c713846 rack1
> {code}
> Note:
> We introduced sleep during replace operation in order to simulate do our
> experiments. We attached code diff that does it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16302) HintsWriteThenReadTest & LongStreamingTest flaky
[ https://issues.apache.org/jira/browse/CASSANDRA-16302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-16302: --- Since Version: 4.0-beta3 Source Control Link: https://github.com/apache/cassandra/commit/a7f50c9a693453d548bd464943fb0a3a620e9607 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed into 3.11 as a7f50c9a693453d548bd464943fb0a3a620e9607 and merged into trunk. > HintsWriteThenReadTest & LongStreamingTest flaky > > > Key: CASSANDRA-16302 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16302 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0 > > Time Spent: 2h > Remaining Estimate: 0h > > Flaky tests for the last > [100%|https://ci-cassandra.apache.org/job/Cassandra-trunk/167/#showFailuresLink] > runs -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16302) HintsWriteThenReadTest & LongStreamingTest flaky
[ https://issues.apache.org/jira/browse/CASSANDRA-16302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-16302: --- Status: Ready to Commit (was: Review In Progress) > HintsWriteThenReadTest & LongStreamingTest flaky > > > Key: CASSANDRA-16302 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16302 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0 > > Time Spent: 2h > Remaining Estimate: 0h > > Flaky tests for the last > [100%|https://ci-cassandra.apache.org/job/Cassandra-trunk/167/#showFailuresLink] > runs -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14688) Update protocol spec and class level doc with protocol checksumming details
[
https://issues.apache.org/jira/browse/CASSANDRA-14688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-14688:
Status: Open (was: Patch Available)
> Update protocol spec and class level doc with protocol checksumming details
> ---
>
> Key: CASSANDRA-14688
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14688
> Project: Cassandra
> Issue Type: Task
> Components: Legacy/Documentation and Website
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-rc
>
>
> CASSANDRA-13304 provides an option to add checksumming to the frame body of
> native protocol messages. The native protocol spec needs to be updated to
> reflect this ASAP. We should also verify that the javadoc comments describing
> the on-wire format in
> {{o.a.c.transport.frame.checksum.ChecksummingTransformer}} are up to date.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14688) Update protocol spec and class level doc with protocol checksumming details
[
https://issues.apache.org/jira/browse/CASSANDRA-14688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-14688:
Status: Patch Available (was: Ready to Commit)
Following CASSANDRA-15299 the implementation has changed significantly, so this
needs to be revisited. It should include updating the protocol spec itself as
well as a suitable NEWS.txt entry.
> Update protocol spec and class level doc with protocol checksumming details
> ---
>
> Key: CASSANDRA-14688
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14688
> Project: Cassandra
> Issue Type: Task
> Components: Legacy/Documentation and Website
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-rc
>
>
> CASSANDRA-13304 provides an option to add checksumming to the frame body of
> native protocol messages. The native protocol spec needs to be updated to
> reflect this ASAP. We should also verify that the javadoc comments describing
> the on-wire format in
> {{o.a.c.transport.frame.checksum.ChecksummingTransformer}} are up to date.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch cassandra-3.11 into trunk
This is an automated email from the ASF dual-hosted git repository. blerer pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 3757a7327bbeb9fb6591532547aeeb4d4c62dcc4 Merge: b470487 a7f50c9 Author: Benjamin Lerer AuthorDate: Mon Nov 30 11:57:53 2020 +0100 Merge branch cassandra-3.11 into trunk - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (b470487 -> 3757a73)
This is an automated email from the ASF dual-hosted git repository. blerer pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from b470487 Merge branch cassandra-3.11 into trunk add a7f50c9 Fix HintsWriteThenReadTest and LongStreamingTest new 3757a73 Merge branch cassandra-3.11 into trunk The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (080280d -> a7f50c9)
This is an automated email from the ASF dual-hosted git repository. blerer pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 080280d Merge commit '2d0b16804785660e8515aca9944784fb3733c619' into cassandra-3.11 add a7f50c9 Fix HintsWriteThenReadTest and LongStreamingTest No new revisions were added by this update. Summary of changes: .../apache/cassandra/db/commitlog/AbstractCommitLogSegmentManager.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 02/02: Merge branch cassandra-3.11 into trunk
This is an automated email from the ASF dual-hosted git repository.
blerer pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit b47048797f2d377c0e766a1861c3eef6f197881a
Merge: 9a3ca00 2807c7a
Author: Benjamin Lerer
AuthorDate: Mon Nov 30 11:42:26 2020 +0100
Merge branch cassandra-3.11 into trunk
.../commitlog/AbstractCommitLogSegmentManager.java | 3 ++-
.../cassandra/hints/HintsWriteThenReadTest.java| 24 ++
2 files changed, 18 insertions(+), 9 deletions(-)
diff --cc test/long/org/apache/cassandra/hints/HintsWriteThenReadTest.java
index 7970d59,fd880cb..24a9a78
--- a/test/long/org/apache/cassandra/hints/HintsWriteThenReadTest.java
+++ b/test/long/org/apache/cassandra/hints/HintsWriteThenReadTest.java
@@@ -28,25 -28,25 +28,25 @@@ import java.util.concurrent.TimeUnit
import java.util.zip.CRC32;
import com.google.common.collect.Iterables;
++
import org.junit.Test;
import org.apache.cassandra.SchemaLoader;
- import org.apache.cassandra.schema.TableMetadata;
- import org.apache.cassandra.schema.Schema;
-import org.apache.cassandra.config.CFMetaData;
-import org.apache.cassandra.config.Schema;
import org.apache.cassandra.db.Mutation;
import org.apache.cassandra.db.RowUpdateBuilder;
import org.apache.cassandra.db.rows.Cell;
import org.apache.cassandra.db.rows.Row;
import org.apache.cassandra.io.util.FileUtils;
import org.apache.cassandra.schema.KeyspaceParams;
++import org.apache.cassandra.schema.Schema;
++import org.apache.cassandra.schema.TableMetadata;
import org.apache.cassandra.utils.FBUtilities;
--import static junit.framework.Assert.assertEquals;
--import static junit.framework.Assert.assertNotNull;
--import static junit.framework.Assert.assertTrue;
--
import static org.apache.cassandra.Util.dk;
import static org.apache.cassandra.utils.ByteBufferUtil.bytes;
++import static org.junit.Assert.assertEquals;
++import static org.junit.Assert.assertNotNull;
++import static org.junit.Assert.assertTrue;
public class HintsWriteThenReadTest
{
@@@ -129,10 -129,10 +129,10 @@@
Row row =
mutation.getPartitionUpdates().iterator().next().iterator().next();
assertEquals(1, Iterables.size(row.cells()));
--assertEquals(bytes(index), row.clustering().get(0));
-Cell cell = row.cells().iterator().next();
++assertEquals(bytes(index),
toByteBuffer(row.clustering().get(0)));
+Cell cell = row.cells().iterator().next();
assertNotNull(cell);
--assertEquals(bytes(index), cell.value());
++assertEquals(bytes(index), toByteBuffer(cell.value()));
assertEquals(timestamp * 1000, cell.timestamp());
index++;
@@@ -188,4 -188,4 +188,12 @@@
return (int) crc.getValue();
}
++
++private ByteBuffer toByteBuffer(Object obj)
++{
++if (obj instanceof ByteBuffer)
++ return (ByteBuffer) obj;
++
++return ByteBuffer.wrap((byte[]) obj);
++}
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/02: Fix HintsWriteThenReadTest and LongStreamingTest
This is an automated email from the ASF dual-hosted git repository. blerer pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 2807c7a5a3e65bfa371f2254aa0510ee8684b7ed Author: Bereng AuthorDate: Fri Nov 27 06:31:00 2020 +0100 Fix HintsWriteThenReadTest and LongStreamingTest patch by Berenguer Blasi; reviewed by Benjamin Lerer for CASSANDRA-16302 --- .../apache/cassandra/db/commitlog/AbstractCommitLogSegmentManager.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/java/org/apache/cassandra/db/commitlog/AbstractCommitLogSegmentManager.java b/src/java/org/apache/cassandra/db/commitlog/AbstractCommitLogSegmentManager.java index 12dec60..df2e4f3 100755 --- a/src/java/org/apache/cassandra/db/commitlog/AbstractCommitLogSegmentManager.java +++ b/src/java/org/apache/cassandra/db/commitlog/AbstractCommitLogSegmentManager.java @@ -494,7 +494,8 @@ public abstract class AbstractCommitLogSegmentManager for (CommitLogSegment segment : activeSegments) segment.close(); -bufferPool.emptyBufferPool(); +if (bufferPool != null) +bufferPool.emptyBufferPool(); } /** - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (9a3ca00 -> b470487)
This is an automated email from the ASF dual-hosted git repository. blerer pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 9a3ca00 Merge commit '080280dc0177da6176dd4ba970e5a35aa7e2a729' into trunk new 2807c7a Fix HintsWriteThenReadTest and LongStreamingTest new b470487 Merge branch cassandra-3.11 into trunk The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../commitlog/AbstractCommitLogSegmentManager.java | 3 ++- .../cassandra/hints/HintsWriteThenReadTest.java| 24 ++ 2 files changed, 18 insertions(+), 9 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16274) Improve performance when calculating StreamTasks with optimised streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-16274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-16274: --- Reviewers: Sam Tunnicliffe (was: Benjamin Lerer, Sam Tunnicliffe) > Improve performance when calculating StreamTasks with optimised streaming > - > > Key: CASSANDRA-16274 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16274 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Repair >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0-beta4 > > > The way stream tasks are calculated currently is quite inefficient, improve > that. > Also, we currently try to distribute the streaming nodes evenly, this creates > many more sstables than necessary - instead we should try to stream > everything from a single peer, this should reduce the number of sstables > created on the out-of-sync node. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16274) Improve performance when calculating StreamTasks with optimised streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-16274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240635#comment-17240635 ] Benjamin Lerer commented on CASSANDRA-16274: I am lagging behind on my reviews and I do not want to hold that ticket back if [~samt] has already reviewed it (I can hardly do better than him). Removing myself from the reviewers. > Improve performance when calculating StreamTasks with optimised streaming > - > > Key: CASSANDRA-16274 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16274 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Repair >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0-beta4 > > > The way stream tasks are calculated currently is quite inefficient, improve > that. > Also, we currently try to distribute the streaming nodes evenly, this creates > many more sstables than necessary - instead we should try to stream > everything from a single peer, this should reduce the number of sstables > created on the out-of-sync node. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
