[jira] [Created] (CASSANDRA-15795) Cannot read data from a 3-node cluster which has two nodes down

2020-05-06 Thread YCozy (Jira)
YCozy created CASSANDRA-15795:
-

 Summary: Cannot read data from a 3-node cluster which has two 
nodes down
 Key: CASSANDRA-15795
 URL: https://issues.apache.org/jira/browse/CASSANDRA-15795
 Project: Cassandra
  Issue Type: Bug
  Components: Cluster/Gossip, Consistency/Coordination
Reporter: YCozy


I start up a 3 nodes cluster, and write a row with 'replication_factor' : '3'. 
The consistency level is ONE.
Then I kill two nodes, and try to get the row that I just inserted by cqlsh.
But cqlsh returns NoHostAvailable.
My Cassandra version is 3.11.6.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15748) Provide ability to configure IAuditLogger

2020-05-06 Thread Stefan Miklosovic (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Miklosovic updated CASSANDRA-15748:
--
Test and Documentation Plan:     
 Status: Patch Available  (was: Open)

https://github.com/apache/cassandra/pull/555

> Provide ability to configure IAuditLogger
> -
>
> Key: CASSANDRA-15748
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15748
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Other
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Currently there is a parameterless constructor expected for IAuditLogger 
> instances. There is not any way how to "inject" configuration properties from 
> cassandra.yaml into these concrete classes. This would be very handy for 
> custom IAuditLogger implementations.
>  
> PR: [https://github.com/apache/cassandra/pull/555]
> [|https://github.com/smiklosovic/cassandra/tree/CASSANDRA-15748]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15158) Wait for schema agreement rather then in flight schema requests when bootstrapping

2020-05-06 Thread Stefan Miklosovic (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Miklosovic updated CASSANDRA-15158:
--
Authors: Stefan Miklosovic  (was: Ben Bromhead)
Test and Documentation Plan: There are unit tests as part of PR testing 
this feature.
 Status: Patch Available  (was: Open)

for 3.11

 

[https://github.com/smiklosovic/cassandra/tree/CASSANDRA-15158]

 

for 3.0

 

https://github.com/smiklosovic/cassandra/tree/CASSANDRA-15158-cassandra-3

> Wait for schema agreement rather then in flight schema requests when 
> bootstrapping
> --
>
> Key: CASSANDRA-15158
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15158
> Project: Cassandra
>  Issue Type: Bug
>  Components: Cluster/Gossip, Cluster/Schema
>Reporter: Vincent White
>Assignee: Ben Bromhead
>Priority: Normal
>
> Currently when a node is bootstrapping we use a set of latches 
> (org.apache.cassandra.service.MigrationTask#inflightTasks) to keep track of 
> in-flight schema pull requests, and we don't proceed with 
> bootstrapping/stream until all the latches are released (or we timeout 
> waiting for each one). One issue with this is that if we have a large schema, 
> or the retrieval of the schema from the other nodes was unexpectedly slow 
> then we have no explicit check in place to ensure we have actually received a 
> schema before we proceed.
> While it's possible to increase "migration_task_wait_in_seconds" to force the 
> node to wait on each latche longer, there are cases where this doesn't help 
> because the callbacks for the schema pull requests have expired off the 
> messaging service's callback map 
> (org.apache.cassandra.net.MessagingService#callbacks) after 
> request_timeout_in_ms (default 10 seconds) before the other nodes were able 
> to respond to the new node.
> This patch checks for schema agreement between the bootstrapping node and the 
> rest of the live nodes before proceeding with bootstrapping. It also adds a 
> check to prevent the new node from flooding existing nodes with simultaneous 
> schema pull requests as can happen in large clusters.
> Removing the latch system should also prevent new nodes in large clusters 
> getting stuck for extended amounts of time as they wait 
> `migration_task_wait_in_seconds` on each of the latches left orphaned by the 
> timed out callbacks.
>  
> ||3.11||
> |[PoC|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:check_for_schema]|
> |[dtest|https://github.com/apache/cassandra-dtest/compare/master...vincewhite:wait_for_schema_agreement]|
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15406) Add command to show the progress of data streaming and index build

2020-05-06 Thread Stefan Miklosovic (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Miklosovic updated CASSANDRA-15406:
--
Test and Documentation Plan: https://github.com/apache/cassandra/pull/558
 Status: Patch Available  (was: Open)

https://github.com/apache/cassandra/pull/558

> Add command to show the progress of data streaming and index build 
> ---
>
> Key: CASSANDRA-15406
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15406
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Consistency/Streaming, Legacy/Streaming and Messaging, 
> Tool/nodetool
>Reporter: maxwellguo
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0, 4.x
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I found that we should supply a command to show the progress of streaming 
> when we do the operation of bootstrap/move/decommission/removenode. For when 
> do data streaming , noboday knows which steps there program are in , so I 
> think a command to show the joing/leaving node's is needed .
>  
> PR [https://github.com/apache/cassandra/pull/558]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRASC-14) set up sidecar packaging

2020-05-06 Thread Jon Haddad (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRASC-14?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jon Haddad updated CASSANDRASC-14:
--
Reviewers: Dinesh Joshi, T Jake Luciani  (was: Jon Haddad)

> set up sidecar packaging
> 
>
> Key: CASSANDRASC-14
> URL: https://issues.apache.org/jira/browse/CASSANDRASC-14
> Project: Sidecar for Apache Cassandra
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jon Haddad
>Assignee: Jon Haddad
>Priority: Normal
> Fix For: 1.0
>
>
> Sidecar will eventually need the ability to generate RPMs, DEB packages and 
> Docker images.  Using the ospackage and jib Gradle plugins makes this pretty 
> easy and will make adoption easier.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-15794) Upgraded C* (4.x) fail to start because of Compact Tables & downgraded C* (3.11.4) fail to start because of commit log

2020-05-06 Thread Zhuqi Jin (Jira)
Zhuqi Jin created CASSANDRA-15794:
-

 Summary: Upgraded C* (4.x) fail to start because of Compact Tables 
& downgraded C* (3.11.4) fail to start because of commit log
 Key: CASSANDRA-15794
 URL: https://issues.apache.org/jira/browse/CASSANDRA-15794
 Project: Cassandra
  Issue Type: Bug
Reporter: Zhuqi Jin


We tried to test upgrading a 3.11.4 C* cluster to 4.x and run into the 
following problems. 
 * We started a single 3.11.4 C* node. 
 * We ran cassandra-stress like this

{code:java}
./cassandra-stress write n = 30 -rate threads = 10 -node  172.17.0.2 {code}
 * We stopped this node, and started a C* node running C* compiled from trunk 
(git commit: e394dc0bb32f612a476269010930c617dd1ed3cb)
 * New C* failed to start with the following error message

{code:java}
ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 - Error while 
loading schema: ERROR [main] 2020-05-07 00:58:18,503 CassandraDaemon.java:245 - 
Error while loading schema: java.lang.IllegalArgumentException: Compact Tables 
are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... DROP 
COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to migrate off 
Compact Storage. at 
org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965) 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924) 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
 at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
 at 
org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
 at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at 
org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653) 
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770)Exception
 (java.lang.IllegalArgumentException) encountered during startup: Compact 
Tables are not allowed in Cassandra starting with 4.0 version. Use `ALTER ... 
DROP COMPACT STORAGE` command supplied in 3.x/3.11 Cassandra in order to 
migrate off Compact Storage.ERROR [main] 2020-05-07 00:58:18,520 
CassandraDaemon.java:792 - Exception encountered during 
startupjava.lang.IllegalArgumentException: Compact Tables are not allowed in 
Cassandra starting with 4.0 version. Use `ALTER ... DROP COMPACT STORAGE` 
command supplied in 3.x/3.11 Cassandra in order to migrate off Compact Storage. 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:965) 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:924) 
at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:883)
 at 
org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:874)
 at 
org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:862)
 at org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:102) at 
org.apache.cassandra.schema.Schema.loadFromDisk(Schema.java:91) at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:653) 
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:770){code}
 * We stopped the trunk version C* and started the 3.11.4 version C*. 
 * 3.11.4 C* failed to start with the following error messages: 

{code:java}
Exception (java.lang.IllegalStateException) encountered during startup: Unknown 
commitlog version 7Exception (java.lang.IllegalStateException) encountered 
during startup: Unknown commitlog version 7ERROR [main] 2020-05-07 01:13:26,798 
CassandraDaemon.java:749 - Exception encountered during 
startupjava.lang.IllegalStateException: Unknown commitlog version 7 at 
org.apache.cassandra.db.commitlog.CommitLogDescriptor.getMessagingVersion(CommitLogDescriptor.java:227)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLogReader.shouldSkipSegmentId(CommitLogReader.java:276)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:220)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLogReader.readAllFiles(CommitLogReader.java:132)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLogReplayer.replayFiles(CommitLogReplayer.java:137)
 ~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLog.recoverFiles(CommitLog.java:167) 
~[main/:na] at 
org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:148)
 ~[main/:na] at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:324) 
[main/:na] at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.

[jira] [Comment Edited] (CASSANDRA-15793) Update Python driver for cqlsh to 3.23

2020-05-06 Thread Dinesh Joshi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101287#comment-17101287
 ] 

Dinesh Joshi edited comment on CASSANDRA-15793 at 5/7/20, 12:50 AM:


[Patch|https://github.com/dineshjoshi/cassandra/tree/CASSANDRA-15793-trunk] 
[CI|https://circleci.com/workflow-run/dc9cd32a-7989-40c8-94c1-54b633ebd68c]

CI is passing:

!image-2020-05-06-17-49-56-758.png|width=402,height=600!


was (Author: djoshi3):
[Patch|https://github.com/dineshjoshi/cassandra/tree/CASSANDRA-15793-trunk] 
[CI|https://circleci.com/workflow-run/dc9cd32a-7989-40c8-94c1-54b633ebd68c]

> Update Python driver for cqlsh to 3.23
> --
>
> Key: CASSANDRA-15793
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15793
> Project: Cassandra
>  Issue Type: Task
>  Components: Legacy/Tools
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: image-2020-05-06-17-49-56-758.png
>
>
> We should update cqlsh python driver to v3.23 which includes support for 
> transient replication and other fixes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15793) Update Python driver for cqlsh to 3.23

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15793:
-
Test and Documentation Plan: dtests
 Status: Patch Available  (was: Open)

[Patch|https://github.com/dineshjoshi/cassandra/tree/CASSANDRA-15793-trunk] 
[CI|https://circleci.com/workflow-run/dc9cd32a-7989-40c8-94c1-54b633ebd68c]

> Update Python driver for cqlsh to 3.23
> --
>
> Key: CASSANDRA-15793
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15793
> Project: Cassandra
>  Issue Type: Task
>  Components: Legacy/Tools
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> We should update cqlsh python driver to v3.23 which includes support for 
> transient replication and other fixes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15783) test_optimized_primary_range_repair - transient_replication_test.TestTransientReplication

2020-05-06 Thread Dinesh Joshi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101286#comment-17101286
 ] 

Dinesh Joshi commented on CASSANDRA-15783:
--

I've opened CASSANDRA-15792 for fixing the driver related issue. It fixes these 
warnings. However, the test still breaks. I noticed that the sstable is missing 
on 2 out of the 3 replicas. I haven't had a lot of time to investigate this, if 
anybody has the time to dig into it, please do so.

> test_optimized_primary_range_repair - 
> transient_replication_test.TestTransientReplication
> -
>
> Key: CASSANDRA-15783
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15783
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Dtest failure.
> Example:
> https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/118/workflows/9e57522d-52fa-4d44-88d8-5cec0e87f517/jobs/585/tests



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15793) Update Python driver for cqlsh to 3.23

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15793:
-
Change Category: Operability
 Complexity: Low Hanging Fruit
  Fix Version/s: 4.0-alpha
 Status: Open  (was: Triage Needed)

> Update Python driver for cqlsh to 3.23
> --
>
> Key: CASSANDRA-15793
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15793
> Project: Cassandra
>  Issue Type: Task
>  Components: Legacy/Tools
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> We should update cqlsh python driver to v3.23 which includes support for 
> transient replication and other fixes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-15793) Update Python driver for cqlsh to 3.23

2020-05-06 Thread Dinesh Joshi (Jira)
Dinesh Joshi created CASSANDRA-15793:


 Summary: Update Python driver for cqlsh to 3.23
 Key: CASSANDRA-15793
 URL: https://issues.apache.org/jira/browse/CASSANDRA-15793
 Project: Cassandra
  Issue Type: Task
  Components: Legacy/Tools
Reporter: Dinesh Joshi
Assignee: Dinesh Joshi


We should update cqlsh python driver to v3.23 which includes support for 
transient replication and other fixes.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15763) Tunable RangeTombstoneList initial size and resize factor

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15763:
-
  Fix Version/s: 4.0-alpha
Source Control Link: 
https://github.com/apache/cassandra/commit/da95e4b3fb36294b5117846b8fbb8cdc01e427d5
  (was: https://github.com/apache/cassandra/pull/573)
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

Committed, thanks.

> Tunable RangeTombstoneList initial size and resize factor
> -
>
> Key: CASSANDRA-15763
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15763
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Cassandra allocates a backing array with a fixed size of one when creating a 
> new RangeTombstoneList object. The resize factor is fixed to 1.5.
> The initial size and resize factor could be made tunable so it could reduce 
> cost in some cases. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra] branch trunk updated: Add tunable initial size and growth factor to RangeTombstoneList

2020-05-06 Thread djoshi
This is an automated email from the ASF dual-hosted git repository.

djoshi pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/trunk by this push:
 new da95e4b  Add tunable initial size and growth factor to 
RangeTombstoneList
da95e4b is described below

commit da95e4b3fb36294b5117846b8fbb8cdc01e427d5
Author: Yifan Cai 
AuthorDate: Tue Apr 28 13:51:15 2020 -0700

Add tunable initial size and growth factor to RangeTombstoneList

Added `initial_range_tombstone_list_allocation_size` and
`range_tombstone_list_growth_factor` in config, which can be altered via 
JMX.

Patch By Michael Kjellman and Yifan Cai; Reviewed by Dinesh Joshi for 
CASSANDRA-15763

Co-Authored-By: Yifan Cai 
Co-Authored-By: Michael Kjellman 
---
 CHANGES.txt|  1 +
 src/java/org/apache/cassandra/config/Config.java   |  9 +++
 .../cassandra/config/DatabaseDescriptor.java   | 19 +
 .../apache/cassandra/db/MutableDeletionInfo.java   |  5 +-
 .../apache/cassandra/db/RangeTombstoneList.java|  8 +-
 .../apache/cassandra/service/StorageService.java   | 35 +
 .../cassandra/service/StorageServiceMBean.java | 11 +++
 .../cassandra/db/RangeTombstoneListTest.java   | 91 +-
 8 files changed, 172 insertions(+), 7 deletions(-)

diff --git a/CHANGES.txt b/CHANGES.txt
index 9b682b9..0c50b0a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0-alpha5
+ * Add tunable initial size and growth factor to RangeTombstoneList 
(CASSANDRA-15763)
  * Improve debug logging in SSTableReader for index summary (CASSANDRA-15755)
  * bin/sstableverify should support user provided token ranges 
(CASSANDRA-15753)
  * Improve logging when mutation passed to commit log is too large 
(CASSANDRA-14781)
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index 3525715..ed637b0 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -486,6 +486,15 @@ public class Config
 public volatile int validation_preview_purge_head_start_in_sec = 60 * 60;
 
 /**
+ * The intial capacity for creating RangeTombstoneList.
+ */
+public volatile int initial_range_tombstone_list_allocation_size = 1;
+/**
+ * The growth factor to enlarge a RangeTombstoneList.
+ */
+public volatile double range_tombstone_list_growth_factor = 1.5;
+
+/**
  * @deprecated migrate to {@link DatabaseDescriptor#isClientInitialized()}
  */
 @Deprecated
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 698fb45..85a107f 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -3090,4 +3090,23 @@ public class DatabaseDescriptor
 return Math.max(seconds, 0);
 }
 
+public static int getInitialRangeTombstoneListAllocationSize()
+{
+return conf.initial_range_tombstone_list_allocation_size;
+}
+
+public static void setInitialRangeTombstoneListAllocationSize(int size)
+{
+conf.initial_range_tombstone_list_allocation_size = size;
+}
+
+public static double getRangeTombstoneListGrowthFactor()
+{
+return conf.range_tombstone_list_growth_factor;
+}
+
+public static void setRangeTombstoneListGrowthFactor(double resizeFactor)
+{
+conf.range_tombstone_list_growth_factor = resizeFactor;
+}
 }
diff --git a/src/java/org/apache/cassandra/db/MutableDeletionInfo.java 
b/src/java/org/apache/cassandra/db/MutableDeletionInfo.java
index 39728c5..356d763 100644
--- a/src/java/org/apache/cassandra/db/MutableDeletionInfo.java
+++ b/src/java/org/apache/cassandra/db/MutableDeletionInfo.java
@@ -22,6 +22,7 @@ import java.util.Iterator;
 
 import com.google.common.base.Objects;
 
+import org.apache.cassandra.config.DatabaseDescriptor;
 import org.apache.cassandra.db.rows.*;
 import org.apache.cassandra.db.rows.EncodingStats;
 import org.apache.cassandra.utils.ObjectSizes;
@@ -113,8 +114,8 @@ public class MutableDeletionInfo implements DeletionInfo
 
 public void add(RangeTombstone tombstone, ClusteringComparator comparator)
 {
-if (ranges == null)
-ranges = new RangeTombstoneList(comparator, 1);
+if (ranges == null) // Introduce getInitialRangeTombstoneAllocationSize
+ranges = new RangeTombstoneList(comparator, 
DatabaseDescriptor.getInitialRangeTombstoneListAllocationSize());
 
 ranges.add(tombstone);
 }
diff --git a/src/java/org/apache/cassandra/db/RangeTombstoneList.java 
b/src/java/org/apache/cassandra/db/RangeTombstoneList.java
index 401ff7b..7034d22 100644
--- a/src/java/org/apache/cassandra/db/RangeTombstoneList.java
++

[jira] [Commented] (CASSANDRA-15783) test_optimized_primary_range_repair - transient_replication_test.TestTransientReplication

2020-05-06 Thread Dinesh Joshi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101240#comment-17101240
 ] 

Dinesh Joshi commented on CASSANDRA-15783:
--

I think we need to update to the latest Python driver that supports creating 
transient replicas. Here's what I get when I run the test locally:

{code:bash}
15:39:54,773 cassandra.metadata WARNING Failed creating NetworkTopologyStrategy 
with options {'datacenter1': '3/1'}: invalid literal for int() with base 10: 
'3/1'
15:39:54,778 cassandra.metadata WARNING Failed creating NetworkTopologyStrategy 
with options {'datacenter1': '3/1'}: invalid literal for int() with base 10: 
'3/1'
15:39:54,962 cassandra.metadata WARNING Failed creating NetworkTopologyStrategy 
with options {'datacenter1': '3/1'}: invalid literal for int() with base 10: 
'3/1'{code}

> test_optimized_primary_range_repair - 
> transient_replication_test.TestTransientReplication
> -
>
> Key: CASSANDRA-15783
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15783
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Dtest failure.
> Example:
> https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/118/workflows/9e57522d-52fa-4d44-88d8-5cec0e87f517/jobs/585/tests



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Comment Edited] (CASSANDRA-15783) test_optimized_primary_range_repair - transient_replication_test.TestTransientReplication

2020-05-06 Thread Ekaterina Dimitrova (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101222#comment-17101222
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-15783 at 5/6/20, 10:05 PM:
---

Thanks [~jasonstack]
Same as you, I am also not sure about the behavior. It sounds to me as we need 
to rework actually the test now and that behavior is expected but I am not 100% 
sure?
[~djoshi] do you have any vision?
Or maybe [~ifesdjeen] has some advice here?



was (Author: e.dimitrova):
Thanks [~jasonstack]
Same as you, I am also not sure about the behavior. It sounds to me as we need 
to rework actually the test now and that behavior is expected but I am not 100% 
sure?
[~djoshi] do you have any vision?


> test_optimized_primary_range_repair - 
> transient_replication_test.TestTransientReplication
> -
>
> Key: CASSANDRA-15783
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15783
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Dtest failure.
> Example:
> https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/118/workflows/9e57522d-52fa-4d44-88d8-5cec0e87f517/jobs/585/tests



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15783) test_optimized_primary_range_repair - transient_replication_test.TestTransientReplication

2020-05-06 Thread Ekaterina Dimitrova (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101222#comment-17101222
 ] 

Ekaterina Dimitrova commented on CASSANDRA-15783:
-

Thanks [~jasonstack]
Same as you, I am also not sure about the behavior. It sounds to me as we need 
to rework actually the test now and that behavior is expected but I am not 100% 
sure?
[~djoshi] do you have any vision?


> test_optimized_primary_range_repair - 
> transient_replication_test.TestTransientReplication
> -
>
> Key: CASSANDRA-15783
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15783
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Dtest failure.
> Example:
> https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/118/workflows/9e57522d-52fa-4d44-88d8-5cec0e87f517/jobs/585/tests



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15755) Add a debug line informing of building summary during sstable open

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15755:
-
  Fix Version/s: (was: 4.x)
 4.0-alpha
Source Control Link: 
https://github.com/apache/cassandra/commit/6cf352ea9c0c114d998da7b3e2dc6b53609a925b
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

Committed, thanks for the patch [~sumanth.pasupuleti]!

> Add a debug line informing of building summary during sstable open
> --
>
> Key: CASSANDRA-15755
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15755
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Logging
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Fix For: 4.0-alpha
>
> Attachments: 15755-4.0.txt
>
>
> We've had a situation where our backup-restore flow did not include 
> summary.db component for a large file with a lot of keys. This caused 
> Cassandra on the restored node to take a long time to boot (while it was 
> building the summary).
> The last message was about opening a few sstables. It would be nice to also 
> include a debug line about building summary, which then operators can use to 
> check if a missing summary file is expected. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra] branch trunk updated (2d27c76 -> 6cf352e)

2020-05-06 Thread djoshi
This is an automated email from the ASF dual-hosted git repository.

djoshi pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from 2d27c76  bin/sstableverify should support user provided token ranges
 add 6cf352e  Adding debug logging to indicate missing summary component 
file and when building Index Summary

No new revisions were added by this update.

Summary of changes:
 CHANGES.txt   |  1 +
 .../org/apache/cassandra/io/sstable/format/SSTableReader.java | 11 +--
 2 files changed, 10 insertions(+), 2 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15755) Add a debug line informing of building summary during sstable open

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15755:
-
Status: Ready to Commit  (was: Review In Progress)

> Add a debug line informing of building summary during sstable open
> --
>
> Key: CASSANDRA-15755
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15755
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Logging
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Fix For: 4.x
>
> Attachments: 15755-4.0.txt
>
>
> We've had a situation where our backup-restore flow did not include 
> summary.db component for a large file with a lot of keys. This caused 
> Cassandra on the restored node to take a long time to boot (while it was 
> building the summary).
> The last message was about opening a few sstables. It would be nice to also 
> include a debug line about building summary, which then operators can use to 
> check if a missing summary file is expected. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15755) Add a debug line informing of building summary during sstable open

2020-05-06 Thread Dinesh Joshi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101204#comment-17101204
 ] 

Dinesh Joshi commented on CASSANDRA-15755:
--

thanks [~sumanth.pasupuleti]. Looks like the the {{ViewTest}} is flaky. It 
passed after a few tries.

> Add a debug line informing of building summary during sstable open
> --
>
> Key: CASSANDRA-15755
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15755
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Logging
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Fix For: 4.x
>
> Attachments: 15755-4.0.txt
>
>
> We've had a situation where our backup-restore flow did not include 
> summary.db component for a large file with a lot of keys. This caused 
> Cassandra on the restored node to take a long time to boot (while it was 
> building the summary).
> The last message was about opening a few sstables. It would be nice to also 
> include a debug line about building summary, which then operators can use to 
> check if a missing summary file is expected. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15755) Add a debug line informing of building summary during sstable open

2020-05-06 Thread Sumanth Pasupuleti (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101172#comment-17101172
 ] 

Sumanth Pasupuleti commented on CASSANDRA-15755:


+1 [~djoshi]!

> Add a debug line informing of building summary during sstable open
> --
>
> Key: CASSANDRA-15755
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15755
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Logging
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Fix For: 4.x
>
> Attachments: 15755-4.0.txt
>
>
> We've had a situation where our backup-restore flow did not include 
> summary.db component for a large file with a lot of keys. This caused 
> Cassandra on the restored node to take a long time to boot (while it was 
> building the summary).
> The last message was about opening a few sstables. It would be nice to also 
> include a debug line about building summary, which then operators can use to 
> check if a missing summary file is expected. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15753) bin/sstableverify should support user provided token ranges

2020-05-06 Thread Dinesh Joshi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101173#comment-17101173
 ] 

Dinesh Joshi commented on CASSANDRA-15753:
--

Thanks for the patch [~dcapwell]!

> bin/sstableverify should support user provided token ranges
> ---
>
> Key: CASSANDRA-15753
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15753
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tool/sstable
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-alpha
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Verify should allow options for token ranges so the stand alone verifier is 
> able to include user supplied ranges; this is useful for tools to verify 
> outside of the cluster.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-13047) Point cqlsh help to the new doc

2020-05-06 Thread Josh McKenzie (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101171#comment-17101171
 ] 

Josh McKenzie commented on CASSANDRA-13047:
---

Ping on this. Anyone need a hand?

> Point cqlsh help to the new doc
> ---
>
> Key: CASSANDRA-13047
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13047
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/CQL
>Reporter: Sylvain Lebresne
>Assignee: Patrick Bannister
>Priority: Normal
> Fix For: 4.0
>
>
> Cqlsh still points to the "old" textile CQL doc, but that's not really 
> maintain anymore now that we have the new doc (which include everything the 
> old doc had and more). We should update cqlsh to point to the new doc so we 
> can remove the old one completely.
> Any takers?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-14361) Allow SimpleSeedProvider to resolve multiple IPs per DNS name

2020-05-06 Thread Josh McKenzie (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101169#comment-17101169
 ] 

Josh McKenzie commented on CASSANDRA-14361:
---

Did you need a review on this [~benbromhead] or did I just miss it on a PR and 
we need to update the ticket here?

> Allow SimpleSeedProvider to resolve multiple IPs per DNS name
> -
>
> Key: CASSANDRA-14361
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14361
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: Ben Bromhead
>Assignee: Ben Bromhead
>Priority: Low
> Fix For: 4.0
>
>
> Currently SimpleSeedProvider can accept a comma separated string of IPs or 
> hostnames as the set of Cassandra seeds. hostnames are resolved via 
> InetAddress.getByName, which will only return the first IP associated with an 
> A,  or CNAME record.
> By changing to InetAddress.getAllByName, existing behavior is preserved, but 
> now Cassandra can discover multiple IP address per record, allowing seed 
> discovery by DNS to be a little easier.
> Some examples of improved workflows with this change include: 
>  * specify the DNS name of a headless service in Kubernetes which will 
> resolve to all IP addresses of pods within that service. 
>  * seed discovery for multi-region clusters via AWS route53, AzureDNS etc
>  * Other common DNS service discovery mechanisms.
> The only behavior this is likely to impact would be where users are relying 
> on the fact that getByName only returns a single IP address.
> I can't imagine any scenario where that is a sane choice. Even when that 
> choice has been made, it only impacts the first startup of Cassandra and 
> would not be on any critical path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-14606) Add documentation for java 11 support

2020-05-06 Thread Josh McKenzie (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101168#comment-17101168
 ] 

Josh McKenzie commented on CASSANDRA-14606:
---

That documentation looks like developer docs for working on C*. Do we have 
operator documentation for what to expect and not expect running C* on JDK11?

> Add documentation for java 11 support
> -
>
> Key: CASSANDRA-14606
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14606
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/Documentation and Website
>Reporter: Jason Brown
>Priority: Low
>  Labels: Java11
> Fix For: 4.0
>
>
> Let's add some documentation for operators around the java 11 support that 
> was introduced in CASSANDRA-9608. Also, we should point out changes in the 
> scripts that might affect automation that operators have in place.
> Parking on [~snazy] just 'cuz ;)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15753) bin/sstableverify should support user provided token ranges

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15753:
-
  Fix Version/s: 4.0-alpha
Source Control Link: 
https://github.com/apache/cassandra/commit/2d27c769479f877eb3af308b49d73f8e124f4703
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

> bin/sstableverify should support user provided token ranges
> ---
>
> Key: CASSANDRA-15753
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15753
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tool/sstable
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-alpha
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Verify should allow options for token ranges so the stand alone verifier is 
> able to include user supplied ranges; this is useful for tools to verify 
> outside of the cluster.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra] branch trunk updated: bin/sstableverify should support user provided token ranges

2020-05-06 Thread djoshi
This is an automated email from the ASF dual-hosted git repository.

djoshi pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/trunk by this push:
 new 2d27c76  bin/sstableverify should support user provided token ranges
2d27c76 is described below

commit 2d27c769479f877eb3af308b49d73f8e124f4703
Author: David Capwell 
AuthorDate: Wed Apr 22 11:13:24 2020 -0700

bin/sstableverify should support user provided token ranges

Patch by David Capwell; reviewed by Dinesh Joshi for CASSANDRA-15753
---
 CHANGES.txt|  1 +
 .../apache/cassandra/db/compaction/Verifier.java   | 36 +++-
 .../org/apache/cassandra/tools/BulkLoader.java | 17 
 .../apache/cassandra/tools/StandaloneVerifier.java | 49 +-
 4 files changed, 92 insertions(+), 11 deletions(-)

diff --git a/CHANGES.txt b/CHANGES.txt
index 4b13ec5..ddf2d63 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0-alpha5
+ * bin/sstableverify should support user provided token ranges 
(CASSANDRA-15753)
  * Improve logging when mutation passed to commit log is too large 
(CASSANDRA-14781)
  * replace LZ4FastDecompressor with LZ4SafeDecompressor (CASSANDRA-15560)
  * Fix buffer pool NPE with concurrent release due to in-progress tiny pool 
eviction (CASSANDRA-15726)
diff --git a/src/java/org/apache/cassandra/db/compaction/Verifier.java 
b/src/java/org/apache/cassandra/db/compaction/Verifier.java
index 551043a..2500a24 100644
--- a/src/java/org/apache/cassandra/db/compaction/Verifier.java
+++ b/src/java/org/apache/cassandra/db/compaction/Verifier.java
@@ -59,8 +59,8 @@ import java.nio.ByteBuffer;
 import java.nio.file.Files;
 import java.nio.file.Paths;
 import java.util.*;
+import java.util.function.Function;
 import java.util.function.LongPredicate;
-import java.util.function.Predicate;
 
 public class Verifier implements Closeable
 {
@@ -76,6 +76,12 @@ public class Verifier implements Closeable
 private final RowIndexEntry.IndexSerializer rowIndexEntrySerializer;
 private final Options options;
 private final boolean isOffline;
+/**
+ * Given a keyspace, return the set of local and pending token ranges.  By 
default {@link StorageService#getLocalAndPendingRanges(String)}
+ * is expected, but for the standalone verifier case we can't use that, so 
this is here to allow the CLI to provide
+ * the token ranges.
+ */
+private final Function>> 
tokenLookup;
 
 private int goodRows;
 
@@ -103,6 +109,7 @@ public class Verifier implements Closeable
 this.verifyInfo = new VerifyInfo(dataFile, sstable);
 this.options = options;
 this.isOffline = isOffline;
+this.tokenLookup = options.tokenLookup;
 }
 
 public void verify()
@@ -130,7 +137,7 @@ public class Verifier implements Closeable
 }
 catch (Throwable t)
 {
-outputHandler.debug(t.getMessage());
+outputHandler.warn(t.getMessage());
 markAndThrow(false);
 }
 
@@ -141,7 +148,7 @@ public class Verifier implements Closeable
 }
 catch (Throwable t)
 {
-outputHandler.debug(t.getMessage());
+outputHandler.warn(t.getMessage());
 markAndThrow();
 }
 
@@ -153,7 +160,7 @@ public class Verifier implements Closeable
 catch (Throwable t)
 {
 outputHandler.output("Index summary is corrupt - if it is removed 
it will get rebuilt on startup 
"+sstable.descriptor.filenameFor(Component.SUMMARY));
-outputHandler.debug(t.getMessage());
+outputHandler.warn(t.getMessage());
 markAndThrow(false);
 }
 
@@ -165,7 +172,7 @@ public class Verifier implements Closeable
 }
 catch (Throwable t)
 {
-outputHandler.debug(t.getMessage());
+outputHandler.warn(t.getMessage());
 markAndThrow();
 }
 
@@ -174,7 +181,7 @@ public class Verifier implements Closeable
 outputHandler.debug("Checking that all tokens are owned by the 
current node");
 try (KeyIterator iter = new KeyIterator(sstable.descriptor, 
sstable.metadata()))
 {
-List> ownedRanges = 
Range.normalize(StorageService.instance.getLocalAndPendingRanges(cfs.metadata.keyspace));
+List> ownedRanges = 
Range.normalize(tokenLookup.apply(cfs.metadata.keyspace));
 if (ownedRanges.isEmpty())
 return;
 RangeOwnHelper rangeOwnHelper = new 
RangeOwnHelper(ownedRanges);
@@ -213,7 +220,7 @@ public class Verifier implements Closeable
 }
 catch (IOException e)
 {
-outputHandler.debug(e.getMessage());
+outputHandler.warn(e.getMessage());
 markAndThrow();
 }
 finally
@@ -235,7 +242

[jira] [Updated] (CASSANDRA-15756) in-jvm dtest IInstance and ICoordinator should use QueryResult as the base API

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15756:
-
Reviewers: Alex Petrov, Dinesh Joshi, Dinesh Joshi  (was: Alex Petrov, 
Dinesh Joshi)
   Alex Petrov, Dinesh Joshi, Dinesh Joshi  (was: Alex Petrov, 
Dinesh Joshi)
   Status: Review In Progress  (was: Patch Available)

> in-jvm dtest IInstance and ICoordinator should use QueryResult as the base API
> --
>
> Key: CASSANDRA-15756
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15756
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> ICoordinator was modified to have a executeWithResult which returns a 
> QueryResult, but not all query APIs have this. We should fix it so the base 
> interface is executeWithResult and return QueryResult, but update all 
> Object[][] methods to default to calling this.
> This would be a breaking change so would need to be updated in each branch.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15756) in-jvm dtest IInstance and ICoordinator should use QueryResult as the base API

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15756:
-
  Fix Version/s: 4.0
Source Control Link: 
https://github.com/apache/cassandra-in-jvm-dtest-api/commit/cc3e43c710a0fb683b7e955f641e221ccc2e5d54
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

> in-jvm dtest IInstance and ICoordinator should use QueryResult as the base API
> --
>
> Key: CASSANDRA-15756
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15756
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> ICoordinator was modified to have a executeWithResult which returns a 
> QueryResult, but not all query APIs have this. We should fix it so the base 
> interface is executeWithResult and return QueryResult, but update all 
> Object[][] methods to default to calling this.
> This would be a breaking change so would need to be updated in each branch.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15756) in-jvm dtest IInstance and ICoordinator should use QueryResult as the base API

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15756:
-
Status: Ready to Commit  (was: Review In Progress)

> in-jvm dtest IInstance and ICoordinator should use QueryResult as the base API
> --
>
> Key: CASSANDRA-15756
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15756
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> ICoordinator was modified to have a executeWithResult which returns a 
> QueryResult, but not all query APIs have this. We should fix it so the base 
> interface is executeWithResult and return QueryResult, but update all 
> Object[][] methods to default to calling this.
> This would be a breaking change so would need to be updated in each branch.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers

2020-05-06 Thread Dinesh Joshi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101149#comment-17101149
 ] 

Dinesh Joshi commented on CASSANDRA-14825:
--

[~blerer] any updates?

> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Robert Stupp
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the 
> system table values. This is very difficult to keep up to date and buggy 
> enough that its only even supported in Java and Python drivers. Cassandra 
> already has some limited output available for snapshots that we could provide 
> in a virtual table or new query that the drivers can fetch. This can greatly 
> reduce the complexity of drivers while also reducing bugs like 
> CASSANDRA-14822 as the underlying schema and properties change.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15776) python dtest regression caused by CASSANDRA-15637

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15776:
-
Status: Changes Suggested  (was: Review In Progress)

> python dtest regression caused by CASSANDRA-15637
> -
>
> Key: CASSANDRA-15776
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15776
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-alpha
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> CASSANDRA-15637 deprecated size_estimates in favor of table_estimates to 
> allow for local primary range estimates (needed for MapReduce).  This appears 
> to have caused a regression in the python dtest nodetool_test.TestNodetool. 
> test_refresh_size_estimates_clears_invalid_entries (as seen by [Circle 
> CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra/255/workflows/21907001-93ed-4963-9314-6a0ac6ea0f1d/jobs/1246/tests]
>   and 
> [Jenkins|https://ci-cassandra.apache.org/job/Cassandra-trunk-dtest/56/]).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15776) python dtest regression caused by CASSANDRA-15637

2020-05-06 Thread Dinesh Joshi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101145#comment-17101145
 ] 

Dinesh Joshi commented on CASSANDRA-15776:
--

[~dcapwell] I took a look and left a comment. I have a couple concerns that 
other folks share (two scans vs one). But I think thats unavoidable. The only 
other concern I have is you might be clobbering values if the same keyspace 
exists in both tables. We should ideally guard against such situations.

> python dtest regression caused by CASSANDRA-15637
> -
>
> Key: CASSANDRA-15776
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15776
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-alpha
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> CASSANDRA-15637 deprecated size_estimates in favor of table_estimates to 
> allow for local primary range estimates (needed for MapReduce).  This appears 
> to have caused a regression in the python dtest nodetool_test.TestNodetool. 
> test_refresh_size_estimates_clears_invalid_entries (as seen by [Circle 
> CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra/255/workflows/21907001-93ed-4963-9314-6a0ac6ea0f1d/jobs/1246/tests]
>   and 
> [Jenkins|https://ci-cassandra.apache.org/job/Cassandra-trunk-dtest/56/]).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15776) python dtest regression caused by CASSANDRA-15637

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15776:
-
Reviewers: Dinesh Joshi, Dinesh Joshi  (was: Dinesh Joshi)
   Dinesh Joshi, Dinesh Joshi  (was: Dinesh Joshi)
   Status: Review In Progress  (was: Patch Available)

> python dtest regression caused by CASSANDRA-15637
> -
>
> Key: CASSANDRA-15776
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15776
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-alpha
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> CASSANDRA-15637 deprecated size_estimates in favor of table_estimates to 
> allow for local primary range estimates (needed for MapReduce).  This appears 
> to have caused a regression in the python dtest nodetool_test.TestNodetool. 
> test_refresh_size_estimates_clears_invalid_entries (as seen by [Circle 
> CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra/255/workflows/21907001-93ed-4963-9314-6a0ac6ea0f1d/jobs/1246/tests]
>   and 
> [Jenkins|https://ci-cassandra.apache.org/job/Cassandra-trunk-dtest/56/]).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15229) BufferPool Regression

2020-05-06 Thread Josh McKenzie (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101139#comment-17101139
 ] 

Josh McKenzie commented on CASSANDRA-15229:
---

If it's not too much bother, could we update the Since Version on this and add 
a little detail in the description as to the impact of this regression on the 
system?

> BufferPool Regression
> -
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Caching
>Reporter: Benedict Elliott Smith
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: 15229-count.png, 15229-direct.png, 15229-hit-rate.png, 
> 15229-recirculate-count.png, 15229-recirculate-hit-rate.png, 
> 15229-recirculate-size.png, 15229-recirculate.png, 15229-size.png, 
> 15229-unsafe.png
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we 
> need to either change our behaviour to handle uncorrelated lifetimes or use 
> something else.  This is particularly important with the default chunk size 
> for compressed sstables being reduced.  If we address the problem, we should 
> also utilise the BufferPool for native transport connections like we do for 
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used 
> for things with uncorrelated lifetimes, which essentially boils down to 
> tracking those chunks that have not been freed and re-circulating them when 
> we run out of completely free blocks.  We should probably also permit 
> instantiating separate {{BufferPool}}, so that we can insulate internode 
> messaging from the {{ChunkCache}}, or at least have separate memory bounds 
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk 
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce 
> the amount of global coordination and per-allocation overhead.  We don’t need 
> 1KiB granularity for allocations, nor 16 byte granularity for tiny 
> allocations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15788) Add tests to cover CacheMetrics

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15788:
-
Status: Patch Available  (was: Open)

> Add tests to cover CacheMetrics
> ---
>
> Key: CASSANDRA-15788
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15788
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/unit
>Reporter: Stephen Mallette
>Assignee: Stephen Mallette
>Priority: Normal
>
> {{CacheMetrics}} and {{ChunkCacheMetrics}} do not have unit tests covering 
> them.  {{CachingBench}} seems to provide some coverage but those tests (which 
> don't appear to run as part of the standard run of unit tests) are failing 
> and do not assert against all defined metrics, nor do they seem to assert 
> code in {{InstrumentingCache}} which also incremented metrics. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15788) Add tests to cover CacheMetrics

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15788:
-
Reviewers: David Capwell, Dinesh Joshi
   Status: Open  (was: Triage Needed)

> Add tests to cover CacheMetrics
> ---
>
> Key: CASSANDRA-15788
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15788
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/unit
>Reporter: Stephen Mallette
>Assignee: Stephen Mallette
>Priority: Normal
>
> {{CacheMetrics}} and {{ChunkCacheMetrics}} do not have unit tests covering 
> them.  {{CachingBench}} seems to provide some coverage but those tests (which 
> don't appear to run as part of the standard run of unit tests) are failing 
> and do not assert against all defined metrics, nor do they seem to assert 
> code in {{InstrumentingCache}} which also incremented metrics. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra-website] branch asf-staging updated (4db8d07 -> 442a3aa)

2020-05-06 Thread git-site-role
This is an automated email from the ASF dual-hosted git repository.

git-site-role pushed a change to branch asf-staging
in repository https://gitbox.apache.org/repos/asf/cassandra-website.git.


omit 4db8d07  generate docs for eac91582
 add 06e2ad8  Update README.md for new simpler development cycle. Remove 
content/ folder as it is now served from asf-staging and asf-site branches.
 add 1ad5e74  Add a dummy file to keep the content directory in git, as it 
needs to exist to be mounted as a volume in the docker run.
 new 442a3aa  generate docs for 1ad5e747

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (4db8d07)
\
 N -- N -- N   refs/heads/asf-staging (442a3aa)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .asf.yaml  |  2 +-
 README.md  | 32 +-
 content/doc/4.0-alpha5/searchindex.js  |  2 +-
 content/doc/4.0-alpha5/tools/cassandra_stress.html |  2 ++
 content/doc/latest/searchindex.js  |  2 +-
 content/doc/latest/tools/cassandra_stress.html |  2 ++
 content/feed.xml   |  2 +-
 .../_sources/tools/cassandra_stress.rst.txt|  6 +++-
 src/doc/4.0-alpha5/searchindex.js  |  2 +-
 src/doc/4.0-alpha5/tools/cassandra_stress.html |  2 ++
 10 files changed, 35 insertions(+), 19 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15773) Add test to cover metrics related to the BufferPool

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15773:
-
Status: Review In Progress  (was: Changes Suggested)

> Add test to cover metrics related to the BufferPool
> ---
>
> Key: CASSANDRA-15773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15773
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/unit
>Reporter: Stephen Mallette
>Assignee: Stephen Mallette
>Priority: Normal
>
> At this time there do not appear to be unit tests to validate 
> {{BufferPoolMetrics}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15773) Add test to cover metrics related to the BufferPool

2020-05-06 Thread Dinesh Joshi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101123#comment-17101123
 ] 

Dinesh Joshi commented on CASSANDRA-15773:
--

[~spmallette] I think the test is ok. I don't think you're not leaking memory 
([~dcapwell] please correct me if I am missing something here). A few things to 
think about -

# You're making gets of the same size. I feel randomizing the size of requested 
buffers would be better
# We should add negative test cases i.e. request buffers with bad sizes

> Add test to cover metrics related to the BufferPool
> ---
>
> Key: CASSANDRA-15773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15773
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/unit
>Reporter: Stephen Mallette
>Assignee: Stephen Mallette
>Priority: Normal
>
> At this time there do not appear to be unit tests to validate 
> {{BufferPoolMetrics}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15773) Add test to cover metrics related to the BufferPool

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15773:
-
Status: Changes Suggested  (was: Review In Progress)

> Add test to cover metrics related to the BufferPool
> ---
>
> Key: CASSANDRA-15773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15773
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/unit
>Reporter: Stephen Mallette
>Assignee: Stephen Mallette
>Priority: Normal
>
> At this time there do not appear to be unit tests to validate 
> {{BufferPoolMetrics}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-15792) test_speculative_data_request - read_repair_test.TestSpeculativeReadRepair

2020-05-06 Thread Ekaterina Dimitrova (Jira)
Ekaterina Dimitrova created CASSANDRA-15792:
---

 Summary: test_speculative_data_request - 
read_repair_test.TestSpeculativeReadRepair
 Key: CASSANDRA-15792
 URL: https://issues.apache.org/jira/browse/CASSANDRA-15792
 Project: Cassandra
  Issue Type: Bug
Reporter: Ekaterina Dimitrova


Failing on the latest trunk here:
https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/127/workflows/dfba669d-4a5c-4553-b6a2-85647d0d8d2b/jobs/668/tests
Failing once in 30 times as per Jenkins:
https://jenkins-cm4.apache.org/job/Cassandra-trunk-dtest/69/testReport/dtest.read_repair_test/TestSpeculativeReadRepair/test_speculative_data_request/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15792) test_speculative_data_request - read_repair_test.TestSpeculativeReadRepair

2020-05-06 Thread Ekaterina Dimitrova (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-15792:

 Bug Category: Parent values: Correctness(12982)
   Complexity: Normal
  Component/s: Test/dtest
Discovered By: User Report
Fix Version/s: 4.0-alpha
 Severity: Normal
   Status: Open  (was: Triage Needed)

> test_speculative_data_request - read_repair_test.TestSpeculativeReadRepair
> --
>
> Key: CASSANDRA-15792
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15792
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Failing on the latest trunk here:
> https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/127/workflows/dfba669d-4a5c-4553-b6a2-85647d0d8d2b/jobs/668/tests
> Failing once in 30 times as per Jenkins:
> https://jenkins-cm4.apache.org/job/Cassandra-trunk-dtest/69/testReport/dtest.read_repair_test/TestSpeculativeReadRepair/test_speculative_data_request/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra-website] 01/01: generate docs for 1ad5e747

2020-05-06 Thread git-site-role
This is an automated email from the ASF dual-hosted git repository.

git-site-role pushed a commit to branch asf-staging
in repository https://gitbox.apache.org/repos/asf/cassandra-website.git

commit cedc8a3f7e58fa6ac363ffc68d77dddf56b73127
Author: jenkins 
AuthorDate: Wed May 6 19:16:37 2020 +

generate docs for 1ad5e747
---
 content/.keepdir   | 1 -
 src/doc/4.0-alpha5/_sources/tools/cassandra_stress.rst.txt | 6 +-
 src/doc/4.0-alpha5/searchindex.js  | 2 +-
 src/doc/4.0-alpha5/tools/cassandra_stress.html | 2 ++
 4 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/content/.keepdir b/content/.keepdir
deleted file mode 100644
index 0488e10..000
--- a/content/.keepdir
+++ /dev/null
@@ -1 +0,0 @@
-# dummy file to keep git directory
diff --git a/src/doc/4.0-alpha5/_sources/tools/cassandra_stress.rst.txt 
b/src/doc/4.0-alpha5/_sources/tools/cassandra_stress.rst.txt
index bcac54e..c59d058 100644
--- a/src/doc/4.0-alpha5/_sources/tools/cassandra_stress.rst.txt
+++ b/src/doc/4.0-alpha5/_sources/tools/cassandra_stress.rst.txt
@@ -266,4 +266,8 @@ Use the schema option making sure to either escape the 
parenthesis or enclose in
 
 Use the transport option::
 
-cassandra-stress "write n=100k cl=ONE no-warmup" -transport 
"truststore=$HOME/jks/truststore.jks truststore-password=cassandra"
\ No newline at end of file
+cassandra-stress "write n=100k cl=ONE no-warmup" -transport 
"truststore=$HOME/jks/truststore.jks truststore-password=cassandra"
+
+**Is Cassandra Stress a secured tool?**
+
+Cassandra stress is not a secured tool. Serialization and other aspects of the 
tool offer no security guarantees.
diff --git a/src/doc/4.0-alpha5/searchindex.js 
b/src/doc/4.0-alpha5/searchindex.js
index 124c41e..3480911 100644
--- a/src/doc/4.0-alpha5/searchindex.js
+++ b/src/doc/4.0-alpha5/searchindex.js
@@ -1 +1 @@
-Search.setIndex({docnames:["architecture/dynamo","architecture/guarantees","architecture/index","architecture/overview","architecture/storage_engine","bugs","configuration/cassandra_config_file","configuration/index","contactus","cql/appendices","cql/changes","cql/ddl","cql/definitions","cql/dml","cql/functions","cql/index","cql/indexes","cql/json","cql/mvs","cql/operators","cql/security","cql/triggers","cql/types","data_modeling/data_modeling_conceptual","data_modeling/data_modeling_log
 [...]
\ No newline at end of file
+Search.setIndex({docnames:["architecture/dynamo","architecture/guarantees","architecture/index","architecture/overview","architecture/storage_engine","bugs","configuration/cassandra_config_file","configuration/index","contactus","cql/appendices","cql/changes","cql/ddl","cql/definitions","cql/dml","cql/functions","cql/index","cql/indexes","cql/json","cql/mvs","cql/operators","cql/security","cql/triggers","cql/types","data_modeling/data_modeling_conceptual","data_modeling/data_modeling_log
 [...]
\ No newline at end of file
diff --git a/src/doc/4.0-alpha5/tools/cassandra_stress.html 
b/src/doc/4.0-alpha5/tools/cassandra_stress.html
index fd18a64..c866b46 100644
--- a/src/doc/4.0-alpha5/tools/cassandra_stress.html
+++ b/src/doc/4.0-alpha5/tools/cassandra_stress.html
@@ -331,6 +331,8 @@ to fulfill lightweight transaction condition(s).
 cassandra-stress "write n=100k cl=ONE no-warmup" 
-transport "truststore=$HOME/jks/truststore.jks 
truststore-password=cassandra"
 
 
+Is Cassandra Stress a secured tool?
+Cassandra stress is not a secured tool. Serialization and other aspects of 
the tool offer no security guarantees.
 
 
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15755) Add a debug line informing of building summary during sstable open

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15755:
-
Status: Patch Available  (was: Ready to Commit)

> Add a debug line informing of building summary during sstable open
> --
>
> Key: CASSANDRA-15755
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15755
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Logging
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Fix For: 4.x
>
> Attachments: 15755-4.0.txt
>
>
> We've had a situation where our backup-restore flow did not include 
> summary.db component for a large file with a lot of keys. This caused 
> Cassandra on the restored node to take a long time to boot (while it was 
> building the summary).
> The last message was about opening a few sstables. It would be nice to also 
> include a debug line about building summary, which then operators can use to 
> check if a missing summary file is expected. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15755) Add a debug line informing of building summary during sstable open

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15755:
-
Status: Ready to Commit  (was: Review In Progress)

> Add a debug line informing of building summary during sstable open
> --
>
> Key: CASSANDRA-15755
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15755
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Logging
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Fix For: 4.x
>
> Attachments: 15755-4.0.txt
>
>
> We've had a situation where our backup-restore flow did not include 
> summary.db component for a large file with a lot of keys. This caused 
> Cassandra on the restored node to take a long time to boot (while it was 
> building the summary).
> The last message was about opening a few sstables. It would be nice to also 
> include a debug line about building summary, which then operators can use to 
> check if a missing summary file is expected. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15755) Add a debug line informing of building summary during sstable open

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15755:
-
Reviewers: Dinesh Joshi, Dinesh Joshi  (was: Dinesh Joshi)
   Dinesh Joshi, Dinesh Joshi  (was: Dinesh Joshi)
   Status: Review In Progress  (was: Patch Available)

> Add a debug line informing of building summary during sstable open
> --
>
> Key: CASSANDRA-15755
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15755
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Logging
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Fix For: 4.x
>
> Attachments: 15755-4.0.txt
>
>
> We've had a situation where our backup-restore flow did not include 
> summary.db component for a large file with a lot of keys. This caused 
> Cassandra on the restored node to take a long time to boot (while it was 
> building the summary).
> The last message was about opening a few sstables. It would be nice to also 
> include a debug line about building summary, which then operators can use to 
> check if a missing summary file is expected. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15755) Add a debug line informing of building summary during sstable open

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15755:
-
Test and Documentation Plan: Manual testing
 Status: Patch Available  (was: Open)

> Add a debug line informing of building summary during sstable open
> --
>
> Key: CASSANDRA-15755
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15755
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Logging
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Fix For: 4.x
>
> Attachments: 15755-4.0.txt
>
>
> We've had a situation where our backup-restore flow did not include 
> summary.db component for a large file with a lot of keys. This caused 
> Cassandra on the restored node to take a long time to boot (while it was 
> building the summary).
> The last message was about opening a few sstables. It would be nice to also 
> include a debug line about building summary, which then operators can use to 
> check if a missing summary file is expected. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15755) Add a debug line informing of building summary during sstable open

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15755:
-
Reviewers: Dinesh Joshi, Dinesh Joshi  (was: Dinesh Joshi)
   Dinesh Joshi, Dinesh Joshi  (was: Dinesh Joshi)
   Status: Review In Progress  (was: Patch Available)

> Add a debug line informing of building summary during sstable open
> --
>
> Key: CASSANDRA-15755
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15755
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Logging
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Fix For: 4.x
>
> Attachments: 15755-4.0.txt
>
>
> We've had a situation where our backup-restore flow did not include 
> summary.db component for a large file with a lot of keys. This caused 
> Cassandra on the restored node to take a long time to boot (while it was 
> building the summary).
> The last message was about opening a few sstables. It would be nice to also 
> include a debug line about building summary, which then operators can use to 
> check if a missing summary file is expected. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15755) Add a debug line informing of building summary during sstable open

2020-05-06 Thread Dinesh Joshi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101099#comment-17101099
 ] 

Dinesh Joshi commented on CASSANDRA-15755:
--

Hi [~sumanth.pasupuleti] thanks for the patch. Overall it looks good. I've 
added some minor changes to it. Let me know if they're good and I'll commit it.

[trunk|https://github.com/dineshjoshi/cassandra/tree/15755-trunk] 
[CI|https://circleci.com/workflow-run/67d85eed-e46e-4066-b95c-7ddb60b9e799]

> Add a debug line informing of building summary during sstable open
> --
>
> Key: CASSANDRA-15755
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15755
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Logging
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Fix For: 4.x
>
> Attachments: 15755-4.0.txt
>
>
> We've had a situation where our backup-restore flow did not include 
> summary.db component for a large file with a lot of keys. This caused 
> Cassandra on the restored node to take a long time to boot (while it was 
> building the summary).
> The last message was about opening a few sstables. It would be nice to also 
> include a debug line about building summary, which then operators can use to 
> check if a missing summary file is expected. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15755) Add a debug line informing of building summary during sstable open

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15755:
-
Change Category: Operability
 Complexity: Low Hanging Fruit
 Status: Open  (was: Triage Needed)

> Add a debug line informing of building summary during sstable open
> --
>
> Key: CASSANDRA-15755
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15755
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Logging
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Fix For: 4.x
>
> Attachments: 15755-4.0.txt
>
>
> We've had a situation where our backup-restore flow did not include 
> summary.db component for a large file with a lot of keys. This caused 
> Cassandra on the restored node to take a long time to boot (while it was 
> building the summary).
> The last message was about opening a few sstables. It would be nice to also 
> include a debug line about building summary, which then operators can use to 
> check if a missing summary file is expected. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15791) dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/

2020-05-06 Thread Ekaterina Dimitrova (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-15791:

Fix Version/s: (was: 4.0-beta)
   4.0-alpha

> dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/
> --
>
> Key: CASSANDRA-15791
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15791
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Flakey dtest, failure details below:
> https://jenkins-cm4.apache.org/job/Cassandra-trunk-dtest/69/testReport/junit/dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra-website] branch master updated: Add a dummy file to keep the content directory in git, as it needs to exist to be mounted as a volume in the docker run.

2020-05-06 Thread mck
This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-website.git


The following commit(s) were added to refs/heads/master by this push:
 new 1ad5e74  Add a dummy file to keep the content directory in git, as it 
needs to exist to be mounted as a volume in the docker run.
1ad5e74 is described below

commit 1ad5e747a39bcda133cf747db3c917d427497cdb
Author: Mick Semb Wever 
AuthorDate: Wed May 6 21:07:14 2020 +0200

Add a dummy file to keep the content directory in git, as it needs to exist 
to be mounted as a volume in the docker run.
---
 content/.keepdir | 1 +
 1 file changed, 1 insertion(+)

diff --git a/content/.keepdir b/content/.keepdir
new file mode 100644
index 000..0488e10
--- /dev/null
+++ b/content/.keepdir
@@ -0,0 +1 @@
+# dummy file to keep git directory


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15791) dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/

2020-05-06 Thread Ekaterina Dimitrova (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-15791:

Fix Version/s: 4.0-beta

> dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/
> --
>
> Key: CASSANDRA-15791
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15791
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Flakey dtest, failure details below:
> https://jenkins-cm4.apache.org/job/Cassandra-trunk-dtest/69/testReport/junit/dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15791) dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/

2020-05-06 Thread Ekaterina Dimitrova (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-15791:

 Bug Category: Parent values: Correctness(12982)
   Complexity: Normal
  Component/s: Test/dtest
Discovered By: User Report
 Severity: Normal
   Status: Open  (was: Triage Needed)

> dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/
> --
>
> Key: CASSANDRA-15791
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15791
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Ekaterina Dimitrova
>Priority: Normal
>
> Flakey dtest, failure details below:
> https://jenkins-cm4.apache.org/job/Cassandra-trunk-dtest/69/testReport/junit/dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-15791) dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/

2020-05-06 Thread Ekaterina Dimitrova (Jira)
Ekaterina Dimitrova created CASSANDRA-15791:
---

 Summary: 
dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/
 Key: CASSANDRA-15791
 URL: https://issues.apache.org/jira/browse/CASSANDRA-15791
 Project: Cassandra
  Issue Type: Bug
Reporter: Ekaterina Dimitrova


Flakey dtest, failure details below:
https://jenkins-cm4.apache.org/job/Cassandra-trunk-dtest/69/testReport/junit/dtest.consistency_test/TestAccuracy/test_simple_strategy_each_quorum_counters/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15763) Tunable RangeTombstoneList initial size and resize factor

2020-05-06 Thread Dinesh Joshi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101090#comment-17101090
 ] 

Dinesh Joshi commented on CASSANDRA-15763:
--

Thanks [~yifanc] for the patch. +1.

> Tunable RangeTombstoneList initial size and resize factor
> -
>
> Key: CASSANDRA-15763
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15763
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
>
> Cassandra allocates a backing array with a fixed size of one when creating a 
> new RangeTombstoneList object. The resize factor is fixed to 1.5.
> The initial size and resize factor could be made tunable so it could reduce 
> cost in some cases. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15763) Tunable RangeTombstoneList initial size and resize factor

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15763:
-
Status: Ready to Commit  (was: Review In Progress)

> Tunable RangeTombstoneList initial size and resize factor
> -
>
> Key: CASSANDRA-15763
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15763
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
>
> Cassandra allocates a backing array with a fixed size of one when creating a 
> new RangeTombstoneList object. The resize factor is fixed to 1.5.
> The initial size and resize factor could be made tunable so it could reduce 
> cost in some cases. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15763) Tunable RangeTombstoneList initial size and resize factor

2020-05-06 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-15763:
-
Reviewers: Dinesh Joshi, Dinesh Joshi  (was: Dinesh Joshi)
   Dinesh Joshi, Dinesh Joshi
   Status: Review In Progress  (was: Patch Available)

> Tunable RangeTombstoneList initial size and resize factor
> -
>
> Key: CASSANDRA-15763
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15763
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
>
> Cassandra allocates a backing array with a fixed size of one when creating a 
> new RangeTombstoneList object. The resize factor is fixed to 1.5.
> The initial size and resize factor could be made tunable so it could reduce 
> cost in some cases. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta

2020-05-06 Thread Olivier Michallat (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101000#comment-17101000
 ] 

Olivier Michallat commented on CASSANDRA-15299:
---

{quote}we keep accumulating messages into the payload until the max 
*uncompressed* size is reached
{quote}
So I guess for large messages we would use a similar approach: find out that 
the _uncompressed_ size is above the limit, split it into 128 KiB chunks, then 
compress each chunk separately and wrap it into its own (uncontained) outer 
frame?

I think that's the only approach that will work because we need to decompress 
the first chunk separately on the receiving end, in order to read the expected 
message body size that tells us how many more frames follow. So we can't split 
after compression.

If that's correct, then uncontained frames are never recoverable when 
compression is enabled: we need to decompress all the chunks to keep track of 
the total size.

> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol 
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Sam Tunnicliffe
>Priority: Normal
>  Labels: protocolv5
> Fix For: 4.0-beta
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it 
> introduced checksumming/CRC32 to request and response bodies. It’s an 
> important step forward, but it doesn’t cover the entire stream. In 
> particular, the message header is not covered by a checksum or a crc, which 
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of 
> computing the CRC32 on the bytes written on the wire - losing the properties 
> of the CRC32. In some cases, due to this sequencing, attempting to decompress 
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order, 
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for 
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since 
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression 
> operate on individual message bodies rather than frames of multiple complete 
> messages. In reality, this has several important additional downsides. To 
> name a couple:
> # For compression, we are getting poor compression ratios for smaller 
> messages - when operating on tiny sequences of bytes. In reality, for most 
> small requests and responses we are discarding the compressed value as it’d 
> be smaller than the uncompressed one - incurring both redundant allocations 
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages. 
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the 
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we 
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode 
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed, 
> and that we do it before native protocol v5 graduates from beta. See 
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
>  and 
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderLZ4.java.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15757) CustomIndexTest.indexBuildingPagesLargePartitions is flaky

2020-05-06 Thread Jon Meredith (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100994#comment-17100994
 ] 

Jon Meredith commented on CASSANDRA-15757:
--

Test runs fine for me in a loop now (3744 passes before I got bored).  I had a 
quick glance at the patch and it looks reasonable. Thanks for picking it up and 
fixing it.

> CustomIndexTest.indexBuildingPagesLargePartitions is flaky
> --
>
> Key: CASSANDRA-15757
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15757
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Jon Meredith
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.0-alpha
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> CustomIndexTest.indexBuildingPagesLargePartitions is flaky. Failed in CI and 
> was able to reproduce failure inside IntelliJ by setting test Repeat to ‘Run 
> Until Failure’. Failed after 459 iterations.
> {code}
> java.lang.AssertionError
>   at org.junit.Assert.fail(Assert.java:86)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertTrue(Assert.java:52)
>   at 
> org.apache.cassandra.index.CustomIndexTest.lambda$indexBuildingPagesLargePartitions$1(CustomIndexTest.java:687)
>   at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
>   at 
> org.apache.cassandra.index.CustomIndexTest.indexBuildingPagesLargePartitions(CustomIndexTest.java:687)
>   at jdk.internal.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>   at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>   at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>   at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
>   at 
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
>   at 
> com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:53)
>   at 
> com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
>   at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15583) 4.0 quality testing: Tooling, Bundled and First Party

2020-05-06 Thread Josh McKenzie (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh McKenzie updated CASSANDRA-15583:
--
Fix Version/s: (was: 4.0-rc)
   4.0-beta

> 4.0 quality testing: Tooling, Bundled and First Party
> -
>
> Key: CASSANDRA-15583
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15583
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest
>Reporter: Josh McKenzie
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Test plans should cover bundled first-party tooling and CLIs such as 
> nodetool, cqlsh, and new tools supporting full query and audit logging 
> (CASSANDRA-13983, CASSANDRA-12151).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15579) 4.0 quality testing: Distributed Read/Write Path: Coordination, Replication, and Read Repair

2020-05-06 Thread Josh McKenzie (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh McKenzie updated CASSANDRA-15579:
--
Fix Version/s: (was: 4.0-rc)
   4.0-beta

> 4.0 quality testing: Distributed Read/Write Path: Coordination, Replication, 
> and Read Repair
> 
>
> Key: CASSANDRA-15579
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15579
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/unit
>Reporter: Josh McKenzie
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Testing in this area focuses on non-node-local aspects of the read-write 
> path: coordination, replication, read repair, etc.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15353) GSoD Documentation - Cassandra 4.0

2020-05-06 Thread Josh McKenzie (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh McKenzie updated CASSANDRA-15353:
--
Fix Version/s: (was: 4.0-rc)
   4.0-beta

> GSoD Documentation - Cassandra 4.0
> --
>
> Key: CASSANDRA-15353
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15353
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: DeepakVohra
>Assignee: Deepak Vohra
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Please review, and add comments to, some of the documentation preview for 
> Apache Cassandra 4.0.
> New Features
> ---
> *Audit Logging*
> https://docs.google.com/document/d/1WsQYjDQ9UZ7f8P3eAATht6Vz5oNibTmhsH8cu9viLTk/edit?usp=sharing
> *Full Query Logging*
> https://docs.google.com/document/d/1ZNfs68ZNJXqDc_2XGM6W-tpvnejUfSh8HmkbG8u35Es/edit?usp=sharing
> *Support for Java 11*
> https://docs.google.com/document/d/1v7ffccqk_5Son4iwfuwZae8YUam41quewKi_6Z_PQGw/edit?usp=sharing
> *Transient Replication and Cheap Quorums*
> https://docs.google.com/document/d/1GyJVCmwoMb7OO9NkbGdk26YxHi9fWtfzuTBTiIEpli8/edit?usp=sharing
> *Virtual Tables*
> https://docs.google.com/document/d/1U8qAzQnA2HcoIzTg5IXFUySUZXD1tEIl2bam-514we8/edit?usp=sharing
> Improvements 
> 
> *Improved Streaming*
> https://docs.google.com/document/d/1zd6AuHBgC82v598cuDtVIkxJVDV9c0A2rrMEVFHyB4Y/edit?usp=sharing
> *Improved Internode Messaging*
> https://docs.google.com/document/d/1ub2DBHE7hNEKe4tuCOpdbCZ7qGqCdMAYvSTC-YamuMA/edit?usp=sharing



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15584) 4.0 quality testing: Tooling - External Ecosystem

2020-05-06 Thread Josh McKenzie (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh McKenzie updated CASSANDRA-15584:
--
Fix Version/s: (was: 4.0-rc)
   4.0-beta

> 4.0 quality testing: Tooling - External Ecosystem
> -
>
> Key: CASSANDRA-15584
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15584
> Project: Cassandra
>  Issue Type: Task
>Reporter: Josh McKenzie
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Many users of Apache Cassandra employ open source tooling to automate 
> Cassandra configuration, runtime management, and repair scheduling. Prior to 
> release, we need to confirm that popular third-party tools such as Reaper, 
> Priam, etc. function properly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15580) 4.0 quality testing: Repair

2020-05-06 Thread Josh McKenzie (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh McKenzie updated CASSANDRA-15580:
--
Fix Version/s: (was: 4.0-rc)
   4.0-beta

> 4.0 quality testing: Repair
> ---
>
> Key: CASSANDRA-15580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15580
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest
>Reporter: Josh McKenzie
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0-beta
>
>
> We aim for 4.0 to have the first fully functioning incremental repair 
> solution (CASSANDRA-9143)! Furthermore we aim to verify that all types of 
> repair: (full range, sub range, incremental) function as expected as well as 
> ensuring community tools such as Reaper work. CASSANDRA-3200 adds an 
> experimental option to reduce the amount of data streamed during repair, we 
> should write more tests and see how it works with big nodes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15538) 4.0 quality testing: Local Read/Write Path: Other Areas

2020-05-06 Thread Josh McKenzie (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh McKenzie updated CASSANDRA-15538:
--
Fix Version/s: (was: 4.0)
   4.0-beta

> 4.0 quality testing: Local Read/Write Path: Other Areas
> ---
>
> Key: CASSANDRA-15538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15538
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest
>Reporter: Josh McKenzie
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Testing in this area refers to the local read/write path (StorageProxy, 
> ColumnFamilyStore, Memtable, SSTable reading/writing, etc). We are still 
> finding numerous bugs and issues with the 3.0 storage engine rewrite 
> (CASSANDRA-8099). For 4.0 we want to ensure that we thoroughly cover the 
> local read/write path with techniques such as property-based testing, fuzzing 
> ([example|http://cassandra.apache.org/blog/2018/10/17/finding_bugs_with_property_based_testing.html]),
>  and a source audit.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15588) 4.0 quality testing: Cluster Upgrade

2020-05-06 Thread Josh McKenzie (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh McKenzie updated CASSANDRA-15588:
--
Fix Version/s: (was: 4.0-rc)
   4.0-beta

> 4.0 quality testing: Cluster Upgrade
> 
>
> Key: CASSANDRA-15588
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15588
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest
>Reporter: Josh McKenzie
>Assignee: Sean McCarthy
>Priority: Normal
> Fix For: 4.0-beta
>
>
> We've historically had numerous bugs concerning upgrading clusters from one 
> version to the other. Let's establish the supported upgrade path and ensure 
> that users can safely perform the upgrades in production.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15585) 4.0 quality testing: Test Frameworks, Tooling, Infra / Automation

2020-05-06 Thread Josh McKenzie (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh McKenzie updated CASSANDRA-15585:
--
Fix Version/s: (was: 4.0-rc)
   4.0-beta

> 4.0 quality testing: Test Frameworks, Tooling, Infra / Automation
> -
>
> Key: CASSANDRA-15585
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15585
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest
>Reporter: Josh McKenzie
>Assignee: Jordan West
>Priority: Normal
> Fix For: 4.0-beta
>
>
> This area refers to contributions to test frameworks/tooling (e.g., dtests, 
> QuickTheories, CASSANDRA-14821), and automation enabling those tools to be 
> applied at scale (e.g., replay testing via Spark-based replay of captured FQL 
> logs).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15582) 4.0 quality testing: metrics

2020-05-06 Thread Josh McKenzie (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh McKenzie updated CASSANDRA-15582:
--
Fix Version/s: (was: 4.0-rc)
   4.0-beta

> 4.0 quality testing: metrics
> 
>
> Key: CASSANDRA-15582
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15582
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest
>Reporter: Josh McKenzie
>Assignee: Romain Hardouin
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: Screen Shot 2020-04-07 at 5.47.17 PM.png
>
>
> In past releases we've unknowingly broken metrics integrations and introduced 
> performance regressions in metrics collection and reporting. We strive in 4.0 
> to not do that. Metrics should work well!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15537) 4.0 quality testing: Local Read/Write Path: Upgrade and Diff Test

2020-05-06 Thread Josh McKenzie (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh McKenzie updated CASSANDRA-15537:
--
Fix Version/s: (was: 4.0)
   4.0-beta

> 4.0 quality testing: Local Read/Write Path: Upgrade and Diff Test
> -
>
> Key: CASSANDRA-15537
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15537
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest
>Reporter: Josh McKenzie
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Execution of upgrade and diff tests via cassandra-diff have proven to be one 
> of the most effective approaches toward identifying issues with the local 
> read/write path. These include instances of data loss, data corruption, data 
> resurrection, incorrect responses to queries, incomplete responses, and 
> others. Upgrade and diff tests can be executed concurrent with fault 
> injection (such as host or network failure); as well as during mixed-version 
> scenarios (such as upgrading half of the instances in a cluster, and running 
> upgradesstables on only half of the upgraded instances).
> Upgrade and diff tests are expected to continue through the release cycle, 
> and are a great way for contributors to gain confidence in the correctness of 
> the database under their own workloads.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-14697) Transient Replication 4.0 pre-release followup work

2020-05-06 Thread Josh McKenzie (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh McKenzie updated CASSANDRA-14697:
--
Fix Version/s: (was: 4.0)
   4.0-beta

> Transient Replication 4.0 pre-release followup work
> ---
>
> Key: CASSANDRA-14697
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14697
> Project: Cassandra
>  Issue Type: Task
>  Components: Legacy/Core
>Reporter: Ariel Weisberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> This is an umbrella ticket for linking work done post CASSANDRA-14404.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-14746) Ensure Netty Internode Messaging Refactor is Solid

2020-05-06 Thread Josh McKenzie (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh McKenzie updated CASSANDRA-14746:
--
Fix Version/s: (was: 4.0)

> Ensure Netty Internode Messaging Refactor is Solid
> --
>
> Key: CASSANDRA-14746
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14746
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/Streaming and Messaging
>Reporter: Joey Lynch
>Assignee: Joey Lynch
>Priority: Normal
>  Labels: 4.0-QA
> Fix For: 4.0-beta
>
>
> Before we release 4.0 let's ensure that the internode messaging refactor is 
> 100% solid. As internode messaging is naturally used in many code paths and 
> widely configurable we have a large number of cluster configurations and test 
> configurations that must be vetted.
> We plan to vary the following:
>  * Version of Cassandra 3.0.17 vs 4.0-alpha
>  * Cluster sizes with *multi-dc* deployments ranging from 6 - 100 nodes
>  * Client request rates varying between 1k QPS and 100k QPS of varying sizes 
> and shapes (BATCH, INSERT, SELECT point, SELECT range, etc ...)
>  * Internode compression
>  * Internode SSL (as well as openssl vs jdk)
>  * Internode Coalescing options
> We are looking to measure the following as appropriate:
>  * Latency distributions of reads and writes (lower is better)
>  * Scaling limit, aka maximum throughput before violating p99 latency 
> deadline of 10ms @ LOCAL_QUORUM, on a fixed hardware deployment for 100% 
> writes, 100% reads and 50-50 writes+reads (higher is better)
>  * Thread counts (lower is better)
>  * Context switches (lower is better)
>  * On-CPU time of tasks (higher periods without context switch is better)
>  * GC allocation rates / throughput for a fixed size heap (lower allocation 
> better)
>  * Streaming recovery time for a single node failure, i.e. can Cassandra 
> saturate the NIC
>  
> The goal is that 4.0 should have better latency, more throughput, fewer 
> threads, fewer context switches, less GC allocation, and faster recovery 
> time. I'm putting Jason Brown as the reviewer since he implemented most of 
> the internode refactor.
> Current collaborators driving this QA task: Dinesh Joshi, Jordan West, Joey 
> Lynch (Netflix), Vinay Chella (Netflix)
> Owning committer(s): Jason Brown



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15581) 4.0 quality testing: Compaction

2020-05-06 Thread Josh McKenzie (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh McKenzie updated CASSANDRA-15581:
--
Fix Version/s: (was: 4.0-rc)
   4.0-beta

> 4.0 quality testing: Compaction
> ---
>
> Key: CASSANDRA-15581
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15581
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest
>Reporter: Josh McKenzie
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Alongside the local and distributed read/write paths, we'll also want to 
> validate compaction. CASSANDRA-6696 introduced substantial 
> changes/improvements that require testing (esp. JBOD).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15586) 4.0 quality testing: Cluster Setup and Maintenance

2020-05-06 Thread Josh McKenzie (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh McKenzie updated CASSANDRA-15586:
--
Fix Version/s: (was: 4.0-rc)
   4.0-beta

> 4.0 quality testing: Cluster Setup and Maintenance
> --
>
> Key: CASSANDRA-15586
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15586
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest
>Reporter: Josh McKenzie
>Assignee: Ekaterina Dimitrova
>Priority: Normal
>  Labels: 4.0-QA
> Fix For: 4.0-beta
>
>
> We want 4.0 to be easy for users to setup out of the box and just work. This 
> means having low friction when users download the Cassandra package and start 
> running it. For example, users should be able to easily configure and start 
> new 4.0 clusters and have tokens distributed evenly. Another example is 
> packaging, it should be easy to install Cassandra on all supported platforms 
> (e.g. packaging) and have Cassandra use standard platform integrations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15587) 4.0 quality testing: Platforms and Runtimes

2020-05-06 Thread Josh McKenzie (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh McKenzie updated CASSANDRA-15587:
--
Fix Version/s: (was: 4.0-rc)
   4.0-beta

> 4.0 quality testing: Platforms and Runtimes
> ---
>
> Key: CASSANDRA-15587
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15587
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest
>Reporter: Josh McKenzie
>Priority: Normal
> Fix For: 4.0-beta
>
>
> CASSANDRA-9608 introduces support for Java 11. We'll want to verify that 
> Cassandra under Java 11 meets expectations of stability.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15348) Harry: generator library and extensible framework for fuzz testing Apache Cassandra

2020-05-06 Thread Josh McKenzie (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh McKenzie updated CASSANDRA-15348:
--
Fix Version/s: 4.0-beta

> Harry: generator library and extensible framework for fuzz testing Apache 
> Cassandra
> ---
>
> Key: CASSANDRA-15348
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15348
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 4.0-beta
>
>
> h2. Description:
> This ticket introduces Harry, a component for fuzz testing and verification 
> of the Apache Cassandra clusters at scale. 
> h2. Motivation: 
> Current testing tooling largely tests for common- and edge-cases, and most of 
> the tests use predefined datasets. Property-based tests can help explore a 
> broader range of states, but often require either a complex model or a large 
> state to test against.
> h2. What problems Harry solves:
> Harry allows to run tests that are able to validate state of both dense nodes 
> (to test local read-write path) and large clusters (to test distributed 
> read-write path), and do it efficiently. Main goals, and what sets it apart 
> from the other testing tools is:
>  * The state required for verification should remain as compact as possible.
>  * The verification process itself should be as performant as possible.
>  * Ideally, we'd want a way to verify database state while _continuing_ 
> running state change queries against it.
> h2. What Harry does: 
> To achieve this, Harry defines a model that holds the state of the database, 
> generators that produce reproducible, pseudo-random schemas, mutations, and 
> queries, and a validator that asserts the correctness of the model following 
> execution of generated traffic.
> h2. Harry consists of multiple reusable components:
>  * Generator library: how to create a library of invertible, order-preserving 
> generators for simple and composite data types.
>  * Model and checker: how to use the properties of generators to validate the 
> output of an eventually-consistent database in a linear time.
>  * Runner library: how to create a scheme for reproducible runs, despite the 
> concurrent nature of database and fuzzer itself.
> h2. Short and somewhat approximate description of how Harry achieves this:
> Generation and validation define strict mathematical relations between the 
> generated values and pseudorandom numbers they were generated from. Using 
> these properties, we can store minimal state and check if these properties 
> hold during validation.
> Since Cassandra stores data in rows, we should be able to "inflate" data to 
> insert a row into the database from a single number we call _descriptor_. 
> Each value in the row read from the database can be "deflated" back to the 
> descriptor it was generated from. This way, to precisely verify the state of 
> the row, we only need to know the descriptor it was generated from and a 
> timestamp at which it was inserted.
> Similarly, keys for the inserted row can be "inflated" from a single 64-bit 
> integer, and then "deflated" back to it. To efficiently search for keys, 
> while allowing range scans, our generation scheme preserves the order of the 
> original 64-bit integer. Every pair of keys generated from two 64-bit 
> integers would sort the same way as these integers.
> This way, in order to validate a state of the range of rows queried from the 
> database, it is sufficient to "deflate" its key and data values, use deflated 
> 64-bit key representation to find all descriptors these rows were generated 
> from, and ensure that the given sequence of descriptors could have resulted 
> in the state that database has responded with.
> Using this scheme, we keep a minimum possible amount of data per row, can 
> efficiently generate the data, and backtrack values to the numbers they were 
> generated from. Most of the time, we operate on 64-bit integer values and 
> only use "inflated" objects when running queries against database state, 
> minimizing the amount of required memory.
> h2. Name: 
> Harry (verb). 
> According to Marriam-Webster: 
>   * to torment by or as if by constant attack
>   * persistently carry out attacks on (an enemy or an enemy's territory)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-13606) Improve handling of 2i initialization failures

2020-05-06 Thread Berenguer Blasi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-13606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100877#comment-17100877
 ] 

Berenguer Blasi commented on CASSANDRA-13606:
-

[~adelapena] I think I addressed your comments in the 
[PR|https://github.com/apache/cassandra/pull/570]. New CI runs are also 
included there.

> Improve handling of 2i initialization failures
> --
>
> Key: CASSANDRA-13606
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13606
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Feature/2i Index
>Reporter: Sergio Bossa
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> CASSANDRA-10130 fixes the 2i build management, but initialization failures 
> are still not properly handled, most notably because:
> * Initialization failures make the index non-queryable, but it can still be 
> written to.
> * Initialization failures can be recovered via full rebuilds.
> Both points above are probably suboptimal because the initialization logic 
> could be more complex than just an index build, hence it shouldn't be made 
> recoverable via a simple rebuild, and could cause the index to be fully 
> unavailable not just for reads, but for writes as well.
> So, we should better handle initialization failures by:
> * Allowing the index implementation to specify if unavailable for reads, 
> writes, or both. 
> * Providing a proper method to recover, distinct from index rebuilds.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-14280) Fix timeout test - org.apache.cassandra.cql3.ViewTest

2020-05-06 Thread Ekaterina Dimitrova (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-14280:

Resolution: Duplicate
Status: Resolved  (was: Open)

> Fix timeout test - org.apache.cassandra.cql3.ViewTest
> -
>
> Key: CASSANDRA-14280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14280
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Testing
>Reporter: Dikang Gu
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> The test timeout very often, it seems too big, try to split it into multiple 
> tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-14280) Fix timeout test - org.apache.cassandra.cql3.ViewTest

2020-05-06 Thread Ekaterina Dimitrova (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-14280:

Status: In Progress  (was: Patch Available)

> Fix timeout test - org.apache.cassandra.cql3.ViewTest
> -
>
> Key: CASSANDRA-14280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14280
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Testing
>Reporter: Dikang Gu
>Assignee: Dikang Gu
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> The test timeout very often, it seems too big, try to split it into multiple 
> tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-14280) Fix timeout test - org.apache.cassandra.cql3.ViewTest

2020-05-06 Thread Ekaterina Dimitrova (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-14280:

Status: Patch Available  (was: Ready to Commit)

> Fix timeout test - org.apache.cassandra.cql3.ViewTest
> -
>
> Key: CASSANDRA-14280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14280
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Testing
>Reporter: Dikang Gu
>Assignee: Dikang Gu
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> The test timeout very often, it seems too big, try to split it into multiple 
> tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Assigned] (CASSANDRA-14280) Fix timeout test - org.apache.cassandra.cql3.ViewTest

2020-05-06 Thread Ekaterina Dimitrova (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova reassigned CASSANDRA-14280:
---

Assignee: Ekaterina Dimitrova  (was: Dikang Gu)

> Fix timeout test - org.apache.cassandra.cql3.ViewTest
> -
>
> Key: CASSANDRA-14280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14280
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Testing
>Reporter: Dikang Gu
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> The test timeout very often, it seems too big, try to split it into multiple 
> tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-14280) Fix timeout test - org.apache.cassandra.cql3.ViewTest

2020-05-06 Thread Ekaterina Dimitrova (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100860#comment-17100860
 ] 

Ekaterina Dimitrova commented on CASSANDRA-14280:
-

Looks like this has been fixed long time ago.
https://ci-cassandra.apache.org/job/Cassandra-trunk/lastCompletedBuild/testReport/org.apache.cassandra.cql3/ViewTest/


> Fix timeout test - org.apache.cassandra.cql3.ViewTest
> -
>
> Key: CASSANDRA-14280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14280
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Testing
>Reporter: Dikang Gu
>Assignee: Dikang Gu
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> The test timeout very often, it seems too big, try to split it into multiple 
> tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15262) server_encryption_options is not backwards compatible with 3.11

2020-05-06 Thread Ekaterina Dimitrova (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-15262:

Test and Documentation Plan: 
[trunk|https://github.com/apache/cassandra/compare/trunk...jolynch:CASSANDRA-15262]
[dtest|https://github.com/apache/cassandra-dtest/commit/98c0be8789f1a016a1038bf3337c0fbbc8580bd6]


 Status: Patch Available  (was: In Progress)

> server_encryption_options is not backwards compatible with 3.11
> ---
>
> Key: CASSANDRA-15262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15262
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Config
>Reporter: Joey Lynch
>Assignee: Joey Lynch
>Priority: Normal
> Fix For: 4.0, 4.0-alpha
>
>
> The current `server_encryption_options` configuration options are as follows:
> {noformat}
> server_encryption_options:
> # set to true for allowing secure incoming connections
> enabled: false
> # If enabled and optional are both set to true, encrypted and unencrypted 
> connections are handled on the storage_port
> optional: false
> # if enabled, will open up an encrypted listening socket on 
> ssl_storage_port. Should be used
> # during upgrade to 4.0; otherwise, set to false.
> enable_legacy_ssl_storage_port: false
> # on outbound connections, determine which type of peers to securely 
> connect to. 'enabled' must be set to true.
> internode_encryption: none
> keystore: conf/.keystore
> keystore_password: cassandra
> truststore: conf/.truststore
> truststore_password: cassandra
> # More advanced defaults below:
> # protocol: TLS
> # store_type: JKS
> # cipher_suites: 
> [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA]
> # require_client_auth: false
> # require_endpoint_verification: false
> {noformat}
> A couple of issues here:
> 1. optional defaults to false, which will break existing TLS configurations 
> for (from what I can tell) no particularly good reason
> 2. The provided protocol and cipher suites are not good ideas (in particular 
> encouraging anyone to use CBC ciphers is a bad plan
> I propose that before the 4.0 cut we fixup server_encryption_options and even 
> client_encryption_options :
> # Change the default {{optional}} setting to true. As the new Netty code 
> intelligently decides to open a TLS connection or not this is the more 
> sensible default (saves operators a step while transitioning to TLS as well)
> # Update the defaults to what netty actually defaults to



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15262) server_encryption_options is not backwards compatible with 3.11

2020-05-06 Thread Ekaterina Dimitrova (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-15262:

Reviewers: Benedict Elliott Smith, Ekaterina Dimitrova, Ekaterina Dimitrova 
 (was: Benedict Elliott Smith, Ekaterina Dimitrova)
   Benedict Elliott Smith, Ekaterina Dimitrova, Ekaterina Dimitrova 
 (was: Benedict Elliott Smith)
   Status: Review In Progress  (was: Patch Available)

> server_encryption_options is not backwards compatible with 3.11
> ---
>
> Key: CASSANDRA-15262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15262
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Config
>Reporter: Joey Lynch
>Assignee: Joey Lynch
>Priority: Normal
> Fix For: 4.0, 4.0-alpha
>
>
> The current `server_encryption_options` configuration options are as follows:
> {noformat}
> server_encryption_options:
> # set to true for allowing secure incoming connections
> enabled: false
> # If enabled and optional are both set to true, encrypted and unencrypted 
> connections are handled on the storage_port
> optional: false
> # if enabled, will open up an encrypted listening socket on 
> ssl_storage_port. Should be used
> # during upgrade to 4.0; otherwise, set to false.
> enable_legacy_ssl_storage_port: false
> # on outbound connections, determine which type of peers to securely 
> connect to. 'enabled' must be set to true.
> internode_encryption: none
> keystore: conf/.keystore
> keystore_password: cassandra
> truststore: conf/.truststore
> truststore_password: cassandra
> # More advanced defaults below:
> # protocol: TLS
> # store_type: JKS
> # cipher_suites: 
> [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA]
> # require_client_auth: false
> # require_endpoint_verification: false
> {noformat}
> A couple of issues here:
> 1. optional defaults to false, which will break existing TLS configurations 
> for (from what I can tell) no particularly good reason
> 2. The provided protocol and cipher suites are not good ideas (in particular 
> encouraging anyone to use CBC ciphers is a bad plan
> I propose that before the 4.0 cut we fixup server_encryption_options and even 
> client_encryption_options :
> # Change the default {{optional}} setting to true. As the new Netty code 
> intelligently decides to open a TLS connection or not this is the more 
> sensible default (saves operators a step while transitioning to TLS as well)
> # Update the defaults to what netty actually defaults to



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-14280) Fix timeout test - org.apache.cassandra.cql3.ViewTest

2020-05-06 Thread Ekaterina Dimitrova (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100764#comment-17100764
 ] 

Ekaterina Dimitrova commented on CASSANDRA-14280:
-

[~dikanggu] and [~jay.zhuang],
Do you need a hand with this one? I see CASSANDRA-14194 was closed

> Fix timeout test - org.apache.cassandra.cql3.ViewTest
> -
>
> Key: CASSANDRA-14280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14280
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Testing
>Reporter: Dikang Gu
>Assignee: Dikang Gu
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> The test timeout very often, it seems too big, try to split it into multiple 
> tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15674) liveDiskSpaceUsed and totalDiskSpaceUsed get corrupted if IndexSummaryRedistribution gets interrupted

2020-05-06 Thread Ekaterina Dimitrova (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100762#comment-17100762
 ] 

Ekaterina Dimitrova commented on CASSANDRA-15674:
-

Hi [~dcapwell] and [~marcuse],
Is there anything else required for this one?


> liveDiskSpaceUsed and totalDiskSpaceUsed get corrupted if 
> IndexSummaryRedistribution gets interrupted
> -
>
> Key: CASSANDRA-15674
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15674
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction, Observability/Metrics
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> IndexSummaryRedistribution is a compaction task and as such extends Holder 
> and supports cancelation by throwing a CompactionInterruptedException.  The 
> issue is that IndexSummaryRedistribution tries to use transactions, but 
> mutates the sstable in-place; transaction is unable to roll back.
> This would be fine (only updates summary) if it wasn’t for the fact the task 
> attempts to also mutate the two metrics liveDiskSpaceUsed and 
> totalDiskSpaceUsed, since these can’t be rolled back any cancelation could 
> corrupt these metrics.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Comment Edited] (CASSANDRA-15685) flaky testWithMismatchingPending - org.apache.cassandra.distributed.test.PreviewRepairTest

2020-05-06 Thread Ekaterina Dimitrova (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100759#comment-17100759
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-15685 at 5/6/20, 12:33 PM:
---

Hi [~dcapwell],
Still working on this one? Let me know if you need a hand


was (Author: e.dimitrova):
Hi [~dcapwell],
Still working on this one?

> flaky testWithMismatchingPending - 
> org.apache.cassandra.distributed.test.PreviewRepairTest
> --
>
> Key: CASSANDRA-15685
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15685
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Kevin Gallardo
>Assignee: David Capwell
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Observed in: 
> https://app.circleci.com/pipelines/github/newkek/cassandra/34/workflows/1c6b157d-13c3-48a9-85fb-9fe8c153256b/jobs/191/tests
> Failure:
> {noformat}
> testWithMismatchingPending - 
> org.apache.cassandra.distributed.test.PreviewRepairTest
> junit.framework.AssertionFailedError
>   at 
> org.apache.cassandra.distributed.test.PreviewRepairTest.testWithMismatchingPending(PreviewRepairTest.java:97)
> {noformat}
> [Circle 
> CI|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FCASSANDRA-15685]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15685) flaky testWithMismatchingPending - org.apache.cassandra.distributed.test.PreviewRepairTest

2020-05-06 Thread Ekaterina Dimitrova (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100759#comment-17100759
 ] 

Ekaterina Dimitrova commented on CASSANDRA-15685:
-

Hi [~dcapwell],
Still working on this one?

> flaky testWithMismatchingPending - 
> org.apache.cassandra.distributed.test.PreviewRepairTest
> --
>
> Key: CASSANDRA-15685
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15685
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Kevin Gallardo
>Assignee: David Capwell
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Observed in: 
> https://app.circleci.com/pipelines/github/newkek/cassandra/34/workflows/1c6b157d-13c3-48a9-85fb-9fe8c153256b/jobs/191/tests
> Failure:
> {noformat}
> testWithMismatchingPending - 
> org.apache.cassandra.distributed.test.PreviewRepairTest
> junit.framework.AssertionFailedError
>   at 
> org.apache.cassandra.distributed.test.PreviewRepairTest.testWithMismatchingPending(PreviewRepairTest.java:97)
> {noformat}
> [Circle 
> CI|https://circleci.com/gh/dcapwell/cassandra/tree/bug%2FCASSANDRA-15685]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15773) Add test to cover metrics related to the BufferPool

2020-05-06 Thread Stephen Mallette (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17100695#comment-17100695
 ] 

Stephen Mallette commented on CASSANDRA-15773:
--

[~dcapwell] thanks for the feedback:

> Since we don't free, and the metric is how many chunks are leant out, >= 
> totalBytesRequestedFromPool should hold true and would be more accurate than 
> > 0

yes - that sounds smart to me - I've made that change.

> the test makes an assumption at buffer pool size, which may not be true in CI,

I've initialized the test with a fixed size to the pool: 
https://github.com/apache/cassandra/compare/trunk...spmallette:CASSANDRA-15773-trunk#diff-790a43fcd6ed69f4122cd0fe205c847bR48-R52

> leaking, should cleanup after the test is over; same for the other test

I might not understand something but I thought that this would prevent such 
problems:

https://github.com/apache/cassandra/compare/trunk...spmallette:CASSANDRA-15773-trunk#diff-790a43fcd6ed69f4122cd0fe205c847bR54-R58

which effectively calls:

{code}
static void unsafeReset()
{
localPool.get().unsafeRecycle();
globalPool.unsafeFree();
}
{code}

on {{BufferPool}} itself. Do I need to do something more?



> Add test to cover metrics related to the BufferPool
> ---
>
> Key: CASSANDRA-15773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15773
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/unit
>Reporter: Stephen Mallette
>Assignee: Stephen Mallette
>Priority: Normal
>
> At this time there do not appear to be unit tests to validate 
> {{BufferPoolMetrics}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15729) Jenkins Test Results Report in plaintext for ASF ML

2020-05-06 Thread Michael Semb Wever (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-15729:
---
  Fix Version/s: (was: 4.0-beta)
 4.0-alpha
 3.11.7
 3.0.21
 2.2.17
Source Control Link: 
https://github.com/apache/cassandra/commit/94aee924e2d6b57ce3ecf90d39114246c4f7ec70
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

Committed as 94aee924e2d6b57ce3ecf90d39114246c4f7ec70

> Jenkins Test Results Report in plaintext for ASF ML
> ---
>
> Key: CASSANDRA-15729
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15729
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>  Labels: Jenkins
> Fix For: 2.2.17, 3.0.21, 3.11.7, 4.0-alpha
>
> Attachments: .Screenshot 2020-05-01 at 01.12.28.png
>
>
> The Jenkins pipeline builds now aggregate all test reports.
> For example: 
> - https://ci-cassandra.apache.org/job/Cassandra-trunk/68/testReport/
> - 
> https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-trunk/detail/Cassandra-trunk/68/tests
> But Jenkins can only keep a limited amount of build history, so those links 
> are not permanent, can't be used as references, and don't help for bisecting 
> and blame on regressions (and flakey tests) over a longer period of time.
> The builds@ ML can provide a permanent record of test results. 
> This was first brought up in these two threads: 
> - 
> https://lists.apache.org/thread.html/re8122e4fdd8629e7fbca2abf27d72054b3bc0e3690ece8b8e66f618b%40%3Cdev.cassandra.apache.org%3E
> - 
> https://lists.apache.org/thread.html/ra5f6aeea89546825fe7ccc4a80898c62f8ed57decabf709d81d9c720%40%3Cdev.cassandra.apache.org%3E
> An example plaintext report, to demonstrate feasibility, is available here: 
> https://lists.apache.org/thread.html/r80d13f7af706bf8dfbf2387fab46004c1fbd3917b7bc339c49e69aa8%40%3Cbuilds.cassandra.apache.org%3E
> Hurdles:
>  - the ASF mailing lists won't accept html, attachments, or any message body 
> over 1MB.
>  - packages are used as a differentiator in the final aggregated report. The 
> cqlsh and dtests currently don't specify it. It needs to be added as a 
> "dot-separated" prefix to the testsuite and testcase name.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra] branch cassandra-3.0 updated (196e544 -> bc7ca56)

2020-05-06 Thread mck
This is an automated email from the ASF dual-hosted git repository.

mck pushed a change to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from 196e544  Merge branch 'cassandra-2.2' into cassandra-3.0
 new 94aee92  Jenkins Test Results Report in plaintext for ASF ML
 new bc7ca56  Merge branch 'cassandra-2.2' into cassandra-3.0

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .jenkins/Jenkinsfile | 38 ++
 1 file changed, 26 insertions(+), 12 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk

2020-05-06 Thread mck
This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git

commit 4d27eb424e6971746f41a32bb70a964a386cc30a
Merge: 183812b 2f8fe37
Author: Mick Semb Wever 
AuthorDate: Wed May 6 13:11:25 2020 +0200

Merge branch 'cassandra-3.11' into trunk

 .jenkins/Jenkinsfile | 30 --
 1 file changed, 20 insertions(+), 10 deletions(-)

diff --cc .jenkins/Jenkinsfile
index 3b75ee2,767db54..0a453b8
--- a/.jenkins/Jenkinsfile
+++ b/.jenkins/Jenkinsfile
@@@ -26,157 -26,132 +26,153 @@@
  pipeline {
agent { label 'cassandra' }
stages {
 -  stage('Init') {
 -steps {
 -cleanWs()
 -}
 +stage('Init') {
 +  steps {
 +  cleanWs()
}
 -  stage('Build') {
 -steps {
 -build job: "${env.JOB_NAME}-artifacts"
 -}
 +}
 +stage('Build') {
 +  steps {
 +  build job: "${env.JOB_NAME}-artifacts"
}
 -  stage('Test') {
 -  parallel {
 -stage('stress') {
 -  steps {
 -  build job: "${env.JOB_NAME}-stress-test"
 +}
 +stage('Test') {
 +parallel {
 +  stage('stress') {
 +steps {
- warnError('Tests unstable') {
- build job: "${env.JOB_NAME}-stress-test"
- }
++build job: "${env.JOB_NAME}-stress-test"
 +}
 +post {
 +  success {
 +  warnError('missing test xml files') {
 +  script {
 +  copyTestResults('stress-test')
 +  }
 +  }
}
 -  post {
 -success {
 -warnError('missing test xml files') {
 -script {
 -copyTestResults('stress-test')
 -}
 -}
 -}
 -unstable {
 -warnError('missing test xml files') {
 -script {
 -copyTestResults('stress-test')
 -}
 -}
 -}
 +  unstable {
 +  warnError('missing test xml files') {
 +  script {
 +  copyTestResults('stress-test')
 +  }
 +  }
}
  }
 -stage('JVM DTests') {
 -  steps {
 -  warnError('Tests unstable') {
 -build job: "${env.JOB_NAME}-jvm-dtest"
 +  }
 +  stage('fqltool') {
 +steps {
- warnError('Tests unstable') {
- build job: "${env.JOB_NAME}-fqltool-test"
- }
++build job: "${env.JOB_NAME}-fqltool-test"
 +}
 +post {
 +  success {
 +  warnError('missing test xml files') {
 +  script {
 +  copyTestResults('fqltool-test')
 +  }
}
}
 -  post {
 -success {
 -warnError('missing test xml files') {
 -script {
 -copyTestResults('jvm-dtest')
 -}
 -}
 -}
 -unstable {
 -warnError('missing test xml files') {
 -script {
 -copyTestResults('jvm-dtest')
 -}
 -}
 -}
 +  unstable {
 +  warnError('missing test xml files') {
 +  script {
 +  copyTestResults('fqltool-test')
 +  }
 +  }
}
  }
 -stage('units') {
 -steps {
 -  warnError('Tests unstable') {
 -build job: "${env.JOB_NAME}-test"
 -  }
 -}
 -  post {
 -success {
 -warnError('missing test xml files') {
 -script {
 -copyTestResults('test')
 -}
 -}
 -}
 -unstable {
 -warnError('missing test xml files') {
 -script {
 -copyTestResults('test')
 -}
 -}
 +  }
 +  stage('JVM DTests') {
 +steps {
 +warnError('Tests unstable') {
 +  build job: "${env.JOB_NAME}-jvm-dtest"
  }
 -  }
 

[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0

2020-05-06 Thread mck
This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git

commit bc7ca568eada26867fb9c6c1cdacb9e37aa4a54b
Merge: 196e544 94aee92
Author: Mick Semb Wever 
AuthorDate: Wed May 6 13:09:08 2020 +0200

Merge branch 'cassandra-2.2' into cassandra-3.0

 .jenkins/Jenkinsfile | 38 ++
 1 file changed, 26 insertions(+), 12 deletions(-)



-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra] branch cassandra-2.2 updated: Jenkins Test Results Report in plaintext for ASF ML

2020-05-06 Thread mck
This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch cassandra-2.2
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/cassandra-2.2 by this push:
 new 94aee92  Jenkins Test Results Report in plaintext for ASF ML
94aee92 is described below

commit 94aee924e2d6b57ce3ecf90d39114246c4f7ec70
Author: Mick Semb Wever 
AuthorDate: Fri Apr 17 19:06:22 2020 +0200

Jenkins Test Results Report in plaintext for ASF ML

 patch by Mick Semb Wever; reviewed by Eduard Tudenhöfner for 
CASSANDRA-15729
---
 .jenkins/Jenkinsfile | 38 ++
 1 file changed, 26 insertions(+), 12 deletions(-)

diff --git a/.jenkins/Jenkinsfile b/.jenkins/Jenkinsfile
index 5af2b5a..f7cbefc 100644
--- a/.jenkins/Jenkinsfile
+++ b/.jenkins/Jenkinsfile
@@ -21,7 +21,7 @@
 //
https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy
 //
 // Validate/lint this file using the following command
-// `curl -X POST  -F "jenkinsfile=<.jenkins/Jenkinsfile" 
https://builds.apache.org/pipeline-model-converter/validate`
+// `curl -X POST  -F "jenkinsfile=<.jenkins/Jenkinsfile" 
https://ci-cassandra.apache.org/pipeline-model-converter/validate`
 
 pipeline {
   agent { label 'cassandra' }
@@ -250,19 +250,33 @@ pipeline {
   }
 }
   }
-  }
-  stage('Summary') {
-steps {
-junit '**/TEST*.xml,**/cqlshlib.xml,**/nosetests.xml'
-
-// the following should fail on any installation other than 
builds.apache.org
-//  TODO: keep jenkins infrastructure related settings in 
`cassandra_job_dsl_seed.groovy`
-warnError('cannot send notifications') {
-  slackSend channel: '#cassandra-builds', message: 
"${currentBuild.fullDisplayName} completed: ${currentBuild.result}. See 
${env.BUILD_URL}"
-  emailext to: 'bui...@cassandra.apache.org', subject: "Build 
complete: ${currentBuild.fullDisplayName} [${currentBuild.result}] 
${env.GIT_COMMIT}", body: '${CHANGES}  ${JELLY_SCRIPT,template="text"}'
-}
 }
+stage('Summary') {
+  steps {
+  sh "git clone 
https://gitbox.apache.org/repos/asf/cassandra-builds.git";
+  sh "./cassandra-builds/build-scripts/cassandra-test-report.sh"
+  junit '**/build/test/**/TEST*.xml,**/cqlshlib.xml,**/nosetests.xml'
+
+  // the following should fail on any installation other than 
builds.apache.org
+  //  TODO: keep jenkins infrastructure related settings in 
`cassandra_job_dsl_seed.groovy`
+  warnError('cannot send notifications') {
+  slackSend channel: '#cassandra-builds', message: ":apache: 
<${env.BUILD_URL}|${currentBuild.fullDisplayName}> completed: 
${currentBuild.result}. ${env.GIT_COMMIT}\n${CHANGES}"
+  emailext to: 'bui...@cassandra.apache.org', subject: "Build 
complete: ${currentBuild.fullDisplayName} [${currentBuild.result}] 
${env.GIT_COMMIT}", body: '''
+---
+Build ${ENV,var="JOB_NAME"} #${BUILD_NUMBER} ${BUILD_STATUS}
+URL: ${BUILD_URL}
+---
+Changes:
+${CHANGES}
+---
+Failed Tests:
+${FAILED_TESTS,maxTests=500,showMessage=false,showStack=false}
+---
+${FILE,path="cassandra-test-report.txt"}
+'''
+  }
   }
+}
   }
 }
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra] branch trunk updated (183812b -> 4d27eb4)

2020-05-06 Thread mck
This is an automated email from the ASF dual-hosted git repository.

mck pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from 183812b  Merge branch 'cassandra-3.11' into trunk
 new 94aee92  Jenkins Test Results Report in plaintext for ASF ML
 new bc7ca56  Merge branch 'cassandra-2.2' into cassandra-3.0
 new 2f8fe37  Merge branch 'cassandra-3.0' into cassandra-3.11
 new 4d27eb4  Merge branch 'cassandra-3.11' into trunk

The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .jenkins/Jenkinsfile | 30 --
 1 file changed, 20 insertions(+), 10 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11

2020-05-06 Thread mck
This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git

commit 2f8fe3754d68ebfaf55c1f9fe9e534e4e1800e3a
Merge: 996c0ad bc7ca56
Author: Mick Semb Wever 
AuthorDate: Wed May 6 13:09:58 2020 +0200

Merge branch 'cassandra-3.0' into cassandra-3.11

 .jenkins/Jenkinsfile | 42 +++---
 1 file changed, 27 insertions(+), 15 deletions(-)

diff --cc .jenkins/Jenkinsfile
index 384bcab,22fe692..767db54
--- a/.jenkins/Jenkinsfile
+++ b/.jenkins/Jenkinsfile
@@@ -38,29 -38,6 +38,27 @@@ pipeline 
}
stage('Test') {
parallel {
 +stage('stress') {
 +  steps {
-   warnError('Tests unstable') {
-   build job: "${env.JOB_NAME}-stress-test"
-   }
++  build job: "${env.JOB_NAME}-stress-test"
 +  }
 +  post {
 +success {
 +warnError('missing test xml files') {
 +script {
 +copyTestResults('stress-test')
 +}
 +}
 +}
 +unstable {
 +warnError('missing test xml files') {
 +script {
 +copyTestResults('stress-test')
 +}
 +}
 +}
 +  }
 +}
  stage('JVM DTests') {
steps {
warnError('Tests unstable') {
@@@ -295,43 -249,34 +293,57 @@@
  }
}
  }
 +stage('dtest-offheap') {
 +  steps {
 +  warnError('Tests unstable') {
 +build job: "${env.JOB_NAME}-dtest-offheap"
 +  }
 +  }
 +  post {
 +success {
 +warnError('missing test xml files') {
 +script {
 +copyTestResults('dtest-offheap')
 +}
 +}
 +}
 +unstable {
 +warnError('missing test xml files') {
 +script {
 +copyTestResults('dtest-offheap')
 +}
 +}
 +}
 +  }
 +}
}
-   }
-   stage('Summary') {
- steps {
- junit '**/TEST*.xml,**/cqlshlib.xml,**/nosetests.xml'
- 
- // the following should fail on any installation other than 
builds.apache.org
- //  TODO: keep jenkins infrastructure related settings in 
`cassandra_job_dsl_seed.groovy`
- warnError('cannot send notifications') {
-   slackSend channel: '#cassandra-builds', message: 
"${currentBuild.fullDisplayName} completed: ${currentBuild.result}. See 
${env.BUILD_URL}"
-   emailext to: 'bui...@cassandra.apache.org', subject: "Build 
complete: ${currentBuild.fullDisplayName} [${currentBuild.result}] 
${env.GIT_COMMIT}", body: '${CHANGES}  ${JELLY_SCRIPT,template="text"}'
- }
  }
+ stage('Summary') {
+   steps {
+   sh "git clone 
https://gitbox.apache.org/repos/asf/cassandra-builds.git";
+   sh "./cassandra-builds/build-scripts/cassandra-test-report.sh"
+   junit '**/build/test/**/TEST*.xml,**/cqlshlib.xml,**/nosetests.xml'
+ 
+   // the following should fail on any installation other than 
builds.apache.org
+   //  TODO: keep jenkins infrastructure related settings in 
`cassandra_job_dsl_seed.groovy`
+   warnError('cannot send notifications') {
+   slackSend channel: '#cassandra-builds', message: ":apache: 
<${env.BUILD_URL}|${currentBuild.fullDisplayName}> completed: 
${currentBuild.result}. ${env.GIT_COMMIT}\n${CHANGES}"
+   emailext to: 'bui...@cassandra.apache.org', subject: "Build 
complete: ${currentBuild.fullDisplayName} [${currentBuild.result}] 
${env.GIT_COMMIT}", body: '''
+ 
---
+ Build ${ENV,var="JOB_NAME"} #${BUILD_NUMBER} ${BUILD_STATUS}
+ URL: ${BUILD_URL}
+ 
---
+ Changes:
+ ${CHANGES}
+ 
---
+ Failed Tests:
+ ${FAILED_TESTS,maxTests=500,showMessage=false,showStack=false}
+ 
---
+ ${FILE,path="cassandra-test-report.txt"}
+ '''
+   }
}
+ }
}
  }
  


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional co

[cassandra] branch cassandra-3.11 updated (996c0ad -> 2f8fe37)

2020-05-06 Thread mck
This is an automated email from the ASF dual-hosted git repository.

mck pushed a change to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from 996c0ad  Merge branch 'cassandra-3.0' into cassandra-3.11
 new 94aee92  Jenkins Test Results Report in plaintext for ASF ML
 new bc7ca56  Merge branch 'cassandra-2.2' into cassandra-3.0
 new 2f8fe37  Merge branch 'cassandra-3.0' into cassandra-3.11

The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .jenkins/Jenkinsfile | 42 +++---
 1 file changed, 27 insertions(+), 15 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15729) Jenkins Test Results Report in plaintext for ASF ML

2020-05-06 Thread Michael Semb Wever (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-15729:
---
Reviewers: David Capwell, Eduard Tudenhoefner  (was: David Capwell)

> Jenkins Test Results Report in plaintext for ASF ML
> ---
>
> Key: CASSANDRA-15729
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15729
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>  Labels: Jenkins
> Fix For: 4.0-beta
>
> Attachments: .Screenshot 2020-05-01 at 01.12.28.png
>
>
> The Jenkins pipeline builds now aggregate all test reports.
> For example: 
> - https://ci-cassandra.apache.org/job/Cassandra-trunk/68/testReport/
> - 
> https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-trunk/detail/Cassandra-trunk/68/tests
> But Jenkins can only keep a limited amount of build history, so those links 
> are not permanent, can't be used as references, and don't help for bisecting 
> and blame on regressions (and flakey tests) over a longer period of time.
> The builds@ ML can provide a permanent record of test results. 
> This was first brought up in these two threads: 
> - 
> https://lists.apache.org/thread.html/re8122e4fdd8629e7fbca2abf27d72054b3bc0e3690ece8b8e66f618b%40%3Cdev.cassandra.apache.org%3E
> - 
> https://lists.apache.org/thread.html/ra5f6aeea89546825fe7ccc4a80898c62f8ed57decabf709d81d9c720%40%3Cdev.cassandra.apache.org%3E
> An example plaintext report, to demonstrate feasibility, is available here: 
> https://lists.apache.org/thread.html/r80d13f7af706bf8dfbf2387fab46004c1fbd3917b7bc339c49e69aa8%40%3Cbuilds.cassandra.apache.org%3E
> Hurdles:
>  - the ASF mailing lists won't accept html, attachments, or any message body 
> over 1MB.
>  - packages are used as a differentiator in the final aggregated report. The 
> cqlsh and dtests currently don't specify it. It needs to be added as a 
> "dot-separated" prefix to the testsuite and testcase name.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15729) Jenkins Test Results Report in plaintext for ASF ML

2020-05-06 Thread Michael Semb Wever (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-15729:
---
Status: Ready to Commit  (was: Review In Progress)

> Jenkins Test Results Report in plaintext for ASF ML
> ---
>
> Key: CASSANDRA-15729
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15729
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>  Labels: Jenkins
> Fix For: 4.0-beta
>
> Attachments: .Screenshot 2020-05-01 at 01.12.28.png
>
>
> The Jenkins pipeline builds now aggregate all test reports.
> For example: 
> - https://ci-cassandra.apache.org/job/Cassandra-trunk/68/testReport/
> - 
> https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-trunk/detail/Cassandra-trunk/68/tests
> But Jenkins can only keep a limited amount of build history, so those links 
> are not permanent, can't be used as references, and don't help for bisecting 
> and blame on regressions (and flakey tests) over a longer period of time.
> The builds@ ML can provide a permanent record of test results. 
> This was first brought up in these two threads: 
> - 
> https://lists.apache.org/thread.html/re8122e4fdd8629e7fbca2abf27d72054b3bc0e3690ece8b8e66f618b%40%3Cdev.cassandra.apache.org%3E
> - 
> https://lists.apache.org/thread.html/ra5f6aeea89546825fe7ccc4a80898c62f8ed57decabf709d81d9c720%40%3Cdev.cassandra.apache.org%3E
> An example plaintext report, to demonstrate feasibility, is available here: 
> https://lists.apache.org/thread.html/r80d13f7af706bf8dfbf2387fab46004c1fbd3917b7bc339c49e69aa8%40%3Cbuilds.cassandra.apache.org%3E
> Hurdles:
>  - the ASF mailing lists won't accept html, attachments, or any message body 
> over 1MB.
>  - packages are used as a differentiator in the final aggregated report. The 
> cqlsh and dtests currently don't specify it. It needs to be added as a 
> "dot-separated" prefix to the testsuite and testcase name.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



  1   2   >