[jira] [Commented] (CASSANDRA-13889) cfstats should take sorting and limit parameters

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459445#comment-16459445
 ] 

Marcus Eriksson commented on CASSANDRA-13889:
-

Haha yes it is my fault, sorry

> cfstats should take sorting and limit parameters
> 
>
> Key: CASSANDRA-13889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13889
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jon Haddad
>Assignee: Patrick Bannister
>Priority: Major
> Fix For: 4.0
>
> Attachments: 13889-trunk.txt, sample_output_normal.txt, 
> sample_output_sorted.txt, sample_output_sorted_top3.txt
>
>
> When looking at a problematic node I'm not familiar with, one of the first 
> things I do is check cfstats to identify the tables with the most reads, 
> writes, and data.  This is fine as long as there aren't a lot of tables but 
> once it goes above a dozen it's quite difficult.  cfstats should allow me to 
> sort the results and limit to top K tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-9220) Hostname verification for node-to-node encryption

[Jeremy Hanna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremy Hanna updated CASSANDRA-9220:

Labels: security  (was: )

> Hostname verification for node-to-node encryption
> -
>
> Key: CASSANDRA-9220
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9220
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>  Labels: security
> Fix For: 3.6
>
>
> This patch will will introduce a new ssl server option: 
> {{require_endpoint_verification}}. 
> Setting it will enable hostname verification for inter-node SSL 
> communication. This is necessary to prevent man-in-the-middle attacks when 
> building a trust chain against a common CA. See 
> [here|https://tersesystems.com/2014/03/23/fixing-hostname-verification/] for 
> background details. 
> Clusters that solely rely on importing all node certificates into each trust 
> store (as described 
> [here|http://docs.datastax.com/en/cassandra/2.0/cassandra/security/secureSSLCertificates_t.html])
>  are not effected. 
> Clusters that use the same common CA to sign node certificates are 
> potentially affected. In case the CA signing process will allow other parties 
> to generate certs for different purposes, those certificates could in turn be 
> used for MITM attacks. The provided patch will allow to enable hostname 
> verification to make sure not only to check if the cert is valid but also if 
> it has been created for the host that we're about to connect.
> Corresponding dtest: [Test for 
> CASSANDRA-9220|https://github.com/riptano/cassandra-dtest/pull/237]
> Related patches from the client perspective: 
> [Java|https://datastax-oss.atlassian.net/browse/JAVA-716], 
> [Python|https://datastax-oss.atlassian.net/browse/PYTHON-296]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14197) SSTable upgrade should be automatic

[Kurt Greaves (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459380#comment-16459380
 ] 

Kurt Greaves commented on CASSANDRA-14197:
--

bq. no, the idea is still to only run these upgrade sstable tasks if there are 
no other compactions to run for a table, just that we restrict the number of 
parallel upgrade tasks
Right, I was getting confused with manual upgrades and should have just looked 
at the patch again.

+1 from me too.


> SSTable upgrade should be automatic
> ---
>
> Key: CASSANDRA-14197
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14197
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 4.x
>
>
> Upgradesstables should run automatically on node upgrade



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13889) cfstats should take sorting and limit parameters

[Patrick Bannister (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459378#comment-16459378
 ] 

Patrick Bannister commented on CASSANDRA-13889:
---

[~krummas], it looks like the failure is caused by the stat "Old SSTable 
count," which doesn't appear in the test's expected output for tablestats. I 
don't recognize that stat - are you testing changes that include modifications 
to TableStatsPrinter?

For what it's worth, I synced my fork with the trunk branch on the 
apache/cassandra repo and reran these tests, and they still pass in trunk on my 
environment (Ubuntu 16.04 LTS on a 64-bit architecture).

> cfstats should take sorting and limit parameters
> 
>
> Key: CASSANDRA-13889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13889
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jon Haddad
>Assignee: Patrick Bannister
>Priority: Major
> Fix For: 4.0
>
> Attachments: 13889-trunk.txt, sample_output_normal.txt, 
> sample_output_sorted.txt, sample_output_sorted_top3.txt
>
>
> When looking at a problematic node I'm not familiar with, one of the first 
> things I do is check cfstats to identify the tables with the most reads, 
> writes, and data.  This is fine as long as there aren't a lot of tables but 
> once it goes above a dozen it's quite difficult.  cfstats should allow me to 
> sort the results and limit to top K tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-11163) Summaries are needlessly rebuilt when the BF FP ratio is changed

[Kurt Greaves (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459370#comment-16459370
 ] 

Kurt Greaves commented on CASSANDRA-11163:
--

Hm, sorry about that. Originally started with second delays but changed it to 
speed up the test. Thought ms would be safe in this day and age. 2018 and all 
that.

{quote}How should we test against that?
{quote}
We don't, just have to go with the lowest resolution which will hopefully be 
seconds. No point putting workarounds in to speed up a test by a few seconds on 
_some_ platforms. Having said that, what filesystem is jenkins running? Seems 
odd it'd only have second precision. But then again it's probably an ancient 
server.

> Summaries are needlessly rebuilt when the BF FP ratio is changed
> 
>
> Key: CASSANDRA-11163
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11163
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Brandon Williams
>Assignee: Kurt Greaves
>Priority: Major
> Fix For: 4.0, 3.0.17, 3.11.3
>
>
> This is from trunk, but I also saw this happen on 2.0:
> Before:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 221460
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-6-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-7-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root104178 Feb 11 23:50 ma-5-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> root@bw-1:/srv/cassandra# md5sum 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ma-5-big-Summary.db
> 5fca154fc790f7cfa37e8ad6d1c7552c
> {noformat}
> BF ratio changed, node restarted:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 242168
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> -rw-r--r-- 1 

[jira] [Commented] (CASSANDRA-11163) Summaries are needlessly rebuilt when the BF FP ratio is changed

[mck (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459274#comment-16459274
 ] 

mck commented on CASSANDRA-11163:
-

[~cnlwsu], oh! not os but filesystem, and circleci and jenkins differ there. 
Ouch. 
How should we test against that?

> Summaries are needlessly rebuilt when the BF FP ratio is changed
> 
>
> Key: CASSANDRA-11163
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11163
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Brandon Williams
>Assignee: Kurt Greaves
>Priority: Major
> Fix For: 4.0, 3.0.17, 3.11.3
>
>
> This is from trunk, but I also saw this happen on 2.0:
> Before:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 221460
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-6-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-7-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root104178 Feb 11 23:50 ma-5-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> root@bw-1:/srv/cassandra# md5sum 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ma-5-big-Summary.db
> 5fca154fc790f7cfa37e8ad6d1c7552c
> {noformat}
> BF ratio changed, node restarted:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 242168
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 12 00:03 ma-8-big-TOC.txt
> -rw-r--r-- 1 root root 14902 Feb 12 00:03 ma-8-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 12 00:03 ma-8-big-Statistics.db
> -rw-r--r-- 1 root root   1458631 Feb 12 00:03 ma-8-big-Index.db
> -rw-r--r-- 1 root root 10808 Feb 12 00:03 ma-8-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 12 00:03 ma-8-big-Digest.crc32
> -rw-r--r-- 1 root root  19660275 Feb 12 00:03 

[jira] [Commented] (CASSANDRA-11163) Summaries are needlessly rebuilt when the BF FP ratio is changed

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459269#comment-16459269
 ] 

Chris Lohfink commented on CASSANDRA-11163:
---

the granularity of the file timestamp depends on the filesystem

> Summaries are needlessly rebuilt when the BF FP ratio is changed
> 
>
> Key: CASSANDRA-11163
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11163
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Brandon Williams
>Assignee: Kurt Greaves
>Priority: Major
> Fix For: 4.0, 3.0.17, 3.11.3
>
>
> This is from trunk, but I also saw this happen on 2.0:
> Before:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 221460
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-6-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-7-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root104178 Feb 11 23:50 ma-5-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> root@bw-1:/srv/cassandra# md5sum 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ma-5-big-Summary.db
> 5fca154fc790f7cfa37e8ad6d1c7552c
> {noformat}
> BF ratio changed, node restarted:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 242168
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 12 00:03 ma-8-big-TOC.txt
> -rw-r--r-- 1 root root 14902 Feb 12 00:03 ma-8-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 12 00:03 ma-8-big-Statistics.db
> -rw-r--r-- 1 root root   1458631 Feb 12 00:03 ma-8-big-Index.db
> -rw-r--r-- 1 root root 10808 Feb 12 00:03 ma-8-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 12 00:03 ma-8-big-Digest.crc32
> -rw-r--r-- 1 root root  19660275 Feb 12 00:03 ma-8-big-Data.db
> -rw-r--r-- 1 

[jira] [Commented] (CASSANDRA-11163) Summaries are needlessly rebuilt when the BF FP ratio is changed

[mck (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459267#comment-16459267
 ] 

mck commented on CASSANDRA-11163:
-

{quote}Cherry-plicked into 3.0 as 
[e16f0ed0698c5cb47ab2bb0a0b04966d5bdbcde0|https://github.com/apache/cassandra/commit/e16f0ed0698c5cb47ab2bb0a0b04966d5bdbcde0]
 and merged upwards
{quote}
Thanks [~iamaleksey] for fixing this.
{quote}We should be more careful about committing things.
{quote}
Yes, I should have noticed that {{SSTableReaderTest}} was a new failure in  
[https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-3.0-test-all/248/#showFailuresLink]
 

Up until now I've only been using circleci unit test reports, as illustrated in 
the "branch|testall|dtest" comment above, which reported it all green.

As CircleCI hasn't failed on this test, what's the difference here? 
Aleksey's circleci build #234 [passed 
SSTableReaderTest|https://circleci.com/gh/iamaleksey/cassandra/234#tests/containers/2]
 , as did Kurt's 
[here|https://circleci.com/gh/kgreav/cassandra/116#tests/containers/1]

 

If CircleCi and Jenkins can both identify failures separately then "test-all" 
column should be split into two, to link to each.

Otherwise, does Jenkins have any form of alerting or reporting on tests that go 
from stable to flakey? Would have been ideal to have automatically have receive 
the breakage information before it affected you [~iamaleksey]

> Summaries are needlessly rebuilt when the BF FP ratio is changed
> 
>
> Key: CASSANDRA-11163
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11163
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Brandon Williams
>Assignee: Kurt Greaves
>Priority: Major
> Fix For: 4.0, 3.0.17, 3.11.3
>
>
> This is from trunk, but I also saw this happen on 2.0:
> Before:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 221460
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-6-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-7-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root104178 Feb 11 23:50 ma-5-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> root@bw-1:/srv/cassandra# md5sum 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ma-5-big-Summary.db
> 5fca154fc790f7cfa37e8ad6d1c7552c
> {noformat}
> BF ratio changed, node restarted:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 242168
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 

[jira] [Commented] (CASSANDRA-12464) Investigate the potential improvement of parallelism on higher level compactions in LCS

[Wei Deng (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458922#comment-16458922
 ] 

Wei Deng commented on CASSANDRA-12464:
--

Thanks. Updated the JIRA description.

> Investigate the potential improvement of parallelism on higher level 
> compactions in LCS
> ---
>
> Key: CASSANDRA-12464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12464
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Wei Deng
>Priority: Major
>  Labels: lcs, lhf, performance
>
> According to LevelDB's design doc 
> [here|https://github.com/google/leveldb/blob/master/doc/impl.md#compactions], 
> "A compaction merges the contents of the picked files to produce a sequence 
> of level-(L+1) files", it will "switch to producing a new level-(L+1) file 
> after the current output file has reached the target file size" (in our case 
> 160MB), it will also "switch to a new output file when the key range of the 
> current output file has grown enough to overlap more than ten level-(L+2) 
> files". This is to ensure "that a later compaction of a level-(L+1) file will 
> not pick up too much data from level-(L+2)."
> Our current code in LeveledCompactionStrategy doesn't implement this last 
> rule, but we might be able to quickly implement it and see how much a 
> compaction throughput improvement it can deliver. Potentially we can create a 
> scenario where a number of large L0 SSTables are present (e.g. 200GB after 
> switching from STCS) and let it to create thousands of L1 SSTables overflow, 
> and see how fast LCS can digest this much data from L1 and properly 
> upper-level them to completion.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-12464) Investigate the potential improvement of parallelism on higher level compactions in LCS

[Wei Deng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wei Deng updated CASSANDRA-12464:
-
Description: 
According to LevelDB's design doc 
[here|https://github.com/google/leveldb/blob/master/doc/impl.md#compactions], 
"A compaction merges the contents of the picked files to produce a sequence of 
level-(L+1) files", it will "switch to producing a new level-(L+1) file after 
the current output file has reached the target file size" (in our case 160MB), 
it will also "switch to a new output file when the key range of the current 
output file has grown enough to overlap more than ten level-(L+2) files". This 
is to ensure "that a later compaction of a level-(L+1) file will not pick up 
too much data from level-(L+2)."

Our current code in LeveledCompactionStrategy doesn't implement this last rule, 
but we might be able to quickly implement it and see how much a compaction 
throughput improvement it can deliver. Potentially we can create a scenario 
where a number of large L0 SSTables are present (e.g. 200GB after switching 
from STCS) and let it to create thousands of L1 SSTables overflow, and see how 
fast LCS can digest this much data from L1 and properly upper-level them to 
completion.

  was:
According to LevelDB's design doc 
[here|https://github.com/google/leveldb/blob/master/doc/impl.html#L115-L116], 
"A compaction merges the contents of the picked files to produce a sequence of 
level-(L+1) files", it will "switch to producing a new level-(L+1) file after 
the current output file has reached the target file size" (in our case 160MB), 
it will also "switch to a new output file when the key range of the current 
output file has grown enough to overlap more than ten level-(L+2) files". This 
is to ensure "that a later compaction of a level-(L+1) file will not pick up 
too much data from level-(L+2)."

Our current code in LeveledCompactionStrategy doesn't implement this last rule, 
but we might be able to quickly implement it and see how much a compaction 
throughput improvement it can deliver. Potentially we can create a scenario 
where a number of large L0 SSTables are present (e.g. 200GB after switching 
from STCS) and let it to create thousands of L1 SSTables overflow, and see how 
fast LCS can digest this much data from L1 and properly upper-level them to 
completion.


> Investigate the potential improvement of parallelism on higher level 
> compactions in LCS
> ---
>
> Key: CASSANDRA-12464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12464
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Wei Deng
>Priority: Major
>  Labels: lcs, lhf, performance
>
> According to LevelDB's design doc 
> [here|https://github.com/google/leveldb/blob/master/doc/impl.md#compactions], 
> "A compaction merges the contents of the picked files to produce a sequence 
> of level-(L+1) files", it will "switch to producing a new level-(L+1) file 
> after the current output file has reached the target file size" (in our case 
> 160MB), it will also "switch to a new output file when the key range of the 
> current output file has grown enough to overlap more than ten level-(L+2) 
> files". This is to ensure "that a later compaction of a level-(L+1) file will 
> not pick up too much data from level-(L+2)."
> Our current code in LeveledCompactionStrategy doesn't implement this last 
> rule, but we might be able to quickly implement it and see how much a 
> compaction throughput improvement it can deliver. Potentially we can create a 
> scenario where a number of large L0 SSTables are present (e.g. 200GB after 
> switching from STCS) and let it to create thousands of L1 SSTables overflow, 
> and see how fast LCS can digest this much data from L1 and properly 
> upper-level them to completion.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12743) Assertion error while running compaction

[Jay Zhuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458854#comment-16458854
 ] 

Jay Zhuang commented on CASSANDRA-12743:


There're a few dTest failures for each branch. Trying to confirm they're not 
introduced by this patch.

> Assertion error while running compaction 
> -
>
> Key: CASSANDRA-12743
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12743
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: unix
>Reporter: Jean-Baptiste Le Duigou
>Assignee: Jay Zhuang
>Priority: Major
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
>
> While running compaction I run into an error sometimes :
> {noformat}
> nodetool compact
> error: null
> -- StackTrace --
> java.lang.AssertionError
> at 
> org.apache.cassandra.io.compress.CompressionMetadata$Chunk.(CompressionMetadata.java:463)
> at 
> org.apache.cassandra.io.compress.CompressionMetadata.chunkFor(CompressionMetadata.java:228)
> at 
> org.apache.cassandra.io.util.CompressedSegmentedFile.createMappedSegments(CompressedSegmentedFile.java:80)
> at 
> org.apache.cassandra.io.util.CompressedPoolingSegmentedFile.(CompressedPoolingSegmentedFile.java:38)
> at 
> org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:101)
> at 
> org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:198)
> at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.openEarly(BigTableWriter.java:315)
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.maybeReopenEarly(SSTableRewriter.java:171)
> at 
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:116)
> at 
> org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.append(DefaultCompactionWriter.java:64)
> at 
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:184)
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at 
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:74)
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> at 
> org.apache.cassandra.db.compaction.CompactionManager$8.runMayThrow(CompactionManager.java:599)
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> Why is that happening?
> Is there anyway to provide more details (e.g. which SSTable cannot be 
> compacted)?
> We are using Cassandra 2.2.7



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14387) SSTableReaderTest#testOpeningSSTable fails on macOS

[Dinesh Joshi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458843#comment-16458843
 ] 

Dinesh Joshi commented on CASSANDRA-14387:
--

Thank you for taking care of this [~iamaleksey]

> SSTableReaderTest#testOpeningSSTable fails on macOS
> ---
>
> Key: CASSANDRA-14387
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14387
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Minor
> Fix For: 4.0, 3.0.17, 3.11.3
>
>
> I ran into an issue with {{SSTableReaderTest#testOpeningSSTable}} test 
> failure on macOS. The reason for failure seems that on macOS, the file 
> modification timestamps are at a second granularity (See: 
> https://stackoverflow.com/questions/18403588/how-to-return-millisecond-information-for-file-access-on-mac-os-x-in-java
>  and 
> https://developer.apple.com/legacy/library/technotes/tn/tn1150.html#HFSPlusDates).
>  The fix is simple - bumping up the sleep time to 1 second instead of 10ms.
> {noformat}
> [junit] -  ---
> [junit] Testcase: 
> testOpeningSSTable(org.apache.cassandra.io.sstable.SSTableReaderTest):  FAILED
> [junit] Bloomfilter was not recreated
> [junit] junit.framework.AssertionFailedError: Bloomfilter was not 
> recreated
> [junit]   at 
> org.apache.cassandra.io.sstable.SSTableReaderTest.testOpeningSSTable(SSTableReaderTest.java:421)
> [junit]
> [junit]
> [junit] Test org.apache.cassandra.io.sstable.SSTableReaderTest FAILED
> {noformat}
> Related issue: CASSANDRA-11163



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14430) Flaky dTest: repair_tests.repair_test.TestRepair.test_simple_sequential_repair

[Jay Zhuang (JIRA)]

Jay Zhuang created CASSANDRA-14430:
--

 Summary: Flaky dTest:  
repair_tests.repair_test.TestRepair.test_simple_sequential_repair
 Key: CASSANDRA-14430
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14430
 Project: Cassandra
  Issue Type: Bug
  Components: Testing
Reporter: Jay Zhuang


https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-dtest/lastCompletedBuild/testReport/repair_tests.repair_test/TestRepair/test_simple_sequential_repair/

 
{noformat}
Failed 5 times in the last 30 runs. Flakiness: 24%, Stability: 83%

Error Message
cassandra.cluster.NoHostAvailable: ('Unable to complete the operation against 
any hosts', {: ConnectionException('Host has been 
marked down or removed',), : 
ConnectionException('Pool is shutdown',)})
Stacktrace
self = 

def test_simple_sequential_repair(self):
"""
Calls simple repair test with a sequential repair
"""
>   self._simple_repair(sequential=True)

repair_tests/repair_test.py:350: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
repair_tests/repair_test.py:440: in _simple_repair
self._populate_cluster()
repair_tests/repair_test.py:115: in _populate_cluster
insert_c1c2(session, keys=list(range(1001, 2001)), 
consistency=ConsistencyLevel.ALL)
tools/data.py:29: in insert_c1c2
execute_concurrent_with_args(session, statement, [['k{}'.format(k)] for k 
in keys])
../venv/src/cassandra-driver/cassandra/concurrent.py:239: in 
execute_concurrent_with_args
return execute_concurrent(session, zip(cycle((statement,)), parameters), 
*args, **kwargs)
../venv/src/cassandra-driver/cassandra/concurrent.py:95: in execute_concurrent
return executor.execute(concurrency, raise_on_first_error)
../venv/src/cassandra-driver/cassandra/concurrent.py:202: in execute
return super(ConcurrentExecutorListResults, self).execute(concurrency, 
fail_fast)
../venv/src/cassandra-driver/cassandra/concurrent.py:121: in execute
return self._results()
../venv/src/cassandra-driver/cassandra/concurrent.py:220: in _results
self._raise(self._exception)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

exc = NoHostAvailable('Unable to complete the operation against any hosts', 
{: ConnectionException('Host has been marked down 
or removed',), : ConnectionException('Pool is 
shutdown',)})

@staticmethod
def _raise(exc):
if six.PY2 and isinstance(exc, tuple):
(exc_type, value, traceback) = exc
six.reraise(exc_type, value, traceback)
else:
>   raise exc
E   cassandra.cluster.NoHostAvailable: ('Unable to complete the 
operation against any hosts', {: 
ConnectionException('Host has been marked down or removed',), : ConnectionException('Pool is shutdown',)})

../venv/src/cassandra-driver/cassandra/concurrent.py:168: NoHostAvailable
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14429) Flaky dTest: bootstrap_test.TestBootstrap.test_local_quorum_bootstrap

[Jay Zhuang (JIRA)]

Jay Zhuang created CASSANDRA-14429:
--

 Summary: Flaky dTest: 
bootstrap_test.TestBootstrap.test_local_quorum_bootstrap
 Key: CASSANDRA-14429
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14429
 Project: Cassandra
  Issue Type: Bug
  Components: Testing
Reporter: Jay Zhuang


https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-dtest/lastCompletedBuild/testReport/bootstrap_test/TestBootstrap/test_local_quorum_bootstrap/

{noformat}
Failed 13 times in the last 30 runs. Flakiness: 41%, Stability: 56%

Error Message
Failed: Timeout >900.0s
Stacktrace
self = 

def test_local_quorum_bootstrap(self):
"""
Test that CL local_quorum works while a node is bootstrapping.
@jira_ticket CASSANDRA-8058
"""
cluster = self.cluster
cluster.populate([1, 1])
cluster.start()

node1 = cluster.nodes['node1']
yaml_config = """
# Create the keyspace and table
keyspace: keyspace1
keyspace_definition: |
  CREATE KEYSPACE keyspace1 WITH replication = {'class': 
'NetworkTopologyStrategy', 'dc1': 1, 'dc2': 1};
table: users
table_definition:
  CREATE TABLE users (
username text,
first_name text,
last_name text,
email text,
PRIMARY KEY(username)
  ) WITH compaction = {'class':'SizeTieredCompactionStrategy'};
insert:
  partitions: fixed(1)
  batchtype: UNLOGGED
queries:
  read:
cql: select * from users where username = ?
fields: samerow
"""
with tempfile.NamedTemporaryFile(mode='w+') as stress_config:
stress_config.write(yaml_config)
stress_config.flush()
node1.stress(['user', 'profile=' + stress_config.name, 'n=2M', 
'no-warmup',
> 'ops(insert=1)', '-rate', 'threads=50'])

bootstrap_test.py:439: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../venv/src/ccm/ccmlib/node.py:1339: in stress
return handle_external_tool_process(p, ['stress'] + stress_options)
../venv/src/ccm/ccmlib/node.py:2101: in handle_external_tool_process
out, err = process.communicate()
/usr/lib/python3.6/subprocess.py:843: in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
/usr/lib/python3.6/subprocess.py:1514: in _communicate
ready = selector.select(timeout)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = , timeout = None

def select(self, timeout=None):
if timeout is None:
timeout = None
elif timeout <= 0:
timeout = 0
else:
# poll() has a resolution of 1 millisecond, round away from
# zero to wait *at least* timeout seconds.
timeout = math.ceil(timeout * 1e3)
ready = []
try:
>   fd_event_list = self._poll.poll(timeout)
E   Failed: Timeout >900.0s

/usr/lib/python3.6/selectors.py:376: Failed
{noformat}




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14387) SSTableReaderTest#testOpeningSSTable fails on macOS

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458824#comment-16458824
 ] 

Aleksey Yeschenko commented on CASSANDRA-14387:
---

Cherry-plicked into 3.0 as 
[e16f0ed0698c5cb47ab2bb0a0b04966d5bdbcde0|https://github.com/apache/cassandra/commit/e16f0ed0698c5cb47ab2bb0a0b04966d5bdbcde0]
 and merged upwards, replacing the somewhat silly millis to millis conversion 
with {{TimeUnit.sleep()}} in the process.

> SSTableReaderTest#testOpeningSSTable fails on macOS
> ---
>
> Key: CASSANDRA-14387
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14387
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Minor
> Fix For: 4.0, 3.0.17, 3.11.3
>
>
> I ran into an issue with {{SSTableReaderTest#testOpeningSSTable}} test 
> failure on macOS. The reason for failure seems that on macOS, the file 
> modification timestamps are at a second granularity (See: 
> https://stackoverflow.com/questions/18403588/how-to-return-millisecond-information-for-file-access-on-mac-os-x-in-java
>  and 
> https://developer.apple.com/legacy/library/technotes/tn/tn1150.html#HFSPlusDates).
>  The fix is simple - bumping up the sleep time to 1 second instead of 10ms.
> {noformat}
> [junit] -  ---
> [junit] Testcase: 
> testOpeningSSTable(org.apache.cassandra.io.sstable.SSTableReaderTest):  FAILED
> [junit] Bloomfilter was not recreated
> [junit] junit.framework.AssertionFailedError: Bloomfilter was not 
> recreated
> [junit]   at 
> org.apache.cassandra.io.sstable.SSTableReaderTest.testOpeningSSTable(SSTableReaderTest.java:421)
> [junit]
> [junit]
> [junit] Test org.apache.cassandra.io.sstable.SSTableReaderTest FAILED
> {noformat}
> Related issue: CASSANDRA-11163



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14387) SSTableReaderTest#testOpeningSSTable fails on macOS

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-14387:
--
Fix Version/s: 3.11.3
   3.0.17

> SSTableReaderTest#testOpeningSSTable fails on macOS
> ---
>
> Key: CASSANDRA-14387
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14387
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Minor
> Fix For: 4.0, 3.0.17, 3.11.3
>
>
> I ran into an issue with {{SSTableReaderTest#testOpeningSSTable}} test 
> failure on macOS. The reason for failure seems that on macOS, the file 
> modification timestamps are at a second granularity (See: 
> https://stackoverflow.com/questions/18403588/how-to-return-millisecond-information-for-file-access-on-mac-os-x-in-java
>  and 
> https://developer.apple.com/legacy/library/technotes/tn/tn1150.html#HFSPlusDates).
>  The fix is simple - bumping up the sleep time to 1 second instead of 10ms.
> {noformat}
> [junit] -  ---
> [junit] Testcase: 
> testOpeningSSTable(org.apache.cassandra.io.sstable.SSTableReaderTest):  FAILED
> [junit] Bloomfilter was not recreated
> [junit] junit.framework.AssertionFailedError: Bloomfilter was not 
> recreated
> [junit]   at 
> org.apache.cassandra.io.sstable.SSTableReaderTest.testOpeningSSTable(SSTableReaderTest.java:421)
> [junit]
> [junit]
> [junit] Test org.apache.cassandra.io.sstable.SSTableReaderTest FAILED
> {noformat}
> Related issue: CASSANDRA-11163



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[aleksey]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2fe4b9dc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2fe4b9dc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2fe4b9dc

Branch: refs/heads/trunk
Commit: 2fe4b9dc69a919cadd6f78e9a4e259e6740b127f
Parents: 9fecf94 b67d6fb
Author: Aleksey Yeshchenko 
Authored: Mon Apr 30 19:12:16 2018 +0100
Committer: Aleksey Yeshchenko 
Committed: Mon Apr 30 19:12:16 2018 +0100

--
 .../org/apache/cassandra/io/sstable/SSTableReaderTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2fe4b9dc/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[1/6] cassandra git commit: Fix junit failure for SSTableReaderTest#testOpeningSSTable

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 b2f6ce961 -> e16f0ed06
  refs/heads/cassandra-3.11 32f9e6763 -> b67d6fb60
  refs/heads/trunk 9fecf9477 -> 2fe4b9dc6


Fix junit failure for SSTableReaderTest#testOpeningSSTable

Patch by Dinesh Joshi; Reviewed by Chris Lohfink for CASSANDRA-14387


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e16f0ed0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e16f0ed0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e16f0ed0

Branch: refs/heads/cassandra-3.0
Commit: e16f0ed0698c5cb47ab2bb0a0b04966d5bdbcde0
Parents: b2f6ce9
Author: Dinesh A. Joshi 
Authored: Mon Apr 16 10:18:22 2018 -0700
Committer: Aleksey Yeshchenko 
Committed: Mon Apr 30 19:09:01 2018 +0100

--
 .../org/apache/cassandra/io/sstable/SSTableReaderTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e16f0ed0/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
--
diff --git a/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java 
b/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
index 50519b2..4ca6ec0 100644
--- a/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
+++ b/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
@@ -385,7 +385,7 @@ public class SSTableReaderTest
 long bloomModified = Files.getLastModifiedTime(bloomPath).toMillis();
 long summaryModified = 
Files.getLastModifiedTime(summaryPath).toMillis();
 
-Thread.sleep(TimeUnit.MILLISECONDS.toMillis(10)); // sleep to ensure 
modified time will be different
+TimeUnit.MILLISECONDS.sleep(1000); // sleep to ensure modified time 
will be different
 
 // Offline tests
 // check that bloomfilter/summary ARE NOT regenerated
@@ -432,7 +432,7 @@ public class SSTableReaderTest
 summaryModified = Files.getLastModifiedTime(summaryPath).toMillis();
 summaryFile.delete();
 
-Thread.sleep(TimeUnit.MILLISECONDS.toMillis(10)); // sleep to ensure 
modified time will be different
+TimeUnit.MILLISECONDS.sleep(1000); // sleep to ensure modified time 
will be different
 bloomModified = Files.getLastModifiedTime(bloomPath).toMillis();
 
 target = SSTableReader.open(desc, components, store.metadata);
@@ -449,7 +449,7 @@ public class SSTableReaderTest
 summaryModified = Files.getLastModifiedTime(summaryPath).toMillis();
 target = SSTableReader.open(desc, components, store.metadata, false, 
false);
 
-Thread.sleep(TimeUnit.MILLISECONDS.toMillis(10)); // sleep to ensure 
modified time will be different
+TimeUnit.MILLISECONDS.sleep(1000); // sleep to ensure modified time 
will be different
 assertEquals(bloomModified, 
Files.getLastModifiedTime(bloomPath).toMillis());
 assertEquals(summaryModified, 
Files.getLastModifiedTime(summaryPath).toMillis());
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[aleksey]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b67d6fb6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b67d6fb6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b67d6fb6

Branch: refs/heads/trunk
Commit: b67d6fb6003640ee93b2a5232249ee362097aca8
Parents: 32f9e67 e16f0ed
Author: Aleksey Yeshchenko 
Authored: Mon Apr 30 19:10:23 2018 +0100
Committer: Aleksey Yeshchenko 
Committed: Mon Apr 30 19:10:23 2018 +0100

--
 .../org/apache/cassandra/io/sstable/SSTableReaderTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b67d6fb6/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[aleksey]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b67d6fb6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b67d6fb6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b67d6fb6

Branch: refs/heads/cassandra-3.11
Commit: b67d6fb6003640ee93b2a5232249ee362097aca8
Parents: 32f9e67 e16f0ed
Author: Aleksey Yeshchenko 
Authored: Mon Apr 30 19:10:23 2018 +0100
Committer: Aleksey Yeshchenko 
Committed: Mon Apr 30 19:10:23 2018 +0100

--
 .../org/apache/cassandra/io/sstable/SSTableReaderTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b67d6fb6/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
--


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[3/6] cassandra git commit: Fix junit failure for SSTableReaderTest#testOpeningSSTable

[aleksey]

Fix junit failure for SSTableReaderTest#testOpeningSSTable

Patch by Dinesh Joshi; Reviewed by Chris Lohfink for CASSANDRA-14387


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e16f0ed0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e16f0ed0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e16f0ed0

Branch: refs/heads/trunk
Commit: e16f0ed0698c5cb47ab2bb0a0b04966d5bdbcde0
Parents: b2f6ce9
Author: Dinesh A. Joshi 
Authored: Mon Apr 16 10:18:22 2018 -0700
Committer: Aleksey Yeshchenko 
Committed: Mon Apr 30 19:09:01 2018 +0100

--
 .../org/apache/cassandra/io/sstable/SSTableReaderTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e16f0ed0/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
--
diff --git a/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java 
b/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
index 50519b2..4ca6ec0 100644
--- a/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
+++ b/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
@@ -385,7 +385,7 @@ public class SSTableReaderTest
 long bloomModified = Files.getLastModifiedTime(bloomPath).toMillis();
 long summaryModified = 
Files.getLastModifiedTime(summaryPath).toMillis();
 
-Thread.sleep(TimeUnit.MILLISECONDS.toMillis(10)); // sleep to ensure 
modified time will be different
+TimeUnit.MILLISECONDS.sleep(1000); // sleep to ensure modified time 
will be different
 
 // Offline tests
 // check that bloomfilter/summary ARE NOT regenerated
@@ -432,7 +432,7 @@ public class SSTableReaderTest
 summaryModified = Files.getLastModifiedTime(summaryPath).toMillis();
 summaryFile.delete();
 
-Thread.sleep(TimeUnit.MILLISECONDS.toMillis(10)); // sleep to ensure 
modified time will be different
+TimeUnit.MILLISECONDS.sleep(1000); // sleep to ensure modified time 
will be different
 bloomModified = Files.getLastModifiedTime(bloomPath).toMillis();
 
 target = SSTableReader.open(desc, components, store.metadata);
@@ -449,7 +449,7 @@ public class SSTableReaderTest
 summaryModified = Files.getLastModifiedTime(summaryPath).toMillis();
 target = SSTableReader.open(desc, components, store.metadata, false, 
false);
 
-Thread.sleep(TimeUnit.MILLISECONDS.toMillis(10)); // sleep to ensure 
modified time will be different
+TimeUnit.MILLISECONDS.sleep(1000); // sleep to ensure modified time 
will be different
 assertEquals(bloomModified, 
Files.getLastModifiedTime(bloomPath).toMillis());
 assertEquals(summaryModified, 
Files.getLastModifiedTime(summaryPath).toMillis());
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[2/6] cassandra git commit: Fix junit failure for SSTableReaderTest#testOpeningSSTable

[aleksey]

Fix junit failure for SSTableReaderTest#testOpeningSSTable

Patch by Dinesh Joshi; Reviewed by Chris Lohfink for CASSANDRA-14387


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e16f0ed0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e16f0ed0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e16f0ed0

Branch: refs/heads/cassandra-3.11
Commit: e16f0ed0698c5cb47ab2bb0a0b04966d5bdbcde0
Parents: b2f6ce9
Author: Dinesh A. Joshi 
Authored: Mon Apr 16 10:18:22 2018 -0700
Committer: Aleksey Yeshchenko 
Committed: Mon Apr 30 19:09:01 2018 +0100

--
 .../org/apache/cassandra/io/sstable/SSTableReaderTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e16f0ed0/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
--
diff --git a/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java 
b/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
index 50519b2..4ca6ec0 100644
--- a/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
+++ b/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
@@ -385,7 +385,7 @@ public class SSTableReaderTest
 long bloomModified = Files.getLastModifiedTime(bloomPath).toMillis();
 long summaryModified = 
Files.getLastModifiedTime(summaryPath).toMillis();
 
-Thread.sleep(TimeUnit.MILLISECONDS.toMillis(10)); // sleep to ensure 
modified time will be different
+TimeUnit.MILLISECONDS.sleep(1000); // sleep to ensure modified time 
will be different
 
 // Offline tests
 // check that bloomfilter/summary ARE NOT regenerated
@@ -432,7 +432,7 @@ public class SSTableReaderTest
 summaryModified = Files.getLastModifiedTime(summaryPath).toMillis();
 summaryFile.delete();
 
-Thread.sleep(TimeUnit.MILLISECONDS.toMillis(10)); // sleep to ensure 
modified time will be different
+TimeUnit.MILLISECONDS.sleep(1000); // sleep to ensure modified time 
will be different
 bloomModified = Files.getLastModifiedTime(bloomPath).toMillis();
 
 target = SSTableReader.open(desc, components, store.metadata);
@@ -449,7 +449,7 @@ public class SSTableReaderTest
 summaryModified = Files.getLastModifiedTime(summaryPath).toMillis();
 target = SSTableReader.open(desc, components, store.metadata, false, 
false);
 
-Thread.sleep(TimeUnit.MILLISECONDS.toMillis(10)); // sleep to ensure 
modified time will be different
+TimeUnit.MILLISECONDS.sleep(1000); // sleep to ensure modified time 
will be different
 assertEquals(bloomModified, 
Files.getLastModifiedTime(bloomPath).toMillis());
 assertEquals(summaryModified, 
Files.getLastModifiedTime(summaryPath).toMillis());
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13529) cassandra-stress light-weight transaction support

[Jaydeepkumar Chovatia (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458821#comment-16458821
 ] 

Jaydeepkumar Chovatia commented on CASSANDRA-13529:
---

A gentle ping [~jasobrown]

> cassandra-stress light-weight transaction support
> -
>
> Key: CASSANDRA-13529
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13529
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Stress
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Minor
> Fix For: 4.x
>
> Attachments: 13529.txt, lwttest.yaml
>
>
> It would be nice to have a light-weight transaction support in 
> cassandra-stress.
> Although currently in cassandra-stress we can achieve light-weight 
> transaction partially by using static conditions like "IF col1 != null" or 
> "IF not EXIST". 
> If would be ideal to have full fledged light-weight transaction support like 
> "IF col1 = ? and col2 = ?". One way to implement is to read values from 
> Cassandra and use that in the condition so it will execute all the paxos 
> phases in Cassandra.
> Please find git link for the patch: 
> https://github.com/apache/cassandra/compare/trunk...jaydeepkumar1984:13529-trunk?expand=1
> ||trunk|
> |[branch|https://github.com/jaydeepkumar1984/cassandra/tree/13529-trunk]|
> |[utests|https://circleci.com/gh/jaydeepkumar1984/cassandra/8]|



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13740) Orphan hint file gets created while node is being removed from cluster

[Jaydeepkumar Chovatia (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458820#comment-16458820
 ] 

Jaydeepkumar Chovatia commented on CASSANDRA-13740:
---

Thank You!!! [~iamaleksey]

> Orphan hint file gets created while node is being removed from cluster
> --
>
> Key: CASSANDRA-13740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13740
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core, Hints
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Minor
> Fix For: 4.0, 3.0.17, 3.11.3
>
> Attachments: 13740-3.0.15.txt, gossip_hang_test.py
>
>
> I have found this new issue during my test, whenever node is being removed 
> then hint file for that node gets written and stays inside the hint directory 
> forever. I debugged the code and found that it is due to the race condition 
> between [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L195]
>  and [HintsWriteExecutor.java::closeWriter | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L106]
> . 
>  
> *Time t1* Node is down, as a result Hints are being written by 
> [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L195]
> *Time t2* Node is removed from cluster as a result it calls 
> [HintsService.java-exciseStore | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L327]
>  which removes hint files for the node being removed
> *Time t3* Mutation stage keeps pumping Hints through [HintService.java::write 
> | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L145]
>  which again calls [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L215]
>  and new orphan file gets created
> I was writing a new dtest for {CASSANDRA-13562, CASSANDRA-13308} and that 
> helped me reproduce this new bug. I will submit patch for this new dtest 
> later.
> I also tried following to check how this orphan hint file responds:
> 1. I tried {{nodetool truncatehints }} but it fails as node is no 
> longer part of the ring
> 2. I then tried {{nodetool truncatehints}}, that still doesn’t remove hint 
> file because it is not yet included in the [dispatchDequeue | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsStore.java#L53]
> Reproducible steps:
> Please find dTest python file {{gossip_hang_test.py}} attached which 
> reproduces this bug.
> Solution:
> This is due to race condition as mentioned above. Since 
> {{HintsWriteExecutor.java}} creates thread pool with only 1 worker, so 
> solution becomes little simple. Whenever we [HintService.java::excise | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L303]
>  a host, just store it in-memory, and check for already evicted host inside 
> [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L215].
>  If already evicted host is found then ignore hints.
> Jaydeep



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13740) Orphan hint file gets created while node is being removed from cluster

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-13740:
--
   Resolution: Fixed
Fix Version/s: (was: 3.11.x)
   (was: 3.0.x)
   3.11.3
   3.0.17
   4.0
   Status: Resolved  (was: Patch Available)

> Orphan hint file gets created while node is being removed from cluster
> --
>
> Key: CASSANDRA-13740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13740
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core, Hints
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Minor
> Fix For: 4.0, 3.0.17, 3.11.3
>
> Attachments: 13740-3.0.15.txt, gossip_hang_test.py
>
>
> I have found this new issue during my test, whenever node is being removed 
> then hint file for that node gets written and stays inside the hint directory 
> forever. I debugged the code and found that it is due to the race condition 
> between [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L195]
>  and [HintsWriteExecutor.java::closeWriter | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L106]
> . 
>  
> *Time t1* Node is down, as a result Hints are being written by 
> [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L195]
> *Time t2* Node is removed from cluster as a result it calls 
> [HintsService.java-exciseStore | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L327]
>  which removes hint files for the node being removed
> *Time t3* Mutation stage keeps pumping Hints through [HintService.java::write 
> | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L145]
>  which again calls [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L215]
>  and new orphan file gets created
> I was writing a new dtest for {CASSANDRA-13562, CASSANDRA-13308} and that 
> helped me reproduce this new bug. I will submit patch for this new dtest 
> later.
> I also tried following to check how this orphan hint file responds:
> 1. I tried {{nodetool truncatehints }} but it fails as node is no 
> longer part of the ring
> 2. I then tried {{nodetool truncatehints}}, that still doesn’t remove hint 
> file because it is not yet included in the [dispatchDequeue | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsStore.java#L53]
> Reproducible steps:
> Please find dTest python file {{gossip_hang_test.py}} attached which 
> reproduces this bug.
> Solution:
> This is due to race condition as mentioned above. Since 
> {{HintsWriteExecutor.java}} creates thread pool with only 1 worker, so 
> solution becomes little simple. Whenever we [HintService.java::excise | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L303]
>  a host, just store it in-memory, and check for already evicted host inside 
> [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L215].
>  If already evicted host is found then ignore hints.
> Jaydeep



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13740) Orphan hint file gets created while node is being removed from cluster

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458817#comment-16458817
 ] 

Aleksey Yeschenko commented on CASSANDRA-13740:
---

Made some changes and committed to 3.0 as 
[b2f6ce961f38a3e4cd744e102026bf7a471056c9|https://github.com/apache/cassandra/commit/b2f6ce961f38a3e4cd744e102026bf7a471056c9]
 and merged upwards.

Changes made:
- fixed {{excise()}} to properly handle non-existing stores instead of 
re-initializing them
- changed the delay to be min rpc timeout + write rpc timeout, which roughly 
the time you may expect a new hint written after node decom

Thank you for you patience and for the added tests.

> Orphan hint file gets created while node is being removed from cluster
> --
>
> Key: CASSANDRA-13740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13740
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core, Hints
>Reporter: Jaydeepkumar Chovatia
>Assignee: Jaydeepkumar Chovatia
>Priority: Minor
> Fix For: 3.0.x, 3.11.x
>
> Attachments: 13740-3.0.15.txt, gossip_hang_test.py
>
>
> I have found this new issue during my test, whenever node is being removed 
> then hint file for that node gets written and stays inside the hint directory 
> forever. I debugged the code and found that it is due to the race condition 
> between [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L195]
>  and [HintsWriteExecutor.java::closeWriter | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L106]
> . 
>  
> *Time t1* Node is down, as a result Hints are being written by 
> [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L195]
> *Time t2* Node is removed from cluster as a result it calls 
> [HintsService.java-exciseStore | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L327]
>  which removes hint files for the node being removed
> *Time t3* Mutation stage keeps pumping Hints through [HintService.java::write 
> | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L145]
>  which again calls [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L215]
>  and new orphan file gets created
> I was writing a new dtest for {CASSANDRA-13562, CASSANDRA-13308} and that 
> helped me reproduce this new bug. I will submit patch for this new dtest 
> later.
> I also tried following to check how this orphan hint file responds:
> 1. I tried {{nodetool truncatehints }} but it fails as node is no 
> longer part of the ring
> 2. I then tried {{nodetool truncatehints}}, that still doesn’t remove hint 
> file because it is not yet included in the [dispatchDequeue | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsStore.java#L53]
> Reproducible steps:
> Please find dTest python file {{gossip_hang_test.py}} attached which 
> reproduces this bug.
> Solution:
> This is due to race condition as mentioned above. Since 
> {{HintsWriteExecutor.java}} creates thread pool with only 1 worker, so 
> solution becomes little simple. Whenever we [HintService.java::excise | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsService.java#L303]
>  a host, just store it in-memory, and check for already evicted host inside 
> [HintsWriteExecutor.java::flush | 
> https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/hints/HintsWriteExecutor.java#L215].
>  If already evicted host is found then ignore hints.
> Jaydeep



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[3/6] cassandra git commit: Delay hints store excise by write timeout to avoid race with decommission

[aleksey]

Delay hints store excise by write timeout to avoid race with decommission

patch by Jaydeepkumar Chovatia; reviewed by Aleksey Yeschenko for
CASSANDRA-13740


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b2f6ce96
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b2f6ce96
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b2f6ce96

Branch: refs/heads/trunk
Commit: b2f6ce961f38a3e4cd744e102026bf7a471056c9
Parents: bc1f841
Author: Jaydeepkumar Chovatia 
Authored: Thu Aug 3 15:34:26 2017 -0700
Committer: Aleksey Yeshchenko 
Committed: Mon Apr 30 17:32:05 2018 +0100

--
 CHANGES.txt |   1 +
 .../apache/cassandra/hints/HintsCatalog.java|   7 ++
 .../apache/cassandra/hints/HintsService.java|   2 +-
 .../org/apache/cassandra/hints/HintsStore.java  |   7 ++
 .../cassandra/service/StorageService.java   |   7 +-
 .../cassandra/hints/HintsCatalogTest.java   | 114 ++-
 6 files changed, 135 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2f6ce96/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index cf470d6..857cf96 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.17
+ * Delay hints store excise by write timeout to avoid race with decommission 
(CASSANDRA-13740)
  * Deprecate background repair and probablistic read_repair_chance table 
options
(CASSANDRA-13910)
  * Add missed CQL keywords to documentation (CASSANDRA-14359)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2f6ce96/src/java/org/apache/cassandra/hints/HintsCatalog.java
--
diff --git a/src/java/org/apache/cassandra/hints/HintsCatalog.java 
b/src/java/org/apache/cassandra/hints/HintsCatalog.java
index 6d01629..d1f6fba 100644
--- a/src/java/org/apache/cassandra/hints/HintsCatalog.java
+++ b/src/java/org/apache/cassandra/hints/HintsCatalog.java
@@ -23,6 +23,7 @@ import java.nio.file.Files;
 import java.util.*;
 import java.util.concurrent.ConcurrentHashMap;
 import java.util.stream.Stream;
+import javax.annotation.Nullable;
 
 import com.google.common.collect.ImmutableMap;
 
@@ -94,6 +95,12 @@ final class HintsCatalog
  : store;
 }
 
+@Nullable
+HintsStore getNullable(UUID hostId)
+{
+return stores.get(hostId);
+}
+
 /**
  * Delete all hints for all host ids.
  *

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2f6ce96/src/java/org/apache/cassandra/hints/HintsService.java
--
diff --git a/src/java/org/apache/cassandra/hints/HintsService.java 
b/src/java/org/apache/cassandra/hints/HintsService.java
index 9cd4ed3..268ee1f 100644
--- a/src/java/org/apache/cassandra/hints/HintsService.java
+++ b/src/java/org/apache/cassandra/hints/HintsService.java
@@ -302,7 +302,7 @@ public final class HintsService implements HintsServiceMBean
  */
 public void excise(UUID hostId)
 {
-HintsStore store = catalog.get(hostId);
+HintsStore store = catalog.getNullable(hostId);
 if (store == null)
 return;
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2f6ce96/src/java/org/apache/cassandra/hints/HintsStore.java
--
diff --git a/src/java/org/apache/cassandra/hints/HintsStore.java 
b/src/java/org/apache/cassandra/hints/HintsStore.java
index c066331..bb3aa0f 100644
--- a/src/java/org/apache/cassandra/hints/HintsStore.java
+++ b/src/java/org/apache/cassandra/hints/HintsStore.java
@@ -25,6 +25,7 @@ import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.ConcurrentLinkedDeque;
 import java.util.concurrent.ConcurrentLinkedQueue;
 
+import com.google.common.annotations.VisibleForTesting;
 import com.google.common.collect.ImmutableMap;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -77,6 +78,12 @@ final class HintsStore
 return new HintsStore(hostId, hintsDirectory, writerParams, 
descriptors);
 }
 
+@VisibleForTesting
+int getDispatchQueueSize()
+{
+return dispatchDequeue.size();
+}
+
 InetAddress address()
 {
 return StorageService.instance.getEndpointForHostId(hostId);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2f6ce96/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 

[1/6] cassandra git commit: Delay hints store excise by write timeout to avoid race with decommission

[aleksey]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 bc1f84129 -> b2f6ce961
  refs/heads/cassandra-3.11 684120deb -> 32f9e6763
  refs/heads/trunk 71a27ee2b -> 9fecf9477


Delay hints store excise by write timeout to avoid race with decommission

patch by Jaydeepkumar Chovatia; reviewed by Aleksey Yeschenko for
CASSANDRA-13740


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b2f6ce96
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b2f6ce96
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b2f6ce96

Branch: refs/heads/cassandra-3.0
Commit: b2f6ce961f38a3e4cd744e102026bf7a471056c9
Parents: bc1f841
Author: Jaydeepkumar Chovatia 
Authored: Thu Aug 3 15:34:26 2017 -0700
Committer: Aleksey Yeshchenko 
Committed: Mon Apr 30 17:32:05 2018 +0100

--
 CHANGES.txt |   1 +
 .../apache/cassandra/hints/HintsCatalog.java|   7 ++
 .../apache/cassandra/hints/HintsService.java|   2 +-
 .../org/apache/cassandra/hints/HintsStore.java  |   7 ++
 .../cassandra/service/StorageService.java   |   7 +-
 .../cassandra/hints/HintsCatalogTest.java   | 114 ++-
 6 files changed, 135 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2f6ce96/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index cf470d6..857cf96 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.17
+ * Delay hints store excise by write timeout to avoid race with decommission 
(CASSANDRA-13740)
  * Deprecate background repair and probablistic read_repair_chance table 
options
(CASSANDRA-13910)
  * Add missed CQL keywords to documentation (CASSANDRA-14359)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2f6ce96/src/java/org/apache/cassandra/hints/HintsCatalog.java
--
diff --git a/src/java/org/apache/cassandra/hints/HintsCatalog.java 
b/src/java/org/apache/cassandra/hints/HintsCatalog.java
index 6d01629..d1f6fba 100644
--- a/src/java/org/apache/cassandra/hints/HintsCatalog.java
+++ b/src/java/org/apache/cassandra/hints/HintsCatalog.java
@@ -23,6 +23,7 @@ import java.nio.file.Files;
 import java.util.*;
 import java.util.concurrent.ConcurrentHashMap;
 import java.util.stream.Stream;
+import javax.annotation.Nullable;
 
 import com.google.common.collect.ImmutableMap;
 
@@ -94,6 +95,12 @@ final class HintsCatalog
  : store;
 }
 
+@Nullable
+HintsStore getNullable(UUID hostId)
+{
+return stores.get(hostId);
+}
+
 /**
  * Delete all hints for all host ids.
  *

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2f6ce96/src/java/org/apache/cassandra/hints/HintsService.java
--
diff --git a/src/java/org/apache/cassandra/hints/HintsService.java 
b/src/java/org/apache/cassandra/hints/HintsService.java
index 9cd4ed3..268ee1f 100644
--- a/src/java/org/apache/cassandra/hints/HintsService.java
+++ b/src/java/org/apache/cassandra/hints/HintsService.java
@@ -302,7 +302,7 @@ public final class HintsService implements HintsServiceMBean
  */
 public void excise(UUID hostId)
 {
-HintsStore store = catalog.get(hostId);
+HintsStore store = catalog.getNullable(hostId);
 if (store == null)
 return;
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2f6ce96/src/java/org/apache/cassandra/hints/HintsStore.java
--
diff --git a/src/java/org/apache/cassandra/hints/HintsStore.java 
b/src/java/org/apache/cassandra/hints/HintsStore.java
index c066331..bb3aa0f 100644
--- a/src/java/org/apache/cassandra/hints/HintsStore.java
+++ b/src/java/org/apache/cassandra/hints/HintsStore.java
@@ -25,6 +25,7 @@ import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.ConcurrentLinkedDeque;
 import java.util.concurrent.ConcurrentLinkedQueue;
 
+import com.google.common.annotations.VisibleForTesting;
 import com.google.common.collect.ImmutableMap;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -77,6 +78,12 @@ final class HintsStore
 return new HintsStore(hostId, hintsDirectory, writerParams, 
descriptors);
 }
 
+@VisibleForTesting
+int getDispatchQueueSize()
+{
+return dispatchDequeue.size();
+}
+
 InetAddress address()
 {
 return StorageService.instance.getEndpointForHostId(hostId);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2f6ce96/src/java/org/apache/cassandra/service/StorageService.java

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[aleksey]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/32f9e676
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/32f9e676
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/32f9e676

Branch: refs/heads/cassandra-3.11
Commit: 32f9e6763b7a857a015da0b20af1d3ff486da0c9
Parents: 684120d b2f6ce9
Author: Aleksey Yeshchenko 
Authored: Mon Apr 30 18:48:08 2018 +0100
Committer: Aleksey Yeshchenko 
Committed: Mon Apr 30 18:48:08 2018 +0100

--
 CHANGES.txt |   1 +
 .../apache/cassandra/hints/HintsCatalog.java|   7 ++
 .../apache/cassandra/hints/HintsService.java|   2 +-
 .../org/apache/cassandra/hints/HintsStore.java  |   7 ++
 .../cassandra/service/StorageService.java   |   7 +-
 .../cassandra/hints/HintsCatalogTest.java   | 120 ++-
 6 files changed, 138 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/32f9e676/CHANGES.txt
--
diff --cc CHANGES.txt
index 5450322,857cf96..c392059
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,5 +1,16 @@@
 -3.0.17
 +3.11.3
 + * Allow existing nodes to use all peers in shadow round (CASSANDRA-13851)
 + * Fix cqlsh to read connection.ssl cqlshrc option again (CASSANDRA-14299)
 + * Downgrade log level to trace for CommitLogSegmentManager (CASSANDRA-14370)
 + * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
 + * Serialize empty buffer as empty string for json output format 
(CASSANDRA-14245)
 + * Allow logging implementation to be interchanged for embedded testing 
(CASSANDRA-13396)
 + * SASI tokenizer for simple delimiter based entries (CASSANDRA-14247)
 + * Fix Loss of digits when doing CAST from varint/bigint to decimal 
(CASSANDRA-14170)
 + * RateBasedBackPressure unnecessarily invokes a lock on the Guava 
RateLimiter (CASSANDRA-14163)
 + * Fix wildcard GROUP BY queries (CASSANDRA-14209)
 +Merged from 3.0:
+  * Delay hints store excise by write timeout to avoid race with decommission 
(CASSANDRA-13740)
   * Deprecate background repair and probablistic read_repair_chance table 
options
 (CASSANDRA-13910)
   * Add missed CQL keywords to documentation (CASSANDRA-14359)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32f9e676/src/java/org/apache/cassandra/hints/HintsService.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32f9e676/src/java/org/apache/cassandra/hints/HintsStore.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32f9e676/src/java/org/apache/cassandra/service/StorageService.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32f9e676/test/unit/org/apache/cassandra/hints/HintsCatalogTest.java
--
diff --cc test/unit/org/apache/cassandra/hints/HintsCatalogTest.java
index a255338,928fd31..68acd0c
--- a/test/unit/org/apache/cassandra/hints/HintsCatalogTest.java
+++ b/test/unit/org/apache/cassandra/hints/HintsCatalogTest.java
@@@ -43,7 -69,21 +69,7 @@@ public class HintsCatalogTes
  }
  }
  
- public static void loadCompletenessAndOrderTest(File directory) throws 
IOException
 -@Test
 -public void exciseHintFiles() throws IOException
 -{
 -File directory = Files.createTempDirectory(null).toFile();
 -try
 -{
 -exciseHintFiles(directory);
 -}
 -finally
 -{
 -directory.deleteOnExit();
 -}
 -}
 -
+ private void loadCompletenessAndOrderTest(File directory) throws 
IOException
  {
  UUID hostId1 = UUID.randomUUID();
  UUID hostId2 = UUID.randomUUID();
@@@ -79,37 -119,39 +105,84 @@@
  assertNull(store2.poll());
  }
  
 +@Test
 +public void deleteHintsTest() throws IOException
 +{
 +File directory = Files.createTempDirectory(null).toFile();
 +UUID hostId1 = UUID.randomUUID();
 +UUID hostId2 = UUID.randomUUID();
 +long now = System.currentTimeMillis();
 +writeDescriptor(directory, new HintsDescriptor(hostId1, now));
- writeDescriptor(directory, new HintsDescriptor(hostId1, now+1));
- writeDescriptor(directory, new HintsDescriptor(hostId2, now+2));
- writeDescriptor(directory, new HintsDescriptor(hostId2, now+3));
++writeDescriptor(directory, new HintsDescriptor(hostId1, now + 1));
++writeDescriptor(directory, new HintsDescriptor(hostId2, 

[2/6] cassandra git commit: Delay hints store excise by write timeout to avoid race with decommission

[aleksey]

Delay hints store excise by write timeout to avoid race with decommission

patch by Jaydeepkumar Chovatia; reviewed by Aleksey Yeschenko for
CASSANDRA-13740


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b2f6ce96
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b2f6ce96
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b2f6ce96

Branch: refs/heads/cassandra-3.11
Commit: b2f6ce961f38a3e4cd744e102026bf7a471056c9
Parents: bc1f841
Author: Jaydeepkumar Chovatia 
Authored: Thu Aug 3 15:34:26 2017 -0700
Committer: Aleksey Yeshchenko 
Committed: Mon Apr 30 17:32:05 2018 +0100

--
 CHANGES.txt |   1 +
 .../apache/cassandra/hints/HintsCatalog.java|   7 ++
 .../apache/cassandra/hints/HintsService.java|   2 +-
 .../org/apache/cassandra/hints/HintsStore.java  |   7 ++
 .../cassandra/service/StorageService.java   |   7 +-
 .../cassandra/hints/HintsCatalogTest.java   | 114 ++-
 6 files changed, 135 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2f6ce96/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index cf470d6..857cf96 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.17
+ * Delay hints store excise by write timeout to avoid race with decommission 
(CASSANDRA-13740)
  * Deprecate background repair and probablistic read_repair_chance table 
options
(CASSANDRA-13910)
  * Add missed CQL keywords to documentation (CASSANDRA-14359)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2f6ce96/src/java/org/apache/cassandra/hints/HintsCatalog.java
--
diff --git a/src/java/org/apache/cassandra/hints/HintsCatalog.java 
b/src/java/org/apache/cassandra/hints/HintsCatalog.java
index 6d01629..d1f6fba 100644
--- a/src/java/org/apache/cassandra/hints/HintsCatalog.java
+++ b/src/java/org/apache/cassandra/hints/HintsCatalog.java
@@ -23,6 +23,7 @@ import java.nio.file.Files;
 import java.util.*;
 import java.util.concurrent.ConcurrentHashMap;
 import java.util.stream.Stream;
+import javax.annotation.Nullable;
 
 import com.google.common.collect.ImmutableMap;
 
@@ -94,6 +95,12 @@ final class HintsCatalog
  : store;
 }
 
+@Nullable
+HintsStore getNullable(UUID hostId)
+{
+return stores.get(hostId);
+}
+
 /**
  * Delete all hints for all host ids.
  *

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2f6ce96/src/java/org/apache/cassandra/hints/HintsService.java
--
diff --git a/src/java/org/apache/cassandra/hints/HintsService.java 
b/src/java/org/apache/cassandra/hints/HintsService.java
index 9cd4ed3..268ee1f 100644
--- a/src/java/org/apache/cassandra/hints/HintsService.java
+++ b/src/java/org/apache/cassandra/hints/HintsService.java
@@ -302,7 +302,7 @@ public final class HintsService implements HintsServiceMBean
  */
 public void excise(UUID hostId)
 {
-HintsStore store = catalog.get(hostId);
+HintsStore store = catalog.getNullable(hostId);
 if (store == null)
 return;
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2f6ce96/src/java/org/apache/cassandra/hints/HintsStore.java
--
diff --git a/src/java/org/apache/cassandra/hints/HintsStore.java 
b/src/java/org/apache/cassandra/hints/HintsStore.java
index c066331..bb3aa0f 100644
--- a/src/java/org/apache/cassandra/hints/HintsStore.java
+++ b/src/java/org/apache/cassandra/hints/HintsStore.java
@@ -25,6 +25,7 @@ import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.ConcurrentLinkedDeque;
 import java.util.concurrent.ConcurrentLinkedQueue;
 
+import com.google.common.annotations.VisibleForTesting;
 import com.google.common.collect.ImmutableMap;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -77,6 +78,12 @@ final class HintsStore
 return new HintsStore(hostId, hintsDirectory, writerParams, 
descriptors);
 }
 
+@VisibleForTesting
+int getDispatchQueueSize()
+{
+return dispatchDequeue.size();
+}
+
 InetAddress address()
 {
 return StorageService.instance.getEndpointForHostId(hostId);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2f6ce96/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 

[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk

[aleksey]

Merge branch 'cassandra-3.11' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9fecf947
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9fecf947
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9fecf947

Branch: refs/heads/trunk
Commit: 9fecf947729e6113a79cd8069a52082b4c335629
Parents: 71a27ee 32f9e67
Author: Aleksey Yeshchenko 
Authored: Mon Apr 30 18:52:43 2018 +0100
Committer: Aleksey Yeshchenko 
Committed: Mon Apr 30 18:52:43 2018 +0100

--
 CHANGES.txt |   1 +
 .../apache/cassandra/hints/HintsCatalog.java|   7 ++
 .../apache/cassandra/hints/HintsService.java|   2 +-
 .../org/apache/cassandra/hints/HintsStore.java  |   7 ++
 .../cassandra/service/StorageService.java   |   7 +-
 .../cassandra/hints/HintsCatalogTest.java   | 119 ++-
 6 files changed, 137 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9fecf947/CHANGES.txt
--
diff --cc CHANGES.txt
index 15c2c2e,c392059..33c81d1
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -245,6 -10,9 +245,7 @@@
   * RateBasedBackPressure unnecessarily invokes a lock on the Guava 
RateLimiter (CASSANDRA-14163)
   * Fix wildcard GROUP BY queries (CASSANDRA-14209)
  Merged from 3.0:
+  * Delay hints store excise by write timeout to avoid race with decommission 
(CASSANDRA-13740)
 - * Deprecate background repair and probablistic read_repair_chance table 
options
 -   (CASSANDRA-13910)
   * Add missed CQL keywords to documentation (CASSANDRA-14359)
   * Fix unbounded validation compactions on repair / revert CASSANDRA-13797 
(CASSANDRA-14332)
   * Avoid deadlock when running nodetool refresh before node is fully up 
(CASSANDRA-14310)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9fecf947/src/java/org/apache/cassandra/hints/HintsService.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9fecf947/src/java/org/apache/cassandra/hints/HintsStore.java
--
diff --cc src/java/org/apache/cassandra/hints/HintsStore.java
index bbf57f5,032de5a..9f95884
--- a/src/java/org/apache/cassandra/hints/HintsStore.java
+++ b/src/java/org/apache/cassandra/hints/HintsStore.java
@@@ -77,7 -78,13 +78,13 @@@ final class HintsStor
  return new HintsStore(hostId, hintsDirectory, writerParams, 
descriptors);
  }
  
+ @VisibleForTesting
+ int getDispatchQueueSize()
+ {
+ return dispatchDequeue.size();
+ }
+ 
 -InetAddress address()
 +InetAddressAndPort address()
  {
  return StorageService.instance.getEndpointForHostId(hostId);
  }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9fecf947/src/java/org/apache/cassandra/service/StorageService.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9fecf947/test/unit/org/apache/cassandra/hints/HintsCatalogTest.java
--
diff --cc test/unit/org/apache/cassandra/hints/HintsCatalogTest.java
index a255338,68acd0c..92cfc71
--- a/test/unit/org/apache/cassandra/hints/HintsCatalogTest.java
+++ b/test/unit/org/apache/cassandra/hints/HintsCatalogTest.java
@@@ -23,6 -24,13 +24,12 @@@ import java.nio.file.Files
  import java.util.*;
  
  import com.google.common.collect.ImmutableMap;
+ import org.apache.cassandra.SchemaLoader;
 -import org.apache.cassandra.config.Schema;
+ import org.apache.cassandra.db.Mutation;
 -import org.apache.cassandra.db.RowUpdateBuilder;
+ import org.apache.cassandra.schema.KeyspaceParams;
++import org.apache.cassandra.schema.Schema;
+ import org.apache.cassandra.utils.FBUtilities;
+ import org.junit.BeforeClass;
  import org.junit.Test;
  
  import static junit.framework.Assert.*;
@@@ -117,4 -190,43 +189,43 @@@ public class HintsCatalogTes
  {
  }
  }
+ 
+ private static Mutation createMutation(String key, long now)
+ {
+ Mutation.SimpleBuilder builder = Mutation.simpleBuilder(KEYSPACE, 
dk(key));
+ 
 -builder.update(Schema.instance.getCFMetaData(KEYSPACE, TABLE0))
++builder.update(Schema.instance.getTableMetadata(KEYSPACE, TABLE0))
+.timestamp(now)
+.row("column0")
+.add("val", "value0");
+ 
 -builder.update(Schema.instance.getCFMetaData(KEYSPACE, TABLE1))
++builder.update(Schema.instance.getTableMetadata(KEYSPACE, TABLE1))
+.timestamp(now + 1)
+.row("column1")
+.add("val", 

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11

[aleksey]

Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/32f9e676
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/32f9e676
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/32f9e676

Branch: refs/heads/trunk
Commit: 32f9e6763b7a857a015da0b20af1d3ff486da0c9
Parents: 684120d b2f6ce9
Author: Aleksey Yeshchenko 
Authored: Mon Apr 30 18:48:08 2018 +0100
Committer: Aleksey Yeshchenko 
Committed: Mon Apr 30 18:48:08 2018 +0100

--
 CHANGES.txt |   1 +
 .../apache/cassandra/hints/HintsCatalog.java|   7 ++
 .../apache/cassandra/hints/HintsService.java|   2 +-
 .../org/apache/cassandra/hints/HintsStore.java  |   7 ++
 .../cassandra/service/StorageService.java   |   7 +-
 .../cassandra/hints/HintsCatalogTest.java   | 120 ++-
 6 files changed, 138 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/32f9e676/CHANGES.txt
--
diff --cc CHANGES.txt
index 5450322,857cf96..c392059
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,5 +1,16 @@@
 -3.0.17
 +3.11.3
 + * Allow existing nodes to use all peers in shadow round (CASSANDRA-13851)
 + * Fix cqlsh to read connection.ssl cqlshrc option again (CASSANDRA-14299)
 + * Downgrade log level to trace for CommitLogSegmentManager (CASSANDRA-14370)
 + * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
 + * Serialize empty buffer as empty string for json output format 
(CASSANDRA-14245)
 + * Allow logging implementation to be interchanged for embedded testing 
(CASSANDRA-13396)
 + * SASI tokenizer for simple delimiter based entries (CASSANDRA-14247)
 + * Fix Loss of digits when doing CAST from varint/bigint to decimal 
(CASSANDRA-14170)
 + * RateBasedBackPressure unnecessarily invokes a lock on the Guava 
RateLimiter (CASSANDRA-14163)
 + * Fix wildcard GROUP BY queries (CASSANDRA-14209)
 +Merged from 3.0:
+  * Delay hints store excise by write timeout to avoid race with decommission 
(CASSANDRA-13740)
   * Deprecate background repair and probablistic read_repair_chance table 
options
 (CASSANDRA-13910)
   * Add missed CQL keywords to documentation (CASSANDRA-14359)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32f9e676/src/java/org/apache/cassandra/hints/HintsService.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32f9e676/src/java/org/apache/cassandra/hints/HintsStore.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32f9e676/src/java/org/apache/cassandra/service/StorageService.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32f9e676/test/unit/org/apache/cassandra/hints/HintsCatalogTest.java
--
diff --cc test/unit/org/apache/cassandra/hints/HintsCatalogTest.java
index a255338,928fd31..68acd0c
--- a/test/unit/org/apache/cassandra/hints/HintsCatalogTest.java
+++ b/test/unit/org/apache/cassandra/hints/HintsCatalogTest.java
@@@ -43,7 -69,21 +69,7 @@@ public class HintsCatalogTes
  }
  }
  
- public static void loadCompletenessAndOrderTest(File directory) throws 
IOException
 -@Test
 -public void exciseHintFiles() throws IOException
 -{
 -File directory = Files.createTempDirectory(null).toFile();
 -try
 -{
 -exciseHintFiles(directory);
 -}
 -finally
 -{
 -directory.deleteOnExit();
 -}
 -}
 -
+ private void loadCompletenessAndOrderTest(File directory) throws 
IOException
  {
  UUID hostId1 = UUID.randomUUID();
  UUID hostId2 = UUID.randomUUID();
@@@ -79,37 -119,39 +105,84 @@@
  assertNull(store2.poll());
  }
  
 +@Test
 +public void deleteHintsTest() throws IOException
 +{
 +File directory = Files.createTempDirectory(null).toFile();
 +UUID hostId1 = UUID.randomUUID();
 +UUID hostId2 = UUID.randomUUID();
 +long now = System.currentTimeMillis();
 +writeDescriptor(directory, new HintsDescriptor(hostId1, now));
- writeDescriptor(directory, new HintsDescriptor(hostId1, now+1));
- writeDescriptor(directory, new HintsDescriptor(hostId2, now+2));
- writeDescriptor(directory, new HintsDescriptor(hostId2, now+3));
++writeDescriptor(directory, new HintsDescriptor(hostId1, now + 1));
++writeDescriptor(directory, new HintsDescriptor(hostId2, now + 

[jira] [Commented] (CASSANDRA-14424) Gossip EchoMessages not being handled somewhere after node restart

[Joseph Lynch (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458787#comment-16458787
 ] 

Joseph Lynch commented on CASSANDRA-14424:
--

[~alienth] yea this looks really similar (in particular the 10-20 minute delay 
looks a lot like Linux's default tcp_retries2 parameter). Note that if this is 
14358 you won't get retransmits on the restarted node, but the old node that 
has a bad {{OutboundTCPConnection}}. The restarted node is 100% fine, it's the 
old node that is endlessly transmitting on a blackholing TCP connection.

By any chance are you using VPC or security groups (sg in particular)? The core 
finding of CASSANDRA-14358 imo is that those network products can swallow 
packets on an {{ESTABLISHED}} TCP connection (rather than the more useful 
{{RST}}). If you are able to get a reproduction, you can see if this is 
happening by running the following on the _old_ host (the one that the 
restarted node can't see, not the restarted node):
{noformat}
sudo netstat -o -n -p  | grep  | grep 
{noformat}
If it is the same issue as CASSANDRA-14358, you should see {{ESTABLISHED}} 
connections from an ephemeral port on the old node to the restarted node with 
non zero retransmission counters (e.g. keepalive (X/Y/Z) where Y > 0). On the 
restarted node you should not see that the ephemeral port from the 
non-restarted node in an {{ESTABLISHED}} state.

I know that reproducing this issue is tricky. If you can't reproduce it to 
check the connection state, you could try testing the proposed mitigation in 
CASSANDRA-14358 by setting the sysctl on the entire cluster lower and then 
restarting nodes:
{noformat}
sudo sysctl net.ipv4.tcp_retries2=5
{noformat}
If it is 14358, then your partitions should reduce to ~2 minutes rather than 
15-20.

> Gossip EchoMessages not being handled somewhere after node restart
> --
>
> Key: CASSANDRA-14424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14424
> Project: Cassandra
>  Issue Type: Bug
> Environment: cassandra 3.11.2 - brand new ring - 18 nodes.
> ubuntu 16.04
> AWS - cross AZ, with GossipingPropertyFileSnitch setting the rack to the AZs.
>Reporter: Jason Harvey
>Priority: Major
> Fix For: 3.11.x, 4.x
>
>
> Noticing this behaviour on a brand new 3.11.2 ring:
>  # Restart a random node in the ring.
>  # When that node comes back up, around 30% of the time it sees a single 
> other node down. No other node in the ring sees that node is down.
>  # After 10-20 minutes, the DOWN node suddenly appears UP to the restarted 
> node.
>  
> After digging through tracing logs, here's what I know:
>  
> The node seen as DOWN has not gone down, but simply hasn't been seen as UP 
> yet. The restarted node is attempting to `markAlive()` the target node. 
> Relevant logs from the restarted node's POV:
>  
> {{INFO [GossipStage:1] 2018-04-27 14:03:50,950 Gossiper.java:1053 - Node 
> /10.0.225.147 has restarted, now UP}}
>  {{INFO [GossipStage:1] 2018-04-27 14:03:50,969 StorageService.java:2292 - 
> Node /10.0.225.147 state jump to NORMAL}}
>  {{INFO [HANDSHAKE-/10.0.225.147] 2018-04-27 14:03:50,976 
> OutboundTcpConnection.java:560 - Handshaking version with /10.0.225.147}}
>  {{INFO [GossipStage:1] 2018-04-27 14:03:50,977 TokenMetadata.java:479 - 
> Updating topology for /10.0.225.147}}
>  {{INFO [GossipStage:1] 2018-04-27 14:03:50,977 TokenMetadata.java:479 - 
> Updating topology for /10.0.225.147}}
>  
> (note that despite the Gossip seeing the DOWN node as 'UP', nodetool status 
> still shows it as 'DOWN', as markAlive has not completed, and will not 
> actually be seen as 'UP' for 20 more minutes)
>  
> The restarted node is repeatedly sending Echo messages to the DOWN node as 
> part of the `markAlive()` call. The DOWN node is receiving those, and claims 
> to be sending a response. However, the restarted node is not marking the DOWN 
> node as UP even after the DOWN node sends the Echo response.
>  
> Relevant logs from the restarted node's POV:
>  
> {{TRACE [GossipStage:1] 2018-04-27 14:11:28,792 MessagingService.java:945 - 
> 10.0.103.45 sending ECHO to 99248@/10.0.225.147}}
> {{TRACE [GossipTasks:1] 2018-04-27 14:11:29,792 MessagingService.java:945 - 
> 10.0.103.45 sending GOSSIP_DIGEST_SYN to 99631@/10.0.225.147}}
> {{TRACE [GossipStage:1] 2018-04-27 14:11:29,792 MessagingService.java:945 - 
> 10.0.103.45 sending ECHO to 99632@/10.0.225.147}}
> {{TRACE [GossipStage:1] 2018-04-27 14:11:29,793 MessagingService.java:945 - 
> 10.0.103.45 sending GOSSIP_DIGEST_ACK2 to 99633@/10.0.225.147}}
> {{TRACE [GossipStage:1] 2018-04-27 14:11:29,793 MessagingService.java:945 - 
> 10.0.103.45 sending ECHO to 99635@/10.0.225.147}}
> {{TRACE [GossipStage:1] 2018-04-27 14:11:31,794 MessagingService.java:945 - 
> 

[jira] [Commented] (CASSANDRA-14197) SSTable upgrade should be automatic

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458762#comment-16458762
 ] 

Ariel Weisberg commented on CASSANDRA-14197:


+1 LGTM

> SSTable upgrade should be automatic
> ---
>
> Key: CASSANDRA-14197
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14197
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 4.x
>
>
> Upgradesstables should run automatically on node upgrade



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14428) Run ant eclipse-warnings in circleci

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458722#comment-16458722
 ] 

Ariel Weisberg commented on CASSANDRA-14428:


+1 to your fixes of my code.

Can we run eclipse warnings inside the unit test job so we don't have to spin 
up the environment and build Cassandra again? That would save a bit less than 
two minutes. You could have it bail out if it's not container index 0 or just 
have all of them run it.

> Run ant eclipse-warnings in circleci
> 
>
> Key: CASSANDRA-14428
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14428
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 4.x
>
>
> We should run ant eclipse-warnings in circle-ci



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14379) Better handling of missing partition columns in system_schema.columns during startup

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-14379:
--
Status: Ready to Commit  (was: Patch Available)

> Better handling of missing partition columns in system_schema.columns during 
> startup
> 
>
> Key: CASSANDRA-14379
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14379
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Distributed Metadata
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Major
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Follow up for CASSANDRA-13180, during table deletion/creation, we saw one 
> table having partially deleted columns (no partition column, only regular 
> column). It's blocking node from startup:
> {noformat}
> java.lang.AssertionError: null
> at 
> org.apache.cassandra.db.marshal.CompositeType.getInstance(CompositeType.java:103)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.config.CFMetaData.rebuild(CFMetaData.java:308) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:288) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.CFMetaData.create(CFMetaData.java:363) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:1028)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:987)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:945)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:922)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:910)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:138) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:128) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) 
> [apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:567)
>  [apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:695) 
> [apache-cassandra-3.0.14.x.jar:3.0.14.x]
> {noformat}
> As partition column is mandatory, it should throw 
> [{{MissingColumns}}|https://github.com/apache/cassandra/blob/60563f4e8910fb59af141fd24f1fc1f98f34f705/src/java/org/apache/cassandra/schema/SchemaKeyspace.java#L1351],
>  the same as CASSANDRA-13180, so the user is able to cleanup the schema.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14379) Better handling of missing partition columns in system_schema.columns during startup

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-14379:
--
Fix Version/s: 4.x
   3.11.x
   3.0.x

> Better handling of missing partition columns in system_schema.columns during 
> startup
> 
>
> Key: CASSANDRA-14379
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14379
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Distributed Metadata
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Major
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> Follow up for CASSANDRA-13180, during table deletion/creation, we saw one 
> table having partially deleted columns (no partition column, only regular 
> column). It's blocking node from startup:
> {noformat}
> java.lang.AssertionError: null
> at 
> org.apache.cassandra.db.marshal.CompositeType.getInstance(CompositeType.java:103)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.config.CFMetaData.rebuild(CFMetaData.java:308) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:288) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.CFMetaData.create(CFMetaData.java:363) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:1028)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:987)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:945)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:922)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:910)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:138) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:128) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) 
> [apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:567)
>  [apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:695) 
> [apache-cassandra-3.0.14.x.jar:3.0.14.x]
> {noformat}
> As partition column is mandatory, it should throw 
> [{{MissingColumns}}|https://github.com/apache/cassandra/blob/60563f4e8910fb59af141fd24f1fc1f98f34f705/src/java/org/apache/cassandra/schema/SchemaKeyspace.java#L1351],
>  the same as CASSANDRA-13180, so the user is able to cleanup the schema.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14379) Better handling of missing partition columns in system_schema.columns during startup

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458657#comment-16458657
 ] 

Aleksey Yeschenko commented on CASSANDRA-14379:
---

The patch is basically ok, so feel free to commit, +1.

I would do it a tiny bit differently myself:
- instead of replacing "No columns found for table" with "No column or 
partition column found for table", I would replace it with "No partition 
columns found for table" - with correct tense and without redundance, as 
"partition columns" is essentially a subset of "columns"
- replace your changes to {{fetchColumns()}} with
{code}
private static List fetchColumns(String keyspace, String 
table, Types types)
{
String query = format("SELECT * FROM %s.%s WHERE keyspace_name = ? AND 
table_name = ?", NAME, COLUMNS);
UntypedResultSet columnRows = query(query, keyspace, table);

List columns = new ArrayList<>();
columnRows.forEach(row -> columns.add(createColumnFromRow(row, types)));

if (columns.stream().noneMatch(ColumnDefinition::isPartitionKey))
throw new MissingColumns("No partition key columns found in schema 
table for " + keyspace + "." + table);

return columns;
}
{code}

But, again, this is a small patch and is essentially ok enough, so it's up to 
you how you want to proceed - I'm +1 either way.

> Better handling of missing partition columns in system_schema.columns during 
> startup
> 
>
> Key: CASSANDRA-14379
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14379
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Distributed Metadata
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Major
>
> Follow up for CASSANDRA-13180, during table deletion/creation, we saw one 
> table having partially deleted columns (no partition column, only regular 
> column). It's blocking node from startup:
> {noformat}
> java.lang.AssertionError: null
> at 
> org.apache.cassandra.db.marshal.CompositeType.getInstance(CompositeType.java:103)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.config.CFMetaData.rebuild(CFMetaData.java:308) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:288) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.CFMetaData.create(CFMetaData.java:363) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:1028)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:987)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:945)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:922)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:910)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:138) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:128) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) 
> [apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:567)
>  [apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:695) 
> [apache-cassandra-3.0.14.x.jar:3.0.14.x]
> {noformat}
> As partition column is mandatory, it should throw 
> [{{MissingColumns}}|https://github.com/apache/cassandra/blob/60563f4e8910fb59af141fd24f1fc1f98f34f705/src/java/org/apache/cassandra/schema/SchemaKeyspace.java#L1351],
>  the same as CASSANDRA-13180, so the user is able to cleanup the schema.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12464) Investigate the potential improvement of parallelism on higher level compactions in LCS

[Iuri Sitinschi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458637#comment-16458637
 ] 

Iuri Sitinschi commented on CASSANDRA-12464:


Link in the description is broken. It should be 
https://github.com/google/leveldb/blob/master/doc/impl.md#compactions

> Investigate the potential improvement of parallelism on higher level 
> compactions in LCS
> ---
>
> Key: CASSANDRA-12464
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12464
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Compaction
>Reporter: Wei Deng
>Priority: Major
>  Labels: lcs, lhf, performance
>
> According to LevelDB's design doc 
> [here|https://github.com/google/leveldb/blob/master/doc/impl.html#L115-L116], 
> "A compaction merges the contents of the picked files to produce a sequence 
> of level-(L+1) files", it will "switch to producing a new level-(L+1) file 
> after the current output file has reached the target file size" (in our case 
> 160MB), it will also "switch to a new output file when the key range of the 
> current output file has grown enough to overlap more than ten level-(L+2) 
> files". This is to ensure "that a later compaction of a level-(L+1) file will 
> not pick up too much data from level-(L+2)."
> Our current code in LeveledCompactionStrategy doesn't implement this last 
> rule, but we might be able to quickly implement it and see how much a 
> compaction throughput improvement it can deliver. Potentially we can create a 
> scenario where a number of large L0 SSTables are present (e.g. 200GB after 
> switching from STCS) and let it to create thousands of L1 SSTables overflow, 
> and see how fast LCS can digest this much data from L1 and properly 
> upper-level them to completion.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14428) Run ant eclipse-warnings in circleci

[Marcus Eriksson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-14428:

Fix Version/s: 4.x
   Status: Patch Available  (was: Open)

to get the unit tests passing on 
[builds.apache.org|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-test-all/]
 again we need to fix the eclipse-warnings, and to make sure we don't start 
failing again we should run them in circleci
 
[https://github.com/krummas/cassandra/commits/marcuse/eclipse-warnings]
[https://circleci.com/gh/krummas/cassandra/tree/marcuse%2Feclipse-warnings]

[~aweisberg]: could you have a look at the fixes in {{MessageIn}} and 
{{MessageInHandler}} from CASSANDRA-7544? and [~bdeggleston] could you look at 
the rest? Looks like they are just left-out {{@SuppressWarnings("resource")}} 
from CASSANDRA-14115 and CASSANDRA-14118

> Run ant eclipse-warnings in circleci
> 
>
> Key: CASSANDRA-14428
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14428
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 4.x
>
>
> We should run ant eclipse-warnings in circle-ci



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-14428) Run ant eclipse-warnings in circleci

[Marcus Eriksson (JIRA)]

Marcus Eriksson created CASSANDRA-14428:
---

 Summary: Run ant eclipse-warnings in circleci
 Key: CASSANDRA-14428
 URL: https://issues.apache.org/jira/browse/CASSANDRA-14428
 Project: Cassandra
  Issue Type: Improvement
Reporter: Marcus Eriksson
Assignee: Marcus Eriksson


We should run ant eclipse-warnings in circle-ci



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13889) cfstats should take sorting and limit parameters

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458573#comment-16458573
 ] 

Marcus Eriksson commented on CASSANDRA-13889:
-

[~ptbannister] the tests for this seem to always fail in circleci for me: 
https://circleci.com/gh/krummas/cassandra/400#tests/containers/1 - could you 
have a look?

> cfstats should take sorting and limit parameters
> 
>
> Key: CASSANDRA-13889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13889
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Jon Haddad
>Assignee: Patrick Bannister
>Priority: Major
> Fix For: 4.0
>
> Attachments: 13889-trunk.txt, sample_output_normal.txt, 
> sample_output_sorted.txt, sample_output_sorted_top3.txt
>
>
> When looking at a problematic node I'm not familiar with, one of the first 
> things I do is check cfstats to identify the tables with the most reads, 
> writes, and data.  This is fine as long as there aren't a lot of tables but 
> once it goes above a dozen it's quite difficult.  cfstats should allow me to 
> sort the results and limit to top K tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14387) SSTableReaderTest#testOpeningSSTable fails on macOS

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458572#comment-16458572
 ] 

Aleksey Yeschenko commented on CASSANDRA-14387:
---

Can you please commit to 3.0 and 3.11 too? Because tests are failing on Jenkins 
:\ Cheers.

> SSTableReaderTest#testOpeningSSTable fails on macOS
> ---
>
> Key: CASSANDRA-14387
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14387
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Minor
> Fix For: 4.0
>
>
> I ran into an issue with {{SSTableReaderTest#testOpeningSSTable}} test 
> failure on macOS. The reason for failure seems that on macOS, the file 
> modification timestamps are at a second granularity (See: 
> https://stackoverflow.com/questions/18403588/how-to-return-millisecond-information-for-file-access-on-mac-os-x-in-java
>  and 
> https://developer.apple.com/legacy/library/technotes/tn/tn1150.html#HFSPlusDates).
>  The fix is simple - bumping up the sleep time to 1 second instead of 10ms.
> {noformat}
> [junit] -  ---
> [junit] Testcase: 
> testOpeningSSTable(org.apache.cassandra.io.sstable.SSTableReaderTest):  FAILED
> [junit] Bloomfilter was not recreated
> [junit] junit.framework.AssertionFailedError: Bloomfilter was not 
> recreated
> [junit]   at 
> org.apache.cassandra.io.sstable.SSTableReaderTest.testOpeningSSTable(SSTableReaderTest.java:421)
> [junit]
> [junit]
> [junit] Test org.apache.cassandra.io.sstable.SSTableReaderTest FAILED
> {noformat}
> Related issue: CASSANDRA-11163



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-12151) Audit logging for database activity

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458567#comment-16458567
 ] 

Jason Brown edited comment on CASSANDRA-12151 at 4/30/18 1:29 PM:
--

After rereading this patch, I've merged the FQL and AuditLog code paths as they 
do the same thing from the outside: record user-initiated events. To that end, 
I've moved the management of {{FullQueryLogger}} under {{AuditLogManager}}, 
thus making {{AuditLogManager}} the hub for logging these actions. Further, 
instead of {{BinAuditLogger}} inherit from {{FullQueryLogger}} (which never 
felt quite right), I've pulled the shared componentry into a base class 
({{BinLogAuditLogger}}), and {{BinAuditLogger}} / {{FullQueryLogger}} inherit 
from that.

I refactored {{AuditLogEntry}} to use the builder pattern for construction as 
all the {{static #getLogEntry()}} with the {{set*()}} methods was rather 
confusing. Further, we're using the builder pattern elsewhere in the code. 
{{#toString()}} is typically used for debugging, so I'd prefer a different 
method name - I used {{#getLogString()}}. If you want the debug dump to have 
the same output, you can have {{toString()}} call that renamed function.

We need an extra filter in {{AuditLogManager#log(AuditLogEntry)}} to make sure 
we don't send unintended messages to FQL. Can you figure out what that is? I'm 
not sure if we can simply use the entry's {{AuditLogEntryType}} or the type's 
category.

Why did you remove the reloadFilters functionality? 

I feel like the {{category}} in AuditLogEntryType should be an enum, as well. 
wdyt?

I haven't looked at the circleci results yet for utests/dtests, but let's make 
sure this is right path forward first.

UPDATE: branch (squashed as single patch) is 
[here|https://github.com/jasobrown/cassandra/tree/trunk_CASSANDRA-12151]


was (Author: jasobrown):
After rereading this patch, I've merged the FQL and AuditLog code paths as they 
do the same thing from the outside: record user-initiated events. To that end, 
I've moved the management of {{FullQueryLogger}} under {{AuditLogManager}}, 
thus making {{AuditLogManager}} the hub for logging these actions. Further, 
instead of {{BinAuditLogger}} inherit from {{FullQueryLogger}} (which never 
felt quite right), I've pulled the shared componentry into a base class 
({{BinLogAuditLogger}}), and {{BinAuditLogger}} / {{FullQueryLogger}} inherit 
from that.

I refactored {{AuditLogEntry}} to use the builder pattern for construction as 
all the {{static #getLogEntry()}} with the {{set*()}} methods was rather 
confusing. Further, we're using the builder pattern elsewhere in the code. 
{{#toString()}} is typically used for debugging, so I'd prefer a different 
method name - I used {{#getLogString()}}. If you want the debug dump to have 
the same output, you can have {{toString()}} call that renamed function.

We need an extra filter in {{AuditLogManager#log(AuditLogEntry)}} to make sure 
we don't send unintended messages to FQL. Can you figure out what that is? I'm 
not sure if we can simply use the entry's {{AuditLogEntryType}} or the type's 
category.

Why did you remove the reloadFilters functionality? 

I feel like the {{category}} in AuditLogEntryType should be an enum, as well. 
wdyt?

I haven't looked at the circleci results yet for utests/dtests, but let's make 
sure this is right path forward first.


> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12151) Audit logging for database activity

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458567#comment-16458567
 ] 

Jason Brown commented on CASSANDRA-12151:
-

After rereading this patch, I've merged the FQL and AuditLog code paths as they 
do the same thing from the outside: record user-initiated events. To that end, 
I've moved the management of {{FullQueryLogger}} under {{AuditLogManager}}, 
thus making {{AuditLogManager}} the hub for logging these actions. Further, 
instead of {{BinAuditLogger}} inherit from {{FullQueryLogger}} (which never 
felt quite right), I've pulled the shared componentry into a base class 
({{BinLogAuditLogger}}), and {{BinAuditLogger}} / {{FullQueryLogger}} inherit 
from that.

I refactored {{AuditLogEntry}} to use the builder pattern for construction as 
all the {{static #getLogEntry()}} with the {{set*()}} methods was rather 
confusing. Further, we're using the builder pattern elsewhere in the code. 
{{#toString()}} is typically used for debugging, so I'd prefer a different 
method name - I used {{#getLogString()}}. If you want the debug dump to have 
the same output, you can have {{toString()}} call that renamed function.

We need an extra filter in {{AuditLogManager#log(AuditLogEntry)}} to make sure 
we don't send unintended messages to FQL. Can you figure out what that is? I'm 
not sure if we can simply use the entry's {{AuditLogEntryType}} or the type's 
category.

Why did you remove the reloadFilters functionality? 

I feel like the {{category}} in AuditLogEntryType should be an enum, as well. 
wdyt?

I haven't looked at the circleci results yet for utests/dtests, but let's make 
sure this is right path forward first.


> Audit logging for database activity
> ---
>
> Key: CASSANDRA-12151
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12151
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: stefan setyadi
>Assignee: Vinay Chella
>Priority: Major
> Fix For: 4.x
>
> Attachments: 12151.txt, CASSANDRA_12151-benchmark.html, 
> DesignProposal_AuditingFeature_ApacheCassandra_v1.docx
>
>
> we would like a way to enable cassandra to log database activity being done 
> on our server.
> It should show username, remote address, timestamp, action type, keyspace, 
> column family, and the query statement.
> it should also be able to log connection attempt and changes to the 
> user/roles.
> I was thinking of making a new keyspace and insert an entry for every 
> activity that occurs.
> Then It would be possible to query for specific activity or a query targeting 
> a specific keyspace and column family.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-11163) Summaries are needlessly rebuilt when the BF FP ratio is changed

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458561#comment-16458561
 ] 

Aleksey Yeschenko commented on CASSANDRA-11163:
---

Yeah. As you (and Marcus, offline) mentioned, CASSANDRA-14387 was only 
committed to trunk, so 3.0 tests are still failing elsewhere (incl. Jenkins, 
but not Circle). We just need to commit 14387 to 3.0 and 3.11.

> Summaries are needlessly rebuilt when the BF FP ratio is changed
> 
>
> Key: CASSANDRA-11163
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11163
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Brandon Williams
>Assignee: Kurt Greaves
>Priority: Major
> Fix For: 4.0, 3.0.17, 3.11.3
>
>
> This is from trunk, but I also saw this happen on 2.0:
> Before:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 221460
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-6-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-7-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root104178 Feb 11 23:50 ma-5-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> root@bw-1:/srv/cassandra# md5sum 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ma-5-big-Summary.db
> 5fca154fc790f7cfa37e8ad6d1c7552c
> {noformat}
> BF ratio changed, node restarted:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 242168
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 12 00:03 ma-8-big-TOC.txt
> -rw-r--r-- 1 root root 14902 Feb 12 00:03 ma-8-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 12 00:03 ma-8-big-Statistics.db
> -rw-r--r-- 1 root root   1458631 Feb 12 00:03 ma-8-big-Index.db
> -rw-r--r-- 1 root root 10808 Feb 12 00:03 

[jira] [Commented] (CASSANDRA-11163) Summaries are needlessly rebuilt when the BF FP ratio is changed

[Chris Lohfink (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458556#comment-16458556
 ] 

Chris Lohfink commented on CASSANDRA-11163:
---

thought this was fixed in https://issues.apache.org/jira/browse/CASSANDRA-14387 
? time resolution being different on causing it to fail on some systems.

> Summaries are needlessly rebuilt when the BF FP ratio is changed
> 
>
> Key: CASSANDRA-11163
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11163
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Brandon Williams
>Assignee: Kurt Greaves
>Priority: Major
> Fix For: 4.0, 3.0.17, 3.11.3
>
>
> This is from trunk, but I also saw this happen on 2.0:
> Before:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 221460
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-6-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-7-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root104178 Feb 11 23:50 ma-5-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> root@bw-1:/srv/cassandra# md5sum 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ma-5-big-Summary.db
> 5fca154fc790f7cfa37e8ad6d1c7552c
> {noformat}
> BF ratio changed, node restarted:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 242168
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 12 00:03 ma-8-big-TOC.txt
> -rw-r--r-- 1 root root 14902 Feb 12 00:03 ma-8-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 12 00:03 ma-8-big-Statistics.db
> -rw-r--r-- 1 root root   1458631 Feb 12 00:03 ma-8-big-Index.db
> -rw-r--r-- 1 root root 10808 Feb 12 00:03 ma-8-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 12 00:03 

[jira] [Commented] (CASSANDRA-11163) Summaries are needlessly rebuilt when the BF FP ratio is changed

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458549#comment-16458549
 ] 

Aleksey Yeschenko commented on CASSANDRA-11163:
---

This commit broke 3.0 unit tests (maybe 3.11 too, haven't checked).

See 
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-3.0-test-all/lastCompletedBuild/testReport/org.apache.cassandra.io.sstable/SSTableReaderTest/testOpeningSSTable/history/
 and 
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-3.0-test-all/248/changes.
 It's also failing locally and I'm pretty sure will be failing 
[here|https://circleci.com/gh/iamaleksey/cassandra/234] too.

We should be more careful about committing things. But in the meantime, 
[~michaelsembwever] can you please fix this or revert this until there is a 
fix? Cheers.

> Summaries are needlessly rebuilt when the BF FP ratio is changed
> 
>
> Key: CASSANDRA-11163
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11163
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Brandon Williams
>Assignee: Kurt Greaves
>Priority: Major
> Fix For: 4.0, 3.0.17, 3.11.3
>
>
> This is from trunk, but I also saw this happen on 2.0:
> Before:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 221460
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-6-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-7-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root104178 Feb 11 23:50 ma-5-big-Summary.db
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root  8508 Feb 11 23:50 ma-5-big-CRC.db
> root@bw-1:/srv/cassandra# md5sum 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ma-5-big-Summary.db
> 5fca154fc790f7cfa37e8ad6d1c7552c
> {noformat}
> BF ratio changed, node restarted:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr 
> /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 242168
> drwxr-xr-x 2 root root  4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root  2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root  2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 

[jira] [Updated] (CASSANDRA-13998) Cassandra stress distribution does not affect the result

[Venkata Harikrishna Nukala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Venkata Harikrishna Nukala updated CASSANDRA-13998:
---
Attachment: 13998-trunk.txt

> Cassandra stress distribution does not affect the result
> 
>
> Key: CASSANDRA-13998
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13998
> Project: Cassandra
>  Issue Type: Task
>  Components: Stress
> Environment: Widnows 10
>Reporter: Mikhail Pliskovsky
>Assignee: Venkata Harikrishna Nukala
>Priority: Trivial
> Fix For: 3.11.x
>
> Attachments: 13998-trunk.txt, cqlstress-example.yaml
>
>
> When testing my schema on single-node cluster, I am getting the identical 
> data for each stress-test run
> I specify my cassandra-stress.yaml file 
> Table and column spec
> {code:java}
> table_definition: |
>   CREATE TABLE files (
> id uuid PRIMARY KEY,
> data blob
>   ) 
> columnspec:
>   - name: data
> size: UNIFORM(10..100)
> population: UNIFORM(1..100B)
> {code}
> But when query table rows after test, I am getting data as identical string 
> in each row
> Command to run the test
> {code:java}
> cassandra-stress user profile=..\cqlstress-example.yaml n=20 ops(insert=5) 
> -rate threads=8
> {code}
> What I am doing wrong? 
> My wish is to have the data of variable length



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13998) Cassandra stress distribution does not affect the result

[Venkata Harikrishna Nukala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Venkata Harikrishna Nukala updated CASSANDRA-13998:
---
Reproduced In: 4.0
   Status: Patch Available  (was: Open)

> Cassandra stress distribution does not affect the result
> 
>
> Key: CASSANDRA-13998
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13998
> Project: Cassandra
>  Issue Type: Task
>  Components: Stress
> Environment: Widnows 10
>Reporter: Mikhail Pliskovsky
>Assignee: Venkata Harikrishna Nukala
>Priority: Trivial
> Fix For: 3.11.x
>
> Attachments: 13998-trunk.txt, cqlstress-example.yaml
>
>
> When testing my schema on single-node cluster, I am getting the identical 
> data for each stress-test run
> I specify my cassandra-stress.yaml file 
> Table and column spec
> {code:java}
> table_definition: |
>   CREATE TABLE files (
> id uuid PRIMARY KEY,
> data blob
>   ) 
> columnspec:
>   - name: data
> size: UNIFORM(10..100)
> population: UNIFORM(1..100B)
> {code}
> But when query table rows after test, I am getting data as identical string 
> in each row
> Command to run the test
> {code:java}
> cassandra-stress user profile=..\cqlstress-example.yaml n=20 ops(insert=5) 
> -rate threads=8
> {code}
> What I am doing wrong? 
> My wish is to have the data of variable length



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13998) Cassandra stress distribution does not affect the result

[Venkata Harikrishna Nukala (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458489#comment-16458489
 ] 

Venkata Harikrishna Nukala commented on CASSANDRA-13998:


This is the issue with UUID generation. UUID is generated with same MSB (most 
significant bits) and LSB (least significant bits). Because of this, the XOR of 
MSB and LSB is always zero. _PartitionIterator.seed(Object object, AbstractType 
type, long seed)_ is using the _MSB ^ LSB_ to generate the idSeed which is 
always zero and the initial value of idSeed is zero too. This idSeed is used as 
the seed for value columns too. Since the seed is same (zero) all the time, 
same values with the same size are being generated. So fixed UUID generation 
using FasterRandom.

After this change, I can see data generated with different size & values. 
Tested with a table having clustering columns and different types of primary 
keys.

Attaching the patch to this ticket.

> Cassandra stress distribution does not affect the result
> 
>
> Key: CASSANDRA-13998
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13998
> Project: Cassandra
>  Issue Type: Task
>  Components: Stress
> Environment: Widnows 10
>Reporter: Mikhail Pliskovsky
>Assignee: Venkata Harikrishna Nukala
>Priority: Trivial
> Fix For: 3.11.x
>
> Attachments: cqlstress-example.yaml
>
>
> When testing my schema on single-node cluster, I am getting the identical 
> data for each stress-test run
> I specify my cassandra-stress.yaml file 
> Table and column spec
> {code:java}
> table_definition: |
>   CREATE TABLE files (
> id uuid PRIMARY KEY,
> data blob
>   ) 
> columnspec:
>   - name: data
> size: UNIFORM(10..100)
> population: UNIFORM(1..100B)
> {code}
> But when query table rows after test, I am getting data as identical string 
> in each row
> Command to run the test
> {code:java}
> cassandra-stress user profile=..\cqlstress-example.yaml n=20 ops(insert=5) 
> -rate threads=8
> {code}
> What I am doing wrong? 
> My wish is to have the data of variable length



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14379) Better handling of missing partition columns in system_schema.columns during startup

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-14379:
--
Reviewer: Aleksey Yeschenko

> Better handling of missing partition columns in system_schema.columns during 
> startup
> 
>
> Key: CASSANDRA-14379
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14379
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Distributed Metadata
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Major
>
> Follow up for CASSANDRA-13180, during table deletion/creation, we saw one 
> table having partially deleted columns (no partition column, only regular 
> column). It's blocking node from startup:
> {noformat}
> java.lang.AssertionError: null
> at 
> org.apache.cassandra.db.marshal.CompositeType.getInstance(CompositeType.java:103)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.config.CFMetaData.rebuild(CFMetaData.java:308) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.CFMetaData.(CFMetaData.java:288) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.CFMetaData.create(CFMetaData.java:363) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTable(SchemaKeyspace.java:1028)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchTables(SchemaKeyspace.java:987)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspace(SchemaKeyspace.java:945)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchKeyspacesWithout(SchemaKeyspace.java:922)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.schema.SchemaKeyspace.fetchNonSystemKeyspaces(SchemaKeyspace.java:910)
>  ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:138) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:128) 
> ~[apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:241) 
> [apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:567)
>  [apache-cassandra-3.0.14.x.jar:3.0.14.x]
> at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:695) 
> [apache-cassandra-3.0.14.x.jar:3.0.14.x]
> {noformat}
> As partition column is mandatory, it should throw 
> [{{MissingColumns}}|https://github.com/apache/cassandra/blob/60563f4e8910fb59af141fd24f1fc1f98f34f705/src/java/org/apache/cassandra/schema/SchemaKeyspace.java#L1351],
>  the same as CASSANDRA-13180, so the user is able to cleanup the schema.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14197) SSTable upgrade should be automatic

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458407#comment-16458407
 ] 

Marcus Eriksson edited comment on CASSANDRA-14197 at 4/30/18 9:07 AM:
--

bq. Doesn't this mean that only one compaction task can be running at a time?
yeah that was a leftover, pushed the correct version

bq. Worth logging the sstable filename here?
It is logged in {{CompactionStrategyManager#findUpgradeTask}}, removed that log 
line instead

also added tests to make sure we don't exceed the limit
tests running here:
https://circleci.com/gh/krummas/cassandra/tree/marcuse%2Fautoupgrade-v2

bq. Is there a max value for max_concurrent_automatic_sstable_upgrades to 
either error on or clamp it to?
added a warn log line if we exceed {{concurrent_compactors}}


was (Author: krummas):
bq. Doesn't this mean that only one compaction task can be running at a time?
yeah that was a leftover, pushed the correct version

bq. Worth logging the sstable filename here?
It is logged in {{CompactionStrategyManager#findUpgradeTask}}, removed that log 
line instead

also added tests to make sure we don't exceed the limit
tests running here:
https://circleci.com/gh/krummas/cassandra/tree/marcuse%2Fautoupgrade-v2

> SSTable upgrade should be automatic
> ---
>
> Key: CASSANDRA-14197
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14197
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 4.x
>
>
> Upgradesstables should run automatically on node upgrade



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14197) SSTable upgrade should be automatic

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458407#comment-16458407
 ] 

Marcus Eriksson commented on CASSANDRA-14197:
-

bq. Doesn't this mean that only one compaction task can be running at a time?
yeah that was a leftover, pushed the correct version

bq. Worth logging the sstable filename here?
It is logged in {{CompactionStrategyManager#findUpgradeTask}}, removed that log 
line instead

also added tests to make sure we don't exceed the limit

> SSTable upgrade should be automatic
> ---
>
> Key: CASSANDRA-14197
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14197
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 4.x
>
>
> Upgradesstables should run automatically on node upgrade



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14197) SSTable upgrade should be automatic

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458407#comment-16458407
 ] 

Marcus Eriksson edited comment on CASSANDRA-14197 at 4/30/18 8:55 AM:
--

bq. Doesn't this mean that only one compaction task can be running at a time?
yeah that was a leftover, pushed the correct version

bq. Worth logging the sstable filename here?
It is logged in {{CompactionStrategyManager#findUpgradeTask}}, removed that log 
line instead

also added tests to make sure we don't exceed the limit
tests running here:
https://circleci.com/gh/krummas/cassandra/tree/marcuse%2Fautoupgrade-v2


was (Author: krummas):
bq. Doesn't this mean that only one compaction task can be running at a time?
yeah that was a leftover, pushed the correct version

bq. Worth logging the sstable filename here?
It is logged in {{CompactionStrategyManager#findUpgradeTask}}, removed that log 
line instead

also added tests to make sure we don't exceed the limit

> SSTable upgrade should be automatic
> ---
>
> Key: CASSANDRA-14197
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14197
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 4.x
>
>
> Upgradesstables should run automatically on node upgrade



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458271#comment-16458271
 ] 

Lerh Chuan Low edited comment on CASSANDRA-14427 at 4/30/18 6:59 AM:
-

Github branch if preferred:

[https://github.com/juiceblender/cassandra/tree/jackson-update] 
[https://github.com/juiceblender/cassandra/tree/jackson-update-3.X
https://github.com/juiceblender/cassandra/tree/jackson-update-3.0
https://github.com/juiceblender/cassandra/tree/jackson-update-2.2
https://github.com/juiceblender/cassandra/tree/jackson-update-2.1|https://github.com/juiceblender/cassandra/tree/jackson-update-3.X]
 
CCI:

[https://circleci.com/gh/juiceblender/cassandra/76] (trunk)
 [https://circleci.com/gh/juiceblender/cassandra/77] (3.X)
 [https://circleci.com/gh/juiceblender/cassandra/78] (3.0)
 [https://circleci.com/gh/juiceblender/cassandra/79] (2.2)
 [https://circleci.com/gh/juiceblender/cassandra/80] (2.1)

I get the feeling some of the CCIs may fail (to my knowledge they currently 
don't work on 3.X and 3.0, not sure about 2.Xs). 


was (Author: lerh low):
Github branch if preferred:

[https://github.com/juiceblender/cassandra/tree/jackson-update] 
 [https://github.com/juiceblender/cassandra/tree/jackson-update-3.X
https://github.com/juiceblender/cassandra/tree/jackson-update-3.0
|https://github.com/juiceblender/cassandra/tree/jackson-update-3.X] 
[https://github.com/juiceblender/cassandra/tree/jackson-update-2|https://github.com/juiceblender/cassandra/tree/jackson-update-3.0].2
[https://github.com/juiceblender/cassandra/tree/jackson-update-2|https://github.com/juiceblender/cassandra/tree/jackson-update-3.0].1

CCI:

[https://circleci.com/gh/juiceblender/cassandra/76] (trunk)
[https://circleci.com/gh/juiceblender/cassandra/77] (3.X)
[https://circleci.com/gh/juiceblender/cassandra/78] (3.0)
 [https://circleci.com/gh/juiceblender/cassandra/79] (2.2)
 [https://circleci.com/gh/juiceblender/cassandra/80] (2.1)

I get the feeling some of the CCIs may fail (to my knowledge they currently 
don't work on 3.X and 3.0, not sure about 2.Xs). 

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458271#comment-16458271
 ] 

Lerh Chuan Low edited comment on CASSANDRA-14427 at 4/30/18 6:59 AM:
-

Github branch if preferred:

[https://github.com/juiceblender/cassandra/tree/jackson-update] 
 [https://github.com/juiceblender/cassandra/tree/jackson-update-3.0]
 [https://github.com/juiceblender/cassandra/tree/jackson-update-2.2]
[https://github.com/juiceblender/cassandra/tree/jackson-update-2.1]


  
 CCI:

[https://circleci.com/gh/juiceblender/cassandra/76] (trunk)
 [https://circleci.com/gh/juiceblender/cassandra/77] (3.X)
 [https://circleci.com/gh/juiceblender/cassandra/78] (3.0)
 [https://circleci.com/gh/juiceblender/cassandra/79] (2.2)
 [https://circleci.com/gh/juiceblender/cassandra/80] (2.1)

I get the feeling some of the CCIs may fail (to my knowledge they currently 
don't work on 3.X and 3.0, not sure about 2.Xs). 


was (Author: lerh low):
Github branch if preferred:

[https://github.com/juiceblender/cassandra/tree/jackson-update] 
[https://github.com/juiceblender/cassandra/tree/jackson-update-3.X
https://github.com/juiceblender/cassandra/tree/jackson-update-3.0
https://github.com/juiceblender/cassandra/tree/jackson-update-2.2
https://github.com/juiceblender/cassandra/tree/jackson-update-2.1|https://github.com/juiceblender/cassandra/tree/jackson-update-3.X]
 
CCI:

[https://circleci.com/gh/juiceblender/cassandra/76] (trunk)
 [https://circleci.com/gh/juiceblender/cassandra/77] (3.X)
 [https://circleci.com/gh/juiceblender/cassandra/78] (3.0)
 [https://circleci.com/gh/juiceblender/cassandra/79] (2.2)
 [https://circleci.com/gh/juiceblender/cassandra/80] (2.1)

I get the feeling some of the CCIs may fail (to my knowledge they currently 
don't work on 3.X and 3.0, not sure about 2.Xs). 

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458271#comment-16458271
 ] 

Lerh Chuan Low edited comment on CASSANDRA-14427 at 4/30/18 7:00 AM:
-

Github branch if preferred:

[https://github.com/juiceblender/cassandra/tree/jackson-update] 
[https://github.com/juiceblender/cassandra/tree/jackson-update-3.X]
 [https://github.com/juiceblender/cassandra/tree/jackson-update-3.0]
 [https://github.com/juiceblender/cassandra/tree/jackson-update-2.2]
 [https://github.com/juiceblender/cassandra/tree/jackson-update-2.1]

 
 CCI:

[https://circleci.com/gh/juiceblender/cassandra/76] (trunk)
 [https://circleci.com/gh/juiceblender/cassandra/77] (3.X)
 [https://circleci.com/gh/juiceblender/cassandra/78] (3.0)
 [https://circleci.com/gh/juiceblender/cassandra/79] (2.2)
 [https://circleci.com/gh/juiceblender/cassandra/80] (2.1)

I get the feeling some of the CCIs may fail (to my knowledge they currently 
don't work on 3.X and 3.0, not sure about 2.Xs). 


was (Author: lerh low):
Github branch if preferred:

[https://github.com/juiceblender/cassandra/tree/jackson-update] 
 [https://github.com/juiceblender/cassandra/tree/jackson-update-3.0]
 [https://github.com/juiceblender/cassandra/tree/jackson-update-2.2]
[https://github.com/juiceblender/cassandra/tree/jackson-update-2.1]


  
 CCI:

[https://circleci.com/gh/juiceblender/cassandra/76] (trunk)
 [https://circleci.com/gh/juiceblender/cassandra/77] (3.X)
 [https://circleci.com/gh/juiceblender/cassandra/78] (3.0)
 [https://circleci.com/gh/juiceblender/cassandra/79] (2.2)
 [https://circleci.com/gh/juiceblender/cassandra/80] (2.1)

I get the feeling some of the CCIs may fail (to my knowledge they currently 
don't work on 3.X and 3.0, not sure about 2.Xs). 

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458271#comment-16458271
 ] 

Lerh Chuan Low edited comment on CASSANDRA-14427 at 4/30/18 6:58 AM:
-

Github branch if preferred:

[https://github.com/juiceblender/cassandra/tree/jackson-update] 
 [https://github.com/juiceblender/cassandra/tree/jackson-update-3.X
https://github.com/juiceblender/cassandra/tree/jackson-update-3.0
|https://github.com/juiceblender/cassandra/tree/jackson-update-3.X] 
[https://github.com/juiceblender/cassandra/tree/jackson-update-2|https://github.com/juiceblender/cassandra/tree/jackson-update-3.0].2
[https://github.com/juiceblender/cassandra/tree/jackson-update-2|https://github.com/juiceblender/cassandra/tree/jackson-update-3.0].1

CCI:

[https://circleci.com/gh/juiceblender/cassandra/76] (trunk)
[https://circleci.com/gh/juiceblender/cassandra/77] (3.X)
[https://circleci.com/gh/juiceblender/cassandra/78] (3.0)
 [https://circleci.com/gh/juiceblender/cassandra/79] (2.2)
 [https://circleci.com/gh/juiceblender/cassandra/80] (2.1)

I get the feeling some of the CCIs may fail (to my knowledge they currently 
don't work on 3.X and 3.0, not sure about 2.Xs). 


was (Author: lerh low):
Github branch if preferred: 

[https://github.com/juiceblender/cassandra/tree/jackson-update] [
https://github.com/juiceblender/cassandra/tree/jackson-update-3.X|https://github.com/juiceblender/cassandra/tree/jackson-update]

[https://github.com/juiceblender/cassandra/tree/jackson-update-3.0]

[https://github.com/juiceblender/cassandra/tree/jackson-update-2|https://github.com/juiceblender/cassandra/tree/jackson-update-3.0].2

[https://github.com/juiceblender/cassandra/tree/jackson-update-2|https://github.com/juiceblender/cassandra/tree/jackson-update-3.0].1

 

CCI: 

[https://circleci.com/gh/juiceblender/cassandra/76] (trunk)

[https://circleci.com/gh/juiceblender/cassandra/77] (3.X)

[https://circleci.com/gh/juiceblender/cassandra/78] (3.0)
[https://circleci.com/gh/juiceblender/cassandra/79] (2.2)
[https://circleci.com/gh/juiceblender/cassandra/80] (2.1)

I get the feeling some of the CCIs may fail (to my knowledge they currently 
don't work on 3.X and 3.0, not sure about 2.Xs). 

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13404) Hostname verification for client-to-node encryption

[Jeremy Hanna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremy Hanna updated CASSANDRA-13404:
-
Labels: security  (was: )

> Hostname verification for client-to-node encryption
> ---
>
> Key: CASSANDRA-13404
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13404
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Jan Karlsson
>Assignee: Per Otterström
>Priority: Major
>  Labels: security
> Fix For: 4.x
>
> Attachments: 13404-trunk-v2.patch, 13404-trunk.txt
>
>
> Similarily to CASSANDRA-9220, Cassandra should support hostname verification 
> for client-node connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458271#comment-16458271
 ] 

Lerh Chuan Low edited comment on CASSANDRA-14427 at 4/30/18 6:57 AM:
-

Github branch if preferred: 

[https://github.com/juiceblender/cassandra/tree/jackson-update] [
https://github.com/juiceblender/cassandra/tree/jackson-update-3.X|https://github.com/juiceblender/cassandra/tree/jackson-update]

[https://github.com/juiceblender/cassandra/tree/jackson-update-3.0]

[https://github.com/juiceblender/cassandra/tree/jackson-update-2|https://github.com/juiceblender/cassandra/tree/jackson-update-3.0].2

[https://github.com/juiceblender/cassandra/tree/jackson-update-2|https://github.com/juiceblender/cassandra/tree/jackson-update-3.0].1

 

CCI: 

[https://circleci.com/gh/juiceblender/cassandra/76] (trunk)

[https://circleci.com/gh/juiceblender/cassandra/77] (3.X)

[https://circleci.com/gh/juiceblender/cassandra/78] (3.0)
[https://circleci.com/gh/juiceblender/cassandra/79] (2.2)
[https://circleci.com/gh/juiceblender/cassandra/80] (2.1)

I get the feeling some of the CCIs may fail (to my knowledge they currently 
don't work on 3.X and 3.0, not sure about 2.Xs). 


was (Author: lerh low):
Github branch if preferred: 
[https://github.com/juiceblender/cassandra/tree/jackson-update]

CCI: [https://circleci.com/gh/juiceblender/cassandra/76]

Not sure if these should include all the previous versions (I think it should), 
let me know if I'm on the right track + if I should create patches for 
2.1/2.2/3.0/3. Thanks!

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14427) Bump jackson version to >= 2.9.5

[Lerh Chuan Low (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lerh Chuan Low updated CASSANDRA-14427:
---
Attachment: 3.X-14427.txt
3.0-14427.txt
2.2-14427.txt
2.1-14427.txt

> Bump jackson version to >= 2.9.5
> 
>
> Key: CASSANDRA-14427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14427
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Lerh Chuan Low
>Assignee: Lerh Chuan Low
>Priority: Major
> Attachments: 2.1-14427.txt, 2.2-14427.txt, 3.0-14427.txt, 
> 3.X-14427.txt, trunk-14427.txt
>
>
> The Jackson being used by Cassandra is really old (1.9.2, and still 
> references codehaus (Jackson 1) instead of fasterxml (Jackson 2)). 
> There have been a few jackson vulnerabilities recently (mostly around 
> deserialization which allows arbitrary code execution)
> [https://nvd.nist.gov/vuln/detail/CVE-2017-7525]
>  [https://nvd.nist.gov/vuln/detail/CVE-2017-15095]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-1327]
>  [https://nvd.nist.gov/vuln/detail/CVE-2018-7489]
> Given that Jackson in Cassandra is really old and seems to be used also for 
> reading in values, it looks worthwhile to update Jackson to 2.9.5. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13404) Hostname verification for client-to-node encryption

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458359#comment-16458359
 ] 

Per Otterström commented on CASSANDRA-13404:


Thanks, and I appreciate your patience with me. :)

The patch is small, so shouldn't be to much. One thing I should mention though. 
In the dtests I'm using the openssl command to convert keys to pem format. I 
suppose this brings in a "new dependency" to the dtests. Is that OK?

> Hostname verification for client-to-node encryption
> ---
>
> Key: CASSANDRA-13404
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13404
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Jan Karlsson
>Assignee: Per Otterström
>Priority: Major
> Fix For: 4.x
>
> Attachments: 13404-trunk-v2.patch, 13404-trunk.txt
>
>
> Similarily to CASSANDRA-9220, Cassandra should support hostname verification 
> for client-node connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org