[jira] [Commented] (CASSANDRA-11163) Summaries are needlessly rebuilt when the BF FP ratio is changed
[ https://issues.apache.org/jira/browse/CASSANDRA-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832752#comment-15832752 ] Jeff Jirsa commented on CASSANDRA-11163: It persists until all sstables are rewritten - the bloom filter is recalculated on startup but not persisted to disk until compaction runs. You can force the bloom filter to be rewritten without waiting dircompaction or this bug to be fixed by running "nodetool upgradesstables -a" to rewrite all of the sstables (note that this can be IO intensive and impact your latencies) > Summaries are needlessly rebuilt when the BF FP ratio is changed > > > Key: CASSANDRA-11163 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11163 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Brandon Williams > Fix For: 3.x > > > This is from trunk, but I also saw this happen on 2.0: > Before: > {noformat} > root@bw-1:/srv/cassandra# ls -ltr > /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ > total 221460 > drwxr-xr-x 2 root root 4096 Feb 11 23:34 backups > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt > -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-6-big-Summary.db > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db > -rw-r--r-- 1 root root 2607705 Feb 11 23:50 ma-6-big-Index.db > -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db > -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32 > -rw-r--r-- 1 root root 35212125 Feb 11 23:50 ma-6-big-Data.db > -rw-r--r-- 1 root root 2156 Feb 11 23:50 ma-6-big-CRC.db > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt > -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-7-big-Summary.db > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db > -rw-r--r-- 1 root root 2607614 Feb 11 23:50 ma-7-big-Index.db > -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db > -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32 > -rw-r--r-- 1 root root 35190400 Feb 11 23:50 ma-7-big-Data.db > -rw-r--r-- 1 root root 2152 Feb 11 23:50 ma-7-big-CRC.db > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt > -rw-r--r-- 1 root root104178 Feb 11 23:50 ma-5-big-Summary.db > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db > -rw-r--r-- 1 root root 10289077 Feb 11 23:50 ma-5-big-Index.db > -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db > -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32 > -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db > -rw-r--r-- 1 root root 8508 Feb 11 23:50 ma-5-big-CRC.db > root@bw-1:/srv/cassandra# md5sum > /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ma-5-big-Summary.db > 5fca154fc790f7cfa37e8ad6d1c7552c > {noformat} > BF ratio changed, node restarted: > {noformat} > root@bw-1:/srv/cassandra# ls -ltr > /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ > total 242168 > drwxr-xr-x 2 root root 4096 Feb 11 23:34 backups > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db > -rw-r--r-- 1 root root 2607705 Feb 11 23:50 ma-6-big-Index.db > -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db > -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32 > -rw-r--r-- 1 root root 35212125 Feb 11 23:50 ma-6-big-Data.db > -rw-r--r-- 1 root root 2156 Feb 11 23:50 ma-6-big-CRC.db > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db > -rw-r--r-- 1 root root 2607614 Feb 11 23:50 ma-7-big-Index.db > -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db > -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32 > -rw-r--r-- 1 root root 35190400 Feb 11 23:50 ma-7-big-Data.db > -rw-r--r-- 1 root root 2152 Feb 11 23:50 ma-7-big-CRC.db > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db > -rw-r--r-- 1 root root 10289077 Feb 11 23:50 ma-5-big-Index.db > -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db > -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32 > -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db > -rw-r--r-- 1 root root 8508 Feb 11 23:50 ma-5-big-CRC.db > -rw-r--r-- 1 root root80 Feb 12 00:03 ma-8-big-TOC.txt > -rw-r--r-- 1 root root 14902 Feb 12 00:03 ma-8-big-Summary.db > -rw-r--r-- 1 root root 10264 Feb 12 00:03 ma-8-big-Statistics.db > -rw-r--r-- 1 root root 1458631 Feb 1
[jira] [Commented] (CASSANDRA-13009) Speculative retry bugs
[ https://issues.apache.org/jira/browse/CASSANDRA-13009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832727#comment-15832727 ] Simon Zhou commented on CASSANDRA-13009: [~tjake], do you mind reviewing this patch? Thanks. > Speculative retry bugs > -- > > Key: CASSANDRA-13009 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13009 > Project: Cassandra > Issue Type: Bug >Reporter: Simon Zhou >Assignee: Simon Zhou > Fix For: 3.0.11 > > Attachments: CASSANDRA-13009-v1.patch > > > There are a few issues with speculative retry: > 1. Time unit bugs. These are from ColumnFamilyStore (v3.0.10): > The left hand side is in nanos, as the name suggests, while the right hand > side is in millis. > {code} > sampleLatencyNanos = DatabaseDescriptor.getReadRpcTimeout() / 2; > {code} > Here coordinatorReadLatency is already in nanos and we shouldn't multiple the > value by 1000. This was a regression in 8896a70 when we switch metrics > library and the two libraries use different time units. > {code} > sampleLatencyNanos = (long) > (metric.coordinatorReadLatency.getSnapshot().getValue(retryPolicy.threshold()) > * 1000d); > {code} > 2. Confusing overload protection and retry delay. As the name > "sampleLatencyNanos" suggests, it should be used to keep the actually sampled > read latency. However, we assign it the retry threshold in the case of > CUSTOM. Then we compare the retry threshold with read timeout (defaults to > 5000ms). This means, if we use speculative_retry=10ms for the table, we won't > be able to avoid being overloaded. We should compare the actual read latency > with the read timeout for overload protection. See line 450 of > ColumnFamilyStore.java and line 279 of AbstractReadExecutor.java. > My proposals are: > a. We use sampled p99 delay and compare it with a customizable threshold > (-Dcassandra.overload.threshold) for overload detection. > b. Introduce another variable retryDelayNanos for waiting time before retry. > This is the value from table setting (PERCENTILE or CUSTOM). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13009) Speculative retry bugs
[ https://issues.apache.org/jira/browse/CASSANDRA-13009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Zhou updated CASSANDRA-13009: --- Reviewer: (was: T Jake Luciani) > Speculative retry bugs > -- > > Key: CASSANDRA-13009 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13009 > Project: Cassandra > Issue Type: Bug >Reporter: Simon Zhou >Assignee: Simon Zhou > Fix For: 3.0.11 > > Attachments: CASSANDRA-13009-v1.patch > > > There are a few issues with speculative retry: > 1. Time unit bugs. These are from ColumnFamilyStore (v3.0.10): > The left hand side is in nanos, as the name suggests, while the right hand > side is in millis. > {code} > sampleLatencyNanos = DatabaseDescriptor.getReadRpcTimeout() / 2; > {code} > Here coordinatorReadLatency is already in nanos and we shouldn't multiple the > value by 1000. This was a regression in 8896a70 when we switch metrics > library and the two libraries use different time units. > {code} > sampleLatencyNanos = (long) > (metric.coordinatorReadLatency.getSnapshot().getValue(retryPolicy.threshold()) > * 1000d); > {code} > 2. Confusing overload protection and retry delay. As the name > "sampleLatencyNanos" suggests, it should be used to keep the actually sampled > read latency. However, we assign it the retry threshold in the case of > CUSTOM. Then we compare the retry threshold with read timeout (defaults to > 5000ms). This means, if we use speculative_retry=10ms for the table, we won't > be able to avoid being overloaded. We should compare the actual read latency > with the read timeout for overload protection. See line 450 of > ColumnFamilyStore.java and line 279 of AbstractReadExecutor.java. > My proposals are: > a. We use sampled p99 delay and compare it with a customizable threshold > (-Dcassandra.overload.threshold) for overload detection. > b. Introduce another variable retryDelayNanos for waiting time before retry. > This is the value from table setting (PERCENTILE or CUSTOM). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: ninja-fix followup to CASSANDRA-4663
Repository: cassandra Updated Branches: refs/heads/trunk 4d67639d3 -> 85e4df4bb ninja-fix followup to CASSANDRA-4663 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/85e4df4b Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/85e4df4b Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/85e4df4b Branch: refs/heads/trunk Commit: 85e4df4bbf0ed5447c7328faa7495ad0b0f1cc22 Parents: 4d67639 Author: Jason Brown Authored: Fri Jan 20 13:46:56 2017 -0800 Committer: Jason Brown Committed: Fri Jan 20 13:46:56 2017 -0800 -- test/unit/org/apache/cassandra/dht/BootStrapperTest.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/85e4df4b/test/unit/org/apache/cassandra/dht/BootStrapperTest.java -- diff --git a/test/unit/org/apache/cassandra/dht/BootStrapperTest.java b/test/unit/org/apache/cassandra/dht/BootStrapperTest.java index a2858dc..ba0352d 100644 --- a/test/unit/org/apache/cassandra/dht/BootStrapperTest.java +++ b/test/unit/org/apache/cassandra/dht/BootStrapperTest.java @@ -98,7 +98,7 @@ public class BootStrapperTest InetAddress myEndpoint = InetAddress.getByName("127.0.0.1"); assertEquals(numOldNodes, tmd.sortedTokens().size()); -RangeStreamer s = new RangeStreamer(tmd, null, myEndpoint, "Bootstrap", true, DatabaseDescriptor.getEndpointSnitch(), new StreamStateStore(), false); +RangeStreamer s = new RangeStreamer(tmd, null, myEndpoint, "Bootstrap", true, DatabaseDescriptor.getEndpointSnitch(), new StreamStateStore(), false, 1); IFailureDetector mockFailureDetector = new IFailureDetector() { public boolean isAlive(InetAddress ep)
[jira] [Comment Edited] (CASSANDRA-11163) Summaries are needlessly rebuilt when the BF FP ratio is changed
[ https://issues.apache.org/jira/browse/CASSANDRA-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832423#comment-15832423 ] Soumya Sanyal edited comment on CASSANDRA-11163 at 1/20/17 9:31 PM: FWIW - this is causing startup on nodes in our cluster to take in excess of 15mins. We do have roughly 1.5TB on disk. We're on a patched version of 3.9. What I don't quite understand is why this behavior persists past the first restart of the BF FP? was (Author: _soumya_): FWIW - this is causing startup on nodes in our cluster to take in excess of 15mins. We do have roughly 1.5TB on disk. We're on a patched version of 3.9 > Summaries are needlessly rebuilt when the BF FP ratio is changed > > > Key: CASSANDRA-11163 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11163 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Brandon Williams > Fix For: 3.x > > > This is from trunk, but I also saw this happen on 2.0: > Before: > {noformat} > root@bw-1:/srv/cassandra# ls -ltr > /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ > total 221460 > drwxr-xr-x 2 root root 4096 Feb 11 23:34 backups > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt > -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-6-big-Summary.db > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db > -rw-r--r-- 1 root root 2607705 Feb 11 23:50 ma-6-big-Index.db > -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db > -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32 > -rw-r--r-- 1 root root 35212125 Feb 11 23:50 ma-6-big-Data.db > -rw-r--r-- 1 root root 2156 Feb 11 23:50 ma-6-big-CRC.db > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt > -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-7-big-Summary.db > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db > -rw-r--r-- 1 root root 2607614 Feb 11 23:50 ma-7-big-Index.db > -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db > -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32 > -rw-r--r-- 1 root root 35190400 Feb 11 23:50 ma-7-big-Data.db > -rw-r--r-- 1 root root 2152 Feb 11 23:50 ma-7-big-CRC.db > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt > -rw-r--r-- 1 root root104178 Feb 11 23:50 ma-5-big-Summary.db > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db > -rw-r--r-- 1 root root 10289077 Feb 11 23:50 ma-5-big-Index.db > -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db > -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32 > -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db > -rw-r--r-- 1 root root 8508 Feb 11 23:50 ma-5-big-CRC.db > root@bw-1:/srv/cassandra# md5sum > /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ma-5-big-Summary.db > 5fca154fc790f7cfa37e8ad6d1c7552c > {noformat} > BF ratio changed, node restarted: > {noformat} > root@bw-1:/srv/cassandra# ls -ltr > /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ > total 242168 > drwxr-xr-x 2 root root 4096 Feb 11 23:34 backups > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db > -rw-r--r-- 1 root root 2607705 Feb 11 23:50 ma-6-big-Index.db > -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db > -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32 > -rw-r--r-- 1 root root 35212125 Feb 11 23:50 ma-6-big-Data.db > -rw-r--r-- 1 root root 2156 Feb 11 23:50 ma-6-big-CRC.db > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db > -rw-r--r-- 1 root root 2607614 Feb 11 23:50 ma-7-big-Index.db > -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db > -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32 > -rw-r--r-- 1 root root 35190400 Feb 11 23:50 ma-7-big-Data.db > -rw-r--r-- 1 root root 2152 Feb 11 23:50 ma-7-big-CRC.db > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db > -rw-r--r-- 1 root root 10289077 Feb 11 23:50 ma-5-big-Index.db > -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db > -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32 > -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db > -rw-r--r-- 1 root root 8508 Feb 11 23:50 ma-5-big-CRC.db > -rw-r--r-- 1 root root80 Feb 12 00:03 ma-8-big-TOC.txt > -rw-r--r-- 1 root root 14902 Feb 12 00:03 ma-8-big-Summary.db > -r
[jira] [Comment Edited] (CASSANDRA-11163) Summaries are needlessly rebuilt when the BF FP ratio is changed
[ https://issues.apache.org/jira/browse/CASSANDRA-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832423#comment-15832423 ] Soumya Sanyal edited comment on CASSANDRA-11163 at 1/20/17 9:27 PM: FWIW - this is causing startup on nodes in our cluster to take in excess of 15mins. We do have roughly 1.5TB on disk. We're on a patched version of 3.9 was (Author: _soumya_): FWIW - this is causing startup on nodes in our cluster to take in excess of 15mins. We do have roughly 1.5TB on disk. > Summaries are needlessly rebuilt when the BF FP ratio is changed > > > Key: CASSANDRA-11163 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11163 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Brandon Williams > Fix For: 3.x > > > This is from trunk, but I also saw this happen on 2.0: > Before: > {noformat} > root@bw-1:/srv/cassandra# ls -ltr > /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ > total 221460 > drwxr-xr-x 2 root root 4096 Feb 11 23:34 backups > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt > -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-6-big-Summary.db > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db > -rw-r--r-- 1 root root 2607705 Feb 11 23:50 ma-6-big-Index.db > -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db > -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32 > -rw-r--r-- 1 root root 35212125 Feb 11 23:50 ma-6-big-Data.db > -rw-r--r-- 1 root root 2156 Feb 11 23:50 ma-6-big-CRC.db > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt > -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-7-big-Summary.db > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db > -rw-r--r-- 1 root root 2607614 Feb 11 23:50 ma-7-big-Index.db > -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db > -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32 > -rw-r--r-- 1 root root 35190400 Feb 11 23:50 ma-7-big-Data.db > -rw-r--r-- 1 root root 2152 Feb 11 23:50 ma-7-big-CRC.db > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt > -rw-r--r-- 1 root root104178 Feb 11 23:50 ma-5-big-Summary.db > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db > -rw-r--r-- 1 root root 10289077 Feb 11 23:50 ma-5-big-Index.db > -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db > -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32 > -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db > -rw-r--r-- 1 root root 8508 Feb 11 23:50 ma-5-big-CRC.db > root@bw-1:/srv/cassandra# md5sum > /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ma-5-big-Summary.db > 5fca154fc790f7cfa37e8ad6d1c7552c > {noformat} > BF ratio changed, node restarted: > {noformat} > root@bw-1:/srv/cassandra# ls -ltr > /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ > total 242168 > drwxr-xr-x 2 root root 4096 Feb 11 23:34 backups > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db > -rw-r--r-- 1 root root 2607705 Feb 11 23:50 ma-6-big-Index.db > -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db > -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32 > -rw-r--r-- 1 root root 35212125 Feb 11 23:50 ma-6-big-Data.db > -rw-r--r-- 1 root root 2156 Feb 11 23:50 ma-6-big-CRC.db > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db > -rw-r--r-- 1 root root 2607614 Feb 11 23:50 ma-7-big-Index.db > -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db > -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32 > -rw-r--r-- 1 root root 35190400 Feb 11 23:50 ma-7-big-Data.db > -rw-r--r-- 1 root root 2152 Feb 11 23:50 ma-7-big-CRC.db > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db > -rw-r--r-- 1 root root 10289077 Feb 11 23:50 ma-5-big-Index.db > -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db > -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32 > -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db > -rw-r--r-- 1 root root 8508 Feb 11 23:50 ma-5-big-CRC.db > -rw-r--r-- 1 root root80 Feb 12 00:03 ma-8-big-TOC.txt > -rw-r--r-- 1 root root 14902 Feb 12 00:03 ma-8-big-Summary.db > -rw-r--r-- 1 root root 10264 Feb 12 00:03 ma-8-big-Statistics.db > -rw-r--r-- 1 root root 1458631 Feb 12 00:03 ma-8-big-Index.db >
[jira] [Commented] (CASSANDRA-11163) Summaries are needlessly rebuilt when the BF FP ratio is changed
[ https://issues.apache.org/jira/browse/CASSANDRA-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832423#comment-15832423 ] Soumya Sanyal commented on CASSANDRA-11163: --- FWIW - this is causing startup on nodes in our cluster to take in excess of 15mins. We do have roughly 1.5TB on disk. > Summaries are needlessly rebuilt when the BF FP ratio is changed > > > Key: CASSANDRA-11163 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11163 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Brandon Williams > Fix For: 3.x > > > This is from trunk, but I also saw this happen on 2.0: > Before: > {noformat} > root@bw-1:/srv/cassandra# ls -ltr > /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ > total 221460 > drwxr-xr-x 2 root root 4096 Feb 11 23:34 backups > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt > -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-6-big-Summary.db > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db > -rw-r--r-- 1 root root 2607705 Feb 11 23:50 ma-6-big-Index.db > -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db > -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32 > -rw-r--r-- 1 root root 35212125 Feb 11 23:50 ma-6-big-Data.db > -rw-r--r-- 1 root root 2156 Feb 11 23:50 ma-6-big-CRC.db > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt > -rw-r--r-- 1 root root 26518 Feb 11 23:50 ma-7-big-Summary.db > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db > -rw-r--r-- 1 root root 2607614 Feb 11 23:50 ma-7-big-Index.db > -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db > -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32 > -rw-r--r-- 1 root root 35190400 Feb 11 23:50 ma-7-big-Data.db > -rw-r--r-- 1 root root 2152 Feb 11 23:50 ma-7-big-CRC.db > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt > -rw-r--r-- 1 root root104178 Feb 11 23:50 ma-5-big-Summary.db > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db > -rw-r--r-- 1 root root 10289077 Feb 11 23:50 ma-5-big-Index.db > -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db > -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32 > -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db > -rw-r--r-- 1 root root 8508 Feb 11 23:50 ma-5-big-CRC.db > root@bw-1:/srv/cassandra# md5sum > /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ma-5-big-Summary.db > 5fca154fc790f7cfa37e8ad6d1c7552c > {noformat} > BF ratio changed, node restarted: > {noformat} > root@bw-1:/srv/cassandra# ls -ltr > /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ > total 242168 > drwxr-xr-x 2 root root 4096 Feb 11 23:34 backups > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-6-big-TOC.txt > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-6-big-Statistics.db > -rw-r--r-- 1 root root 2607705 Feb 11 23:50 ma-6-big-Index.db > -rw-r--r-- 1 root root192440 Feb 11 23:50 ma-6-big-Filter.db > -rw-r--r-- 1 root root10 Feb 11 23:50 ma-6-big-Digest.crc32 > -rw-r--r-- 1 root root 35212125 Feb 11 23:50 ma-6-big-Data.db > -rw-r--r-- 1 root root 2156 Feb 11 23:50 ma-6-big-CRC.db > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-7-big-TOC.txt > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-7-big-Statistics.db > -rw-r--r-- 1 root root 2607614 Feb 11 23:50 ma-7-big-Index.db > -rw-r--r-- 1 root root192432 Feb 11 23:50 ma-7-big-Filter.db > -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-7-big-Digest.crc32 > -rw-r--r-- 1 root root 35190400 Feb 11 23:50 ma-7-big-Data.db > -rw-r--r-- 1 root root 2152 Feb 11 23:50 ma-7-big-CRC.db > -rw-r--r-- 1 root root80 Feb 11 23:50 ma-5-big-TOC.txt > -rw-r--r-- 1 root root 10264 Feb 11 23:50 ma-5-big-Statistics.db > -rw-r--r-- 1 root root 10289077 Feb 11 23:50 ma-5-big-Index.db > -rw-r--r-- 1 root root757384 Feb 11 23:50 ma-5-big-Filter.db > -rw-r--r-- 1 root root 9 Feb 11 23:50 ma-5-big-Digest.crc32 > -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db > -rw-r--r-- 1 root root 8508 Feb 11 23:50 ma-5-big-CRC.db > -rw-r--r-- 1 root root80 Feb 12 00:03 ma-8-big-TOC.txt > -rw-r--r-- 1 root root 14902 Feb 12 00:03 ma-8-big-Summary.db > -rw-r--r-- 1 root root 10264 Feb 12 00:03 ma-8-big-Statistics.db > -rw-r--r-- 1 root root 1458631 Feb 12 00:03 ma-8-big-Index.db > -rw-r--r-- 1 root root 10808 Feb 12 00:03 ma-8-big-Filter.db > -rw-r--r-- 1 root root10 Feb 12 00:03 ma-8-big-Digest.crc32 > -rw-r--r-- 1 root root 19660275 Feb 12 00:03 ma-8-big-Data.db > -rw-r--r-- 1 root root
[jira] [Updated] (CASSANDRA-12847) cqlsh DESCRIBE output doesn't properly quote index names
[ https://issues.apache.org/jira/browse/CASSANDRA-12847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-12847: Fix Version/s: (was: 2.2.x) Status: Patch Available (was: In Progress) > cqlsh DESCRIBE output doesn't properly quote index names > > > Key: CASSANDRA-12847 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12847 > Project: Cassandra > Issue Type: Bug >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe > Labels: cqlsh > Fix For: 3.0.x, 3.x > > > CASSANDRA-8365 fixed the CQL grammar so that quoting index names preserves > case. The output of DESCRIBE in cqlsh wasn't updated however so this doesn't > round-trip properly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12847) cqlsh DESCRIBE output doesn't properly quote index names
[ https://issues.apache.org/jira/browse/CASSANDRA-12847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832409#comment-15832409 ] Sam Tunnicliffe commented on CASSANDRA-12847: - I've updated the bundled driver and kicked off CI runs (for 3.0 & 3.11 branches only as the trunk build is broken currently, I'll do the same for that when it's fixed). Also updated the dtest to only expect the new behaviour with the correct versions. ||branch||testall||dtest|| |[12847-3.0|https://github.com/beobal/cassandra/tree/12847-3.0]|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-12847-3.0-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-12847-3.0-dtest]| |[12847-3.11|https://github.com/beobal/cassandra/tree/12847-3.11]|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-12847-3.11-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-12847-3.11-dtest]| |[12847-trunk|https://github.com/beobal/cassandra/tree/12847-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-12847-trunk-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-12847-trunk-dtest]| cc [~aholmber] [~ifesdjeen] [~philipthompson] > cqlsh DESCRIBE output doesn't properly quote index names > > > Key: CASSANDRA-12847 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12847 > Project: Cassandra > Issue Type: Bug >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe > Labels: cqlsh > Fix For: 3.0.x, 3.x > > > CASSANDRA-8365 fixed the CQL grammar so that quoting index names preserves > case. The output of DESCRIBE in cqlsh wasn't updated however so this doesn't > round-trip properly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13125) Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9
[ https://issues.apache.org/jira/browse/CASSANDRA-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832400#comment-15832400 ] Michael Shuler commented on CASSANDRA-13125: Would this have something to do with it? {noformat} (cassandra-3.11)mshuler@hana:~/git/cassandra$ grep -r sigar-bin conf/ conf/cassandra-env.ps1:$env:JVM_OPTS = "$env:JVM_OPTS -Djava.library.path=""$env:CASSANDRA_HOME\lib\sigar-bin""" conf/cassandra-env.sh:JVM_OPTS="$JVM_OPTS -Djava.library.path=$CASSANDRA_HOME/lib/sigar-bin" (cassandra-3.11)mshuler@hana:~/git/cassandra$ grep -r sigar-bin test/conf/ (cassandra-3.11)mshuler@hana:~/git/cassandra$ {noformat} > Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9 > > > Key: CASSANDRA-13125 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13125 > Project: Cassandra > Issue Type: Bug >Reporter: Zhongxiang Zheng >Assignee: Yasuharu Goto >Priority: Critical > Attachments: diff-a.patch, diff-b.patch > > > I found that rows are splitting and duplicated after upgrading the cluster > from 2.1.x to 3.0.x. > I found the way to reproduce the problem as below. > {code} > $ ccm create test -v 2.1.16 -n 3 -s > > Current cluster is now: test > $ ccm node1 cqlsh -e "CREATE KEYSPACE test WITH replication = > {'class':'SimpleStrategy', 'replication_factor':3}" > $ ccm node1 cqlsh -e "CREATE TABLE test.test (id text PRIMARY KEY, value1 > set, value2 set);" > # Upgrade node1 > $ for i in 1; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm > node${i} start;ccm node${i} nodetool upgradesstables; done > # Insert a row through node1(3.0.10) > $ ccm node1 cqlsh -e "INSERT INTO test.test (id, value1, value2) values > ('aaa', {'aaa', 'bbb'}, {'ccc', 'ddd'});" > # Insert a row through node2(2.1.16) > $ ccm node2 cqlsh -e "INSERT INTO test.test (id, value1, value2) values > ('bbb', {'aaa', 'bbb'}, {'ccc', 'ddd'});" > # The row inserted from node1 is splitting > $ ccm node1 cqlsh -e "SELECT * FROM test.test ;" > id | value1 | value2 > -++ > aaa | null | null > aaa | {'aaa', 'bbb'} | {'ccc', 'ddd'} > bbb | {'aaa', 'bbb'} | {'ccc', 'ddd'} > $ for i in 1 2; do ccm node${i} nodetool flush; done > # Results of sstable2json of node2. The row inserted from node1(3.0.10) is > different from the row inserted from node2(2.1.16). > $ ccm node2 json -k test -c test > running > ['/home/zzheng/.ccm/test/node2/data0/test/test-5406ee80dbdb11e6a175f57c4c7c85f3/test-test-ka-1-Data.db'] > -- test-test-ka-1-Data.db - > [ > {"key": "aaa", > "cells": [["","",1484564624769577], >["value1","value2:!",1484564624769576,"t",1484564624], >["value1:616161","",1484564624769577], >["value1:626262","",1484564624769577], >["value2:636363","",1484564624769577], >["value2:646464","",1484564624769577]]}, > {"key": "bbb", > "cells": [["","",1484564634508029], >["value1:_","value1:!",1484564634508028,"t",1484564634], >["value1:616161","",1484564634508029], >["value1:626262","",1484564634508029], >["value2:_","value2:!",1484564634508028,"t",1484564634], >["value2:636363","",1484564634508029], >["value2:646464","",1484564634508029]]} > ] > # Upgrade node2,3 > $ for i in `seq 2 3`; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm > node${i} start;ccm node${i} nodetool upgradesstables; done > # After upgrade node2,3, the row inserted from node1 is splitting in node2,3 > $ ccm node2 cqlsh -e "SELECT * FROM test.test ;" > > id | value1 | value2 > -++ > aaa | null | null > aaa | {'aaa', 'bbb'} | {'ccc', 'ddd'} > bbb | {'aaa', 'bbb'} | {'ccc', 'ddd'} > (3 rows) > # Results of sstabledump > # node1 > [ > { > "partition" : { > "key" : [ "aaa" ], > "position" : 0 > }, > "rows" : [ > { > "type" : "row", > "position" : 17, > "liveness_info" : { "tstamp" : "2017-01-16T11:03:44.769577Z" }, > "cells" : [ > { "name" : "value1", "deletion_info" : { "marked_deleted" : > "2017-01-16T11:03:44.769576Z", "local_delete_time" : "2017-01-16T11:03:44Z" } > }, > { "name" : "value1", "path" : [ "aaa" ], "value" : "" }, > { "name" : "value1", "path" : [ "bbb" ], "value" : "" }, > { "name" : "value2", "deletion_info" : { "marked_deleted" : > "2017-01-16T11:03:44.769576Z", "local_delete_time" : "2017-01-16T11:03:44Z" } > }, >
[jira] [Commented] (CASSANDRA-13125) Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9
[ https://issues.apache.org/jira/browse/CASSANDRA-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832384#comment-15832384 ] Tyler Hobbs commented on CASSANDRA-13125: - [~slebresne] the patch and new test look good to me. For some reason related to sigar your new unit test is erroring on Jenkins in the 3.11 version -- maybe [~mshuler] knows what's up with that? > Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9 > > > Key: CASSANDRA-13125 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13125 > Project: Cassandra > Issue Type: Bug >Reporter: Zhongxiang Zheng >Assignee: Yasuharu Goto >Priority: Critical > Attachments: diff-a.patch, diff-b.patch > > > I found that rows are splitting and duplicated after upgrading the cluster > from 2.1.x to 3.0.x. > I found the way to reproduce the problem as below. > {code} > $ ccm create test -v 2.1.16 -n 3 -s > > Current cluster is now: test > $ ccm node1 cqlsh -e "CREATE KEYSPACE test WITH replication = > {'class':'SimpleStrategy', 'replication_factor':3}" > $ ccm node1 cqlsh -e "CREATE TABLE test.test (id text PRIMARY KEY, value1 > set, value2 set);" > # Upgrade node1 > $ for i in 1; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm > node${i} start;ccm node${i} nodetool upgradesstables; done > # Insert a row through node1(3.0.10) > $ ccm node1 cqlsh -e "INSERT INTO test.test (id, value1, value2) values > ('aaa', {'aaa', 'bbb'}, {'ccc', 'ddd'});" > # Insert a row through node2(2.1.16) > $ ccm node2 cqlsh -e "INSERT INTO test.test (id, value1, value2) values > ('bbb', {'aaa', 'bbb'}, {'ccc', 'ddd'});" > # The row inserted from node1 is splitting > $ ccm node1 cqlsh -e "SELECT * FROM test.test ;" > id | value1 | value2 > -++ > aaa | null | null > aaa | {'aaa', 'bbb'} | {'ccc', 'ddd'} > bbb | {'aaa', 'bbb'} | {'ccc', 'ddd'} > $ for i in 1 2; do ccm node${i} nodetool flush; done > # Results of sstable2json of node2. The row inserted from node1(3.0.10) is > different from the row inserted from node2(2.1.16). > $ ccm node2 json -k test -c test > running > ['/home/zzheng/.ccm/test/node2/data0/test/test-5406ee80dbdb11e6a175f57c4c7c85f3/test-test-ka-1-Data.db'] > -- test-test-ka-1-Data.db - > [ > {"key": "aaa", > "cells": [["","",1484564624769577], >["value1","value2:!",1484564624769576,"t",1484564624], >["value1:616161","",1484564624769577], >["value1:626262","",1484564624769577], >["value2:636363","",1484564624769577], >["value2:646464","",1484564624769577]]}, > {"key": "bbb", > "cells": [["","",1484564634508029], >["value1:_","value1:!",1484564634508028,"t",1484564634], >["value1:616161","",1484564634508029], >["value1:626262","",1484564634508029], >["value2:_","value2:!",1484564634508028,"t",1484564634], >["value2:636363","",1484564634508029], >["value2:646464","",1484564634508029]]} > ] > # Upgrade node2,3 > $ for i in `seq 2 3`; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm > node${i} start;ccm node${i} nodetool upgradesstables; done > # After upgrade node2,3, the row inserted from node1 is splitting in node2,3 > $ ccm node2 cqlsh -e "SELECT * FROM test.test ;" > > id | value1 | value2 > -++ > aaa | null | null > aaa | {'aaa', 'bbb'} | {'ccc', 'ddd'} > bbb | {'aaa', 'bbb'} | {'ccc', 'ddd'} > (3 rows) > # Results of sstabledump > # node1 > [ > { > "partition" : { > "key" : [ "aaa" ], > "position" : 0 > }, > "rows" : [ > { > "type" : "row", > "position" : 17, > "liveness_info" : { "tstamp" : "2017-01-16T11:03:44.769577Z" }, > "cells" : [ > { "name" : "value1", "deletion_info" : { "marked_deleted" : > "2017-01-16T11:03:44.769576Z", "local_delete_time" : "2017-01-16T11:03:44Z" } > }, > { "name" : "value1", "path" : [ "aaa" ], "value" : "" }, > { "name" : "value1", "path" : [ "bbb" ], "value" : "" }, > { "name" : "value2", "deletion_info" : { "marked_deleted" : > "2017-01-16T11:03:44.769576Z", "local_delete_time" : "2017-01-16T11:03:44Z" } > }, > { "name" : "value2", "path" : [ "ccc" ], "value" : "" }, > { "name" : "value2", "path" : [ "ddd" ], "value" : "" } > ] > } > ] > }, > { > "partition" : { > "key" : [ "bbb" ], > "position" : 48 > }, > "rows" : [ >
[jira] [Commented] (CASSANDRA-13143) Cassandra can accept invalid durations
[ https://issues.apache.org/jira/browse/CASSANDRA-13143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832371#comment-15832371 ] Benjamin Lerer commented on CASSANDRA-13143: [~thobbs] Could you also review that ticket? > Cassandra can accept invalid durations > -- > > Key: CASSANDRA-13143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13143 > Project: Cassandra > Issue Type: Bug >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer > > A duration can be positive or negative. If the duration is positive the > months, days and nanoseconds must be greater or equals to zero. If the > duration is negative the months, days and nanoseconds must be smaller or > equals to zero. > Currently, it is possible to send to C* a duration which does not respect > that rule and the data will not be reject. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13143) Cassandra can accept invalid durations
[ https://issues.apache.org/jira/browse/CASSANDRA-13143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-13143: --- Assignee: Benjamin Lerer Status: Patch Available (was: Open) > Cassandra can accept invalid durations > -- > > Key: CASSANDRA-13143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13143 > Project: Cassandra > Issue Type: Bug >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer > > A duration can be positive or negative. If the duration is positive the > months, days and nanoseconds must be greater or equals to zero. If the > duration is negative the months, days and nanoseconds must be smaller or > equals to zero. > Currently, it is possible to send to C* a duration which does not respect > that rule and the data will not be reject. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13143) Cassandra can accept invalid durations
[ https://issues.apache.org/jira/browse/CASSANDRA-13143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832370#comment-15832370 ] Benjamin Lerer commented on CASSANDRA-13143: ||3.11|[branch|https://github.com/apache/cassandra/compare/trunk...blerer:13143-3.11]|[utest|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-13143-3.11-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-13143-3.11-dtest/]| The patch fix the validation and add some tests for it. > Cassandra can accept invalid durations > -- > > Key: CASSANDRA-13143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13143 > Project: Cassandra > Issue Type: Bug >Reporter: Benjamin Lerer > > A duration can be positive or negative. If the duration is positive the > months, days and nanoseconds must be greater or equals to zero. If the > duration is negative the months, days and nanoseconds must be smaller or > equals to zero. > Currently, it is possible to send to C* a duration which does not respect > that rule and the data will not be reject. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-13143) Cassandra can accept invalid durations
Benjamin Lerer created CASSANDRA-13143: -- Summary: Cassandra can accept invalid durations Key: CASSANDRA-13143 URL: https://issues.apache.org/jira/browse/CASSANDRA-13143 Project: Cassandra Issue Type: Bug Reporter: Benjamin Lerer A duration can be positive or negative. If the duration is positive the months, days and nanoseconds must be greater or equals to zero. If the duration is negative the months, days and nanoseconds must be smaller or equals to zero. Currently, it is possible to send to C* a duration which does not respect that rule and the data will not be reject. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12850) Add the duration type to the protocol V5
[ https://issues.apache.org/jira/browse/CASSANDRA-12850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs updated CASSANDRA-12850: Reviewer: Tyler Hobbs > Add the duration type to the protocol V5 > > > Key: CASSANDRA-12850 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12850 > Project: Cassandra > Issue Type: Sub-task > Components: CQL >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer > Fix For: 3.x > > > The Duration type need to be added to the protocol specifications. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-12443) Remove alter type support
[ https://issues.apache.org/jira/browse/CASSANDRA-12443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer resolved CASSANDRA-12443. Resolution: Fixed Committed the followup patch into 3.0 at f3b452c54dea0366436ebe2bceac747bdf599e90 and merged into 3.11 and trunk > Remove alter type support > - > > Key: CASSANDRA-12443 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12443 > Project: Cassandra > Issue Type: Bug >Reporter: Carl Yeksigian >Assignee: Benjamin Lerer > Fix For: 3.0.11, 3.10 > > > Currently, we allow altering of types. However, because we no longer store > the length for all types anymore, switching from a fixed-width to > variable-width type causes issues. commitlog playback breaking startup, > queries currently in flight getting back bad results, and special casing > required to handle the changes. In addition, this would solve > CASSANDRA-10309, as there is no possibility of the types changing while an > SSTableReader is open. > For fixed-length, compatible types, the alter also doesn't add much over a > cast, so users could use that in order to retrieve the altered type. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-4663) Parallelize streaming of different keyspaces for bootstrap and rebuild
[ https://issues.apache.org/jira/browse/CASSANDRA-4663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Brown updated CASSANDRA-4663: --- Resolution: Fixed Assignee: Corentin Chary Fix Version/s: (was: 3.x) 4.0 Status: Resolved (was: Patch Available) > Parallelize streaming of different keyspaces for bootstrap and rebuild > -- > > Key: CASSANDRA-4663 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4663 > Project: Cassandra > Issue Type: Improvement >Reporter: sankalp kohli >Assignee: Corentin Chary >Priority: Minor > Fix For: 4.0 > > Attachments: > 0001-streaming-add-a-way-to-configure-the-number-of-conne.patch > > > This is not fast enough when someone is using SSD and may be 10G link. We > should try to create multiple connections and send multiple files in > parallel. > Current approach under utilize the link(even 1G). > This change will improve the bootstrapping time of a node. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-4663) Parallelize streaming of different keyspaces for bootstrap and rebuild
[ https://issues.apache.org/jira/browse/CASSANDRA-4663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832322#comment-15832322 ] Jason Brown commented on CASSANDRA-4663: committed as sha {{4d67639d38b3e3a6fd0a3487a99b9755abda469d}} to 4.0. Thanks! > Parallelize streaming of different keyspaces for bootstrap and rebuild > -- > > Key: CASSANDRA-4663 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4663 > Project: Cassandra > Issue Type: Improvement >Reporter: sankalp kohli >Priority: Minor > Fix For: 3.x > > Attachments: > 0001-streaming-add-a-way-to-configure-the-number-of-conne.patch > > > This is not fast enough when someone is using SSD and may be 10G link. We > should try to create multiple connections and send multiple files in > parallel. > Current approach under utilize the link(even 1G). > This change will improve the bootstrapping time of a node. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: Parallelize streaming of different keyspaces for bootstrap and rebuild
Repository: cassandra Updated Branches: refs/heads/trunk d3704d8a0 -> 4d67639d3 Parallelize streaming of different keyspaces for bootstrap and rebuild patch by Corentin Chary; reviewed by jasobrown for CASSANDRA-4663 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4d67639d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4d67639d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4d67639d Branch: refs/heads/trunk Commit: 4d67639d38b3e3a6fd0a3487a99b9755abda469d Parents: d3704d8 Author: Corentin Chary Authored: Wed Dec 7 11:11:06 2016 +0100 Committer: Jason Brown Committed: Fri Jan 20 11:47:18 2017 -0800 -- CHANGES.txt | 1 + conf/cassandra.yaml | 6 ++ src/java/org/apache/cassandra/config/Config.java | 1 + src/java/org/apache/cassandra/config/DatabaseDescriptor.java | 5 + src/java/org/apache/cassandra/dht/BootStrapper.java | 3 ++- src/java/org/apache/cassandra/dht/RangeStreamer.java | 7 +-- src/java/org/apache/cassandra/service/StorageService.java| 3 ++- 7 files changed, 22 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4d67639d/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index ebf161d..86ecbc4 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 4.0 + * Parallelize streaming of different keyspaces (4663) * Improved compactions metrics (CASSANDRA-13015) * Speed-up start-up sequence by avoiding un-needed flushes (CASSANDRA-13031) * Use Caffeine (W-TinyLFU) for on-heap caches (CASSANDRA-10855) http://git-wip-us.apache.org/repos/asf/cassandra/blob/4d67639d/conf/cassandra.yaml -- diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml index 4891706..e728796 100644 --- a/conf/cassandra.yaml +++ b/conf/cassandra.yaml @@ -817,6 +817,12 @@ cross_node_timeout: false # times out in 10 minutes by default # streaming_keep_alive_period_in_secs: 300 +# Limit number of connections per host for streaming +# Increase this when you notice that joins are CPU-bound rather that network +# bound (for example a few nodes with big files). +# streaming_connections_per_host: 1 + + # phi value that must be reached for a host to be marked down. # most users should never need to adjust this. # phi_convict_threshold: 8 http://git-wip-us.apache.org/repos/asf/cassandra/blob/4d67639d/src/java/org/apache/cassandra/config/Config.java -- diff --git a/src/java/org/apache/cassandra/config/Config.java b/src/java/org/apache/cassandra/config/Config.java index f5a8722..6fb999e 100644 --- a/src/java/org/apache/cassandra/config/Config.java +++ b/src/java/org/apache/cassandra/config/Config.java @@ -101,6 +101,7 @@ public class Config @Deprecated public int streaming_socket_timeout_in_ms = 8640; //24 hours +public Integer streaming_connections_per_host = 1; public Integer streaming_keep_alive_period_in_secs = 300; //5 minutes public boolean cross_node_timeout = false; http://git-wip-us.apache.org/repos/asf/cassandra/blob/4d67639d/src/java/org/apache/cassandra/config/DatabaseDescriptor.java -- diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java index c43672a..5aa7065 100644 --- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java +++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java @@ -2030,6 +2030,11 @@ public class DatabaseDescriptor return conf.streaming_keep_alive_period_in_secs; } +public static int getStreamingConnectionsPerHost() +{ +return conf.streaming_connections_per_host; +} + public static String getLocalDataCenter() { return localDC; http://git-wip-us.apache.org/repos/asf/cassandra/blob/4d67639d/src/java/org/apache/cassandra/dht/BootStrapper.java -- diff --git a/src/java/org/apache/cassandra/dht/BootStrapper.java b/src/java/org/apache/cassandra/dht/BootStrapper.java index 1e00f48..15e75fe 100644 --- a/src/java/org/apache/cassandra/dht/BootStrapper.java +++ b/src/java/org/apache/cassandra/dht/BootStrapper.java @@ -77,7 +77,8 @@ public class BootStrapper extends ProgressEventNotifierSupport useStrictConsistency, DatabaseDescriptor.getEndpointSnitch(),
[jira] [Updated] (CASSANDRA-4663) Parallelize streaming of different keyspaces for bootstrap and rebuild
[ https://issues.apache.org/jira/browse/CASSANDRA-4663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Brown updated CASSANDRA-4663: --- Summary: Parallelize streaming of different keyspaces for bootstrap and rebuild (was: Parallelize streaming of different keyspaces) > Parallelize streaming of different keyspaces for bootstrap and rebuild > -- > > Key: CASSANDRA-4663 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4663 > Project: Cassandra > Issue Type: Improvement >Reporter: sankalp kohli >Priority: Minor > Fix For: 3.x > > Attachments: > 0001-streaming-add-a-way-to-configure-the-number-of-conne.patch > > > This is not fast enough when someone is using SSD and may be 10G link. We > should try to create multiple connections and send multiple files in > parallel. > Current approach under utilize the link(even 1G). > This change will improve the bootstrapping time of a node. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-4663) Parallelize streaming of different keyspaces
[ https://issues.apache.org/jira/browse/CASSANDRA-4663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Brown updated CASSANDRA-4663: --- Summary: Parallelize streaming of different keyspaces (was: Streaming sends one file at a time serially. ) > Parallelize streaming of different keyspaces > > > Key: CASSANDRA-4663 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4663 > Project: Cassandra > Issue Type: Improvement >Reporter: sankalp kohli >Priority: Minor > Fix For: 3.x > > Attachments: > 0001-streaming-add-a-way-to-configure-the-number-of-conne.patch > > > This is not fast enough when someone is using SSD and may be 10G link. We > should try to create multiple connections and send multiple files in > parallel. > Current approach under utilize the link(even 1G). > This change will improve the bootstrapping time of a node. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-4663) Streaming sends one file at a time serially.
[ https://issues.apache.org/jira/browse/CASSANDRA-4663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832314#comment-15832314 ] Anubhav Kale commented on CASSANDRA-4663: - OOF 1/20 > Streaming sends one file at a time serially. > - > > Key: CASSANDRA-4663 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4663 > Project: Cassandra > Issue Type: Improvement >Reporter: sankalp kohli >Priority: Minor > Fix For: 3.x > > Attachments: > 0001-streaming-add-a-way-to-configure-the-number-of-conne.patch > > > This is not fast enough when someone is using SSD and may be 10G link. We > should try to create multiple connections and send multiple files in > parallel. > Current approach under utilize the link(even 1G). > This change will improve the bootstrapping time of a node. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-4663) Streaming sends one file at a time serially.
[ https://issues.apache.org/jira/browse/CASSANDRA-4663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832312#comment-15832312 ] Jason Brown commented on CASSANDRA-4663: OK, test look reasonable (no new dtest failures that are not currently failing on trunk). For the record, [~iksaif]'s patch parallelizes the streaming of the keyspaces being sent. If you have multiple keyspaces, you will potentially get an nice win. However, if there is only one keyspace (or most data is in one), there is no parallelization gained. I'll update the title of this jira to reflect that. As part of CASSANDRA-12229, or an immediate follow up, I'll be addressing sending sending files in parallel, so that request won't fall off the radar. +1'ing the ticket and will commit briefly > Streaming sends one file at a time serially. > - > > Key: CASSANDRA-4663 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4663 > Project: Cassandra > Issue Type: Improvement >Reporter: sankalp kohli >Priority: Minor > Fix For: 3.x > > Attachments: > 0001-streaming-add-a-way-to-configure-the-number-of-conne.patch > > > This is not fast enough when someone is using SSD and may be 10G link. We > should try to create multiple connections and send multiple files in > parallel. > Current approach under utilize the link(even 1G). > This change will improve the bootstrapping time of a node. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-12100) Compactions are stuck after TRUNCATE
[ https://issues.apache.org/jira/browse/CASSANDRA-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832261#comment-15832261 ] Sotirios Delimanolis edited comment on CASSANDRA-12100 at 1/20/17 7:13 PM: --- Any chance you can backport this to 2.2.9? (Referring to [this commit|https://github.com/apache/cassandra/commit/05483a962c64c350315fc738c697980b22361cc3].) was (Author: s_delima): Any chance you can backport this to 2.2.9? > Compactions are stuck after TRUNCATE > > > Key: CASSANDRA-12100 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12100 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Stefano Ortolani >Assignee: Stefania > Fix For: 3.0.9, 3.8 > > Attachments: node3_jstack.log > > > Hi, > since the upgrade to C* 3.0.7 I see compaction tasks getting stuck when > truncating the column family. I verified this on all nodes of the cluster. > Pending compactions seem to disappear after restarting the node. > {noformat} > root@node10:~# nodetool -h localhost compactionstats > pending tasks: 6 > id compaction type > keyspacetable completed totalunit progress >24e1ad30-3cac-11e6-870d-5de740693258Compaction > schema table_1 0 57558382 bytes 0.00% >2be2e3b0-3cac-11e6-870d-5de740693258Compaction > schema table_2 0 65063705 bytes 0.00% >54de38f0-3cac-11e6-870d-5de740693258Compaction > schema table_3 0 187031 bytes 0.00% >31926ce0-3cac-11e6-870d-5de740693258Compaction > schema table_4 0 42951119 bytes 0.00% >3911ad00-3cac-11e6-870d-5de740693258Compaction > schema table_5 0 25918949 bytes 0.00% >3e6a8ab0-3cac-11e6-870d-5de740693258Compaction > schema table_6 0 65466210 bytes 0.00% > Active compaction remaining time : 0h00m15s > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12100) Compactions are stuck after TRUNCATE
[ https://issues.apache.org/jira/browse/CASSANDRA-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832261#comment-15832261 ] Sotirios Delimanolis commented on CASSANDRA-12100: -- Any chance you can backport this to 2.2.9? > Compactions are stuck after TRUNCATE > > > Key: CASSANDRA-12100 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12100 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Stefano Ortolani >Assignee: Stefania > Fix For: 3.0.9, 3.8 > > Attachments: node3_jstack.log > > > Hi, > since the upgrade to C* 3.0.7 I see compaction tasks getting stuck when > truncating the column family. I verified this on all nodes of the cluster. > Pending compactions seem to disappear after restarting the node. > {noformat} > root@node10:~# nodetool -h localhost compactionstats > pending tasks: 6 > id compaction type > keyspacetable completed totalunit progress >24e1ad30-3cac-11e6-870d-5de740693258Compaction > schema table_1 0 57558382 bytes 0.00% >2be2e3b0-3cac-11e6-870d-5de740693258Compaction > schema table_2 0 65063705 bytes 0.00% >54de38f0-3cac-11e6-870d-5de740693258Compaction > schema table_3 0 187031 bytes 0.00% >31926ce0-3cac-11e6-870d-5de740693258Compaction > schema table_4 0 42951119 bytes 0.00% >3911ad00-3cac-11e6-870d-5de740693258Compaction > schema table_5 0 25918949 bytes 0.00% >3e6a8ab0-3cac-11e6-870d-5de740693258Compaction > schema table_6 0 65466210 bytes 0.00% > Active compaction remaining time : 0h00m15s > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13125) Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9
[ https://issues.apache.org/jira/browse/CASSANDRA-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832169#comment-15832169 ] Yasuharu Goto commented on CASSANDRA-13125: --- Thank you [~slebresne]! I checked that your patches worked properly in my reproduce procedure. The result is below. Now I could see that C*3.0 generated {{[c-c!][d-d!]}} style range tombstones and the rows are not broken! h4. On 13125-3.0 {noformat} cqlsh> insert into test.test(a,b,c,d,e) values(14,1,{2,3},{4,5},6); cqlsh> select * from test.test; a | b | c | d | e +---+++--- 14 | 1 | {2, 3} | {4, 5} | 6 RangeTombstone(0), start:org.apache.cassandra.db.composites.CompoundSparseCellName@78e3b54a, end:org.apache.cassandra.db.composites.BoundedComposite@6b517b57, markedAt:1484931972134334, delTime:1484931972 RangeTombstone(1), start:org.apache.cassandra.db.composites.CompoundSparseCellName@78e3b54b, end:org.apache.cassandra.db.composites.BoundedComposite@6b517b58, markedAt:1484931972134334, delTime:1484931972 DeletionInfo:{deletedAt=-9223372036854775808, localDeletion=2147483647, ranges=[c-c:!, deletedAt=1484931972134334, localDeletion=1484931972][d-d:!, deletedAt=1484931972134334, localDeletion=1484931972]} from:/127.0.0.1, payload:Mutation(keyspace='test', key='000e', modifications=[ColumnFamily(test -{deletedAt=-9223372036854775808, localDeletion=2147483647, ranges=[c-c:!, deletedAt=1484931972134334, localDeletion=1484931972][d-d:!, deletedAt=1484931972134334, localDeletion=1484931972]}- [:false:0@1484931972134335,b:false:4@1484931972134335,c:0002:false:0@1484931972134335,c:0003:false:0@1484931972134335,d:0004:false:0@1484931972134335,d:0005:false:0@1484931972134335,e:false:4@1484931972134335,])]), verb:MUTATION, version:8 {noformat} h4. On 13125-3.11 {noformat} cqlsh> insert into test.test(a,b,c,d,e) values(14,1,{2,3},{4,5},6); cqlsh> select * from test.test; a | b | c | d | e +---+++--- 14 | 1 | {2, 3} | {4, 5} | 6 Mutation.deserialize() size==1 RangeTombstone(0), start:org.apache.cassandra.db.composites.CompoundSparseCellName@4316af5d, end:org.apache.cassandra.db.composites.BoundedComposite@256e93b0, markedAt:1484933162431359, delTime:1484933162 RangeTombstone(1), start:org.apache.cassandra.db.composites.CompoundSparseCellName@4316af5e, end:org.apache.cassandra.db.composites.BoundedComposite@256e93b1, markedAt:1484933162431359, delTime:1484933162 DeletionInfo:{deletedAt=-9223372036854775808, localDeletion=2147483647, ranges=[c-c:!, deletedAt=1484933162431359, localDeletion=1484933162][d-d:!, deletedAt=1484933162431359, localDeletion=1484933162]} from:/127.0.0.1, payload:Mutation(keyspace='test', key='000e', modifications=[ColumnFamily(test -{deletedAt=-9223372036854775808, localDeletion=2147483647, ranges=[c-c:!, deletedAt=1484933162431359, localDeletion=1484933162][d-d:!, deletedAt=1484933162431359, localDeletion=1484933162]}- [:false:0@1484933162431360,b:false:4@1484933162431360,c:0002:false:0@1484933162431360,c:0003:false:0@1484933162431360,d:0004:false:0@1484933162431360,d:0005:false:0@1484933162431360,e:false:4@1484933162431360,])]), verb:MUTATION, version:8 {noformat} > Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9 > > > Key: CASSANDRA-13125 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13125 > Project: Cassandra > Issue Type: Bug >Reporter: Zhongxiang Zheng >Assignee: Yasuharu Goto >Priority: Critical > Attachments: diff-a.patch, diff-b.patch > > > I found that rows are splitting and duplicated after upgrading the cluster > from 2.1.x to 3.0.x. > I found the way to reproduce the problem as below. > {code} > $ ccm create test -v 2.1.16 -n 3 -s > > Current cluster is now: test > $ ccm node1 cqlsh -e "CREATE KEYSPACE test WITH replication = > {'class':'SimpleStrategy', 'replication_factor':3}" > $ ccm node1 cqlsh -e "CREATE TABLE test.test (id text PRIMARY KEY, value1 > set, value2 set);" > # Upgrade node1 > $ for i in 1; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm > node${i} start;ccm node${i} nodetool upgradesstables; done > # Insert a row through node1(3.0.10) > $ ccm node1 cqlsh -e "INSERT INTO test.test (id, value1, value2) values > ('aaa', {'aaa', 'bbb'}, {'ccc', 'ddd'});" > # Insert a row through node2(2.1.16) > $ ccm node2 cqlsh -e "INSERT INTO test.test (id, value1, value2) values > ('bbb', {'aaa', 'bbb'}, {'ccc', 'ddd'});" > # The row inserted from node1 is splitting > $ ccm node1 cqlsh -e "SELECT * FROM test.test ;" > id | value1 | value2 > -+-
[jira] [Comment Edited] (CASSANDRA-13142) Upgradesstables cancels compactions unnecessarily
[ https://issues.apache.org/jira/browse/CASSANDRA-13142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832087#comment-15832087 ] Tom van der Woerdt edited comment on CASSANDRA-13142 at 1/20/17 4:49 PM: - Cancelling view/index builds is not a "minor" issue. :) Granted, a restart should resume the view build, but that's not obvious to operators at all. was (Author: tvdw): Cancelling view/index builds is not a "minor" issue. :) > Upgradesstables cancels compactions unnecessarily > - > > Key: CASSANDRA-13142 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13142 > Project: Cassandra > Issue Type: Bug >Reporter: Kurt Greaves >Assignee: Kurt Greaves >Priority: Minor > > Since at least 1.2 upgradesstables will cancel any compactions bar > validations when run. This was originally determined as a non-issue in > CASSANDRA-3430 however can be quite annoying (especially with STCS) as a > compaction will output the new version anyway. Furthermore, as per > CASSANDRA-12243 it also stops things like view builds and I assume secondary > index builds as well which is not ideal. > We should avoid cancelling compactions unnecessarily. > I'd class this as a very minor bug, should at least be fixed from 3.0, but up > for debate. The main negative effect is annoying operators (which I tend to > rate pretty highly), but other than that there are no serious consequences. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13142) Upgradesstables cancels compactions unnecessarily
[ https://issues.apache.org/jira/browse/CASSANDRA-13142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832087#comment-15832087 ] Tom van der Woerdt commented on CASSANDRA-13142: Cancelling view/index builds is not a "minor" issue. :) > Upgradesstables cancels compactions unnecessarily > - > > Key: CASSANDRA-13142 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13142 > Project: Cassandra > Issue Type: Bug >Reporter: Kurt Greaves >Assignee: Kurt Greaves >Priority: Minor > > Since at least 1.2 upgradesstables will cancel any compactions bar > validations when run. This was originally determined as a non-issue in > CASSANDRA-3430 however can be quite annoying (especially with STCS) as a > compaction will output the new version anyway. Furthermore, as per > CASSANDRA-12243 it also stops things like view builds and I assume secondary > index builds as well which is not ideal. > We should avoid cancelling compactions unnecessarily. > I'd class this as a very minor bug, should at least be fixed from 3.0, but up > for debate. The main negative effect is annoying operators (which I tend to > rate pretty highly), but other than that there are no serious consequences. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13142) Upgradesstables cancels compactions unnecessarily
[ https://issues.apache.org/jira/browse/CASSANDRA-13142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832084#comment-15832084 ] Kurt Greaves commented on CASSANDRA-13142: -- I've already done all the necessary reconnaissance so will write a patch soon, I think best solution would be to provide a more modular way to specify compaction types to "ignore" in {{CompactionManager.interruptCompactionsForCFs()}}. > Upgradesstables cancels compactions unnecessarily > - > > Key: CASSANDRA-13142 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13142 > Project: Cassandra > Issue Type: Bug >Reporter: Kurt Greaves >Assignee: Kurt Greaves >Priority: Minor > > Since at least 1.2 upgradesstables will cancel any compactions bar > validations when run. This was originally determined as a non-issue in > CASSANDRA-3430 however can be quite annoying (especially with STCS) as a > compaction will output the new version anyway. Furthermore, as per > CASSANDRA-12243 it also stops things like view builds and I assume secondary > index builds as well which is not ideal. > We should avoid cancelling compactions unnecessarily. > I'd class this as a very minor bug, should at least be fixed from 3.0, but up > for debate. The main negative effect is annoying operators (which I tend to > rate pretty highly), but other than that there are no serious consequences. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-13142) Upgradesstables cancels compactions unnecessarily
[ https://issues.apache.org/jira/browse/CASSANDRA-13142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Greaves reassigned CASSANDRA-13142: Assignee: Kurt Greaves > Upgradesstables cancels compactions unnecessarily > - > > Key: CASSANDRA-13142 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13142 > Project: Cassandra > Issue Type: Bug >Reporter: Kurt Greaves >Assignee: Kurt Greaves >Priority: Minor > > Since at least 1.2 upgradesstables will cancel any compactions bar > validations when run. This was originally determined as a non-issue in > CASSANDRA-3430 however can be quite annoying (especially with STCS) as a > compaction will output the new version anyway. Furthermore, as per > CASSANDRA-12243 it also stops things like view builds and I assume secondary > index builds as well which is not ideal. > We should avoid cancelling compactions unnecessarily. > I'd class this as a very minor bug, should at least be fixed from 3.0, but up > for debate. The main negative effect is annoying operators (which I tend to > rate pretty highly), but other than that there are no serious consequences. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-12243) upgradesstables cancels view build
[ https://issues.apache.org/jira/browse/CASSANDRA-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Greaves resolved CASSANDRA-12243. -- Resolution: Duplicate Marking as duplicate in favor of CASSANDRA-13142 as it encapsulates this. > upgradesstables cancels view build > -- > > Key: CASSANDRA-12243 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12243 > Project: Cassandra > Issue Type: Bug >Reporter: Tom van der Woerdt > > It appears that running upgradesstables cancels view builds, even if there > are no sstables to upgrade : > {code} > $ nodetool compactionstats -H > pending tasks: 54 > id compaction type keyspace > table completedtotal unit progress >1cfee290-4dbe-11e6-b619-fb674e694d11View build mykeyspace > mytable 666 bytes 1015 bytes ranges 65.62% > Active compaction remaining time :n/a > $ nodetool upgradesstables > $ nodetool compactionstats -H > pending tasks: 53 > {code} > Easy to reproduce. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-13142) Upgradesstables cancels compactions unnecessarily
Kurt Greaves created CASSANDRA-13142: Summary: Upgradesstables cancels compactions unnecessarily Key: CASSANDRA-13142 URL: https://issues.apache.org/jira/browse/CASSANDRA-13142 Project: Cassandra Issue Type: Bug Reporter: Kurt Greaves Priority: Minor Since at least 1.2 upgradesstables will cancel any compactions bar validations when run. This was originally determined as a non-issue in CASSANDRA-3430 however can be quite annoying (especially with STCS) as a compaction will output the new version anyway. Furthermore, as per CASSANDRA-12243 it also stops things like view builds and I assume secondary index builds as well which is not ideal. We should avoid cancelling compactions unnecessarily. I'd class this as a very minor bug, should at least be fixed from 3.0, but up for debate. The main negative effect is annoying operators (which I tend to rate pretty highly), but other than that there are no serious consequences. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13141) test failure in cql_tests.MiscellaneousCQLTester.prepared_statement_invalidation_test
[ https://issues.apache.org/jira/browse/CASSANDRA-13141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean McCarthy updated CASSANDRA-13141: -- Component/s: Testing > test failure in > cql_tests.MiscellaneousCQLTester.prepared_statement_invalidation_test > - > > Key: CASSANDRA-13141 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13141 > Project: Cassandra > Issue Type: Bug > Components: Testing >Reporter: Sean McCarthy > Labels: dtest, test-failure > Attachments: node1_debug.log, node1_gc.log, node1.log > > > example failure: > http://cassci.datastax.com/job/trunk_dtest/1471/testReport/cql_tests/MiscellaneousCQLTester/prepared_statement_invalidation_test > {code} > Error Message > Error from server: code=2200 [Invalid query] message="Altering of types is > not allowed" > {code}{code} > Stacktrace > File "/usr/lib/python2.7/unittest/case.py", line 329, in run > testMethod() > File "/home/automaton/cassandra-dtest/cql_tests.py", line 633, in > prepared_statement_invalidation_test > session.execute("ALTER TABLE test ALTER d TYPE blob") > File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line > 1998, in execute > return self.execute_async(query, parameters, trace, custom_payload, > timeout, execution_profile, paging_state).result() > File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line > 3784, in result > raise self._final_exception > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-13141) test failure in cql_tests.MiscellaneousCQLTester.prepared_statement_invalidation_test
Sean McCarthy created CASSANDRA-13141: - Summary: test failure in cql_tests.MiscellaneousCQLTester.prepared_statement_invalidation_test Key: CASSANDRA-13141 URL: https://issues.apache.org/jira/browse/CASSANDRA-13141 Project: Cassandra Issue Type: Bug Reporter: Sean McCarthy Attachments: node1_debug.log, node1_gc.log, node1.log example failure: http://cassci.datastax.com/job/trunk_dtest/1471/testReport/cql_tests/MiscellaneousCQLTester/prepared_statement_invalidation_test {code} Error Message Error from server: code=2200 [Invalid query] message="Altering of types is not allowed" {code}{code} Stacktrace File "/usr/lib/python2.7/unittest/case.py", line 329, in run testMethod() File "/home/automaton/cassandra-dtest/cql_tests.py", line 633, in prepared_statement_invalidation_test session.execute("ALTER TABLE test ALTER d TYPE blob") File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 1998, in execute return self.execute_async(query, parameters, trace, custom_payload, timeout, execution_profile, paging_state).result() File "/home/automaton/src/cassandra-driver/cassandra/cluster.py", line 3784, in result raise self._final_exception {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-13140) test failure in cqlsh_tests.cqlsh_tests.CqlshSmokeTest.test_alter_table
Sean McCarthy created CASSANDRA-13140: - Summary: test failure in cqlsh_tests.cqlsh_tests.CqlshSmokeTest.test_alter_table Key: CASSANDRA-13140 URL: https://issues.apache.org/jira/browse/CASSANDRA-13140 Project: Cassandra Issue Type: Bug Reporter: Sean McCarthy Attachments: node1_debug.log, node1_gc.log, node1.log example failure: http://cassci.datastax.com/job/trunk_dtest/1471/testReport/cqlsh_tests.cqlsh_tests/CqlshSmokeTest/test_alter_table {code} Error Message [u'test', u'i', u'ascii'] unexpectedly found in [[u'test', u'key', u'text'], [u'test', u'i', u'ascii']] {code}{code} Stacktrace File "/usr/lib/python2.7/unittest/case.py", line 329, in run testMethod() File "/home/automaton/cassandra-dtest/cqlsh_tests/cqlsh_tests.py", line 1839, in test_alter_table self.assertNotIn(old_column_spec, new_columns) File "/usr/lib/python2.7/unittest/case.py", line 810, in assertNotIn self.fail(self._formatMessage(msg, standardMsg)) File "/usr/lib/python2.7/unittest/case.py", line 410, in fail raise self.failureException(msg) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-13139) test failure in hintedhandoff_test.TestHintedHandoff.hintedhandoff_decom_test
Sean McCarthy created CASSANDRA-13139: - Summary: test failure in hintedhandoff_test.TestHintedHandoff.hintedhandoff_decom_test Key: CASSANDRA-13139 URL: https://issues.apache.org/jira/browse/CASSANDRA-13139 Project: Cassandra Issue Type: Bug Components: Testing Reporter: Sean McCarthy Attachments: node1_debug.log, node1_gc.log, node1.log, node2_debug.log, node2_gc.log, node2.log, node3_debug.log, node3_gc.log, node3.log, node4_debug.log, node4_gc.log, node4.log example failure: http://cassci.datastax.com/job/trunk_novnode_dtest/503/testReport/hintedhandoff_test/TestHintedHandoff/hintedhandoff_decom_test {code} Error Message Subprocess ['nodetool', '-h', 'localhost', '-p', '7200', ['decommission']] exited with non-zero status; exit status: 1; stdout: nodetool: Unsupported operation: Not enough live nodes to maintain replication factor in keyspace system_distributed (RF = 3, N = 3). Perform a forceful decommission to ignore. See 'nodetool help' or 'nodetool help '. {code}{code} Stacktrace File "/usr/lib/python2.7/unittest/case.py", line 329, in run testMethod() File "/home/automaton/cassandra-dtest/hintedhandoff_test.py", line 169, in hintedhandoff_decom_test node2.decommission() File "/usr/local/lib/python2.7/dist-packages/ccmlib/node.py", line 1314, in decommission self.nodetool("decommission") File "/usr/local/lib/python2.7/dist-packages/ccmlib/node.py", line 783, in nodetool return handle_external_tool_process(p, ['nodetool', '-h', 'localhost', '-p', str(self.jmx_port), cmd.split()]) File "/usr/local/lib/python2.7/dist-packages/ccmlib/node.py", line 1993, in handle_external_tool_process raise ToolError(cmd_args, rc, out, err) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[1/2] cassandra git commit: Follow up 12443
Repository: cassandra Updated Branches: refs/heads/cassandra-3.11 e3d26b6aa -> 1a56dd063 Follow up 12443 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f3b452c5 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f3b452c5 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f3b452c5 Branch: refs/heads/cassandra-3.11 Commit: f3b452c54dea0366436ebe2bceac747bdf599e90 Parents: 7a06df7 Author: Benjamin Lerer Authored: Fri Jan 20 15:53:34 2017 +0100 Committer: Benjamin Lerer Committed: Fri Jan 20 15:53:34 2017 +0100 -- src/java/org/apache/cassandra/config/CFMetaData.java | 13 +++-- 1 file changed, 7 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f3b452c5/src/java/org/apache/cassandra/config/CFMetaData.java -- diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java b/src/java/org/apache/cassandra/config/CFMetaData.java index a3370dc..44f3a96 100644 --- a/src/java/org/apache/cassandra/config/CFMetaData.java +++ b/src/java/org/apache/cassandra/config/CFMetaData.java @@ -84,9 +84,9 @@ public final class CFMetaData private final boolean isView; private final boolean isIndex; -public final ClusteringComparator comparator; // bytes, long, timeuuid, utf8, etc. This is built directly from clusteringColumns +public volatile ClusteringComparator comparator; // bytes, long, timeuuid, utf8, etc. This is built directly from clusteringColumns public final IPartitioner partitioner;// partitioner the table uses -private final AbstractType keyValidator; +private volatile AbstractType keyValidator; private final Serializers serializers; @@ -285,10 +285,6 @@ public final class CFMetaData this.serializers = new Serializers(this); -this.comparator = new ClusteringComparator(extractTypes(clusteringColumns)); -List> keyTypes = extractTypes(partitionKeyColumns); -this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : CompositeType.getInstance(keyTypes); - rebuild(); } @@ -296,6 +292,8 @@ public final class CFMetaData // are kept because they are often useful in a different format. private void rebuild() { +this.comparator = new ClusteringComparator(extractTypes(clusteringColumns)); + Map newColumnMetadata = new HashMap<>(); for (ColumnDefinition def : partitionKeyColumns) newColumnMetadata.put(def.name.bytes, def); @@ -306,6 +304,9 @@ public final class CFMetaData this.columnMetadata = newColumnMetadata; +List> keyTypes = extractTypes(partitionKeyColumns); +this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : CompositeType.getInstance(keyTypes); + if (isCompactTable()) this.compactValueColumn = CompactTables.getCompactValueColumn(partitionColumns, isSuper()); }
[3/3] cassandra git commit: Merge branch cassandra-3.11 into trunk
Merge branch cassandra-3.11 into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d3704d8a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d3704d8a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d3704d8a Branch: refs/heads/trunk Commit: d3704d8a06927e234250463796fc515e2b14d2cb Parents: d6da7b7 1a56dd0 Author: Benjamin Lerer Authored: Fri Jan 20 16:03:30 2017 +0100 Committer: Benjamin Lerer Committed: Fri Jan 20 16:03:37 2017 +0100 -- src/java/org/apache/cassandra/config/CFMetaData.java | 13 +++-- 1 file changed, 7 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d3704d8a/src/java/org/apache/cassandra/config/CFMetaData.java -- diff --cc src/java/org/apache/cassandra/config/CFMetaData.java index 097f29d,d0932ed..c8180f3 --- a/src/java/org/apache/cassandra/config/CFMetaData.java +++ b/src/java/org/apache/cassandra/config/CFMetaData.java @@@ -89,10 -90,12 +89,10 @@@ public final class CFMetaDat private final boolean isView; private final boolean isIndex; - public final ClusteringComparator comparator; // bytes, long, timeuuid, utf8, etc. This is built directly from clusteringColumns + public volatile ClusteringComparator comparator; // bytes, long, timeuuid, utf8, etc. This is built directly from clusteringColumns public final IPartitioner partitioner;// partitioner the table uses - private final AbstractType keyValidator; + private volatile AbstractType keyValidator; -private final Serializers serializers; - // non-final, for now public volatile TableParams params = TableParams.DEFAULT; @@@ -315,8 -320,13 +313,11 @@@ this.columnMetadata = newColumnMetadata; + List> keyTypes = extractTypes(partitionKeyColumns); + this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : CompositeType.getInstance(keyTypes); + if (isCompactTable()) this.compactValueColumn = CompactTables.getCompactValueColumn(partitionColumns, isSuper()); - -this.allColumnFilter = ColumnFilter.all(this); } public Indexes getIndexes()
[1/3] cassandra git commit: Follow up 12443
Repository: cassandra Updated Branches: refs/heads/trunk d6da7b799 -> d3704d8a0 Follow up 12443 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f3b452c5 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f3b452c5 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f3b452c5 Branch: refs/heads/trunk Commit: f3b452c54dea0366436ebe2bceac747bdf599e90 Parents: 7a06df7 Author: Benjamin Lerer Authored: Fri Jan 20 15:53:34 2017 +0100 Committer: Benjamin Lerer Committed: Fri Jan 20 15:53:34 2017 +0100 -- src/java/org/apache/cassandra/config/CFMetaData.java | 13 +++-- 1 file changed, 7 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f3b452c5/src/java/org/apache/cassandra/config/CFMetaData.java -- diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java b/src/java/org/apache/cassandra/config/CFMetaData.java index a3370dc..44f3a96 100644 --- a/src/java/org/apache/cassandra/config/CFMetaData.java +++ b/src/java/org/apache/cassandra/config/CFMetaData.java @@ -84,9 +84,9 @@ public final class CFMetaData private final boolean isView; private final boolean isIndex; -public final ClusteringComparator comparator; // bytes, long, timeuuid, utf8, etc. This is built directly from clusteringColumns +public volatile ClusteringComparator comparator; // bytes, long, timeuuid, utf8, etc. This is built directly from clusteringColumns public final IPartitioner partitioner;// partitioner the table uses -private final AbstractType keyValidator; +private volatile AbstractType keyValidator; private final Serializers serializers; @@ -285,10 +285,6 @@ public final class CFMetaData this.serializers = new Serializers(this); -this.comparator = new ClusteringComparator(extractTypes(clusteringColumns)); -List> keyTypes = extractTypes(partitionKeyColumns); -this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : CompositeType.getInstance(keyTypes); - rebuild(); } @@ -296,6 +292,8 @@ public final class CFMetaData // are kept because they are often useful in a different format. private void rebuild() { +this.comparator = new ClusteringComparator(extractTypes(clusteringColumns)); + Map newColumnMetadata = new HashMap<>(); for (ColumnDefinition def : partitionKeyColumns) newColumnMetadata.put(def.name.bytes, def); @@ -306,6 +304,9 @@ public final class CFMetaData this.columnMetadata = newColumnMetadata; +List> keyTypes = extractTypes(partitionKeyColumns); +this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : CompositeType.getInstance(keyTypes); + if (isCompactTable()) this.compactValueColumn = CompactTables.getCompactValueColumn(partitionColumns, isSuper()); }
[jira] [Commented] (CASSANDRA-13123) Draining a node might fail to delete all inactive commitlogs
[ https://issues.apache.org/jira/browse/CASSANDRA-13123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831886#comment-15831886 ] Jason Brown commented on CASSANDRA-13123: - Pushed code for cassci to run tests. I think the change to {{CommitLogSegmentManager}} is probably legit, but I want to look at the test a little more before +1'ing it ||3.0||3.11||trunk|| |[branch|https://github.com/jasobrown/cassandra/tree/13123-3.0]|[branch|https://github.com/jasobrown/cassandra/tree/13123-3.11]|[branch|https://github.com/jasobrown/cassandra/tree/13123-trunk]| |[dtest|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-13123-3.0-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-13123-3.11-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-13123-trunk-dtest/]| |[testall|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-13123-3.0-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-13123-3.11-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-13123-trunk-testall/]| > Draining a node might fail to delete all inactive commitlogs > > > Key: CASSANDRA-13123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13123 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths >Reporter: Jan Urbański >Assignee: Jan Urbański > Fix For: 3.8 > > Attachments: 13123-2.2.8.txt, 13123-3.0.10.txt, 13123-3.9.txt, > 13123-trunk.txt > > > After issuing a drain command, it's possible that not all of the inactive > commitlogs are removed. > The drain command shuts down the CommitLog instance, which in turn shuts down > the CommitLogSegmentManager. This has the effect of discarding any pending > management tasks it might have, like the removal of inactive commitlogs. > This in turn leads to an excessive amount of commitlogs being left behind > after a drain and a lengthy recovery after a restart. With a fleet of dozens > of nodes, each of them leaving several GB of commitlogs after a drain and > taking up to two minutes to recover them on restart, the additional time > required to restart the entire fleet becomes noticeable. > This problem is not present in 3.x or trunk because of the CLSM rewrite done > in CASSANDRA-8844. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[2/3] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.11
Merge branch cassandra-3.0 into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1a56dd06 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1a56dd06 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1a56dd06 Branch: refs/heads/trunk Commit: 1a56dd063d8d032c33427013ff0e990f4e5b4760 Parents: e3d26b6 f3b452c Author: Benjamin Lerer Authored: Fri Jan 20 16:02:27 2017 +0100 Committer: Benjamin Lerer Committed: Fri Jan 20 16:02:27 2017 +0100 -- src/java/org/apache/cassandra/config/CFMetaData.java | 13 +++-- 1 file changed, 7 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a56dd06/src/java/org/apache/cassandra/config/CFMetaData.java -- diff --cc src/java/org/apache/cassandra/config/CFMetaData.java index 1d3bb3a,44f3a96..d0932ed --- a/src/java/org/apache/cassandra/config/CFMetaData.java +++ b/src/java/org/apache/cassandra/config/CFMetaData.java @@@ -297,22 -283,18 +297,20 @@@ public final class CFMetaDat this.clusteringColumns = clusteringColumns; this.partitionColumns = partitionColumns; - this.comparator = new ClusteringComparator(extractTypes(clusteringColumns)); - List> keyTypes = extractTypes(partitionKeyColumns); - this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : CompositeType.getInstance(keyTypes); -this.serializers = new Serializers(this); -- rebuild(); + +this.resource = DataResource.table(ksName, cfName); +this.serializers = new Serializers(this); } // This rebuild informations that are intrinsically duplicate of the table definition but // are kept because they are often useful in a different format. private void rebuild() { + this.comparator = new ClusteringComparator(extractTypes(clusteringColumns)); + -Map newColumnMetadata = new HashMap<>(); +Map newColumnMetadata = Maps.newHashMapWithExpectedSize(partitionKeyColumns.size() + clusteringColumns.size() + partitionColumns.size()); + for (ColumnDefinition def : partitionKeyColumns) newColumnMetadata.put(def.name.bytes, def); for (ColumnDefinition def : clusteringColumns) @@@ -322,10 -304,11 +320,13 @@@ this.columnMetadata = newColumnMetadata; + List> keyTypes = extractTypes(partitionKeyColumns); + this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : CompositeType.getInstance(keyTypes); + if (isCompactTable()) this.compactValueColumn = CompactTables.getCompactValueColumn(partitionColumns, isSuper()); + +this.allColumnFilter = ColumnFilter.all(this); } public Indexes getIndexes()
[2/2] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.11
Merge branch cassandra-3.0 into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1a56dd06 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1a56dd06 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1a56dd06 Branch: refs/heads/cassandra-3.11 Commit: 1a56dd063d8d032c33427013ff0e990f4e5b4760 Parents: e3d26b6 f3b452c Author: Benjamin Lerer Authored: Fri Jan 20 16:02:27 2017 +0100 Committer: Benjamin Lerer Committed: Fri Jan 20 16:02:27 2017 +0100 -- src/java/org/apache/cassandra/config/CFMetaData.java | 13 +++-- 1 file changed, 7 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/1a56dd06/src/java/org/apache/cassandra/config/CFMetaData.java -- diff --cc src/java/org/apache/cassandra/config/CFMetaData.java index 1d3bb3a,44f3a96..d0932ed --- a/src/java/org/apache/cassandra/config/CFMetaData.java +++ b/src/java/org/apache/cassandra/config/CFMetaData.java @@@ -297,22 -283,18 +297,20 @@@ public final class CFMetaDat this.clusteringColumns = clusteringColumns; this.partitionColumns = partitionColumns; - this.comparator = new ClusteringComparator(extractTypes(clusteringColumns)); - List> keyTypes = extractTypes(partitionKeyColumns); - this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : CompositeType.getInstance(keyTypes); -this.serializers = new Serializers(this); -- rebuild(); + +this.resource = DataResource.table(ksName, cfName); +this.serializers = new Serializers(this); } // This rebuild informations that are intrinsically duplicate of the table definition but // are kept because they are often useful in a different format. private void rebuild() { + this.comparator = new ClusteringComparator(extractTypes(clusteringColumns)); + -Map newColumnMetadata = new HashMap<>(); +Map newColumnMetadata = Maps.newHashMapWithExpectedSize(partitionKeyColumns.size() + clusteringColumns.size() + partitionColumns.size()); + for (ColumnDefinition def : partitionKeyColumns) newColumnMetadata.put(def.name.bytes, def); for (ColumnDefinition def : clusteringColumns) @@@ -322,10 -304,11 +320,13 @@@ this.columnMetadata = newColumnMetadata; + List> keyTypes = extractTypes(partitionKeyColumns); + this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : CompositeType.getInstance(keyTypes); + if (isCompactTable()) this.compactValueColumn = CompactTables.getCompactValueColumn(partitionColumns, isSuper()); + +this.allColumnFilter = ColumnFilter.all(this); } public Indexes getIndexes()
cassandra git commit: Follow up 12443
Repository: cassandra Updated Branches: refs/heads/cassandra-3.0 7a06df79d -> f3b452c54 Follow up 12443 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f3b452c5 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f3b452c5 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f3b452c5 Branch: refs/heads/cassandra-3.0 Commit: f3b452c54dea0366436ebe2bceac747bdf599e90 Parents: 7a06df7 Author: Benjamin Lerer Authored: Fri Jan 20 15:53:34 2017 +0100 Committer: Benjamin Lerer Committed: Fri Jan 20 15:53:34 2017 +0100 -- src/java/org/apache/cassandra/config/CFMetaData.java | 13 +++-- 1 file changed, 7 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f3b452c5/src/java/org/apache/cassandra/config/CFMetaData.java -- diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java b/src/java/org/apache/cassandra/config/CFMetaData.java index a3370dc..44f3a96 100644 --- a/src/java/org/apache/cassandra/config/CFMetaData.java +++ b/src/java/org/apache/cassandra/config/CFMetaData.java @@ -84,9 +84,9 @@ public final class CFMetaData private final boolean isView; private final boolean isIndex; -public final ClusteringComparator comparator; // bytes, long, timeuuid, utf8, etc. This is built directly from clusteringColumns +public volatile ClusteringComparator comparator; // bytes, long, timeuuid, utf8, etc. This is built directly from clusteringColumns public final IPartitioner partitioner;// partitioner the table uses -private final AbstractType keyValidator; +private volatile AbstractType keyValidator; private final Serializers serializers; @@ -285,10 +285,6 @@ public final class CFMetaData this.serializers = new Serializers(this); -this.comparator = new ClusteringComparator(extractTypes(clusteringColumns)); -List> keyTypes = extractTypes(partitionKeyColumns); -this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : CompositeType.getInstance(keyTypes); - rebuild(); } @@ -296,6 +292,8 @@ public final class CFMetaData // are kept because they are often useful in a different format. private void rebuild() { +this.comparator = new ClusteringComparator(extractTypes(clusteringColumns)); + Map newColumnMetadata = new HashMap<>(); for (ColumnDefinition def : partitionKeyColumns) newColumnMetadata.put(def.name.bytes, def); @@ -306,6 +304,9 @@ public final class CFMetaData this.columnMetadata = newColumnMetadata; +List> keyTypes = extractTypes(partitionKeyColumns); +this.keyValidator = keyTypes.size() == 1 ? keyTypes.get(0) : CompositeType.getInstance(keyTypes); + if (isCompactTable()) this.compactValueColumn = CompactTables.getCompactValueColumn(partitionColumns, isSuper()); }
[jira] [Commented] (CASSANDRA-12443) Remove alter type support
[ https://issues.apache.org/jira/browse/CASSANDRA-12443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831854#comment-15831854 ] Aleksey Yeschenko commented on CASSANDRA-12443: --- Had a look offline already. +1 > Remove alter type support > - > > Key: CASSANDRA-12443 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12443 > Project: Cassandra > Issue Type: Bug >Reporter: Carl Yeksigian >Assignee: Benjamin Lerer > Fix For: 3.0.11, 3.10 > > > Currently, we allow altering of types. However, because we no longer store > the length for all types anymore, switching from a fixed-width to > variable-width type causes issues. commitlog playback breaking startup, > queries currently in flight getting back bad results, and special casing > required to handle the changes. In addition, this would solve > CASSANDRA-10309, as there is no possibility of the types changing while an > SSTableReader is open. > For fixed-length, compatible types, the alter also doesn't add much over a > cast, so users could use that in order to retrieve the altered type. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12443) Remove alter type support
[ https://issues.apache.org/jira/browse/CASSANDRA-12443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831853#comment-15831853 ] Benjamin Lerer commented on CASSANDRA-12443: The followup patch is [here|https://github.com/apache/cassandra/compare/trunk...blerer:12443-3.0-followup]. > Remove alter type support > - > > Key: CASSANDRA-12443 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12443 > Project: Cassandra > Issue Type: Bug >Reporter: Carl Yeksigian >Assignee: Benjamin Lerer > Fix For: 3.0.11, 3.10 > > > Currently, we allow altering of types. However, because we no longer store > the length for all types anymore, switching from a fixed-width to > variable-width type causes issues. commitlog playback breaking startup, > queries currently in flight getting back bad results, and special casing > required to handle the changes. In addition, this would solve > CASSANDRA-10309, as there is no possibility of the types changing while an > SSTableReader is open. > For fixed-length, compatible types, the alter also doesn't add much over a > cast, so users could use that in order to retrieve the altered type. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13125) Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9
[ https://issues.apache.org/jira/browse/CASSANDRA-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831816#comment-15831816 ] Sylvain Lebresne commented on CASSANDRA-13125: -- Thanks a lot for the very detailed reproduction case and analysis. You are absolutely right that it's abnormal for 3.0 to transform a range like {{\[c-c!\]\[d-d!\]}} into {{\[c-c!\]\[c-d!\]}}. And having duplicate row entries is only one of the problem this triggers, as with more collections you could this ending up to delete data that it shouldn't. In any case, the problem is definitively in 3.0 generating this in the first place, so that's what we should fix, so fixing this in LegacyRangeTombstone constructor (Plan-A above) is too late. So the problem is when building the {{LegacyRangeTombstonList}} on 3.0, though it's not in the {{insertFrom}} method and the Plan-B patch would likely create other problems. The problem is in the {{LegacyBoundComparator}}: it delegates to {{clusteringComparator.compare()}} before it even checks if the bounds it compares have a {{collectionName}}, but that's incorrect. Anyway, attaching below a fix for this as well as a simple unit test showing where the problem lies: | [13125-3.0|https://github.com/pcmanus/cassandra/commits/13125-3.0] | [utests|http://cassci.datastax.com/job/pcmanus-13125-3.0-testall] | [dtests|http://cassci.datastax.com/job/pcmanus-13125-3.0-dtest] | | [13125-3.11|https://github.com/pcmanus/cassandra/commits/13125-3.11] | [utests|http://cassci.datastax.com/job/pcmanus-13125-3.11-testall] | [dtests|http://cassci.datastax.com/job/pcmanus-13125-3.11-dtest] | [~Yasuharu], if you could check if this patch does properly work in your case, that would be much appreciated. [~thobbs], since you're familiar with 3.0 compatibility code, would you also mind having a quick look at the patch? > Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9 > > > Key: CASSANDRA-13125 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13125 > Project: Cassandra > Issue Type: Bug >Reporter: Zhongxiang Zheng >Assignee: Yasuharu Goto >Priority: Critical > Attachments: diff-a.patch, diff-b.patch > > > I found that rows are splitting and duplicated after upgrading the cluster > from 2.1.x to 3.0.x. > I found the way to reproduce the problem as below. > {code} > $ ccm create test -v 2.1.16 -n 3 -s > > Current cluster is now: test > $ ccm node1 cqlsh -e "CREATE KEYSPACE test WITH replication = > {'class':'SimpleStrategy', 'replication_factor':3}" > $ ccm node1 cqlsh -e "CREATE TABLE test.test (id text PRIMARY KEY, value1 > set, value2 set);" > # Upgrade node1 > $ for i in 1; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm > node${i} start;ccm node${i} nodetool upgradesstables; done > # Insert a row through node1(3.0.10) > $ ccm node1 cqlsh -e "INSERT INTO test.test (id, value1, value2) values > ('aaa', {'aaa', 'bbb'}, {'ccc', 'ddd'});" > # Insert a row through node2(2.1.16) > $ ccm node2 cqlsh -e "INSERT INTO test.test (id, value1, value2) values > ('bbb', {'aaa', 'bbb'}, {'ccc', 'ddd'});" > # The row inserted from node1 is splitting > $ ccm node1 cqlsh -e "SELECT * FROM test.test ;" > id | value1 | value2 > -++ > aaa | null | null > aaa | {'aaa', 'bbb'} | {'ccc', 'ddd'} > bbb | {'aaa', 'bbb'} | {'ccc', 'ddd'} > $ for i in 1 2; do ccm node${i} nodetool flush; done > # Results of sstable2json of node2. The row inserted from node1(3.0.10) is > different from the row inserted from node2(2.1.16). > $ ccm node2 json -k test -c test > running > ['/home/zzheng/.ccm/test/node2/data0/test/test-5406ee80dbdb11e6a175f57c4c7c85f3/test-test-ka-1-Data.db'] > -- test-test-ka-1-Data.db - > [ > {"key": "aaa", > "cells": [["","",1484564624769577], >["value1","value2:!",1484564624769576,"t",1484564624], >["value1:616161","",1484564624769577], >["value1:626262","",1484564624769577], >["value2:636363","",1484564624769577], >["value2:646464","",1484564624769577]]}, > {"key": "bbb", > "cells": [["","",1484564634508029], >["value1:_","value1:!",1484564634508028,"t",1484564634], >["value1:616161","",1484564634508029], >["value1:626262","",1484564634508029], >["value2:_","value2:!",1484564634508028,"t",1484564634], >["value2:636363","",1484564634508029], >["value2:646464","",1484564634508029]]} > ] > # Upgrade node2,3 > $ for i in `seq 2 3`; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm > node${i} start;ccm node${i} nodetool upgrades
[jira] [Updated] (CASSANDRA-13125) Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9
[ https://issues.apache.org/jira/browse/CASSANDRA-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-13125: - Priority: Critical (was: Major) > Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9 > > > Key: CASSANDRA-13125 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13125 > Project: Cassandra > Issue Type: Bug >Reporter: Zhongxiang Zheng >Assignee: Yasuharu Goto >Priority: Critical > Attachments: diff-a.patch, diff-b.patch > > > I found that rows are splitting and duplicated after upgrading the cluster > from 2.1.x to 3.0.x. > I found the way to reproduce the problem as below. > {code} > $ ccm create test -v 2.1.16 -n 3 -s > > Current cluster is now: test > $ ccm node1 cqlsh -e "CREATE KEYSPACE test WITH replication = > {'class':'SimpleStrategy', 'replication_factor':3}" > $ ccm node1 cqlsh -e "CREATE TABLE test.test (id text PRIMARY KEY, value1 > set, value2 set);" > # Upgrade node1 > $ for i in 1; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm > node${i} start;ccm node${i} nodetool upgradesstables; done > # Insert a row through node1(3.0.10) > $ ccm node1 cqlsh -e "INSERT INTO test.test (id, value1, value2) values > ('aaa', {'aaa', 'bbb'}, {'ccc', 'ddd'});" > # Insert a row through node2(2.1.16) > $ ccm node2 cqlsh -e "INSERT INTO test.test (id, value1, value2) values > ('bbb', {'aaa', 'bbb'}, {'ccc', 'ddd'});" > # The row inserted from node1 is splitting > $ ccm node1 cqlsh -e "SELECT * FROM test.test ;" > id | value1 | value2 > -++ > aaa | null | null > aaa | {'aaa', 'bbb'} | {'ccc', 'ddd'} > bbb | {'aaa', 'bbb'} | {'ccc', 'ddd'} > $ for i in 1 2; do ccm node${i} nodetool flush; done > # Results of sstable2json of node2. The row inserted from node1(3.0.10) is > different from the row inserted from node2(2.1.16). > $ ccm node2 json -k test -c test > running > ['/home/zzheng/.ccm/test/node2/data0/test/test-5406ee80dbdb11e6a175f57c4c7c85f3/test-test-ka-1-Data.db'] > -- test-test-ka-1-Data.db - > [ > {"key": "aaa", > "cells": [["","",1484564624769577], >["value1","value2:!",1484564624769576,"t",1484564624], >["value1:616161","",1484564624769577], >["value1:626262","",1484564624769577], >["value2:636363","",1484564624769577], >["value2:646464","",1484564624769577]]}, > {"key": "bbb", > "cells": [["","",1484564634508029], >["value1:_","value1:!",1484564634508028,"t",1484564634], >["value1:616161","",1484564634508029], >["value1:626262","",1484564634508029], >["value2:_","value2:!",1484564634508028,"t",1484564634], >["value2:636363","",1484564634508029], >["value2:646464","",1484564634508029]]} > ] > # Upgrade node2,3 > $ for i in `seq 2 3`; do ccm node${i} stop; ccm node${i} setdir -v3.0.10; ccm > node${i} start;ccm node${i} nodetool upgradesstables; done > # After upgrade node2,3, the row inserted from node1 is splitting in node2,3 > $ ccm node2 cqlsh -e "SELECT * FROM test.test ;" > > id | value1 | value2 > -++ > aaa | null | null > aaa | {'aaa', 'bbb'} | {'ccc', 'ddd'} > bbb | {'aaa', 'bbb'} | {'ccc', 'ddd'} > (3 rows) > # Results of sstabledump > # node1 > [ > { > "partition" : { > "key" : [ "aaa" ], > "position" : 0 > }, > "rows" : [ > { > "type" : "row", > "position" : 17, > "liveness_info" : { "tstamp" : "2017-01-16T11:03:44.769577Z" }, > "cells" : [ > { "name" : "value1", "deletion_info" : { "marked_deleted" : > "2017-01-16T11:03:44.769576Z", "local_delete_time" : "2017-01-16T11:03:44Z" } > }, > { "name" : "value1", "path" : [ "aaa" ], "value" : "" }, > { "name" : "value1", "path" : [ "bbb" ], "value" : "" }, > { "name" : "value2", "deletion_info" : { "marked_deleted" : > "2017-01-16T11:03:44.769576Z", "local_delete_time" : "2017-01-16T11:03:44Z" } > }, > { "name" : "value2", "path" : [ "ccc" ], "value" : "" }, > { "name" : "value2", "path" : [ "ddd" ], "value" : "" } > ] > } > ] > }, > { > "partition" : { > "key" : [ "bbb" ], > "position" : 48 > }, > "rows" : [ > { > "type" : "row", > "position" : 65, > "liveness_info" : { "tstamp" : "2017-01-16T11:03:54.508029Z" }, > "cells" : [ > { "name" : "value1", "deletion_info" : { "mar
[jira] [Commented] (CASSANDRA-4663) Streaming sends one file at a time serially.
[ https://issues.apache.org/jira/browse/CASSANDRA-4663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831782#comment-15831782 ] Jason Brown commented on CASSANDRA-4663: Yeah, I think you are correct here. I didn't look that far up the stack (into {{RangeStreamer}}), but that will surely parallelize transfers by keyspace. I've kicked off the tests here: ||trunk|| |[branch|https://github.com/jasobrown/cassandra/tree/4663-trunk]| |[dtest|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-4663-trunk-dtest/]| |[testall|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-4663-trunk-testall/]| If everything looks good with the tests, I'll rename the ticket to something more appropriate and commit. > Streaming sends one file at a time serially. > - > > Key: CASSANDRA-4663 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4663 > Project: Cassandra > Issue Type: Improvement >Reporter: sankalp kohli >Priority: Minor > Fix For: 3.x > > Attachments: > 0001-streaming-add-a-way-to-configure-the-number-of-conne.patch > > > This is not fast enough when someone is using SSD and may be 10G link. We > should try to create multiple connections and send multiple files in > parallel. > Current approach under utilize the link(even 1G). > This change will improve the bootstrapping time of a node. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12202) LongLeveledCompactionStrategyTest flapping in 2.1, 2.2, 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-12202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831737#comment-15831737 ] Jason Brown commented on CASSANDRA-12202: - [~yukim] Did this ever get committed? > LongLeveledCompactionStrategyTest flapping in 2.1, 2.2, 3.0 > --- > > Key: CASSANDRA-12202 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12202 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x > > > We actually fixed this for 3.7+ in CASSANDRA-11657, need to backport that fix > to 2.1+ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-13103) incorrect jvm metric names
[ https://issues.apache.org/jira/browse/CASSANDRA-13103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Brown updated CASSANDRA-13103: Assignee: Alain Rastoul > incorrect jvm metric names > -- > > Key: CASSANDRA-13103 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13103 > Project: Cassandra > Issue Type: Bug > Components: Observability >Reporter: Alain Rastoul >Assignee: Alain Rastoul >Priority: Minor > Labels: lhf > Fix For: 3.9, 3.x > > Attachments: Fix-incorrect-jvm-metric-names-CASSANDRA-13103.patch > > > Some jvm metrics have a double dot in name like: > jvm.memory..total.max , jvm.memory..total.init (etc). > it seems that an extra dot is added at the end of the name in > CassandraDaemon.java, around line 367 (in 3.0.10): > ... > // enable metrics provided by metrics-jvm.jar > CassandraMetricsRegistry.Metrics.register("jvm.buffers.", new > BufferPoolMetricSet(ManagementFactory.getPlatformMBeanServer())); > CassandraMetricsRegistry.Metrics.register("jvm.gc.", new > GarbageCollectorMetricSet()); > CassandraMetricsRegistry.Metrics.register("jvm.memory.", new > MemoryUsageGaugeSet()); > and also added in append method of MetricRegistry. > Call stack is: > MetricRegistry>>registerAll(String prefix, MetricSet metrics) > MetricRegistry>>static String name(String name, String... names) > MetricRegistry>>static void append(StringBuilder builder, String part) > and in append the dot is also added: > ... > if(builder.length() > 0) { > builder.append('.'); > } > builder.append(part); > ... > The codahale MetricRegistry class seems to have no recent modification of > name or append methods, so it look like a small bug. > May be the fix could be to simply not to add the final dot in the metric > name, ie "jvm.buffers" instead of "jvm.buffers." -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12850) Add the duration type to the protocol V5
[ https://issues.apache.org/jira/browse/CASSANDRA-12850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831687#comment-15831687 ] Benjamin Lerer commented on CASSANDRA-12850: [~thobbs] could you review? > Add the duration type to the protocol V5 > > > Key: CASSANDRA-12850 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12850 > Project: Cassandra > Issue Type: Sub-task > Components: CQL >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer > Fix For: 3.x > > > The Duration type need to be added to the protocol specifications. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12850) Add the duration type to the protocol V5
[ https://issues.apache.org/jira/browse/CASSANDRA-12850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-12850: --- Status: Patch Available (was: In Progress) > Add the duration type to the protocol V5 > > > Key: CASSANDRA-12850 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12850 > Project: Cassandra > Issue Type: Sub-task > Components: CQL >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer > Fix For: 3.x > > > The Duration type need to be added to the protocol specifications. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12850) Add the duration type to the protocol V5
[ https://issues.apache.org/jira/browse/CASSANDRA-12850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831685#comment-15831685 ] Benjamin Lerer commented on CASSANDRA-12850: ||[3.11|https://github.com/apache/cassandra/compare/trunk...blerer:12850-3.11]|[utest|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-12850-3.11-testall/]|[dtest|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-12850-3.11-dtest/]| ||[trunk|https://github.com/apache/cassandra/compare/trunk...blerer:12850-trunk]|[utest|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-12850-trunk-testall/]|[dtest|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-12850-trunk-dtest/]| The patch add {{Duration}} to the protocol specification and to the {{DataType}}. While running the tests, I faced an {{IllegalStateException}} in {{OptionCodec}} caused by the fact that 2 {{DataType}}s where having the same id (the one of {{CUSTOM}}. As {{OptionCodec}} was only used by {{DataType}} I simplified it and converted it into an inner class. That change helped to fix the problem as the {{protocolVersion}} of the {{DataType}} was needed. > Add the duration type to the protocol V5 > > > Key: CASSANDRA-12850 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12850 > Project: Cassandra > Issue Type: Sub-task > Components: CQL >Reporter: Benjamin Lerer >Assignee: Benjamin Lerer > Fix For: 3.x > > > The Duration type need to be added to the protocol specifications. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-13138) SASI tries to fetch an extra page when resultset size is same size as page size
Alex Petrov created CASSANDRA-13138: --- Summary: SASI tries to fetch an extra page when resultset size is same size as page size Key: CASSANDRA-13138 URL: https://issues.apache.org/jira/browse/CASSANDRA-13138 Project: Cassandra Issue Type: Bug Reporter: Alex Petrov For example, in a dataset that would return 10 rows, SASI would try (and return an empty page) to fetch the next page, while filtering and 2i will return results correctly: {code} pk | ck1 | ck2 | reg1 | reg2 | reg3 +-+-+--+--+-- 6 | 5 | 5 |5 |5 | 10 7 | 5 | 5 |5 |5 | 10 9 | 5 | 5 |5 |5 | 10 4 | 5 | 5 |5 |5 | 10 3 | 5 | 5 |5 |5 | 10 5 | 5 | 5 |5 |5 | 10 0 | 5 | 5 |5 |5 | 10 8 | 5 | 5 |5 |5 | 10 2 | 5 | 5 |5 |5 | 10 1 | 5 | 5 |5 |5 | 10 ---MORE--- (10 rows) {code} (that {{--MORE--}} shouldn't have been there) This might be an inherent limitation, although even if it is we can opt out for fetching limit+1 if the data limits aren't exhausted. Although it seems that there should be a solution for it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)