[jira] [Commented] (CASSANDRA-11517) o.a.c.utils.UUIDGen could handle contention better
[ https://issues.apache.org/jira/browse/CASSANDRA-11517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273683#comment-15273683 ] Joel Knighton commented on CASSANDRA-11517: --- Implementation after Sylvain's points looks good to me. The test problem with lastTimestamp never being updated didn't seem to be fixed - lastTimestamp was always zero, so we were testing only that the timestamp of a UUID was greater than 0. I've updated the test to fix this such that the condition asserts that UUID timestamps are strictly increasing within each test thread. I also slightly decreased the test iterations (runtime was reaching 6s on my dev machines, which seemed a bit long), rebased on master, and pushed for updated CI. ||branch||testall||dtest|| |[CASSANDRA-11517-trunk|https://github.com/jkni/cassandra/tree/CASSANDRA-11517-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/jkni/job/jkni-CASSANDRA-11517-trunk-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/jkni/job/jkni-CASSANDRA-11517-trunk-dtest]| CI looks good relative to upstream. If you're fine with the test changes [~aweisberg], this LGTM. Side note: out of curiosity, I wrote a small JMH benchmark and ran it on Windows, Linux, and FreeBSD. It confirmed a throughput increase on all platforms. That test is available here and could be added as a microbenchmark: [CASSANDRA-11517-trunk-jmh|https://github.com/jkni/cassandra/tree/CASSANDRA-11517-trunk-jmh]. > o.a.c.utils.UUIDGen could handle contention better > -- > > Key: CASSANDRA-11517 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11517 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Minor > Fix For: 3.x > > > I noticed this profiling a query handler implementation that uses UUIDGen to > get handles to track queries for logging purposes. > Under contention threads are being unscheduled instead of spinning until the > lock is available. I would have expected intrinsic locks to be able to adapt > to this based on profiling information. > Either way it's seems pretty straightforward to rewrite this to use a CAS > loop and test that it generally produces unique values. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11706) Tracing payload not passed through newSession(..)
[
https://issues.apache.org/jira/browse/CASSANDRA-11706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mck updated CASSANDRA-11706:
Attachment: (was: 11706-trunk.txt)
> Tracing payload not passed through newSession(..)
> -
>
> Key: CASSANDRA-11706
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11706
> Project: Cassandra
> Issue Type: Bug
>Reporter: mck
>Assignee: mck
>Priority: Minor
> Labels: tracing
> Fix For: 3.6
>
> Attachments: 11706-trunk.txt
>
>
> Caused by CASSANDRA-10392
> There's a small bug in
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/tracing/Tracing.java#L153
> {noformat}
> public UUID newSession(UUID sessionId, Map
> customPayload)
> {
> return newSession(sessionId, TraceType.QUERY, Collections.EMPTY_MAP);
> }{noformat}
> in that it passes on an {{EMPTY_MAP}} instead of the {{customPayload}}.
> I've marked this as "minor" as custom tracing plugins can easily enough
> workaround it by also overriding the {{newSession(UUID sessionId,
> Map customPayload)}} method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11706) Tracing payload not passed through newSession(..)
[
https://issues.apache.org/jira/browse/CASSANDRA-11706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mck updated CASSANDRA-11706:
Attachment: 11706-trunk.txt
> Tracing payload not passed through newSession(..)
> -
>
> Key: CASSANDRA-11706
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11706
> Project: Cassandra
> Issue Type: Bug
>Reporter: mck
>Assignee: mck
>Priority: Minor
> Labels: tracing
> Fix For: 3.6
>
> Attachments: 11706-trunk.txt
>
>
> Caused by CASSANDRA-10392
> There's a small bug in
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/tracing/Tracing.java#L153
> {noformat}
> public UUID newSession(UUID sessionId, Map
> customPayload)
> {
> return newSession(sessionId, TraceType.QUERY, Collections.EMPTY_MAP);
> }{noformat}
> in that it passes on an {{EMPTY_MAP}} instead of the {{customPayload}}.
> I've marked this as "minor" as custom tracing plugins can easily enough
> workaround it by also overriding the {{newSession(UUID sessionId,
> Map customPayload)}} method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11706) Tracing payload not passed through newSession(..)
[
https://issues.apache.org/jira/browse/CASSANDRA-11706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273673#comment-15273673
]
mck commented on CASSANDRA-11706:
-
Updated the patch so that the log line improvement now uses String.format(..)
> Tracing payload not passed through newSession(..)
> -
>
> Key: CASSANDRA-11706
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11706
> Project: Cassandra
> Issue Type: Bug
>Reporter: mck
>Assignee: mck
>Priority: Minor
> Labels: tracing
> Fix For: 3.6
>
> Attachments: 11706-trunk.txt
>
>
> Caused by CASSANDRA-10392
> There's a small bug in
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/tracing/Tracing.java#L153
> {noformat}
> public UUID newSession(UUID sessionId, Map
> customPayload)
> {
> return newSession(sessionId, TraceType.QUERY, Collections.EMPTY_MAP);
> }{noformat}
> in that it passes on an {{EMPTY_MAP}} instead of the {{customPayload}}.
> I've marked this as "minor" as custom tracing plugins can easily enough
> workaround it by also overriding the {{newSession(UUID sessionId,
> Map customPayload)}} method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11724) False Failure Detection in Big Cassandra Cluster
[ https://issues.apache.org/jira/browse/CASSANDRA-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey F. Lukman updated CASSANDRA-11724: -- Description: We are running some testing on Cassandra v2.2.5 stable in a big cluster. The setting in our testing is that each machine has 16-cores and runs 8 cassandra instances, and our testing is 32, 64, 128, 256, and 512 instances of Cassandra. We use the default number of vnodes for each instance which is 256. The data and log directories are on in-memory tmpfs file system. We run several types of workloads on this Cassandra cluster: Workload1: Just start the cluster Workload2: Start half of the cluster, wait until it gets into a stable condition, and run another half of the cluster Workload3: Start half of the cluster, wait until it gets into a stable condition, load some data, and run another half of the cluster Workload4: Start the cluster, wait until it gets into a stable condition, load some data and decommission one node For this testing, we measure the total numbers of false failure detection inside the cluster. By false failure detection, we mean that, for example, instance-1 marks the instance-2 down, but the instance-2 is not down. We dig deeper into the root cause and find out that instance-1 has not received any heartbeat after some time from instance-2 because the instance-2 run a long computation process. Here I attach the graphs of each workload result. was: We are running some testing on Cassandra v2.2.5 stable in a big cluster. The setting in our testing is that each machine has 16-cores and runs 8 cassandra instances, and our testing is 32, 64, 128, 256, and 512 instances of Cassandra. We use the default number of vnodes for each instance which is 256. The data and log directories are on in-memory tmpfs file system. We run several types of workloads on this Cassandra cluster: Workload1: Just start the cluster Workload2: Start half of the cluster, wait until it gets into a stable condition, and run another half of the cluster Workload3: Start half of the cluster, wait until it gets into a stable condition, load some data, and run another half of the cluster Workload4: Start the cluster, wait until it gets into a stable condition and decommission one node For this testing, we measure the total numbers of false failure detection inside the cluster. By false failure detection, we mean that, for example, instance-1 marks the instance-2 down, but the instance-2 is not down. We dig deeper into the root cause and find out that instance-1 has not received any heartbeat after some time from instance-2 because the instance-2 run a long computation process. Here I attach the graphs of each workload result. > False Failure Detection in Big Cassandra Cluster > > > Key: CASSANDRA-11724 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11724 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Jeffrey F. Lukman > Labels: gossip, node-failure > Attachments: Workload1.jpg, Workload2.jpg, Workload3.jpg, > Workload4.jpg > > > We are running some testing on Cassandra v2.2.5 stable in a big cluster. The > setting in our testing is that each machine has 16-cores and runs 8 cassandra > instances, and our testing is 32, 64, 128, 256, and 512 instances of > Cassandra. We use the default number of vnodes for each instance which is > 256. The data and log directories are on in-memory tmpfs file system. > We run several types of workloads on this Cassandra cluster: > Workload1: Just start the cluster > Workload2: Start half of the cluster, wait until it gets into a stable > condition, and run another half of the cluster > Workload3: Start half of the cluster, wait until it gets into a stable > condition, load some data, and run another half of the cluster > Workload4: Start the cluster, wait until it gets into a stable condition, > load some data and decommission one node > For this testing, we measure the total numbers of false failure detection > inside the cluster. By false failure detection, we mean that, for example, > instance-1 marks the instance-2 down, but the instance-2 is not down. We dig > deeper into the root cause and find out that instance-1 has not received any > heartbeat after some time from instance-2 because the instance-2 run a long > computation process. > Here I attach the graphs of each workload result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11724) False Failure Detection in Big Cassandra Cluster
[ https://issues.apache.org/jira/browse/CASSANDRA-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey F. Lukman updated CASSANDRA-11724: -- Description: We are running some testing on Cassandra v2.2.5 stable in a big cluster. The setting in our testing is that each machine has 16-cores and runs 8 cassandra instances, and our testing is 32, 64, 128, 256, and 512 instances of Cassandra. We use the default number of vnodes for each instance which is 256. The data and log directories are on in-memory tmpfs file system. We run several types of workloads on this Cassandra cluster: Workload1: Just start the cluster Workload2: Start half of the cluster, wait until it gets into a stable condition, and run another half of the cluster Workload3: Start half of the cluster, wait until it gets into a stable condition, load some data, and run another half of the cluster Workload4: Start the cluster, wait until it gets into a stable condition and decommission one node For this testing, we measure the total numbers of false failure detection inside the cluster. By false failure detection, we mean that, for example, instance-1 marks the instance-2 down, but the instance-2 is not down. We dig deeper into the root cause and find out that instance-1 has not received any heartbeat after some time from instance-2 because the instance-2 run a long computation process. Here I attach the graphs of each workload result. was: We are running some testing on Cassandra v2.2.5 stable in a big cluster. The setting in our testing is that each machine runs 8 cassandra instances, and our testing is 32, 64, 128, 256, and 512 instances of Cassandra. We use the default number of vnodes for each instance which is 256. The data and log directories are on in-memory tmpfs file system. We run several types of workloads on this Cassandra cluster: Workload1: Just start the cluster Workload2: Start half of the cluster, wait until it gets into a stable condition, and run another half of the cluster Workload3: Start half of the cluster, wait until it gets into a stable condition, load some data, and run another half of the cluster Workload4: Start the cluster, wait until it gets into a stable condition and decommission one node For this testing, we measure the total numbers of false failure detection inside the cluster. By false failure detection, we mean that, for example, instance-1 marks the instance-2 down, but the instance-2 is not down. We dig deeper into the root cause and find out that instance-1 has not received any heartbeat after some time from instance-2 because the instance-2 run a long computation process. Here I attach the graphs of each workload result. > False Failure Detection in Big Cassandra Cluster > > > Key: CASSANDRA-11724 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11724 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Jeffrey F. Lukman > Labels: gossip, node-failure > Attachments: Workload1.jpg, Workload2.jpg, Workload3.jpg, > Workload4.jpg > > > We are running some testing on Cassandra v2.2.5 stable in a big cluster. The > setting in our testing is that each machine has 16-cores and runs 8 cassandra > instances, and our testing is 32, 64, 128, 256, and 512 instances of > Cassandra. We use the default number of vnodes for each instance which is > 256. The data and log directories are on in-memory tmpfs file system. > We run several types of workloads on this Cassandra cluster: > Workload1: Just start the cluster > Workload2: Start half of the cluster, wait until it gets into a stable > condition, and run another half of the cluster > Workload3: Start half of the cluster, wait until it gets into a stable > condition, load some data, and run another half of the cluster > Workload4: Start the cluster, wait until it gets into a stable condition and > decommission one node > For this testing, we measure the total numbers of false failure detection > inside the cluster. By false failure detection, we mean that, for example, > instance-1 marks the instance-2 down, but the instance-2 is not down. We dig > deeper into the root cause and find out that instance-1 has not received any > heartbeat after some time from instance-2 because the instance-2 run a long > computation process. > Here I attach the graphs of each workload result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11724) False Failure Detection in Big Cassandra Cluster
[ https://issues.apache.org/jira/browse/CASSANDRA-11724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey F. Lukman updated CASSANDRA-11724: -- Fix Version/s: (was: 2.2.5) > False Failure Detection in Big Cassandra Cluster > > > Key: CASSANDRA-11724 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11724 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Jeffrey F. Lukman > Labels: gossip, node-failure > Attachments: Workload1.jpg, Workload2.jpg, Workload3.jpg, > Workload4.jpg > > > We are running some testing on Cassandra v2.2.5 stable in a big cluster. The > setting in our testing is that each machine runs 8 cassandra instances, and > our testing is 32, 64, 128, 256, and 512 instances of Cassandra. We use the > default number of vnodes for each instance which is 256. The data and log > directories are on in-memory tmpfs file system. > We run several types of workloads on this Cassandra cluster: > Workload1: Just start the cluster > Workload2: Start half of the cluster, wait until it gets into a stable > condition, and run another half of the cluster > Workload3: Start half of the cluster, wait until it gets into a stable > condition, load some data, and run another half of the cluster > Workload4: Start the cluster, wait until it gets into a stable condition and > decommission one node > For this testing, we measure the total numbers of false failure detection > inside the cluster. By false failure detection, we mean that, for example, > instance-1 marks the instance-2 down, but the instance-2 is not down. We dig > deeper into the root cause and find out that instance-1 has not received any > heartbeat after some time from instance-2 because the instance-2 run a long > computation process. > Here I attach the graphs of each workload result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11724) False Failure Detection in Big Cassandra Cluster
Jeffrey F. Lukman created CASSANDRA-11724: - Summary: False Failure Detection in Big Cassandra Cluster Key: CASSANDRA-11724 URL: https://issues.apache.org/jira/browse/CASSANDRA-11724 Project: Cassandra Issue Type: Bug Components: Core Reporter: Jeffrey F. Lukman Fix For: 2.2.5 Attachments: Workload1.jpg, Workload2.jpg, Workload3.jpg, Workload4.jpg We are running some testing on Cassandra v2.2.5 stable in a big cluster. The setting in our testing is that each machine runs 8 cassandra instances, and our testing is 32, 64, 128, 256, and 512 instances of Cassandra. We use the default number of vnodes for each instance which is 256. The data and log directories are on in-memory tmpfs file system. We run several types of workloads on this Cassandra cluster: Workload1: Just start the cluster Workload2: Start half of the cluster, wait until it gets into a stable condition, and run another half of the cluster Workload3: Start half of the cluster, wait until it gets into a stable condition, load some data, and run another half of the cluster Workload4: Start the cluster, wait until it gets into a stable condition and decommission one node For this testing, we measure the total numbers of false failure detection inside the cluster. By false failure detection, we mean that, for example, instance-1 marks the instance-2 down, but the instance-2 is not down. We dig deeper into the root cause and find out that instance-1 has not received any heartbeat after some time from instance-2 because the instance-2 run a long computation process. Here I attach the graphs of each workload result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11031) MultiTenant : support “ALLOW FILTERING" for First Partition Key
[ https://issues.apache.org/jira/browse/CASSANDRA-11031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhaoYang updated CASSANDRA-11031: - Attachment: (was: CASSANDRA-11031-3.7.patch) > MultiTenant : support “ALLOW FILTERING" for First Partition Key > --- > > Key: CASSANDRA-11031 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11031 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Minor > Fix For: 3.x > > Attachments: CASSANDRA-11031-3.7.patch > > > Currently, Allow Filtering only works for secondary Index column or > clustering columns. And it's slow, because Cassandra will read all data from > SSTABLE from hard-disk to memory to filter. > But we can support allow filtering on Partition Key, as far as I know, > Partition Key is in memory, so we can easily filter them, and then read > required data from SSTable. > This will similar to "Select * from table" which scan through entire cluster. > CREATE TABLE multi_tenant_table ( > tenant_id text, > pk2 text, > c1 text, > c2 text, > v1 text, > v2 text, > PRIMARY KEY ((tenant_id,pk2),c1,c2) > ) ; > Select * from multi_tenant_table where tenant_id = "datastax" allow filtering; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11031) MultiTenant : support “ALLOW FILTERING" for First Partition Key
[ https://issues.apache.org/jira/browse/CASSANDRA-11031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhaoYang updated CASSANDRA-11031: - Attachment: CASSANDRA-11031-3.7.patch > MultiTenant : support “ALLOW FILTERING" for First Partition Key > --- > > Key: CASSANDRA-11031 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11031 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Minor > Fix For: 3.x > > Attachments: CASSANDRA-11031-3.7.patch > > > Currently, Allow Filtering only works for secondary Index column or > clustering columns. And it's slow, because Cassandra will read all data from > SSTABLE from hard-disk to memory to filter. > But we can support allow filtering on Partition Key, as far as I know, > Partition Key is in memory, so we can easily filter them, and then read > required data from SSTable. > This will similar to "Select * from table" which scan through entire cluster. > CREATE TABLE multi_tenant_table ( > tenant_id text, > pk2 text, > c1 text, > c2 text, > v1 text, > v2 text, > PRIMARY KEY ((tenant_id,pk2),c1,c2) > ) ; > Select * from multi_tenant_table where tenant_id = "datastax" allow filtering; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11541) correct the java documentation for SlabAllocator and NativeAllocator
[ https://issues.apache.org/jira/browse/CASSANDRA-11541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273537#comment-15273537 ] ZhaoYang commented on CASSANDRA-11541: -- [~JoshuaMcKenzie] sorry. that was for 2.1.15. I added patch for 2.2. > correct the java documentation for SlabAllocator and NativeAllocator > > > Key: CASSANDRA-11541 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11541 > Project: Cassandra > Issue Type: Improvement >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Trivial > Labels: document, lhf > Fix For: 2.2.x > > Attachments: CASSANDRA-11541-2.1.15.patch, CASSANDRA-11541-2.2.patch > > > I heard a lot that Cassandra uses 2MB slab allocation strategy for memtable > to improve its memory efficiency. But in fact, it has been changed from 2MB > to 1 MB. > And in NativeAllocator, it's logarithmically from 8kb to 1mb. > large number of tables may not cause too much trouble in term of memtable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11541) correct the java documentation for SlabAllocator and NativeAllocator
[ https://issues.apache.org/jira/browse/CASSANDRA-11541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhaoYang updated CASSANDRA-11541: - Attachment: (was: CASSANDRA-11541.patch) > correct the java documentation for SlabAllocator and NativeAllocator > > > Key: CASSANDRA-11541 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11541 > Project: Cassandra > Issue Type: Improvement >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Trivial > Labels: document, lhf > Fix For: 2.2.x > > Attachments: CASSANDRA-11541-2.1.15.patch, CASSANDRA-11541-2.2.patch > > > I heard a lot that Cassandra uses 2MB slab allocation strategy for memtable > to improve its memory efficiency. But in fact, it has been changed from 2MB > to 1 MB. > And in NativeAllocator, it's logarithmically from 8kb to 1mb. > large number of tables may not cause too much trouble in term of memtable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11541) correct the java documentation for SlabAllocator and NativeAllocator
[ https://issues.apache.org/jira/browse/CASSANDRA-11541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhaoYang updated CASSANDRA-11541: - Attachment: CASSANDRA-11541-2.1.15.patch CASSANDRA-11541-2.2.patch > correct the java documentation for SlabAllocator and NativeAllocator > > > Key: CASSANDRA-11541 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11541 > Project: Cassandra > Issue Type: Improvement >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Trivial > Labels: document, lhf > Fix For: 2.2.x > > Attachments: CASSANDRA-11541-2.1.15.patch, CASSANDRA-11541-2.2.patch > > > I heard a lot that Cassandra uses 2MB slab allocation strategy for memtable > to improve its memory efficiency. But in fact, it has been changed from 2MB > to 1 MB. > And in NativeAllocator, it's logarithmically from 8kb to 1mb. > large number of tables may not cause too much trouble in term of memtable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-11540) The JVM should exit if jmx fails to bind
[
https://issues.apache.org/jira/browse/CASSANDRA-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler resolved CASSANDRA-11540.
Resolution: Fixed
I just found the config change that helps out with my env situation:
{{-Dcassandra.jmx.local.port=7199}} instead of
{{-Dcom.sun.management.jmxremote.port=7199}}
> The JVM should exit if jmx fails to bind
>
>
> Key: CASSANDRA-11540
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11540
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Brandon Williams
>Assignee: Alex Petrov
> Labels: lhf
> Fix For: 2.2.7, 3.7, 3.0.7
>
>
> If you are already running a cassandra instance, but for some reason try to
> start another one, this happens:
> {noformat}
> INFO 20:57:09 JNA mlockall successful
> WARN 20:57:09 JMX is not enabled to receive remote connections. Please see
> cassandra-env.sh for more info.
> ERROR 20:57:10 Error starting local jmx server:
> java.rmi.server.ExportException: Port already in use: 7199; nested exception
> is:
> java.net.BindException: Address already in use
> at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:340)
> ~[na:1.7.0_76]
> at
> sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:248)
> ~[na:1.7.0_76]
> at
> sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411)
> ~[na:1.7.0_76]
> at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
> ~[na:1.7.0_76]
> at
> sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:207)
> ~[na:1.7.0_76]
> at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:122)
> ~[na:1.7.0_76]
> at sun.rmi.registry.RegistryImpl.(RegistryImpl.java:98)
> ~[na:1.7.0_76]
> at
> java.rmi.registry.LocateRegistry.createRegistry(LocateRegistry.java:239)
> ~[na:1.7.0_76]
> at
> org.apache.cassandra.service.CassandraDaemon.maybeInitJmx(CassandraDaemon.java:100)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:222)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:564)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:653)
> [main/:na]
> Caused by: java.net.BindException: Address already in use
> at java.net.PlainSocketImpl.socketBind(Native Method) ~[na:1.7.0_76]
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
> ~[na:1.7.0_76]
> at java.net.ServerSocket.bind(ServerSocket.java:376) ~[na:1.7.0_76]
> at java.net.ServerSocket.(ServerSocket.java:237) ~[na:1.7.0_76]
> at
> javax.net.DefaultServerSocketFactory.createServerSocket(ServerSocketFactory.java:231)
> ~[na:1.7.0_76]
> at
> org.apache.cassandra.utils.RMIServerSocketFactoryImpl.createServerSocket(RMIServerSocketFactoryImpl.java:13)
> ~[main/:na]
> at
> sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:666)
> ~[na:1.7.0_76]
> at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:329)
> ~[na:1.7.0_76]
> ... 11 common frames omitted
> {noformat}
> However the startup continues, and ends up replaying commitlogs, which is
> probably not a good thing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11723) Cassandra upgrade from 2.1.11 to 3.0.5 leads to unstable nodes
[
https://issues.apache.org/jira/browse/CASSANDRA-11723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefano Ortolani updated CASSANDRA-11723:
-
Priority: Major (was: Critical)
> Cassandra upgrade from 2.1.11 to 3.0.5 leads to unstable nodes
> --
>
> Key: CASSANDRA-11723
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11723
> Project: Cassandra
> Issue Type: Bug
>Reporter: Stefano Ortolani
>
> Upgrade seems fine, but any restart of the node might lead to a situation
> where the node just dies after 30 seconds / 1 minute.
> Nothing in the logs besides many "FailureDetector.java:456 - Ignoring
> interval time of 3000892567 for /10.12.a.x" output every second (against all
> other nodes) in debug.log plus some spurious GraphiteErrors/ReadRepair
> notifications:
> {code:xml}
> DEBUG [GossipStage:1] 2016-05-05 22:29:03,921 FailureDetector.java:456 -
> Ignoring interval time of 2373187360 for /10.12.a.x
> DEBUG [GossipStage:1] 2016-05-05 22:29:03,921 FailureDetector.java:456 -
> Ignoring interval time of 2000276196 for /10.12.a.y
> DEBUG [ReadRepairStage:24] 2016-05-05 22:29:03,990 ReadCallback.java:234 -
> Digest mismatch:
> org.apache.cassandra.service.DigestMismatchException: Mismatch for key
> DecoratedKey(-152946356843306763, e859fdd2f264485f42030ce261e4e12e)
> (d6e617ece3b7bec6138b52b8974b8cab vs 31becca666a62b3c4b2fc0bab9902718)
> at
> org.apache.cassandra.service.DigestResolver.resolve(DigestResolver.java:85)
> ~[apache-cassandra-3.0.5.jar:3.0.5]
> at
> org.apache.cassandra.service.ReadCallback$AsyncRepairRunner.run(ReadCallback.java:225)
> ~[apache-cassandra-3.0.5.jar:3.0.5]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_60]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
> DEBUG [GossipStage:1] 2016-05-05 22:29:04,841 FailureDetector.java:456 -
> Ignoring interval time of 3000299340 for /10.12.33.5
> ERROR [metrics-graphite-reporter-1-thread-1] 2016-05-05 22:29:05,692
> ScheduledReporter.java:119 - RuntimeException thrown from
> GraphiteReporter#report. Exception was suppressed.
> java.lang.IllegalStateException: Unable to compute ceiling for max when
> histogram overflowed
> at
> org.apache.cassandra.utils.EstimatedHistogram.rawMean(EstimatedHistogram.java:231)
> ~[apache-cassandra-3.0.5.jar:3.0.5]
> at
> org.apache.cassandra.metrics.EstimatedHistogramReservoir$HistogramSnapshot.getMean(EstimatedHistogramReservoir.java:103)
> ~[apache-cassandra-3.0.5.jar:3.0.5]
> at
> com.codahale.metrics.graphite.GraphiteReporter.reportHistogram(GraphiteReporter.java:252)
> ~[metrics-graphite-3.1.0.jar:3.1.0]
> at
> com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:166)
> ~[metrics-graphite-3.1.0.jar:3.1.0]
> at
> com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:162)
> ~[metrics-core-3.1.0.jar:3.1.0]
> at
> com.codahale.metrics.ScheduledReporter$1.run(ScheduledReporter.java:117)
> ~[metrics-core-3.1.0.jar:3.1.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_60]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_60]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_60]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_60]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
> {code}
> I know this is not much but nothing else gets to dmesg or to any other log.
> Any suggestion how to debug this further?
> I upgraded two nodes so far, and it happened on both nodes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (CASSANDRA-11540) The JVM should exit if jmx fails to bind
[
https://issues.apache.org/jira/browse/CASSANDRA-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler reopened CASSANDRA-11540:
> The JVM should exit if jmx fails to bind
>
>
> Key: CASSANDRA-11540
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11540
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Brandon Williams
>Assignee: Alex Petrov
> Labels: lhf
> Fix For: 2.2.7, 3.7, 3.0.7
>
>
> If you are already running a cassandra instance, but for some reason try to
> start another one, this happens:
> {noformat}
> INFO 20:57:09 JNA mlockall successful
> WARN 20:57:09 JMX is not enabled to receive remote connections. Please see
> cassandra-env.sh for more info.
> ERROR 20:57:10 Error starting local jmx server:
> java.rmi.server.ExportException: Port already in use: 7199; nested exception
> is:
> java.net.BindException: Address already in use
> at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:340)
> ~[na:1.7.0_76]
> at
> sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:248)
> ~[na:1.7.0_76]
> at
> sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411)
> ~[na:1.7.0_76]
> at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
> ~[na:1.7.0_76]
> at
> sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:207)
> ~[na:1.7.0_76]
> at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:122)
> ~[na:1.7.0_76]
> at sun.rmi.registry.RegistryImpl.(RegistryImpl.java:98)
> ~[na:1.7.0_76]
> at
> java.rmi.registry.LocateRegistry.createRegistry(LocateRegistry.java:239)
> ~[na:1.7.0_76]
> at
> org.apache.cassandra.service.CassandraDaemon.maybeInitJmx(CassandraDaemon.java:100)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:222)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:564)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:653)
> [main/:na]
> Caused by: java.net.BindException: Address already in use
> at java.net.PlainSocketImpl.socketBind(Native Method) ~[na:1.7.0_76]
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
> ~[na:1.7.0_76]
> at java.net.ServerSocket.bind(ServerSocket.java:376) ~[na:1.7.0_76]
> at java.net.ServerSocket.(ServerSocket.java:237) ~[na:1.7.0_76]
> at
> javax.net.DefaultServerSocketFactory.createServerSocket(ServerSocketFactory.java:231)
> ~[na:1.7.0_76]
> at
> org.apache.cassandra.utils.RMIServerSocketFactoryImpl.createServerSocket(RMIServerSocketFactoryImpl.java:13)
> ~[main/:na]
> at
> sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:666)
> ~[na:1.7.0_76]
> at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:329)
> ~[na:1.7.0_76]
> ... 11 common frames omitted
> {noformat}
> However the startup continues, and ends up replaying commitlogs, which is
> probably not a good thing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-11540) The JVM should exit if jmx fails to bind
[
https://issues.apache.org/jira/browse/CASSANDRA-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273271#comment-15273271
]
Michael Shuler edited comment on CASSANDRA-11540 at 5/5/16 10:56 PM:
-
Just a quick note on behavior after this commit, as we have found in cstar_perf
(trunk will not start at all, currently).
If a user happens to set an environment variable {{JVM_OPTS="$JVM_OPTS
-Dcom.sun.management.jmxremote.port=7199 ..."}} startup fails. There's no other
Cassandra running and nothing was listening to 7199 prior to startup.
I believe that commit 7b0c716 introduced for CASSANDRA-10091 does something a
little wonky with regards to JVM_OPTS. The previous commit, ad7e36b, does not
throw an error when JVM_OPTS are passed in the user env.
If this hard failure is working as intended, please let me know!
Repro on trunk HEAD:
{noformat}
(trunk)mshuler@mana:~/git/cassandra$ sudo netstat -atunp | grep 7199
(trunk)mshuler@mana:~/git/cassandra$
(trunk)mshuler@mana:~/git/cassandra$ export
JVM_OPTS="-Dcom.sun.management.jmxremote.port=7199
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
-Djava.rmi.server.hostname=127.0.0.1"
(trunk)mshuler@mana:~/git/cassandra$
(trunk)mshuler@mana:~/git/cassandra$ cassandra -f
<...>
INFO 22:00:18 Not submitting build tasks for views in keyspace system as
storage service is not initialized
ERROR 22:00:18 Port already in use: 7199; nested exception is:
java.net.BindException: Address already in use
java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method) ~[na:1.8.0_92]
at
java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
~[na:1.8.0_92]
at java.net.ServerSocket.bind(ServerSocket.java:375) ~[na:1.8.0_92]
at java.net.ServerSocket.(ServerSocket.java:237) ~[na:1.8.0_92]
at
javax.net.DefaultServerSocketFactory.createServerSocket(ServerSocketFactory.java:231)
~[na:1.8.0_92]
at
org.apache.cassandra.utils.RMIServerSocketFactoryImpl.createServerSocket(RMIServerSocketFactoryImpl.java:21)
~[main/:na]
at
sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:666)
~[na:1.8.0_92]
at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:330)
~[na:1.8.0_92]
at
sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:249)
~[na:1.8.0_92]
at sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411)
~[na:1.8.0_92]
at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
~[na:1.8.0_92]
at
sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:212)
~[na:1.8.0_92]
at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:152)
~[na:1.8.0_92]
at sun.rmi.registry.RegistryImpl.(RegistryImpl.java:112)
~[na:1.8.0_92]
at
java.rmi.registry.LocateRegistry.createRegistry(LocateRegistry.java:239)
~[na:1.8.0_92]
at
org.apache.cassandra.utils.JMXServerUtils.createJMXServer(JMXServerUtils.java:94)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.maybeInitJmx(CassandraDaemon.java:124)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:193)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:568)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:697)
[main/:na]
(trunk)mshuler@mana:~/git/cassandra$
{noformat}
was (Author: mshuler):
Just a quick note on behavior after this commit, as we have found in cstar_perf
(trunk will not start at all, currently).
If a user happens to set and environment variable {{JVM_OPTS="$JVM_OPTS
-Dcom.sun.management.jmxremote.port=7199 ..."}} startup fails. There's no other
Cassandra running and nothing was listening to 7199 prior to startup.
I believe that commit 7b0c716 introduced for CASSANDRA-10091 does something a
little wonky with regards to JVM_OPTS. The previous commit, ad7e36b, does not
throw an error when JVM_OPTS are passed in the user env.
If this hard failure is working as intended, please let me know!
Repro on trunk HEAD:
{noformat}
(trunk)mshuler@mana:~/git/cassandra$ sudo netstat -atunp | grep 7199
(trunk)mshuler@mana:~/git/cassandra$
(trunk)mshuler@mana:~/git/cassandra$ export
JVM_OPTS="-Dcom.sun.management.jmxremote.port=7199
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
-Djava.rmi.server.hostname=127.0.0.1"
(trunk)mshuler@mana:~/git/cassandra$
(trunk)mshuler@mana:~/git/cassandra$ cassandra -f
<...>
INFO 22:00:18 Not submitting build tasks for views in keyspace system as
storage service is not initialized
ERROR 22:00:18 Port already in use: 7199; nested exception is:
java.net.BindException: Address alr
[jira] [Commented] (CASSANDRA-11540) The JVM should exit if jmx fails to bind
[
https://issues.apache.org/jira/browse/CASSANDRA-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273271#comment-15273271
]
Michael Shuler commented on CASSANDRA-11540:
Just a quick note on behavior after this commit, as we have found in cstar_perf
(trunk will not start at all, currently).
If a user happens to set and environment variable {{JVM_OPTS="$JVM_OPTS
-Dcom.sun.management.jmxremote.port=7199 ..."}} startup fails. There's no other
Cassandra running and nothing was listening to 7199 prior to startup.
I believe that commit 7b0c716 introduced for CASSANDRA-10091 does something a
little wonky with regards to JVM_OPTS. The previous commit, ad7e36b, does not
throw an error when JVM_OPTS are passed in the user env.
If this hard failure is working as intended, please let me know!
Repro on trunk HEAD:
{noformat}
(trunk)mshuler@mana:~/git/cassandra$ sudo netstat -atunp | grep 7199
(trunk)mshuler@mana:~/git/cassandra$
(trunk)mshuler@mana:~/git/cassandra$ export
JVM_OPTS="-Dcom.sun.management.jmxremote.port=7199
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
-Djava.rmi.server.hostname=127.0.0.1"
(trunk)mshuler@mana:~/git/cassandra$
(trunk)mshuler@mana:~/git/cassandra$ cassandra -f
<...>
INFO 22:00:18 Not submitting build tasks for views in keyspace system as
storage service is not initialized
ERROR 22:00:18 Port already in use: 7199; nested exception is:
java.net.BindException: Address already in use
java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method) ~[na:1.8.0_92]
at
java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
~[na:1.8.0_92]
at java.net.ServerSocket.bind(ServerSocket.java:375) ~[na:1.8.0_92]
at java.net.ServerSocket.(ServerSocket.java:237) ~[na:1.8.0_92]
at
javax.net.DefaultServerSocketFactory.createServerSocket(ServerSocketFactory.java:231)
~[na:1.8.0_92]
at
org.apache.cassandra.utils.RMIServerSocketFactoryImpl.createServerSocket(RMIServerSocketFactoryImpl.java:21)
~[main/:na]
at
sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:666)
~[na:1.8.0_92]
at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:330)
~[na:1.8.0_92]
at
sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:249)
~[na:1.8.0_92]
at sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411)
~[na:1.8.0_92]
at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
~[na:1.8.0_92]
at
sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:212)
~[na:1.8.0_92]
at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:152)
~[na:1.8.0_92]
at sun.rmi.registry.RegistryImpl.(RegistryImpl.java:112)
~[na:1.8.0_92]
at
java.rmi.registry.LocateRegistry.createRegistry(LocateRegistry.java:239)
~[na:1.8.0_92]
at
org.apache.cassandra.utils.JMXServerUtils.createJMXServer(JMXServerUtils.java:94)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.maybeInitJmx(CassandraDaemon.java:124)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:193)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:568)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:697)
[main/:na]
(trunk)mshuler@mana:~/git/cassandra$
{noformat}
> The JVM should exit if jmx fails to bind
>
>
> Key: CASSANDRA-11540
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11540
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Brandon Williams
>Assignee: Alex Petrov
> Labels: lhf
> Fix For: 2.2.7, 3.7, 3.0.7
>
>
> If you are already running a cassandra instance, but for some reason try to
> start another one, this happens:
> {noformat}
> INFO 20:57:09 JNA mlockall successful
> WARN 20:57:09 JMX is not enabled to receive remote connections. Please see
> cassandra-env.sh for more info.
> ERROR 20:57:10 Error starting local jmx server:
> java.rmi.server.ExportException: Port already in use: 7199; nested exception
> is:
> java.net.BindException: Address already in use
> at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:340)
> ~[na:1.7.0_76]
> at
> sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:248)
> ~[na:1.7.0_76]
> at
> sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411)
> ~[na:1.7.0_76]
> at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
> ~[na:1.7.0_76]
> at
> sun.rmi.server.UnicastServerRef.exportObject(UnicastServerR
[jira] [Updated] (CASSANDRA-11578) remove DatabaseDescriptor dependency from FileUtil
[
https://issues.apache.org/jira/browse/CASSANDRA-11578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuki Morishita updated CASSANDRA-11578:
---
Resolution: Fixed
Fix Version/s: 3.8
Status: Resolved (was: Ready to Commit)
Committed as {[1dd33eca1b72e6cf1261471f88e7c3990be3f097}}, thanks!
> remove DatabaseDescriptor dependency from FileUtil
> --
>
> Key: CASSANDRA-11578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11578
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 3.8
>
>
> {{FileUtil}} has dependencies to {{DatabaseDescriptor}} and other online
> related classes like {{StorageService}} when handling FS error.
> This is used in handling error in SSTable as well, so when one wants to use
> SSTableReader/Writer offline, they has a chance to initializing unnecessary
> staff at error.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-11578) remove DatabaseDescriptor dependency from FileUtil
[
https://issues.apache.org/jira/browse/CASSANDRA-11578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273251#comment-15273251
]
Yuki Morishita edited comment on CASSANDRA-11578 at 5/5/16 10:41 PM:
-
Committed as {{1dd33eca1b72e6cf1261471f88e7c3990be3f097}}, thanks!
was (Author: yukim):
Committed as {[1dd33eca1b72e6cf1261471f88e7c3990be3f097}}, thanks!
> remove DatabaseDescriptor dependency from FileUtil
> --
>
> Key: CASSANDRA-11578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11578
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 3.8
>
>
> {{FileUtil}} has dependencies to {{DatabaseDescriptor}} and other online
> related classes like {{StorageService}} when handling FS error.
> This is used in handling error in SSTable as well, so when one wants to use
> SSTableReader/Writer offline, they has a chance to initializing unnecessary
> staff at error.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
cassandra git commit: Remove DatabaseDescriptor dependency from FileUtils
Repository: cassandra
Updated Branches:
refs/heads/trunk 89eb3e58b -> 1dd33eca1
Remove DatabaseDescriptor dependency from FileUtils
patch by yukim; reviewed by snazy for CASSANDRA-11578
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1dd33eca
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1dd33eca
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1dd33eca
Branch: refs/heads/trunk
Commit: 1dd33eca1b72e6cf1261471f88e7c3990be3f097
Parents: 89eb3e5
Author: Yuki Morishita
Authored: Tue Apr 12 11:26:14 2016 -0500
Committer: Yuki Morishita
Committed: Thu May 5 17:33:10 2016 -0500
--
CHANGES.txt | 1 +
.../org/apache/cassandra/io/FSErrorHandler.java | 30 ++
.../org/apache/cassandra/io/util/FileUtils.java | 75 +++---
.../cassandra/service/CassandraDaemon.java | 2 +
.../service/DefaultFSErrorHandler.java | 101 +++
.../apache/cassandra/db/DirectoriesTest.java| 2 +
6 files changed, 150 insertions(+), 61 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1dd33eca/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 45d9fb1..ba64a19 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.8
+ * Remove DatabaseDescriptor dependency from FileUtils (CASSANDRA-11578)
* Faster streaming (CASSANDRA-9766)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1dd33eca/src/java/org/apache/cassandra/io/FSErrorHandler.java
--
diff --git a/src/java/org/apache/cassandra/io/FSErrorHandler.java
b/src/java/org/apache/cassandra/io/FSErrorHandler.java
new file mode 100644
index 000..081ec0b
--- /dev/null
+++ b/src/java/org/apache/cassandra/io/FSErrorHandler.java
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.io;
+
+import org.apache.cassandra.io.sstable.CorruptSSTableException;
+
+/**
+ * Interface for handling file system errors.
+ */
+public interface FSErrorHandler
+{
+void handleCorruptSSTable(CorruptSSTableException e);
+void handleFSError(FSError e);
+}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1dd33eca/src/java/org/apache/cassandra/io/util/FileUtils.java
--
diff --git a/src/java/org/apache/cassandra/io/util/FileUtils.java
b/src/java/org/apache/cassandra/io/util/FileUtils.java
index 6b58e85..d522c27 100644
--- a/src/java/org/apache/cassandra/io/util/FileUtils.java
+++ b/src/java/org/apache/cassandra/io/util/FileUtils.java
@@ -28,27 +28,25 @@ import java.text.DecimalFormat;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
-
-import sun.nio.ch.DirectBuffer;
+import java.util.Optional;
+import java.util.concurrent.atomic.AtomicReference;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+import sun.nio.ch.DirectBuffer;
import org.apache.cassandra.concurrent.ScheduledExecutors;
-import org.apache.cassandra.config.DatabaseDescriptor;
-import org.apache.cassandra.db.BlacklistedDirectories;
-import org.apache.cassandra.db.Keyspace;
import org.apache.cassandra.io.FSError;
+import org.apache.cassandra.io.FSErrorHandler;
import org.apache.cassandra.io.FSReadError;
import org.apache.cassandra.io.FSWriteError;
import org.apache.cassandra.io.sstable.CorruptSSTableException;
-import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.utils.JVMStabilityInspector;
import static org.apache.cassandra.utils.Throwables.maybeFail;
import static org.apache.cassandra.utils.Throwables.merge;
-public class FileUtils
+public final class FileUtils
{
public static final Charset CHARSET = StandardCharsets.UTF_8;
@@ -60,6 +58,7 @@ public class FileUtils
private static final DecimalFormat df = new DecimalFormat("#.##");
private static final boolean canC
[jira] [Updated] (CASSANDRA-11723) Cassandra upgrade from 2.1.11 to 3.0.5 leads to unstable nodes
[
https://issues.apache.org/jira/browse/CASSANDRA-11723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefano Ortolani updated CASSANDRA-11723:
-
Description:
Upgrade seems fine, but any restart of the node might lead to a situation where
the node just dies after 30 seconds / 1 minute.
Nothing in the logs besides many "FailureDetector.java:456 - Ignoring interval
time of 3000892567 for /10.12.a.x" output every second (against all other
nodes) in debug.log plus some spurious GraphiteErrors/ReadRepair notifications:
{code:xml}
DEBUG [GossipStage:1] 2016-05-05 22:29:03,921 FailureDetector.java:456 -
Ignoring interval time of 2373187360 for /10.12.a.x
DEBUG [GossipStage:1] 2016-05-05 22:29:03,921 FailureDetector.java:456 -
Ignoring interval time of 2000276196 for /10.12.a.y
DEBUG [ReadRepairStage:24] 2016-05-05 22:29:03,990 ReadCallback.java:234 -
Digest mismatch:
org.apache.cassandra.service.DigestMismatchException: Mismatch for key
DecoratedKey(-152946356843306763, e859fdd2f264485f42030ce261e4e12e)
(d6e617ece3b7bec6138b52b8974b8cab vs 31becca666a62b3c4b2fc0bab9902718)
at
org.apache.cassandra.service.DigestResolver.resolve(DigestResolver.java:85)
~[apache-cassandra-3.0.5.jar:3.0.5]
at
org.apache.cassandra.service.ReadCallback$AsyncRepairRunner.run(ReadCallback.java:225)
~[apache-cassandra-3.0.5.jar:3.0.5]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_60]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_60]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
DEBUG [GossipStage:1] 2016-05-05 22:29:04,841 FailureDetector.java:456 -
Ignoring interval time of 3000299340 for /10.12.33.5
ERROR [metrics-graphite-reporter-1-thread-1] 2016-05-05 22:29:05,692
ScheduledReporter.java:119 - RuntimeException thrown from
GraphiteReporter#report. Exception was suppressed.
java.lang.IllegalStateException: Unable to compute ceiling for max when
histogram overflowed
at
org.apache.cassandra.utils.EstimatedHistogram.rawMean(EstimatedHistogram.java:231)
~[apache-cassandra-3.0.5.jar:3.0.5]
at
org.apache.cassandra.metrics.EstimatedHistogramReservoir$HistogramSnapshot.getMean(EstimatedHistogramReservoir.java:103)
~[apache-cassandra-3.0.5.jar:3.0.5]
at
com.codahale.metrics.graphite.GraphiteReporter.reportHistogram(GraphiteReporter.java:252)
~[metrics-graphite-3.1.0.jar:3.1.0]
at
com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:166)
~[metrics-graphite-3.1.0.jar:3.1.0]
at
com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:162)
~[metrics-core-3.1.0.jar:3.1.0]
at
com.codahale.metrics.ScheduledReporter$1.run(ScheduledReporter.java:117)
~[metrics-core-3.1.0.jar:3.1.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_60]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[na:1.8.0_60]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_60]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_60]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_60]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_60]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
{code}
I know this is not much but nothing else gets to dmesg or to any other log. Any
suggestion how to debug this further?
I upgraded two nodes so far, and it happened on both nodes.
was:
Upgrade seems fine, but any restart of the node might lead to a situation where
the node just dies after 30 seconds / 1 minute.
Nothing in the logs besides many "FailureDetector.java:456 - Ignoring interval
time of 3000892567 for /10.12.a.x" output every second (against all other
nodes) in debug.log plus some spurious GraphiteErrors/ReadRepair notifications:
{{
DEBUG [GossipStage:1] 2016-05-05 22:29:03,921 FailureDetector.java:456 -
Ignoring interval time of 2373187360 for /10.12.a.x
DEBUG [GossipStage:1] 2016-05-05 22:29:03,921 FailureDetector.java:456 -
Ignoring interval time of 2000276196 for /10.12.a.y
DEBUG [ReadRepairStage:24] 2016-05-05 22:29:03,990 ReadCallback.java:234 -
Digest mismatch:
org.apache.cassandra.service.DigestMismatchException: Mismatch for key
DecoratedKey(-152946356843306763, e859fdd2f264485f42030ce261e4e12e)
(d6e617ece3b7bec6138b52b8974b8cab vs 31becca666a62b3c4b2fc0bab9902718)
at
org.apache.cassandra.service.DigestResolver.resolve(DigestResolver.java:85)
~[apache-cassandra-3.0.5.jar:3.0.5]
at
org.apache.cassandra.service.ReadCallback$AsyncRepairRunner.run(ReadCal
[jira] [Updated] (CASSANDRA-11723) Cassandra upgrade from 2.1.11 to 3.0.5 leads to unstable nodes
[
https://issues.apache.org/jira/browse/CASSANDRA-11723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefano Ortolani updated CASSANDRA-11723:
-
Description:
Upgrade seems fine, but any restart of the node might lead to a situation where
the node just dies after 30 seconds / 1 minute.
Nothing in the logs besides many "FailureDetector.java:456 - Ignoring interval
time of 3000892567 for /10.12.a.x" output every second (against all other
nodes) in debug.log plus some spurious GraphiteErrors/ReadRepair notifications:
{{
DEBUG [GossipStage:1] 2016-05-05 22:29:03,921 FailureDetector.java:456 -
Ignoring interval time of 2373187360 for /10.12.a.x
DEBUG [GossipStage:1] 2016-05-05 22:29:03,921 FailureDetector.java:456 -
Ignoring interval time of 2000276196 for /10.12.a.y
DEBUG [ReadRepairStage:24] 2016-05-05 22:29:03,990 ReadCallback.java:234 -
Digest mismatch:
org.apache.cassandra.service.DigestMismatchException: Mismatch for key
DecoratedKey(-152946356843306763, e859fdd2f264485f42030ce261e4e12e)
(d6e617ece3b7bec6138b52b8974b8cab vs 31becca666a62b3c4b2fc0bab9902718)
at
org.apache.cassandra.service.DigestResolver.resolve(DigestResolver.java:85)
~[apache-cassandra-3.0.5.jar:3.0.5]
at
org.apache.cassandra.service.ReadCallback$AsyncRepairRunner.run(ReadCallback.java:225)
~[apache-cassandra-3.0.5.jar:3.0.5]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_60]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_60]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
DEBUG [GossipStage:1] 2016-05-05 22:29:04,841 FailureDetector.java:456 -
Ignoring interval time of 3000299340 for /10.12.33.5
ERROR [metrics-graphite-reporter-1-thread-1] 2016-05-05 22:29:05,692
ScheduledReporter.java:119 - RuntimeException thrown from
GraphiteReporter#report. Exception was suppressed.
java.lang.IllegalStateException: Unable to compute ceiling for max when
histogram overflowed
at
org.apache.cassandra.utils.EstimatedHistogram.rawMean(EstimatedHistogram.java:231)
~[apache-cassandra-3.0.5.jar:3.0.5]
at
org.apache.cassandra.metrics.EstimatedHistogramReservoir$HistogramSnapshot.getMean(EstimatedHistogramReservoir.java:103)
~[apache-cassandra-3.0.5.jar:3.0.5]
at
com.codahale.metrics.graphite.GraphiteReporter.reportHistogram(GraphiteReporter.java:252)
~[metrics-graphite-3.1.0.jar:3.1.0]
at
com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:166)
~[metrics-graphite-3.1.0.jar:3.1.0]
at
com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:162)
~[metrics-core-3.1.0.jar:3.1.0]
at
com.codahale.metrics.ScheduledReporter$1.run(ScheduledReporter.java:117)
~[metrics-core-3.1.0.jar:3.1.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_60]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[na:1.8.0_60]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_60]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_60]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_60]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_60]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
}}
I know this is not much but nothing else gets to dmesg or to any other log. Any
suggestion how to debug this further?
I upgraded two nodes so far, and it happened on both nodes.
was:
Upgrade seems fine, but any restart of the node might lead to a situation where
the node just dies after 30 seconds / 1 minute.
Nothing in the logs besides many "FailureDetector.java:456 - Ignoring interval
time of 3000892567 for /10.12.x.y" output every second (against all other
nodes) in debug.log plus some spurious GraphiteErrors/ReadRepair notifications:
{{{
DEBUG [GossipStage:1] 2016-05-05 22:29:03,921 FailureDetector.java:456 -
Ignoring interval time of 2373187360 for /10.12.33.6
DEBUG [GossipStage:1] 2016-05-05 22:29:03,921 FailureDetector.java:456 -
Ignoring interval time of 2000276196 for /10.12.33.7
DEBUG [ReadRepairStage:24] 2016-05-05 22:29:03,990 ReadCallback.java:234 -
Digest mismatch:
org.apache.cassandra.service.DigestMismatchException: Mismatch for key
DecoratedKey(-152946356843306763, e859fdd2f264485f42030ce261e4e12e)
(d6e617ece3b7bec6138b52b8974b8cab vs 31becca666a62b3c4b2fc0bab9902718)
at
org.apache.cassandra.service.DigestResolver.resolve(DigestResolver.java:85)
~[apache-cassandra-3.0.5.jar:3.0.5]
at
org.apache.cassandra.service.ReadCallback$AsyncRepairRunner.run(ReadCallback.jav
[jira] [Created] (CASSANDRA-11723) Cassandra upgrade from 2.1.11 to 3.0.5 leads to unstable nodes
Stefano Ortolani created CASSANDRA-11723:
Summary: Cassandra upgrade from 2.1.11 to 3.0.5 leads to unstable
nodes
Key: CASSANDRA-11723
URL: https://issues.apache.org/jira/browse/CASSANDRA-11723
Project: Cassandra
Issue Type: Bug
Reporter: Stefano Ortolani
Priority: Critical
Upgrade seems fine, but any restart of the node might lead to a situation where
the node just dies after 30 seconds / 1 minute.
Nothing in the logs besides many "FailureDetector.java:456 - Ignoring interval
time of 3000892567 for /10.12.x.y" output every second (against all other
nodes) in debug.log plus some spurious GraphiteErrors/ReadRepair notifications:
{{{
DEBUG [GossipStage:1] 2016-05-05 22:29:03,921 FailureDetector.java:456 -
Ignoring interval time of 2373187360 for /10.12.33.6
DEBUG [GossipStage:1] 2016-05-05 22:29:03,921 FailureDetector.java:456 -
Ignoring interval time of 2000276196 for /10.12.33.7
DEBUG [ReadRepairStage:24] 2016-05-05 22:29:03,990 ReadCallback.java:234 -
Digest mismatch:
org.apache.cassandra.service.DigestMismatchException: Mismatch for key
DecoratedKey(-152946356843306763, e859fdd2f264485f42030ce261e4e12e)
(d6e617ece3b7bec6138b52b8974b8cab vs 31becca666a62b3c4b2fc0bab9902718)
at
org.apache.cassandra.service.DigestResolver.resolve(DigestResolver.java:85)
~[apache-cassandra-3.0.5.jar:3.0.5]
at
org.apache.cassandra.service.ReadCallback$AsyncRepairRunner.run(ReadCallback.java:225)
~[apache-cassandra-3.0.5.jar:3.0.5]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_60]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_60]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
DEBUG [GossipStage:1] 2016-05-05 22:29:04,841 FailureDetector.java:456 -
Ignoring interval time of 3000299340 for /10.12.33.5
ERROR [metrics-graphite-reporter-1-thread-1] 2016-05-05 22:29:05,692
ScheduledReporter.java:119 - RuntimeException thrown from
GraphiteReporter#report. Exception was suppressed.
java.lang.IllegalStateException: Unable to compute ceiling for max when
histogram overflowed
at
org.apache.cassandra.utils.EstimatedHistogram.rawMean(EstimatedHistogram.java:231)
~[apache-cassandra-3.0.5.jar:3.0.5]
at
org.apache.cassandra.metrics.EstimatedHistogramReservoir$HistogramSnapshot.getMean(EstimatedHistogramReservoir.java:103)
~[apache-cassandra-3.0.5.jar:3.0.5]
at
com.codahale.metrics.graphite.GraphiteReporter.reportHistogram(GraphiteReporter.java:252)
~[metrics-graphite-3.1.0.jar:3.1.0]
at
com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:166)
~[metrics-graphite-3.1.0.jar:3.1.0]
at
com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:162)
~[metrics-core-3.1.0.jar:3.1.0]
at
com.codahale.metrics.ScheduledReporter$1.run(ScheduledReporter.java:117)
~[metrics-core-3.1.0.jar:3.1.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_60]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[na:1.8.0_60]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_60]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_60]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_60]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_60]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
}}}
I know this is not much but nothing else gets to dmesg or to any other log. Any
suggestion how to debug this further?
I upgraded two nodes so far, and it happened on both nodes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11541) correct the java documentation for SlabAllocator and NativeAllocator
[ https://issues.apache.org/jira/browse/CASSANDRA-11541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273206#comment-15273206 ] Joshua McKenzie commented on CASSANDRA-11541: - [~jasonstack]: patch doesn't apply on 2.2. Is FixVersion wrong on this ticket? > correct the java documentation for SlabAllocator and NativeAllocator > > > Key: CASSANDRA-11541 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11541 > Project: Cassandra > Issue Type: Improvement >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Trivial > Labels: document, lhf > Fix For: 2.2.x > > Attachments: CASSANDRA-11541.patch > > > I heard a lot that Cassandra uses 2MB slab allocation strategy for memtable > to improve its memory efficiency. But in fact, it has been changed from 2MB > to 1 MB. > And in NativeAllocator, it's logarithmically from 8kb to 1mb. > large number of tables may not cause too much trouble in term of memtable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11518) o.a.c.utils.UUIDGen clock generation is not very high in entropy
[
https://issues.apache.org/jira/browse/CASSANDRA-11518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-11518:
Resolution: Fixed
Fix Version/s: (was: 3.0.x)
(was: 3.x)
3.0.7
3.7
Status: Resolved (was: Ready to Commit)
[Committed|https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=commit;h=c19066ee0251fc9d3d7d8217949b93a65088a8e9].
> o.a.c.utils.UUIDGen clock generation is not very high in entropy
>
>
> Key: CASSANDRA-11518
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11518
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Trivial
> Fix For: 3.7, 3.0.7
>
>
> makeClockSeqAndNode uses {{java.util.Random}} to generate the clock.
> {{Random}} only has 48-bits of internal state so it's not going to generate
> the best bits for clock and in addition to that it uses a collision prone
> seed that sort of defeats the purpose of clock sequence.
> A better approach to get the most out of those 14-bits would be to use
> {{SecureRandom}} with something like SHA1PRNG.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7
Merge branch 'cassandra-3.0' into cassandra-3.7 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4e364d71 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4e364d71 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4e364d71 Branch: refs/heads/cassandra-3.7 Commit: 4e364d71edd63a9ee2ca2b05d463d81b24d725a4 Parents: 022edf4 c19066e Author: Josh McKenzie Authored: Thu May 5 18:08:46 2016 -0400 Committer: Josh McKenzie Committed: Thu May 5 18:08:46 2016 -0400 -- src/java/org/apache/cassandra/utils/UUIDGen.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4e364d71/src/java/org/apache/cassandra/utils/UUIDGen.java --
[3/6] cassandra git commit: CASSANDRA-11518, use SecureRandom instead of java.util.Random to initialize UUIDGen clock sequence.
CASSANDRA-11518, use SecureRandom instead of java.util.Random to initialize
UUIDGen clock sequence.
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c19066ee
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c19066ee
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c19066ee
Branch: refs/heads/trunk
Commit: c19066ee0251fc9d3d7d8217949b93a65088a8e9
Parents: b8f2011
Author: Ariel Weisberg
Authored: Wed Apr 6 17:05:39 2016 -0400
Committer: Josh McKenzie
Committed: Thu May 5 18:08:27 2016 -0400
--
src/java/org/apache/cassandra/utils/UUIDGen.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c19066ee/src/java/org/apache/cassandra/utils/UUIDGen.java
--
diff --git a/src/java/org/apache/cassandra/utils/UUIDGen.java
b/src/java/org/apache/cassandra/utils/UUIDGen.java
index df07e1f..f50545b 100644
--- a/src/java/org/apache/cassandra/utils/UUIDGen.java
+++ b/src/java/org/apache/cassandra/utils/UUIDGen.java
@@ -21,6 +21,7 @@ import java.net.InetAddress;
import java.nio.ByteBuffer;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
import java.util.Collection;
import java.util.Random;
import java.util.UUID;
@@ -239,7 +240,7 @@ public class UUIDGen
private static long makeClockSeqAndNode()
{
-long clock = new Random(System.currentTimeMillis()).nextLong();
+long clock = new SecureRandom().nextLong();
long lsb = 0;
lsb |= 0x8000L; // variant (2 bits)
[1/6] cassandra git commit: CASSANDRA-11518, use SecureRandom instead of java.util.Random to initialize UUIDGen clock sequence.
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.0 b8f201172 -> c19066ee0
refs/heads/cassandra-3.7 022edf436 -> 4e364d71e
refs/heads/trunk 95df20929 -> 89eb3e58b
CASSANDRA-11518, use SecureRandom instead of java.util.Random to initialize
UUIDGen clock sequence.
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c19066ee
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c19066ee
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c19066ee
Branch: refs/heads/cassandra-3.0
Commit: c19066ee0251fc9d3d7d8217949b93a65088a8e9
Parents: b8f2011
Author: Ariel Weisberg
Authored: Wed Apr 6 17:05:39 2016 -0400
Committer: Josh McKenzie
Committed: Thu May 5 18:08:27 2016 -0400
--
src/java/org/apache/cassandra/utils/UUIDGen.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c19066ee/src/java/org/apache/cassandra/utils/UUIDGen.java
--
diff --git a/src/java/org/apache/cassandra/utils/UUIDGen.java
b/src/java/org/apache/cassandra/utils/UUIDGen.java
index df07e1f..f50545b 100644
--- a/src/java/org/apache/cassandra/utils/UUIDGen.java
+++ b/src/java/org/apache/cassandra/utils/UUIDGen.java
@@ -21,6 +21,7 @@ import java.net.InetAddress;
import java.nio.ByteBuffer;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
import java.util.Collection;
import java.util.Random;
import java.util.UUID;
@@ -239,7 +240,7 @@ public class UUIDGen
private static long makeClockSeqAndNode()
{
-long clock = new Random(System.currentTimeMillis()).nextLong();
+long clock = new SecureRandom().nextLong();
long lsb = 0;
lsb |= 0x8000L; // variant (2 bits)
[6/6] cassandra git commit: Merge branch 'cassandra-3.7' into trunk
Merge branch 'cassandra-3.7' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/89eb3e58 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/89eb3e58 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/89eb3e58 Branch: refs/heads/trunk Commit: 89eb3e58b6f2229cb6be3b382722fd6832dc1a2a Parents: 95df209 4e364d7 Author: Josh McKenzie Authored: Thu May 5 18:08:53 2016 -0400 Committer: Josh McKenzie Committed: Thu May 5 18:08:53 2016 -0400 -- src/java/org/apache/cassandra/utils/UUIDGen.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) --
[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7
Merge branch 'cassandra-3.0' into cassandra-3.7 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4e364d71 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4e364d71 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4e364d71 Branch: refs/heads/trunk Commit: 4e364d71edd63a9ee2ca2b05d463d81b24d725a4 Parents: 022edf4 c19066e Author: Josh McKenzie Authored: Thu May 5 18:08:46 2016 -0400 Committer: Josh McKenzie Committed: Thu May 5 18:08:46 2016 -0400 -- src/java/org/apache/cassandra/utils/UUIDGen.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4e364d71/src/java/org/apache/cassandra/utils/UUIDGen.java --
[2/6] cassandra git commit: CASSANDRA-11518, use SecureRandom instead of java.util.Random to initialize UUIDGen clock sequence.
CASSANDRA-11518, use SecureRandom instead of java.util.Random to initialize
UUIDGen clock sequence.
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c19066ee
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c19066ee
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c19066ee
Branch: refs/heads/cassandra-3.7
Commit: c19066ee0251fc9d3d7d8217949b93a65088a8e9
Parents: b8f2011
Author: Ariel Weisberg
Authored: Wed Apr 6 17:05:39 2016 -0400
Committer: Josh McKenzie
Committed: Thu May 5 18:08:27 2016 -0400
--
src/java/org/apache/cassandra/utils/UUIDGen.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c19066ee/src/java/org/apache/cassandra/utils/UUIDGen.java
--
diff --git a/src/java/org/apache/cassandra/utils/UUIDGen.java
b/src/java/org/apache/cassandra/utils/UUIDGen.java
index df07e1f..f50545b 100644
--- a/src/java/org/apache/cassandra/utils/UUIDGen.java
+++ b/src/java/org/apache/cassandra/utils/UUIDGen.java
@@ -21,6 +21,7 @@ import java.net.InetAddress;
import java.nio.ByteBuffer;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
import java.util.Collection;
import java.util.Random;
import java.util.UUID;
@@ -239,7 +240,7 @@ public class UUIDGen
private static long makeClockSeqAndNode()
{
-long clock = new Random(System.currentTimeMillis()).nextLong();
+long clock = new SecureRandom().nextLong();
long lsb = 0;
lsb |= 0x8000L; // variant (2 bits)
[jira] [Updated] (CASSANDRA-10962) Cassandra should not create snapshot at restart for compactions_in_progress
[
https://issues.apache.org/jira/browse/CASSANDRA-10962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-10962:
Resolution: Fixed
Fix Version/s: (was: 2.2.x)
2.2.7
Reproduced In: 2.1.12, 2.0.10 (was: 2.0.10, 2.1.12)
Status: Resolved (was: Ready to Commit)
[Committed|https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=commit;h=5a923f65c2c02d08b687874dc8ac45d2d032b811].
> Cassandra should not create snapshot at restart for compactions_in_progress
> ---
>
> Key: CASSANDRA-10962
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10962
> Project: Cassandra
> Issue Type: Bug
> Environment: Ubuntu 14.04.3 LTS
>Reporter: FACORAT
>Assignee: Alex Petrov
>Priority: Minor
> Fix For: 2.2.7
>
>
> If auto_snapshot is set to true in cassandra.yaml, each time you restart
> Cassandra, a snapshot is created for system.compactions_in_progress as the
> table is truncated at cassandra start.
> However as datas in this table are temporary, Cassandra should not create
> snapshot for this table (or maybe even for system.* tables). This will be
> coherent with the fact that "nodetool listsnapshots" doesn't even list this
> table.
> Exemple:
> {code}
> $ nodetool listsnapshots | grep compactions
> $ ls -lh
> system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/snapshots/
> total 16K
> drwxr-xr-x 2 cassandra cassandra 4.0K Nov 30 13:12
> 1448885530280-compactions_in_progress
> drwxr-xr-x 2 cassandra cassandra 4.0K Dec 7 15:36
> 1449498977181-compactions_in_progress
> drwxr-xr-x 2 cassandra cassandra 4.0K Dec 14 18:20
> 1450113621506-compactions_in_progress
> drwxr-xr-x 2 cassandra cassandra 4.0K Jan 4 12:53
> 1451908396364-compactions_in_progress
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b8f20117 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b8f20117 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b8f20117 Branch: refs/heads/cassandra-3.0 Commit: b8f201172ee8511d745b83940b991f8586cc1a4d Parents: d689da3 5a923f6 Author: Josh McKenzie Authored: Thu May 5 18:04:12 2016 -0400 Committer: Josh McKenzie Committed: Thu May 5 18:04:12 2016 -0400 -- --
[03/10] cassandra git commit: Avoid creating a snapshot at restart for compactions_in_progress
Avoid creating a snapshot at restart for compactions_in_progress
Patch by Alex Petrov; reviewed by Joel Knighton for CASSANDRA-10962
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a923f65
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a923f65
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a923f65
Branch: refs/heads/cassandra-3.7
Commit: 5a923f65c2c02d08b687874dc8ac45d2d032b811
Parents: 5de9de1
Author: Alex Petrov
Authored: Thu Apr 28 16:20:19 2016 +0200
Committer: Josh McKenzie
Committed: Thu May 5 18:03:48 2016 -0400
--
.../org/apache/cassandra/db/ColumnFamilyStore.java | 16 ++--
.../org/apache/cassandra/db/SystemKeyspace.java | 2 +-
2 files changed, 15 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a923f65/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index bb2b5c9..513a138 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -2681,6 +2681,18 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
*/
public void truncateBlocking()
{
+truncateBlocking(DatabaseDescriptor.isAutoSnapshot());
+}
+
+/**
+ * Truncate deletes the column family's data with no expensive tombstone
creation,
+ * optionally snapshotting the data.
+ *
+ * @param takeSnapshot whether or not to take a snapshot true
if snapshot should be taken,
+ * false otherwise
+ */
+public void truncateBlocking(final boolean takeSnapshot)
+{
// We have two goals here:
// - truncate should delete everything written before truncate was
invoked
// - but not delete anything that isn't part of the snapshot we create.
@@ -2695,7 +2707,7 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
// position in the System keyspace.
logger.trace("truncating {}", name);
-if (keyspace.getMetadata().durableWrites ||
DatabaseDescriptor.isAutoSnapshot())
+if (keyspace.getMetadata().durableWrites || takeSnapshot)
{
// flush the CF being truncated before forcing the new segment
forceBlockingFlush();
@@ -2724,7 +2736,7 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
final long truncatedAt = System.currentTimeMillis();
data.notifyTruncated(truncatedAt);
-if (DatabaseDescriptor.isAutoSnapshot())
+if (takeSnapshot)
snapshot(Keyspace.getTimestampedSnapshotName(name));
ReplayPosition replayAfter = discardSSTables(truncatedAt);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a923f65/src/java/org/apache/cassandra/db/SystemKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java
b/src/java/org/apache/cassandra/db/SystemKeyspace.java
index 46de0db..1ce599d 100644
--- a/src/java/org/apache/cassandra/db/SystemKeyspace.java
+++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java
@@ -402,7 +402,7 @@ public final class SystemKeyspace
public static void discardCompactionsInProgress()
{
ColumnFamilyStore compactionLog =
Keyspace.open(NAME).getColumnFamilyStore(COMPACTIONS_IN_PROGRESS);
-compactionLog.truncateBlocking();
+compactionLog.truncateBlocking(false);
}
public static void updateCompactionHistory(String ksname,
[01/10] cassandra git commit: Avoid creating a snapshot at restart for compactions_in_progress
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.2 5de9de1f5 -> 5a923f65c
refs/heads/cassandra-3.0 d689da367 -> b8f201172
refs/heads/cassandra-3.7 58d3b9a90 -> 022edf436
refs/heads/trunk 07385b6ce -> 95df20929
Avoid creating a snapshot at restart for compactions_in_progress
Patch by Alex Petrov; reviewed by Joel Knighton for CASSANDRA-10962
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a923f65
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a923f65
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a923f65
Branch: refs/heads/cassandra-2.2
Commit: 5a923f65c2c02d08b687874dc8ac45d2d032b811
Parents: 5de9de1
Author: Alex Petrov
Authored: Thu Apr 28 16:20:19 2016 +0200
Committer: Josh McKenzie
Committed: Thu May 5 18:03:48 2016 -0400
--
.../org/apache/cassandra/db/ColumnFamilyStore.java | 16 ++--
.../org/apache/cassandra/db/SystemKeyspace.java | 2 +-
2 files changed, 15 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a923f65/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index bb2b5c9..513a138 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -2681,6 +2681,18 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
*/
public void truncateBlocking()
{
+truncateBlocking(DatabaseDescriptor.isAutoSnapshot());
+}
+
+/**
+ * Truncate deletes the column family's data with no expensive tombstone
creation,
+ * optionally snapshotting the data.
+ *
+ * @param takeSnapshot whether or not to take a snapshot true
if snapshot should be taken,
+ * false otherwise
+ */
+public void truncateBlocking(final boolean takeSnapshot)
+{
// We have two goals here:
// - truncate should delete everything written before truncate was
invoked
// - but not delete anything that isn't part of the snapshot we create.
@@ -2695,7 +2707,7 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
// position in the System keyspace.
logger.trace("truncating {}", name);
-if (keyspace.getMetadata().durableWrites ||
DatabaseDescriptor.isAutoSnapshot())
+if (keyspace.getMetadata().durableWrites || takeSnapshot)
{
// flush the CF being truncated before forcing the new segment
forceBlockingFlush();
@@ -2724,7 +2736,7 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
final long truncatedAt = System.currentTimeMillis();
data.notifyTruncated(truncatedAt);
-if (DatabaseDescriptor.isAutoSnapshot())
+if (takeSnapshot)
snapshot(Keyspace.getTimestampedSnapshotName(name));
ReplayPosition replayAfter = discardSSTables(truncatedAt);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a923f65/src/java/org/apache/cassandra/db/SystemKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java
b/src/java/org/apache/cassandra/db/SystemKeyspace.java
index 46de0db..1ce599d 100644
--- a/src/java/org/apache/cassandra/db/SystemKeyspace.java
+++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java
@@ -402,7 +402,7 @@ public final class SystemKeyspace
public static void discardCompactionsInProgress()
{
ColumnFamilyStore compactionLog =
Keyspace.open(NAME).getColumnFamilyStore(COMPACTIONS_IN_PROGRESS);
-compactionLog.truncateBlocking();
+compactionLog.truncateBlocking(false);
}
public static void updateCompactionHistory(String ksname,
[04/10] cassandra git commit: Avoid creating a snapshot at restart for compactions_in_progress
Avoid creating a snapshot at restart for compactions_in_progress
Patch by Alex Petrov; reviewed by Joel Knighton for CASSANDRA-10962
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a923f65
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a923f65
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a923f65
Branch: refs/heads/trunk
Commit: 5a923f65c2c02d08b687874dc8ac45d2d032b811
Parents: 5de9de1
Author: Alex Petrov
Authored: Thu Apr 28 16:20:19 2016 +0200
Committer: Josh McKenzie
Committed: Thu May 5 18:03:48 2016 -0400
--
.../org/apache/cassandra/db/ColumnFamilyStore.java | 16 ++--
.../org/apache/cassandra/db/SystemKeyspace.java | 2 +-
2 files changed, 15 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a923f65/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index bb2b5c9..513a138 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -2681,6 +2681,18 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
*/
public void truncateBlocking()
{
+truncateBlocking(DatabaseDescriptor.isAutoSnapshot());
+}
+
+/**
+ * Truncate deletes the column family's data with no expensive tombstone
creation,
+ * optionally snapshotting the data.
+ *
+ * @param takeSnapshot whether or not to take a snapshot true
if snapshot should be taken,
+ * false otherwise
+ */
+public void truncateBlocking(final boolean takeSnapshot)
+{
// We have two goals here:
// - truncate should delete everything written before truncate was
invoked
// - but not delete anything that isn't part of the snapshot we create.
@@ -2695,7 +2707,7 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
// position in the System keyspace.
logger.trace("truncating {}", name);
-if (keyspace.getMetadata().durableWrites ||
DatabaseDescriptor.isAutoSnapshot())
+if (keyspace.getMetadata().durableWrites || takeSnapshot)
{
// flush the CF being truncated before forcing the new segment
forceBlockingFlush();
@@ -2724,7 +2736,7 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
final long truncatedAt = System.currentTimeMillis();
data.notifyTruncated(truncatedAt);
-if (DatabaseDescriptor.isAutoSnapshot())
+if (takeSnapshot)
snapshot(Keyspace.getTimestampedSnapshotName(name));
ReplayPosition replayAfter = discardSSTables(truncatedAt);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a923f65/src/java/org/apache/cassandra/db/SystemKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java
b/src/java/org/apache/cassandra/db/SystemKeyspace.java
index 46de0db..1ce599d 100644
--- a/src/java/org/apache/cassandra/db/SystemKeyspace.java
+++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java
@@ -402,7 +402,7 @@ public final class SystemKeyspace
public static void discardCompactionsInProgress()
{
ColumnFamilyStore compactionLog =
Keyspace.open(NAME).getColumnFamilyStore(COMPACTIONS_IN_PROGRESS);
-compactionLog.truncateBlocking();
+compactionLog.truncateBlocking(false);
}
public static void updateCompactionHistory(String ksname,
[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7
Merge branch 'cassandra-3.0' into cassandra-3.7 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/022edf43 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/022edf43 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/022edf43 Branch: refs/heads/cassandra-3.7 Commit: 022edf43601eeb5225b9a64fdff9801fd5929c3e Parents: 58d3b9a b8f2011 Author: Josh McKenzie Authored: Thu May 5 18:04:24 2016 -0400 Committer: Josh McKenzie Committed: Thu May 5 18:04:24 2016 -0400 -- --
[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7
Merge branch 'cassandra-3.0' into cassandra-3.7 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/022edf43 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/022edf43 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/022edf43 Branch: refs/heads/trunk Commit: 022edf43601eeb5225b9a64fdff9801fd5929c3e Parents: 58d3b9a b8f2011 Author: Josh McKenzie Authored: Thu May 5 18:04:24 2016 -0400 Committer: Josh McKenzie Committed: Thu May 5 18:04:24 2016 -0400 -- --
[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b8f20117 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b8f20117 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b8f20117 Branch: refs/heads/cassandra-3.7 Commit: b8f201172ee8511d745b83940b991f8586cc1a4d Parents: d689da3 5a923f6 Author: Josh McKenzie Authored: Thu May 5 18:04:12 2016 -0400 Committer: Josh McKenzie Committed: Thu May 5 18:04:12 2016 -0400 -- --
[02/10] cassandra git commit: Avoid creating a snapshot at restart for compactions_in_progress
Avoid creating a snapshot at restart for compactions_in_progress
Patch by Alex Petrov; reviewed by Joel Knighton for CASSANDRA-10962
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a923f65
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a923f65
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a923f65
Branch: refs/heads/cassandra-3.0
Commit: 5a923f65c2c02d08b687874dc8ac45d2d032b811
Parents: 5de9de1
Author: Alex Petrov
Authored: Thu Apr 28 16:20:19 2016 +0200
Committer: Josh McKenzie
Committed: Thu May 5 18:03:48 2016 -0400
--
.../org/apache/cassandra/db/ColumnFamilyStore.java | 16 ++--
.../org/apache/cassandra/db/SystemKeyspace.java | 2 +-
2 files changed, 15 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a923f65/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index bb2b5c9..513a138 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -2681,6 +2681,18 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
*/
public void truncateBlocking()
{
+truncateBlocking(DatabaseDescriptor.isAutoSnapshot());
+}
+
+/**
+ * Truncate deletes the column family's data with no expensive tombstone
creation,
+ * optionally snapshotting the data.
+ *
+ * @param takeSnapshot whether or not to take a snapshot true
if snapshot should be taken,
+ * false otherwise
+ */
+public void truncateBlocking(final boolean takeSnapshot)
+{
// We have two goals here:
// - truncate should delete everything written before truncate was
invoked
// - but not delete anything that isn't part of the snapshot we create.
@@ -2695,7 +2707,7 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
// position in the System keyspace.
logger.trace("truncating {}", name);
-if (keyspace.getMetadata().durableWrites ||
DatabaseDescriptor.isAutoSnapshot())
+if (keyspace.getMetadata().durableWrites || takeSnapshot)
{
// flush the CF being truncated before forcing the new segment
forceBlockingFlush();
@@ -2724,7 +2736,7 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean
final long truncatedAt = System.currentTimeMillis();
data.notifyTruncated(truncatedAt);
-if (DatabaseDescriptor.isAutoSnapshot())
+if (takeSnapshot)
snapshot(Keyspace.getTimestampedSnapshotName(name));
ReplayPosition replayAfter = discardSSTables(truncatedAt);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a923f65/src/java/org/apache/cassandra/db/SystemKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java
b/src/java/org/apache/cassandra/db/SystemKeyspace.java
index 46de0db..1ce599d 100644
--- a/src/java/org/apache/cassandra/db/SystemKeyspace.java
+++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java
@@ -402,7 +402,7 @@ public final class SystemKeyspace
public static void discardCompactionsInProgress()
{
ColumnFamilyStore compactionLog =
Keyspace.open(NAME).getColumnFamilyStore(COMPACTIONS_IN_PROGRESS);
-compactionLog.truncateBlocking();
+compactionLog.truncateBlocking(false);
}
public static void updateCompactionHistory(String ksname,
[10/10] cassandra git commit: Merge branch 'cassandra-3.7' into trunk
Merge branch 'cassandra-3.7' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/95df2092 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/95df2092 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/95df2092 Branch: refs/heads/trunk Commit: 95df209290fafa6f3b1e9983c7abec1add6cc7ee Parents: 07385b6 022edf4 Author: Josh McKenzie Authored: Thu May 5 18:04:31 2016 -0400 Committer: Josh McKenzie Committed: Thu May 5 18:04:31 2016 -0400 -- --
[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b8f20117 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b8f20117 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b8f20117 Branch: refs/heads/trunk Commit: b8f201172ee8511d745b83940b991f8586cc1a4d Parents: d689da3 5a923f6 Author: Josh McKenzie Authored: Thu May 5 18:04:12 2016 -0400 Committer: Josh McKenzie Committed: Thu May 5 18:04:12 2016 -0400 -- --
[jira] [Commented] (CASSANDRA-11463) dtest failure in compaction_test.TestCompaction_with_LeveledCompactionStrategy.bloomfilter_size_test
[
https://issues.apache.org/jira/browse/CASSANDRA-11463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273182#comment-15273182
]
Paulo Motta commented on CASSANDRA-11463:
-
+1, can you make the PR? thanks!
> dtest failure in
> compaction_test.TestCompaction_with_LeveledCompactionStrategy.bloomfilter_size_test
>
>
> Key: CASSANDRA-11463
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11463
> Project: Cassandra
> Issue Type: Test
>Reporter: Philip Thompson
>Assignee: Marcus Eriksson
> Labels: dtest
> Fix For: 2.1.x
>
>
> The final assertion {{self.assertGreaterEqual(bfSize, min_bf_size)}} is
> failing with {{44728 not greater than or equal to 5}} on 2.1, pretty
> consistently.
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_dtest/439/testReport/compaction_test/TestCompaction_with_LeveledCompactionStrategy/bloomfilter_size_test
> Failed on CassCI build cassandra-2.1_dtest #439
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11463) dtest failure in compaction_test.TestCompaction_with_LeveledCompactionStrategy.bloomfilter_size_test
[
https://issues.apache.org/jira/browse/CASSANDRA-11463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta updated CASSANDRA-11463:
Status: Ready to Commit (was: Patch Available)
> dtest failure in
> compaction_test.TestCompaction_with_LeveledCompactionStrategy.bloomfilter_size_test
>
>
> Key: CASSANDRA-11463
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11463
> Project: Cassandra
> Issue Type: Test
>Reporter: Philip Thompson
>Assignee: Marcus Eriksson
> Labels: dtest
> Fix For: 2.1.x
>
>
> The final assertion {{self.assertGreaterEqual(bfSize, min_bf_size)}} is
> failing with {{44728 not greater than or equal to 5}} on 2.1, pretty
> consistently.
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_dtest/439/testReport/compaction_test/TestCompaction_with_LeveledCompactionStrategy/bloomfilter_size_test
> Failed on CassCI build cassandra-2.1_dtest #439
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10526) Add dtest for CASSANDRA-10406
[
https://issues.apache.org/jira/browse/CASSANDRA-10526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273174#comment-15273174
]
Paulo Motta commented on CASSANDRA-10526:
-
Looks good so far, but we're only testing that the whole range is transferred,
so if {{-ts}} parameter is ignored server side this will still work. Can you do
add a partial rebuild step and check that only some keys are transferred?
> Add dtest for CASSANDRA-10406
> -
>
> Key: CASSANDRA-10526
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10526
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 3.x
>
>
> CASSANDRA-10406 adds new function to nodetool rebuild, so it needs to be
> tested with cassandra-dtest.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8377) Coordinated Commitlog Replay
[ https://issues.apache.org/jira/browse/CASSANDRA-8377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273135#comment-15273135 ] Chris Lohfink edited comment on CASSANDRA-8377 at 5/5/16 9:37 PM: -- its been awhile... out of curiosity whats thoughts of making this a new tool vs a jmx thing? like sstableloader we can have a "commitlogloader". Can isolate it a little from the server. That said tacking it on to the existing commit log replay stuff is easier, but its awkward from nodetool to send a "file", kinda has to be a path that would then have to exist on the C* node that nodetool connects to. was (Author: cnlwsu): its been awhile... out of curiosity whats thoughts of making this a new tool vs a jmx thing? like sstableloader we can have a "commitlogloader". Can isolate it a little from the server. > Coordinated Commitlog Replay > > > Key: CASSANDRA-8377 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8377 > Project: Cassandra > Issue Type: New Feature >Reporter: Nick Bailey >Assignee: Chris Lohfink > Fix For: 3.x > > Attachments: CASSANDRA-8377.txt > > > Commit log archiving and replay can be used to support point in time restores > on a cluster. Unfortunately, at the moment that is only true when the > topology of the cluster is exactly the same as when the commitlogs were > archived. This is because commitlogs need to be replayed on a node that is a > replica for those writes. > To support replaying commitlogs when the topology has changed we should have > a tool that replays the writes in a commitlog as if they were writes from a > client and will get coordinated to the correct replicas. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8377) Coordinated Commitlog Replay
[ https://issues.apache.org/jira/browse/CASSANDRA-8377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273135#comment-15273135 ] Chris Lohfink commented on CASSANDRA-8377: -- its been awhile... out of curiosity whats thoughts of making this a new tool vs a jmx thing? like sstableloader we can have a "commitlogloader". Can isolate it a little from the server. > Coordinated Commitlog Replay > > > Key: CASSANDRA-8377 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8377 > Project: Cassandra > Issue Type: New Feature >Reporter: Nick Bailey >Assignee: Chris Lohfink > Fix For: 3.x > > Attachments: CASSANDRA-8377.txt > > > Commit log archiving and replay can be used to support point in time restores > on a cluster. Unfortunately, at the moment that is only true when the > topology of the cluster is exactly the same as when the commitlogs were > archived. This is because commitlogs need to be replayed on a node that is a > replica for those writes. > To support replaying commitlogs when the topology has changed we should have > a tool that replays the writes in a commitlog as if they were writes from a > client and will get coordinated to the correct replicas. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11722) No existant cassandra-topology.properties file causes repeating error
[ https://issues.apache.org/jira/browse/CASSANDRA-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Jolly updated CASSANDRA-11722: --- Description: So starting at the same time on two of my Cassandra Nodes this error was repeatedly thrown: May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: ERROR 21:00:18 Timed run of class org.apache.cassandra.locator.PropertyFileSnitch$1 failed. May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: org.apache.cassandra.exceptions.ConfigurationException: unable to locate cassandra-topology.properties May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at org.apache.cassandra.utils.FBUtilities.resourceToFile(FBUtilities.java:299) ~[apache-cassandra-3.0.5.jar:3.0.5] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at org.apache.cassandra.utils.ResourceWatcher$WatchedResource.run(ResourceWatcher.java:53) ~[apache-cassandra-3.0.5.jar:3.0.5] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118) [apache-cassandra-3.0.5.jar:3.0.5] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65] I believe it is because it tired to fallback to the cassandra-topology.properties file which does not exist. Only after I restarted did it stop erroring. I am running 3.0.5. was: So sudden on two of my Cassandra Nodes at the same time they started repeatedly throwing this error: May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: ERROR 21:00:18 Timed run of class org.apache.cassandra.locator.PropertyFileSnitch$1 failed. May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: org.apache.cassandra.exceptions.ConfigurationException: unable to locate cassandra-topology.properties May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at org.apache.cassandra.utils.FBUtilities.resourceToFile(FBUtilities.java:299) ~[apache-cassandra-3.0.5.jar:3.0.5] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at org.apache.cassandra.utils.ResourceWatcher$WatchedResource.run(ResourceWatcher.java:53) ~[apache-cassandra-3.0.5.jar:3.0.5] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118) [apache-cassandra-3.0.5.jar:3.0.5] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65] I believe it is because it tired to fallback to the cassandra-topology.properties file which does not exist. A restart fixed the issue. I am running 3.0.5. > No existant cassandra-topology.properties file causes repeating error > - > > Key: CASSANDRA-11722 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11722 > Project: Cassandra > Issue Type: Bug >Reporter: Luke Jolly > >
[jira] [Updated] (CASSANDRA-11722) No existant cassandra-topology.properties file causes repeating error
[ https://issues.apache.org/jira/browse/CASSANDRA-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Jolly updated CASSANDRA-11722: --- Description: So sudden on two of my Cassandra Nodes at the same time they started repeatedly throwing this error: May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: ERROR 21:00:18 Timed run of class org.apache.cassandra.locator.PropertyFileSnitch$1 failed. May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: org.apache.cassandra.exceptions.ConfigurationException: unable to locate cassandra-topology.properties May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at org.apache.cassandra.utils.FBUtilities.resourceToFile(FBUtilities.java:299) ~[apache-cassandra-3.0.5.jar:3.0.5] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at org.apache.cassandra.utils.ResourceWatcher$WatchedResource.run(ResourceWatcher.java:53) ~[apache-cassandra-3.0.5.jar:3.0.5] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118) [apache-cassandra-3.0.5.jar:3.0.5] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65] I believe it is because it tired to fallback to the cassandra-topology.properties file which does not exist. A restart fixed the issue. I am running 3.0.5. was: So sudden on two of my Cassandra Nodes at the same time they started repeatedly throwing this error: May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: ERROR 21:00:18 Timed run of class org.apache.cassandra.locator.PropertyFileSnitch$1 failed. May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: org.apache.cassandra.exceptions.ConfigurationException: unable to locate cassandra-topology.properties May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at org.apache.cassandra.utils.FBUtilities.resourceToFile(FBUtilities.java:299) ~[apache-cassandra-3.0.5.jar:3.0.5] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at org.apache.cassandra.utils.ResourceWatcher$WatchedResource.run(ResourceWatcher.java:53) ~[apache-cassandra-3.0.5.jar:3.0.5] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118) [apache-cassandra-3.0.5.jar:3.0.5] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65] I believe it is because it tired to fallback to the cassandra-topology.properties file which does not exist. I am running 3.0.5 > No existant cassandra-topology.properties file causes repeating error > - > > Key: CASSANDRA-11722 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11722 > Project: Cassandra > Issue Type: Bug >Reporter: Luke Jolly > > So sudden on two of my Cassandra Nodes
[jira] [Created] (CASSANDRA-11722) No existant cassandra-topology.properties file causes repeating error
Luke Jolly created CASSANDRA-11722: -- Summary: No existant cassandra-topology.properties file causes repeating error Key: CASSANDRA-11722 URL: https://issues.apache.org/jira/browse/CASSANDRA-11722 Project: Cassandra Issue Type: Bug Reporter: Luke Jolly So sudden on two of my Cassandra Nodes at the same time they started repeatedly throwing this error: May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: ERROR 21:00:18 Timed run of class org.apache.cassandra.locator.PropertyFileSnitch$1 failed. May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: org.apache.cassandra.exceptions.ConfigurationException: unable to locate cassandra-topology.properties May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at org.apache.cassandra.utils.FBUtilities.resourceToFile(FBUtilities.java:299) ~[apache-cassandra-3.0.5.jar:3.0.5] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at org.apache.cassandra.utils.ResourceWatcher$WatchedResource.run(ResourceWatcher.java:53) ~[apache-cassandra-3.0.5.jar:3.0.5] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118) [apache-cassandra-3.0.5.jar:3.0.5] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_65] May 05 21:00:18 tammy.11-e.ninja cassandra[19471]: at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65] I believe it is because it tired to fallback to the cassandra-topology.properties file which does not exist. I am running 3.0.5 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9633) Add ability to encrypt sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-9633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273091#comment-15273091 ] Blake Eggleston commented on CASSANDRA-9633: Thanks Jason. Sorry for the delay getting to this. This is getting there, although there are a few points that can be improved: * Since sstable chunks don't have any relation to each other (from the perspective of encryption/decryption), iv values shouldn't be shared between them. The cipher should be reinitialized with a fresh iv for each chunk. This should also simplify the patch, since there's a decent amount of logic supporting attaching ivs to sstables, copying compressors, having separate encryption and decryption contexts, and so forth. * Not every cipher mode supports initialization vectors. For instance, trying to init a cipher using 'AES/ECB/NoPadding' with an iv, even one with 0 length, will throw an exception. So there needs to be some logic determining if an iv is required / can be used. * Enabling encryption on a table silently discards any compression settings on the table. Making encryption part of the compression path (and sstable metadata) makes sense as an implementation detail, but the two should be kept separate from the user's perspective. I know you're creating an lz4 compressor in the encrypting compressor, but since users can (and do) create their own ICompressor implementations, the two shouldn't be mutually exclusive. * I'd like to see some higher level testing of this. Specifically: ** Tests demonstrating the sequence of table creation, insert, flush, and select working properly. ** Tests demonstrating streaming works properly. ** Tests showing data can't be read if the key is removed. * The key alias used to encrypt an sstable should be saved with the sstable metadata, and used when decrypting. Otherwise, I don't see a way to re-encrypt your data with another key if the existing key is compromised. Misc nits: * The ternary conditional logic in CompressedSegmentedFile is getting pretty intense. Can we move that into a static method? * The tde comments in cassandra.yaml need to be updated. > Add ability to encrypt sstables > --- > > Key: CASSANDRA-9633 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9633 > Project: Cassandra > Issue Type: New Feature >Reporter: Jason Brown >Assignee: Jason Brown > Labels: encryption, security, sstable > Fix For: 3.x > > > Add option to allow encrypting of sstables. > I have a version of this functionality built on cassandra 2.0 that > piggy-backs on the existing sstable compression functionality and ICompressor > interface (similar in nature to what DataStax Enterprise does). However, if > we're adding the feature to the main OSS product, I'm not sure if we want to > use the pluggable compression framework or if it's worth investigating a > different path. I think there's a lot of upside in reusing the sstable > compression scheme, but perhaps add a new component in cqlsh for table > encryption and a corresponding field in CFMD. > Encryption configuration in the yaml can use the same mechanism as > CASSANDRA-6018 (which is currently pending internal review). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8911) Consider Mutation-based Repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272986#comment-15272986 ] Paulo Motta commented on CASSANDRA-8911: thanks for the input! IMO the number of threads for running repair should probably be configurable. my initial concern was if this was going to be a shared pool for all tables, or continuously-running globally-throttled pool-per-table, because if we need to prioritize between different tables/ranges on a shared pool this is already going to be provided by CASSANDRA-10070 so we can probably reuse that to automatize MBRs. but as you said, this is probably a side-concern that can be further discussed on CASSANDRA-10070 or another ticket. > Consider Mutation-based Repairs > --- > > Key: CASSANDRA-8911 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8911 > Project: Cassandra > Issue Type: Improvement >Reporter: Tyler Hobbs >Assignee: Marcus Eriksson > Fix For: 3.x > > > We should consider a mutation-based repair to replace the existing streaming > repair. While we're at it, we could do away with a lot of the complexity > around merkle trees. > I have not planned this out in detail, but here's roughly what I'm thinking: > * Instead of building an entire merkle tree up front, just send the "leaves" > one-by-one. Instead of dealing with token ranges, make the leaves primary > key ranges. The PK ranges would need to be contiguous, so that the start of > each range would match the end of the previous range. (The first and last > leaves would need to be open-ended on one end of the PK range.) This would be > similar to doing a read with paging. > * Once one page of data is read, compute a hash of it and send it to the > other replicas along with the PK range that it covers and a row count. > * When the replicas receive the hash, the perform a read over the same PK > range (using a LIMIT of the row count + 1) and compare hashes (unless the row > counts don't match, in which case this can be skipped). > * If there is a mismatch, the replica will send a mutation covering that > page's worth of data (ignoring the row count this time) to the source node. > Here are the advantages that I can think of: > * With the current repair behavior of streaming, vnode-enabled clusters may > need to stream hundreds of small SSTables. This results in increased compact > ion load on the receiving node. With the mutation-based approach, memtables > would naturally merge these. > * It's simple to throttle. For example, you could give a number of rows/sec > that should be repaired. > * It's easy to see what PK range has been repaired so far. This could make > it simpler to resume a repair that fails midway. > * Inconsistencies start to be repaired almost right away. > * Less special code \(?\) > * Wide partitions are no longer a problem. > There are a few problems I can think of: > * Counters. I don't know if this can be made safe, or if they need to be > skipped. > * To support incremental repair, we need to be able to read from only > repaired sstables. Probably not too difficult to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11721) Have a per operation truncate ddl "no snapshot" option
Jeremy Hanna created CASSANDRA-11721: Summary: Have a per operation truncate ddl "no snapshot" option Key: CASSANDRA-11721 URL: https://issues.apache.org/jira/browse/CASSANDRA-11721 Project: Cassandra Issue Type: Wish Components: CQL Reporter: Jeremy Hanna Priority: Minor Right now with truncate, it will always create a snapshot. That is the right thing to do most of the time. 'auto_snapshot' exists as an option to disable that but it is server wide and requires a restart to change. There are data models, however, that require rotating through a handful of tables and periodically truncating them. Currently you either have to operate with no safety net (some actually do this) or manually clear those snapshots out periodically. Both are less than optimal. In HDFS, you generally delete something where it goes to the trash. If you don't want that safety net, you can do something like 'rm -rf -skiptrash /jeremy/stuff' in one command. It would be nice to have something in the truncate ddl to skip the snapshot on a per operation basis. Perhaps 'TRUNCATE solarsystem.earth NO SNAPSHOT'. This might also be useful in those situations where you're just playing with data and you don't want something to take a snapshot in a development system. If that's the case, this would also be useful for the DROP operation, but that convenience is not the main reason for this option. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8911) Consider Mutation-based Repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272946#comment-15272946 ] Jeremiah Jordan commented on CASSANDRA-8911: If we are going to discuss it here, then for the single vs multi-thread thing, I know that DataStax OpsCenter does multi-threaded for their current repair service, because depending on your cluster size/data per node single threaded can't finish in 7 days. > Consider Mutation-based Repairs > --- > > Key: CASSANDRA-8911 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8911 > Project: Cassandra > Issue Type: Improvement >Reporter: Tyler Hobbs >Assignee: Marcus Eriksson > Fix For: 3.x > > > We should consider a mutation-based repair to replace the existing streaming > repair. While we're at it, we could do away with a lot of the complexity > around merkle trees. > I have not planned this out in detail, but here's roughly what I'm thinking: > * Instead of building an entire merkle tree up front, just send the "leaves" > one-by-one. Instead of dealing with token ranges, make the leaves primary > key ranges. The PK ranges would need to be contiguous, so that the start of > each range would match the end of the previous range. (The first and last > leaves would need to be open-ended on one end of the PK range.) This would be > similar to doing a read with paging. > * Once one page of data is read, compute a hash of it and send it to the > other replicas along with the PK range that it covers and a row count. > * When the replicas receive the hash, the perform a read over the same PK > range (using a LIMIT of the row count + 1) and compare hashes (unless the row > counts don't match, in which case this can be skipped). > * If there is a mismatch, the replica will send a mutation covering that > page's worth of data (ignoring the row count this time) to the source node. > Here are the advantages that I can think of: > * With the current repair behavior of streaming, vnode-enabled clusters may > need to stream hundreds of small SSTables. This results in increased compact > ion load on the receiving node. With the mutation-based approach, memtables > would naturally merge these. > * It's simple to throttle. For example, you could give a number of rows/sec > that should be repaired. > * It's easy to see what PK range has been repaired so far. This could make > it simpler to resume a repair that fails midway. > * Inconsistencies start to be repaired almost right away. > * Less special code \(?\) > * Wide partitions are no longer a problem. > There are a few problems I can think of: > * Counters. I don't know if this can be made safe, or if they need to be > skipped. > * To support incremental repair, we need to be able to read from only > repaired sstables. Probably not too difficult to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8969) Add indication in cassandra.yaml that rpc timeouts going too high will cause memory build up
[ https://issues.apache.org/jira/browse/CASSANDRA-8969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paulo Motta updated CASSANDRA-8969: --- Reviewer: Sylvain Lebresne (was: Paulo Motta) > Add indication in cassandra.yaml that rpc timeouts going too high will cause > memory build up > > > Key: CASSANDRA-8969 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8969 > Project: Cassandra > Issue Type: Improvement > Components: Configuration >Reporter: Jeremy Hanna >Assignee: Jeremy Hanna >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: 8969.txt > > > It would be helpful to communicate that setting the rpc timeouts too high may > cause memory problems on the server as it can become overloaded and has to > retain the in flight requests in memory. I'll get this done but just adding > the ticket as a placeholder for memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8969) Add indication in cassandra.yaml that rpc timeouts going too high will cause memory build up
[ https://issues.apache.org/jira/browse/CASSANDRA-8969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paulo Motta updated CASSANDRA-8969: --- Status: Awaiting Feedback (was: Open) > Add indication in cassandra.yaml that rpc timeouts going too high will cause > memory build up > > > Key: CASSANDRA-8969 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8969 > Project: Cassandra > Issue Type: Improvement > Components: Configuration >Reporter: Jeremy Hanna >Assignee: Jeremy Hanna >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: 8969.txt > > > It would be helpful to communicate that setting the rpc timeouts too high may > cause memory problems on the server as it can become overloaded and has to > retain the in flight requests in memory. I'll get this done but just adding > the ticket as a placeholder for memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8969) Add indication in cassandra.yaml that rpc timeouts going too high will cause memory build up
[ https://issues.apache.org/jira/browse/CASSANDRA-8969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paulo Motta updated CASSANDRA-8969: --- Status: Open (was: Patch Available) > Add indication in cassandra.yaml that rpc timeouts going too high will cause > memory build up > > > Key: CASSANDRA-8969 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8969 > Project: Cassandra > Issue Type: Improvement > Components: Configuration >Reporter: Jeremy Hanna >Assignee: Jeremy Hanna >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: 8969.txt > > > It would be helpful to communicate that setting the rpc timeouts too high may > cause memory problems on the server as it can become overloaded and has to > retain the in flight requests in memory. I'll get this done but just adding > the ticket as a placeholder for memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8911) Consider Mutation-based Repairs
[
https://issues.apache.org/jira/browse/CASSANDRA-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272917#comment-15272917
]
Paulo Motta commented on CASSANDRA-8911:
bq. not sure here, would be nice to be able to prioritise a given range for a
table, say we lose an sstable for example, automatically repairing the range
that sstable covered immediately.
bq. It should probably be just a single thread, one table after the other. And
then maybe having a priority queue or something with ranges to repair
immediately, wdyt Paulo Motta? This prio queue thing might be a bit of gold
plating that we could do later.
Sounds good! While we will remove the need for coordination between inter-node
repairs (if we replace traditional repairs with this in the future), we will
still need some level of management/scheduling for MBRs that we need to
consider when designing/implementing the repair scheduling/management framework
of CASSANDRA-10070, so it can also be useful to MBR in the future. With this
said, we should probably focus on core MBR functionality here and take any
auto-repair considerations to CASSANDRA-10070 so we can have a single interface
for managing repairs (MBRs or not).
As for incremental release of this, I agree that we should have a minimum level
of confidence that this is promising (at least for some use cases) before
releasing even if it's only experimental, but throwing it into the wild early
will certainly give us some important feedback that we can use to validate and
improve this. Furthermore if this proves out worthy it will probably live
alongside traditional repairs for a long time (maybe forever for
counters/DTCS?).
Some early bikeshedding here (since it's not ready for review yet), when
possible I think we should add new options to the existing repair interfaces
rather than creating specific interfaces for MBR (for instance, {{nodetool
repair --mutation-based}} instead of {{nodetool mutationbasedrepair}},
{{StorageService.repairAsync}} instead of {{CFS.enableMutationBasedRepair}}),
since this is just a different implementation of the same functionality (unless
there`s something that does not fit well or is not covered by existing
interfaces) and will also allow existing tools to try out MBR with little
change.
> Consider Mutation-based Repairs
> ---
>
> Key: CASSANDRA-8911
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8911
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tyler Hobbs
>Assignee: Marcus Eriksson
> Fix For: 3.x
>
>
> We should consider a mutation-based repair to replace the existing streaming
> repair. While we're at it, we could do away with a lot of the complexity
> around merkle trees.
> I have not planned this out in detail, but here's roughly what I'm thinking:
> * Instead of building an entire merkle tree up front, just send the "leaves"
> one-by-one. Instead of dealing with token ranges, make the leaves primary
> key ranges. The PK ranges would need to be contiguous, so that the start of
> each range would match the end of the previous range. (The first and last
> leaves would need to be open-ended on one end of the PK range.) This would be
> similar to doing a read with paging.
> * Once one page of data is read, compute a hash of it and send it to the
> other replicas along with the PK range that it covers and a row count.
> * When the replicas receive the hash, the perform a read over the same PK
> range (using a LIMIT of the row count + 1) and compare hashes (unless the row
> counts don't match, in which case this can be skipped).
> * If there is a mismatch, the replica will send a mutation covering that
> page's worth of data (ignoring the row count this time) to the source node.
> Here are the advantages that I can think of:
> * With the current repair behavior of streaming, vnode-enabled clusters may
> need to stream hundreds of small SSTables. This results in increased compact
> ion load on the receiving node. With the mutation-based approach, memtables
> would naturally merge these.
> * It's simple to throttle. For example, you could give a number of rows/sec
> that should be repaired.
> * It's easy to see what PK range has been repaired so far. This could make
> it simpler to resume a repair that fails midway.
> * Inconsistencies start to be repaired almost right away.
> * Less special code \(?\)
> * Wide partitions are no longer a problem.
> There are a few problems I can think of:
> * Counters. I don't know if this can be made safe, or if they need to be
> skipped.
> * To support incremental repair, we need to be able to read from only
> repaired sstables. Probably not too difficult to do.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11718) entry-weighers in QueryProcessor should respect partitionKeyBindIndexes field
[ https://issues.apache.org/jira/browse/CASSANDRA-11718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-11718: Reviewer: Benjamin Lerer > entry-weighers in QueryProcessor should respect partitionKeyBindIndexes field > - > > Key: CASSANDRA-11718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11718 > Project: Cassandra > Issue Type: Bug >Reporter: Robert Stupp >Assignee: Robert Stupp >Priority: Minor > Labels: lhf > Fix For: 3.x > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11717) ParsedStatement.Prepared#partitionKeyBindIndexes should be of type short[]
[
https://issues.apache.org/jira/browse/CASSANDRA-11717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-11717:
Reviewer: Benjamin Lerer
> ParsedStatement.Prepared#partitionKeyBindIndexes should be of type short[]
> --
>
> Key: CASSANDRA-11717
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11717
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
> Labels: lhf
> Fix For: 3.x
>
>
> {{o.a.c.cql3.statements.ParsedStatement.Prepared#partitionKeyBindIndexes}} is
> a {{Short[]}} but could easily be a {{short[]}}.
> The value of {{partitionKeyBindIndexes}} is always the result of
> {{o.a.c.cql3.VariableSpecifications#getPartitionKeyBindIndexes}}, which can
> never return a {{null}} array element.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11708) sstableloader not work with ssl options
[
https://issues.apache.org/jira/browse/CASSANDRA-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-11708:
Reviewer: Alex Petrov
> sstableloader not work with ssl options
> ---
>
> Key: CASSANDRA-11708
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11708
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Bin Chen
>Assignee: Yuki Morishita
>Priority: Minor
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> My cassandra server start with client_encryption_options.
> {code}
> client_encryption_options:
> enabled: true
> keystore: /path/to/keystore.node0
> keystore_password: cassandra
> require_client_auth: true
> truststore: /path/to/truststore.node0
> truststore_password: cassandra
> {code}
> when I use sstableloader with -ts/-tspw/-ks/-kspw, I get
> {code}
> All host(s) tried for query failed (tried: localhost/127.0.0.1:9042
> (com.datastax.driver.core.TransportException: [localhost/127.0.0.1:9042]
> Channel has been closed))
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: localhost/127.0.0.1:9042
> (com.datastax.driver.core.TransportException: [localhost/127.0.0.1:9042]
> Channel has been closed))
> {code}
> after looked at the code, I found only one way to active ssl is use -f with a
> configuration yaml file which has
> {code}
> client_encryption_options:
> enabled: true
> {code}
> so I think it's a bug, better to active ssl when I set -ts and -tspw.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11679) Cassandra Driver returns different number of results depending on fetchsize
[
https://issues.apache.org/jira/browse/CASSANDRA-11679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-11679:
Reviewer: Tyler Hobbs
> Cassandra Driver returns different number of results depending on fetchsize
> ---
>
> Key: CASSANDRA-11679
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11679
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Varun Barala
>Assignee: Benjamin Lerer
> Fix For: 2.1.x
>
> Attachments: 11679-2.1.txt
>
>
> I'm trying to fetch all distinct keys from a CF using cassandra-driver
> (2.1.7.1) and I observed some strange behavior :-
> The total distinct rows are 498 so If I perform a query get All distinctKeys
> It returns 503 instead of 498(five keys twice).
> But If I define the fetch size in select statement more than 498 then it
> returns exact 498 rows.
> And If I execute same statement on Dev-center it returns 498 rows (because
> the default fetch size is 5000). In `cqlsh` it returns 503 rows (because
> cqlsh uses fetch size=100).
> Some Additional and useful information :-
> ---
> Cassandra-2.1.13 (C)* version
> Consistency level: ONE
> local machine(ubuntu 14.04)
> Table Schema:-
> --
> {code:xml}
> CREATE TABLE sample (
> pk1 text,
> pk2 text,
> row_id uuid,
> value blob,
> PRIMARY KEY (( pk1, pk2))
> ) WITH bloom_filter_fp_chance = 0.01
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
> AND compression = {'sstable_compression':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {code}
> query :-
>
> {code:xml}
> SELECT DISTINCT pk2, pk1 FROM sample LIMIT 2147483647;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11031) MultiTenant : support “ALLOW FILTERING" for First Partition Key
[ https://issues.apache.org/jira/browse/CASSANDRA-11031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-11031: Reviewer: Sylvain Lebresne > MultiTenant : support “ALLOW FILTERING" for First Partition Key > --- > > Key: CASSANDRA-11031 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11031 > Project: Cassandra > Issue Type: New Feature > Components: CQL >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Minor > Fix For: 3.x > > Attachments: CASSANDRA-11031-3.7.patch > > > Currently, Allow Filtering only works for secondary Index column or > clustering columns. And it's slow, because Cassandra will read all data from > SSTABLE from hard-disk to memory to filter. > But we can support allow filtering on Partition Key, as far as I know, > Partition Key is in memory, so we can easily filter them, and then read > required data from SSTable. > This will similar to "Select * from table" which scan through entire cluster. > CREATE TABLE multi_tenant_table ( > tenant_id text, > pk2 text, > c1 text, > c2 text, > v1 text, > v2 text, > PRIMARY KEY ((tenant_id,pk2),c1,c2) > ) ; > Select * from multi_tenant_table where tenant_id = "datastax" allow filtering; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11706) Tracing payload not passed through newSession(..)
[
https://issues.apache.org/jira/browse/CASSANDRA-11706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-11706:
Reviewer: Alex Petrov
> Tracing payload not passed through newSession(..)
> -
>
> Key: CASSANDRA-11706
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11706
> Project: Cassandra
> Issue Type: Bug
>Reporter: mck
>Assignee: mck
>Priority: Minor
> Labels: tracing
> Fix For: 3.6
>
> Attachments: 11706-trunk.txt
>
>
> Caused by CASSANDRA-10392
> There's a small bug in
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/tracing/Tracing.java#L153
> {noformat}
> public UUID newSession(UUID sessionId, Map
> customPayload)
> {
> return newSession(sessionId, TraceType.QUERY, Collections.EMPTY_MAP);
> }{noformat}
> in that it passes on an {{EMPTY_MAP}} instead of the {{customPayload}}.
> I've marked this as "minor" as custom tracing plugins can easily enough
> workaround it by also overriding the {{newSession(UUID sessionId,
> Map customPayload)}} method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8911) Consider Mutation-based Repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272748#comment-15272748 ] Joshua McKenzie commented on CASSANDRA-8911: bq. you don't think we should release this incrementally? I'm all for us releasing something incrementally, I'm just concerned about the idea of us committing code w/out a clear idea of the benefits it adds and the strong possibility of having to remove it in the future. I'd expect we should be able to show the value of each step in the chain you've mentioned. I dislike the idea of us adding this complexity and another knob into the code-base if we're not clear that it has value either over or alongside the traditional repairs. > Consider Mutation-based Repairs > --- > > Key: CASSANDRA-8911 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8911 > Project: Cassandra > Issue Type: Improvement >Reporter: Tyler Hobbs >Assignee: Marcus Eriksson > Fix For: 3.x > > > We should consider a mutation-based repair to replace the existing streaming > repair. While we're at it, we could do away with a lot of the complexity > around merkle trees. > I have not planned this out in detail, but here's roughly what I'm thinking: > * Instead of building an entire merkle tree up front, just send the "leaves" > one-by-one. Instead of dealing with token ranges, make the leaves primary > key ranges. The PK ranges would need to be contiguous, so that the start of > each range would match the end of the previous range. (The first and last > leaves would need to be open-ended on one end of the PK range.) This would be > similar to doing a read with paging. > * Once one page of data is read, compute a hash of it and send it to the > other replicas along with the PK range that it covers and a row count. > * When the replicas receive the hash, the perform a read over the same PK > range (using a LIMIT of the row count + 1) and compare hashes (unless the row > counts don't match, in which case this can be skipped). > * If there is a mismatch, the replica will send a mutation covering that > page's worth of data (ignoring the row count this time) to the source node. > Here are the advantages that I can think of: > * With the current repair behavior of streaming, vnode-enabled clusters may > need to stream hundreds of small SSTables. This results in increased compact > ion load on the receiving node. With the mutation-based approach, memtables > would naturally merge these. > * It's simple to throttle. For example, you could give a number of rows/sec > that should be repaired. > * It's easy to see what PK range has been repaired so far. This could make > it simpler to resume a repair that fails midway. > * Inconsistencies start to be repaired almost right away. > * Less special code \(?\) > * Wide partitions are no longer a problem. > There are a few problems I can think of: > * Counters. I don't know if this can be made safe, or if they need to be > skipped. > * To support incremental repair, we need to be able to read from only > repaired sstables. Probably not too difficult to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8911) Consider Mutation-based Repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272689#comment-15272689 ] Jeremiah Jordan edited comment on CASSANDRA-8911 at 5/5/16 5:35 PM: bq. not sure here, would be nice to be able to prioritise a given range for a table, say we lose an sstable for example, automatically repairing the range that sstable covered immediately. bq. It should probably be just a single thread, one table after the other. And then maybe having a priority queue or something with ranges to repair immediately, wdyt Paulo Motta? This prio queue thing might be a bit of gold plating that we could do later. This discussion should probably be on CASSANDRA-10070? was (Author: jjordan): bq. not sure here, would be nice to be able to prioritise a given range for a table, say we lose an sstable for example, automatically repairing the range that sstable covered immediately. bq. It should probably be just a single thread, one table after the other. And then maybe having a priority queue or something with ranges to repair immediately, wdyt Paulo Motta? This prio queue thing might be a bit of gold plating that we could do later. This discussion should probably be on the other ticket? > Consider Mutation-based Repairs > --- > > Key: CASSANDRA-8911 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8911 > Project: Cassandra > Issue Type: Improvement >Reporter: Tyler Hobbs >Assignee: Marcus Eriksson > Fix For: 3.x > > > We should consider a mutation-based repair to replace the existing streaming > repair. While we're at it, we could do away with a lot of the complexity > around merkle trees. > I have not planned this out in detail, but here's roughly what I'm thinking: > * Instead of building an entire merkle tree up front, just send the "leaves" > one-by-one. Instead of dealing with token ranges, make the leaves primary > key ranges. The PK ranges would need to be contiguous, so that the start of > each range would match the end of the previous range. (The first and last > leaves would need to be open-ended on one end of the PK range.) This would be > similar to doing a read with paging. > * Once one page of data is read, compute a hash of it and send it to the > other replicas along with the PK range that it covers and a row count. > * When the replicas receive the hash, the perform a read over the same PK > range (using a LIMIT of the row count + 1) and compare hashes (unless the row > counts don't match, in which case this can be skipped). > * If there is a mismatch, the replica will send a mutation covering that > page's worth of data (ignoring the row count this time) to the source node. > Here are the advantages that I can think of: > * With the current repair behavior of streaming, vnode-enabled clusters may > need to stream hundreds of small SSTables. This results in increased compact > ion load on the receiving node. With the mutation-based approach, memtables > would naturally merge these. > * It's simple to throttle. For example, you could give a number of rows/sec > that should be repaired. > * It's easy to see what PK range has been repaired so far. This could make > it simpler to resume a repair that fails midway. > * Inconsistencies start to be repaired almost right away. > * Less special code \(?\) > * Wide partitions are no longer a problem. > There are a few problems I can think of: > * Counters. I don't know if this can be made safe, or if they need to be > skipped. > * To support incremental repair, we need to be able to read from only > repaired sstables. Probably not too difficult to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8911) Consider Mutation-based Repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272689#comment-15272689 ] Jeremiah Jordan commented on CASSANDRA-8911: bq. not sure here, would be nice to be able to prioritise a given range for a table, say we lose an sstable for example, automatically repairing the range that sstable covered immediately. bq. It should probably be just a single thread, one table after the other. And then maybe having a priority queue or something with ranges to repair immediately, wdyt Paulo Motta? This prio queue thing might be a bit of gold plating that we could do later. This discussion should probably be on the other ticket? > Consider Mutation-based Repairs > --- > > Key: CASSANDRA-8911 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8911 > Project: Cassandra > Issue Type: Improvement >Reporter: Tyler Hobbs >Assignee: Marcus Eriksson > Fix For: 3.x > > > We should consider a mutation-based repair to replace the existing streaming > repair. While we're at it, we could do away with a lot of the complexity > around merkle trees. > I have not planned this out in detail, but here's roughly what I'm thinking: > * Instead of building an entire merkle tree up front, just send the "leaves" > one-by-one. Instead of dealing with token ranges, make the leaves primary > key ranges. The PK ranges would need to be contiguous, so that the start of > each range would match the end of the previous range. (The first and last > leaves would need to be open-ended on one end of the PK range.) This would be > similar to doing a read with paging. > * Once one page of data is read, compute a hash of it and send it to the > other replicas along with the PK range that it covers and a row count. > * When the replicas receive the hash, the perform a read over the same PK > range (using a LIMIT of the row count + 1) and compare hashes (unless the row > counts don't match, in which case this can be skipped). > * If there is a mismatch, the replica will send a mutation covering that > page's worth of data (ignoring the row count this time) to the source node. > Here are the advantages that I can think of: > * With the current repair behavior of streaming, vnode-enabled clusters may > need to stream hundreds of small SSTables. This results in increased compact > ion load on the receiving node. With the mutation-based approach, memtables > would naturally merge these. > * It's simple to throttle. For example, you could give a number of rows/sec > that should be repaired. > * It's easy to see what PK range has been repaired so far. This could make > it simpler to resume a repair that fails midway. > * Inconsistencies start to be repaired almost right away. > * Less special code \(?\) > * Wide partitions are no longer a problem. > There are a few problems I can think of: > * Counters. I don't know if this can be made safe, or if they need to be > skipped. > * To support incremental repair, we need to be able to read from only > repaired sstables. Probably not too difficult to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8911) Consider Mutation-based Repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272675#comment-15272675 ] Marcus Eriksson commented on CASSANDRA-8911: bq. is the idea to have one thread per table running continuously, or a fixed-size pool for all tables? If the second option we may still need to prioritize which table should run MBR next. not sure here, would be nice to be able to prioritise a given range for a table, say we lose an sstable for example, automatically repairing the range that sstable covered immediately. It should probably be just a single thread, one table after the other. And then maybe having a priority queue or something with ranges to repair immediately, wdyt [~pauloricardomg]? This prio queue thing might be a bit of gold plating that we could do later. > Consider Mutation-based Repairs > --- > > Key: CASSANDRA-8911 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8911 > Project: Cassandra > Issue Type: Improvement >Reporter: Tyler Hobbs >Assignee: Marcus Eriksson > Fix For: 3.x > > > We should consider a mutation-based repair to replace the existing streaming > repair. While we're at it, we could do away with a lot of the complexity > around merkle trees. > I have not planned this out in detail, but here's roughly what I'm thinking: > * Instead of building an entire merkle tree up front, just send the "leaves" > one-by-one. Instead of dealing with token ranges, make the leaves primary > key ranges. The PK ranges would need to be contiguous, so that the start of > each range would match the end of the previous range. (The first and last > leaves would need to be open-ended on one end of the PK range.) This would be > similar to doing a read with paging. > * Once one page of data is read, compute a hash of it and send it to the > other replicas along with the PK range that it covers and a row count. > * When the replicas receive the hash, the perform a read over the same PK > range (using a LIMIT of the row count + 1) and compare hashes (unless the row > counts don't match, in which case this can be skipped). > * If there is a mismatch, the replica will send a mutation covering that > page's worth of data (ignoring the row count this time) to the source node. > Here are the advantages that I can think of: > * With the current repair behavior of streaming, vnode-enabled clusters may > need to stream hundreds of small SSTables. This results in increased compact > ion load on the receiving node. With the mutation-based approach, memtables > would naturally merge these. > * It's simple to throttle. For example, you could give a number of rows/sec > that should be repaired. > * It's easy to see what PK range has been repaired so far. This could make > it simpler to resume a repair that fails midway. > * Inconsistencies start to be repaired almost right away. > * Less special code \(?\) > * Wide partitions are no longer a problem. > There are a few problems I can think of: > * Counters. I don't know if this can be made safe, or if they need to be > skipped. > * To support incremental repair, we need to be able to read from only > repaired sstables. Probably not too difficult to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8911) Consider Mutation-based Repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272661#comment-15272661 ] Marcus Eriksson commented on CASSANDRA-8911: bq. We shouldn't commit anything to the code-base if it doesn't work in production environments That is not what I meant, the nodetool command should of course work as advertised, but we might remove it if the benefit over old style repairs are too small for example. [~JoshuaMcKenzie] so, you don't think we should release this incrementally? Being able to release small but still usable things is a good thing imo. > Consider Mutation-based Repairs > --- > > Key: CASSANDRA-8911 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8911 > Project: Cassandra > Issue Type: Improvement >Reporter: Tyler Hobbs >Assignee: Marcus Eriksson > Fix For: 3.x > > > We should consider a mutation-based repair to replace the existing streaming > repair. While we're at it, we could do away with a lot of the complexity > around merkle trees. > I have not planned this out in detail, but here's roughly what I'm thinking: > * Instead of building an entire merkle tree up front, just send the "leaves" > one-by-one. Instead of dealing with token ranges, make the leaves primary > key ranges. The PK ranges would need to be contiguous, so that the start of > each range would match the end of the previous range. (The first and last > leaves would need to be open-ended on one end of the PK range.) This would be > similar to doing a read with paging. > * Once one page of data is read, compute a hash of it and send it to the > other replicas along with the PK range that it covers and a row count. > * When the replicas receive the hash, the perform a read over the same PK > range (using a LIMIT of the row count + 1) and compare hashes (unless the row > counts don't match, in which case this can be skipped). > * If there is a mismatch, the replica will send a mutation covering that > page's worth of data (ignoring the row count this time) to the source node. > Here are the advantages that I can think of: > * With the current repair behavior of streaming, vnode-enabled clusters may > need to stream hundreds of small SSTables. This results in increased compact > ion load on the receiving node. With the mutation-based approach, memtables > would naturally merge these. > * It's simple to throttle. For example, you could give a number of rows/sec > that should be repaired. > * It's easy to see what PK range has been repaired so far. This could make > it simpler to resume a repair that fails midway. > * Inconsistencies start to be repaired almost right away. > * Less special code \(?\) > * Wide partitions are no longer a problem. > There are a few problems I can think of: > * Counters. I don't know if this can be made safe, or if they need to be > skipped. > * To support incremental repair, we need to be able to read from only > repaired sstables. Probably not too difficult to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8377) Coordinated Commitlog Replay
[ https://issues.apache.org/jira/browse/CASSANDRA-8377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272648#comment-15272648 ] Nick Bailey commented on CASSANDRA-8377: Not sure if Chris has had time to do anything with this lately but I don't think we should close this ticket if thats what you mean. > Coordinated Commitlog Replay > > > Key: CASSANDRA-8377 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8377 > Project: Cassandra > Issue Type: New Feature >Reporter: Nick Bailey >Assignee: Chris Lohfink > Fix For: 3.x > > Attachments: CASSANDRA-8377.txt > > > Commit log archiving and replay can be used to support point in time restores > on a cluster. Unfortunately, at the moment that is only true when the > topology of the cluster is exactly the same as when the commitlogs were > archived. This is because commitlogs need to be replayed on a node that is a > replica for those writes. > To support replaying commitlogs when the topology has changed we should have > a tool that replays the writes in a commitlog as if they were writes from a > client and will get coordinated to the correct replicas. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11713) Add ability to log thread dump when NTR pool is blocked
[
https://issues.apache.org/jira/browse/CASSANDRA-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-11713:
Reviewer: Joshua McKenzie
> Add ability to log thread dump when NTR pool is blocked
> ---
>
> Key: CASSANDRA-11713
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11713
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability
>Reporter: Paulo Motta
>Assignee: Paulo Motta
>Priority: Minor
>
> Thread dumps are very useful for troubleshooting Native-Transport-Requests
> contention issues like CASSANDRA-11363 and CASSANDRA-11529.
> While they could be generated externally with {{jstack}}, sometimes the
> conditions are transient and it's hard to catch the exact moment when they
> happen, so it could be useful to generate and log them upon user request when
> certain internal condition happens.
> I propose adding a {{logThreadDumpOnNextContention}} flag to {{SEPExecutor}}
> that when enabled via JMX generates and logs a single thread dump on the
> system log when the thread pool queue is full.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8911) Consider Mutation-based Repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272630#comment-15272630 ] Joshua McKenzie commented on CASSANDRA-8911: bq. Reason I prefer to release it incrementally like this is that I don't want us to waste a lot of time if the approach does not really work in real life We shouldn't commit anything to the code-base if it doesn't work in production environments, incremental release and experimental feature or not. Certainly we can test this to determine if it's viable from a performance perspective and give recommendations regarding its limitations in general usage before committing. > Consider Mutation-based Repairs > --- > > Key: CASSANDRA-8911 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8911 > Project: Cassandra > Issue Type: Improvement >Reporter: Tyler Hobbs >Assignee: Marcus Eriksson > Fix For: 3.x > > > We should consider a mutation-based repair to replace the existing streaming > repair. While we're at it, we could do away with a lot of the complexity > around merkle trees. > I have not planned this out in detail, but here's roughly what I'm thinking: > * Instead of building an entire merkle tree up front, just send the "leaves" > one-by-one. Instead of dealing with token ranges, make the leaves primary > key ranges. The PK ranges would need to be contiguous, so that the start of > each range would match the end of the previous range. (The first and last > leaves would need to be open-ended on one end of the PK range.) This would be > similar to doing a read with paging. > * Once one page of data is read, compute a hash of it and send it to the > other replicas along with the PK range that it covers and a row count. > * When the replicas receive the hash, the perform a read over the same PK > range (using a LIMIT of the row count + 1) and compare hashes (unless the row > counts don't match, in which case this can be skipped). > * If there is a mismatch, the replica will send a mutation covering that > page's worth of data (ignoring the row count this time) to the source node. > Here are the advantages that I can think of: > * With the current repair behavior of streaming, vnode-enabled clusters may > need to stream hundreds of small SSTables. This results in increased compact > ion load on the receiving node. With the mutation-based approach, memtables > would naturally merge these. > * It's simple to throttle. For example, you could give a number of rows/sec > that should be repaired. > * It's easy to see what PK range has been repaired so far. This could make > it simpler to resume a repair that fails midway. > * Inconsistencies start to be repaired almost right away. > * Less special code \(?\) > * Wide partitions are no longer a problem. > There are a few problems I can think of: > * Counters. I don't know if this can be made safe, or if they need to be > skipped. > * To support incremental repair, we need to be able to read from only > repaired sstables. Probably not too difficult to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9712) Refactor CFMetaData
[
https://issues.apache.org/jira/browse/CASSANDRA-9712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-9712:
Reviewer: (was: Robert Stupp)
> Refactor CFMetaData
> ---
>
> Key: CASSANDRA-9712
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9712
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
> Fix For: 3.x
>
>
> As part of CASSANDRA-9425 and a follow-up to CASSANDRA-9665, and a
> pre-requisite for new schema change protocol, this ticket will do the
> following
> 1. Make the triggers {{HashMap}} immutable (new {{Triggers}} class)
> 2. Allow multiple 2i definitions per column in CFMetaData
> 3.
> 4. Rename and move {{config.CFMetaData}} to {{schema.TableMetadata}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8377) Coordinated Commitlog Replay
[ https://issues.apache.org/jira/browse/CASSANDRA-8377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272611#comment-15272611 ] T Jake Luciani commented on CASSANDRA-8377: --- Any update or should we close this out? > Coordinated Commitlog Replay > > > Key: CASSANDRA-8377 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8377 > Project: Cassandra > Issue Type: New Feature >Reporter: Nick Bailey >Assignee: Chris Lohfink > Fix For: 3.x > > Attachments: CASSANDRA-8377.txt > > > Commit log archiving and replay can be used to support point in time restores > on a cluster. Unfortunately, at the moment that is only true when the > topology of the cluster is exactly the same as when the commitlogs were > archived. This is because commitlogs need to be replayed on a node that is a > replica for those writes. > To support replaying commitlogs when the topology has changed we should have > a tool that replays the writes in a commitlog as if they were writes from a > client and will get coordinated to the correct replicas. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10984) Cassandra should not depend on netty-all
[
https://issues.apache.org/jira/browse/CASSANDRA-10984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
T Jake Luciani updated CASSANDRA-10984:
---
Resolution: Won't Fix
Status: Resolved (was: Patch Available)
> Cassandra should not depend on netty-all
>
>
> Key: CASSANDRA-10984
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10984
> Project: Cassandra
> Issue Type: Improvement
>Reporter: James Roper
>Assignee: Alex Petrov
>Priority: Minor
> Attachments:
> 0001-Use-separate-netty-depencencies-instead-of-netty-all.patch,
> 0001-with-binaries.patch
>
>
> netty-all is a jar that bundles all the individual netty dependencies for
> convenience together for people trying out netty to get started quickly.
> Serious projects like Cassandra should never ever ever use it, since it's a
> recipe for classpath disasters.
> To illustrate, I'm running Cassandra embedded in an app, and I get this error:
> {noformat}
> [JVM-1] java.lang.NoSuchMethodError:
> io.netty.util.internal.PlatformDependent.newLongCounter()Lio/netty/util/internal/LongCounter;
> [JVM-1] at io.netty.buffer.PoolArena.(PoolArena.java:64)
> ~[netty-buffer-4.0.33.Final.jar:4.0.33.Final]
> [JVM-1] at
> io.netty.buffer.PoolArena$HeapArena.(PoolArena.java:593)
> ~[netty-buffer-4.0.33.Final.jar:4.0.33.Final]
> [JVM-1] at
> io.netty.buffer.PooledByteBufAllocator.(PooledByteBufAllocator.java:179)
> ~[netty-buffer-4.0.33.Final.jar:4.0.33.Final]
> [JVM-1] at
> io.netty.buffer.PooledByteBufAllocator.(PooledByteBufAllocator.java:153)
> ~[netty-buffer-4.0.33.Final.jar:4.0.33.Final]
> [JVM-1] at
> io.netty.buffer.PooledByteBufAllocator.(PooledByteBufAllocator.java:145)
> ~[netty-buffer-4.0.33.Final.jar:4.0.33.Final]
> [JVM-1] at
> io.netty.buffer.PooledByteBufAllocator.(PooledByteBufAllocator.java:128)
> ~[netty-buffer-4.0.33.Final.jar:4.0.33.Final]
> [JVM-1] at
> org.apache.cassandra.transport.CBUtil.(CBUtil.java:56)
> ~[cassandra-all-3.0.0.jar:3.0.0]
> [JVM-1] at org.apache.cassandra.transport.Server.start(Server.java:134)
> ~[cassandra-all-3.0.0.jar:3.0.0]
> {noformat}
> {{PlatformDependent}} comes from netty-common, of which version 4.0.33 is on
> the classpath, but it's also provided by netty-all, which has version 4.0.23
> brought in by cassandra. By a fluke of classpath ordering, the classloader
> has loaded the netty buffer classes from netty-buffer 4.0.33, but the
> PlatformDependent class from netty-all 4.0.23, and these two versions are not
> binary compatible, hence the linkage error.
> Essentially to avoid these problems in serious projects, anyone that ever
> brings in cassandra is going to have to exclude the netty dependency from it,
> which is error prone, and when you get it wrong, due to the nature of
> classpath ordering bugs, it might not be till you deploy to production that
> you actually find out there's a problem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11425) Add prepared query parameter to trace for "Execute CQL3 prepared query" session
[
https://issues.apache.org/jira/browse/CASSANDRA-11425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-11425:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
Patch LGTM.
Adding rawCQLStatement to weighing is fine.
To verify that the patch works as expected, I created a simple unit test
(please add one yourself in the future ;) ).
While reviewing your patch, I found that the old _TODO_ in
{{org.apache.cassandra.transport.messages.ExecuteMessage#execute}} is now
possible and opened CASSANDRA-11719 for this. [~Yasuharu] Do you want to tackle
11719, too?
[Rebased branch plus unit
test|https://github.com/apache/cassandra/compare/trunk...snazy:11425-trunk]
[testall|http://cassci.datastax.com/job/snazy-11425-trunk-testall/lastBuild/]
[dtest|http://cassci.datastax.com/job/snazy-11425-trunk-dtest/lastBuild/]
Utests + dtests look good. Committed as
07385b6ce90d3a230f00f2812b22e3ff158cc2d6 to trunk.
Thanks for the patch!
> Add prepared query parameter to trace for "Execute CQL3 prepared query"
> session
> ---
>
> Key: CASSANDRA-11425
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11425
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Yasuharu Goto
>Assignee: Yasuharu Goto
>Priority: Minor
>
> For now, the system_traces.sessions rows for "Execute CQL3 prepared query" do
> not show us any information about the prepared query which is executed on the
> session. So we can't see what query is the session executing.
> I think this makes performance tuning difficult on Cassandra.
> So, In this ticket, I'd like to add the prepared query parameter on Execute
> session trace like this.
> {noformat}
> cqlsh:system_traces> select * from sessions ;
> session_id | client| command | coordinator |
> duration | parameters
>
> | request | started_at
> --+---+-+-+--+--+-+-
> a001ec00-f1c5-11e5-b14a-6fe1292cf9f1 | 127.0.0.1 | QUERY | 127.0.0.1 |
>666 | \{'consistency_level': 'ONE', 'page_size': '5000', 'query':
> 'SELECT * FROM test.test2 WHERE id=? LIMIT 1', 'serial_consistency_level':
> 'SERIAL'\} | Execute CQL3 prepared query | 2016-03-24 13:38:00.00+
> a0019de0-f1c5-11e5-b14a-6fe1292cf9f1 | 127.0.0.1 | QUERY | 127.0.0.1 |
>109 |
>{'query': 'SELECT * FROM test.test2 WHERE id=? LIMIT
> 1'} |Preparing CQL3 query | 2016-03-24 13:37:59.998000+
> a0014fc0-f1c5-11e5-b14a-6fe1292cf9f1 | 127.0.0.1 | QUERY | 127.0.0.1 |
>126 |
> {'query': 'INSERT INTO test.test2(id,value) VALUES
> (?,?)'} |Preparing CQL3 query | 2016-03-24 13:37:59.996000+
> a0019de1-f1c5-11e5-b14a-6fe1292cf9f1 | 127.0.0.1 | QUERY | 127.0.0.1 |
>764 | {'consistency_level': 'ONE', 'page_size': '5000', 'query':
> 'SELECT * FROM test.test2 WHERE id=? LIMIT 1', 'serial_consistency_level':
> 'SERIAL'} | Execute CQL3 prepared query | 2016-03-24 13:37:59.998000+
> a00176d0-f1c5-11e5-b14a-6fe1292cf9f1 | 127.0.0.1 | QUERY | 127.0.0.1 |
>857 | {'consistency_level': 'QUORUM', 'page_size': '5000', 'query':
> 'INSERT INTO test.test2(id,value) VALUES (?,?)', 'serial_consistency_level':
> 'SERIAL'} | Execute CQL3 prepared query | 2016-03-24 13:37:59.997000+
> {noformat}
> Now, "Execute CQL3 prepared query" session displays its query.
> I believe that this additional information would help operators a lot.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
cassandra git commit: Add prepared query parameter to trace for "Execute CQL3 prepared query" session
Repository: cassandra
Updated Branches:
refs/heads/trunk b57b047e5 -> 07385b6ce
Add prepared query parameter to trace for "Execute CQL3 prepared query" session
patch by Yasuharu Goto; reviewed by Robert Stupp for CASSANDRA-11425
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/07385b6c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/07385b6c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/07385b6c
Branch: refs/heads/trunk
Commit: 07385b6ce90d3a230f00f2812b22e3ff158cc2d6
Parents: b57b047
Author: Yasuharu Goto
Authored: Thu May 5 18:46:05 2016 +0200
Committer: Robert Stupp
Committed: Thu May 5 18:46:05 2016 +0200
--
.../apache/cassandra/cql3/QueryProcessor.java | 5 +-
.../cql3/statements/ParsedStatement.java| 9
.../transport/messages/ExecuteMessage.java | 1 +
.../org/apache/cassandra/cql3/TraceCqlTest.java | 51
4 files changed, 64 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/07385b6c/src/java/org/apache/cassandra/cql3/QueryProcessor.java
--
diff --git a/src/java/org/apache/cassandra/cql3/QueryProcessor.java
b/src/java/org/apache/cassandra/cql3/QueryProcessor.java
index 39a9c03..feedfc4 100644
--- a/src/java/org/apache/cassandra/cql3/QueryProcessor.java
+++ b/src/java/org/apache/cassandra/cql3/QueryProcessor.java
@@ -71,7 +71,7 @@ public class QueryProcessor implements QueryHandler
@Override
public int weightOf(MD5Digest key, ParsedStatement.Prepared value)
{
-return Ints.checkedCast(measure(key) + measure(value.statement) +
measure(value.boundNames));
+return Ints.checkedCast(measure(key) +
measure(value.rawCQLStatement) + measure(value.statement) +
measure(value.boundNames));
}
};
@@ -80,7 +80,7 @@ public class QueryProcessor implements QueryHandler
@Override
public int weightOf(Integer key, ParsedStatement.Prepared value)
{
-return Ints.checkedCast(measure(key) + measure(value.statement) +
measure(value.boundNames));
+return Ints.checkedCast(measure(key) +
measure(value.rawCQLStatement) + measure(value.statement) +
measure(value.boundNames));
}
};
@@ -386,6 +386,7 @@ public class QueryProcessor implements QueryHandler
return existing;
ParsedStatement.Prepared prepared = getStatement(queryString,
clientState);
+prepared.rawCQLStatement = queryString;
int boundTerms = prepared.statement.getBoundTerms();
if (boundTerms > FBUtilities.MAX_UNSIGNED_SHORT)
throw new InvalidRequestException(String.format("Too many
markers(?). %d markers exceed the allowed maximum of %d", boundTerms,
FBUtilities.MAX_UNSIGNED_SHORT));
http://git-wip-us.apache.org/repos/asf/cassandra/blob/07385b6c/src/java/org/apache/cassandra/cql3/statements/ParsedStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/ParsedStatement.java
b/src/java/org/apache/cassandra/cql3/statements/ParsedStatement.java
index 4c3f8a9..5f6c42c 100644
--- a/src/java/org/apache/cassandra/cql3/statements/ParsedStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/ParsedStatement.java
@@ -48,6 +48,14 @@ public abstract class ParsedStatement
public static class Prepared
{
+/**
+ * Contains the CQL statement source if the statement has been
"regularly" perpared via
+ * {@link
org.apache.cassandra.cql3.QueryProcessor#prepare(java.lang.String,
org.apache.cassandra.service.ClientState, boolean)} /
+ * {@link QueryHandler#prepare(java.lang.String,
org.apache.cassandra.service.QueryState, java.util.Map)}.
+ * Other usages of this class may or may not contain the CQL statement
source.
+ */
+public String rawCQLStatement;
+
public final CQLStatement statement;
public final List boundNames;
public final Short[] partitionKeyBindIndexes;
@@ -57,6 +65,7 @@ public abstract class ParsedStatement
this.statement = statement;
this.boundNames = boundNames;
this.partitionKeyBindIndexes = partitionKeyBindIndexes;
+this.rawCQLStatement = "";
}
public Prepared(CQLStatement statement, VariableSpecifications names,
Short[] partitionKeyBindIndexes)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/07385b6c/src/java/org/apache/cassandra/transport/messages/ExecuteMessage.java
--
diff --git
a/src/java/org/apache/cassandra/transport/messages/Execut
[jira] [Commented] (CASSANDRA-11615) cassandra-stress blocks when connecting to a big cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-11615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272600#comment-15272600
]
T Jake Luciani commented on CASSANDRA-11615:
[branch| https://github.com/tjake/cassandra/tree/update-driver-3.0.1]
[testall|http://cassci.datastax.com/job/tjake-update-driver-3.0.1-testall/]
[dtest|http://cassci.datastax.com/job/tjake-update-driver-3.0.1-dtest/]
> cassandra-stress blocks when connecting to a big cluster
>
>
> Key: CASSANDRA-11615
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11615
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Eduard Tudenhoefner
>Assignee: Eduard Tudenhoefner
> Fix For: 3.0.x
>
> Attachments: 11615-3.0-2nd.patch, 11615-3.0.patch
>
>
> I had a *100* node cluster and was running
> {code}
> cassandra-stress read n=100 no-warmup cl=LOCAL_QUORUM -rate 'threads=20'
> 'limit=1000/s'
> {code}
> Based on the thread dump it looks like it's been blocked at
> https://github.com/apache/cassandra/blob/cassandra-3.0/tools/stress/src/org/apache/cassandra/stress/util/JavaDriverClient.java#L96
> {code}
> "Thread-20" #245 prio=5 os_prio=0 tid=0x7f3781822000 nid=0x46c4 waiting
> for monitor entry [0x7f36cc788000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.cassandra.stress.util.JavaDriverClient.prepare(JavaDriverClient.java:96)
> - waiting to lock <0x0005c003d920> (a
> java.util.concurrent.ConcurrentHashMap)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation$JavaDriverWrapper.createPreparedStatement(CqlOperation.java:314)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:77)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:109)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:261)
> at
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:327)
> "Thread-19" #244 prio=5 os_prio=0 tid=0x7f378182 nid=0x46c3 waiting
> for monitor entry [0x7f36cc889000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.cassandra.stress.util.JavaDriverClient.prepare(JavaDriverClient.java:96)
> - waiting to lock <0x0005c003d920> (a
> java.util.concurrent.ConcurrentHashMap)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation$JavaDriverWrapper.createPreparedStatement(CqlOperation.java:314)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:77)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:109)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:261)
> at
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:327)
> {code}
> I was trying the same with with a smaller cluster (50 nodes) and it was
> working fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11626) cqlsh fails and exits on non-ascii chars
[
https://issues.apache.org/jira/browse/CASSANDRA-11626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tyler Hobbs updated CASSANDRA-11626:
Resolution: Fixed
Fix Version/s: (was: 3.0.x)
(was: 2.2.x)
(was: 3.x)
3.0.7
3.7
2.2.7
Reproduced In: 3.5, 3.0.4 (was: 3.0.4, 3.5)
Status: Resolved (was: Ready to Commit)
Thanks, committed as
[{{5de9de1f5832f2a0e92783e2f4412874423e6e15}}|https://github.com/apache/cassandra/commit/5de9de1f5832f2a0e92783e2f4412874423e6e15]
to 2.2 and merged up to 3.0, 3.7, and trunk.
> cqlsh fails and exits on non-ascii chars
>
>
> Key: CASSANDRA-11626
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11626
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Robert Stupp
>Assignee: Tyler Hobbs
>Priority: Minor
> Labels: cqlsh
> Fix For: 2.2.7, 3.7, 3.0.7
>
>
> Just seen on cqlsh on current trunk:
> To repro, copy {{ä}} (german umlaut) to cqlsh and press return.
> cqlsh errors out and immediately exits.
> {code}
> $ bin/cqlsh
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 2.1.13-SNAPSHOT | CQL spec 3.2.1 | Native protocol
> v3]
> Use HELP for help.
> cqlsh> ä
> Invalid syntax at line 1, char 1
> Traceback (most recent call last):
> File "/Users/snazy/devel/cassandra/trunk/bin/cqlsh.py", line 2636, in
>
> main(*read_options(sys.argv[1:], os.environ))
> File "/Users/snazy/devel/cassandra/trunk/bin/cqlsh.py", line 2625, in main
> shell.cmdloop()
> File "/Users/snazy/devel/cassandra/trunk/bin/cqlsh.py", line 1114, in
> cmdloop
> if self.onecmd(self.statement.getvalue()):
> File "/Users/snazy/devel/cassandra/trunk/bin/cqlsh.py", line 1139, in onecmd
> self.printerr(' %s' % statementline)
> File "/Users/snazy/devel/cassandra/trunk/bin/cqlsh.py", line 2314, in
> printerr
> self.writeresult(text, color, newline=newline, out=sys.stderr)
> File "/Users/snazy/devel/cassandra/trunk/bin/cqlsh.py", line 2303, in
> writeresult
> out.write(self.applycolor(str(text), color) + ('\n' if newline else ''))
> UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position
> 2: ordinal not in range(128)
> $
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[4/4] cassandra git commit: Merge branch 'cassandra-3.7' into trunk
Merge branch 'cassandra-3.7' into trunk Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b57b047e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b57b047e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b57b047e Branch: refs/heads/trunk Commit: b57b047e54fff01d904893ae68f0a607c7fda2e7 Parents: 1e92ce4 58d3b9a Author: Tyler Hobbs Authored: Thu May 5 11:37:17 2016 -0500 Committer: Tyler Hobbs Committed: Thu May 5 11:37:17 2016 -0500 -- CHANGES.txt| 6 ++ bin/cqlsh.py | 19 ++- pylib/cqlshlib/copyutil.py | 16 ++-- 3 files changed, 30 insertions(+), 11 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b57b047e/CHANGES.txt -- diff --cc CHANGES.txt index c2165db,882be7c..45d9fb1 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,6 -1,7 +1,12 @@@ +3.8 + * Faster streaming (CASSANDRA-9766) + ++ + 3.7 + Merged from 2.2: + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626) + ++ 3.6 * Enhanced Compaction Logging (CASSANDRA-10805) * Make prepared statement cache size configurable (CASSANDRA-11555)
[1/4] cassandra git commit: cqlsh: Handle non-ascii chars in error messages
Repository: cassandra
Updated Branches:
refs/heads/trunk 1e92ce43a -> b57b047e5
cqlsh: Handle non-ascii chars in error messages
Patch by Tyler Hobbs; reviewed by Paulo Motta for CASSANDRA-11626
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5de9de1f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5de9de1f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5de9de1f
Branch: refs/heads/trunk
Commit: 5de9de1f5832f2a0e92783e2f4412874423e6e15
Parents: 93c5bc6
Author: Tyler Hobbs
Authored: Thu May 5 11:33:35 2016 -0500
Committer: Tyler Hobbs
Committed: Thu May 5 11:33:35 2016 -0500
--
CHANGES.txt| 1 +
bin/cqlsh.py | 20 ++--
pylib/cqlshlib/copyutil.py | 16 ++--
3 files changed, 25 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5de9de1f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0d9d3e9..a46aa56 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.7
+ * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
* Exit JVM if JMX server fails to startup (CASSANDRA-11540)
* Produce a heap dump when exiting on OOM (CASSANDRA-9861)
* Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5de9de1f/bin/cqlsh.py
--
diff --git a/bin/cqlsh.py b/bin/cqlsh.py
index d135317..85605ae 100644
--- a/bin/cqlsh.py
+++ b/bin/cqlsh.py
@@ -36,7 +36,6 @@ import codecs
import ConfigParser
import csv
import getpass
-import locale
import optparse
import os
import platform
@@ -883,7 +882,7 @@ class Shell(cmd.Cmd):
if ksname is None:
ksname = self.current_keyspace
layout = self.get_table_meta(ksname, cfname)
-return [str(col) for col in layout.columns]
+return [unicode(col) for col in layout.columns]
def get_usertype_names(self, ksname=None):
if ksname is None:
@@ -1110,7 +1109,7 @@ class Shell(cmd.Cmd):
except EOFError:
self.handle_eof()
except CQL_ERRORS, cqlerr:
-self.printerr(str(cqlerr))
+self.printerr(unicode(cqlerr))
except KeyboardInterrupt:
self.reset_statement()
print
@@ -1257,10 +1256,10 @@ class Shell(cmd.Cmd):
break
except cassandra.OperationTimedOut, err:
self.refresh_schema_metadata_best_effort()
-self.printerr(str(err.__class__.__name__) + ": " + str(err))
+self.printerr(unicode(err.__class__.__name__) + u": " +
unicode(err))
return False, None
except CQL_ERRORS, err:
-self.printerr(str(err.__class__.__name__) + ": " + str(err))
+self.printerr(unicode(err.__class__.__name__) + u": " +
unicode(err))
return False, None
except Exception, err:
import traceback
@@ -2237,7 +2236,16 @@ class Shell(cmd.Cmd):
def writeresult(self, text, color=None, newline=True, out=None):
if out is None:
out = self.query_out
-out.write(self.applycolor(str(text), color) + ('\n' if newline else
''))
+
+# convert Exceptions, etc to text
+if not isinstance(text, (unicode, str)):
+text = unicode(text)
+
+if isinstance(text, unicode):
+text = text.encode(self.encoding)
+
+to_write = self.applycolor(text, color) + ('\n' if newline else '')
+out.write(to_write)
def flush_output(self):
self.query_out.flush()
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5de9de1f/pylib/cqlshlib/copyutil.py
--
diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
index a7a6671..ea49692 100644
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@ -76,8 +76,9 @@ def printdebugmsg(msg):
printmsg(msg)
-def printmsg(msg, eol='\n'):
-sys.stdout.write(msg + eol)
+def printmsg(msg, eol='\n', encoding='utf8'):
+sys.stdout.write(msg.encode(encoding))
+sys.stdout.write(eol)
sys.stdout.flush()
@@ -219,6 +220,7 @@ class CopyTask(object):
self.options = self.parse_options(opts, direction)
self.num_processes = self.options.copy['numprocesses']
+self.encoding = self.options.copy['encoding']
self.printmsg('Using %d child processes' % (self.num_processes,))
if direction == 'from':
@@ -5
[1/2] cassandra git commit: cqlsh: Handle non-ascii chars in error messages
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.0 5980b336f -> d689da367
cqlsh: Handle non-ascii chars in error messages
Patch by Tyler Hobbs; reviewed by Paulo Motta for CASSANDRA-11626
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5de9de1f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5de9de1f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5de9de1f
Branch: refs/heads/cassandra-3.0
Commit: 5de9de1f5832f2a0e92783e2f4412874423e6e15
Parents: 93c5bc6
Author: Tyler Hobbs
Authored: Thu May 5 11:33:35 2016 -0500
Committer: Tyler Hobbs
Committed: Thu May 5 11:33:35 2016 -0500
--
CHANGES.txt| 1 +
bin/cqlsh.py | 20 ++--
pylib/cqlshlib/copyutil.py | 16 ++--
3 files changed, 25 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5de9de1f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0d9d3e9..a46aa56 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.7
+ * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
* Exit JVM if JMX server fails to startup (CASSANDRA-11540)
* Produce a heap dump when exiting on OOM (CASSANDRA-9861)
* Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5de9de1f/bin/cqlsh.py
--
diff --git a/bin/cqlsh.py b/bin/cqlsh.py
index d135317..85605ae 100644
--- a/bin/cqlsh.py
+++ b/bin/cqlsh.py
@@ -36,7 +36,6 @@ import codecs
import ConfigParser
import csv
import getpass
-import locale
import optparse
import os
import platform
@@ -883,7 +882,7 @@ class Shell(cmd.Cmd):
if ksname is None:
ksname = self.current_keyspace
layout = self.get_table_meta(ksname, cfname)
-return [str(col) for col in layout.columns]
+return [unicode(col) for col in layout.columns]
def get_usertype_names(self, ksname=None):
if ksname is None:
@@ -1110,7 +1109,7 @@ class Shell(cmd.Cmd):
except EOFError:
self.handle_eof()
except CQL_ERRORS, cqlerr:
-self.printerr(str(cqlerr))
+self.printerr(unicode(cqlerr))
except KeyboardInterrupt:
self.reset_statement()
print
@@ -1257,10 +1256,10 @@ class Shell(cmd.Cmd):
break
except cassandra.OperationTimedOut, err:
self.refresh_schema_metadata_best_effort()
-self.printerr(str(err.__class__.__name__) + ": " + str(err))
+self.printerr(unicode(err.__class__.__name__) + u": " +
unicode(err))
return False, None
except CQL_ERRORS, err:
-self.printerr(str(err.__class__.__name__) + ": " + str(err))
+self.printerr(unicode(err.__class__.__name__) + u": " +
unicode(err))
return False, None
except Exception, err:
import traceback
@@ -2237,7 +2236,16 @@ class Shell(cmd.Cmd):
def writeresult(self, text, color=None, newline=True, out=None):
if out is None:
out = self.query_out
-out.write(self.applycolor(str(text), color) + ('\n' if newline else
''))
+
+# convert Exceptions, etc to text
+if not isinstance(text, (unicode, str)):
+text = unicode(text)
+
+if isinstance(text, unicode):
+text = text.encode(self.encoding)
+
+to_write = self.applycolor(text, color) + ('\n' if newline else '')
+out.write(to_write)
def flush_output(self):
self.query_out.flush()
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5de9de1f/pylib/cqlshlib/copyutil.py
--
diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
index a7a6671..ea49692 100644
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@ -76,8 +76,9 @@ def printdebugmsg(msg):
printmsg(msg)
-def printmsg(msg, eol='\n'):
-sys.stdout.write(msg + eol)
+def printmsg(msg, eol='\n', encoding='utf8'):
+sys.stdout.write(msg.encode(encoding))
+sys.stdout.write(eol)
sys.stdout.flush()
@@ -219,6 +220,7 @@ class CopyTask(object):
self.options = self.parse_options(opts, direction)
self.num_processes = self.options.copy['numprocesses']
+self.encoding = self.options.copy['encoding']
self.printmsg('Using %d child processes' % (self.num_processes,))
if direction
[3/4] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7
Merge branch 'cassandra-3.0' into cassandra-3.7 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/58d3b9a9 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/58d3b9a9 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/58d3b9a9 Branch: refs/heads/trunk Commit: 58d3b9a90461806d44dd85bf4aa928e575d5fb6c Parents: 68580c7 d689da3 Author: Tyler Hobbs Authored: Thu May 5 11:36:29 2016 -0500 Committer: Tyler Hobbs Committed: Thu May 5 11:36:29 2016 -0500 -- CHANGES.txt| 2 ++ bin/cqlsh.py | 19 ++- pylib/cqlshlib/copyutil.py | 16 ++-- 3 files changed, 26 insertions(+), 11 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/58d3b9a9/CHANGES.txt -- diff --cc CHANGES.txt index 705a32f,0679e11..882be7c --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,72 -1,4 +1,74 @@@ -3.0.6 +3.7 ++Merged from 2.2: ++ * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626) + +3.6 + * Enhanced Compaction Logging (CASSANDRA-10805) + * Make prepared statement cache size configurable (CASSANDRA-11555) + * Integrated JMX authentication and authorization (CASSANDRA-10091) + * Add units to stress ouput (CASSANDRA-11352) + * Fix PER PARTITION LIMIT for single and multi partitions queries (CASSANDRA-11603) + * Add uncompressed chunk cache for RandomAccessReader (CASSANDRA-5863) + * Clarify ClusteringPrefix hierarchy (CASSANDRA-11213) + * Always perform collision check before joining ring (CASSANDRA-10134) + * SSTableWriter output discrepancy (CASSANDRA-11646) + * Fix potential timeout in NativeTransportService.testConcurrentDestroys (CASSANDRA-10756) + * Support large partitions on the 3.0 sstable format (CASSANDRA-11206) + * Add support to rebuild from specific range (CASSANDRA-10406) + * Optimize the overlapping lookup by calculating all the + bounds in advance (CASSANDRA-11571) + * Support json/yaml output in noetool tablestats (CASSANDRA-5977) + * (stress) Add datacenter option to -node options (CASSANDRA-11591) + * Fix handling of empty slices (CASSANDRA-11513) + * Make number of cores used by cqlsh COPY visible to testing code (CASSANDRA-11437) + * Allow filtering on clustering columns for queries without secondary indexes (CASSANDRA-11310) + * Refactor Restriction hierarchy (CASSANDRA-11354) + * Eliminate allocations in R/W path (CASSANDRA-11421) + * Update Netty to 4.0.36 (CASSANDRA-11567) + * Fix PER PARTITION LIMIT for queries requiring post-query ordering (CASSANDRA-11556) + * Allow instantiation of UDTs and tuples in UDFs (CASSANDRA-10818) + * Support UDT in CQLSSTableWriter (CASSANDRA-10624) + * Support for non-frozen user-defined types, updating + individual fields of user-defined types (CASSANDRA-7423) + * Make LZ4 compression level configurable (CASSANDRA-11051) + * Allow per-partition LIMIT clause in CQL (CASSANDRA-7017) + * Make custom filtering more extensible with UserExpression (CASSANDRA-11295) + * Improve field-checking and error reporting in cassandra.yaml (CASSANDRA-10649) + * Print CAS stats in nodetool proxyhistograms (CASSANDRA-11507) + * More user friendly error when providing an invalid token to nodetool (CASSANDRA-9348) + * Add static column support to SASI index (CASSANDRA-11183) + * Support EQ/PREFIX queries in SASI CONTAINS mode without tokenization (CASSANDRA-11434) + * Support LIKE operator in prepared statements (CASSANDRA-11456) + * Add a command to see if a Materialized View has finished building (CASSANDRA-9967) + * Log endpoint and port associated with streaming operation (CASSANDRA-8777) + * Print sensible units for all log messages (CASSANDRA-9692) + * Upgrade Netty to version 4.0.34 (CASSANDRA-11096) + * Break the CQL grammar into separate Parser and Lexer (CASSANDRA-11372) + * Compress only inter-dc traffic by default (CASSANDRA-) + * Add metrics to track write amplification (CASSANDRA-11420) + * cassandra-stress: cannot handle "value-less" tables (CASSANDRA-7739) + * Add/drop multiple columns in one ALTER TABLE statement (CASSANDRA-10411) + * Add require_endpoint_verification opt for internode encryption (CASSANDRA-9220) + * Add auto import java.util for UDF code block (CASSANDRA-11392) + * Add --hex-format option to nodetool getsstables (CASSANDRA-11337) + * sstablemetadata should print sstable min/max token (CASSANDRA-7159) + * Do not wrap CassandraException in TriggerExecutor (CASSANDRA-9421) + * COPY TO should have higher double precision (CASSANDRA-11255) + * Stress should exit with non-zero status after failure (CASSANDRA-10340) + * Add client to cqlsh SHOW_SESSION (CASSANDRA-8958) + * Fix nodetool tablestats keyspace level
[3/3] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7
Merge branch 'cassandra-3.0' into cassandra-3.7 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/58d3b9a9 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/58d3b9a9 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/58d3b9a9 Branch: refs/heads/cassandra-3.7 Commit: 58d3b9a90461806d44dd85bf4aa928e575d5fb6c Parents: 68580c7 d689da3 Author: Tyler Hobbs Authored: Thu May 5 11:36:29 2016 -0500 Committer: Tyler Hobbs Committed: Thu May 5 11:36:29 2016 -0500 -- CHANGES.txt| 2 ++ bin/cqlsh.py | 19 ++- pylib/cqlshlib/copyutil.py | 16 ++-- 3 files changed, 26 insertions(+), 11 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/58d3b9a9/CHANGES.txt -- diff --cc CHANGES.txt index 705a32f,0679e11..882be7c --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,72 -1,4 +1,74 @@@ -3.0.6 +3.7 ++Merged from 2.2: ++ * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626) + +3.6 + * Enhanced Compaction Logging (CASSANDRA-10805) + * Make prepared statement cache size configurable (CASSANDRA-11555) + * Integrated JMX authentication and authorization (CASSANDRA-10091) + * Add units to stress ouput (CASSANDRA-11352) + * Fix PER PARTITION LIMIT for single and multi partitions queries (CASSANDRA-11603) + * Add uncompressed chunk cache for RandomAccessReader (CASSANDRA-5863) + * Clarify ClusteringPrefix hierarchy (CASSANDRA-11213) + * Always perform collision check before joining ring (CASSANDRA-10134) + * SSTableWriter output discrepancy (CASSANDRA-11646) + * Fix potential timeout in NativeTransportService.testConcurrentDestroys (CASSANDRA-10756) + * Support large partitions on the 3.0 sstable format (CASSANDRA-11206) + * Add support to rebuild from specific range (CASSANDRA-10406) + * Optimize the overlapping lookup by calculating all the + bounds in advance (CASSANDRA-11571) + * Support json/yaml output in noetool tablestats (CASSANDRA-5977) + * (stress) Add datacenter option to -node options (CASSANDRA-11591) + * Fix handling of empty slices (CASSANDRA-11513) + * Make number of cores used by cqlsh COPY visible to testing code (CASSANDRA-11437) + * Allow filtering on clustering columns for queries without secondary indexes (CASSANDRA-11310) + * Refactor Restriction hierarchy (CASSANDRA-11354) + * Eliminate allocations in R/W path (CASSANDRA-11421) + * Update Netty to 4.0.36 (CASSANDRA-11567) + * Fix PER PARTITION LIMIT for queries requiring post-query ordering (CASSANDRA-11556) + * Allow instantiation of UDTs and tuples in UDFs (CASSANDRA-10818) + * Support UDT in CQLSSTableWriter (CASSANDRA-10624) + * Support for non-frozen user-defined types, updating + individual fields of user-defined types (CASSANDRA-7423) + * Make LZ4 compression level configurable (CASSANDRA-11051) + * Allow per-partition LIMIT clause in CQL (CASSANDRA-7017) + * Make custom filtering more extensible with UserExpression (CASSANDRA-11295) + * Improve field-checking and error reporting in cassandra.yaml (CASSANDRA-10649) + * Print CAS stats in nodetool proxyhistograms (CASSANDRA-11507) + * More user friendly error when providing an invalid token to nodetool (CASSANDRA-9348) + * Add static column support to SASI index (CASSANDRA-11183) + * Support EQ/PREFIX queries in SASI CONTAINS mode without tokenization (CASSANDRA-11434) + * Support LIKE operator in prepared statements (CASSANDRA-11456) + * Add a command to see if a Materialized View has finished building (CASSANDRA-9967) + * Log endpoint and port associated with streaming operation (CASSANDRA-8777) + * Print sensible units for all log messages (CASSANDRA-9692) + * Upgrade Netty to version 4.0.34 (CASSANDRA-11096) + * Break the CQL grammar into separate Parser and Lexer (CASSANDRA-11372) + * Compress only inter-dc traffic by default (CASSANDRA-) + * Add metrics to track write amplification (CASSANDRA-11420) + * cassandra-stress: cannot handle "value-less" tables (CASSANDRA-7739) + * Add/drop multiple columns in one ALTER TABLE statement (CASSANDRA-10411) + * Add require_endpoint_verification opt for internode encryption (CASSANDRA-9220) + * Add auto import java.util for UDF code block (CASSANDRA-11392) + * Add --hex-format option to nodetool getsstables (CASSANDRA-11337) + * sstablemetadata should print sstable min/max token (CASSANDRA-7159) + * Do not wrap CassandraException in TriggerExecutor (CASSANDRA-9421) + * COPY TO should have higher double precision (CASSANDRA-11255) + * Stress should exit with non-zero status after failure (CASSANDRA-10340) + * Add client to cqlsh SHOW_SESSION (CASSANDRA-8958) + * Fix nodetool tablestats keyspac
[2/4] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d689da36 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d689da36 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d689da36 Branch: refs/heads/trunk Commit: d689da3677f20160c077e96028b83e69fe0206f9 Parents: 5980b33 5de9de1 Author: Tyler Hobbs Authored: Thu May 5 11:34:56 2016 -0500 Committer: Tyler Hobbs Committed: Thu May 5 11:34:56 2016 -0500 -- CHANGES.txt| 1 + bin/cqlsh.py | 20 ++-- pylib/cqlshlib/copyutil.py | 16 ++-- 3 files changed, 25 insertions(+), 12 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d689da36/CHANGES.txt -- diff --cc CHANGES.txt index 860a4c2,a46aa56..0679e11 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,26 -1,8 +1,27 @@@ -2.2.7 +3.0.6 + * Disallow creating view with a static column (CASSANDRA-11602) + * Reduce the amount of object allocations caused by the getFunctions methods (CASSANDRA-11593) + * Potential error replaying commitlog with smallint/tinyint/date/time types (CASSANDRA-11618) + * Fix queries with filtering on counter columns (CASSANDRA-11629) + * Improve tombstone printing in sstabledump (CASSANDRA-11655) + * Fix paging for range queries where all clustering columns are specified (CASSANDRA-11669) + * Don't require HEAP_NEW_SIZE to be set when using G1 (CASSANDRA-11600) + * Fix sstabledump not showing cells after tombstone marker (CASSANDRA-11654) + * Ignore all LocalStrategy keyspaces for streaming and other related + operations (CASSANDRA-11627) + * Ensure columnfilter covers indexed columns for thrift 2i queries (CASSANDRA-11523) + * Only open one sstable scanner per sstable (CASSANDRA-11412) + * Option to specify ProtocolVersion in cassandra-stress (CASSANDRA-11410) + * ArithmeticException in avgFunctionForDecimal (CASSANDRA-11485) + * LogAwareFileLister should only use OLD sstable files in current folder to determine disk consistency (CASSANDRA-11470) + * Notify indexers of expired rows during compaction (CASSANDRA-11329) + * Properly respond with ProtocolError when a v1/v2 native protocol + header is received (CASSANDRA-11464) + * Validate that num_tokens and initial_token are consistent with one another (CASSANDRA-10120) +Merged from 2.2: + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626) * Exit JVM if JMX server fails to startup (CASSANDRA-11540) * Produce a heap dump when exiting on OOM (CASSANDRA-9861) - * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) * JSON datetime formatting needs timezone (CASSANDRA-11137) * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502) http://git-wip-us.apache.org/repos/asf/cassandra/blob/d689da36/bin/cqlsh.py -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d689da36/pylib/cqlshlib/copyutil.py --
[2/2] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d689da36 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d689da36 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d689da36 Branch: refs/heads/cassandra-3.0 Commit: d689da3677f20160c077e96028b83e69fe0206f9 Parents: 5980b33 5de9de1 Author: Tyler Hobbs Authored: Thu May 5 11:34:56 2016 -0500 Committer: Tyler Hobbs Committed: Thu May 5 11:34:56 2016 -0500 -- CHANGES.txt| 1 + bin/cqlsh.py | 20 ++-- pylib/cqlshlib/copyutil.py | 16 ++-- 3 files changed, 25 insertions(+), 12 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d689da36/CHANGES.txt -- diff --cc CHANGES.txt index 860a4c2,a46aa56..0679e11 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,26 -1,8 +1,27 @@@ -2.2.7 +3.0.6 + * Disallow creating view with a static column (CASSANDRA-11602) + * Reduce the amount of object allocations caused by the getFunctions methods (CASSANDRA-11593) + * Potential error replaying commitlog with smallint/tinyint/date/time types (CASSANDRA-11618) + * Fix queries with filtering on counter columns (CASSANDRA-11629) + * Improve tombstone printing in sstabledump (CASSANDRA-11655) + * Fix paging for range queries where all clustering columns are specified (CASSANDRA-11669) + * Don't require HEAP_NEW_SIZE to be set when using G1 (CASSANDRA-11600) + * Fix sstabledump not showing cells after tombstone marker (CASSANDRA-11654) + * Ignore all LocalStrategy keyspaces for streaming and other related + operations (CASSANDRA-11627) + * Ensure columnfilter covers indexed columns for thrift 2i queries (CASSANDRA-11523) + * Only open one sstable scanner per sstable (CASSANDRA-11412) + * Option to specify ProtocolVersion in cassandra-stress (CASSANDRA-11410) + * ArithmeticException in avgFunctionForDecimal (CASSANDRA-11485) + * LogAwareFileLister should only use OLD sstable files in current folder to determine disk consistency (CASSANDRA-11470) + * Notify indexers of expired rows during compaction (CASSANDRA-11329) + * Properly respond with ProtocolError when a v1/v2 native protocol + header is received (CASSANDRA-11464) + * Validate that num_tokens and initial_token are consistent with one another (CASSANDRA-10120) +Merged from 2.2: + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626) * Exit JVM if JMX server fails to startup (CASSANDRA-11540) * Produce a heap dump when exiting on OOM (CASSANDRA-9861) - * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) * JSON datetime formatting needs timezone (CASSANDRA-11137) * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502) http://git-wip-us.apache.org/repos/asf/cassandra/blob/d689da36/bin/cqlsh.py -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d689da36/pylib/cqlshlib/copyutil.py --
[2/3] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d689da36 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d689da36 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d689da36 Branch: refs/heads/cassandra-3.7 Commit: d689da3677f20160c077e96028b83e69fe0206f9 Parents: 5980b33 5de9de1 Author: Tyler Hobbs Authored: Thu May 5 11:34:56 2016 -0500 Committer: Tyler Hobbs Committed: Thu May 5 11:34:56 2016 -0500 -- CHANGES.txt| 1 + bin/cqlsh.py | 20 ++-- pylib/cqlshlib/copyutil.py | 16 ++-- 3 files changed, 25 insertions(+), 12 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d689da36/CHANGES.txt -- diff --cc CHANGES.txt index 860a4c2,a46aa56..0679e11 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,26 -1,8 +1,27 @@@ -2.2.7 +3.0.6 + * Disallow creating view with a static column (CASSANDRA-11602) + * Reduce the amount of object allocations caused by the getFunctions methods (CASSANDRA-11593) + * Potential error replaying commitlog with smallint/tinyint/date/time types (CASSANDRA-11618) + * Fix queries with filtering on counter columns (CASSANDRA-11629) + * Improve tombstone printing in sstabledump (CASSANDRA-11655) + * Fix paging for range queries where all clustering columns are specified (CASSANDRA-11669) + * Don't require HEAP_NEW_SIZE to be set when using G1 (CASSANDRA-11600) + * Fix sstabledump not showing cells after tombstone marker (CASSANDRA-11654) + * Ignore all LocalStrategy keyspaces for streaming and other related + operations (CASSANDRA-11627) + * Ensure columnfilter covers indexed columns for thrift 2i queries (CASSANDRA-11523) + * Only open one sstable scanner per sstable (CASSANDRA-11412) + * Option to specify ProtocolVersion in cassandra-stress (CASSANDRA-11410) + * ArithmeticException in avgFunctionForDecimal (CASSANDRA-11485) + * LogAwareFileLister should only use OLD sstable files in current folder to determine disk consistency (CASSANDRA-11470) + * Notify indexers of expired rows during compaction (CASSANDRA-11329) + * Properly respond with ProtocolError when a v1/v2 native protocol + header is received (CASSANDRA-11464) + * Validate that num_tokens and initial_token are consistent with one another (CASSANDRA-10120) +Merged from 2.2: + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626) * Exit JVM if JMX server fails to startup (CASSANDRA-11540) * Produce a heap dump when exiting on OOM (CASSANDRA-9861) - * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) * JSON datetime formatting needs timezone (CASSANDRA-11137) * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502) http://git-wip-us.apache.org/repos/asf/cassandra/blob/d689da36/bin/cqlsh.py -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d689da36/pylib/cqlshlib/copyutil.py --
[1/3] cassandra git commit: cqlsh: Handle non-ascii chars in error messages
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.7 68580c7a8 -> 58d3b9a90
cqlsh: Handle non-ascii chars in error messages
Patch by Tyler Hobbs; reviewed by Paulo Motta for CASSANDRA-11626
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5de9de1f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5de9de1f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5de9de1f
Branch: refs/heads/cassandra-3.7
Commit: 5de9de1f5832f2a0e92783e2f4412874423e6e15
Parents: 93c5bc6
Author: Tyler Hobbs
Authored: Thu May 5 11:33:35 2016 -0500
Committer: Tyler Hobbs
Committed: Thu May 5 11:33:35 2016 -0500
--
CHANGES.txt| 1 +
bin/cqlsh.py | 20 ++--
pylib/cqlshlib/copyutil.py | 16 ++--
3 files changed, 25 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5de9de1f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0d9d3e9..a46aa56 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.7
+ * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
* Exit JVM if JMX server fails to startup (CASSANDRA-11540)
* Produce a heap dump when exiting on OOM (CASSANDRA-9861)
* Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5de9de1f/bin/cqlsh.py
--
diff --git a/bin/cqlsh.py b/bin/cqlsh.py
index d135317..85605ae 100644
--- a/bin/cqlsh.py
+++ b/bin/cqlsh.py
@@ -36,7 +36,6 @@ import codecs
import ConfigParser
import csv
import getpass
-import locale
import optparse
import os
import platform
@@ -883,7 +882,7 @@ class Shell(cmd.Cmd):
if ksname is None:
ksname = self.current_keyspace
layout = self.get_table_meta(ksname, cfname)
-return [str(col) for col in layout.columns]
+return [unicode(col) for col in layout.columns]
def get_usertype_names(self, ksname=None):
if ksname is None:
@@ -1110,7 +1109,7 @@ class Shell(cmd.Cmd):
except EOFError:
self.handle_eof()
except CQL_ERRORS, cqlerr:
-self.printerr(str(cqlerr))
+self.printerr(unicode(cqlerr))
except KeyboardInterrupt:
self.reset_statement()
print
@@ -1257,10 +1256,10 @@ class Shell(cmd.Cmd):
break
except cassandra.OperationTimedOut, err:
self.refresh_schema_metadata_best_effort()
-self.printerr(str(err.__class__.__name__) + ": " + str(err))
+self.printerr(unicode(err.__class__.__name__) + u": " +
unicode(err))
return False, None
except CQL_ERRORS, err:
-self.printerr(str(err.__class__.__name__) + ": " + str(err))
+self.printerr(unicode(err.__class__.__name__) + u": " +
unicode(err))
return False, None
except Exception, err:
import traceback
@@ -2237,7 +2236,16 @@ class Shell(cmd.Cmd):
def writeresult(self, text, color=None, newline=True, out=None):
if out is None:
out = self.query_out
-out.write(self.applycolor(str(text), color) + ('\n' if newline else
''))
+
+# convert Exceptions, etc to text
+if not isinstance(text, (unicode, str)):
+text = unicode(text)
+
+if isinstance(text, unicode):
+text = text.encode(self.encoding)
+
+to_write = self.applycolor(text, color) + ('\n' if newline else '')
+out.write(to_write)
def flush_output(self):
self.query_out.flush()
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5de9de1f/pylib/cqlshlib/copyutil.py
--
diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
index a7a6671..ea49692 100644
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@ -76,8 +76,9 @@ def printdebugmsg(msg):
printmsg(msg)
-def printmsg(msg, eol='\n'):
-sys.stdout.write(msg + eol)
+def printmsg(msg, eol='\n', encoding='utf8'):
+sys.stdout.write(msg.encode(encoding))
+sys.stdout.write(eol)
sys.stdout.flush()
@@ -219,6 +220,7 @@ class CopyTask(object):
self.options = self.parse_options(opts, direction)
self.num_processes = self.options.copy['numprocesses']
+self.encoding = self.options.copy['encoding']
self.printmsg('Using %d child processes' % (self.num_processes,))
if direction
cassandra git commit: cqlsh: Handle non-ascii chars in error messages
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.2 93c5bc616 -> 5de9de1f5
cqlsh: Handle non-ascii chars in error messages
Patch by Tyler Hobbs; reviewed by Paulo Motta for CASSANDRA-11626
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5de9de1f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5de9de1f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5de9de1f
Branch: refs/heads/cassandra-2.2
Commit: 5de9de1f5832f2a0e92783e2f4412874423e6e15
Parents: 93c5bc6
Author: Tyler Hobbs
Authored: Thu May 5 11:33:35 2016 -0500
Committer: Tyler Hobbs
Committed: Thu May 5 11:33:35 2016 -0500
--
CHANGES.txt| 1 +
bin/cqlsh.py | 20 ++--
pylib/cqlshlib/copyutil.py | 16 ++--
3 files changed, 25 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5de9de1f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0d9d3e9..a46aa56 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.7
+ * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626)
* Exit JVM if JMX server fails to startup (CASSANDRA-11540)
* Produce a heap dump when exiting on OOM (CASSANDRA-9861)
* Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5de9de1f/bin/cqlsh.py
--
diff --git a/bin/cqlsh.py b/bin/cqlsh.py
index d135317..85605ae 100644
--- a/bin/cqlsh.py
+++ b/bin/cqlsh.py
@@ -36,7 +36,6 @@ import codecs
import ConfigParser
import csv
import getpass
-import locale
import optparse
import os
import platform
@@ -883,7 +882,7 @@ class Shell(cmd.Cmd):
if ksname is None:
ksname = self.current_keyspace
layout = self.get_table_meta(ksname, cfname)
-return [str(col) for col in layout.columns]
+return [unicode(col) for col in layout.columns]
def get_usertype_names(self, ksname=None):
if ksname is None:
@@ -1110,7 +1109,7 @@ class Shell(cmd.Cmd):
except EOFError:
self.handle_eof()
except CQL_ERRORS, cqlerr:
-self.printerr(str(cqlerr))
+self.printerr(unicode(cqlerr))
except KeyboardInterrupt:
self.reset_statement()
print
@@ -1257,10 +1256,10 @@ class Shell(cmd.Cmd):
break
except cassandra.OperationTimedOut, err:
self.refresh_schema_metadata_best_effort()
-self.printerr(str(err.__class__.__name__) + ": " + str(err))
+self.printerr(unicode(err.__class__.__name__) + u": " +
unicode(err))
return False, None
except CQL_ERRORS, err:
-self.printerr(str(err.__class__.__name__) + ": " + str(err))
+self.printerr(unicode(err.__class__.__name__) + u": " +
unicode(err))
return False, None
except Exception, err:
import traceback
@@ -2237,7 +2236,16 @@ class Shell(cmd.Cmd):
def writeresult(self, text, color=None, newline=True, out=None):
if out is None:
out = self.query_out
-out.write(self.applycolor(str(text), color) + ('\n' if newline else
''))
+
+# convert Exceptions, etc to text
+if not isinstance(text, (unicode, str)):
+text = unicode(text)
+
+if isinstance(text, unicode):
+text = text.encode(self.encoding)
+
+to_write = self.applycolor(text, color) + ('\n' if newline else '')
+out.write(to_write)
def flush_output(self):
self.query_out.flush()
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5de9de1f/pylib/cqlshlib/copyutil.py
--
diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
index a7a6671..ea49692 100644
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@ -76,8 +76,9 @@ def printdebugmsg(msg):
printmsg(msg)
-def printmsg(msg, eol='\n'):
-sys.stdout.write(msg + eol)
+def printmsg(msg, eol='\n', encoding='utf8'):
+sys.stdout.write(msg.encode(encoding))
+sys.stdout.write(eol)
sys.stdout.flush()
@@ -219,6 +220,7 @@ class CopyTask(object):
self.options = self.parse_options(opts, direction)
self.num_processes = self.options.copy['numprocesses']
+self.encoding = self.options.copy['encoding']
self.printmsg('Using %d child processes' % (self.num_processes,))
if direction
[jira] [Comment Edited] (CASSANDRA-10202) simplify CommitLogSegmentManager
[ https://issues.apache.org/jira/browse/CASSANDRA-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272494#comment-15272494 ] Branimir Lambov edited comment on CASSANDRA-10202 at 5/5/16 3:12 PM: - Dtests are failing, this is not ready for review yet. was (Author: blambov): Dtests are failing. > simplify CommitLogSegmentManager > > > Key: CASSANDRA-10202 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10202 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths >Reporter: Jonathan Ellis >Assignee: Branimir Lambov >Priority: Minor > > Now that we only keep one active segment around we can simplify this from the > old recycling design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10202) simplify CommitLogSegmentManager
[ https://issues.apache.org/jira/browse/CASSANDRA-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Branimir Lambov updated CASSANDRA-10202: Status: Open (was: Patch Available) Dtests are failing. > simplify CommitLogSegmentManager > > > Key: CASSANDRA-10202 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10202 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths >Reporter: Jonathan Ellis >Assignee: Branimir Lambov >Priority: Minor > > Now that we only keep one active segment around we can simplify this from the > old recycling design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11152) SOURCE command in CQLSH 3.2 requires that "use keyspace" is in the cql file that you are sourcing
[
https://issues.apache.org/jira/browse/CASSANDRA-11152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272452#comment-15272452
]
Robert Stupp commented on CASSANDRA-11152:
--
I'm not sure, whether it's an oversight, that the keyspace is not passed to the
subshell used for {{source}} command.
If it is an oversight in CASSANDRA-6307, adding {{keyspace = self.keyspace}} to
the subshell's Shell constructor in {{Shell#do_source}} should fix it.
Thoughts?
> SOURCE command in CQLSH 3.2 requires that "use keyspace" is in the cql file
> that you are sourcing
> -
>
> Key: CASSANDRA-11152
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11152
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
> Environment: CQLSH 3.2.1
>Reporter: Francesco Animali
>
> a difference in behaviour between SOURCE command in CQLSH 3.1 and 3.2.
> In CQLSH 3.1 SOURCE will NOT require "use keyspace" in the cql file that you
> execute: the "keyspace" directive in the qlshrc file will work and the cql
> file will be executed.
> In CQLSH 3.2.1, SOURCE command requires that "use keyspace" is in the cql
> file that you are sourcing, otherwise it throws this error:
> "No keyspace has been specified. USE a keyspace, or explicitly specify
> keyspace.tablename".
> The "keyspace" directive in cqlshrc is overridden by source command.
> steps to reproduce:
> create a file called select.cql in your home directory:
> {noformat}
> echo "CONSISTENCY ONE;" > select.cql
> echo "select * from tab;" >> select.cql
> {noformat}
> in cqlsh:
> {noformat}
> create KEYSPACE kspace WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> create TABLE tab ( id int primary key);
> insert into tab (id) VALUES ( 1);
> {noformat}
> Add this to cqlsgrc:
> {noformat}
> [authentication]
> keyspace = kspace
> {noformat}
> Then exit cqlsh and rerun cqlsh using the cqlshrc just modified.
> Note that you are in keyspace "kspace".
> execute:
> {noformat}
> source 'select.cql'
> {noformat}
> this will have different behaviour in CQLSH 3.2 and 3.1
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11541) correct the java documentation for SlabAllocator and NativeAllocator
[ https://issues.apache.org/jira/browse/CASSANDRA-11541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Branimir Lambov updated CASSANDRA-11541: Status: Ready to Commit (was: Patch Available) > correct the java documentation for SlabAllocator and NativeAllocator > > > Key: CASSANDRA-11541 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11541 > Project: Cassandra > Issue Type: Improvement >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Trivial > Labels: document, lhf > Fix For: 2.2.x > > Attachments: CASSANDRA-11541.patch > > > I heard a lot that Cassandra uses 2MB slab allocation strategy for memtable > to improve its memory efficiency. But in fact, it has been changed from 2MB > to 1 MB. > And in NativeAllocator, it's logarithmically from 8kb to 1mb. > large number of tables may not cause too much trouble in term of memtable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11541) correct the java documentation for SlabAllocator and NativeAllocator
[ https://issues.apache.org/jira/browse/CASSANDRA-11541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272445#comment-15272445 ] Branimir Lambov commented on CASSANDRA-11541: - LGTM > correct the java documentation for SlabAllocator and NativeAllocator > > > Key: CASSANDRA-11541 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11541 > Project: Cassandra > Issue Type: Improvement >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Trivial > Labels: document, lhf > Fix For: 2.2.x > > Attachments: CASSANDRA-11541.patch > > > I heard a lot that Cassandra uses 2MB slab allocation strategy for memtable > to improve its memory efficiency. But in fact, it has been changed from 2MB > to 1 MB. > And in NativeAllocator, it's logarithmically from 8kb to 1mb. > large number of tables may not cause too much trouble in term of memtable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11541) correct the java documentation for SlabAllocator and NativeAllocator
[ https://issues.apache.org/jira/browse/CASSANDRA-11541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272444#comment-15272444 ] ZhaoYang commented on CASSANDRA-11541: -- [~blambov] thanks for the advice. I updated the patch. > correct the java documentation for SlabAllocator and NativeAllocator > > > Key: CASSANDRA-11541 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11541 > Project: Cassandra > Issue Type: Improvement >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Trivial > Labels: document, lhf > Fix For: 2.2.x > > Attachments: CASSANDRA-11541.patch > > > I heard a lot that Cassandra uses 2MB slab allocation strategy for memtable > to improve its memory efficiency. But in fact, it has been changed from 2MB > to 1 MB. > And in NativeAllocator, it's logarithmically from 8kb to 1mb. > large number of tables may not cause too much trouble in term of memtable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11541) correct the java documentation for SlabAllocator and NativeAllocator
[ https://issues.apache.org/jira/browse/CASSANDRA-11541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhaoYang updated CASSANDRA-11541: - Attachment: CASSANDRA-11541.patch > correct the java documentation for SlabAllocator and NativeAllocator > > > Key: CASSANDRA-11541 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11541 > Project: Cassandra > Issue Type: Improvement >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Trivial > Labels: document, lhf > Fix For: 2.2.x > > Attachments: CASSANDRA-11541.patch > > > I heard a lot that Cassandra uses 2MB slab allocation strategy for memtable > to improve its memory efficiency. But in fact, it has been changed from 2MB > to 1 MB. > And in NativeAllocator, it's logarithmically from 8kb to 1mb. > large number of tables may not cause too much trouble in term of memtable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11541) correct the java documentation for SlabAllocator and NativeAllocator
[ https://issues.apache.org/jira/browse/CASSANDRA-11541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhaoYang updated CASSANDRA-11541: - Attachment: (was: CASSANDRA-11541.patch) > correct the java documentation for SlabAllocator and NativeAllocator > > > Key: CASSANDRA-11541 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11541 > Project: Cassandra > Issue Type: Improvement >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Trivial > Labels: document, lhf > Fix For: 2.2.x > > Attachments: CASSANDRA-11541.patch > > > I heard a lot that Cassandra uses 2MB slab allocation strategy for memtable > to improve its memory efficiency. But in fact, it has been changed from 2MB > to 1 MB. > And in NativeAllocator, it's logarithmically from 8kb to 1mb. > large number of tables may not cause too much trouble in term of memtable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-11714) shutdown script should wait more than 30 seconds before force kill
[
https://issues.apache.org/jira/browse/CASSANDRA-11714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei Deng reassigned CASSANDRA-11714:
Assignee: Wei Deng
> shutdown script should wait more than 30 seconds before force kill
> --
>
> Key: CASSANDRA-11714
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11714
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Wei Deng
>Assignee: Wei Deng
>Priority: Minor
> Labels: lhf
>
> I'm running with 3.0.6 and it appears that if I let it run for multiple days
> (ingesting a never-ending streaming data from Kafka), then the next time I'm
> restarting the JVM by "service cassandra restart", it will take 9 minutes to
> replay the commit log segments (the commitlog_total_space_in_mb is using the
> default 8GB and I observed close to 8GB commitlog segment files before the
> restart). Here is a fragment of the system.log showing how long it takes:
> {noformat}
> INFO [main] 2016-05-04 19:06:54,356 CommitLog.java:168 - Replaying
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248187.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248191.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248195.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248198.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248201.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248205.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248209.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248213.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248217.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248221.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248225.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248230.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248234.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248238.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248242.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248246.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248250.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248254.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248258.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248262.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248266.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248270.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248274.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248278.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248282.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248286.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248290.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248294.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248299.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248303.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248307.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248311.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248315.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248319.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248323.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248327.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248331.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248335.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248340.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248344.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248348.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248352.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248356.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248360.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248364.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248368.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248372.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248376.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248380.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248384.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248388.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248392.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248396.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248397.log,
> /mnt/
[jira] [Updated] (CASSANDRA-11714) shutdown script should wait more than 30 seconds before force kill
[
https://issues.apache.org/jira/browse/CASSANDRA-11714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremy Hanna updated CASSANDRA-11714:
-
Labels: lhf (was: )
> shutdown script should wait more than 30 seconds before force kill
> --
>
> Key: CASSANDRA-11714
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11714
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Wei Deng
>Priority: Minor
> Labels: lhf
>
> I'm running with 3.0.6 and it appears that if I let it run for multiple days
> (ingesting a never-ending streaming data from Kafka), then the next time I'm
> restarting the JVM by "service cassandra restart", it will take 9 minutes to
> replay the commit log segments (the commitlog_total_space_in_mb is using the
> default 8GB and I observed close to 8GB commitlog segment files before the
> restart). Here is a fragment of the system.log showing how long it takes:
> {noformat}
> INFO [main] 2016-05-04 19:06:54,356 CommitLog.java:168 - Replaying
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248187.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248191.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248195.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248198.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248201.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248205.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248209.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248213.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248217.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248221.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248225.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248230.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248234.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248238.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248242.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248246.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248250.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248254.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248258.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248262.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248266.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248270.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248274.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248278.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248282.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248286.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248290.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248294.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248299.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248303.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248307.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248311.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248315.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248319.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248323.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248327.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248331.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248335.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248340.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248344.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248348.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248352.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248356.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248360.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248364.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248368.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248372.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248376.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248380.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248384.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248388.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248392.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248396.log,
> /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248397.log,
> /mnt/ephemeral/cassandra/commitlog
