[jira] [Commented] (CASSANDRA-5717) Repair causes streaming errors
[
https://issues.apache.org/jira/browse/CASSANDRA-5717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201595#comment-14201595
]
Michael A. Fiedler commented on CASSANDRA-5717:
---
Seeing this problem currently on a 1.2.19 node, and chance a fix for this might
be backported, or is there some other workaround? My scenario doesn't change
any schema definitions, just a node replacing an existing node.
> Repair causes streaming errors
> --
>
> Key: CASSANDRA-5717
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5717
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 1.2.6
> Environment: CentOS release 6.3 (Final)
>Reporter: Yoan Arnaudov
>
> I've changed the replication factor for one of keyspaces and now I'm running
> repairs on column families (manually). I have 3 nodes cluster. Here is the
> error in the error log for one of the nodes.
> {code:title=Error Log}
> ERROR [Streaming to /NODE_IP:1] 2013-07-01 09:31:29,819 CassandraDaemon.java
> (line 192) Exception in thread Thread[Streaming to /NODE_IP:1,5,main]
> java.lang.RuntimeException: java.io.IOException: Broken pipe
> at com.google.common.base.Throwables.propagate(Throwables.java:160)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
> Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.IOException: Broken pipe
> at sun.nio.ch.FileChannelImpl.transferTo0(Native Method)
> at sun.nio.ch.FileChannelImpl.transferToDirectly(Unknown Source)
> at sun.nio.ch.FileChannelImpl.transferTo(Unknown Source)
> at
> org.apache.cassandra.streaming.compress.CompressedFileStreamTask.stream(CompressedFileStreamTask.java:93)
> at
> org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:91)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ... 3 more
> ERROR [Streaming to /NODE_IP:2] 2013-07-01 09:44:18,372 CassandraDaemon.java
> (line 192) Exception in thread Thread[Streaming to /NODE_IP:2,5,main]
> java.lang.RuntimeException: java.io.IOException: Broken pipe
> at com.google.common.base.Throwables.propagate(Throwables.java:160)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
> Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.IOException: Broken pipe
> at sun.nio.ch.FileChannelImpl.transferTo0(Native Method)
> at sun.nio.ch.FileChannelImpl.transferToDirectly(Unknown Source)
> at sun.nio.ch.FileChannelImpl.transferTo(Unknown Source)
> at
> org.apache.cassandra.streaming.compress.CompressedFileStreamTask.stream(CompressedFileStreamTask.java:93)
> at
> org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:91)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ... 3 more
> {code}
> {code:title=netstats for one of the nodes}
> Mode: NORMAL
> Not sending any streams.
> Streaming from: /IP
>KEYSPACE:
> /var/lib/cassandra/data/KEYSPACE/ns_history_org/KEYSPACE-ns_history_org-hf-282-Data.db
> sections=1 progress=0/1393915753 - 0%
>KEYSPACE:
> /var/lib/cassandra/data/KEYSPACE/ns_history_org/KEYSPACE-ns_history_org-ic-455-Data.db
> sections=1 progress=0/792707 - 0%
> Streaming from: /IP
>KEYSPACE:
> /var/lib/cassandra/data/KEYSPACE/ns_history_org/KEYSPACE-ns_history_org-hf-255-Data.db
> sections=1 progress=0/1398197628 - 0%
>KEYSPACE:
> /var/lib/cassandra/data/KEYSPACE/ns_history_biz/KEYSPACE-ns_history_biz-ic-341-Data.db
> sections=1 progress=0/6153542 - 0%
>KEYSPACE:
> /var/lib/cassandra/data/KEYSPACE/listing/KEYSPACE-listing-hf-539-Data.db
> sections=1 progress=0/86968194 - 0%
>KEYSPACE:
> /var/lib/cassandra/data/KEYSPACE/ns_history_biz/KEYSPACE-ns_history_biz-hf-244-Data.db
> sections=1 progress=0/322197762 - 0%
>KEYSPACE:
> /var/lib/cassandra/data/KEYSPACE/ns_history_biz/KEYSPACE-ns_history_biz-ic-346-Data.db
> sections=1 progress=0/6219503 - 0%
>KEYSPACE:
> /var/lib/cassandra/data/KEYSPACE/listing/KEYSPACE-listing-hf-487-Data.db
> sections=1 progress=0/2689291466 - 0%
>KEYSPACE:
> /var/lib/cassandra/data/KEYSPACE/listing/KEYSPACE-listing-ic-684-Data.db
> sections=1 progress=0/3717513 - 0%
>KEYSPACE:
> /var/lib/cassandra/data/KEYSPACE/ns_history_org/KEYSPACE-ns_history_org-ic-413-Data.db
> sections=1 progress=0/22256993 - 0%
>
[jira] [Commented] (CASSANDRA-8022) cqlsh hangs indefinitely within a Docker container connecting to itself with hostname
[
https://issues.apache.org/jira/browse/CASSANDRA-8022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201420#comment-14201420
]
Mikhail Stepura commented on CASSANDRA-8022:
I doubt that is network related. I suspect it's a way cqlsh is launched. It
prints no prompt, so I guess cqlsh 'thinks' it is launched not from a terminal
( i.e. {{sys.stdin.isatty() == false}}).
> cqlsh hangs indefinitely within a Docker container connecting to itself with
> hostname
> -
>
> Key: CASSANDRA-8022
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8022
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: Ubuntu 14.04, running Docker, run inside a Ubuntu 14.04
> container.
>Reporter: Matthew O'Riordan
> Labels: cqlsh
>
> I am unable to use the `cqlsh` tool within a Docker container running
> Cassandra. Previously I would use the Java & Thrift based `cqlsh` tool as
> follows:
> ```
> cqlsh --username cassandra --password whatever $(hostname)
> ```
> When I run the `cqlsh` command after attaching to a running container (I use
> LXC containerisation that allows attaching to a running container and running
> a console), it simply hangs and never reports an error. With the `--debug`
> flag on, I get the following with:
> **cqlsh 4.1.1**
> ```
> $ cqlsh --debug --username cassandra --password obfuscated $(hostname)
> Using CQL driver: '/usr/share/cassandra/lib/cql-internal-only-1.4.1.zip/cql-1.4.1/cql/__init__.py'>
> Using thrift lib: '/usr/share/cassandra/lib/thrift-python-internal-only-0.9.1.zip/thrift/__init__.py'>
> ```
> It then hangs in this state indefinitely, I have no errors from `cqlsh` and
> no errors in the Cassandra log.
> **cqlsh 5.0.1**
> ```
> $ cqlsh --debug --username cassandra --passwor obfuscated $(hostname)
> Using CQL driver: '/usr/share/cassandra/lib/cassandra-driver-internal-only-2.1.0.post.zip/cassandra-driver-2.1.0.post/cassandra/__init__.py'>
> ```
> It then also hangs in this state indefinitely, I have no errors from `cqlsh`
> and no errors in the Cassandra log.
> What's interesting, and quite confusing is that:
> * I can telnet within the container as follows `telnet $(hostname) 9042` and
> I get a socket. When trying to issue some commands, I see Protocol errors in
> the Cassandra log thus verifying that the port is indeed open on the host
> that resolves from $(hostname)
> * If I `cqlsh` from another container or another host to the Cassandra
> container it works just fine.
> * I have tried disabling authentication altogether and using the
> AllowAllAuthenticator, and I experience the same problem.
> * `nodetool` works fine
> In the mean time, I am forced to `cqlsh` from another container as a
> workaround. Happy to try and do anything require to diagnose the cause of
> this problem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8274) Node fails to rejoin cluster on EC2 if private IP is changed
[ https://issues.apache.org/jira/browse/CASSANDRA-8274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Clark updated CASSANDRA-8274: Description: Nodes in Amazon AWS EC2 Classic (not a VPC) may be assigned a new private IP if the node is stopped and then started again. In this case we have puppet update the configured listen_address to the new private IP. However, once the cassandra service starts, it is unable to communicate with the existing nodes(single region) and vice versa. 'nodetool status' shows that each node believes that it is 'UN' and the other node is 'DN'. 'nodetool gossipinfo' on the node that remained running shows the *old* private IP listed as the 'INTERNAL_IP' of the node that was stopped and restarted. The situation is resolved by restarting the cassandra service on the node that remained running. Once it has restarted, the INTERNAL_IP is correctly updated to the new private IP. 'nodetool status' shows that both nodes are up and the cluster appears to function normally. This appears to me to be the root cause of https://issues.apache.org/jira/browse/CASSANDRA-7292. -Possibly https://issues.apache.org/jira/browse/CASSANDRA-8072 as well, but I am not convinced they are actually duplicates.- was: Nodes in Amazon AWS EC2 Classic (not a VPC) may be assigned a new private IP if the node is stopped and then started again. In this case we have puppet update the configured listen_address to the new private IP. However, once the cassandra service starts, it is unable to communicate with the existing nodes(single region) and vice versa. 'nodetool status' shows that each node believes that it is 'UN' and the other node is 'DN'. 'nodetool gossipinfo' on the node that remained running shows the *old* private IP listed as the 'INTERNAL_IP' of the node that was stopped and restarted. The situation is resolved by restarting the cassandra service on the node that remained running. Once it has restarted, the INTERNAL_IP is correctly updated to the new private IP. 'nodetool status' shows that both nodes are up and the cluster appears to function normally. This appears to me to be the root cause of https://issues.apache.org/jira/browse/CASSANDRA-7292. Possibly https://issues.apache.org/jira/browse/CASSANDRA-8072 as well, but I am not convinced they are actually duplicates. > Node fails to rejoin cluster on EC2 if private IP is changed > > > Key: CASSANDRA-8274 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8274 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Amazon EC2 >Reporter: Joseph Clark > > Nodes in Amazon AWS EC2 Classic (not a VPC) may be assigned a new private IP > if the node is stopped and then started again. In this case we have puppet > update the configured listen_address to the new private IP. However, once the > cassandra service starts, it is unable to communicate with the existing > nodes(single region) and vice versa. > 'nodetool status' shows that each node believes that it is 'UN' and the other > node is 'DN'. > 'nodetool gossipinfo' on the node that remained running shows the *old* > private IP listed as the 'INTERNAL_IP' of the node that was stopped and > restarted. > The situation is resolved by restarting the cassandra service on the node > that remained running. Once it has restarted, the INTERNAL_IP is correctly > updated to the new private IP. 'nodetool status' shows that both nodes are up > and the cluster appears to function normally. > This appears to me to be the root cause of > https://issues.apache.org/jira/browse/CASSANDRA-7292. -Possibly > https://issues.apache.org/jira/browse/CASSANDRA-8072 as well, but I am not > convinced they are actually duplicates.- -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8222) Unable to connect to "any" server with cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-8222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler resolved CASSANDRA-8222.
---
Resolution: Not a Problem
> Unable to connect to "any" server with cqlsh
>
>
> Key: CASSANDRA-8222
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8222
> Project: Cassandra
> Issue Type: Bug
> Environment: java -version
> java version "1.7.0_25"
> Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
> Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)
> python --version
> Python 2.7.8
>Reporter: RADOMIRS CIRSKIS
> Labels: cqlsh
>
> I was going through the http://wiki.apache.org/cassandra/GettingStarted
> 1. Installed cassandra
> 2. run cassandra: bin/cassandra -f
> 3. in another terminal attempt to run cqlsh: cqlsh
> 4. also I have tried explicitly: cqlsh 127.0.0.1 9042
> 5. it produces a message:
> {noformat}
> Connection error: ('Unable to connect to any servers', {'127.0.0.1':
> AttributeError("'module' object has no attribute 'loads'",)})
> {noformat}
> Cassandra is running and listening on the port 9042:
> {noformat}
> ...
> INFO 03:36:08 Using Netty Version:
> [netty-buffer=netty-buffer-4.0.23.Final.208198c,
> netty-codec=netty-codec-4.0.23.Final.208198c,
> netty-codec-http=netty-codec-http-4.0.23.Final.208198c,
> netty-codec-socks=netty-codec-socks-4.0.23.Final.208198c,
> netty-common=netty-common-4.0.23.Final.208198c,
> netty-handler=netty-handler-4.0.23.Final.208198c,
> netty-transport=netty-transport-4.0.23.Final.208198c,
> netty-transport-rxtx=netty-transport-rxtx-4.0.23.Final.208198c,
> netty-transport-sctp=netty-transport-sctp-4.0.23.Final.208198c,
> netty-transport-udt=netty-transport-udt-4.0.23.Final.208198c]
> INFO 03:36:08 Starting listening for CQL clients on
> localhost/127.0.0.1:9042...
> INFO 03:36:09 Binding thrift service to localhost/127.0.0.1:9160
> INFO 03:36:09 Listening for thrift clients...
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8274) Node fails to rejoin cluster on EC2 if private IP is changed
[ https://issues.apache.org/jira/browse/CASSANDRA-8274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201345#comment-14201345 ] Joseph Clark commented on CASSANDRA-8274: - at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1193) at org.apache.cassandra.service.StorageService.*prepareReplacementInfo*(StorageService.java:419) at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:650) at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1200) at org.apache.cassandra.service.StorageService.*checkForEndpointCollision*(StorageService.java:444) at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:655) Slightly different :) Even if they were the same, I believe that they are still separate issues. 7292 and this bug both involve starting a node that the cluster *already knows about* as identified by the public lP. Another similarity between 7292 and this bug, in my case at least and likely the original reporter as well, is that the private IP/listen_address has been changed. As far as I can tell, 8072 occurs with brand new nodes that aren't replacing another node in the cluster. > Node fails to rejoin cluster on EC2 if private IP is changed > > > Key: CASSANDRA-8274 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8274 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Amazon EC2 >Reporter: Joseph Clark > > Nodes in Amazon AWS EC2 Classic (not a VPC) may be assigned a new private IP > if the node is stopped and then started again. In this case we have puppet > update the configured listen_address to the new private IP. However, once the > cassandra service starts, it is unable to communicate with the existing > nodes(single region) and vice versa. > 'nodetool status' shows that each node believes that it is 'UN' and the other > node is 'DN'. > 'nodetool gossipinfo' on the node that remained running shows the *old* > private IP listed as the 'INTERNAL_IP' of the node that was stopped and > restarted. > The situation is resolved by restarting the cassandra service on the node > that remained running. Once it has restarted, the INTERNAL_IP is correctly > updated to the new private IP. 'nodetool status' shows that both nodes are up > and the cluster appears to function normally. > This appears to me to be the root cause of > https://issues.apache.org/jira/browse/CASSANDRA-7292. Possibly > https://issues.apache.org/jira/browse/CASSANDRA-8072 as well, but I am not > convinced they are actually duplicates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8222) Unable to connect to "any" server with cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-8222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201313#comment-14201313
]
RADOMIRS CIRSKIS commented on CASSANDRA-8222:
-
[~mishail], thanks :). you are right. I had globally overwritten PYTHONPATH
that had included some 3rd party libs for Python 2.6.
{{unset PYTHONPATH}} solved the issue:
{code}
# unset PYTHONPATH ; python -c "import json; print dir(json); print
json.__version__"
['JSONDecoder', 'JSONEncoder', '__all__', '__author__', '__builtins__',
'__doc__', '__file__', '__name__', '__package__', '__path__', '__version__',
'_default_decoder', '_default_encoder', 'decoder', 'dump', 'dumps', 'encoder',
'load', 'loads', 'scanner']
2.0.9
# cqlsh
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 2.1.1 | CQL spec 3.2.0 | Native protocol v3]
Use HELP for help.
cqlsh>
{code}
> Unable to connect to "any" server with cqlsh
>
>
> Key: CASSANDRA-8222
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8222
> Project: Cassandra
> Issue Type: Bug
> Environment: java -version
> java version "1.7.0_25"
> Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
> Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)
> python --version
> Python 2.7.8
>Reporter: RADOMIRS CIRSKIS
> Labels: cqlsh
>
> I was going through the http://wiki.apache.org/cassandra/GettingStarted
> 1. Installed cassandra
> 2. run cassandra: bin/cassandra -f
> 3. in another terminal attempt to run cqlsh: cqlsh
> 4. also I have tried explicitly: cqlsh 127.0.0.1 9042
> 5. it produces a message:
> {noformat}
> Connection error: ('Unable to connect to any servers', {'127.0.0.1':
> AttributeError("'module' object has no attribute 'loads'",)})
> {noformat}
> Cassandra is running and listening on the port 9042:
> {noformat}
> ...
> INFO 03:36:08 Using Netty Version:
> [netty-buffer=netty-buffer-4.0.23.Final.208198c,
> netty-codec=netty-codec-4.0.23.Final.208198c,
> netty-codec-http=netty-codec-http-4.0.23.Final.208198c,
> netty-codec-socks=netty-codec-socks-4.0.23.Final.208198c,
> netty-common=netty-common-4.0.23.Final.208198c,
> netty-handler=netty-handler-4.0.23.Final.208198c,
> netty-transport=netty-transport-4.0.23.Final.208198c,
> netty-transport-rxtx=netty-transport-rxtx-4.0.23.Final.208198c,
> netty-transport-sctp=netty-transport-sctp-4.0.23.Final.208198c,
> netty-transport-udt=netty-transport-udt-4.0.23.Final.208198c]
> INFO 03:36:08 Starting listening for CQL clients on
> localhost/127.0.0.1:9042...
> INFO 03:36:09 Binding thrift service to localhost/127.0.0.1:9160
> INFO 03:36:09 Listening for thrift clients...
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8222) Unable to connect to "any" server with cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-8222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201290#comment-14201290
]
Mikhail Stepura commented on CASSANDRA-8222:
[~nad2000] it looks like a package/module conflict. Do you have some custom/3rd
party "json" module on system?
{code}
16:33 $ python -c "import json; print dir(json); print json.__version__"
['JSONDecoder', 'JSONEncoder', '__all__', '__author__', '__builtins__',
'__doc__', '__file__', '__name__', '__package__', '__path__', '__version__',
'_default_decoder', '_default_encoder', 'decoder', 'dump', 'dumps', 'encoder',
'load', 'loads', 'scanner']
2.0.9
{code}
> Unable to connect to "any" server with cqlsh
>
>
> Key: CASSANDRA-8222
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8222
> Project: Cassandra
> Issue Type: Bug
> Environment: java -version
> java version "1.7.0_25"
> Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
> Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)
> python --version
> Python 2.7.8
>Reporter: RADOMIRS CIRSKIS
> Labels: cqlsh
>
> I was going through the http://wiki.apache.org/cassandra/GettingStarted
> 1. Installed cassandra
> 2. run cassandra: bin/cassandra -f
> 3. in another terminal attempt to run cqlsh: cqlsh
> 4. also I have tried explicitly: cqlsh 127.0.0.1 9042
> 5. it produces a message:
> {noformat}
> Connection error: ('Unable to connect to any servers', {'127.0.0.1':
> AttributeError("'module' object has no attribute 'loads'",)})
> {noformat}
> Cassandra is running and listening on the port 9042:
> {noformat}
> ...
> INFO 03:36:08 Using Netty Version:
> [netty-buffer=netty-buffer-4.0.23.Final.208198c,
> netty-codec=netty-codec-4.0.23.Final.208198c,
> netty-codec-http=netty-codec-http-4.0.23.Final.208198c,
> netty-codec-socks=netty-codec-socks-4.0.23.Final.208198c,
> netty-common=netty-common-4.0.23.Final.208198c,
> netty-handler=netty-handler-4.0.23.Final.208198c,
> netty-transport=netty-transport-4.0.23.Final.208198c,
> netty-transport-rxtx=netty-transport-rxtx-4.0.23.Final.208198c,
> netty-transport-sctp=netty-transport-sctp-4.0.23.Final.208198c,
> netty-transport-udt=netty-transport-udt-4.0.23.Final.208198c]
> INFO 03:36:08 Starting listening for CQL clients on
> localhost/127.0.0.1:9042...
> INFO 03:36:09 Binding thrift service to localhost/127.0.0.1:9160
> INFO 03:36:09 Listening for thrift clients...
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8222) Unable to connect to "any" server with cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-8222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201280#comment-14201280
]
RADOMIRS CIRSKIS commented on CASSANDRA-8222:
-
[~mishail],
{code}
python -c "import json; print dir(json); print json.__version__"
['JsonReader', 'JsonWriter', 'ReadException', 'WriteException',
'_StringGenerator', '__builtins__', '__doc__', '__file__', '__name__',
'__package__', 'read', 'string', 'types', 'write']
Traceback (most recent call last):
File "", line 1, in
AttributeError: 'module' object has no attribute '__version__'
{code}
> Unable to connect to "any" server with cqlsh
>
>
> Key: CASSANDRA-8222
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8222
> Project: Cassandra
> Issue Type: Bug
> Environment: java -version
> java version "1.7.0_25"
> Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
> Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)
> python --version
> Python 2.7.8
>Reporter: RADOMIRS CIRSKIS
> Labels: cqlsh
>
> I was going through the http://wiki.apache.org/cassandra/GettingStarted
> 1. Installed cassandra
> 2. run cassandra: bin/cassandra -f
> 3. in another terminal attempt to run cqlsh: cqlsh
> 4. also I have tried explicitly: cqlsh 127.0.0.1 9042
> 5. it produces a message:
> {noformat}
> Connection error: ('Unable to connect to any servers', {'127.0.0.1':
> AttributeError("'module' object has no attribute 'loads'",)})
> {noformat}
> Cassandra is running and listening on the port 9042:
> {noformat}
> ...
> INFO 03:36:08 Using Netty Version:
> [netty-buffer=netty-buffer-4.0.23.Final.208198c,
> netty-codec=netty-codec-4.0.23.Final.208198c,
> netty-codec-http=netty-codec-http-4.0.23.Final.208198c,
> netty-codec-socks=netty-codec-socks-4.0.23.Final.208198c,
> netty-common=netty-common-4.0.23.Final.208198c,
> netty-handler=netty-handler-4.0.23.Final.208198c,
> netty-transport=netty-transport-4.0.23.Final.208198c,
> netty-transport-rxtx=netty-transport-rxtx-4.0.23.Final.208198c,
> netty-transport-sctp=netty-transport-sctp-4.0.23.Final.208198c,
> netty-transport-udt=netty-transport-udt-4.0.23.Final.208198c]
> INFO 03:36:08 Starting listening for CQL clients on
> localhost/127.0.0.1:9042...
> INFO 03:36:09 Binding thrift service to localhost/127.0.0.1:9160
> INFO 03:36:09 Listening for thrift clients...
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8274) Node fails to rejoin cluster on EC2 if private IP is changed
[ https://issues.apache.org/jira/browse/CASSANDRA-8274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201278#comment-14201278 ] Brandon Williams commented on CASSANDRA-8274: - Given that the stracktraces in both are exactly the same: yes :) > Node fails to rejoin cluster on EC2 if private IP is changed > > > Key: CASSANDRA-8274 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8274 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Amazon EC2 >Reporter: Joseph Clark > > Nodes in Amazon AWS EC2 Classic (not a VPC) may be assigned a new private IP > if the node is stopped and then started again. In this case we have puppet > update the configured listen_address to the new private IP. However, once the > cassandra service starts, it is unable to communicate with the existing > nodes(single region) and vice versa. > 'nodetool status' shows that each node believes that it is 'UN' and the other > node is 'DN'. > 'nodetool gossipinfo' on the node that remained running shows the *old* > private IP listed as the 'INTERNAL_IP' of the node that was stopped and > restarted. > The situation is resolved by restarting the cassandra service on the node > that remained running. Once it has restarted, the INTERNAL_IP is correctly > updated to the new private IP. 'nodetool status' shows that both nodes are up > and the cluster appears to function normally. > This appears to me to be the root cause of > https://issues.apache.org/jira/browse/CASSANDRA-7292. Possibly > https://issues.apache.org/jira/browse/CASSANDRA-8072 as well, but I am not > convinced they are actually duplicates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8274) Node fails to rejoin cluster on EC2 if private IP is changed
[ https://issues.apache.org/jira/browse/CASSANDRA-8274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201244#comment-14201244 ] Joseph Clark edited comment on CASSANDRA-8274 at 11/6/14 11:59 PM: --- 7292 and 8072 have different stack traces[edit: though admittedly I'm unsure if this is just a result of different versions]. Which did you reproduce? was (Author: jw.clark): 7292 and 8072 have different stack traces. Which did you reproduce? > Node fails to rejoin cluster on EC2 if private IP is changed > > > Key: CASSANDRA-8274 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8274 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Amazon EC2 >Reporter: Joseph Clark > > Nodes in Amazon AWS EC2 Classic (not a VPC) may be assigned a new private IP > if the node is stopped and then started again. In this case we have puppet > update the configured listen_address to the new private IP. However, once the > cassandra service starts, it is unable to communicate with the existing > nodes(single region) and vice versa. > 'nodetool status' shows that each node believes that it is 'UN' and the other > node is 'DN'. > 'nodetool gossipinfo' on the node that remained running shows the *old* > private IP listed as the 'INTERNAL_IP' of the node that was stopped and > restarted. > The situation is resolved by restarting the cassandra service on the node > that remained running. Once it has restarted, the INTERNAL_IP is correctly > updated to the new private IP. 'nodetool status' shows that both nodes are up > and the cluster appears to function normally. > This appears to me to be the root cause of > https://issues.apache.org/jira/browse/CASSANDRA-7292. Possibly > https://issues.apache.org/jira/browse/CASSANDRA-8072 as well, but I am not > convinced they are actually duplicates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8274) Node fails to rejoin cluster on EC2 if private IP is changed
[ https://issues.apache.org/jira/browse/CASSANDRA-8274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201244#comment-14201244 ] Joseph Clark commented on CASSANDRA-8274: - 7292 and 8072 have different stack traces. Which did you reproduce? > Node fails to rejoin cluster on EC2 if private IP is changed > > > Key: CASSANDRA-8274 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8274 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Amazon EC2 >Reporter: Joseph Clark > > Nodes in Amazon AWS EC2 Classic (not a VPC) may be assigned a new private IP > if the node is stopped and then started again. In this case we have puppet > update the configured listen_address to the new private IP. However, once the > cassandra service starts, it is unable to communicate with the existing > nodes(single region) and vice versa. > 'nodetool status' shows that each node believes that it is 'UN' and the other > node is 'DN'. > 'nodetool gossipinfo' on the node that remained running shows the *old* > private IP listed as the 'INTERNAL_IP' of the node that was stopped and > restarted. > The situation is resolved by restarting the cassandra service on the node > that remained running. Once it has restarted, the INTERNAL_IP is correctly > updated to the new private IP. 'nodetool status' shows that both nodes are up > and the cluster appears to function normally. > This appears to me to be the root cause of > https://issues.apache.org/jira/browse/CASSANDRA-7292. Possibly > https://issues.apache.org/jira/browse/CASSANDRA-8072 as well, but I am not > convinced they are actually duplicates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8072) Exception during startup: Unable to gossip with any seeds
[ https://issues.apache.org/jira/browse/CASSANDRA-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201236#comment-14201236 ] Joseph Clark edited comment on CASSANDRA-8072 at 11/6/14 11:52 PM: --- [CASSANDRA-8274|https://issues.apache.org/jira/browse/CASSANDRA-8274] appears to me to be the root cause, in my situation at least, to [CASSANDRA-7292|https://issues.apache.org/jira/browse/CASSANDRA-7292]. I'm still not convinced that 7292 and 8072 are duplicate issues. was (Author: jw.clark): [CASSANDRA-8274|https://issues.apache.org/jira/browse/CASSANDRA-8274] appears to me to be the root cause, in my situation at least, to [CASSANDRA-7292|https://issues.apache.org/jira/browse/CASSANDRA-7292]. I'm still not convinced that 7292 is a duplicate issue. > Exception during startup: Unable to gossip with any seeds > - > > Key: CASSANDRA-8072 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8072 > Project: Cassandra > Issue Type: Bug >Reporter: Ryan Springer >Assignee: Brandon Williams > > When Opscenter 4.1.4 or 5.0.1 tries to provision a 2-node DSC 2.0.10 cluster > in either ec2 or locally, an error occurs sometimes with one of the nodes > refusing to start C*. The error in the /var/log/cassandra/system.log is: > ERROR [main] 2014-10-06 15:54:52,292 CassandraDaemon.java (line 513) > Exception encountered during startup > java.lang.RuntimeException: Unable to gossip with any seeds > at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1200) > at > org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:444) > at > org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:655) > at > org.apache.cassandra.service.StorageService.initServer(StorageService.java:609) > at > org.apache.cassandra.service.StorageService.initServer(StorageService.java:502) > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378) > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496) > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585) > INFO [StorageServiceShutdownHook] 2014-10-06 15:54:52,326 Gossiper.java > (line 1279) Announcing shutdown > INFO [StorageServiceShutdownHook] 2014-10-06 15:54:54,326 > MessagingService.java (line 701) Waiting for messaging service to quiesce > INFO [ACCEPT-localhost/127.0.0.1] 2014-10-06 15:54:54,327 > MessagingService.java (line 941) MessagingService has terminated the accept() > thread > This errors does not always occur when provisioning a 2-node cluster, but > probably around half of the time on only one of the nodes. I haven't been > able to reproduce this error with DSC 2.0.9, and there have been no code or > definition file changes in Opscenter. > I can reproduce locally with the above steps. I'm happy to test any proposed > fixes since I'm the only person able to reproduce reliably so far. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8072) Exception during startup: Unable to gossip with any seeds
[ https://issues.apache.org/jira/browse/CASSANDRA-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201236#comment-14201236 ] Joseph Clark commented on CASSANDRA-8072: - [CASSANDRA-8274|https://issues.apache.org/jira/browse/CASSANDRA-8274] appears to me to be the root cause, in my situation at least, to [CASSANDRA-7292|https://issues.apache.org/jira/browse/CASSANDRA-7292]. I'm still not convinced that 7292 is a duplicate issue. > Exception during startup: Unable to gossip with any seeds > - > > Key: CASSANDRA-8072 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8072 > Project: Cassandra > Issue Type: Bug >Reporter: Ryan Springer >Assignee: Brandon Williams > > When Opscenter 4.1.4 or 5.0.1 tries to provision a 2-node DSC 2.0.10 cluster > in either ec2 or locally, an error occurs sometimes with one of the nodes > refusing to start C*. The error in the /var/log/cassandra/system.log is: > ERROR [main] 2014-10-06 15:54:52,292 CassandraDaemon.java (line 513) > Exception encountered during startup > java.lang.RuntimeException: Unable to gossip with any seeds > at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1200) > at > org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:444) > at > org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:655) > at > org.apache.cassandra.service.StorageService.initServer(StorageService.java:609) > at > org.apache.cassandra.service.StorageService.initServer(StorageService.java:502) > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378) > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496) > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585) > INFO [StorageServiceShutdownHook] 2014-10-06 15:54:52,326 Gossiper.java > (line 1279) Announcing shutdown > INFO [StorageServiceShutdownHook] 2014-10-06 15:54:54,326 > MessagingService.java (line 701) Waiting for messaging service to quiesce > INFO [ACCEPT-localhost/127.0.0.1] 2014-10-06 15:54:54,327 > MessagingService.java (line 941) MessagingService has terminated the accept() > thread > This errors does not always occur when provisioning a 2-node cluster, but > probably around half of the time on only one of the nodes. I haven't been > able to reproduce this error with DSC 2.0.9, and there have been no code or > definition file changes in Opscenter. > I can reproduce locally with the above steps. I'm happy to test any proposed > fixes since I'm the only person able to reproduce reliably so far. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8274) Node fails to rejoin cluster on EC2 if private IP is changed
[ https://issues.apache.org/jira/browse/CASSANDRA-8274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201230#comment-14201230 ] Brandon Williams commented on CASSANDRA-8274: - bq. This appears to me to be the root cause of https://issues.apache.org/jira/browse/CASSANDRA-7292. Possibly https://issues.apache.org/jira/browse/CASSANDRA-8072 Neither, because I've repro'd it and don't use EC2. > Node fails to rejoin cluster on EC2 if private IP is changed > > > Key: CASSANDRA-8274 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8274 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Amazon EC2 >Reporter: Joseph Clark > > Nodes in Amazon AWS EC2 Classic (not a VPC) may be assigned a new private IP > if the node is stopped and then started again. In this case we have puppet > update the configured listen_address to the new private IP. However, once the > cassandra service starts, it is unable to communicate with the existing > nodes(single region) and vice versa. > 'nodetool status' shows that each node believes that it is 'UN' and the other > node is 'DN'. > 'nodetool gossipinfo' on the node that remained running shows the *old* > private IP listed as the 'INTERNAL_IP' of the node that was stopped and > restarted. > The situation is resolved by restarting the cassandra service on the node > that remained running. Once it has restarted, the INTERNAL_IP is correctly > updated to the new private IP. 'nodetool status' shows that both nodes are up > and the cluster appears to function normally. > This appears to me to be the root cause of > https://issues.apache.org/jira/browse/CASSANDRA-7292. Possibly > https://issues.apache.org/jira/browse/CASSANDRA-8072 as well, but I am not > convinced they are actually duplicates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8274) Node fails to rejoin cluster on EC2 if private IP is changed
Joseph Clark created CASSANDRA-8274: --- Summary: Node fails to rejoin cluster on EC2 if private IP is changed Key: CASSANDRA-8274 URL: https://issues.apache.org/jira/browse/CASSANDRA-8274 Project: Cassandra Issue Type: Bug Components: Core Environment: Amazon EC2 Reporter: Joseph Clark Nodes in Amazon AWS EC2 Classic (not a VPC) may be assigned a new private IP if the node is stopped and then started again. In this case we have puppet update the configured listen_address to the new private IP. However, once the cassandra service starts, it is unable to communicate with the existing nodes(single region) and vice versa. 'nodetool status' shows that each node believes that it is 'UN' and the other node is 'DN'. 'nodetool gossipinfo' on the node that remained running shows the *old* private IP listed as the 'INTERNAL_IP' of the node that was stopped and restarted. The situation is resolved by restarting the cassandra service on the node that remained running. Once it has restarted, the INTERNAL_IP is correctly updated to the new private IP. 'nodetool status' shows that both nodes are up and the cluster appears to function normally. This appears to me to be the root cause of https://issues.apache.org/jira/browse/CASSANDRA-7292. Possibly https://issues.apache.org/jira/browse/CASSANDRA-8072 as well, but I am not convinced they are actually duplicates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7973) cqlsh connect error "member_descriptor' object is not callable"
[
https://issues.apache.org/jira/browse/CASSANDRA-7973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler updated CASSANDRA-7973:
--
Labels: cqlsh lhf (was: cqlsh)
> cqlsh connect error "member_descriptor' object is not callable"
> ---
>
> Key: CASSANDRA-7973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7973
> Project: Cassandra
> Issue Type: Bug
> Environment: Cassandra 2.1.0
>Reporter: Digant Modha
>Priority: Minor
> Labels: cqlsh, lhf
>
> When using cqlsh (Cassandra 2.1.0) with ssl, python 2.6.9. I get Connection
> error: ('Unable to connect to any servers', {...:
> TypeError("'member_descriptor' object is not callable",)})
> I am able to connect from another machine using python 2.7.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7973) cqlsh connect error "member_descriptor' object is not callable"
[
https://issues.apache.org/jira/browse/CASSANDRA-7973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201179#comment-14201179
]
Michael Shuler commented on CASSANDRA-7973:
---
I wonder if we need to s/python (>= 2.5)/python (>= 2.7)/ in debian/control and
document python requirement to 2.7+ somewhere for tar installs?
> cqlsh connect error "member_descriptor' object is not callable"
> ---
>
> Key: CASSANDRA-7973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7973
> Project: Cassandra
> Issue Type: Bug
> Environment: Cassandra 2.1.0
>Reporter: Digant Modha
>Priority: Minor
> Labels: cqlsh
>
> When using cqlsh (Cassandra 2.1.0) with ssl, python 2.6.9. I get Connection
> error: ('Unable to connect to any servers', {...:
> TypeError("'member_descriptor' object is not callable",)})
> I am able to connect from another machine using python 2.7.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8010) cassandra-stress needs better docs for rate options
[ https://issues.apache.org/jira/browse/CASSANDRA-8010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Shuler updated CASSANDRA-8010: -- Labels: lhf (was: ) > cassandra-stress needs better docs for rate options > --- > > Key: CASSANDRA-8010 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8010 > Project: Cassandra > Issue Type: Bug > Components: Documentation & website, Examples, Tools >Reporter: Matt Stump >Priority: Minor > Labels: lhf > > It's not obvious how to use the rate option. I wasn't able to figure it out > via the source, or from the docs. I kept trying to do -rate= or -threads=. I > had to search confluence for usage examples. > Need something like this in the docs: > -rate threads=900 > -rate threads<=900 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-7083) Authentication Support for CqlRecordWriter
[
https://issues.apache.org/jira/browse/CASSANDRA-7083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams resolved CASSANDRA-7083.
-
Resolution: Duplicate
Fix Version/s: (was: 2.0.12)
Reproduced In: 2.0.5, 1.2.16 (was: 1.2.16, 2.0.5)
Well that simplifies things :)
> Authentication Support for CqlRecordWriter
> --
>
> Key: CASSANDRA-7083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7083
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
>Reporter: Henning Kropp
>Assignee: Brandon Williams
> Labels: authentication, pig
> Attachments: auth_cql.patch
>
>
> The {{CqlRecordWriter}} seems not to support authentication. When the
> keyspace in Cassandra is to set to use authentication our Pig job fails with,
> when credentials are provided using the URI ('cql://username:password...):
> {code}
> java.lang.RuntimeException: InvalidRequestException(why:You have not logged
> in)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:123)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:90)
> at
> org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:76)
> at
> org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:57)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84)
> at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: InvalidRequestException(why:You have not logged in)
> at
> org.apache.cassandra.thrift.Cassandra$execute_cql3_query_result.read(Cassandra.java:38677)
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
> at
> org.apache.cassandra.thrift.Cassandra$Client.recv_execute_cql3_query(Cassandra.java:1597)
> at
> org.apache.cassandra.thrift.Cassandra$Client.execute_cql3_query(Cassandra.java:1582)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.retrievePartitionKeyValidator(CqlRecordWriter.java:332)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:108)
> ... 7 more
> {code}
> If not supplied in the URI but as only in the {{JobConf}} the exception is:
> {code}
> Output Location Validation Failed for: 'cql://...' More info to follow:
> InvalidRequestException(why:You have not logged in)
> at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$
> {code}
> Which let to the finding, that authentication is correctly supplied for
> {{CqlStorage}} but not for the {{CqlRecordWriter}}.
> Maybe it would make sense to put the authentication part into
> {{ConfigHelper.getClientFromAddressList()}}? Then in {{CqlStorage}} the
> username and password in the conf would need to be set from the URI. If so
> the {{ConfigHelper}} has all the information to authenticate and already
> returns the client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7083) Authentication Support for CqlRecordWriter
[
https://issues.apache.org/jira/browse/CASSANDRA-7083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201074#comment-14201074
]
Henning Kropp commented on CASSANDRA-7083:
--
Looking in current trunk it seems like it has already been fixed.
{{AbstractColumnFamilyOutputFormat}} has the login function and the client is
correctly logged in. Looks good to me.
By the way it seems like your commit: 4b8bb86e272d8ddab62f671d5109e02c32c07728
:)
> Authentication Support for CqlRecordWriter
> --
>
> Key: CASSANDRA-7083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7083
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
>Reporter: Henning Kropp
>Assignee: Brandon Williams
> Labels: authentication, pig
> Fix For: 2.0.12
>
> Attachments: auth_cql.patch
>
>
> The {{CqlRecordWriter}} seems not to support authentication. When the
> keyspace in Cassandra is to set to use authentication our Pig job fails with,
> when credentials are provided using the URI ('cql://username:password...):
> {code}
> java.lang.RuntimeException: InvalidRequestException(why:You have not logged
> in)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:123)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:90)
> at
> org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:76)
> at
> org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:57)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84)
> at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: InvalidRequestException(why:You have not logged in)
> at
> org.apache.cassandra.thrift.Cassandra$execute_cql3_query_result.read(Cassandra.java:38677)
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
> at
> org.apache.cassandra.thrift.Cassandra$Client.recv_execute_cql3_query(Cassandra.java:1597)
> at
> org.apache.cassandra.thrift.Cassandra$Client.execute_cql3_query(Cassandra.java:1582)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.retrievePartitionKeyValidator(CqlRecordWriter.java:332)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:108)
> ... 7 more
> {code}
> If not supplied in the URI but as only in the {{JobConf}} the exception is:
> {code}
> Output Location Validation Failed for: 'cql://...' More info to follow:
> InvalidRequestException(why:You have not logged in)
> at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$
> {code}
> Which let to the finding, that authentication is correctly supplied for
> {{CqlStorage}} but not for the {{CqlRecordWriter}}.
> Maybe it would make sense to put the authentication part into
> {{ConfigHelper.getClientFromAddressList()}}? Then in {{CqlStorage}} the
> username and password in the conf would need to be set from the URI. If so
> the {{ConfigHelper}} has all the information to authenticate and already
> returns the client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8221) Specify keyspace in error message when streaming fails due to missing replicas
[ https://issues.apache.org/jira/browse/CASSANDRA-8221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201067#comment-14201067 ] Rajanarayanan Thottuvaikkatumana commented on CASSANDRA-8221: - Attached patches for both the 2.0 and 2.1 branches. Thanks > Specify keyspace in error message when streaming fails due to missing replicas > -- > > Key: CASSANDRA-8221 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8221 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Tyler Hobbs >Assignee: Rajanarayanan Thottuvaikkatumana >Priority: Trivial > Labels: lhf > Fix For: 2.0.12, 2.1.2 > > Attachments: cassandra-2.0.11-8221.txt, cassandra-2.1.1-8221.txt > > > When there aren't sufficient live replicas for streaming (during bootstrap, > etc), you'll get an error message like "unable to find sufficient sources for > streaming range". It would be helpful to include the keyspace that this > failed for, since each keyspace can have different replication settings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8221) Specify keyspace in error message when streaming fails due to missing replicas
[ https://issues.apache.org/jira/browse/CASSANDRA-8221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajanarayanan Thottuvaikkatumana updated CASSANDRA-8221: Attachment: cassandra-2.0.11-8221.txt Attached the patch for the 2.0.11 branch > Specify keyspace in error message when streaming fails due to missing replicas > -- > > Key: CASSANDRA-8221 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8221 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Tyler Hobbs >Assignee: Rajanarayanan Thottuvaikkatumana >Priority: Trivial > Labels: lhf > Fix For: 2.0.12, 2.1.2 > > Attachments: cassandra-2.0.11-8221.txt, cassandra-2.1.1-8221.txt > > > When there aren't sufficient live replicas for streaming (during bootstrap, > etc), you'll get an error message like "unable to find sufficient sources for > streaming range". It would be helpful to include the keyspace that this > failed for, since each keyspace can have different replication settings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8221) Specify keyspace in error message when streaming fails due to missing replicas
[ https://issues.apache.org/jira/browse/CASSANDRA-8221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajanarayanan Thottuvaikkatumana updated CASSANDRA-8221: Attachment: cassandra-2.1.1-8221.txt Attaching the patch for 2.1.1 after making the intendation changes. Thanks > Specify keyspace in error message when streaming fails due to missing replicas > -- > > Key: CASSANDRA-8221 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8221 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Tyler Hobbs >Assignee: Rajanarayanan Thottuvaikkatumana >Priority: Trivial > Labels: lhf > Fix For: 2.0.12, 2.1.2 > > Attachments: cassandra-2.1.1-8221.txt > > > When there aren't sufficient live replicas for streaming (during bootstrap, > etc), you'll get an error message like "unable to find sufficient sources for > streaming range". It would be helpful to include the keyspace that this > failed for, since each keyspace can have different replication settings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8221) Specify keyspace in error message when streaming fails due to missing replicas
[ https://issues.apache.org/jira/browse/CASSANDRA-8221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajanarayanan Thottuvaikkatumana updated CASSANDRA-8221: Attachment: (was: cassandra-2.1.1-8221.txt) > Specify keyspace in error message when streaming fails due to missing replicas > -- > > Key: CASSANDRA-8221 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8221 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Tyler Hobbs >Assignee: Rajanarayanan Thottuvaikkatumana >Priority: Trivial > Labels: lhf > Fix For: 2.0.12, 2.1.2 > > > When there aren't sufficient live replicas for streaming (during bootstrap, > etc), you'll get an error message like "unable to find sufficient sources for > streaming range". It would be helpful to include the keyspace that this > failed for, since each keyspace can have different replication settings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7970) JSON support for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200992#comment-14200992
]
Jonathan Ellis commented on CASSANDRA-7970:
---
I think we should apply Postel's principle here. If we have a map
and someone gives us {'1': 'foo'} we should turn it into an integer rather than
rejecting it. Especially since, as you point out, there's really no other
option absent transform functions that we're not implementing initially.
> JSON support for CQL
>
>
> Key: CASSANDRA-7970
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7970
> Project: Cassandra
> Issue Type: Bug
> Components: API
>Reporter: Jonathan Ellis
>Assignee: Tyler Hobbs
> Fix For: 3.0
>
>
> JSON is popular enough that not supporting it is becoming a competitive
> weakness. We can add JSON support in a way that is compatible with our
> performance goals by *mapping* JSON to an existing schema: one JSON documents
> maps to one CQL row.
> Thus, it is NOT a goal to support schemaless documents, which is a misfeature
> [1] [2] [3]. Rather, it is to allow a convenient way to easily turn a JSON
> document from a service or a user into a CQL row, with all the validation
> that entails.
> Since we are not looking to support schemaless documents, we will not be
> adding a JSON data type (CASSANDRA-6833) a la postgresql. Rather, we will
> map the JSON to UDT, collections, and primitive CQL types.
> Here's how this might look:
> {code}
> CREATE TYPE address (
> street text,
> city text,
> zip_code int,
> phones set
> );
> CREATE TABLE users (
> id uuid PRIMARY KEY,
> name text,
> addresses map
> );
> INSERT INTO users JSON
> {‘id’: 4b856557-7153,
>‘name’: ‘jbellis’,
>‘address’: {“home”: {“street”: “123 Cassandra Dr”,
> “city”: “Austin”,
> “zip_code”: 78747,
> “phones”: [2101234567]}}};
> SELECT JSON id, address FROM users;
> {code}
> (We would also want to_json and from_json functions to allow mapping a single
> column's worth of data. These would not require extra syntax.)
> [1] http://rustyrazorblade.com/2014/07/the-myth-of-schema-less/
> [2] https://blog.compose.io/schema-less-is-usually-a-lie/
> [3] http://dl.acm.org/citation.cfm?id=2481247
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8272) 2ndary indexes can return stale data
[
https://issues.apache.org/jira/browse/CASSANDRA-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-8272:
--
Fix Version/s: 2.1.3
Tagging 2.1.3 rather than earlier based on the reasoning that since this has
been around since 2i was added in 0.7 with nobody hitting it in the wild, it
must not be a very common occurrence.
> 2ndary indexes can return stale data
>
>
> Key: CASSANDRA-8272
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8272
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
> Fix For: 2.1.3
>
>
> When replica return 2ndary index results, it's possible for a single replica
> to return a stale result and that result will be sent back to the user,
> potentially failing the CL contract.
> For instance, consider 3 replicas A, B and C, and the following situation:
> {noformat}
> CREATE TABLE test (k int PRIMARY KEY, v text);
> CREATE INDEX ON test(v);
> INSERT INTO test(k, v) VALUES (0, 'foo');
> {noformat}
> with every replica up to date. Now, suppose that the following queries are
> done at {{QUORUM}}:
> {noformat}
> UPDATE test SET v = 'bar' WHERE k = 0;
> SELECT * FROM test WHERE v = 'foo';
> {noformat}
> then, if A and B acknowledge the insert but C respond to the read before
> having applied the insert, then the now stale result will be returned (since
> C will return it and A or B will return nothing).
> A potential solution would be that when we read a tombstone in the index (and
> provided we make the index inherit the gcGrace of it's parent CF), instead of
> skipping that tombstone, we'd insert in the result a corresponding range
> tombstone.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7970) JSON support for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200981#comment-14200981
]
Joshua McKenzie commented on CASSANDRA-7970:
Option 2 seems consistent from a JSON ecosystem perspective as well.
> JSON support for CQL
>
>
> Key: CASSANDRA-7970
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7970
> Project: Cassandra
> Issue Type: Bug
> Components: API
>Reporter: Jonathan Ellis
>Assignee: Tyler Hobbs
> Fix For: 3.0
>
>
> JSON is popular enough that not supporting it is becoming a competitive
> weakness. We can add JSON support in a way that is compatible with our
> performance goals by *mapping* JSON to an existing schema: one JSON documents
> maps to one CQL row.
> Thus, it is NOT a goal to support schemaless documents, which is a misfeature
> [1] [2] [3]. Rather, it is to allow a convenient way to easily turn a JSON
> document from a service or a user into a CQL row, with all the validation
> that entails.
> Since we are not looking to support schemaless documents, we will not be
> adding a JSON data type (CASSANDRA-6833) a la postgresql. Rather, we will
> map the JSON to UDT, collections, and primitive CQL types.
> Here's how this might look:
> {code}
> CREATE TYPE address (
> street text,
> city text,
> zip_code int,
> phones set
> );
> CREATE TABLE users (
> id uuid PRIMARY KEY,
> name text,
> addresses map
> );
> INSERT INTO users JSON
> {‘id’: 4b856557-7153,
>‘name’: ‘jbellis’,
>‘address’: {“home”: {“street”: “123 Cassandra Dr”,
> “city”: “Austin”,
> “zip_code”: 78747,
> “phones”: [2101234567]}}};
> SELECT JSON id, address FROM users;
> {code}
> (We would also want to_json and from_json functions to allow mapping a single
> column's worth of data. These would not require extra syntax.)
> [1] http://rustyrazorblade.com/2014/07/the-myth-of-schema-less/
> [2] https://blog.compose.io/schema-less-is-usually-a-lie/
> [3] http://dl.acm.org/citation.cfm?id=2481247
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7970) JSON support for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200982#comment-14200982
]
Aleksey Yeschenko commented on CASSANDRA-7970:
--
bq. Option 2 would be my preference.
Likewise.
> JSON support for CQL
>
>
> Key: CASSANDRA-7970
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7970
> Project: Cassandra
> Issue Type: Bug
> Components: API
>Reporter: Jonathan Ellis
>Assignee: Tyler Hobbs
> Fix For: 3.0
>
>
> JSON is popular enough that not supporting it is becoming a competitive
> weakness. We can add JSON support in a way that is compatible with our
> performance goals by *mapping* JSON to an existing schema: one JSON documents
> maps to one CQL row.
> Thus, it is NOT a goal to support schemaless documents, which is a misfeature
> [1] [2] [3]. Rather, it is to allow a convenient way to easily turn a JSON
> document from a service or a user into a CQL row, with all the validation
> that entails.
> Since we are not looking to support schemaless documents, we will not be
> adding a JSON data type (CASSANDRA-6833) a la postgresql. Rather, we will
> map the JSON to UDT, collections, and primitive CQL types.
> Here's how this might look:
> {code}
> CREATE TYPE address (
> street text,
> city text,
> zip_code int,
> phones set
> );
> CREATE TABLE users (
> id uuid PRIMARY KEY,
> name text,
> addresses map
> );
> INSERT INTO users JSON
> {‘id’: 4b856557-7153,
>‘name’: ‘jbellis’,
>‘address’: {“home”: {“street”: “123 Cassandra Dr”,
> “city”: “Austin”,
> “zip_code”: 78747,
> “phones”: [2101234567]}}};
> SELECT JSON id, address FROM users;
> {code}
> (We would also want to_json and from_json functions to allow mapping a single
> column's worth of data. These would not require extra syntax.)
> [1] http://rustyrazorblade.com/2014/07/the-myth-of-schema-less/
> [2] https://blog.compose.io/schema-less-is-usually-a-lie/
> [3] http://dl.acm.org/citation.cfm?id=2481247
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7970) JSON support for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200974#comment-14200974
]
Tyler Hobbs commented on CASSANDRA-7970:
There's one problem that I forgot about: the JSON spec only accepts strings for
map keys.
I think that leaves us with two options:
# Accept a string version of anything (including ints, floats, etc). For
example, if we're expecting a float when decoding JSON and the JSON decoder
returns a String, we should try to create a float from it. For {{toJson()}},
we would convert non-strings to strings when they're map keys.
# Don't allow {{toJson()}} and {{fromJson()}} to be used on maps with
non-string keys. If we have transform functions at some point, those could be
used to support these kinds of maps.
Option 2 would be my preference.
> JSON support for CQL
>
>
> Key: CASSANDRA-7970
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7970
> Project: Cassandra
> Issue Type: Bug
> Components: API
>Reporter: Jonathan Ellis
>Assignee: Tyler Hobbs
> Fix For: 3.0
>
>
> JSON is popular enough that not supporting it is becoming a competitive
> weakness. We can add JSON support in a way that is compatible with our
> performance goals by *mapping* JSON to an existing schema: one JSON documents
> maps to one CQL row.
> Thus, it is NOT a goal to support schemaless documents, which is a misfeature
> [1] [2] [3]. Rather, it is to allow a convenient way to easily turn a JSON
> document from a service or a user into a CQL row, with all the validation
> that entails.
> Since we are not looking to support schemaless documents, we will not be
> adding a JSON data type (CASSANDRA-6833) a la postgresql. Rather, we will
> map the JSON to UDT, collections, and primitive CQL types.
> Here's how this might look:
> {code}
> CREATE TYPE address (
> street text,
> city text,
> zip_code int,
> phones set
> );
> CREATE TABLE users (
> id uuid PRIMARY KEY,
> name text,
> addresses map
> );
> INSERT INTO users JSON
> {‘id’: 4b856557-7153,
>‘name’: ‘jbellis’,
>‘address’: {“home”: {“street”: “123 Cassandra Dr”,
> “city”: “Austin”,
> “zip_code”: 78747,
> “phones”: [2101234567]}}};
> SELECT JSON id, address FROM users;
> {code}
> (We would also want to_json and from_json functions to allow mapping a single
> column's worth of data. These would not require extra syntax.)
> [1] http://rustyrazorblade.com/2014/07/the-myth-of-schema-less/
> [2] https://blog.compose.io/schema-less-is-usually-a-lie/
> [3] http://dl.acm.org/citation.cfm?id=2481247
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8273) Allow filtering queries can return stale data
Sylvain Lebresne created CASSANDRA-8273:
---
Summary: Allow filtering queries can return stale data
Key: CASSANDRA-8273
URL: https://issues.apache.org/jira/browse/CASSANDRA-8273
Project: Cassandra
Issue Type: Bug
Reporter: Sylvain Lebresne
Data filtering is done replica side. That means that a single replica with
stale data may make the whole query return that stale data.
For instance, consider 3 replicas A, B and C, and the following situation:
{noformat}
CREATE TABLE test (k int PRIMARY KEY, v1 text, v2 int);
CREATE INDEX ON test(v1);
INSERT INTO test(k, v1, v2) VALUES (0, 'foo', 1);
{noformat}
with every replica up to date. Now, suppose that the following queries are done
at {{QUORUM}}:
{noformat}
UPDATE test SET v2 = 2 WHERE k = 0;
SELECT * FROM test WHERE v1 = 'foo' AND v2 = 1;
{noformat}
then, if A and B acknowledge the insert but C respond to the read before having
applied the insert, then the now stale result will be returned. Let's note that
this is a problem related to filtering, not 2ndary indexes.
This issue share similarity with CASSANDRA-8272 but contrarily to that former
issue, I'm not sure how to fix it. Obviously, moving the filtering to the
coordinator would remove that problem, but doing so would, on top of not being
trivial to implmenent, have serious performance impact since we can't know in
advance how much data will be filtered and we may have to redo query to replica
multiple times.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8272) 2ndary indexes can return stale data
Sylvain Lebresne created CASSANDRA-8272:
---
Summary: 2ndary indexes can return stale data
Key: CASSANDRA-8272
URL: https://issues.apache.org/jira/browse/CASSANDRA-8272
Project: Cassandra
Issue Type: Bug
Reporter: Sylvain Lebresne
When replica return 2ndary index results, it's possible for a single replica to
return a stale result and that result will be sent back to the user,
potentially failing the CL contract.
For instance, consider 3 replicas A, B and C, and the following situation:
{noformat}
CREATE TABLE test (k int PRIMARY KEY, v text);
CREATE INDEX ON test(v);
INSERT INTO test(k, v) VALUES (0, 'foo');
{noformat}
with every replica up to date. Now, suppose that the following queries are done
at {{QUORUM}}:
{noformat}
UPDATE test SET v = 'bar' WHERE k = 0;
SELECT * FROM test WHERE v = 'foo';
{noformat}
then, if A and B acknowledge the insert but C respond to the read before having
applied the insert, then the now stale result will be returned (since C will
return it and A or B will return nothing).
A potential solution would be that when we read a tombstone in the index (and
provided we make the index inherit the gcGrace of it's parent CF), instead of
skipping that tombstone, we'd insert in the result a corresponding range
tombstone.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7826) support non-frozen, nested collections
[ https://issues.apache.org/jira/browse/CASSANDRA-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-7826: -- Component/s: API Fix Version/s: 3.0 Summary: support non-frozen, nested collections (was: support arbitrary nesting of collection) okay, edited title of this ticket to reflect that it's specifically for non-frozen collections, and tagged it 3.0 > support non-frozen, nested collections > -- > > Key: CASSANDRA-7826 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7826 > Project: Cassandra > Issue Type: Improvement > Components: API, Core >Reporter: Tupshin Harper >Assignee: Tyler Hobbs > Labels: ponies > Fix For: 3.0 > > > The inability to nest collections is one of the bigger data modelling > limitations we have right now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8236) Delay "node up" and "node added" notifications until native protocol server is started
[
https://issues.apache.org/jira/browse/CASSANDRA-8236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200941#comment-14200941
]
Brandon Williams commented on CASSANDRA-8236:
-
We could also flip this back to false if the native server is stopped (via
nodetool or the like) but I'm not sure if onDown is really the correct
notification to send for that.
> Delay "node up" and "node added" notifications until native protocol server
> is started
> --
>
> Key: CASSANDRA-8236
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8236
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Brandon Williams
> Fix For: 3.0
>
> Attachments: 8236.txt
>
>
> As discussed in CASSANDRA-7510, there is still a gap between when a "node up"
> or "node added" notification may be sent to native protocol clients (in
> response to a gossip event) and when the native protocol server is ready to
> serve requests.
> Everything in between the call to {{StorageService.instance.initServer()}}
> and creation of the native server in {{CassandraDaemon.setup()}} contributes
> to this delay, but waiting for Gossip to settle introduces the biggest delay.
> We may need to introduce a "STARTING" gossip state for the period inbetween,
> which is why this is scheduled for 3.0. If there's a better option, though,
> it may make sense to put this in 2.1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8236) Delay "node up" and "node added" notifications until native protocol server is started
[
https://issues.apache.org/jira/browse/CASSANDRA-8236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-8236:
Attachment: 8236.txt
> Delay "node up" and "node added" notifications until native protocol server
> is started
> --
>
> Key: CASSANDRA-8236
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8236
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Brandon Williams
> Fix For: 3.0
>
> Attachments: 8236.txt
>
>
> As discussed in CASSANDRA-7510, there is still a gap between when a "node up"
> or "node added" notification may be sent to native protocol clients (in
> response to a gossip event) and when the native protocol server is ready to
> serve requests.
> Everything in between the call to {{StorageService.instance.initServer()}}
> and creation of the native server in {{CassandraDaemon.setup()}} contributes
> to this delay, but waiting for Gossip to settle introduces the biggest delay.
> We may need to introduce a "STARTING" gossip state for the period inbetween,
> which is why this is scheduled for 3.0. If there's a better option, though,
> it may make sense to put this in 2.1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8236) Delay "node up" and "node added" notifications until native protocol server is started
[
https://issues.apache.org/jira/browse/CASSANDRA-8236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-8236:
Attachment: (was: 8236.txt)
> Delay "node up" and "node added" notifications until native protocol server
> is started
> --
>
> Key: CASSANDRA-8236
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8236
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Brandon Williams
> Fix For: 3.0
>
> Attachments: 8236.txt
>
>
> As discussed in CASSANDRA-7510, there is still a gap between when a "node up"
> or "node added" notification may be sent to native protocol clients (in
> response to a gossip event) and when the native protocol server is ready to
> serve requests.
> Everything in between the call to {{StorageService.instance.initServer()}}
> and creation of the native server in {{CassandraDaemon.setup()}} contributes
> to this delay, but waiting for Gossip to settle introduces the biggest delay.
> We may need to introduce a "STARTING" gossip state for the period inbetween,
> which is why this is scheduled for 3.0. If there's a better option, though,
> it may make sense to put this in 2.1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8236) Delay "node up" and "node added" notifications until native protocol server is started
[
https://issues.apache.org/jira/browse/CASSANDRA-8236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-8236:
Attachment: 8236.txt
Patch to add a new ApplicationState, RPC_READY, which is a boolean that
defaults to false at startup. Once the native server is started, it then flips
this to true. If the remote node is older, we preserve the behavior of
CASSANDRA-7510 and give our best effort to notify clients when the server is
ready.
Note that while this didn't break anything, I didn't actually test it with a
native client.
> Delay "node up" and "node added" notifications until native protocol server
> is started
> --
>
> Key: CASSANDRA-8236
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8236
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Brandon Williams
> Fix For: 3.0
>
> Attachments: 8236.txt
>
>
> As discussed in CASSANDRA-7510, there is still a gap between when a "node up"
> or "node added" notification may be sent to native protocol clients (in
> response to a gossip event) and when the native protocol server is ready to
> serve requests.
> Everything in between the call to {{StorageService.instance.initServer()}}
> and creation of the native server in {{CassandraDaemon.setup()}} contributes
> to this delay, but waiting for Gossip to settle introduces the biggest delay.
> We may need to introduce a "STARTING" gossip state for the period inbetween,
> which is why this is scheduled for 3.0. If there's a better option, though,
> it may make sense to put this in 2.1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7826) support arbitrary nesting of collection
[ https://issues.apache.org/jira/browse/CASSANDRA-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200865#comment-14200865 ] Tyler Hobbs commented on CASSANDRA-7826: bq. Do you want to create a separate ticket for frozen nested collections in 2.x? I suppose I wasn't very clear -- CASSANDRA-7859 supports frozen nested collections as-is, so there's no need for a separate ticket. > support arbitrary nesting of collection > --- > > Key: CASSANDRA-7826 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7826 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Tupshin Harper >Assignee: Tyler Hobbs > Labels: ponies > > The inability to nest collections is one of the bigger data modelling > limitations we have right now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ffe69c3d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ffe69c3d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ffe69c3d Branch: refs/heads/trunk Commit: ffe69c3d7dd3b529c90bdd30356d991998289284 Parents: 9685622 6b3f3e9 Author: Tyler Hobbs Authored: Thu Nov 6 14:31:55 2014 -0600 Committer: Tyler Hobbs Committed: Thu Nov 6 14:31:55 2014 -0600 -- CHANGES.txt | 2 ++ .../org/apache/cassandra/locator/PropertyFileSnitch.java | 11 ++- 2 files changed, 12 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ffe69c3d/CHANGES.txt -- diff --cc CHANGES.txt index 89db48e,8a7697b..c1cc083 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,33 -1,6 +1,35 @@@ -2.0.12: +2.1.3 + * Fix overflow on histogram computation (CASSANDRA-8028) + * Have paxos reuse the timestamp generation of normal queries (CASSANDRA-7801) ++Merged from 2.0: + * Improve PropertyFileSnitch logging (CASSANDRA-8183) - * Abort liveRatio calculation if the memtable is flushed (CASSANDRA-8164) + +2.1.2 + * (cqlsh) parse_for_table_meta errors out on queries with undefined + grammars (CASSANDRA-8262) + * (cqlsh) Fix SELECT ... TOKEN() function broken in C* 2.1.1 (CASSANDRA-8258) + * Fix Cassandra crash when running on JDK8 update 40 (CASSANDRA-8209) + * Optimize partitioner tokens (CASSANDRA-8230) + * Improve compaction of repaired/unrepaired sstables (CASSANDRA-8004) + * Make cache serializers pluggable (CASSANDRA-8096) + * Fix issues with CONTAINS (KEY) queries on secondary indexes + (CASSANDRA-8147) + * Fix read-rate tracking of sstables for some queries (CASSANDRA-8239) + * Fix default timestamp in QueryOptions (CASSANDRA-8246) + * Set socket timeout when reading remote version (CASSANDRA-8188) + * Refactor how we track live size (CASSANDRA-7852) + * Make sure unfinished compaction files are removed (CASSANDRA-8124) + * Fix shutdown when run as Windows service (CASSANDRA-8136) + * Fix DESCRIBE TABLE with custom indexes (CASSANDRA-8031) + * Fix race in RecoveryManagerTest (CASSANDRA-8176) + * Avoid IllegalArgumentException while sorting sstables in + IndexSummaryManager (CASSANDRA-8182) + * Shutdown JVM on file descriptor exhaustion (CASSANDRA-7579) + * Add 'die' policy for commit log and disk failure (CASSANDRA-7927) + * Fix installing as service on Windows (CASSANDRA-8115) + * Fix CREATE TABLE for CQL2 (CASSANDRA-8144) + * Avoid boxing in ColumnStats min/max trackers (CASSANDRA-8109) +Merged from 2.0: * Correctly handle non-text column names in cql3 (CASSANDRA-8178) * Fix deletion for indexes on primary key columns (CASSANDRA-8206) * Add 'nodetool statusgossip' (CASSANDRA-8125) http://git-wip-us.apache.org/repos/asf/cassandra/blob/ffe69c3d/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java --
[1/3] cassandra git commit: Improve PropertyFileSnitch log formatting
Repository: cassandra
Updated Branches:
refs/heads/trunk b4cbb28f3 -> a67980e8e
Improve PropertyFileSnitch log formatting
Patch by Liang Xie; reviewed by Tyler Hobbs for CASSANDRA-8183
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6b3f3e96
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6b3f3e96
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6b3f3e96
Branch: refs/heads/trunk
Commit: 6b3f3e9607c6e1113dd1e475b5b7c3ad91de6c03
Parents: 65f7990
Author: Liang Xie
Authored: Thu Nov 6 13:37:03 2014 -0600
Committer: Tyler Hobbs
Committed: Thu Nov 6 13:37:03 2014 -0600
--
CHANGES.txt | 1 +
.../org/apache/cassandra/locator/PropertyFileSnitch.java | 11 ++-
2 files changed, 11 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b3f3e96/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a469bbf..8a7697b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.12:
+ * Improve PropertyFileSnitch logging (CASSANDRA-8183)
* Abort liveRatio calculation if the memtable is flushed (CASSANDRA-8164)
* Correctly handle non-text column names in cql3 (CASSANDRA-8178)
* Fix deletion for indexes on primary key columns (CASSANDRA-8206)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b3f3e96/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
--
diff --git a/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
b/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
index 9138bc2..4f822c6 100644
--- a/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
+++ b/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
@@ -20,6 +20,7 @@ package org.apache.cassandra.locator;
import java.io.InputStream;
import java.net.InetAddress;
import java.net.UnknownHostException;
+import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
@@ -33,6 +34,7 @@ import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.ResourceWatcher;
import org.apache.cassandra.utils.WrappedRunnable;
+import org.apache.commons.lang3.StringUtils;
/**
* Used to determine if two IP's are in the same datacenter or on the same
rack.
@@ -185,7 +187,14 @@ public class PropertyFileSnitch extends
AbstractNetworkTopologySnitch
throw new ConfigurationException(String.format("Snitch definitions
at %s do not define a location for this node's broadcast address %s, nor does
it provides a default",
SNITCH_PROPERTIES_FILENAME, FBUtilities.getBroadcastAddress()));
-logger.debug("loaded network topology {}",
FBUtilities.toString(reloadedMap));
+if (logger.isDebugEnabled())
+{
+StringBuilder sb = new StringBuilder();
+for (Map.Entry entry :
reloadedMap.entrySet())
+
sb.append(entry.getKey()).append(":").append(Arrays.toString(entry.getValue())).append(",
");
+logger.debug("Loaded network topology from property file: {}",
StringUtils.removeEnd(sb.toString(), ", "));
+}
+
endpointMap = reloadedMap;
if (StorageService.instance != null) // null check tolerates circular
dependency; see CASSANDRA-4145
StorageService.instance.getTokenMetadata().invalidateCachedRings();
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a67980e8 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a67980e8 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a67980e8 Branch: refs/heads/trunk Commit: a67980e8ee8a6bc72bb78044ee2a1b20a3121a2a Parents: b4cbb28 ffe69c3 Author: Tyler Hobbs Authored: Thu Nov 6 14:32:23 2014 -0600 Committer: Tyler Hobbs Committed: Thu Nov 6 14:32:23 2014 -0600 -- CHANGES.txt | 2 ++ .../org/apache/cassandra/locator/PropertyFileSnitch.java | 11 ++- 2 files changed, 12 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/a67980e8/CHANGES.txt --
[jira] [Commented] (CASSANDRA-8211) Overlapping sstables in L1+
[
https://issues.apache.org/jira/browse/CASSANDRA-8211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200858#comment-14200858
]
Yuki Morishita commented on CASSANDRA-8211:
---
+1
> Overlapping sstables in L1+
> ---
>
> Key: CASSANDRA-8211
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8211
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 2.0.12
>
> Attachments: 0001-Avoid-overlaps-in-L1-v2.patch,
> 0001-Avoid-overlaps-in-L1.patch
>
>
> Seems we have a bug that can create overlapping sstables in L1:
> {code}
> WARN [main] 2014-10-28 04:09:42,295 LeveledManifest.java (line 164) At level
> 2, SSTableReader(path='') [DecoratedKey(2838397575996053472, 00
> 10066059b210066059b210400100),
> DecoratedKey(5516674013223138308,
> 001000ff2d161000ff2d160
> 00010400100)] overlaps
> SSTableReader(path='') [DecoratedKey(2839992722300822584,
> 0010
> 00229ad21000229ad210400100),
> DecoratedKey(5532836928694021724,
> 0010034b05a610034b05a6100
> 000400100)]. This could be caused by a bug in
> Cassandra 1.1.0 .. 1.1.3 or due to the fact that you have dropped sstables
> from another node into the data directory. Sending back to L0. If
> you didn't drop in sstables, and have not yet run scrub, you should do so
> since you may also have rows out-of-order within an sstable
> {code}
> Which might manifest itself during compaction with this exception:
> {code}
> ERROR [CompactionExecutor:3152] 2014-10-28 00:24:06,134 CassandraDaemon.java
> (line 199) Exception in thread Thread[CompactionExecutor:3152,1,main]
> java.lang.RuntimeException: Last written key
> DecoratedKey(5516674013223138308,
> 001000ff2d161000ff2d1610400100)
> >= current key DecoratedKey(2839992722300822584,
> 001000229ad21000229ad210400100)
> writing into
> {code}
> since we use LeveledScanner when compacting (the backing sstable scanner
> might go beyond the start of the next sstable scanner)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ffe69c3d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ffe69c3d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ffe69c3d Branch: refs/heads/cassandra-2.1 Commit: ffe69c3d7dd3b529c90bdd30356d991998289284 Parents: 9685622 6b3f3e9 Author: Tyler Hobbs Authored: Thu Nov 6 14:31:55 2014 -0600 Committer: Tyler Hobbs Committed: Thu Nov 6 14:31:55 2014 -0600 -- CHANGES.txt | 2 ++ .../org/apache/cassandra/locator/PropertyFileSnitch.java | 11 ++- 2 files changed, 12 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ffe69c3d/CHANGES.txt -- diff --cc CHANGES.txt index 89db48e,8a7697b..c1cc083 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,33 -1,6 +1,35 @@@ -2.0.12: +2.1.3 + * Fix overflow on histogram computation (CASSANDRA-8028) + * Have paxos reuse the timestamp generation of normal queries (CASSANDRA-7801) ++Merged from 2.0: + * Improve PropertyFileSnitch logging (CASSANDRA-8183) - * Abort liveRatio calculation if the memtable is flushed (CASSANDRA-8164) + +2.1.2 + * (cqlsh) parse_for_table_meta errors out on queries with undefined + grammars (CASSANDRA-8262) + * (cqlsh) Fix SELECT ... TOKEN() function broken in C* 2.1.1 (CASSANDRA-8258) + * Fix Cassandra crash when running on JDK8 update 40 (CASSANDRA-8209) + * Optimize partitioner tokens (CASSANDRA-8230) + * Improve compaction of repaired/unrepaired sstables (CASSANDRA-8004) + * Make cache serializers pluggable (CASSANDRA-8096) + * Fix issues with CONTAINS (KEY) queries on secondary indexes + (CASSANDRA-8147) + * Fix read-rate tracking of sstables for some queries (CASSANDRA-8239) + * Fix default timestamp in QueryOptions (CASSANDRA-8246) + * Set socket timeout when reading remote version (CASSANDRA-8188) + * Refactor how we track live size (CASSANDRA-7852) + * Make sure unfinished compaction files are removed (CASSANDRA-8124) + * Fix shutdown when run as Windows service (CASSANDRA-8136) + * Fix DESCRIBE TABLE with custom indexes (CASSANDRA-8031) + * Fix race in RecoveryManagerTest (CASSANDRA-8176) + * Avoid IllegalArgumentException while sorting sstables in + IndexSummaryManager (CASSANDRA-8182) + * Shutdown JVM on file descriptor exhaustion (CASSANDRA-7579) + * Add 'die' policy for commit log and disk failure (CASSANDRA-7927) + * Fix installing as service on Windows (CASSANDRA-8115) + * Fix CREATE TABLE for CQL2 (CASSANDRA-8144) + * Avoid boxing in ColumnStats min/max trackers (CASSANDRA-8109) +Merged from 2.0: * Correctly handle non-text column names in cql3 (CASSANDRA-8178) * Fix deletion for indexes on primary key columns (CASSANDRA-8206) * Add 'nodetool statusgossip' (CASSANDRA-8125) http://git-wip-us.apache.org/repos/asf/cassandra/blob/ffe69c3d/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java --
[1/2] cassandra git commit: Improve PropertyFileSnitch log formatting
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 9685622b3 -> ffe69c3d7
Improve PropertyFileSnitch log formatting
Patch by Liang Xie; reviewed by Tyler Hobbs for CASSANDRA-8183
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6b3f3e96
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6b3f3e96
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6b3f3e96
Branch: refs/heads/cassandra-2.1
Commit: 6b3f3e9607c6e1113dd1e475b5b7c3ad91de6c03
Parents: 65f7990
Author: Liang Xie
Authored: Thu Nov 6 13:37:03 2014 -0600
Committer: Tyler Hobbs
Committed: Thu Nov 6 13:37:03 2014 -0600
--
CHANGES.txt | 1 +
.../org/apache/cassandra/locator/PropertyFileSnitch.java | 11 ++-
2 files changed, 11 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b3f3e96/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a469bbf..8a7697b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.12:
+ * Improve PropertyFileSnitch logging (CASSANDRA-8183)
* Abort liveRatio calculation if the memtable is flushed (CASSANDRA-8164)
* Correctly handle non-text column names in cql3 (CASSANDRA-8178)
* Fix deletion for indexes on primary key columns (CASSANDRA-8206)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b3f3e96/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
--
diff --git a/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
b/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
index 9138bc2..4f822c6 100644
--- a/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
+++ b/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
@@ -20,6 +20,7 @@ package org.apache.cassandra.locator;
import java.io.InputStream;
import java.net.InetAddress;
import java.net.UnknownHostException;
+import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
@@ -33,6 +34,7 @@ import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.ResourceWatcher;
import org.apache.cassandra.utils.WrappedRunnable;
+import org.apache.commons.lang3.StringUtils;
/**
* Used to determine if two IP's are in the same datacenter or on the same
rack.
@@ -185,7 +187,14 @@ public class PropertyFileSnitch extends
AbstractNetworkTopologySnitch
throw new ConfigurationException(String.format("Snitch definitions
at %s do not define a location for this node's broadcast address %s, nor does
it provides a default",
SNITCH_PROPERTIES_FILENAME, FBUtilities.getBroadcastAddress()));
-logger.debug("loaded network topology {}",
FBUtilities.toString(reloadedMap));
+if (logger.isDebugEnabled())
+{
+StringBuilder sb = new StringBuilder();
+for (Map.Entry entry :
reloadedMap.entrySet())
+
sb.append(entry.getKey()).append(":").append(Arrays.toString(entry.getValue())).append(",
");
+logger.debug("Loaded network topology from property file: {}",
StringUtils.removeEnd(sb.toString(), ", "));
+}
+
endpointMap = reloadedMap;
if (StorageService.instance != null) // null check tolerates circular
dependency; see CASSANDRA-4145
StorageService.instance.getTokenMetadata().invalidateCachedRings();
cassandra git commit: Improve PropertyFileSnitch log formatting
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 65f799008 -> 6b3f3e960
Improve PropertyFileSnitch log formatting
Patch by Liang Xie; reviewed by Tyler Hobbs for CASSANDRA-8183
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6b3f3e96
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6b3f3e96
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6b3f3e96
Branch: refs/heads/cassandra-2.0
Commit: 6b3f3e9607c6e1113dd1e475b5b7c3ad91de6c03
Parents: 65f7990
Author: Liang Xie
Authored: Thu Nov 6 13:37:03 2014 -0600
Committer: Tyler Hobbs
Committed: Thu Nov 6 13:37:03 2014 -0600
--
CHANGES.txt | 1 +
.../org/apache/cassandra/locator/PropertyFileSnitch.java | 11 ++-
2 files changed, 11 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b3f3e96/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a469bbf..8a7697b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.12:
+ * Improve PropertyFileSnitch logging (CASSANDRA-8183)
* Abort liveRatio calculation if the memtable is flushed (CASSANDRA-8164)
* Correctly handle non-text column names in cql3 (CASSANDRA-8178)
* Fix deletion for indexes on primary key columns (CASSANDRA-8206)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b3f3e96/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
--
diff --git a/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
b/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
index 9138bc2..4f822c6 100644
--- a/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
+++ b/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
@@ -20,6 +20,7 @@ package org.apache.cassandra.locator;
import java.io.InputStream;
import java.net.InetAddress;
import java.net.UnknownHostException;
+import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
@@ -33,6 +34,7 @@ import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.ResourceWatcher;
import org.apache.cassandra.utils.WrappedRunnable;
+import org.apache.commons.lang3.StringUtils;
/**
* Used to determine if two IP's are in the same datacenter or on the same
rack.
@@ -185,7 +187,14 @@ public class PropertyFileSnitch extends
AbstractNetworkTopologySnitch
throw new ConfigurationException(String.format("Snitch definitions
at %s do not define a location for this node's broadcast address %s, nor does
it provides a default",
SNITCH_PROPERTIES_FILENAME, FBUtilities.getBroadcastAddress()));
-logger.debug("loaded network topology {}",
FBUtilities.toString(reloadedMap));
+if (logger.isDebugEnabled())
+{
+StringBuilder sb = new StringBuilder();
+for (Map.Entry entry :
reloadedMap.entrySet())
+
sb.append(entry.getKey()).append(":").append(Arrays.toString(entry.getValue())).append(",
");
+logger.debug("Loaded network topology from property file: {}",
StringUtils.removeEnd(sb.toString(), ", "));
+}
+
endpointMap = reloadedMap;
if (StorageService.instance != null) // null check tolerates circular
dependency; see CASSANDRA-4145
StorageService.instance.getTokenMetadata().invalidateCachedRings();
[jira] [Commented] (CASSANDRA-7826) support arbitrary nesting of collection
[ https://issues.apache.org/jira/browse/CASSANDRA-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200853#comment-14200853 ] Tupshin Harper commented on CASSANDRA-7826: --- Correct interpretation of my end goal. I'm neutral on the need/benefit of doing frozen/nested in 2.x. Personally I'd be OK deferring full nesting of collections until unfrozen nested support in 3.0. > support arbitrary nesting of collection > --- > > Key: CASSANDRA-7826 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7826 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Tupshin Harper >Assignee: Tyler Hobbs > Labels: ponies > > The inability to nest collections is one of the bigger data modelling > limitations we have right now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7826) support arbitrary nesting of collection
[ https://issues.apache.org/jira/browse/CASSANDRA-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200845#comment-14200845 ] Jonathan Ellis commented on CASSANDRA-7826: --- Right, we want both really. Do you want to create a separate ticket for frozen nested collections in 2.x? > support arbitrary nesting of collection > --- > > Key: CASSANDRA-7826 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7826 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Tupshin Harper >Assignee: Tyler Hobbs > Labels: ponies > > The inability to nest collections is one of the bigger data modelling > limitations we have right now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7124) Use JMX Notifications to Indicate Success/Failure of Long-Running Operations
[
https://issues.apache.org/jira/browse/CASSANDRA-7124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200725#comment-14200725
]
Tyler Hobbs commented on CASSANDRA-7124:
[~rnamboodiri] that sounds correct to me. There will probably be some overlap
for each of the cleanup/compact/etc async operations, so you will probably want
to make a generic Runner class that CleanupRunner, CompactionRunner, etc can
inherit from.
> Use JMX Notifications to Indicate Success/Failure of Long-Running Operations
>
>
> Key: CASSANDRA-7124
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7124
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Tyler Hobbs
>Assignee: Rajanarayanan Thottuvaikkatumana
>Priority: Minor
> Labels: lhf
> Fix For: 3.0
>
>
> If {{nodetool cleanup}} or some other long-running operation takes too long
> to complete, you'll see an error like the one in CASSANDRA-2126, so you can't
> tell if the operation completed successfully or not. CASSANDRA-4767 fixed
> this for repairs with JMX notifications. We should do something similar for
> nodetool cleanup, compact, decommission, move, relocate, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8248) Possible memory leak
[
https://issues.apache.org/jira/browse/CASSANDRA-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200707#comment-14200707
]
Michael Shuler commented on CASSANDRA-8248:
---
How much physical RAM is in the machine and what are your heap configs?
> Possible memory leak
> -
>
> Key: CASSANDRA-8248
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8248
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alexander Sterligov
>
> Sometimes during repair cassandra starts to consume more memory than expected.
> Total amount of data on node is about 20GB.
> Size of the data directory is 66GC because of snapshots.
> Top reports:
> {quote}
> PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND
> 15724 loadbase 20 0 493g 55g 44g S 28 44.2 4043:24 java
> {quote}
> At the /proc/15724/maps there are a lot of deleted file maps
> {quote}
> 7f63a6102000-7f63a6332000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6332000-7f63a6562000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6562000-7f63a6792000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6792000-7f63a69c2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a69c2000-7f63a6bf2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6bf2000-7f63a6e22000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6e22000-7f63a7052000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a7052000-7f63a7282000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a7282000-7f63a74b2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a74b2000-7f63a76e2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a76e2000-7f63a7912000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a7912000-7f63a7b42000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a7b42000-7f63a7d72000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a7d72000-7f63a7fa2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a7fa2000-7f63a81d2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a81d2000-7f63a8402000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a8402000-7f63a8622000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a8622000-7f63a8842000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a8842000-7f63a8a62000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d3
[jira] [Commented] (CASSANDRA-8248) Possible memory leak
[
https://issues.apache.org/jira/browse/CASSANDRA-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200699#comment-14200699
]
Michael Shuler commented on CASSANDRA-8248:
---
Anything interesting in thread and heap dumps? If you are only looking at the
value top gives you, your initial post had 55G resident - you just posted 51G
(assume this to also be resident). I don't see where the leak is.
> Possible memory leak
> -
>
> Key: CASSANDRA-8248
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8248
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alexander Sterligov
>
> Sometimes during repair cassandra starts to consume more memory than expected.
> Total amount of data on node is about 20GB.
> Size of the data directory is 66GC because of snapshots.
> Top reports:
> {quote}
> PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND
> 15724 loadbase 20 0 493g 55g 44g S 28 44.2 4043:24 java
> {quote}
> At the /proc/15724/maps there are a lot of deleted file maps
> {quote}
> 7f63a6102000-7f63a6332000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6332000-7f63a6562000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6562000-7f63a6792000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6792000-7f63a69c2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a69c2000-7f63a6bf2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6bf2000-7f63a6e22000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6e22000-7f63a7052000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a7052000-7f63a7282000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a7282000-7f63a74b2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a74b2000-7f63a76e2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a76e2000-7f63a7912000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a7912000-7f63a7b42000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a7b42000-7f63a7d72000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a7d72000-7f63a7fa2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a7fa2000-7f63a81d2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a81d2000-7f63a8402000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a8402000-7f63a8622000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a8622000-7f63a8842000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-k
[jira] [Commented] (CASSANDRA-8221) Specify keyspace in error message when streaming fails due to missing replicas
[ https://issues.apache.org/jira/browse/CASSANDRA-8221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200697#comment-14200697 ] Tyler Hobbs commented on CASSANDRA-8221: [~rnamboodiri] sorry for the slow response. The patch looks good, except for a whitespace nitpick: please use a space in between method parameters. With that fixed, please go ahead and attach a patch for the 2.0 branch as well. Thanks! > Specify keyspace in error message when streaming fails due to missing replicas > -- > > Key: CASSANDRA-8221 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8221 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Tyler Hobbs >Assignee: Rajanarayanan Thottuvaikkatumana >Priority: Trivial > Labels: lhf > Fix For: 2.0.12, 2.1.2 > > Attachments: cassandra-2.1.1-8221.txt > > > When there aren't sufficient live replicas for streaming (during bootstrap, > etc), you'll get an error message like "unable to find sufficient sources for > streaming range". It would be helpful to include the keyspace that this > failed for, since each keyspace can have different replication settings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7083) Authentication Support for CqlRecordWriter
[
https://issues.apache.org/jira/browse/CASSANDRA-7083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200685#comment-14200685
]
Brandon Williams commented on CASSANDRA-7083:
-
Also this doesn't compile:
{noformat}
src/java/org/apache/cassandra/hadoop/cql3/CqlRecordWriter.java:115: error:
variable password is already defined in constructor
CqlRecordWriter(Configuration)
[javac] String password =
ConfigHelper.getOutputKeyspacePassword(conf);
[javac] ^
{noformat}
> Authentication Support for CqlRecordWriter
> --
>
> Key: CASSANDRA-7083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7083
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
>Reporter: Henning Kropp
>Assignee: Brandon Williams
> Labels: authentication, pig
> Fix For: 2.0.12
>
> Attachments: auth_cql.patch
>
>
> The {{CqlRecordWriter}} seems not to support authentication. When the
> keyspace in Cassandra is to set to use authentication our Pig job fails with,
> when credentials are provided using the URI ('cql://username:password...):
> {code}
> java.lang.RuntimeException: InvalidRequestException(why:You have not logged
> in)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:123)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:90)
> at
> org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:76)
> at
> org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:57)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84)
> at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: InvalidRequestException(why:You have not logged in)
> at
> org.apache.cassandra.thrift.Cassandra$execute_cql3_query_result.read(Cassandra.java:38677)
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
> at
> org.apache.cassandra.thrift.Cassandra$Client.recv_execute_cql3_query(Cassandra.java:1597)
> at
> org.apache.cassandra.thrift.Cassandra$Client.execute_cql3_query(Cassandra.java:1582)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.retrievePartitionKeyValidator(CqlRecordWriter.java:332)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:108)
> ... 7 more
> {code}
> If not supplied in the URI but as only in the {{JobConf}} the exception is:
> {code}
> Output Location Validation Failed for: 'cql://...' More info to follow:
> InvalidRequestException(why:You have not logged in)
> at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$
> {code}
> Which let to the finding, that authentication is correctly supplied for
> {{CqlStorage}} but not for the {{CqlRecordWriter}}.
> Maybe it would make sense to put the authentication part into
> {{ConfigHelper.getClientFromAddressList()}}? Then in {{CqlStorage}} the
> username and password in the conf would need to be set from the URI. If so
> the {{ConfigHelper}} has all the information to authenticate and already
> returns the client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-7217) Native transport performance (with cassandra-stress) drops precipitously past around 1000 threads
[ https://issues.apache.org/jira/browse/CASSANDRA-7217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200604#comment-14200604 ] Shawn Kumar edited comment on CASSANDRA-7217 at 11/6/14 6:46 PM: - I'll be continuing testing on a more cpu-perfomant instance but thought I would briefly try the cstar_perf on bdplab. [Here|http://cstar.datastax.com/graph?stats=dd73c4a6-65d9-11e4-9413-bc764e04482c&metric=op_rate&operation=1_write&smoothing=1&show_aggregates=true&xmin=0&xmax=279.07&ymin=0&ymax=120665.6] are the results - I increase the threads from 500 - 1500 in 250 thread increments from the first operation to the last (ie. 1_write to 5_write) and it seems like there is a noticeable drop in performance especially around 1250 threads. was (Author: shawn.kumar): I'll be continuing testing on a more cpu-perfomant instance but thought I would briefly try the cstar_perf on bdplab. [Here|http://cstar.datastax.com/graph?stats=dd73c4a6-65d9-11e4-9413-bc764e04482c&metric=op_rate&operation=1_write&smoothing=1&show_aggregates=true&xmin=0&xmax=279.07&ymin=0&ymax=120665.6] are the results - I increase the threads from 500 - 1500 in 250 thread increments from the first operation to the last (ie. 1_write to 5_write) and it seems like there is a noticeable drop in performance especially around 1000 threads. > Native transport performance (with cassandra-stress) drops precipitously past > around 1000 threads > - > > Key: CASSANDRA-7217 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7217 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Benedict >Assignee: Shawn Kumar > Labels: performance, triaged > Fix For: 2.1.2 > > > This is obviously bad. Let's figure out why it's happening and put a stop to > it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8252) dtests that involve topology changes should verify system.peers on all nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-8252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson updated CASSANDRA-8252: --- Assignee: Shawn Kumar > dtests that involve topology changes should verify system.peers on all nodes > > > Key: CASSANDRA-8252 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8252 > Project: Cassandra > Issue Type: Test >Reporter: Brandon Williams >Assignee: Shawn Kumar > > This is especially true for replace where I've discovered it's wrong in > 1.2.19, which is sad because now it's too late to fix. We've had a lot of > problems with incorrect/null system.peers, so after any topology change we > should verify it on every live node when everything is finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-4511) Secondary index support for CQL3 collections
[ https://issues.apache.org/jira/browse/CASSANDRA-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200624#comment-14200624 ] Kristine Hahn commented on CASSANDRA-4511: -- Thanks, [~lexlythius]. http://www.datastax.com/documentation/cql/3.1/cql/cql_reference/collection_type_r.html > Secondary index support for CQL3 collections > - > > Key: CASSANDRA-4511 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4511 > Project: Cassandra > Issue Type: Improvement >Affects Versions: 1.2.0 beta 1 >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne > Fix For: 2.1 beta1 > > Attachments: 4511.txt > > > We should allow to 2ndary index on collections. A typical use case would be > to add a 'tag set' to say a user profile and to query users based on > what tag they have. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-7217) Native transport performance (with cassandra-stress) drops precipitously past around 1000 threads
[ https://issues.apache.org/jira/browse/CASSANDRA-7217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200604#comment-14200604 ] Shawn Kumar edited comment on CASSANDRA-7217 at 11/6/14 6:32 PM: - I'll be continuing testing on a more cpu-perfomant instance but thought I would briefly try the cstar_perf on bdplab. [Here|http://cstar.datastax.com/graph?stats=dd73c4a6-65d9-11e4-9413-bc764e04482c&metric=op_rate&operation=1_write&smoothing=1&show_aggregates=true&xmin=0&xmax=279.07&ymin=0&ymax=120665.6] are the results - I increase the threads from 500 - 1500 in 250 thread increments from the first operation to the last (ie. 1_write to 5_write) and it seems like there is a noticeable drop in performance especially around 1000 threads. was (Author: shawn.kumar): I'll be continuing testing on a more cpu-perfomant instance but thought I would briefly try the cstar_perf on bdplab. [Here|http://cstar.datastax.com/graph?stats=dd73c4a6-65d9-11e4-9413-bc764e04482c&metric=op_rate&operation=1_write&smoothing=1&show_aggregates=true&xmin=0&xmax=279.07&ymin=0&ymax=120665.6] are the results - I increase the threads from 500 - 1500 in 250 thread increments from the first operation to the last and it seems like there is a noticeable drop. > Native transport performance (with cassandra-stress) drops precipitously past > around 1000 threads > - > > Key: CASSANDRA-7217 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7217 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Benedict >Assignee: Shawn Kumar > Labels: performance, triaged > Fix For: 2.1.2 > > > This is obviously bad. Let's figure out why it's happening and put a stop to > it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7217) Native transport performance (with cassandra-stress) drops precipitously past around 1000 threads
[ https://issues.apache.org/jira/browse/CASSANDRA-7217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200604#comment-14200604 ] Shawn Kumar commented on CASSANDRA-7217: I'll be continuing testing on a more cpu-perfomant instance but thought I would briefly try the cstar_perf on bdplab. [Here|http://cstar.datastax.com/graph?stats=dd73c4a6-65d9-11e4-9413-bc764e04482c&metric=op_rate&operation=1_write&smoothing=1&show_aggregates=true&xmin=0&xmax=279.07&ymin=0&ymax=120665.6] are the results - I increase the threads from 500 - 1500 in 250 thread increments from the first operation to the last and it seems like there is a noticeable drop. > Native transport performance (with cassandra-stress) drops precipitously past > around 1000 threads > - > > Key: CASSANDRA-7217 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7217 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Benedict >Assignee: Shawn Kumar > Labels: performance, triaged > Fix For: 2.1.2 > > > This is obviously bad. Let's figure out why it's happening and put a stop to > it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7083) Authentication Support for CqlRecordWriter
[
https://issues.apache.org/jira/browse/CASSANDRA-7083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200596#comment-14200596
]
Brandon Williams commented on CASSANDRA-7083:
-
Can you fix the code style problems? Brace on newline is missing in several
places, and the indentation is off everywhere.
> Authentication Support for CqlRecordWriter
> --
>
> Key: CASSANDRA-7083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7083
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
>Reporter: Henning Kropp
>Assignee: Brandon Williams
> Labels: authentication, pig
> Fix For: 2.0.12
>
> Attachments: auth_cql.patch
>
>
> The {{CqlRecordWriter}} seems not to support authentication. When the
> keyspace in Cassandra is to set to use authentication our Pig job fails with,
> when credentials are provided using the URI ('cql://username:password...):
> {code}
> java.lang.RuntimeException: InvalidRequestException(why:You have not logged
> in)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:123)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:90)
> at
> org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:76)
> at
> org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:57)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84)
> at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: InvalidRequestException(why:You have not logged in)
> at
> org.apache.cassandra.thrift.Cassandra$execute_cql3_query_result.read(Cassandra.java:38677)
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
> at
> org.apache.cassandra.thrift.Cassandra$Client.recv_execute_cql3_query(Cassandra.java:1597)
> at
> org.apache.cassandra.thrift.Cassandra$Client.execute_cql3_query(Cassandra.java:1582)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.retrievePartitionKeyValidator(CqlRecordWriter.java:332)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:108)
> ... 7 more
> {code}
> If not supplied in the URI but as only in the {{JobConf}} the exception is:
> {code}
> Output Location Validation Failed for: 'cql://...' More info to follow:
> InvalidRequestException(why:You have not logged in)
> at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$
> {code}
> Which let to the finding, that authentication is correctly supplied for
> {{CqlStorage}} but not for the {{CqlRecordWriter}}.
> Maybe it would make sense to put the authentication part into
> {{ConfigHelper.getClientFromAddressList()}}? Then in {{CqlStorage}} the
> username and password in the conf would need to be set from the URI. If so
> the {{ConfigHelper}} has all the information to authenticate and already
> returns the client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8019) Windows Unit tests and Dtests erroring due to sstable deleting task error
[
https://issues.apache.org/jira/browse/CASSANDRA-8019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200590#comment-14200590
]
Joshua McKenzie commented on CASSANDRA-8019:
Discussed this offline - I'm going to pursue formalizing the relationship
between SSTableScanners and SSTableReaders w/expectations of ordering, as we
don't want to close out SSTR w/scanners still open to them (in general
principle, though it doesn't seem to be hurting anything at this time).
Currently in the 2.1 branch, Windows' intolerance of deletion of files w/open
filehandles gives us a temporary "canary in a coal mine" on these specific
ordering issues but we shouldn't rely on that, and that goes away in 3.0 anyway.
My gut feeling is that codifying the expectation of release/deletion ordering
is best served by _aggressive_v1 but I'm loathe to add *more* users of the ref
counting mechanism on SSTR as it's been such a pain point for us.
I've also created CASSANDRA-8271 to cover SSTableDeletingTask removal in 3.0.
> Windows Unit tests and Dtests erroring due to sstable deleting task error
> -
>
> Key: CASSANDRA-8019
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8019
> Project: Cassandra
> Issue Type: Bug
> Environment: Windows 7
>Reporter: Philip Thompson
>Assignee: Joshua McKenzie
> Labels: windows
> Fix For: 2.1.2
>
> Attachments: 8019_aggressive_v1.txt, 8019_conservative_v1.txt,
> 8019_v2.txt
>
>
> Currently a large number of dtests and unit tests are erroring on windows
> with the following error in the node log:
> {code}
> ERROR [NonPeriodicTasks:1] 2014-09-29 11:05:04,383
> SSTableDeletingTask.java:89 - Unable to delete
> c:\\users\\username\\appdata\\local\\temp\\dtest-vr6qgw\\test\\node1\\data\\system\\local-7ad54392bcdd35a684174e047860b377\\system-local-ka-4-Data.db
> (it will be removed on server restart; we'll also retry after GC)\n
> {code}
> git bisect points to the following commit:
> {code}
> 0e831007760bffced8687f51b99525b650d7e193 is the first bad commit
> commit 0e831007760bffced8687f51b99525b650d7e193
> Author: Benedict Elliott Smith
> Date: Fri Sep 19 18:17:19 2014 +0100
> Fix resource leak in event of corrupt sstable
> patch by benedict; review by yukim for CASSANDRA-7932
> :100644 100644 d3ee7d99179dce03307503a8093eb47bd0161681
> f55e5d27c1c53db3485154cd16201fc5419f32df M CHANGES.txt
> :04 04 194f4c0569b6be9cc9e129c441433c5c14de7249
> 3c62b53b2b2bd4b212ab6005eab38f8a8e228923 M src
> :04 04 64f49266e328b9fdacc516c52ef1921fe42e994f
> de2ca38232bee6d2a6a5e068ed9ee0fbbc5aaebe M test
> {code}
> You can reproduce this by running simple_bootstrap_test.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8271) Remove SSTableDeletingTask
Joshua McKenzie created CASSANDRA-8271: -- Summary: Remove SSTableDeletingTask Key: CASSANDRA-8271 URL: https://issues.apache.org/jira/browse/CASSANDRA-8271 Project: Cassandra Issue Type: Improvement Reporter: Joshua McKenzie Assignee: Joshua McKenzie Priority: Minor Fix For: 3.0 With CASSANDRA-4050 and CASSANDRA-6993, our out-of-order deletion problems w/regards to Windows are resolved and the only outstanding reason to have SSTableDeletingTask would be for support of non-sun VM's w/regards to mmap'ed files. As this is no longer a big concern in the Cassandra ecosystem (non-sun VM's), we should remove SSTableDeletingTask. The one caveat is that if we want to revisit mmap'ed I/O on Windows in the future we may need to re-use this type of "delayed deletion" approach due to Windows not allowing deletion of hard linked files w/memory-mapped segments in the original file, but CASSANDRA-5863 would obviate that concern. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8226) Cannot call a column 'timestamp'
[
https://issues.apache.org/jira/browse/CASSANDRA-8226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Stepura resolved CASSANDRA-8226.
Resolution: Duplicate
Fix Version/s: (was: 2.1.3)
2.1.2
Reproduced In: 2.1.1
Will be fixed by CASSANDRA-8262
> Cannot call a column 'timestamp'
>
>
> Key: CASSANDRA-8226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8226
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: [cqlsh 5.0.1 | Cassandra 2.1.1 | CQL spec 3.2.0 | Native
> protocol v3]
>Reporter: Dave Cunningham
>Assignee: Mikhail Stepura
>Priority: Minor
> Labels: cqlsh
> Fix For: 2.1.2
>
>
> {code}
> create table test(date text, timestamp timeuuid, stuff text, PRIMARY
> KEY(date, timestamp));
> insert into test(date, timestamp, stuff) values ('20141030', now(), 'test');
> insert into test(date, timestamp, stuff) values ('20141030', now(), 'test');
> insert into test(date, timestamp, stuff) values ('20141030', now(), 'test');
> select * from test where date = '20141030' order by timestamp limit 1;
> Traceback (most recent call last):
> File "bin/cqlsh", line 861, in onecmd
> self.handle_statement(st, statementtext)
> File "bin/cqlsh", line 901, in handle_statement
> return self.handle_parse_error(cmdword, tokens, parsed, srcstr)
> File "bin/cqlsh", line 910, in handle_parse_error
> return self.perform_statement(cqlruleset.cql_extract_orig(tokens, srcstr))
> File "bin/cqlsh", line 935, in perform_statement
> result = self.perform_simple_statement(stmt)
> File "bin/cqlsh", line 968, in perform_simple_statement
> self.print_result(rows, self.parse_for_table_meta(statement.query_string))
> File "bin/cqlsh", line 946, in parse_for_table_meta
> parsed = cqlruleset.cql_parse(query_string)[1]
> IndexError: list index out of range
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7083) Authentication Support for CqlRecordWriter
[
https://issues.apache.org/jira/browse/CASSANDRA-7083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200579#comment-14200579
]
Alex Liu commented on CASSANDRA-7083:
-
+1
> Authentication Support for CqlRecordWriter
> --
>
> Key: CASSANDRA-7083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7083
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
>Reporter: Henning Kropp
>Assignee: Brandon Williams
> Labels: authentication, pig
> Fix For: 2.0.12
>
> Attachments: auth_cql.patch
>
>
> The {{CqlRecordWriter}} seems not to support authentication. When the
> keyspace in Cassandra is to set to use authentication our Pig job fails with,
> when credentials are provided using the URI ('cql://username:password...):
> {code}
> java.lang.RuntimeException: InvalidRequestException(why:You have not logged
> in)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:123)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:90)
> at
> org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:76)
> at
> org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:57)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84)
> at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: InvalidRequestException(why:You have not logged in)
> at
> org.apache.cassandra.thrift.Cassandra$execute_cql3_query_result.read(Cassandra.java:38677)
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
> at
> org.apache.cassandra.thrift.Cassandra$Client.recv_execute_cql3_query(Cassandra.java:1597)
> at
> org.apache.cassandra.thrift.Cassandra$Client.execute_cql3_query(Cassandra.java:1582)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.retrievePartitionKeyValidator(CqlRecordWriter.java:332)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:108)
> ... 7 more
> {code}
> If not supplied in the URI but as only in the {{JobConf}} the exception is:
> {code}
> Output Location Validation Failed for: 'cql://...' More info to follow:
> InvalidRequestException(why:You have not logged in)
> at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$
> {code}
> Which let to the finding, that authentication is correctly supplied for
> {{CqlStorage}} but not for the {{CqlRecordWriter}}.
> Maybe it would make sense to put the authentication part into
> {{ConfigHelper.getClientFromAddressList()}}? Then in {{CqlStorage}} the
> username and password in the conf would need to be set from the URI. If so
> the {{ConfigHelper}} has all the information to authenticate and already
> returns the client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-4511) Secondary index support for CQL3 collections
[
https://issues.apache.org/jira/browse/CASSANDRA-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200569#comment-14200569
]
Lex Lythius commented on CASSANDRA-4511:
Note that CQL for Cassandra 2.x documentation (page 48) still says:
{quote}Currently, you cannot create an index on a column of type map, set, or
list.{quote}
whereas CREATE INDEX syntax is accurate:
{code}
CREATE CUSTOM INDEX IF NOT EXISTS index_name
ON keyspace_name.table_name ( KEYS ( column_name ) )
(USING class_name) (WITH OPTIONS = map)
{code}
> Secondary index support for CQL3 collections
> -
>
> Key: CASSANDRA-4511
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4511
> Project: Cassandra
> Issue Type: Improvement
>Affects Versions: 1.2.0 beta 1
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 2.1 beta1
>
> Attachments: 4511.txt
>
>
> We should allow to 2ndary index on collections. A typical use case would be
> to add a 'tag set' to say a user profile and to query users based on
> what tag they have.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7950) Output of nodetool compactionstats and compactionhistory does not work well with long keyspace and column family names.
[ https://issues.apache.org/jira/browse/CASSANDRA-7950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Shuler updated CASSANDRA-7950: -- Labels: lhf (was: ) > Output of nodetool compactionstats and compactionhistory does not work well > with long keyspace and column family names. > - > > Key: CASSANDRA-7950 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7950 > Project: Cassandra > Issue Type: Bug > Environment: CentOS 5, 64bit, Oracle JDK 7, DSE >Reporter: Eugene >Priority: Minor > Labels: lhf > Fix For: 2.0.12 > > Attachments: nodetool-examples.txt > > > When running these commands: > nodetool compactionstats > nodetool compactionhistory > The output can be difficult to grok due to long keyspace names, column family > names, and long values. I have attached an example. > It's difficult for both humans and grep/sed/awk/perl to read. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8244) Token, DecoratedKey, RowPosition and all bound types should not make any hidden references to the database partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-8244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200557#comment-14200557 ] Joshua McKenzie commented on CASSANDRA-8244: Grabbing review as per IRC > Token, DecoratedKey, RowPosition and all bound types should not make any > hidden references to the database partitioner > -- > > Key: CASSANDRA-8244 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8244 > Project: Cassandra > Issue Type: Bug >Reporter: Branimir Lambov >Assignee: Branimir Lambov >Priority: Minor > Fix For: 3.0 > > > Currently some of the functionality of Token refers to > StorageService.getPartitioner() to avoid needing an extra argument. This is > in turn implicitly used by RowPosition and then Range, causing possible > problems, for example when ranges on secondary indices are used in a > murmur-partitioned database. > These references should be removed to force explicit choice of partitioner by > callers; alternatively, the Token interface could be changed to provide a > reference to the partitioner that created it. > (Note: the hidden reference to partitioner in serialization is a separate > issue.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8244) Token, DecoratedKey, RowPosition and all bound types should not make any hidden references to the database partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-8244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-8244: --- Reviewer: Joshua McKenzie (was: Aleksey Yeschenko) > Token, DecoratedKey, RowPosition and all bound types should not make any > hidden references to the database partitioner > -- > > Key: CASSANDRA-8244 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8244 > Project: Cassandra > Issue Type: Bug >Reporter: Branimir Lambov >Assignee: Branimir Lambov >Priority: Minor > Fix For: 3.0 > > > Currently some of the functionality of Token refers to > StorageService.getPartitioner() to avoid needing an extra argument. This is > in turn implicitly used by RowPosition and then Range, causing possible > problems, for example when ranges on secondary indices are used in a > murmur-partitioned database. > These references should be removed to force explicit choice of partitioner by > callers; alternatively, the Token interface could be changed to provide a > reference to the partitioner that created it. > (Note: the hidden reference to partitioner in serialization is a separate > issue.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7083) Authentication Support for CqlRecordWriter
[
https://issues.apache.org/jira/browse/CASSANDRA-7083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler updated CASSANDRA-7083:
--
Assignee: Brandon Williams
> Authentication Support for CqlRecordWriter
> --
>
> Key: CASSANDRA-7083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7083
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
>Reporter: Henning Kropp
>Assignee: Brandon Williams
> Labels: authentication, pig
> Fix For: 2.0.12
>
> Attachments: auth_cql.patch
>
>
> The {{CqlRecordWriter}} seems not to support authentication. When the
> keyspace in Cassandra is to set to use authentication our Pig job fails with,
> when credentials are provided using the URI ('cql://username:password...):
> {code}
> java.lang.RuntimeException: InvalidRequestException(why:You have not logged
> in)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:123)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:90)
> at
> org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:76)
> at
> org.apache.cassandra.hadoop.cql3.CqlOutputFormat.getRecordWriter(CqlOutputFormat.java:57)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84)
> at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: InvalidRequestException(why:You have not logged in)
> at
> org.apache.cassandra.thrift.Cassandra$execute_cql3_query_result.read(Cassandra.java:38677)
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
> at
> org.apache.cassandra.thrift.Cassandra$Client.recv_execute_cql3_query(Cassandra.java:1597)
> at
> org.apache.cassandra.thrift.Cassandra$Client.execute_cql3_query(Cassandra.java:1582)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.retrievePartitionKeyValidator(CqlRecordWriter.java:332)
> at
> org.apache.cassandra.hadoop.cql3.CqlRecordWriter.(CqlRecordWriter.java:108)
> ... 7 more
> {code}
> If not supplied in the URI but as only in the {{JobConf}} the exception is:
> {code}
> Output Location Validation Failed for: 'cql://...' More info to follow:
> InvalidRequestException(why:You have not logged in)
> at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$
> {code}
> Which let to the finding, that authentication is correctly supplied for
> {{CqlStorage}} but not for the {{CqlRecordWriter}}.
> Maybe it would make sense to put the authentication part into
> {{ConfigHelper.getClientFromAddressList()}}? Then in {{CqlStorage}} the
> username and password in the conf would need to be set from the URI. If so
> the {{ConfigHelper}} has all the information to authenticate and already
> returns the client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8248) Possible memory leak
[
https://issues.apache.org/jira/browse/CASSANDRA-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200544#comment-14200544
]
Alexander Sterligov commented on CASSANDRA-8248:
No, it doesn't return to expected resources after forcing GC.
Just in case:
tpstats:
{quote}
Pool NameActive Pending Completed Blocked All
time blocked
CounterMutationStage 0 0 0 0
0
ReadStage 0 06868466 0
0
RequestResponseStage 0 0 408799535 0
0
MutationStage 0 0 198570382 0
0
ReadRepairStage 0 0 46197 0
0
GossipStage 0 02150908 0
0
CacheCleanupExecutor 0 0 0 0
0
MigrationStage0 0126 0
0
ValidationExecutor0 0 72747 0
0
MemtableReclaimMemory 0 0 97987 0
0
InternalResponseStage 0 0 41583 0
0
AntiEntropyStage 0 0 630829 0
0
MiscStage 0 0 0 0
0
CommitLogArchiver 0 0 0 0
0
MemtableFlushWriter 0 0 97987 0
0
PendingRangeCalculator0 0 62 0
0
MemtablePostFlush 0 0 191444 0
0
CompactionExecutor0 04131622 0
0
AntiEntropySessions 0 0512 0
0
HintedHandoff 0 0421 0
0
Message type Dropped
RANGE_SLICE 0
READ_REPAIR 0
PAGED_RANGE 0
BINARY 0
READ 0
MUTATION 30193
_TRACE 0
REQUEST_RESPONSE 0
COUNTER_MUTATION 0
{quote}
compactionstats:
{quote}
pending tasks: 0
Active compaction remaining time :n/a
{quote}
GC was just finished:
{quote}
INFO [Service Thread] 2014-11-06 21:25:05,233 GCInspector.java:88 -
ConcurrentMarkSweep GC in 15275ms. CMS Old Gen: 3596661592 -> 3596726792; Par
Eden Space: 1371921168 -> 0;
{quote}
It still uses ~51g of RAM.
> Possible memory leak
> -
>
> Key: CASSANDRA-8248
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8248
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alexander Sterligov
>
> Sometimes during repair cassandra starts to consume more memory than expected.
> Total amount of data on node is about 20GB.
> Size of the data directory is 66GC because of snapshots.
> Top reports:
> {quote}
> PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND
> 15724 loadbase 20 0 493g 55g 44g S 28 44.2 4043:24 java
> {quote}
> At the /proc/15724/maps there are a lot of deleted file maps
> {quote}
> 7f63a6102000-7f63a6332000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6332000-7f63a6562000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6562000-7f63a6792000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6792000-7f63a69c2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a69c2000-7f63a6bf2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6bf2000-7f63a6e22000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6e22000-7f63a7052000 r--s 08:21 9442763
> /ssd/cassandr
[jira] [Commented] (CASSANDRA-8138) replace_address cannot find node to be replaced node after seed node restart
[
https://issues.apache.org/jira/browse/CASSANDRA-8138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200541#comment-14200541
]
Brandon Williams commented on CASSANDRA-8138:
-
I think I'd much rather say that the edge case of a node dying, and then a full
cluster restart (rolling would still work) is just not supported, rather than
make such invasive changes to support replacement under such strange and rare
conditions. If that happens, it's time to assassinate the node and bootstrap
another one.
> replace_address cannot find node to be replaced node after seed node restart
>
>
> Key: CASSANDRA-8138
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8138
> Project: Cassandra
> Issue Type: Bug
>Reporter: Oleg Anastasyev
>Assignee: Brandon Williams
> Attachments: ReplaceAfterSeedRestart.txt
>
>
> If a node failed and a cluster was restarted (which is common case on massive
> outages), replace_address fails with
> {code}
> Caused by: java.lang.RuntimeException: Cannot replace_address /172.19.56.97
> because it doesn't exist in gossip
> jvm 1|at
> org.apache.cassandra.service.StorageService.prepareReplacementInfo(StorageService.java:472)
> jvm 1|at
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:724)
> jvm 1|at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:686)
> jvm 1|at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:562)
> {code}
> Although neccessary information is saved in system tables on seed nodes, it
> is not loaded to gossip on seed node, so a replacement node cannot get this
> info.
> Attached patch loads all information from system tables to gossip with
> generation 0 and fixes some bugs around this info on shadow gossip round.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8156) Support indexes on composite column components of COMPACT tables
[
https://issues.apache.org/jira/browse/CASSANDRA-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler updated CASSANDRA-8156:
--
Assignee: Sylvain Lebresne
> Support indexes on composite column components of COMPACT tables
>
>
> Key: CASSANDRA-8156
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8156
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Denis Angilella
>Assignee: Sylvain Lebresne
>
> CASSANDRA-5125 added support of indexes on composite column components for
> *non-compact* tables (see CASSANDRA-5125 comments for additional information).
> This is a follow up for *compact* tables.
> Using compact tables it is possible to CREATE INDEX on composite primary key
> columns, but queries returns no results for the tests below.
> {code:sql}
> CREATE TABLE users2 (
>userID uuid,
>fname text,
>zip int,
>state text,
> PRIMARY KEY ((userID, fname))
> ) WITH COMPACT STORAGE;
> CREATE INDEX ON users2 (userID);
> CREATE INDEX ON users2 (fname);
> INSERT INTO users2 (userID, fname, zip, state) VALUES
> (b3e3bc33-b237-4b55-9337-3d41de9a5649, 'John', 10007, 'NY');
> -- the following queries returns 0 rows, instead of 1 expected
> SELECT * FROM users2 WHERE fname='John';
> SELECT * FROM users2 WHERE userid=b3e3bc33-b237-4b55-9337-3d41de9a5649;
> SELECT * FROM users2 WHERE userid=b3e3bc33-b237-4b55-9337-3d41de9a5649 AND
> fname='John';
> -- dropping 2ndary indexes restore normal behavior
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8138) replace_address cannot find node to be replaced node after seed node restart
[
https://issues.apache.org/jira/browse/CASSANDRA-8138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler updated CASSANDRA-8138:
--
Assignee: Brandon Williams
> replace_address cannot find node to be replaced node after seed node restart
>
>
> Key: CASSANDRA-8138
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8138
> Project: Cassandra
> Issue Type: Bug
>Reporter: Oleg Anastasyev
>Assignee: Brandon Williams
> Attachments: ReplaceAfterSeedRestart.txt
>
>
> If a node failed and a cluster was restarted (which is common case on massive
> outages), replace_address fails with
> {code}
> Caused by: java.lang.RuntimeException: Cannot replace_address /172.19.56.97
> because it doesn't exist in gossip
> jvm 1|at
> org.apache.cassandra.service.StorageService.prepareReplacementInfo(StorageService.java:472)
> jvm 1|at
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:724)
> jvm 1|at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:686)
> jvm 1|at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:562)
> {code}
> Although neccessary information is saved in system tables on seed nodes, it
> is not loaded to gossip on seed node, so a replacement node cannot get this
> info.
> Attached patch loads all information from system tables to gossip with
> generation 0 and fixes some bugs around this info on shadow gossip round.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8022) cqlsh hangs indefinitely within a Docker container connecting to itself with hostname
[ https://issues.apache.org/jira/browse/CASSANDRA-8022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200513#comment-14200513 ] Michael Shuler commented on CASSANDRA-8022: --- Without using hostnames, do container IP addresses work with cqlsh? If not, then it seems to be a host<->guest network routing problem. If IPs work, then it's a DNS resolution problem. I would guess that $hostname on the host and in container resolve different IP addresses? > cqlsh hangs indefinitely within a Docker container connecting to itself with > hostname > - > > Key: CASSANDRA-8022 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8022 > Project: Cassandra > Issue Type: Bug > Components: Tools > Environment: Ubuntu 14.04, running Docker, run inside a Ubuntu 14.04 > container. >Reporter: Matthew O'Riordan > Labels: cqlsh > > I am unable to use the `cqlsh` tool within a Docker container running > Cassandra. Previously I would use the Java & Thrift based `cqlsh` tool as > follows: > ``` > cqlsh --username cassandra --password whatever $(hostname) > ``` > When I run the `cqlsh` command after attaching to a running container (I use > LXC containerisation that allows attaching to a running container and running > a console), it simply hangs and never reports an error. With the `--debug` > flag on, I get the following with: > **cqlsh 4.1.1** > ``` > $ cqlsh --debug --username cassandra --password obfuscated $(hostname) > Using CQL driver: '/usr/share/cassandra/lib/cql-internal-only-1.4.1.zip/cql-1.4.1/cql/__init__.py'> > Using thrift lib: '/usr/share/cassandra/lib/thrift-python-internal-only-0.9.1.zip/thrift/__init__.py'> > ``` > It then hangs in this state indefinitely, I have no errors from `cqlsh` and > no errors in the Cassandra log. > **cqlsh 5.0.1** > ``` > $ cqlsh --debug --username cassandra --passwor obfuscated $(hostname) > Using CQL driver: '/usr/share/cassandra/lib/cassandra-driver-internal-only-2.1.0.post.zip/cassandra-driver-2.1.0.post/cassandra/__init__.py'> > ``` > It then also hangs in this state indefinitely, I have no errors from `cqlsh` > and no errors in the Cassandra log. > What's interesting, and quite confusing is that: > * I can telnet within the container as follows `telnet $(hostname) 9042` and > I get a socket. When trying to issue some commands, I see Protocol errors in > the Cassandra log thus verifying that the port is indeed open on the host > that resolves from $(hostname) > * If I `cqlsh` from another container or another host to the Cassandra > container it works just fine. > * I have tried disabling authentication altogether and using the > AllowAllAuthenticator, and I experience the same problem. > * `nodetool` works fine > In the mean time, I am forced to `cqlsh` from another container as a > workaround. Happy to try and do anything require to diagnose the cause of > this problem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8244) Token, DecoratedKey, RowPosition and all bound types should not make any hidden references to the database partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-8244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-8244: - Fix Version/s: 3.0 > Token, DecoratedKey, RowPosition and all bound types should not make any > hidden references to the database partitioner > -- > > Key: CASSANDRA-8244 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8244 > Project: Cassandra > Issue Type: Bug >Reporter: Branimir Lambov >Assignee: Branimir Lambov >Priority: Minor > Fix For: 3.0 > > > Currently some of the functionality of Token refers to > StorageService.getPartitioner() to avoid needing an extra argument. This is > in turn implicitly used by RowPosition and then Range, causing possible > problems, for example when ranges on secondary indices are used in a > murmur-partitioned database. > These references should be removed to force explicit choice of partitioner by > callers; alternatively, the Token interface could be changed to provide a > reference to the partitioner that created it. > (Note: the hidden reference to partitioner in serialization is a separate > issue.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7826) support arbitrary nesting of collection
[ https://issues.apache.org/jira/browse/CASSANDRA-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200506#comment-14200506 ] Tyler Hobbs commented on CASSANDRA-7826: bq. Where does this stand now that we've introduced frozen types? Frozen collections can be nested, and normal (multi-cell) collections can contain frozen collections. I believe [~tupshin] is more interested in having proper nested multi-cell collections, which would allow for updating individual elements in nested collections and (once we have the ability to do this in CQL) retrieving individual elements from nested collections. > support arbitrary nesting of collection > --- > > Key: CASSANDRA-7826 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7826 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Tupshin Harper >Assignee: Tyler Hobbs > Labels: ponies > > The inability to nest collections is one of the bigger data modelling > limitations we have right now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8024) No boot finished or ready message anymore upon startup completion to CLI
[
https://issues.apache.org/jira/browse/CASSANDRA-8024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler resolved CASSANDRA-8024.
---
Resolution: Not a Problem
It appears that you are starting manually with {{./bin/cassandra}} which sends
a small amount of stdout and detaches, logging normally. You can foreground
your startup, if you wish: {{./bin/cassandra -f}}
> No boot finished or ready message anymore upon startup completion to CLI
>
>
> Key: CASSANDRA-8024
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8024
> Project: Cassandra
> Issue Type: Bug
> Environment: Cassandra 2.0.10
>Reporter: Karl Mueller
>Priority: Trivial
>
> This is trivial, but cassandra logs the following to its log:
> ...
> INFO [main] 2014-09-29 23:10:35,793 CassandraDaemon.java (line 575) No
> gossip backlog; proceeding
> INFO [main] 2014-09-29 23:10:35,979 Server.java (line 156) Starting
> listening for CQL clients on
> kaos-cass00.sv.walmartlabs.com/10.93.12.10:9042...
> INFO [main] 2014-09-29 23:10:36,048 ThriftServer.java (line 99) Using
> TFramedTransport with a max frame size of 15728640 bytes.
> However, on the command line I only see:
> INFO 23:10:30,005 Compacted 4 sstables to
> [/data2/data-cassandra/system/compactions_in_progress/system-compactions_in_progress-jb-67,].
> 1,333 bytes to 962 (~72% of original) in 32ms = 0.028670MB/s. 15 total
> partitions merged to 12. Partition merge counts were {1:11, 2:2, }
> INFO 23:10:35,793 No gossip backlog; proceeding
> I would be nice if the "Starting listening for.." or some other "startup
> complete" message went to the command line STDOUT. There used to be one, I
> think, but there isn't anymore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8025) AssertionError during trigger execution
[ https://issues.apache.org/jira/browse/CASSANDRA-8025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200493#comment-14200493 ] Michael Shuler commented on CASSANDRA-8025: --- [~alice] could you please provide a reproduction case, such as a schema, your trigger, a bit of data, and a query that results in your error, so someone can follow up? Thanks! > AssertionError during trigger execution > --- > > Key: CASSANDRA-8025 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8025 > Project: Cassandra > Issue Type: Bug > Environment: Linux CentOS release 6.5 > Cassandra version 2.1.0 >Reporter: Alice > > I am trying to implement a trigger that, as a consequence of an insertion on > a table, will update a counter table. I can get all of the informations from > the ByteBuffer key, ColumnFamily update, but when I try to instanziate a new > CellNameType, using as AbstractType a CounterColumnType, I get an > "Unexpected exception AssertionError: null" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8244) Token, DecoratedKey, RowPosition and all bound types should not make any hidden references to the database partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-8244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-8244: - Reviewer: Aleksey Yeschenko > Token, DecoratedKey, RowPosition and all bound types should not make any > hidden references to the database partitioner > -- > > Key: CASSANDRA-8244 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8244 > Project: Cassandra > Issue Type: Bug >Reporter: Branimir Lambov >Assignee: Branimir Lambov >Priority: Minor > > Currently some of the functionality of Token refers to > StorageService.getPartitioner() to avoid needing an extra argument. This is > in turn implicitly used by RowPosition and then Range, causing possible > problems, for example when ranges on secondary indices are used in a > murmur-partitioned database. > These references should be removed to force explicit choice of partitioner by > callers; alternatively, the Token interface could be changed to provide a > reference to the partitioner that created it. > (Note: the hidden reference to partitioner in serialization is a separate > issue.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7953) RangeTombstones not merging during compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-7953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler updated CASSANDRA-7953:
--
Assignee: Marcus Eriksson
> RangeTombstones not merging during compaction
> -
>
> Key: CASSANDRA-7953
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7953
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.1
>Reporter: Marcus Olsson
>Assignee: Marcus Eriksson
>Priority: Minor
> Labels: compaction, deletes, tombstone
> Attachments: 0001-7953-v2.patch, CASSANDRA-7953-1.patch,
> CASSANDRA-7953.patch
>
>
> When performing a compaction on two sstables that contain the same
> RangeTombstone with different timestamps, the tombstones are not merged in
> the new sstable.
> This has been tested using cassandra 2.1 with the following table:
> {code}
> CREATE TABLE test(
> key text,
> column text,
> data text,
> PRIMARY KEY(key, column)
> );
> {code}
> And then doing the following:
> {code}
> INSERT INTO test (key, column, data) VALUES ("1", "1", "1"); // If the
> sstable only contains tombstones during compaction it seems that the sstable
> either gets removed or isn't created (but that could probably be a separate
> JIRA issue).
> INSERT INTO test (key, column, data) VALUES ("1", "2", "2"); // The inserts
> are not actually needed, since the deletes create tombstones either way.
> DELETE FROM test WHERE key="1" AND column="2";
> nodetool flush
> INSERT INTO test (key, column, data) VALUES ("1", "2", "2");
> DELETE FROM test WHERE key="1" AND column="2";
> nodetool flush
> nodetool compact
> {code}
> When checking with the SSTableExport tool two tombstones exists in the
> compacted sstable. This can be repeated, resulting in more and more
> tombstones.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7859) Extend freezing to collections
[
https://issues.apache.org/jira/browse/CASSANDRA-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200482#comment-14200482
]
Tyler Hobbs commented on CASSANDRA-7859:
bq. To reiter, collection values are sorted (even when used as cell values),
and so it felt fishy to me to consider that changing to a type that change the
internal sorting of those values is "compatible"
Right, I agree with that, and the current patch doesn't change that. I think
an example will clarify this. Take the type {{frozen>}}. If you use
that type for a clustering column, then entire lists will be compared (is [1,
2, 3] > [2, 3, 4]? etc) to sort the cells. In this case, the sorting order of
{{int}} must be maintained when you alter the type. However, if you use
{{frozen>}} for a cell value, that sorting order no longer matters.
It would be safe to alter it to {{frozen>}}.
That does *not* apply to sets, because the items within a set should be sorted.
It also does not apply to map *keys*, but map *values* can be treated like
list values.
Does that clarify things? I was a little tired when I tried to explain it last
night.
bq. Maybe we should just remove the comment in that case.
Yup, already removed it in the last commit.
What do you think about my comment on 8264 and getRequestedColumns?
> Extend freezing to collections
> --
>
> Key: CASSANDRA-7859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7859
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Tyler Hobbs
> Labels: cql
> Fix For: 2.1.2
>
> Attachments: 7859-v1.txt
>
>
> This is the follow-up to CASSANDRA-7857, to extend {{frozen}} to collections.
> This will allow things like {{map>>}} for
> instance, as well as allowing {{frozen}} collections in PK columns.
> Additionally (and that's alsmot a separate ticket but I figured we can start
> discussing it here), we could decide that tuple is a frozen type by default.
> This means that we would allow {{tuple}} without needing to add
> {{frozen}}, but we would require {{frozen}} for complex type inside tuple, so
> {{tuple>}} would be rejected, but not {{tuple frozen>>}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7458) functional indexes
[
https://issues.apache.org/jira/browse/CASSANDRA-7458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-7458:
--
Description:
Indexing information derived from the row can be powerful. For example, using
the hypothetical {{extract_date}} function,
{code}
create table ticks (
symbol text,
ticked_at datetime,
price int,
tags set,
PRIMARY KEY (symbol, ticked_at)
);
CREATE INDEX ticks_by_day ON ticks(extract_date(ticked_at));
SELECT * FROM ticks_by_day WHERE extract_date(ticked_at) = '2014-5-13';
{code}
http://www.postgresql.org/docs/9.3/static/indexes-expressional.html
was:
Indexing information derived from the row can be powerful. For example, using
the hypothetical {{extract_date}} function,
{code}
create table ticks (
symbol text,
ticked_at datetime,
price int,
tags set,
PRIMARY KEY (symbol, ticked_at)
);
CREATE INDEX ticks_by_day ON ticks(extract_date(ticked_at));
{code}
http://www.postgresql.org/docs/9.3/static/indexes-expressional.html
> functional indexes
> --
>
> Key: CASSANDRA-7458
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7458
> Project: Cassandra
> Issue Type: New Feature
> Components: API, Core
>Reporter: Jonathan Ellis
>Assignee: Mikhail Stepura
> Fix For: 3.0
>
>
> Indexing information derived from the row can be powerful. For example,
> using the hypothetical {{extract_date}} function,
> {code}
> create table ticks (
> symbol text,
> ticked_at datetime,
> price int,
> tags set,
> PRIMARY KEY (symbol, ticked_at)
> );
> CREATE INDEX ticks_by_day ON ticks(extract_date(ticked_at));
> SELECT * FROM ticks_by_day WHERE extract_date(ticked_at) = '2014-5-13';
> {code}
> http://www.postgresql.org/docs/9.3/static/indexes-expressional.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8248) Possible memory leak
[
https://issues.apache.org/jira/browse/CASSANDRA-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200472#comment-14200472
]
Michael Shuler commented on CASSANDRA-8248:
---
Repair is an expensive process. After repair has completed and the JVM has
garbage collected, what are you seeing? If things return to expected resource
use, this sounds like expected behavior.
> Possible memory leak
> -
>
> Key: CASSANDRA-8248
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8248
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alexander Sterligov
>
> Sometimes during repair cassandra starts to consume more memory than expected.
> Total amount of data on node is about 20GB.
> Size of the data directory is 66GC because of snapshots.
> Top reports:
> {quote}
> PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND
> 15724 loadbase 20 0 493g 55g 44g S 28 44.2 4043:24 java
> {quote}
> At the /proc/15724/maps there are a lot of deleted file maps
> {quote}
> 7f63a6102000-7f63a6332000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6332000-7f63a6562000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6562000-7f63a6792000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6792000-7f63a69c2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a69c2000-7f63a6bf2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6bf2000-7f63a6e22000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a6e22000-7f63a7052000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a7052000-7f63a7282000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a7282000-7f63a74b2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a74b2000-7f63a76e2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a76e2000-7f63a7912000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a7912000-7f63a7b42000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a7b42000-7f63a7d72000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a7d72000-7f63a7fa2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a7fa2000-7f63a81d2000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a81d2000-7f63a8402000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a8402000-7f63a8622000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (deleted)
> 7f63a8622000-7f63a8842000 r--s 08:21 9442763
> /ssd/cassandra/data/iss/feedback_history-d32bc7e048c011e49b989bc3e8a5a440/iss-feedback_history-tmplink-ka-328671-Index.db
> (delete
[jira] [Commented] (CASSANDRA-7408) System hints corruption - dataSize ... would be larger than file
[
https://issues.apache.org/jira/browse/CASSANDRA-7408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200467#comment-14200467
]
Michael Shuler commented on CASSANDRA-7408:
---
If you get this error on the same sstable, try a {{nodetool scrub}} to see if
that helps.
> System hints corruption - dataSize ... would be larger than file
>
>
> Key: CASSANDRA-7408
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7408
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: RHEL 6.5
> Cassandra 1.2.16
> RF=3
> Thrift
>Reporter: Jeff Griffith
>
> I've found several unresolved JIRA tickets related to SSTable corruption but
> not sure if they apply to the case we are seeing in system/hints. We see
> periodic exceptions such as:
> {noformat}
> dataSize of 144115248479299639 starting at 17209 would be larger than file
> /home/y/var/cassandra/data/system/hints/system-hints-ic-219-Data.db length
> 35542
> {noformat}
> Is there something we could possibly be doing from the application to cause
> this sort of corruption? We also see it on some of our own column families
> also some *negative* lengths which are presumably a similar corruption.
> {noformat}
> ERROR [HintedHandoff:57] 2014-06-17 17:08:04,690 CassandraDaemon.java (line
> 191) Exception in thread Thread[HintedHandoff:57,1,main]
> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> org.apache.cassandra.io.sstable.CorruptSSTableException: java.io.IOException:
> dataSize of 144115248479299639 starting at 17209 would be larger than file
> /home/y/var/cassandra/data/system/hints/system-hints-ic-219-Data.db length
> 35542
> at
> org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:441)
> at
> org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:282)
> at
> org.apache.cassandra.db.HintedHandOffManager.access$300(HintedHandOffManager.java:90)
> at
> org.apache.cassandra.db.HintedHandOffManager$4.run(HintedHandOffManager.java:508)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> org.apache.cassandra.io.sstable.CorruptSSTableException: java.io.IOException:
> dataSize of 144115248479299639 starting at 17209 would be larger than file
> /home/y/var/cassandra/data/system/hints/system-hints-ic-219-Data.db length
> 35542
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:437)
> ... 6 more
> Caused by: org.apache.cassandra.io.sstable.CorruptSSTableException:
> java.io.IOException: dataSize of 144115248479299639 starting at 17209 would
> be larger than file
> /home/y/var/cassandra/data/system/hints/system-hints-ic-219-Data.db length
> 35542
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.(SSTableIdentityIterator.java:167)
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.(SSTableIdentityIterator.java:83)
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.(SSTableIdentityIterator.java:69)
> at
> org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.next(SSTableScanner.java:180)
> at
> org.apache.cassandra.io.sstable.SSTableScanner$KeyScanningIterator.next(SSTableScanner.java:155)
> at
> org.apache.cassandra.io.sstable.SSTableScanner.next(SSTableScanner.java:142)
> at
> org.apache.cassandra.io.sstable.SSTableScanner.next(SSTableScanner.java:38)
> at
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:145)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:122)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:96)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:145)
> at
> org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(Compac
[jira] [Commented] (CASSANDRA-8257) Opscenter Agent does not properly download target cassandra cluster
[ https://issues.apache.org/jira/browse/CASSANDRA-8257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200445#comment-14200445 ] Todd Nine commented on CASSANDRA-8257: -- Already done. The opscenter tag is on the issue in this ticket. http://stackoverflow.com/questions/26722154/opscenter-wont-use-a-separate-cassandra-cluster > Opscenter Agent does not properly download target cassandra cluster > --- > > Key: CASSANDRA-8257 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8257 > Project: Cassandra > Issue Type: Bug > Components: Tools > Environment: Opscenter 5.0.1, Cassandra 1.2.19 >Reporter: Todd Nine > > Rather than re-post the issue, it is outlined here. > http://stackoverflow.com/questions/26722154/opscenter-wont-use-a-separate-cassandra-cluster. > Note that when omitting the target Cassandra cluster, using the same cluster > as the agent works correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8257) Opscenter Agent does not properly download target cassandra cluster
[ https://issues.apache.org/jira/browse/CASSANDRA-8257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200435#comment-14200435 ] Todd Nine edited comment on CASSANDRA-8257 at 11/6/14 4:53 PM: --- [~brandon.williams] I would love to, where is it? I'm assuming it's here. https://datastax.jira.com But I can't actually get in to create an issue. was (Author: tnine): [~brandon.williams] I would love to, where is it? > Opscenter Agent does not properly download target cassandra cluster > --- > > Key: CASSANDRA-8257 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8257 > Project: Cassandra > Issue Type: Bug > Components: Tools > Environment: Opscenter 5.0.1, Cassandra 1.2.19 >Reporter: Todd Nine > > Rather than re-post the issue, it is outlined here. > http://stackoverflow.com/questions/26722154/opscenter-wont-use-a-separate-cassandra-cluster. > Note that when omitting the target Cassandra cluster, using the same cluster > as the agent works correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8257) Opscenter Agent does not properly download target cassandra cluster
[ https://issues.apache.org/jira/browse/CASSANDRA-8257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200440#comment-14200440 ] Brandon Williams commented on CASSANDRA-8257: - I am told you should use the Feedback button in the UI, or post on SO with an opscenter tag. > Opscenter Agent does not properly download target cassandra cluster > --- > > Key: CASSANDRA-8257 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8257 > Project: Cassandra > Issue Type: Bug > Components: Tools > Environment: Opscenter 5.0.1, Cassandra 1.2.19 >Reporter: Todd Nine > > Rather than re-post the issue, it is outlined here. > http://stackoverflow.com/questions/26722154/opscenter-wont-use-a-separate-cassandra-cluster. > Note that when omitting the target Cassandra cluster, using the same cluster > as the agent works correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-5493) Confusing output of CommandDroppedTasks
[
https://issues.apache.org/jira/browse/CASSANDRA-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler resolved CASSANDRA-5493.
---
Resolution: Incomplete
Reproduced In: 1.2.11, 1.2.9 (was: 1.2.9, 1.2.11)
[~ondrej.cernos] please, reopen this ticket if further help is needed.
> Confusing output of CommandDroppedTasks
> ---
>
> Key: CASSANDRA-5493
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5493
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 1.2.3
>Reporter: Ondřej Černoš
>Priority: Minor
>
> We have 2 DCs, 3 nodes in each, using EC2 support. We are debugging nodetool
> repair problems (roughly 1 out of 2 attempts just freezes). We looked into
> the MessagingServiceBean to see what is going on using jmxterm. See the
> following:
> {noformat}
> #mbean = org.apache.cassandra.net:type=MessagingService:
> CommandDroppedTasks = {
> 107.aaa.bbb.ccc = 0;
> 166.ddd.eee.fff = 124320;
> 10.ggg.hhh.iii = 0;
> 107.jjj.kkk.lll = 0;
> 166.mmm.nnn.ooo = 1336699;
> 166.ppp.qqq.rrr = 1329171;
> 10.sss.ttt.uuu = 0;
> 107.vvv.www.xxx = 0;
> };
> {noformat}
> The problem with this output is it has 8 records. The node's neighbours (the
> 107 and 10 nodes) are mentioned twice in the output, once with their public
> IPs and once with their private IPs. The nodes in remote DC (the 166 ones)
> are reported only once. I am pretty sure this is a bug - the node should be
> reported only with one of its addresses in all outputs from Cassandra and it
> should be consistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8257) Opscenter Agent does not properly download target cassandra cluster
[ https://issues.apache.org/jira/browse/CASSANDRA-8257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200435#comment-14200435 ] Todd Nine commented on CASSANDRA-8257: -- [~brandon.williams] I would love to, where is it? > Opscenter Agent does not properly download target cassandra cluster > --- > > Key: CASSANDRA-8257 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8257 > Project: Cassandra > Issue Type: Bug > Components: Tools > Environment: Opscenter 5.0.1, Cassandra 1.2.19 >Reporter: Todd Nine > > Rather than re-post the issue, it is outlined here. > http://stackoverflow.com/questions/26722154/opscenter-wont-use-a-separate-cassandra-cluster. > Note that when omitting the target Cassandra cluster, using the same cluster > as the agent works correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8270) Allow sub slices for composites
[
https://issues.apache.org/jira/browse/CASSANDRA-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremiah Jordan updated CASSANDRA-8270:
---
Description:
For some queries with clustering keys it would be helpful to allow server side
slicing to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c > 3; --return 1
select count(*) from foo where a = 'key' and b > 2 and c > 3; --error
select count(*) from foo where a = 'key' and c > 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include sub-slices we could effectively request
for:
{noformat}
b(2,) c(3,)
b(,) c(3,)
{noformat}
was:
For some queries with clustering keys it would be helpful to allow server side
slicing to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c > 3; --return 1
select count(*) from foo where a = 'key' and b > 2 and c > 3; --error
select count(*) from foo where a = 'key' and c > 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,) c(3,)
b(,) c(3,)
{noformat}
> Allow sub slices for composites
> ---
>
> Key: CASSANDRA-8270
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8270
> Project: Cassandra
> Issue Type: Improvement
>Reporter: T Jake Luciani
>Priority: Minor
>
> For some queries with clustering keys it would be helpful to allow server
> side slicing to avoid processing things you will simply filter out.
> Example schema:
> {code}
> create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
> insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
> insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
> insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
> insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
> {code}
> {code}
>select count(*) from foo where a = 'key' and b = 2 and c > 3; --return 1
>select count(*) from foo where a = 'key' and b > 2 and c > 3; --error
>select count(*) from foo where a = 'key' and c > 3; --error
> {code}
> The first query is only possible because our slices only allow a fixed prefix
> but if we extended slices to include sub-slices we could effectively request
> for:
> {noformat}
> b(2,) c(3,)
> b(,) c(3,)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7859) Extend freezing to collections
[
https://issues.apache.org/jira/browse/CASSANDRA-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200421#comment-14200421
]
Sylvain Lebresne commented on CASSANDRA-7859:
-
bq. When frozen lists and maps are used as cell values, the collection value
ordering doesn't matter.
I'm confused because my earlier comment was exactly to say that I'm
uncomfortable making that assumption and you seemed to agree initially. To
reiter, collection values are sorted (even when used as cell values), and so it
felt fishy to me to consider that changing to a type that change the internal
sorting of those values is "compatible". Typically sending a value to clients
that is not properly sorted feels wrong. And given we lost little in practice
by requiring the sorting not to change, I'd prefer sticking to that for now.
bq. I think it could potentially introduce assertion errors when the behavior
is reasonable.
Fair enough, it's just that the comments in there pretty much says that it
should only be call for frozen collections. Maybe we should just remove the
comment in that case. Not a big deal though.
> Extend freezing to collections
> --
>
> Key: CASSANDRA-7859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7859
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Tyler Hobbs
> Labels: cql
> Fix For: 2.1.2
>
> Attachments: 7859-v1.txt
>
>
> This is the follow-up to CASSANDRA-7857, to extend {{frozen}} to collections.
> This will allow things like {{map>>}} for
> instance, as well as allowing {{frozen}} collections in PK columns.
> Additionally (and that's alsmot a separate ticket but I figured we can start
> discussing it here), we could decide that tuple is a frozen type by default.
> This means that we would allow {{tuple}} without needing to add
> {{frozen}}, but we would require {{frozen}} for complex type inside tuple, so
> {{tuple>}} would be rejected, but not {{tuple frozen>>}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8270) Allow sub slices for composites
[
https://issues.apache.org/jira/browse/CASSANDRA-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremiah Jordan updated CASSANDRA-8270:
---
Description:
For some queries with clustering keys it would be helpful to allow server side
slicing to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c > 3; --return 1
select count(*) from foo where a = 'key' and b > 2 and c > 3; --error
select count(*) from foo where a = 'key' and c > 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,) c(3,)
b(,) c(3,)
{noformat}
was:
For some queries with clustering keys it would be helpful to allow server side
to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c > 3; --return 1
select count(*) from foo where a = 'key' and b > 2 and c > 3; --error
select count(*) from foo where a = 'key' and c > 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,) c(3,)
b(,) c(3,)
{noformat}
> Allow sub slices for composites
> ---
>
> Key: CASSANDRA-8270
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8270
> Project: Cassandra
> Issue Type: Improvement
>Reporter: T Jake Luciani
>Priority: Minor
>
> For some queries with clustering keys it would be helpful to allow server
> side slicing to avoid processing things you will simply filter out.
> Example schema:
> {code}
> create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
> insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
> insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
> insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
> insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
> {code}
> {code}
>select count(*) from foo where a = 'key' and b = 2 and c > 3; --return 1
>select count(*) from foo where a = 'key' and b > 2 and c > 3; --error
>select count(*) from foo where a = 'key' and c > 3; --error
> {code}
> The first query is only possible because our slices only allow a fixed prefix
> but if we extended slices to include slices we could effectively request for:
> {noformat}
> b(2,) c(3,)
> b(,) c(3,)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8270) Allow sub slices for composites
[
https://issues.apache.org/jira/browse/CASSANDRA-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
T Jake Luciani updated CASSANDRA-8270:
--
Description:
For some queries with clustering keys it would be helpful to allow server side
to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c > 3; --return 1
select count(*) from foo where a = 'key' and b > 2 and c > 3; --error
select count(*) from foo where a = 'key' and c > 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,) c(3,)
b(,) c(3,)
{noformat}
was:
For some queries with clustering keys it would be helpful to allow server side
to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c > 3; --return 1
select count(*) from foo where a = 'key' and b > 2 and c > 3; --error
select count(*) from foo where a = 'key' and c > 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,*) c(3,*)
b(*,*) c(3,*)
{noformat}
> Allow sub slices for composites
> ---
>
> Key: CASSANDRA-8270
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8270
> Project: Cassandra
> Issue Type: Improvement
>Reporter: T Jake Luciani
>Priority: Minor
>
> For some queries with clustering keys it would be helpful to allow server
> side to avoid processing things you will simply filter out.
> Example schema:
> {code}
> create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
> insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
> insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
> insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
> insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
> {code}
> {code}
>select count(*) from foo where a = 'key' and b = 2 and c > 3; --return 1
>select count(*) from foo where a = 'key' and b > 2 and c > 3; --error
>select count(*) from foo where a = 'key' and c > 3; --error
> {code}
> The first query is only possible because our slices only allow a fixed prefix
> but if we extended slices to include slices we could effectively request for:
> {noformat}
> b(2,) c(3,)
> b(,) c(3,)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8270) Allow sub slices for composites
[
https://issues.apache.org/jira/browse/CASSANDRA-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
T Jake Luciani updated CASSANDRA-8270:
--
Description:
For some queries with clustering keys it would be helpful to allow server side
to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c > 3; --return 1
select count(*) from foo where a = 'key' and b > 2 and c > 3; --error
select count(*) from foo where a = 'key' and c > 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,*) c(3,*)
b(*,*) c(3,*)
{noformat}
was:
For some queries with clustering keys it would be helpful to allow server side
to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c > 3; --return 1
select count(*) from foo where a = 'key' and b > 2 and c > 3; --error
select count(*) from foo where a = 'key' and c > 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,*) c (3,*)
b (*,*) c (3,*)
{noformat}
> Allow sub slices for composites
> ---
>
> Key: CASSANDRA-8270
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8270
> Project: Cassandra
> Issue Type: Improvement
>Reporter: T Jake Luciani
>Priority: Minor
>
> For some queries with clustering keys it would be helpful to allow server
> side to avoid processing things you will simply filter out.
> Example schema:
> {code}
> create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
> insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
> insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
> insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
> insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
> {code}
> {code}
>select count(*) from foo where a = 'key' and b = 2 and c > 3; --return 1
>select count(*) from foo where a = 'key' and b > 2 and c > 3; --error
>select count(*) from foo where a = 'key' and c > 3; --error
> {code}
> The first query is only possible because our slices only allow a fixed prefix
> but if we extended slices to include slices we could effectively request for:
> {noformat}
> b(2,*) c(3,*)
> b(*,*) c(3,*)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8270) Allow sub slices for composites
[
https://issues.apache.org/jira/browse/CASSANDRA-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
T Jake Luciani updated CASSANDRA-8270:
--
Description:
For some queries with clustering keys it would be helpful to allow server side
to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c > 3; --return 1
select count(*) from foo where a = 'key' and b > 2 and c > 3; --error
select count(*) from foo where a = 'key' and c > 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
{noformat}
b(2,*) c (3,*)
b (*,*) c (3,*)
{noformat}
was:
For some queries with clustering keys it would be helpful to allow server side
to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c > 3; --return 1
select count(*) from foo where a = 'key' and b > 2 and c > 3; --error
select count(*) from foo where a = 'key' and c > 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
b(2,*) c (3,*)
b (*,*) c (3,*)
> Allow sub slices for composites
> ---
>
> Key: CASSANDRA-8270
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8270
> Project: Cassandra
> Issue Type: Improvement
>Reporter: T Jake Luciani
>Priority: Minor
>
> For some queries with clustering keys it would be helpful to allow server
> side to avoid processing things you will simply filter out.
> Example schema:
> {code}
> create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
> insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
> insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
> insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
> insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
> {code}
> {code}
>select count(*) from foo where a = 'key' and b = 2 and c > 3; --return 1
>select count(*) from foo where a = 'key' and b > 2 and c > 3; --error
>select count(*) from foo where a = 'key' and c > 3; --error
> {code}
> The first query is only possible because our slices only allow a fixed prefix
> but if we extended slices to include slices we could effectively request for:
> {noformat}
> b(2,*) c (3,*)
> b (*,*) c (3,*)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8028) Unable to compute when histogram overflowed
[ https://issues.apache.org/jira/browse/CASSANDRA-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-8028: --- Fix Version/s: (was: 2.1.2) 2.1.3 > Unable to compute when histogram overflowed > --- > > Key: CASSANDRA-8028 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8028 > Project: Cassandra > Issue Type: Bug > Components: Tools > Environment: Linux >Reporter: Gianluca Borello >Assignee: Carl Yeksigian > Fix For: 2.1.3 > > Attachments: 8028-2.1-clean.txt, 8028-2.1-v2.txt, 8028-2.1.txt, > 8028-trunk.txt, sstable-histogrambuster.tar.bz2 > > > It seems like with 2.1.0 histograms can't be computed most of the times: > $ nodetool cfhistograms draios top_files_by_agent1 > nodetool: Unable to compute when histogram overflowed > See 'nodetool help' or 'nodetool help '. > I can probably find a way to attach a .cql script to reproduce it, but I > suspect it must be obvious to replicate it as it happens on more than 50% of > my column families. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[1/2] git commit: Fix overflow on histogram computation
Repository: cassandra
Updated Branches:
refs/heads/trunk d5b2fa206 -> b4cbb28f3
Fix overflow on histogram computation
Patch by cyeksigian; reviewed by jmckenzie for CASSANDRA-8028
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9685622b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9685622b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9685622b
Branch: refs/heads/trunk
Commit: 9685622b32d19ffc9315e764a224f41cbe6b6d4a
Parents: 85ea373
Author: Carl Yeksigian
Authored: Thu Nov 6 10:18:09 2014 -0600
Committer: Joshua McKenzie
Committed: Thu Nov 6 10:18:09 2014 -0600
--
CHANGES.txt | 1 +
.../cassandra/metrics/ColumnFamilyMetrics.java | 67
.../org/apache/cassandra/tools/NodeTool.java| 32 --
3 files changed, 80 insertions(+), 20 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9685622b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5348f2f..89db48e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.3
+ * Fix overflow on histogram computation (CASSANDRA-8028)
* Have paxos reuse the timestamp generation of normal queries (CASSANDRA-7801)
2.1.2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9685622b/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
--
diff --git a/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
b/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
index d9d3ed9..2bdaf27 100644
--- a/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
+++ b/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
@@ -18,6 +18,7 @@
package org.apache.cassandra.metrics;
import java.util.HashSet;
+import java.util.Iterator;
import java.util.Set;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.TimeUnit;
@@ -146,6 +147,46 @@ public class ColumnFamilyMetrics
* Stores all metric names created that can be used when unregistering
*/
public final static Set all = Sets.newHashSet();
+
+private interface GetHistogram
+{
+public EstimatedHistogram getHistogram(SSTableReader reader);
+}
+
+private static long[] combineHistograms(Iterable sstables,
GetHistogram getHistogram)
+{
+Iterator iterator = sstables.iterator();
+if (!iterator.hasNext())
+{
+return new long[0];
+}
+long[] firstBucket =
getHistogram.getHistogram(iterator.next()).getBuckets(false);
+long[] values = new long[firstBucket.length];
+System.arraycopy(firstBucket, 0, values, 0, values.length);
+
+while (iterator.hasNext())
+{
+long[] nextBucket =
getHistogram.getHistogram(iterator.next()).getBuckets(false);
+if (nextBucket.length > values.length)
+{
+long[] newValues = new long[nextBucket.length];
+System.arraycopy(firstBucket, 0, newValues, 0,
firstBucket.length);
+for (int i = 0; i < newValues.length; i++)
+{
+newValues[i] += nextBucket[i];
+}
+values = newValues;
+}
+else
+{
+for (int i = 0; i < values.length; i++)
+{
+values[i] += nextBucket[i];
+}
+}
+}
+return values;
+}
/**
* Creates metrics for given {@link ColumnFamilyStore}.
@@ -219,28 +260,26 @@ public class ColumnFamilyMetrics
{
public long[] value()
{
-long[] histogram = new long[90];
-for (SSTableReader sstable : cfs.getSSTables())
+return combineHistograms(cfs.getSSTables(), new GetHistogram()
{
-long[] rowSize =
sstable.getEstimatedRowSize().getBuckets(false);
-for (int i = 0; i < histogram.length; i++)
-histogram[i] += rowSize[i];
-}
-return histogram;
+public EstimatedHistogram getHistogram(SSTableReader
reader)
+{
+return reader.getEstimatedRowSize();
+}
+});
}
});
estimatedColumnCountHistogram =
Metrics.newGauge(factory.createMetricName("EstimatedColumnCountHistogram"), new
Gauge()
{
public long[] value()
{
-long[] histogram = new long[90];
-for (SSTableReader sstable : cfs.
[2/2] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b4cbb28f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b4cbb28f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b4cbb28f Branch: refs/heads/trunk Commit: b4cbb28f3759fc92ab2a43432fcbce33bac60d7a Parents: d5b2fa2 9685622 Author: Joshua McKenzie Authored: Thu Nov 6 10:19:03 2014 -0600 Committer: Joshua McKenzie Committed: Thu Nov 6 10:19:03 2014 -0600 -- CHANGES.txt | 1 + .../cassandra/metrics/ColumnFamilyMetrics.java | 67 .../org/apache/cassandra/tools/NodeTool.java| 32 -- 3 files changed, 80 insertions(+), 20 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b4cbb28f/CHANGES.txt -- diff --cc CHANGES.txt index 79318fc,89db48e..92b2b2c --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,36 -1,5 +1,37 @@@ +3.0 + * Mark sstables as repaired after full repair (CASSANDRA-7586) + * Extend Descriptor to include a format value and refactor reader/writer apis (CASSANDRA-7443) + * Integrate JMH for microbenchmarks (CASSANDRA-8151) + * Keep sstable levels when bootstrapping (CASSANDRA-7460) + * Add Sigar library and perform basic OS settings check on startup (CASSANDRA-7838) + * Support for aggregation functions (CASSANDRA-4914) + * Remove cassandra-cli (CASSANDRA-7920) + * Accept dollar quoted strings in CQL (CASSANDRA-7769) + * Make assassinate a first class command (CASSANDRA-7935) + * Support IN clause on any clustering column (CASSANDRA-4762) + * Improve compaction logging (CASSANDRA-7818) + * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917) + * Do anticompaction in groups (CASSANDRA-6851) + * Support pure user-defined functions (CASSANDRA-7395, 7526, 7562, 7740, 7781, 7929, + 7924, 7812, 8063) + * Permit configurable timestamps with cassandra-stress (CASSANDRA-7416) + * Move sstable RandomAccessReader to nio2, which allows using the + FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050) + * Remove CQL2 (CASSANDRA-5918) + * Add Thrift get_multi_slice call (CASSANDRA-6757) + * Optimize fetching multiple cells by name (CASSANDRA-6933) + * Allow compilation in java 8 (CASSANDRA-7028) + * Make incremental repair default (CASSANDRA-7250) + * Enable code coverage thru JaCoCo (CASSANDRA-7226) + * Switch external naming of 'column families' to 'tables' (CASSANDRA-4369) + * Shorten SSTable path (CASSANDRA-6962) + * Use unsafe mutations for most unit tests (CASSANDRA-6969) + * Fix race condition during calculation of pending ranges (CASSANDRA-7390) + * Fail on very large batch sizes (CASSANDRA-8011) + * improve concurrency of repair (CASSANDRA-6455) + 2.1.3 + * Fix overflow on histogram computation (CASSANDRA-8028) * Have paxos reuse the timestamp generation of normal queries (CASSANDRA-7801) 2.1.2 http://git-wip-us.apache.org/repos/asf/cassandra/blob/b4cbb28f/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b4cbb28f/src/java/org/apache/cassandra/tools/NodeTool.java --
git commit: Fix overflow on histogram computation
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 85ea37356 -> 9685622b3
Fix overflow on histogram computation
Patch by cyeksigian; reviewed by jmckenzie for CASSANDRA-8028
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9685622b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9685622b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9685622b
Branch: refs/heads/cassandra-2.1
Commit: 9685622b32d19ffc9315e764a224f41cbe6b6d4a
Parents: 85ea373
Author: Carl Yeksigian
Authored: Thu Nov 6 10:18:09 2014 -0600
Committer: Joshua McKenzie
Committed: Thu Nov 6 10:18:09 2014 -0600
--
CHANGES.txt | 1 +
.../cassandra/metrics/ColumnFamilyMetrics.java | 67
.../org/apache/cassandra/tools/NodeTool.java| 32 --
3 files changed, 80 insertions(+), 20 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9685622b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5348f2f..89db48e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.3
+ * Fix overflow on histogram computation (CASSANDRA-8028)
* Have paxos reuse the timestamp generation of normal queries (CASSANDRA-7801)
2.1.2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9685622b/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
--
diff --git a/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
b/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
index d9d3ed9..2bdaf27 100644
--- a/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
+++ b/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java
@@ -18,6 +18,7 @@
package org.apache.cassandra.metrics;
import java.util.HashSet;
+import java.util.Iterator;
import java.util.Set;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.TimeUnit;
@@ -146,6 +147,46 @@ public class ColumnFamilyMetrics
* Stores all metric names created that can be used when unregistering
*/
public final static Set all = Sets.newHashSet();
+
+private interface GetHistogram
+{
+public EstimatedHistogram getHistogram(SSTableReader reader);
+}
+
+private static long[] combineHistograms(Iterable sstables,
GetHistogram getHistogram)
+{
+Iterator iterator = sstables.iterator();
+if (!iterator.hasNext())
+{
+return new long[0];
+}
+long[] firstBucket =
getHistogram.getHistogram(iterator.next()).getBuckets(false);
+long[] values = new long[firstBucket.length];
+System.arraycopy(firstBucket, 0, values, 0, values.length);
+
+while (iterator.hasNext())
+{
+long[] nextBucket =
getHistogram.getHistogram(iterator.next()).getBuckets(false);
+if (nextBucket.length > values.length)
+{
+long[] newValues = new long[nextBucket.length];
+System.arraycopy(firstBucket, 0, newValues, 0,
firstBucket.length);
+for (int i = 0; i < newValues.length; i++)
+{
+newValues[i] += nextBucket[i];
+}
+values = newValues;
+}
+else
+{
+for (int i = 0; i < values.length; i++)
+{
+values[i] += nextBucket[i];
+}
+}
+}
+return values;
+}
/**
* Creates metrics for given {@link ColumnFamilyStore}.
@@ -219,28 +260,26 @@ public class ColumnFamilyMetrics
{
public long[] value()
{
-long[] histogram = new long[90];
-for (SSTableReader sstable : cfs.getSSTables())
+return combineHistograms(cfs.getSSTables(), new GetHistogram()
{
-long[] rowSize =
sstable.getEstimatedRowSize().getBuckets(false);
-for (int i = 0; i < histogram.length; i++)
-histogram[i] += rowSize[i];
-}
-return histogram;
+public EstimatedHistogram getHistogram(SSTableReader
reader)
+{
+return reader.getEstimatedRowSize();
+}
+});
}
});
estimatedColumnCountHistogram =
Metrics.newGauge(factory.createMetricName("EstimatedColumnCountHistogram"), new
Gauge()
{
public long[] value()
{
-long[] histogram = new long[90];
-for (SSTableReade
[jira] [Created] (CASSANDRA-8270) Allow sub slices for composites
T Jake Luciani created CASSANDRA-8270:
-
Summary: Allow sub slices for composites
Key: CASSANDRA-8270
URL: https://issues.apache.org/jira/browse/CASSANDRA-8270
Project: Cassandra
Issue Type: Improvement
Reporter: T Jake Luciani
Priority: Minor
For some queries with clustering keys it would be helpful to allow server side
to avoid processing things you will simply filter out.
Example schema:
{code}
create TABLE foo (a text, b int, c int, d int, primary key(a, b, c));
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 2, 4, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 3, 4);
insert into foo (a, b, c, d ) VALUES ( 'key', 3, 4, 4);
{code}
{code}
select count(*) from foo where a = 'key' and b = 2 and c > 3; --return 1
select count(*) from foo where a = 'key' and b > 2 and c > 3; --error
select count(*) from foo where a = 'key' and c > 3; --error
{code}
The first query is only possible because our slices only allow a fixed prefix
but if we extended slices to include slices we could effectively request for:
b(2,*) c (3,*)
b (*,*) c (3,*)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8244) Token, DecoratedKey, RowPosition and all bound types should not make any hidden references to the database partitioner
[
https://issues.apache.org/jira/browse/CASSANDRA-8244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200374#comment-14200374
]
Branimir Lambov commented on CASSANDRA-8244:
Proposed patch is uploaded for review on
https://github.com/blambov/cassandra/compare/8244-partitioner-in-token. Applies
on top of CASSANDRA-8230 on trunk.
Changes tokens and {{RingPosition}} to provide a reference to the partitioner
that created them, which is used to make sure Range/Bounds always use the
correct partitioner for the bounds they contain. Previously this was quite easy
to get wrong without noticing, see e.g. {{DataRange.allData}}. To avoid
increasing the space used by a token, this is done by moving the token type
definitions inside the partitioner, which allows them to have a static
reference to it, except {{LocalToken}} which now replaces the reference to its
comparator with a reference to the issuing partitioner.
Removes custom key construction in {{SecondaryIndex.getIndexKeyFor}} and
delegates the task to the {{indexCfs}} partitioner, the presence of which is
already tested whenever {{getIndexKeyFor}} is used.
Adds static instances of the partitioners and a mechanism to choose them
instead of instantiating a new copy in {{FBUtilities.newPartitioner}}.
Fixes {{KeyCollisionTest}} which wasn't changing the partitioner as expected,
and was also passing with the wrong partitioner.
> Token, DecoratedKey, RowPosition and all bound types should not make any
> hidden references to the database partitioner
> --
>
> Key: CASSANDRA-8244
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8244
> Project: Cassandra
> Issue Type: Bug
>Reporter: Branimir Lambov
>Assignee: Branimir Lambov
>Priority: Minor
>
> Currently some of the functionality of Token refers to
> StorageService.getPartitioner() to avoid needing an extra argument. This is
> in turn implicitly used by RowPosition and then Range, causing possible
> problems, for example when ranges on secondary indices are used in a
> murmur-partitioned database.
> These references should be removed to force explicit choice of partitioner by
> callers; alternatively, the Token interface could be changed to provide a
> reference to the partitioner that created it.
> (Note: the hidden reference to partitioner in serialization is a separate
> issue.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8028) Unable to compute when histogram overflowed
[ https://issues.apache.org/jira/browse/CASSANDRA-8028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Yeksigian updated CASSANDRA-8028: -- Attachment: 8028-2.1-v2.txt The intent was to allow one to be set without the other, just didn't do it correctly. Thanks for catching; I've addressed it in the latest patch. > Unable to compute when histogram overflowed > --- > > Key: CASSANDRA-8028 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8028 > Project: Cassandra > Issue Type: Bug > Components: Tools > Environment: Linux >Reporter: Gianluca Borello >Assignee: Carl Yeksigian > Fix For: 2.1.2 > > Attachments: 8028-2.1-clean.txt, 8028-2.1-v2.txt, 8028-2.1.txt, > 8028-trunk.txt, sstable-histogrambuster.tar.bz2 > > > It seems like with 2.1.0 histograms can't be computed most of the times: > $ nodetool cfhistograms draios top_files_by_agent1 > nodetool: Unable to compute when histogram overflowed > See 'nodetool help' or 'nodetool help '. > I can probably find a way to attach a .cql script to reproduce it, but I > suspect it must be obvious to replicate it as it happens on more than 50% of > my column families. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[1/2] git commit: Make paxos reuse the timestamp generation of normal operation
Repository: cassandra
Updated Branches:
refs/heads/trunk aa6233aa1 -> d5b2fa206
Make paxos reuse the timestamp generation of normal operation
patch by slebresne; reviewed by iamaleksey for CASSANDRA-7801
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/85ea3735
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/85ea3735
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/85ea3735
Branch: refs/heads/trunk
Commit: 85ea37356e666c2780294bbd29daa89a32ebf333
Parents: cdf80d9
Author: Sylvain Lebresne
Authored: Thu Nov 6 16:38:27 2014 +0100
Committer: Sylvain Lebresne
Committed: Thu Nov 6 16:38:27 2014 +0100
--
CHANGES.txt | 3 +
.../cql3/statements/BatchStatement.java | 7 +-
.../cql3/statements/ModificationStatement.java | 3 +-
.../cql3/statements/SelectStatement.java| 8 +-
.../apache/cassandra/service/ClientState.java | 36 +
.../apache/cassandra/service/QueryState.java| 5 +-
.../apache/cassandra/service/StorageProxy.java | 139 ++-
.../service/pager/MultiPartitionPager.java | 13 +-
.../service/pager/NamesQueryPager.java | 7 +-
.../cassandra/service/pager/QueryPagers.java| 30 ++--
.../service/pager/SliceQueryPager.java | 11 +-
.../cassandra/thrift/CassandraServer.java | 35 +++--
12 files changed, 206 insertions(+), 91 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/85ea3735/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2e60f3a..5348f2f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,6 @@
+2.1.3
+ * Have paxos reuse the timestamp generation of normal queries (CASSANDRA-7801)
+
2.1.2
* (cqlsh) parse_for_table_meta errors out on queries with undefined
grammars (CASSANDRA-8262)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/85ea3735/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
b/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
index 17d1771..d54e4fd 100644
--- a/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
@@ -275,7 +275,7 @@ public class BatchStatement implements CQLStatement,
MeasurableForPreparedCache
throw new InvalidRequestException("Invalid empty serial
consistency level");
if (hasConditions)
-return executeWithConditions(options, now);
+return executeWithConditions(options, queryState);
executeWithoutConditions(getMutations(options, local, now),
options.getConsistency());
return new ResultMessage.Void();
@@ -297,9 +297,10 @@ public class BatchStatement implements CQLStatement,
MeasurableForPreparedCache
StorageProxy.mutateWithTriggers(mutations, cl, mutateAtomic);
}
-private ResultMessage executeWithConditions(BatchQueryOptions options,
long now)
+private ResultMessage executeWithConditions(BatchQueryOptions options,
QueryState state)
throws RequestExecutionException, RequestValidationException
{
+long now = state.getTimestamp();
ByteBuffer key = null;
String ksName = null;
String cfName = null;
@@ -339,7 +340,7 @@ public class BatchStatement implements CQLStatement,
MeasurableForPreparedCache
casRequest.addRowUpdate(clusteringPrefix, statement,
statementOptions, timestamp);
}
-ColumnFamily result = StorageProxy.cas(ksName, cfName, key,
casRequest, options.getSerialConsistency(), options.getConsistency());
+ColumnFamily result = StorageProxy.cas(ksName, cfName, key,
casRequest, options.getSerialConsistency(), options.getConsistency(),
state.getClientState());
return new
ResultMessage.Rows(ModificationStatement.buildCasResultSet(ksName, key, cfName,
result, columnsWithConditions, true, options.forStatement(0)));
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/85ea3735/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
--
diff --git
a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
index 974ccc8..846ad3e 100644
--- a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
@@ -524,7 +524,8 @@ public abstract class ModificationStatement impleme
