[jira] [Updated] (CASSANDRA-7973) cqlsh connect error member_descriptor' object is not callable
[
https://issues.apache.org/jira/browse/CASSANDRA-7973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler updated CASSANDRA-7973:
--
Labels: cqlsh lhf (was: cqlsh)
cqlsh connect error member_descriptor' object is not callable
---
Key: CASSANDRA-7973
URL: https://issues.apache.org/jira/browse/CASSANDRA-7973
Project: Cassandra
Issue Type: Bug
Environment: Cassandra 2.1.0
Reporter: Digant Modha
Priority: Minor
Labels: cqlsh, lhf
When using cqlsh (Cassandra 2.1.0) with ssl, python 2.6.9. I get Connection
error: ('Unable to connect to any servers', {...:
TypeError('member_descriptor' object is not callable,)})
I am able to connect from another machine using python 2.7.5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8274) Node fails to rejoin cluster on EC2 if private IP is changed
Joseph Clark created CASSANDRA-8274: --- Summary: Node fails to rejoin cluster on EC2 if private IP is changed Key: CASSANDRA-8274 URL: https://issues.apache.org/jira/browse/CASSANDRA-8274 Project: Cassandra Issue Type: Bug Components: Core Environment: Amazon EC2 Reporter: Joseph Clark Nodes in Amazon AWS EC2 Classic (not a VPC) may be assigned a new private IP if the node is stopped and then started again. In this case we have puppet update the configured listen_address to the new private IP. However, once the cassandra service starts, it is unable to communicate with the existing nodes(single region) and vice versa. 'nodetool status' shows that each node believes that it is 'UN' and the other node is 'DN'. 'nodetool gossipinfo' on the node that remained running shows the *old* private IP listed as the 'INTERNAL_IP' of the node that was stopped and restarted. The situation is resolved by restarting the cassandra service on the node that remained running. Once it has restarted, the INTERNAL_IP is correctly updated to the new private IP. 'nodetool status' shows that both nodes are up and the cluster appears to function normally. This appears to me to be the root cause of https://issues.apache.org/jira/browse/CASSANDRA-7292. Possibly https://issues.apache.org/jira/browse/CASSANDRA-8072 as well, but I am not convinced they are actually duplicates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8274) Node fails to rejoin cluster on EC2 if private IP is changed
[ https://issues.apache.org/jira/browse/CASSANDRA-8274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201230#comment-14201230 ] Brandon Williams commented on CASSANDRA-8274: - bq. This appears to me to be the root cause of https://issues.apache.org/jira/browse/CASSANDRA-7292. Possibly https://issues.apache.org/jira/browse/CASSANDRA-8072 Neither, because I've repro'd it and don't use EC2. Node fails to rejoin cluster on EC2 if private IP is changed Key: CASSANDRA-8274 URL: https://issues.apache.org/jira/browse/CASSANDRA-8274 Project: Cassandra Issue Type: Bug Components: Core Environment: Amazon EC2 Reporter: Joseph Clark Nodes in Amazon AWS EC2 Classic (not a VPC) may be assigned a new private IP if the node is stopped and then started again. In this case we have puppet update the configured listen_address to the new private IP. However, once the cassandra service starts, it is unable to communicate with the existing nodes(single region) and vice versa. 'nodetool status' shows that each node believes that it is 'UN' and the other node is 'DN'. 'nodetool gossipinfo' on the node that remained running shows the *old* private IP listed as the 'INTERNAL_IP' of the node that was stopped and restarted. The situation is resolved by restarting the cassandra service on the node that remained running. Once it has restarted, the INTERNAL_IP is correctly updated to the new private IP. 'nodetool status' shows that both nodes are up and the cluster appears to function normally. This appears to me to be the root cause of https://issues.apache.org/jira/browse/CASSANDRA-7292. Possibly https://issues.apache.org/jira/browse/CASSANDRA-8072 as well, but I am not convinced they are actually duplicates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8072) Exception during startup: Unable to gossip with any seeds
[ https://issues.apache.org/jira/browse/CASSANDRA-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201236#comment-14201236 ] Joseph Clark commented on CASSANDRA-8072: - [CASSANDRA-8274|https://issues.apache.org/jira/browse/CASSANDRA-8274] appears to me to be the root cause, in my situation at least, to [CASSANDRA-7292|https://issues.apache.org/jira/browse/CASSANDRA-7292]. I'm still not convinced that 7292 is a duplicate issue. Exception during startup: Unable to gossip with any seeds - Key: CASSANDRA-8072 URL: https://issues.apache.org/jira/browse/CASSANDRA-8072 Project: Cassandra Issue Type: Bug Reporter: Ryan Springer Assignee: Brandon Williams When Opscenter 4.1.4 or 5.0.1 tries to provision a 2-node DSC 2.0.10 cluster in either ec2 or locally, an error occurs sometimes with one of the nodes refusing to start C*. The error in the /var/log/cassandra/system.log is: ERROR [main] 2014-10-06 15:54:52,292 CassandraDaemon.java (line 513) Exception encountered during startup java.lang.RuntimeException: Unable to gossip with any seeds at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1200) at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:444) at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:655) at org.apache.cassandra.service.StorageService.initServer(StorageService.java:609) at org.apache.cassandra.service.StorageService.initServer(StorageService.java:502) at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378) at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496) at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585) INFO [StorageServiceShutdownHook] 2014-10-06 15:54:52,326 Gossiper.java (line 1279) Announcing shutdown INFO [StorageServiceShutdownHook] 2014-10-06 15:54:54,326 MessagingService.java (line 701) Waiting for messaging service to quiesce INFO [ACCEPT-localhost/127.0.0.1] 2014-10-06 15:54:54,327 MessagingService.java (line 941) MessagingService has terminated the accept() thread This errors does not always occur when provisioning a 2-node cluster, but probably around half of the time on only one of the nodes. I haven't been able to reproduce this error with DSC 2.0.9, and there have been no code or definition file changes in Opscenter. I can reproduce locally with the above steps. I'm happy to test any proposed fixes since I'm the only person able to reproduce reliably so far. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8072) Exception during startup: Unable to gossip with any seeds
[ https://issues.apache.org/jira/browse/CASSANDRA-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201236#comment-14201236 ] Joseph Clark edited comment on CASSANDRA-8072 at 11/6/14 11:52 PM: --- [CASSANDRA-8274|https://issues.apache.org/jira/browse/CASSANDRA-8274] appears to me to be the root cause, in my situation at least, to [CASSANDRA-7292|https://issues.apache.org/jira/browse/CASSANDRA-7292]. I'm still not convinced that 7292 and 8072 are duplicate issues. was (Author: jw.clark): [CASSANDRA-8274|https://issues.apache.org/jira/browse/CASSANDRA-8274] appears to me to be the root cause, in my situation at least, to [CASSANDRA-7292|https://issues.apache.org/jira/browse/CASSANDRA-7292]. I'm still not convinced that 7292 is a duplicate issue. Exception during startup: Unable to gossip with any seeds - Key: CASSANDRA-8072 URL: https://issues.apache.org/jira/browse/CASSANDRA-8072 Project: Cassandra Issue Type: Bug Reporter: Ryan Springer Assignee: Brandon Williams When Opscenter 4.1.4 or 5.0.1 tries to provision a 2-node DSC 2.0.10 cluster in either ec2 or locally, an error occurs sometimes with one of the nodes refusing to start C*. The error in the /var/log/cassandra/system.log is: ERROR [main] 2014-10-06 15:54:52,292 CassandraDaemon.java (line 513) Exception encountered during startup java.lang.RuntimeException: Unable to gossip with any seeds at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1200) at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:444) at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:655) at org.apache.cassandra.service.StorageService.initServer(StorageService.java:609) at org.apache.cassandra.service.StorageService.initServer(StorageService.java:502) at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378) at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496) at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585) INFO [StorageServiceShutdownHook] 2014-10-06 15:54:52,326 Gossiper.java (line 1279) Announcing shutdown INFO [StorageServiceShutdownHook] 2014-10-06 15:54:54,326 MessagingService.java (line 701) Waiting for messaging service to quiesce INFO [ACCEPT-localhost/127.0.0.1] 2014-10-06 15:54:54,327 MessagingService.java (line 941) MessagingService has terminated the accept() thread This errors does not always occur when provisioning a 2-node cluster, but probably around half of the time on only one of the nodes. I haven't been able to reproduce this error with DSC 2.0.9, and there have been no code or definition file changes in Opscenter. I can reproduce locally with the above steps. I'm happy to test any proposed fixes since I'm the only person able to reproduce reliably so far. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8274) Node fails to rejoin cluster on EC2 if private IP is changed
[ https://issues.apache.org/jira/browse/CASSANDRA-8274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201244#comment-14201244 ] Joseph Clark commented on CASSANDRA-8274: - 7292 and 8072 have different stack traces. Which did you reproduce? Node fails to rejoin cluster on EC2 if private IP is changed Key: CASSANDRA-8274 URL: https://issues.apache.org/jira/browse/CASSANDRA-8274 Project: Cassandra Issue Type: Bug Components: Core Environment: Amazon EC2 Reporter: Joseph Clark Nodes in Amazon AWS EC2 Classic (not a VPC) may be assigned a new private IP if the node is stopped and then started again. In this case we have puppet update the configured listen_address to the new private IP. However, once the cassandra service starts, it is unable to communicate with the existing nodes(single region) and vice versa. 'nodetool status' shows that each node believes that it is 'UN' and the other node is 'DN'. 'nodetool gossipinfo' on the node that remained running shows the *old* private IP listed as the 'INTERNAL_IP' of the node that was stopped and restarted. The situation is resolved by restarting the cassandra service on the node that remained running. Once it has restarted, the INTERNAL_IP is correctly updated to the new private IP. 'nodetool status' shows that both nodes are up and the cluster appears to function normally. This appears to me to be the root cause of https://issues.apache.org/jira/browse/CASSANDRA-7292. Possibly https://issues.apache.org/jira/browse/CASSANDRA-8072 as well, but I am not convinced they are actually duplicates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8274) Node fails to rejoin cluster on EC2 if private IP is changed
[ https://issues.apache.org/jira/browse/CASSANDRA-8274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201244#comment-14201244 ] Joseph Clark edited comment on CASSANDRA-8274 at 11/6/14 11:59 PM: --- 7292 and 8072 have different stack traces[edit: though admittedly I'm unsure if this is just a result of different versions]. Which did you reproduce? was (Author: jw.clark): 7292 and 8072 have different stack traces. Which did you reproduce? Node fails to rejoin cluster on EC2 if private IP is changed Key: CASSANDRA-8274 URL: https://issues.apache.org/jira/browse/CASSANDRA-8274 Project: Cassandra Issue Type: Bug Components: Core Environment: Amazon EC2 Reporter: Joseph Clark Nodes in Amazon AWS EC2 Classic (not a VPC) may be assigned a new private IP if the node is stopped and then started again. In this case we have puppet update the configured listen_address to the new private IP. However, once the cassandra service starts, it is unable to communicate with the existing nodes(single region) and vice versa. 'nodetool status' shows that each node believes that it is 'UN' and the other node is 'DN'. 'nodetool gossipinfo' on the node that remained running shows the *old* private IP listed as the 'INTERNAL_IP' of the node that was stopped and restarted. The situation is resolved by restarting the cassandra service on the node that remained running. Once it has restarted, the INTERNAL_IP is correctly updated to the new private IP. 'nodetool status' shows that both nodes are up and the cluster appears to function normally. This appears to me to be the root cause of https://issues.apache.org/jira/browse/CASSANDRA-7292. Possibly https://issues.apache.org/jira/browse/CASSANDRA-8072 as well, but I am not convinced they are actually duplicates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8274) Node fails to rejoin cluster on EC2 if private IP is changed
[ https://issues.apache.org/jira/browse/CASSANDRA-8274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201278#comment-14201278 ] Brandon Williams commented on CASSANDRA-8274: - Given that the stracktraces in both are exactly the same: yes :) Node fails to rejoin cluster on EC2 if private IP is changed Key: CASSANDRA-8274 URL: https://issues.apache.org/jira/browse/CASSANDRA-8274 Project: Cassandra Issue Type: Bug Components: Core Environment: Amazon EC2 Reporter: Joseph Clark Nodes in Amazon AWS EC2 Classic (not a VPC) may be assigned a new private IP if the node is stopped and then started again. In this case we have puppet update the configured listen_address to the new private IP. However, once the cassandra service starts, it is unable to communicate with the existing nodes(single region) and vice versa. 'nodetool status' shows that each node believes that it is 'UN' and the other node is 'DN'. 'nodetool gossipinfo' on the node that remained running shows the *old* private IP listed as the 'INTERNAL_IP' of the node that was stopped and restarted. The situation is resolved by restarting the cassandra service on the node that remained running. Once it has restarted, the INTERNAL_IP is correctly updated to the new private IP. 'nodetool status' shows that both nodes are up and the cluster appears to function normally. This appears to me to be the root cause of https://issues.apache.org/jira/browse/CASSANDRA-7292. Possibly https://issues.apache.org/jira/browse/CASSANDRA-8072 as well, but I am not convinced they are actually duplicates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8222) Unable to connect to any server with cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-8222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201280#comment-14201280
]
RADOMIRS CIRSKIS commented on CASSANDRA-8222:
-
[~mishail],
{code}
python -c import json; print dir(json); print json.__version__
['JsonReader', 'JsonWriter', 'ReadException', 'WriteException',
'_StringGenerator', '__builtins__', '__doc__', '__file__', '__name__',
'__package__', 'read', 'string', 'types', 'write']
Traceback (most recent call last):
File string, line 1, in module
AttributeError: 'module' object has no attribute '__version__'
{code}
Unable to connect to any server with cqlsh
Key: CASSANDRA-8222
URL: https://issues.apache.org/jira/browse/CASSANDRA-8222
Project: Cassandra
Issue Type: Bug
Environment: java -version
java version 1.7.0_25
Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)
python --version
Python 2.7.8
Reporter: RADOMIRS CIRSKIS
Labels: cqlsh
I was going through the http://wiki.apache.org/cassandra/GettingStarted
1. Installed cassandra
2. run cassandra: bin/cassandra -f
3. in another terminal attempt to run cqlsh: cqlsh
4. also I have tried explicitly: cqlsh 127.0.0.1 9042
5. it produces a message:
{noformat}
Connection error: ('Unable to connect to any servers', {'127.0.0.1':
AttributeError('module' object has no attribute 'loads',)})
{noformat}
Cassandra is running and listening on the port 9042:
{noformat}
...
INFO 03:36:08 Using Netty Version:
[netty-buffer=netty-buffer-4.0.23.Final.208198c,
netty-codec=netty-codec-4.0.23.Final.208198c,
netty-codec-http=netty-codec-http-4.0.23.Final.208198c,
netty-codec-socks=netty-codec-socks-4.0.23.Final.208198c,
netty-common=netty-common-4.0.23.Final.208198c,
netty-handler=netty-handler-4.0.23.Final.208198c,
netty-transport=netty-transport-4.0.23.Final.208198c,
netty-transport-rxtx=netty-transport-rxtx-4.0.23.Final.208198c,
netty-transport-sctp=netty-transport-sctp-4.0.23.Final.208198c,
netty-transport-udt=netty-transport-udt-4.0.23.Final.208198c]
INFO 03:36:08 Starting listening for CQL clients on
localhost/127.0.0.1:9042...
INFO 03:36:09 Binding thrift service to localhost/127.0.0.1:9160
INFO 03:36:09 Listening for thrift clients...
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8222) Unable to connect to any server with cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-8222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201290#comment-14201290
]
Mikhail Stepura commented on CASSANDRA-8222:
[~nad2000] it looks like a package/module conflict. Do you have some custom/3rd
party json module on system?
{code}
16:33 $ python -c import json; print dir(json); print json.__version__
['JSONDecoder', 'JSONEncoder', '__all__', '__author__', '__builtins__',
'__doc__', '__file__', '__name__', '__package__', '__path__', '__version__',
'_default_decoder', '_default_encoder', 'decoder', 'dump', 'dumps', 'encoder',
'load', 'loads', 'scanner']
2.0.9
{code}
Unable to connect to any server with cqlsh
Key: CASSANDRA-8222
URL: https://issues.apache.org/jira/browse/CASSANDRA-8222
Project: Cassandra
Issue Type: Bug
Environment: java -version
java version 1.7.0_25
Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)
python --version
Python 2.7.8
Reporter: RADOMIRS CIRSKIS
Labels: cqlsh
I was going through the http://wiki.apache.org/cassandra/GettingStarted
1. Installed cassandra
2. run cassandra: bin/cassandra -f
3. in another terminal attempt to run cqlsh: cqlsh
4. also I have tried explicitly: cqlsh 127.0.0.1 9042
5. it produces a message:
{noformat}
Connection error: ('Unable to connect to any servers', {'127.0.0.1':
AttributeError('module' object has no attribute 'loads',)})
{noformat}
Cassandra is running and listening on the port 9042:
{noformat}
...
INFO 03:36:08 Using Netty Version:
[netty-buffer=netty-buffer-4.0.23.Final.208198c,
netty-codec=netty-codec-4.0.23.Final.208198c,
netty-codec-http=netty-codec-http-4.0.23.Final.208198c,
netty-codec-socks=netty-codec-socks-4.0.23.Final.208198c,
netty-common=netty-common-4.0.23.Final.208198c,
netty-handler=netty-handler-4.0.23.Final.208198c,
netty-transport=netty-transport-4.0.23.Final.208198c,
netty-transport-rxtx=netty-transport-rxtx-4.0.23.Final.208198c,
netty-transport-sctp=netty-transport-sctp-4.0.23.Final.208198c,
netty-transport-udt=netty-transport-udt-4.0.23.Final.208198c]
INFO 03:36:08 Starting listening for CQL clients on
localhost/127.0.0.1:9042...
INFO 03:36:09 Binding thrift service to localhost/127.0.0.1:9160
INFO 03:36:09 Listening for thrift clients...
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8222) Unable to connect to any server with cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-8222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201313#comment-14201313
]
RADOMIRS CIRSKIS commented on CASSANDRA-8222:
-
[~mishail], thanks :). you are right. I had globally overwritten PYTHONPATH
that had included some 3rd party libs for Python 2.6.
{{unset PYTHONPATH}} solved the issue:
{code}
# unset PYTHONPATH ; python -c import json; print dir(json); print
json.__version__
['JSONDecoder', 'JSONEncoder', '__all__', '__author__', '__builtins__',
'__doc__', '__file__', '__name__', '__package__', '__path__', '__version__',
'_default_decoder', '_default_encoder', 'decoder', 'dump', 'dumps', 'encoder',
'load', 'loads', 'scanner']
2.0.9
# cqlsh
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 2.1.1 | CQL spec 3.2.0 | Native protocol v3]
Use HELP for help.
cqlsh
{code}
Unable to connect to any server with cqlsh
Key: CASSANDRA-8222
URL: https://issues.apache.org/jira/browse/CASSANDRA-8222
Project: Cassandra
Issue Type: Bug
Environment: java -version
java version 1.7.0_25
Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)
python --version
Python 2.7.8
Reporter: RADOMIRS CIRSKIS
Labels: cqlsh
I was going through the http://wiki.apache.org/cassandra/GettingStarted
1. Installed cassandra
2. run cassandra: bin/cassandra -f
3. in another terminal attempt to run cqlsh: cqlsh
4. also I have tried explicitly: cqlsh 127.0.0.1 9042
5. it produces a message:
{noformat}
Connection error: ('Unable to connect to any servers', {'127.0.0.1':
AttributeError('module' object has no attribute 'loads',)})
{noformat}
Cassandra is running and listening on the port 9042:
{noformat}
...
INFO 03:36:08 Using Netty Version:
[netty-buffer=netty-buffer-4.0.23.Final.208198c,
netty-codec=netty-codec-4.0.23.Final.208198c,
netty-codec-http=netty-codec-http-4.0.23.Final.208198c,
netty-codec-socks=netty-codec-socks-4.0.23.Final.208198c,
netty-common=netty-common-4.0.23.Final.208198c,
netty-handler=netty-handler-4.0.23.Final.208198c,
netty-transport=netty-transport-4.0.23.Final.208198c,
netty-transport-rxtx=netty-transport-rxtx-4.0.23.Final.208198c,
netty-transport-sctp=netty-transport-sctp-4.0.23.Final.208198c,
netty-transport-udt=netty-transport-udt-4.0.23.Final.208198c]
INFO 03:36:08 Starting listening for CQL clients on
localhost/127.0.0.1:9042...
INFO 03:36:09 Binding thrift service to localhost/127.0.0.1:9160
INFO 03:36:09 Listening for thrift clients...
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8274) Node fails to rejoin cluster on EC2 if private IP is changed
[ https://issues.apache.org/jira/browse/CASSANDRA-8274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201345#comment-14201345 ] Joseph Clark commented on CASSANDRA-8274: - at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1193) at org.apache.cassandra.service.StorageService.*prepareReplacementInfo*(StorageService.java:419) at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:650) at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1200) at org.apache.cassandra.service.StorageService.*checkForEndpointCollision*(StorageService.java:444) at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:655) Slightly different :) Even if they were the same, I believe that they are still separate issues. 7292 and this bug both involve starting a node that the cluster *already knows about* as identified by the public lP. Another similarity between 7292 and this bug, in my case at least and likely the original reporter as well, is that the private IP/listen_address has been changed. As far as I can tell, 8072 occurs with brand new nodes that aren't replacing another node in the cluster. Node fails to rejoin cluster on EC2 if private IP is changed Key: CASSANDRA-8274 URL: https://issues.apache.org/jira/browse/CASSANDRA-8274 Project: Cassandra Issue Type: Bug Components: Core Environment: Amazon EC2 Reporter: Joseph Clark Nodes in Amazon AWS EC2 Classic (not a VPC) may be assigned a new private IP if the node is stopped and then started again. In this case we have puppet update the configured listen_address to the new private IP. However, once the cassandra service starts, it is unable to communicate with the existing nodes(single region) and vice versa. 'nodetool status' shows that each node believes that it is 'UN' and the other node is 'DN'. 'nodetool gossipinfo' on the node that remained running shows the *old* private IP listed as the 'INTERNAL_IP' of the node that was stopped and restarted. The situation is resolved by restarting the cassandra service on the node that remained running. Once it has restarted, the INTERNAL_IP is correctly updated to the new private IP. 'nodetool status' shows that both nodes are up and the cluster appears to function normally. This appears to me to be the root cause of https://issues.apache.org/jira/browse/CASSANDRA-7292. Possibly https://issues.apache.org/jira/browse/CASSANDRA-8072 as well, but I am not convinced they are actually duplicates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8222) Unable to connect to any server with cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-8222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler resolved CASSANDRA-8222.
---
Resolution: Not a Problem
Unable to connect to any server with cqlsh
Key: CASSANDRA-8222
URL: https://issues.apache.org/jira/browse/CASSANDRA-8222
Project: Cassandra
Issue Type: Bug
Environment: java -version
java version 1.7.0_25
Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)
python --version
Python 2.7.8
Reporter: RADOMIRS CIRSKIS
Labels: cqlsh
I was going through the http://wiki.apache.org/cassandra/GettingStarted
1. Installed cassandra
2. run cassandra: bin/cassandra -f
3. in another terminal attempt to run cqlsh: cqlsh
4. also I have tried explicitly: cqlsh 127.0.0.1 9042
5. it produces a message:
{noformat}
Connection error: ('Unable to connect to any servers', {'127.0.0.1':
AttributeError('module' object has no attribute 'loads',)})
{noformat}
Cassandra is running and listening on the port 9042:
{noformat}
...
INFO 03:36:08 Using Netty Version:
[netty-buffer=netty-buffer-4.0.23.Final.208198c,
netty-codec=netty-codec-4.0.23.Final.208198c,
netty-codec-http=netty-codec-http-4.0.23.Final.208198c,
netty-codec-socks=netty-codec-socks-4.0.23.Final.208198c,
netty-common=netty-common-4.0.23.Final.208198c,
netty-handler=netty-handler-4.0.23.Final.208198c,
netty-transport=netty-transport-4.0.23.Final.208198c,
netty-transport-rxtx=netty-transport-rxtx-4.0.23.Final.208198c,
netty-transport-sctp=netty-transport-sctp-4.0.23.Final.208198c,
netty-transport-udt=netty-transport-udt-4.0.23.Final.208198c]
INFO 03:36:08 Starting listening for CQL clients on
localhost/127.0.0.1:9042...
INFO 03:36:09 Binding thrift service to localhost/127.0.0.1:9160
INFO 03:36:09 Listening for thrift clients...
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8274) Node fails to rejoin cluster on EC2 if private IP is changed
[ https://issues.apache.org/jira/browse/CASSANDRA-8274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Clark updated CASSANDRA-8274: Description: Nodes in Amazon AWS EC2 Classic (not a VPC) may be assigned a new private IP if the node is stopped and then started again. In this case we have puppet update the configured listen_address to the new private IP. However, once the cassandra service starts, it is unable to communicate with the existing nodes(single region) and vice versa. 'nodetool status' shows that each node believes that it is 'UN' and the other node is 'DN'. 'nodetool gossipinfo' on the node that remained running shows the *old* private IP listed as the 'INTERNAL_IP' of the node that was stopped and restarted. The situation is resolved by restarting the cassandra service on the node that remained running. Once it has restarted, the INTERNAL_IP is correctly updated to the new private IP. 'nodetool status' shows that both nodes are up and the cluster appears to function normally. This appears to me to be the root cause of https://issues.apache.org/jira/browse/CASSANDRA-7292. -Possibly https://issues.apache.org/jira/browse/CASSANDRA-8072 as well, but I am not convinced they are actually duplicates.- was: Nodes in Amazon AWS EC2 Classic (not a VPC) may be assigned a new private IP if the node is stopped and then started again. In this case we have puppet update the configured listen_address to the new private IP. However, once the cassandra service starts, it is unable to communicate with the existing nodes(single region) and vice versa. 'nodetool status' shows that each node believes that it is 'UN' and the other node is 'DN'. 'nodetool gossipinfo' on the node that remained running shows the *old* private IP listed as the 'INTERNAL_IP' of the node that was stopped and restarted. The situation is resolved by restarting the cassandra service on the node that remained running. Once it has restarted, the INTERNAL_IP is correctly updated to the new private IP. 'nodetool status' shows that both nodes are up and the cluster appears to function normally. This appears to me to be the root cause of https://issues.apache.org/jira/browse/CASSANDRA-7292. Possibly https://issues.apache.org/jira/browse/CASSANDRA-8072 as well, but I am not convinced they are actually duplicates. Node fails to rejoin cluster on EC2 if private IP is changed Key: CASSANDRA-8274 URL: https://issues.apache.org/jira/browse/CASSANDRA-8274 Project: Cassandra Issue Type: Bug Components: Core Environment: Amazon EC2 Reporter: Joseph Clark Nodes in Amazon AWS EC2 Classic (not a VPC) may be assigned a new private IP if the node is stopped and then started again. In this case we have puppet update the configured listen_address to the new private IP. However, once the cassandra service starts, it is unable to communicate with the existing nodes(single region) and vice versa. 'nodetool status' shows that each node believes that it is 'UN' and the other node is 'DN'. 'nodetool gossipinfo' on the node that remained running shows the *old* private IP listed as the 'INTERNAL_IP' of the node that was stopped and restarted. The situation is resolved by restarting the cassandra service on the node that remained running. Once it has restarted, the INTERNAL_IP is correctly updated to the new private IP. 'nodetool status' shows that both nodes are up and the cluster appears to function normally. This appears to me to be the root cause of https://issues.apache.org/jira/browse/CASSANDRA-7292. -Possibly https://issues.apache.org/jira/browse/CASSANDRA-8072 as well, but I am not convinced they are actually duplicates.- -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8022) cqlsh hangs indefinitely within a Docker container connecting to itself with hostname
[
https://issues.apache.org/jira/browse/CASSANDRA-8022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201420#comment-14201420
]
Mikhail Stepura commented on CASSANDRA-8022:
I doubt that is network related. I suspect it's a way cqlsh is launched. It
prints no prompt, so I guess cqlsh 'thinks' it is launched not from a terminal
( i.e. {{sys.stdin.isatty() == false}}).
cqlsh hangs indefinitely within a Docker container connecting to itself with
hostname
-
Key: CASSANDRA-8022
URL: https://issues.apache.org/jira/browse/CASSANDRA-8022
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Ubuntu 14.04, running Docker, run inside a Ubuntu 14.04
container.
Reporter: Matthew O'Riordan
Labels: cqlsh
I am unable to use the `cqlsh` tool within a Docker container running
Cassandra. Previously I would use the Java Thrift based `cqlsh` tool as
follows:
```
cqlsh --username cassandra --password whatever $(hostname)
```
When I run the `cqlsh` command after attaching to a running container (I use
LXC containerisation that allows attaching to a running container and running
a console), it simply hangs and never reports an error. With the `--debug`
flag on, I get the following with:
**cqlsh 4.1.1**
```
$ cqlsh --debug --username cassandra --password obfuscated $(hostname)
Using CQL driver: module 'cql' from
'/usr/share/cassandra/lib/cql-internal-only-1.4.1.zip/cql-1.4.1/cql/__init__.py'
Using thrift lib: module 'thrift' from
'/usr/share/cassandra/lib/thrift-python-internal-only-0.9.1.zip/thrift/__init__.py'
```
It then hangs in this state indefinitely, I have no errors from `cqlsh` and
no errors in the Cassandra log.
**cqlsh 5.0.1**
```
$ cqlsh --debug --username cassandra --passwor obfuscated $(hostname)
Using CQL driver: module 'cassandra' from
'/usr/share/cassandra/lib/cassandra-driver-internal-only-2.1.0.post.zip/cassandra-driver-2.1.0.post/cassandra/__init__.py'
```
It then also hangs in this state indefinitely, I have no errors from `cqlsh`
and no errors in the Cassandra log.
What's interesting, and quite confusing is that:
* I can telnet within the container as follows `telnet $(hostname) 9042` and
I get a socket. When trying to issue some commands, I see Protocol errors in
the Cassandra log thus verifying that the port is indeed open on the host
that resolves from $(hostname)
* If I `cqlsh` from another container or another host to the Cassandra
container it works just fine.
* I have tried disabling authentication altogether and using the
AllowAllAuthenticator, and I experience the same problem.
* `nodetool` works fine
In the mean time, I am forced to `cqlsh` from another container as a
workaround. Happy to try and do anything require to diagnose the cause of
this problem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-5717) Repair causes streaming errors
[
https://issues.apache.org/jira/browse/CASSANDRA-5717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201595#comment-14201595
]
Michael A. Fiedler commented on CASSANDRA-5717:
---
Seeing this problem currently on a 1.2.19 node, and chance a fix for this might
be backported, or is there some other workaround? My scenario doesn't change
any schema definitions, just a node replacing an existing node.
Repair causes streaming errors
--
Key: CASSANDRA-5717
URL: https://issues.apache.org/jira/browse/CASSANDRA-5717
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.2.6
Environment: CentOS release 6.3 (Final)
Reporter: Yoan Arnaudov
I've changed the replication factor for one of keyspaces and now I'm running
repairs on column families (manually). I have 3 nodes cluster. Here is the
error in the error log for one of the nodes.
{code:title=Error Log}
ERROR [Streaming to /NODE_IP:1] 2013-07-01 09:31:29,819 CassandraDaemon.java
(line 192) Exception in thread Thread[Streaming to /NODE_IP:1,5,main]
java.lang.RuntimeException: java.io.IOException: Broken pipe
at com.google.common.base.Throwables.propagate(Throwables.java:160)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: Broken pipe
at sun.nio.ch.FileChannelImpl.transferTo0(Native Method)
at sun.nio.ch.FileChannelImpl.transferToDirectly(Unknown Source)
at sun.nio.ch.FileChannelImpl.transferTo(Unknown Source)
at
org.apache.cassandra.streaming.compress.CompressedFileStreamTask.stream(CompressedFileStreamTask.java:93)
at
org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:91)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
... 3 more
ERROR [Streaming to /NODE_IP:2] 2013-07-01 09:44:18,372 CassandraDaemon.java
(line 192) Exception in thread Thread[Streaming to /NODE_IP:2,5,main]
java.lang.RuntimeException: java.io.IOException: Broken pipe
at com.google.common.base.Throwables.propagate(Throwables.java:160)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: Broken pipe
at sun.nio.ch.FileChannelImpl.transferTo0(Native Method)
at sun.nio.ch.FileChannelImpl.transferToDirectly(Unknown Source)
at sun.nio.ch.FileChannelImpl.transferTo(Unknown Source)
at
org.apache.cassandra.streaming.compress.CompressedFileStreamTask.stream(CompressedFileStreamTask.java:93)
at
org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:91)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
... 3 more
{code}
{code:title=netstats for one of the nodes}
Mode: NORMAL
Not sending any streams.
Streaming from: /IP
KEYSPACE:
/var/lib/cassandra/data/KEYSPACE/ns_history_org/KEYSPACE-ns_history_org-hf-282-Data.db
sections=1 progress=0/1393915753 - 0%
KEYSPACE:
/var/lib/cassandra/data/KEYSPACE/ns_history_org/KEYSPACE-ns_history_org-ic-455-Data.db
sections=1 progress=0/792707 - 0%
Streaming from: /IP
KEYSPACE:
/var/lib/cassandra/data/KEYSPACE/ns_history_org/KEYSPACE-ns_history_org-hf-255-Data.db
sections=1 progress=0/1398197628 - 0%
KEYSPACE:
/var/lib/cassandra/data/KEYSPACE/ns_history_biz/KEYSPACE-ns_history_biz-ic-341-Data.db
sections=1 progress=0/6153542 - 0%
KEYSPACE:
/var/lib/cassandra/data/KEYSPACE/listing/KEYSPACE-listing-hf-539-Data.db
sections=1 progress=0/86968194 - 0%
KEYSPACE:
/var/lib/cassandra/data/KEYSPACE/ns_history_biz/KEYSPACE-ns_history_biz-hf-244-Data.db
sections=1 progress=0/322197762 - 0%
KEYSPACE:
/var/lib/cassandra/data/KEYSPACE/ns_history_biz/KEYSPACE-ns_history_biz-ic-346-Data.db
sections=1 progress=0/6219503 - 0%
KEYSPACE:
/var/lib/cassandra/data/KEYSPACE/listing/KEYSPACE-listing-hf-487-Data.db
sections=1 progress=0/2689291466 - 0%
KEYSPACE:
/var/lib/cassandra/data/KEYSPACE/listing/KEYSPACE-listing-ic-684-Data.db
sections=1 progress=0/3717513 - 0%
KEYSPACE:
/var/lib/cassandra/data/KEYSPACE/ns_history_org/KEYSPACE-ns_history_org-ic-413-Data.db
sections=1 progress=0/22256993 - 0%
KEYSPACE:
/var/lib/cassandra/data/KEYSPACE/listing/KEYSPACE-listing-hf-529-Data.db
