[ https://issues.apache.org/jira/browse/CASSANDRA-8306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Russ Hatch updated CASSANDRA-8306: ---------------------------------- Assignee: (was: Russ Hatch) > exception in nodetool enablebinary > ---------------------------------- > > Key: CASSANDRA-8306 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8306 > Project: Cassandra > Issue Type: Bug > Reporter: Rafał Furmański > Attachments: system.log.zip > > > I was trying to add new node (db4) to existing cluster - with no luck. I > can't see any errors in system.log. nodetool status shows, that node is > joining into cluster (for many hours). Attaching error and cluster info: > {code} > root@db4:~# nodetool enablebinary > error: Error starting native transport: null > -- StackTrace -- > java.lang.RuntimeException: Error starting native transport: null > at > org.apache.cassandra.service.StorageService.startNativeTransport(StorageService.java:350) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75) > at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279) > at > com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112) > at > com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46) > at > com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237) > at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138) > at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252) > at > com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819) > at > com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801) > at > javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487) > at > javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97) > at > javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328) > at > javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420) > at > javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322) > at sun.rmi.transport.Transport$1.run(Transport.java:177) > at sun.rmi.transport.Transport$1.run(Transport.java:174) > at java.security.AccessController.doPrivileged(Native Method) > at sun.rmi.transport.Transport.serviceCall(Transport.java:173) > at > sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556) > at > sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811) > at > sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > {code} > {code} > root@db4:~# nodetool describecluster > Cluster Information: > Name: Production Cluster > Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch > Partitioner: org.apache.cassandra.dht.Murmur3Partitioner > Schema versions: > b7e98bb9-717f-3f59-bac4-84bc19544e90: [10.195.15.163, > 10.195.15.162, 10.195.15.167, 10.195.15.166] > {code} > {code} > root@db4:~# nodetool status > Datacenter: Ashburn > =================== > Status=Up/Down > |/ State=Normal/Leaving/Joining/Moving > -- Address Load Tokens Owns Host ID > Rack > UN 10.195.15.163 12.05 GB 256 ? > 0a9f478c-80b5-4c15-8b2e-e27df6684c69 RAC1 > UN 10.195.15.162 12.8 GB 256 ? > c18d2218-ef84-4165-9c3a-05f592f512e9 RAC1 > UJ 10.195.15.167 18.61 GB 256 ? > 0d3999d9-1e33-4407-bbbd-10cf0a93b3ba RAC1 > UN 10.195.15.166 13.67 GB 256 ? > df8df3b7-da17-48de-8cf6-1c718dc2fde8 RAC1 > {code} > I can't even connect to cassandra using cqlsh: > {code} > root@db4:~# cqlsh 10.195.15.167 > Connection error: ('Unable to connect to any servers', {'10.195.15.167': > error(111, "Tried connecting to [('10.195.15.167', 9042)]. Last error: > Connection refused")}) > {code} > In OpsCenter I can see that node has status: Active - Joining and Gossip is > active. Any ideas what could be wrong with this node? -- This message was sent by Atlassian JIRA (v6.3.4#6332)