Hi, We tried rebooting again Node 1 and this time we observed nodetool status displaying "UN" for all 3 nodes in node1.
Executing nodetool status on "Node 3" displays "UN" for all the nodes. Executing nodetool status on "Node 2" displays "DN" for node 1 (rebooted node) and "UN" for other 2 nodes. We also observed "Node 2" sending "Syn" message to "Node 1", but no "Ack" received from "Node 1" initially for 15min and then it is started receiving it. Please let us know the reason why "Node 2" is not receiving any "Ack" message from "Node 1" for 10 -15 minutes and how it suddenly able to receive "Ac" messages. Inter node communication port is 7000. 3 node cluster, and we have all 3 nodes as seed ips. Thanks and Regards A.SathishKumar On Fri, Dec 22, 2017 at 5:15 PM, sat <sathish.al...@gmail.com> wrote: > Hi, > > We checked and we were able to telnet to port 7000. > > Thanks and Regards > A.SathishKumar > > On Fri, Dec 22, 2017 at 3:43 PM, Nitan Kainth <nitankai...@gmail.com> > wrote: > >> Try telnet on your listen port. It must be network issue due to port or >> firewall issue. >> >> Sent from my iPhone >> >> On Dec 22, 2017, at 5:28 PM, sat <sathish.al...@gmail.com> wrote: >> >> >> >> Hi, >> >> We have 3 nodes in cluster, we rebooted one of the cassandra VM, we >> noticed nodetool status returning "UN" for itself and "DN" for other node, >> although we observe gossip sync and ack messages being shared between these >> nodes. >> >> *Issue in Detail* >> >> *Nodes in cluster* >> Node1 >> Node 2 >> Node 3 >> >> All above nodes formed cluster and nodetool status in all 3 machines were >> "UN" >> >> We rebooted Node 1 and restarted cassandra on node1, then we ran nodetool >> status and observed >> >> Node 1 - UN >> Node 2 - DN >> Node 3 - DN >> >> However when we run nodetool status on other 2 nodes (Node2, and Node 3) >> they claim all 3 nodes are "UN" >> >> We enabled "Trace" level and checked Gossip messages and noticed "SYN", >> "ACK" and "ACK2" initiated and received messages in Node 1 for other 2 >> nodes, but still nodetool status marks other 2 nodes as down. >> >> Please let us know how nodetool detects other nodes as "DOWN". Any help >> is highly appreciated. >> >> Thanks >> A.SathishKumar >> >> > > > -- > A.SathishKumar > 044-24735023 > -- A.SathishKumar 044-24735023
