Re: [ClusterLabs] corosync not able to form cluster

2018-06-08 Thread Prasad Nagaraj
Hi Christine - Thanks for looking into the logs. I also see that the node eventually comes out of GATHER state here: Jun 07 16:56:10 corosync [TOTEM ] entering GATHER state from 0. Jun 07 16:56:10 corosync [TOTEM ] Creating commit token because I am the rep. Does it mean, it has timed out or

Re: [ClusterLabs] corosync not able to form cluster

2018-06-08 Thread Christine Caulfield
On 07/06/18 18:32, Prasad Nagaraj wrote: > Hi Christine - Got it:) > > I have collected few seconds of debug logs from all nodes after startup. > Please find them attached. > Please let me know if this will help us to identify rootcause. > The problem is on the node coro.4 - it never gets out

Re: [ClusterLabs] corosync not able to form cluster

2018-06-07 Thread Christine Caulfield
On 07/06/18 15:53, Prasad Nagaraj wrote: > Hi - As you can see in the corosync.conf details - i have already kept > debug: on > But only in the (disabled) AMF subsystem, not for corosync as a whole :) logger_subsys { subsys: AMF debug: on } Chrissie > > On Thu, 7 Jun 2018,

Re: [ClusterLabs] corosync not able to form cluster

2018-06-07 Thread Prasad Nagaraj
Hi - As you can see in the corosync.conf details - i have already kept debug: on Thanks! On Thu, 7 Jun 2018, 8:03 pm Christine Caulfield, wrote: > On 07/06/18 15:24, Prasad Nagaraj wrote: > > > > No iptables or otherwise firewalls are setup on these nodes. > > > > One observation is that each

Re: [ClusterLabs] corosync not able to form cluster

2018-06-07 Thread Christine Caulfield
On 07/06/18 15:24, Prasad Nagaraj wrote: > > No iptables or otherwise firewalls are setup on these nodes. > > One observation is that each node sends messages on with its own ring > sequence number which is not converging.. I have seen that in a good > cluster, when nodes respond with same

Re: [ClusterLabs] corosync not able to form cluster

2018-06-07 Thread Prasad Nagaraj
No iptables or otherwise firewalls are setup on these nodes. One observation is that each node sends messages on with its own ring sequence number which is not converging.. I have seen that in a good cluster, when nodes respond with same sequence number, the membership is automatically formed.

Re: [ClusterLabs] corosync not able to form cluster

2018-06-07 Thread Christine Caulfield
On 07/06/18 12:00, Prasad Nagaraj wrote: > And here is how startup log looks on one of the nodes. > Its pretty much same on the other nodes as well > That all looks fine to me (from a corosync POV anyway). Have you checked for any iptables rules getting in the way? Chrissie > Jun 07 10:41:37

Re: [ClusterLabs] corosync not able to form cluster

2018-06-07 Thread Prasad Nagaraj
And here is how startup log looks on one of the nodes. Its pretty much same on the other nodes as well Jun 07 10:41:37 corosync [MAIN ] Corosync Cluster Engine ('UNKNOWN'): started and ready to provide service. Jun 07 10:41:37 corosync [MAIN ] Corosync built-in features: nss Jun 07 10:41:37

Re: [ClusterLabs] corosync not able to form cluster

2018-06-07 Thread Prasad Nagaraj
Hi Christine - Thanks for looking into this and here are the details. All the nodes are pingable from each other and actively exchanging corosync packets from each other as seen from tcpdump Here is the ifconfig out from each of the node # ifconfig eth0 Link encap:Ethernet HWaddr

Re: [ClusterLabs] corosync not able to form cluster

2018-06-07 Thread Christine Caulfield
On 07/06/18 09:21, Prasad Nagaraj wrote: > Hi - I am running corosync on  3 nodes of CentOS release 6.9 (Final). > Corosync version is  corosync-1.4.7. > The nodes are not seeing each other and not able to form memberships. > What I see is continuous message about " A processor joined or left the

[ClusterLabs] corosync not able to form cluster

2018-06-07 Thread Prasad Nagaraj
Hi - I am running corosync on 3 nodes of CentOS release 6.9 (Final). Corosync version is corosync-1.4.7. The nodes are not seeing each other and not able to form memberships. What I see is continuous message about " A processor joined or left the membership and a new membership was formed." For