Hi - I am running corosync on 3 nodes of CentOS release 6.9 (Final). Corosync version is corosync-1.4.7. The nodes are not seeing each other and not able to form memberships. What I see is continuous message about " A processor joined or left the membership and a new membership was formed." For example:on node: vm2883711991
Jun 07 07:54:52 corosync [pcmk ] info: pcmk_peer_update: MEMB: vm2883711991 184555180 Jun 07 07:54:52 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed. Jun 07 07:54:52 corosync [CPG ] chosen downlist: sender r(0) ip(172.22.0.11) ; members(old:1 left:0) Jun 07 07:54:52 corosync [MAIN ] Completed service synchronization, ready to provide service. Jun 07 07:55:04 corosync [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 71084: memb=1, new=0, lost=0 Jun 07 07:55:04 corosync [pcmk ] info: pcmk_peer_update: memb: vm2883711991 184555180 Jun 07 07:55:04 corosync [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 71084: memb=1, new=0, lost=0 Jun 07 07:55:04 corosync [pcmk ] info: pcmk_peer_update: MEMB: vm2883711991 184555180 Jun 07 07:55:04 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed. Jun 07 07:55:04 corosync [CPG ] chosen downlist: sender r(0) ip(172.22.0.11) ; members(old:1 left:0) Jun 07 07:55:04 corosync [MAIN ] Completed service synchronization, ready to provide service. Jun 07 07:55:16 corosync [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 71096: memb=1, new=0, lost=0 Jun 07 07:55:16 corosync [pcmk ] info: pcmk_peer_update: memb: vm2883711991 184555180 Jun 07 07:55:16 corosync [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 71096: memb=1, new=0, lost=0 Jun 07 07:55:16 corosync [pcmk ] info: pcmk_peer_update: MEMB: vm2883711991 184555180 Jun 07 07:55:16 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed. Jun 07 07:55:16 corosync [CPG ] chosen downlist: sender r(0) ip(172.22.0.11) ; members(old:1 left:0) Jun 07 07:55:16 corosync [MAIN ] Completed service synchronization, ready to provide service. Jun 07 07:55:28 corosync [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 71108: memb=1, new=0, lost=0 Jun 07 07:55:28 corosync [pcmk ] info: pcmk_peer_update: memb: vm2883711991 184555180 Jun 07 07:55:28 corosync [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 71108: memb=1, new=0, lost=0 Jun 07 07:55:28 corosync [pcmk ] info: pcmk_peer_update: MEMB: vm2883711991 184555180 Jun 07 07:55:28 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed. Jun 07 07:55:28 corosync [CPG ] chosen downlist: sender r(0) ip(172.22.0.11) ; members(old:1 left:0) Jun 07 07:55:28 corosync [MAIN ] Completed service synchronization, ready to provide service. Jun 07 07:55:40 corosync [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 71120: memb=1, new=0, lost=0 Jun 07 07:55:40 corosync [pcmk ] info: pcmk_peer_update: memb: vm2883711991 184555180 Jun 07 07:55:40 corosync [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 71120: memb=1, new=0, lost=0 Jun 07 07:55:40 corosync [pcmk ] info: pcmk_peer_update: MEMB: vm2883711991 184555180 Jun 07 07:55:40 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed. Jun 07 07:55:40 corosync [CPG ] chosen downlist: sender r(0) ip(172.22.0.11) ; members(old:1 left:0) Jun 07 07:55:40 corosync [MAIN ] Completed service synchronization, ready to provide service. Jun 07 07:55:52 corosync [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 71132: memb=1, new=0, lost=0 Jun 07 07:55:52 corosync [pcmk ] info: pcmk_peer_update: memb: vm2883711991 184555180 Jun 07 07:55:52 corosync [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 71132: memb=1, new=0, lost=0 Jun 07 07:55:52 corosync [pcmk ] info: pcmk_peer_update: MEMB: vm2883711991 184555180 Jun 07 07:55:52 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed. Jun 07 07:55:52 corosync [CPG ] chosen downlist: sender r(0) ip(172.22.0.11) ; members(old:1 left:0) Jun 07 07:55:52 corosync [MAIN ] Completed service synchronization, ready to provide service. Jun 07 07:56:04 corosync [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 71144: memb=1, new=0, lost=0 Jun 07 07:56:04 corosync [pcmk ] info: pcmk_peer_update: memb: vm2883711991 184555180 Jun 07 07:56:04 corosync [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 71144: memb=1, new=0, lost=0 Jun 07 07:56:04 corosync [pcmk ] info: pcmk_peer_update: MEMB: vm2883711991 184555180 Jun 07 07:56:04 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed. Jun 07 07:56:17 corosync [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 71156: memb=1, new=0, lost=0 Jun 07 07:56:17 corosync [pcmk ] info: pcmk_peer_update: memb: vm2883711991 184555180 Jun 07 07:56:17 corosync [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 71156: memb=1, new=0, lost=0 Jun 07 07:56:17 corosync [pcmk ] info: pcmk_peer_update: MEMB: vm2883711991 184555180 Jun 07 07:56:17 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed. Jun 07 07:56:17 corosync [CPG ] chosen downlist: sender r(0) ip(172.22.0.11) ; members(old:1 left:0) Jun 07 07:56:17 corosync [MAIN ] Completed service synchronization, ready to provide service. Similarly on node: vme6c95166f0 Jun 07 07:55:12 corosync [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 71088: memb=1, new=0, lost=0 Jun 07 07:55:12 corosync [pcmk ] info: pcmk_peer_update: memb: vme6c95166f0 67114668 Jun 07 07:55:12 corosync [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 71088: memb=1, new=0, lost=0 Jun 07 07:55:12 corosync [pcmk ] info: pcmk_peer_update: MEMB: vme6c95166f0 67114668 Jun 07 07:55:12 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed. Jun 07 07:55:12 corosync [CPG ] chosen downlist: sender r(0) ip(172.22.0.4) ; members(old:1 left:0) Jun 07 07:55:12 corosync [MAIN ] Completed service synchronization, ready to provide service. Jun 07 07:55:24 corosync [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 71100: memb=1, new=0, lost=0 Jun 07 07:55:24 corosync [pcmk ] info: pcmk_peer_update: memb: vme6c95166f0 67114668 Jun 07 07:55:24 corosync [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 71100: memb=1, new=0, lost=0 Jun 07 07:55:24 corosync [pcmk ] info: pcmk_peer_update: MEMB: vme6c95166f0 67114668 Jun 07 07:55:24 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed. Jun 07 07:55:24 corosync [CPG ] chosen downlist: sender r(0) ip(172.22.0.4) ; members(old:1 left:0) Jun 07 07:55:24 corosync [MAIN ] Completed service synchronization, ready to provide service. Jun 07 07:55:37 corosync [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 71112: memb=1, new=0, lost=0 Jun 07 07:55:37 corosync [pcmk ] info: pcmk_peer_update: memb: vme6c95166f0 67114668 Jun 07 07:55:37 corosync [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 71112: memb=1, new=0, lost=0 Jun 07 07:55:37 corosync [pcmk ] info: pcmk_peer_update: MEMB: vme6c95166f0 67114668 Jun 07 07:55:37 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed. Jun 07 07:55:49 corosync [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 71124: memb=1, new=0, lost=0 Jun 07 07:55:49 corosync [pcmk ] info: pcmk_peer_update: memb: vme6c95166f0 67114668 Jun 07 07:55:49 corosync [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 71124: memb=1, new=0, lost=0 Jun 07 07:55:49 corosync [pcmk ] info: pcmk_peer_update: MEMB: vme6c95166f0 67114668 Jun 07 07:55:49 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed. Jun 07 07:55:49 corosync [CPG ] chosen downlist: sender r(0) ip(172.22.0.4) ; members(old:1 left:0) Jun 07 07:55:49 corosync [MAIN ] Completed service synchronization, ready to provide service. Jun 07 07:56:02 corosync [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 71136: memb=1, new=0, lost=0 Jun 07 07:56:02 corosync [pcmk ] info: pcmk_peer_update: memb: vme6c95166f0 67114668 Jun 07 07:56:02 corosync [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 71136: memb=1, new=0, lost=0 Jun 07 07:56:02 corosync [pcmk ] info: pcmk_peer_update: MEMB: vme6c95166f0 67114668 Jun 07 07:56:02 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed. Jun 07 07:56:02 corosync [CPG ] chosen downlist: sender r(0) ip(172.22.0.4) ; members(old:1 left:0) Jun 07 07:56:02 corosync [MAIN ] Completed service synchronization, ready to provide service. Jun 07 07:56:14 corosync [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 71148: memb=1, new=0, lost=0 Jun 07 07:56:14 corosync [pcmk ] info: pcmk_peer_update: memb: vme6c95166f0 67114668 Jun 07 07:56:14 corosync [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 71148: memb=1, new=0, lost=0 Jun 07 07:56:14 corosync [pcmk ] info: pcmk_peer_update: MEMB: vme6c95166f0 67114668 Jun 07 07:56:14 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed. Jun 07 07:56:27 corosync [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 71160: memb=1, new=0, lost=0 Jun 07 07:56:27 corosync [pcmk ] info: pcmk_peer_update: memb: vme6c95166f0 67114668 Jun 07 07:56:27 corosync [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 71160: memb=1, new=0, lost=0 Jun 07 07:56:27 corosync [pcmk ] info: pcmk_peer_update: MEMB: vme6c95166f0 67114668 Jun 07 07:56:27 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed. Jun 07 07:56:39 corosync [pcmk ] notice: pcmk_peer_update: Transitional membership event on ring 71172: memb=1, new=0, lost=0 Jun 07 07:56:39 corosync [pcmk ] info: pcmk_peer_update: memb: vme6c95166f0 67114668 Jun 07 07:56:39 corosync [pcmk ] notice: pcmk_peer_update: Stable membership event on ring 71172: memb=1, new=0, lost=0 Jun 07 07:56:39 corosync [pcmk ] info: pcmk_peer_update: MEMB: vme6c95166f0 67114668 Jun 07 07:56:39 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed. Jun 07 07:56:39 corosync [CPG ] chosen downlist: sender r(0) ip(172.22.0.4) ; members(old:1 left:0) Jun 07 07:56:39 corosync [MAIN ] Completed service synchronization, ready to provide service. and similar message on the third node as well. my corosync.conf is as follows: compatibility: whitetank totem { version: 2 secauth: off threads: 0 interface { member { memberaddr: 172.22.0.4 } member { memberaddr: 172.22.0.11 } member { memberaddr: 172.22.0.13 } bindnetaddr: 172.22.0.4 ringnumber: 0 mcastport: 5405 ttl: 1 } transport: udpu token: 10000 token_retransmits_before_loss_const: 10 } logging { fileline: off to_stderr: yes to_logfile: yes to_syslog: no logfile: /var/log/cluster/corosync.log timestamp: on logger_subsys { subsys: AMF debug: on } } service { name: pacemaker ver: 1 } amf { mode: disabled } In general , this configuration has worked for me in other clusters I have, but this particular setup is running into this issue. Request help on how to debug and resolve this condition. Thanks
_______________________________________________ Users mailing list: Users@clusterlabs.org https://lists.clusterlabs.org/mailman/listinfo/users Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org