---
** [tickets:#1298] redundant 'node left the cluster' messages **
**Status:** unassigned
**Milestone:** 4.5.2
**Created:** Thu Apr 02, 2015 05:45 AM UTC by Sirisha Alla
**Last Updated:** Thu Apr 02, 2015 05:45 AM UTC
**Owner:** nobody
This issue is seen on 46FC changeset 6377.
During switchover payload is removed from the cluster.
syslog on SC-1 which is Active when switchover is issued:
Mar 31 16:29:19 SLES-64BIT-SLOT1 osafamfd[3682]: NO Controller switch over
initiated
Mar 31 16:29:19 SLES-64BIT-SLOT1 osafamfd[3682]: NO ROLE SWITCH Active -->
Quiesced
Mar 31 16:29:19 SLES-64BIT-SLOT1 osafimmnd[3626]: NO Implementer (applier)
connected: 113 (@OpenSafImmReplicatorA) <332, 2010f>
Mar 31 16:29:19 SLES-64BIT-SLOT1 osafntfimcnd[3918]: NO Started
Mar 31 16:29:19 SLES-64BIT-SLOT1 osafimmnd[3626]: NO Implementer disconnected
15 <0, 2030f> (MsgQueueService131855)
Mar 31 16:29:19 SLES-64BIT-SLOT1 osafimmnd[3626]: NO Global discard node
received for nodeId:2030f pid:10165
Mar 31 16:29:19 SLES-64BIT-SLOT1 osafimmnd[3626]: NO Implementer (applier)
connected: 114 (@OpenSafImmReplicatorB) <0, 2020f>
Mar 31 16:29:20 SLES-64BIT-SLOT1 osafrded[3597]: NO RDE role set to QUIESCED
Mar 31 16:29:20 SLES-64BIT-SLOT1 osafimmnd[3626]: NO Implementer connected: 115
(MsgQueueService131855) <0, 2020f>
Mar 31 16:29:20 SLES-64BIT-SLOT1 osafimmnd[3626]: NO Implementer disconnected
115 <0, 2020f> (MsgQueueService131855)
Mar 31 16:29:21 SLES-64BIT-SLOT1 osafimmnd[3626]: NO Implementer disconnected
105 <11, 2010f> (safAmfService)
Mar 31 16:29:21 SLES-64BIT-SLOT1 osafimmnd[3626]: NO Implementer (applier)
connected: 116 (@safAmfService2010f) <11, 2010f>
Mar 31 16:29:23 SLES-64BIT-SLOT1 kernel: [ 871.028080] TIPC: Resetting link
<1.1.1:eth0-1.1.3:eth0>, peer not responding
Mar 31 16:29:23 SLES-64BIT-SLOT1 kernel: [ 871.028087] TIPC: Lost link
<1.1.1:eth0-1.1.3:eth0> on network plane A
Mar 31 16:29:23 SLES-64BIT-SLOT1 kernel: [ 871.028093] TIPC: Lost contact with
<1.1.3>
Mar 31 16:29:23 SLES-64BIT-SLOT1 osafimmnd[3626]: NO Implementer disconnected
104 <0, 2020f> (@safAmfService2020f)
Mar 31 16:29:23 SLES-64BIT-SLOT1 osafimmnd[3626]: NO Implementer connected: 117
(safAmfService) <0, 2020f>
Mar 31 16:29:23 SLES-64BIT-SLOT1 osafamfd[3682]: NO Switching Quiesced -->
StandBy
Mar 31 16:29:23 SLES-64BIT-SLOT1 osafrded[3597]: NO RDE role set to STANDBY
Mar 31 16:29:23 SLES-64BIT-SLOT1 osafamfd[3682]: NO Controller switch over done
syslog of SC-2 which became Active:
Mar 31 16:29:03 SLES-64BIT-SLOT2 osafimmnd[3028]: NO Implementer disconnected
105 <0, 2010f> (safAmfService)
Mar 31 16:29:03 SLES-64BIT-SLOT2 osafimmnd[3028]: NO Implementer (applier)
connected: 116 (@safAmfService2010f) <0, 2010f>
Mar 31 16:29:05 SLES-64BIT-SLOT2 kernel: [ 1181.568068] TIPC: Resetting link
<1.1.2:eth0-1.1.3:eth0>, peer not responding
Mar 31 16:29:05 SLES-64BIT-SLOT2 kernel: [ 1181.568076] TIPC: Lost link
<1.1.2:eth0-1.1.3:eth0> on network plane A
Mar 31 16:29:05 SLES-64BIT-SLOT2 kernel: [ 1181.568112] TIPC: Lost contact with
<1.1.3>
Mar 31 16:29:05 SLES-64BIT-SLOT2 osafamfd[2449]: NO Switching StandBy -->
Active State
Mar 31 16:29:05 SLES-64BIT-SLOT2 osafimmnd[3028]: NO Implementer disconnected
104 <324, 2020f> (@safAmfService2020f)
Mar 31 16:29:05 SLES-64BIT-SLOT2 osafimmnd[3028]: NO Implementer connected: 117
(safAmfService) <324, 2020f>
Mar 31 16:29:05 SLES-64BIT-SLOT2 osafrded[2364]: NO RDE role set to ACTIVE
Mar 31 16:29:05 SLES-64BIT-SLOT2 osafclmd[2430]: NO ACTIVE request
Mar 31 16:29:05 SLES-64BIT-SLOT2 osafamfd[2449]: NO Controller switch over done
Mar 31 16:29:05 SLES-64BIT-SLOT2 osafamfd[2449]: NO Node 'PL-3' left the cluster
Another switchover is triggered and the payload is brought up during the
switchover.
Mar 31 16:29:52 SLES-64BIT-SLOT2 osafamfd[2449]: NO safSi=SC-2N,safApp=OpenSAF
Swap initiated
Mar 31 16:29:52 SLES-64BIT-SLOT2 osafamfnd[2459]: NO Assigning
'safSi=SC-2N,safApp=OpenSAF' QUIESCED to 'safSu=SC-2,safSg=2N,safApp=OpenSAF'
Mar 31 16:29:52 SLES-64BIT-SLOT2 osafimmnd[3028]: NO Implementer locally
disconnected. Marking it as doomed 112 <741, 2020f> (safSmfService)
Mar 31 16:29:52 SLES-64BIT-SLOT2 osafimmnd[3028]: NO Implementer disconnected
106 <310, 2020f> (safMsgGrpService)
Mar 31 16:29:52 SLES-64BIT-SLOT2 osafimmnd[3028]: NO Implementer disconnected
112 <741, 2020f> (safSmfService)
Mar 31 16:29:52 SLES-64BIT-SLOT2 osafimmnd[3028]: NO Implementer disconnected
107 <3, 2020f> (safLogService)
Mar 31 16:29:52 SLES-64BIT-SLOT2 osafimmnd[3028]: NO Implementer disconnected
110 <307, 2020f> (safEvtService)
Mar 31 16:29:53 SLES-64BIT-SLOT2 osafimmnd[3028]: NO Implementer disconnected
111 <322, 2020f> (safClmService)
Mar 31 16:29:53 SLES-64BIT-SLOT2 osafimmnd[3028]: NO Implementer disconnected
109 <308, 2020f> (safLckService)
Mar 31 16:29:53 SLES-64BIT-SLOT2 osafimmnd[3028]: NO Implementer disconnected
108 <306, 2020f> (safCheckPointService)
Mar 31 16:29:53 SLES-64BIT-SLOT2 kernel: [ 1229.947777] TIPC: Established link
<1.1.2:eth0-1.1.3:eth0> on network plane A
Mar 31 16:29:54 SLES-64BIT-SLOT2 osafamfnd[2459]: NO Assigned
'safSi=SC-2N,safApp=OpenSAF' QUIESCED to 'safSu=SC-2,safSg=2N,safApp=OpenSAF'
Mar 31 16:29:54 SLES-64BIT-SLOT2 osafimmnd[3028]: NO Implementer disconnected
113 <0, 2010f> (@OpenSafImmReplicatorA)
syslog on the new Active:
Mar 31 16:30:11 SLES-64BIT-SLOT1 kernel: [ 919.503907] TIPC: Established link
<1.1.1:eth0-1.1.3:eth0> on network plane A
Mar 31 16:30:12 SLES-64BIT-SLOT1 osafamfnd[3692]: NO Assigning
'safSi=SC-2N,safApp=OpenSAF' ACTIVE to 'safSu=SC-1,safSg=2N,safApp=OpenSAF'
Mar 31 16:30:12 SLES-64BIT-SLOT1 osafntfimcnd[3918]: NO exiting on signal 15
Mar 31 16:30:12 SLES-64BIT-SLOT1 osafimmd[3616]: WA IMMD not re-electing coord
for switch-over (si-swap) coord at (2020f)
Mar 31 16:30:12 SLES-64BIT-SLOT1 osafimmnd[3626]: NO Implementer disconnected
113 <332, 2010f> (@OpenSafImmReplicatorA)
Mar 31 16:30:12 SLES-64BIT-SLOT1 osafimmnd[3626]: NO Implementer connected: 118
(safMsgGrpService) <304, 2010f>
......
Mar 31 16:30:13 SLES-64BIT-SLOT1 osafrded[3597]: NO RDE role set to ACTIVE
Mar 31 16:30:13 SLES-64BIT-SLOT1 osafclmd[3663]: NO ACTIVE request
Mar 31 16:30:13 SLES-64BIT-SLOT1 osafamfd[3682]: NO Controller switch over done
Mar 31 16:30:13 SLES-64BIT-SLOT1 osafamfd[3682]: NO Node 'PL-3' left the cluster
Mar 31 16:30:26 SLES-64BIT-SLOT1 osafimmnd[3626]: NO NODE STATE->
IMM_NODE_FULLY_AVAILABLE 17221
Mar 31 16:30:26 SLES-64BIT-SLOT1 osafimmnd[3626]: NO Epoch set to 22 in ImmModel
Mar 31 16:30:26 SLES-64BIT-SLOT1 osafimmd[3616]: NO ACT: New Epoch for IMMND
process at node 2020f old epoch: 21 new epoch:22
Mar 31 16:30:26 SLES-64BIT-SLOT1 osafimmd[3616]: NO ACT: New Epoch for IMMND
process at node 2040f old epoch: 21 new epoch:22
Mar 31 16:30:26 SLES-64BIT-SLOT1 osafimmd[3616]: NO ACT: New Epoch for IMMND
process at node 2010f old epoch: 21 new epoch:22
Mar 31 16:30:26 SLES-64BIT-SLOT1 osafimmd[3616]: NO ACT: New Epoch for IMMND
process at node 2030f old epoch: 0 new epoch:22
Mar 31 16:30:26 SLES-64BIT-SLOT1 osafamfd[3682]: NO Node 'PL-3' joined the
cluster
A redundant PL-3 left the cluster message is seen at the new Active when the
payload is actually joining the cluster.
Multiple outcomes are seen when the test is repeated. It is observed once that
payload does not join the cluster in this scenario. AMFND on PL-3 reports
timeout. Traces can be shared if required. It is also observed once that
payload joins the cluster but opensafd status doesnot show payload as part of
the cluster. The operational state of the node is shown as disabled.
syslog and amf traces are attached.
---
Sent from sourceforge.net because opensaf-tickets@lists.sourceforge.net is
subscribed to https://sourceforge.net/p/opensaf/tickets/
To unsubscribe from further messages, a project admin can change settings at
https://sourceforge.net/p/opensaf/admin/tickets/options. Or, if this is a
mailing list, you can unsubscribe from the mailing list.
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Opensaf-tickets mailing list
Opensaf-tickets@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/opensaf-tickets
