Hi, comments in-line.
/Neel. On Monday 04 May 2015 05:33 PM, Yao Cheng LIANG wrote: > > The error was cause by line below as msq service register with immnd. > Please see this line in immnd log > > Jan 1 8:06:49.884694 osafimmnd [936:immnd_evt.c:0729] WA > immnd_evt_proc_imm_init: PID 0 (1012) for 2040f000003f4, MDS problem? > This happens, when the own node receives node down. when there is link loss or link toggling happened, SC-1 or active controller detected link loss and sends immnd down message, by the time time message is sent the link has established again and the PL-3 received the message. Verify, the link loss messages in active controller syslog messages. > *From:*Neelakanta Reddy [mailto:[email protected]] > *Sent:* Monday, May 04, 2015 6:07 PM > *To:* Yao Cheng LIANG; [email protected] > *Subject:* Re: [users] Fw: log > > Hi, > > Please,share the syslog and osafimmnd traces available at > /var/log/opensaf on all the nodes. > > To enable immnd traces uncomment the below line in all the nodes of > the cluster at /etc/opensaf/immnd.conf: > > # Uncomment the next line to enable trace > args="--tracemask=0xffffffff" > > /Neel. > > On Monday 04 May 2015 03:20 PM, Yao Cheng LIANG wrote: > > Nope. All nodes upgraded to 4.6. /Ted > > Sent from Samsung Mobile > > > > -------- Original message -------- > From: Neelakanta Reddy > Date:2015/05/04 5:03 PM (GMT+08:00) > To: [email protected] > <mailto:[email protected]> > Subject: Re: [users] Fw: log > > Hi, > > Two controllers are upgraded to 4.6 and payloads are still have > 4.2 . > In general upgrade the nodes must be rolling upgrade. > If some nodes are upgraded manually, and some nodes are still in > older > releases then IMM flags need to be toggled accordingly. > > comments below. > > /Neel. > > On Monday 04 May 2015 04:55 AM, Yao Cheng LIANG wrote: > > Dear all, > > > > I recently upgraded my opensaf from 4.2.2 to u4.6.0 for > checkpoint service performance improvement. I have successfully > started on both controllers but I can not do same thing on the > payload. From the log below from playload node, seems Opensaf > started successfully but later on shut itself down for some > errors. I am using imm.xml for my 4.2.2 version. May anyone help? > > > > Thanks. > > > > Ted > > > > Jan 1 08:08:11 (none) user.notice opensafd: Starting OpenSAF > Services (Using TCP) > > Jan 1 08:08:11 (none) local0.notice osafdtmd[914]: Started > > Jan 1 08:08:11 (none) local0.notice osafimmnd[931]: Started > > Jan 1 08:08:11 (none) local0.notice osafdtmd[914]: NO > Established contact with 'WR20-64_32' > > Jan 1 08:08:11 (none) local0.notice osafdtmd[914]: NO > Established contact with 'WR20-64_25' > > Jan 1 08:08:11 (none) local0.notice osafimmnd[931]: NO SERVER > STATE: IMM_SERVER_ANONYMOUS --> IMM_SERVER_CLUSTER_WAITING > > Jan 1 08:08:11 (none) local0.notice osafimmnd[931]: NO SERVER > STATE: IMM_SERVER_CLUSTER_WAITING --> IMM_SERVER_LOADING_PENDING > > Jan 1 08:08:11 (none) local0.notice osafimmnd[931]: NO SERVER > STATE: IMM_SERVER_LOADING_PENDING --> IMM_SERVER_SYNC_PENDING > > Jan 1 08:08:11 (none) local0.notice osafimmnd[931]: NO NODE > STATE-> IMM_NODE_ISOLATED > > Jan 1 08:08:12 (none) local0.notice osafimmnd[931]: NO NODE > STATE-> IMM_NODE_W_AVAILABLE > > Jan 1 08:08:12 (none) local0.notice osafimmnd[931]: NO SERVER > STATE: IMM_SERVER_SYNC_PENDING --> IMM_SERVER_SYNC_CLIENT > > Jan 1 08:08:12 (none) local0.notice osafimmnd[931]: NO NODE > STATE-> IMM_NODE_FULLY_AVAILABLE 2578 > > Jan 1 08:08:12 (none) local0.notice osafimmnd[931]: NO > RepositoryInitModeT is SA_IMM_INIT_FROM_FILE > > Jan 1 08:08:12 (none) local0.warn osafimmnd[931]: WA IMM Access > Control mode is DISABLED! > > Jan 1 08:08:12 (none) local0.notice osafimmnd[931]: NO Epoch > set to 18 in ImmModel > > Jan 1 08:08:12 (none) local0.notice osafimmnd[931]: NO SERVER > STATE: IMM_SERVER_SYNC_CLIENT --> IMM SERVER READY > > Jan 1 08:08:12 (none) local0.notice osafclmna[943]: Started > > Jan 1 08:08:12 (none) local0.notice osafclmna[943]: NO > safNode=PL-4,safCluster=myClmCluster Joined cluster, nodeid=a040f > > Jan 1 08:08:13 (none) local0.notice osafamfnd[953]: Started > > Jan 1 08:08:13 (none) local0.notice osafamfnd[953]: NO > 'safSu=PL-4,safSg=NoRed,safApp=OpenSAF' Presence State > UNINSTANTIATED => INSTANTIATING > > Jan 1 08:08:13 (none) local0.notice osafsmfnd[964]: Started > > Jan 1 08:08:13 (none) local0.notice osafmsgnd[974]: Started > > Jan 1 08:08:13 (none) local0.notice osafimmnd[931]: NO > Implementer connected: 38 (MsgQueueService656399) <51, a040f> > IMMND asserted and restarted again, this is because of some > information > added in 4.6, which may not be compatible with older releases. > since the cluster is mixed version. > > Please go through osaf/services/saf/immsv/README( particularly > Notes on > upgrading from OpenSAF 4.[1,2,3,4,5] to OpenSAF (4.6)). > > once the cluster are upgraded, the flags mentioned needs to be > toggled on. > > > > Jan 1 08:08:13 (none) local0.notice osafimmnd[986]: Started > > Jan 1 08:08:13 (none) local0.notice osafimmnd[986]: NO Fevs > count adjusted to 5871 preLoadPid: 0 > > Jan 1 08:08:13 (none) local0.notice osaflcknd[997]: Started > > Jan 1 08:08:13 (none) local0.notice osafckptnd[1007]: Started > > Jan 1 08:08:13 (none) local0.notice osafimmnd[986]: NO SERVER > STATE: IMM_SERVER_ANONYMOUS --> IMM_SERVER_CLUSTER_WAITING > > Jan 1 08:08:13 (none) local0.notice osafamfwd[1018]: Started > > Jan 1 08:08:13 (none) local0.notice osafamfnd[953]: NO > 'safSu=PL-4,safSg=NoRed,safApp=OpenSAF' Presence State > INSTANTIATING => INSTANTIATED > > Jan 1 08:08:13 (none) local0.notice osafamfnd[953]: NO > Assigning 'safSi=NoRed10,safApp=OpenSAF' ACTIVE to > 'safSu=PL-4,safSg=NoRed,safApp=OpenSAF' > > Jan 1 08:08:13 (none) local0.notice osafamfnd[953]: NO Assigned > 'safSi=NoRed10,safApp=OpenSAF' ACTIVE to > 'safSu=PL-4,safSg=NoRed,safApp=OpenSAF' > > Jan 1 08:08:13 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:13 (none) user.notice opensafd: OpenSAF(4.6.0 - > 6467:3561f9d06464) services successfully started > > Jan 1 08:08:13 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:14 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:14 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:15 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:15 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:15 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:16 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:16 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:17 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:17 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:17 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:18 (none) local0.warn osafimmnd[986]: WA Resending > introduce-me - problems with MDS ? 5 > > Jan 1 08:08:18 (none) local0.warn osafimmnd[986]: WA Resending > introduce-me - problems with MDS ? 5 > > Jan 1 08:08:18 (none) local0.warn osafimmnd[986]: WA Resending > introduce-me - problems with MDS ? 5 > > Jan 1 08:08:18 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:18 (none) local0.warn osafimmnd[986]: WA Resending > introduce-me - problems with MDS ? 5 > > Jan 1 08:08:18 (none) local0.warn osafimmnd[986]: WA Resending > introduce-me - problems with MDS ? 5 > > Jan 1 08:08:18 (none) local0.warn osafimmnd[986]: WA Resending > introduce-me - problems with MDS ? 5 > > Jan 1 08:08:18 (none) local0.warn osafimmnd[986]: WA Resending > introduce-me - problems with MDS ? 5 > > Jan 1 08:08:18 (none) local0.warn osafimmnd[986]: WA Resending > introduce-me - problems with MDS ? 5 > > Jan 1 08:08:18 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:18 (none) local0.warn osafimmnd[986]: WA Resending > introduce-me - problems with MDS ? 5 > > Jan 1 08:08:18 (none) local0.warn osafimmnd[986]: WA Resending > introduce-me - problems with MDS ? 5 > > Jan 1 08:08:19 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:19 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:20 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:20 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:20 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:21 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:21 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:22 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:22 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:22 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:23 (none) local0.warn osafimmnd[986]: WA Resending > introduce-me - problems with MDS ? 10 > > Jan 1 08:08:23 (none) local0.warn osafimmnd[986]: WA Resending > introduce-me - problems with MDS ? 10 > > Jan 1 08:08:23 (none) local0.notice osafimmnd[986]: NO > mds_register_callback: dest a040f000003b9 already exist > > Jan 1 08:08:23 (none) local0.err osafamfnd[953]: > saImmOmInitialize FAILED, rc = 6 > > Jan 1 08:08:23 (none) local0.alert osafimmnd[986]: AL AMF Node > Director is down, terminate this process > > Jan 1 08:08:23 (none) local0.alert osaflcknd[997]: AL AMF Node > Director is down, terminate this process > > Jan 1 08:08:23 (none) local0.crit osafamfwd[1018]: Rebooting > OpenSAF NodeId = 0 EE Name = No EE Mapped, Reason: AMF > unexpectedly crashed, OwnNodeId = 656399, SupervisionTime = 60 > > Jan 1 08:08:23 (none) local0.notice osaflcknd[997]: exiting for > shutdown > > Jan 1 08:08:23 (none) local0.alert osafsmfnd[964]: AL AMF Node > Director is down, terminate this process > > Jan 1 08:08:23 (none) local0.alert osafckptnd[1007]: AL AMF > Node Director is down, terminate this process > > Jan 1 08:08:23 (none) local0.notice osafsmfnd[964]: exiting for > shutdown > > Jan 1 08:08:23 (none) local0.alert osafmsgnd[974]: AL AMF Node > Director is down, terminate this process > > Jan 1 08:08:23 (none) local0.notice osafckptnd[1007]: exiting > for shutdown > > Jan 1 08:08:23 (none) local0.notice osafmsgnd[974]: exiting for > shutdown > > Jan 1 08:08:23 (none) local0.notice osafimmnd[986]: exiting for > shutdown > > Jan 1 08:08:23 (none) local0.notice osafimmnd[931]: NO > Implementer locally disconnected. Marking it as doomed 38 <51, > a040f> (MsgQueueService656399) > > Jan 1 08:08:23 (none) local0.err osafimmnd[931]: ER > immnd_evt_proc_discard_node for *this* node 656399 => Cluster > partitioned ("split brain") - exiting > > Jan 1 08:08:23 (none) user.notice opensaf_reboot: Rebooting > local node; timeout=60 > > > > > > ------------------------------------------------------------------------------ > > One dashboard for servers and applications across > Physical-Virtual-Cloud > > Widest out-of-the-box monitoring support with 50+ applications > > Performance metrics, stats and reports that give you Actionable > Insights > > Deep dive visibility with transaction tracing using APM Insight. > > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y > > _______________________________________________ > > Opensaf-users mailing list > > [email protected] > <mailto:[email protected]> > > https://lists.sourceforge.net/lists/listinfo/opensaf-users > > > > ------------------------------------------------------------------------------ > One dashboard for servers and applications across > Physical-Virtual-Cloud > Widest out-of-the-box monitoring support with 50+ applications > Performance metrics, stats and reports that give you Actionable > Insights > Deep dive visibility with transaction tracing using APM Insight. > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y > _______________________________________________ > Opensaf-users mailing list > [email protected] > <mailto:[email protected]> > https://lists.sourceforge.net/lists/listinfo/opensaf-users > ------------------------------------------------------------------------------ One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y _______________________________________________ Opensaf-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/opensaf-users
