I'll push this version then. Mathi. Sent from my Android phone using TouchDown (www.nitrodesk.com)
-----Original Message----- From: Alex Jones [[email protected]] Received: Tuesday, 17 Mar 2015, 7:02 To: Mathivanan Naickan Palanivelu [[email protected]] CC: [email protected] [[email protected]]; [email protected] [[email protected]] Subject: Re: [PATCH 1 of 1] amfd: fix interaction with plm/clm for node left [#1264] Hi Mathi, I removed the sleep and the exit, and replaced with break. It still works fine. Do you want me to resubmit? Or run another test? Alex On 03/16/2015 11:11 AM, Mathivanan Naickan Palanivelu wrote: > Alex, > > I had understood this far, but thanks for re-explaning. > The two things that is pinching me are: > 1) This sleep solution in AMF doesn't look good. > 2) Even if failover is triggerred by AMF within the clm callback, > things should be able to fall-inline up through the layers! May be > we are hitting some timing issue, that are still fixable when > the controller is the STANDBY. > We need to pursue this later. > > For now, i suggest to skip the sleep in the patch and instead > just return from that case statement when if (node->node_info.nodeId == > avd_cb->node_id_avd). > This will enable the intended design to get hit, i.e. plmcd will invoke > /etc/init.d/opensafd stop, > thereby enabling the failover. (If failover still doesn't works, then we need > to go deep into the problem.) > > Let me know if that works for you. > > Thanks > Mathi. > > ----- [email protected] wrote: > >> Mathi, >> >> I am performing a PLM_ADMIN_RESET of the active controller. >> >> The problem is that amfd on the active controller is forcing a >> failover >> before the "node left" is received on the standby. This results in >> all >> the core OpenSAF services on the standby failing at active assignment >> >> because their saImmOiClassImplementerSet calls fail (because "node >> left" >> hasn't been seen yet). >> >> I can see in the CLM logs on the standby, that the "node left" is >> received from PLM immediately, but since CLM is in standby mode it >> ignores the message. In the mean time, amfd on the active controller >> >> initiates a failover. >> >> Alex >> >> On 03/16/2015 05:30 AM, Mathivanan Naickan Palanivelu wrote: >>> Alex, >>> >>> Just to be sure, please mention the exact PLM admin operation being >> carried out here. >>> I have some doubts here, and want to rule out whether this would be >> a hack in AMF because, plmcd should be invoking '/etc/init.d/opensafd >> stop' in this flow? And that(plmcd calling 'opensafd stop') should >> start the failover operations? >>> In worst case, the node reboot should anyway trigger the failover. >>> >>> Is the problem about "failover not happening during this xyz plm >> admin op"? >>> Thanks, >>> Mathi. >>> >>> >>> >>> ----- [email protected] wrote: >>> >>>> osaf/services/saf/amf/amfd/clm.cc | 8 ++++++++ >>>> 1 files changed, 8 insertions(+), 0 deletions(-) >>>> >>>> >>>> When active controller is hard reset, standby reboots, too, >> because >>>> the OpenSAF >>>> services all fail in active assignment because >>>> saImmOiClassImplementerSet fails. >>>> >>>> When amfd on the active controller receives "node left" for itself >>>> from CLM and >>>> the root cause entity is "safEE", it calls clm_node_exit_complete, >>>> which allows >>>> amfd to do an immediate switchover. This is not right, though, >> because >>>> this node >>>> has left the cluster. >>>> >>>> Add a check to amfd such that if the "node left" node is itself, >> just >>>> exit. We >>>> don't want to do a switch over, but let the "node down" handle the >>>> failover. >>>> >>>> diff --git a/osaf/services/saf/amf/amfd/clm.cc >>>> b/osaf/services/saf/amf/amfd/clm.cc >>>> --- a/osaf/services/saf/amf/amfd/clm.cc >>>> +++ b/osaf/services/saf/amf/amfd/clm.cc >>>> @@ -278,6 +278,14 @@ static void clm_track_cb(const SaClmClus >>>> if(strncmp((char >>>> *)rootCauseEntity->value, "safEE=", 6) == 0) >> { >>>> /* This callback is because of >>>> operation on PLM, so we need >> to >>>> mark the node >>>> absent, because PLCD will >>>> anyway call opensafd stop.*/ >>>> + AVD_AVND *node = >>>> avd_node_find_nodeid(notifItem->clusterNode.nodeId); >>>> + if (node->node_info.nodeId == >>>> avd_cb->node_id_avd) { >>>> + /* Self Termination, wait for >>>> getting terminated by >> opensafd >>>> stop script.*/ >>>> + LOG_NO("self termination: >>>> exiting"); >>>> + sleep(5); >>>> + exit(0); >>>> + } >>>> + >>>> >>>> clm_node_exit_complete(notifItem->clusterNode.nodeId); >>>> } else if (strncmp((char >>>> *)rootCauseEntity->value, >> "safNode=", 8) >>>> == 0) { >>>> /* This callback is because of >>>> operation on CLM.*/ ------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Opensaf-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/opensaf-devel
