On 30-Jun-16 6:08 PM, Kang-Sen Lu wrote: > Hi, Praveen: > > Thanks for your response. > > You asked "... for the configured value of saAmfSIPrefActiveAssignments for > each SI". I am sure it is explicitly set to 1 by our application code. It is > also shown in the immlist output for Sis in the original post. > > I have done some experiment by first configure all 3 Sus. Then configure > SI-1. And I saw only one of the SU was assigned with the SI-1. Then I > configured SI-2, and only 1 SU was assigned to SI-2. > > So I think in normal conditions, the amf is doing the correct thing. > > I suspect when nodes are crashing and restarting, may be the amf database was > not sync'ed up correctly. Because the immfind and immlist was showing > inconsistency, as far as PL-4 is assigned or not. > > Unfortunately, the safLog did not show anything useful. I also lost the > syslog because we did not detect this problem quick enough. > > I was hoping you have some insight about how could immfind and immlist > display inconsistent fact. I think immlist is displaying the current amf > database. But immfind is also displaying amf database. > So before crash, situation is like this: Model: N-Way Active,2 SIs: SI1 and SI2, 3 SUs: SU1 on Node1, SU2 on Node2 and SU3 on Node3. SU1 assigned SI1 on Node1, SU2 assigned SI2 on Node2 and SU3 spare is not assigned on Node3. Now suppose Node1 crashes/restarts then AMF will assigned SI1 on spare Su3 and assignments, thus, will look like this in ideal case: SU1 spare (after Node1 comes up), SU2 is assigned SI2 on Node2(no change) and SU3 is assigned SI1( fresh assignment after Node1 lefts).
But I think what is being observed is SU1 assigned to SI1 (old assignment) is still listed by immlist/immfind. If this is the case Could you please check that after restart/reboot of Node1, SU1 was instantiated and was given callbacks for SI1? If after fresh assignment of SI1 in SU3, AMF is still assigning it to SU1 on Node1 then it is a bug and a ticket can be written for it. But we have had issues in the past when assignments were not present in AMF database but still shown by IMM. This could be because controller failover/switchover happens after Node1 restart and before AMF deletes assignments from IMM. Is the node that is restarting/crashing is a controller node? or If it is a payload node, is the crash of this node is followed by controller role change? Thanks, Praveen > Thanks again. > > Kang-sen > > -----Original Message----- > From: praveen malviya [mailto:[email protected]] > Sent: Thursday, June 30, 2016 2:02 AM > To: Kang-Sen Lu <[email protected]>; [email protected] > Subject: Re: [users] question about 'N WAY ACTIVE' redundancy model > > Please find response below with [Praveen]. > > Thanks, > Praveen > > On 29-Jun-16 8:04 PM, Kang-Sen Lu wrote: >> We are running opensaf 4.4.0. >> >> We have a service group containing 2 SIs (SI1, SI2), and 3 SUs (PL4, PL6, >> PL8). >> >> We are selecting N WAY ACTIVE redundancy model, and expected 1 SI assigned >> to 1 SU, with the 3rd SU as spare. >> >> >From immfind output, we saw PL4 and PL6 are assigned with SI2, PL8 was >> >assigned with SI2. >> >> >From immlist output, we saw PL4 was not assigned with any SI. > [Praveen] PL4 is assigned saAmfSGMaxActiveSIsperSU as per immfind output > below. > > Anyways, primary reason could be that the configured value of > saAmfSGMaxActiveSIsperSU=1 in SG. Since all the SUs got one SI assigned, > because of this criteria now there will be no more assignments. So in this > case SI1 has only one assignments in SG. > > But still I have a doubt, why SI2 got two assignments when in the below > mentioned "immlist safSi=loadBalancerSI-2,safApp=loadBalancerApp" > output, value of saAmfSIPrefActiveAssignments is 1. > Could you please check the configuration for the configured value of > saAmfSIPrefActiveAssignments for each SI. > > > >> >> That is not consistent. >> >> Any idea why this happened? >> >> Here is the immfind output: >> >> root@BHA-IND-WHF-KK-CAE-5:~# immfind | grep -i "safCSIComp=safComp=Lbm_PL-" >> safCSIComp=safComp=Lbm_PL-4\,safSu=loadBalancerSU_PL-4\,safSg=loadBala >> ncerSG\,safApp=loadBalancerApp,safCsi=LbmCSI_SI-2,safSi=loadBalancerSI >> -2,safApp=loadBalancerApp >> safCSIComp=safComp=Lbm_PL-6\,safSu=loadBalancerSU_PL-6\,safSg=loadBala >> ncerSG\,safApp=loadBalancerApp,safCsi=LbmCSI_SI-2,safSi=loadBalancerSI >> -2,safApp=loadBalancerApp >> safCSIComp=safComp=Lbm_PL-8\,safSu=loadBalancerSU_PL-8\,safSg=loadBala >> ncerSG\,safApp=loadBalancerApp,safCsi=LbmCSI_SI-1,safSi=loadBalancerSI >> -1,safApp=loadBalancerApp >> root@BHA-IND-WHF-KK-CAE-5:~# >> >> Here is the immlist output: >> >> root@BHA-IND-WHF-KK-CAE-5:~# immlist >> safSu=loadBalancerSU_PL-4,safSg=loadBalancerSG,safApp=loadBalancerApp >> Name Type Value(s) >> ======================================================================== >> safSu SA_STRING_T >> safSu=loadBalancerSU_PL-4 >> saAmfSUType SA_NAME_T >> safVersion=4.0.0,safSuType=MovikSUType (38) >> saAmfSURestartCount SA_UINT32_T 0 (0x0) >> saAmfSUReadinessState SA_UINT32_T 2 (0x2) >> saAmfSURank SA_UINT32_T 1 (0x1) >> saAmfSUPresenceState SA_UINT32_T 3 (0x3) >> saAmfSUPreInstantiable SA_UINT32_T 1 (0x1) >> saAmfSUOperState SA_UINT32_T 1 (0x1) >> saAmfSUNumCurrStandbySIs SA_UINT32_T 0 (0x0) >> saAmfSUNumCurrActiveSIs SA_UINT32_T 0 (0x0) >> saAmfSUMaintenanceCampaign SA_NAME_T <Empty> >> saAmfSUHostedByNode SA_NAME_T >> safAmfNode=PL-4,safAmfCluster=myAmfCluster (42) >> saAmfSUHostNodeOrNodeGroup SA_NAME_T >> safAmfNode=PL-4,safAmfCluster=myAmfCluster (42) >> saAmfSUFailover SA_UINT32_T <Empty> >> saAmfSUAssignedSIs SA_NAME_T <Empty> >> saAmfSUAdminState SA_UINT32_T 1 (0x1) >> SaImmAttrImplementerName SA_STRING_T safAmfService >> SaImmAttrClassName SA_STRING_T SaAmfSU >> SaImmAttrAdminOwnerName SA_STRING_T <Empty> >> MovikServiceMode SA_STRING_T <Empty> >> >> root@BHA-IND-WHF-KK-CAE-5:~# immlist >> safSu=loadBalancerSU_PL-6,safSg=loadBalancerSG,safApp=loadBalancerApp >> Name Type Value(s) >> ======================================================================== >> safSu SA_STRING_T >> safSu=loadBalancerSU_PL-6 >> saAmfSUType SA_NAME_T >> safVersion=4.0.0,safSuType=MovikSUType (38) >> saAmfSURestartCount SA_UINT32_T 0 (0x0) >> saAmfSUReadinessState SA_UINT32_T 2 (0x2) >> saAmfSURank SA_UINT32_T 1 (0x1) >> saAmfSUPresenceState SA_UINT32_T 3 (0x3) >> saAmfSUPreInstantiable SA_UINT32_T 1 (0x1) >> saAmfSUOperState SA_UINT32_T 1 (0x1) >> saAmfSUNumCurrStandbySIs SA_UINT32_T 0 (0x0) >> saAmfSUNumCurrActiveSIs SA_UINT32_T 1 (0x1) >> saAmfSUMaintenanceCampaign SA_NAME_T <Empty> >> saAmfSUHostedByNode SA_NAME_T >> safAmfNode=PL-6,safAmfCluster=myAmfCluster (42) >> saAmfSUHostNodeOrNodeGroup SA_NAME_T >> safAmfNode=PL-6,safAmfCluster=myAmfCluster (42) >> saAmfSUFailover SA_UINT32_T <Empty> >> saAmfSUAssignedSIs SA_NAME_T <Empty> >> saAmfSUAdminState SA_UINT32_T 1 (0x1) >> SaImmAttrImplementerName SA_STRING_T safAmfService >> SaImmAttrClassName SA_STRING_T SaAmfSU >> SaImmAttrAdminOwnerName SA_STRING_T <Empty> >> MovikServiceMode SA_STRING_T <Empty> >> >> root@BHA-IND-WHF-KK-CAE-5:~# immlist >> safSu=loadBalancerSU_PL-8,safSg=loadBalancerSG,safApp=loadBalancerApp >> Name Type Value(s) >> ======================================================================== >> safSu SA_STRING_T >> safSu=loadBalancerSU_PL-8 >> saAmfSUType SA_NAME_T >> safVersion=4.0.0,safSuType=MovikSUType (38) >> saAmfSURestartCount SA_UINT32_T 0 (0x0) >> saAmfSUReadinessState SA_UINT32_T 2 (0x2) >> saAmfSURank SA_UINT32_T 1 (0x1) >> saAmfSUPresenceState SA_UINT32_T 3 (0x3) >> saAmfSUPreInstantiable SA_UINT32_T 1 (0x1) >> saAmfSUOperState SA_UINT32_T 1 (0x1) >> saAmfSUNumCurrStandbySIs SA_UINT32_T 0 (0x0) >> saAmfSUNumCurrActiveSIs SA_UINT32_T 1 (0x1) >> saAmfSUMaintenanceCampaign SA_NAME_T <Empty> >> saAmfSUHostedByNode SA_NAME_T >> safAmfNode=PL-8,safAmfCluster=myAmfCluster (42) >> saAmfSUHostNodeOrNodeGroup SA_NAME_T >> safAmfNode=PL-8,safAmfCluster=myAmfCluster (42) >> saAmfSUFailover SA_UINT32_T <Empty> >> saAmfSUAssignedSIs SA_NAME_T <Empty> >> saAmfSUAdminState SA_UINT32_T 1 (0x1) >> SaImmAttrImplementerName SA_STRING_T safAmfService >> SaImmAttrClassName SA_STRING_T SaAmfSU >> SaImmAttrAdminOwnerName SA_STRING_T <Empty> >> MovikServiceMode SA_STRING_T <Empty> >> >> root@BHA-IND-WHF-KK-CAE-5:~# >> >> >> >> root@BHA-IND-WHF-KK-CAE-5:~# immlist >> safSi=loadBalancerSI-1,safApp=loadBalancerApp >> Name Type Value(s) >> ======================================================================== >> safSi SA_STRING_T >> safSi=loadBalancerSI-1 >> saAmfSvcType SA_NAME_T >> safVersion=4.0.0,safSvcType=MovikSvcType (40) >> saAmfSIStandbyWeight SA_STRING_T <Empty> >> saAmfSIRank SA_UINT32_T 0 (0x0) >> saAmfSIProtectedbySG SA_NAME_T >> safSg=loadBalancerSG,safApp=loadBalancerApp (43) >> saAmfSIPrefStandbyAssignments SA_UINT32_T 1 (0x1) >> saAmfSIPrefActiveAssignments SA_UINT32_T 1 (0x1) >> saAmfSINumCurrStandbyAssignments SA_UINT32_T 0 (0x0) >> saAmfSINumCurrActiveAssignments SA_UINT32_T 1 (0x1) >> saAmfSIAssignmentState SA_UINT32_T 2 (0x2) >> saAmfSIAdminState SA_UINT32_T 1 (0x1) >> saAmfSIActiveWeight SA_STRING_T <Empty> >> SaImmAttrImplementerName SA_STRING_T safAmfService >> SaImmAttrClassName SA_STRING_T SaAmfSI >> SaImmAttrAdminOwnerName SA_STRING_T <Empty> >> >> root@BHA-IND-WHF-KK-CAE-5:~# immlist >> safSi=loadBalancerSI-2,safApp=loadBalancerApp >> Name Type Value(s) >> ======================================================================== >> safSi SA_STRING_T >> safSi=loadBalancerSI-2 >> saAmfSvcType SA_NAME_T >> safVersion=4.0.0,safSvcType=MovikSvcType (40) >> saAmfSIStandbyWeight SA_STRING_T <Empty> >> saAmfSIRank SA_UINT32_T 0 (0x0) >> saAmfSIProtectedbySG SA_NAME_T >> safSg=loadBalancerSG,safApp=loadBalancerApp (43) >> saAmfSIPrefStandbyAssignments SA_UINT32_T 1 (0x1) >> saAmfSIPrefActiveAssignments SA_UINT32_T 1 (0x1) >> saAmfSINumCurrStandbyAssignments SA_UINT32_T 0 (0x0) >> saAmfSINumCurrActiveAssignments SA_UINT32_T 1 (0x1) >> saAmfSIAssignmentState SA_UINT32_T 2 (0x2) >> saAmfSIAdminState SA_UINT32_T 1 (0x1) >> saAmfSIActiveWeight SA_STRING_T <Empty> >> SaImmAttrImplementerName SA_STRING_T safAmfService >> SaImmAttrClassName SA_STRING_T SaAmfSI >> SaImmAttrAdminOwnerName SA_STRING_T <Empty> >> >> root@BHA-IND-WHF-KK-CAE-5:~# >> >> root@BHA-IND-WHF-KK-CAE-5:~# immlist >> safSg=loadBalancerSG,safApp=loadBalancerApp >> Name Type Value(s) >> ======================================================================== >> safSg SA_STRING_T >> safSg=loadBalancerSG >> saAmfSGType SA_NAME_T >> safVersion=4.0.0,safSgType=MovikSGTypeNWayActive (48) >> saAmfSGSuRestartProb SA_TIME_T <Empty> >> saAmfSGSuRestartMax SA_UINT32_T <Empty> >> saAmfSGSuHostNodeGroup SA_NAME_T <Empty> >> saAmfSGNumPrefStandbySUs SA_UINT32_T 0 (0x0) >> saAmfSGNumPrefInserviceSUs SA_UINT32_T 100 (0x64) >> saAmfSGNumPrefAssignedSUs SA_UINT32_T 100 (0x64) >> saAmfSGNumPrefActiveSUs SA_UINT32_T 100 (0x64) >> saAmfSGNumCurrNonInstantiatedSpareSUs SA_UINT32_T 0 (0x0) >> saAmfSGNumCurrInstantiatedSpareSUs SA_UINT32_T 1 (0x1) >> saAmfSGNumCurrAssignedSUs SA_UINT32_T 2 (0x2) >> saAmfSGMaxStandbySIsperSU SA_UINT32_T <Empty> >> saAmfSGMaxActiveSIsperSU SA_UINT32_T 1 (0x1) >> saAmfSGCompRestartProb SA_TIME_T <Empty> >> saAmfSGCompRestartMax SA_UINT32_T <Empty> >> saAmfSGAutoRepair SA_UINT32_T 1 (0x1) >> saAmfSGAutoAdjustProb SA_TIME_T <Empty> >> saAmfSGAutoAdjust SA_UINT32_T 0 (0x0) >> saAmfSGAdminState SA_UINT32_T 1 (0x1) >> SaImmAttrImplementerName SA_STRING_T safAmfService >> SaImmAttrClassName SA_STRING_T SaAmfSG >> SaImmAttrAdminOwnerName SA_STRING_T <Empty> >> >> Thanks. >> >> Kang-sen >> >> ---------------------------------------------------------------------- >> -------- Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T >> Park in San Francisco, CA to explore cutting-edge tech and listen to >> tech luminaries present their vision of the future. This family event >> has something for everyone, including kids. Get more information and >> register today. >> http://sdm.link/attshape >> _______________________________________________ >> Opensaf-users mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/opensaf-users >> ------------------------------------------------------------------------------ Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San Francisco, CA to explore cutting-edge tech and listen to tech luminaries present their vision of the future. This family event has something for everyone, including kids. Get more information and register today. http://sdm.link/attshape _______________________________________________ Opensaf-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/opensaf-users
