- Attachments has changed:
Diff:
~~~~
--- old
+++ new
@@ -1,2 +1 @@
OpenSAF_AMF_PR_2929.odt (138.5 kB; application/vnd.oasis.opendocument.text)
-amfd.tgz (404.2 kB; application/x-compressed-tar)
~~~~
---
** [tickets:#2929] amfd: Too many assignments after split brain**
**Status:** fixed
**Milestone:** 5.19.01
**Created:** Wed Sep 19, 2018 12:01 PM UTC by Minh Hon Chau
**Last Updated:** Wed Jan 09, 2019 09:23 PM UTC
**Owner:** nobody
**Attachments:**
-
[OpenSAF_AMF_PR_2929.odt](https://sourceforge.net/p/opensaf/tickets/2929/attachment/OpenSAF_AMF_PR_2929.odt)
(138.5 kB; application/vnd.oasis.opendocument.text)
In ticket #2920, amfd has rebooted the nodes that have duplicated 2N
assignments after split-brain. However, if there are spare 2N SUs hosted on
each payloads, amfd does not reboot the nodes that have duplicated assignments
and those assignments are remained intact after split-brain
Configuration:
- start cluster with 2 controllers + 6 payloads
- SC absence is enabled
- 6 SUs hosted on 6 payloads respectively, i.e SU1 hosted on PL3, SU2 on PL4,
...
- SU1 and SU2 initially are given 2N Active/Standby assignments
- Split network to separate PL3 and PL4 apart from cluster. Since SC absence is
enabled, PL3 and PL4 don't reboot
- Now SU3 (PL5) and SU4(PL6) are given 2N Active/Standby assignments
- Split network two partitions, [SC1, PL5, PL6] and [SC2, PL7, PL8]
- The second partition has SC2 becomes active, so the SUs in PL7 and PL8 are
given assignments.
- Restart SCs.
- After both SCs come back from reboot, there are totally 3 Active(s) and 3
Standby(s) assignments
Synced from headless
~~~
2018-09-19 19:00:29.857 PL-3 osafamfnd[193]: NO Synced
SU:safSu=1,safSg=1,safApp=osaftest <0, 1, 3>
2018-09-19 19:00:29.857 PL-3 osafamfnd[193]: NO Synced
SISU:safSi=NoRed4,safApp=OpenSAF,safSu=PL-3,safSg=NoRed,safApp=OpenSAF <1, 3>
2018-09-19 19:00:29.857 PL-4 osafamfnd[193]: NO Synced
SISU:safSi=A,safApp=osaftest,safSu=2,safSg=1,safApp=osaftest <2, 3>
2018-09-19 19:00:29.857 PL-4 osafamfnd[193]: NO Synced
SU:safSu=2,safSg=1,safApp=osaftest <0, 1, 3>
2018-09-19 21:46:23.318 PL-5 osafamfnd[193]: NO Synced
SISU:safSi=A,safApp=osaftest,safSu=3,safSg=1,safApp=osaftest <1, 3>
2018-09-19 21:46:23.319 PL-5 osafamfnd[193]: NO Synced
SU:safSu=3,safSg=1,safApp=osaftest <0, 1, 3>
2018-09-19 21:46:23.319 PL-6 osafamfnd[193]: NO Synced
SISU:safSi=A,safApp=osaftest,safSu=4,safSg=1,safApp=osaftest <2, 3>
2018-09-19 21:46:23.319 PL-6 osafamfnd[193]: NO Synced
SU:safSu=4,safSg=1,safApp=osaftest <0, 1, 3>
2018-09-19 21:46:23.318 PL-8 osafamfnd[193]: NO Synced
SISU:safSi=A,safApp=osaftest,safSu=6,safSg=1,safApp=osaftest <1, 3>
2018-09-19 21:46:23.318 PL-8 osafamfnd[193]: NO Synced
SU:safSu=6,safSg=1,safApp=osaftest <0, 1, 3>
2018-09-19 21:46:23.320 PL-7 osafamfnd[193]: NO Synced
SISU:safSi=A,safApp=osaftest,safSu=5,safSg=1,safApp=osaftest <2, 3>
2018-09-19 21:46:23.320 PL-7 osafamfnd[193]: NO Synced
SU:safSu=5,safSg=1,safApp=osaftest <0, 1, 3>
In avd_sg_2n_act_susi(), amfd always picks the first two assignments, which are
correctly active and standby, thus amfd didn't reboot the nodes
~~~
~~~
<143>1 2018-09-19T21:46:26.819976+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33675"] 262:amf/amfd/sg_2n_fsm.cc:677 >> avd_sg_2n_su_chose_asgn:
'safSg=1,safApp=osaftest'
<143>1 2018-09-19T21:46:26.819982+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33676"] 262:amf/amfd/si_dep.cc:711 >>
avd_sidep_update_si_dep_state_for_all_sis: 'safSg=1,safApp=osaftest'
<143>1 2018-09-19T21:46:26.819989+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33677"] 262:amf/amfd/si_dep.cc:718 <<
avd_sidep_update_si_dep_state_for_all_sis
<143>1 2018-09-19T21:46:26.819995+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33678"] 262:amf/amfd/sg_2n_fsm.cc:522 >> avd_sg_2n_act_susi:
'safSg=1,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820001+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33679"] 262:amf/amfd/sg_2n_fsm.cc:532 TR
si'safSi=A,safApp=osaftest', su'safSu=1,safSg=1,safApp=osaftest',
si'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820008+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33680"] 262:amf/amfd/sg_2n_fsm.cc:536 TR
si'safSi=A,safApp=osaftest', su'safSu=2,safSg=1,safApp=osaftest',
si'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820014+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33681"] 262:amf/amfd/sg_2n_fsm.cc:550 TR
su_1'safSu=1,safSg=1,safApp=osaftest', su_2'safSu=2,safSg=1,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820021+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33682"] 262:amf/amfd/sg_2n_fsm.cc:282 >> su_assigned_susi_find:
'safSu=1,safSg=1,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820027+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33683"] 262:amf/amfd/sg_2n_fsm.cc:288 TR Act
su'safSu=1,safSg=1,safApp=osaftest', si'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820033+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33684"] 262:amf/amfd/sg_2n_fsm.cc:303 TR act_found'1',
quisced_found'0', std_found'0'
<143>1 2018-09-19T21:46:26.82004+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33685"] 262:amf/amfd/sg_2n_fsm.cc:312 TR si'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820046+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33686"] 262:amf/amfd/sg_2n_fsm.cc:317 TR
su'safSu=1,safSg=1,safApp=osaftest', si'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820052+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33687"] 262:amf/amfd/sg_2n_fsm.cc:323 TR
su'safSu=2,safSg=1,safApp=osaftest', si'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820058+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33688"] 262:amf/amfd/sg_2n_fsm.cc:325 TR Act
su'safSu=1,safSg=1,safApp=osaftest', si'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820064+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33689"] 262:amf/amfd/sg_2n_fsm.cc:327 TR Std
su'safSu=2,safSg=1,safApp=osaftest', si'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820071+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33690"] 262:amf/amfd/sg_2n_fsm.cc:346 TR 3. Act
su'safSu=1,safSg=1,safApp=osaftest', si'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820077+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33691"] 262:amf/amfd/sg_2n_fsm.cc:348 TR 3. Std
su'safSu=2,safSg=1,safApp=osaftest', si'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820083+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33692"] 262:amf/amfd/sg_2n_fsm.cc:493 << su_assigned_susi_find: act
su: 'safSu=1,safSg=1,safApp=osaftest', stdby su:
'safSu=2,safSg=1,safApp=osaftest', si: 'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.82009+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33693"] 262:amf/amfd/sg_2n_fsm.cc:282 >> su_assigned_susi_find:
'safSu=2,safSg=1,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820097+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33694"] 262:amf/amfd/sg_2n_fsm.cc:297 TR Stdby
su'safSu=2,safSg=1,safApp=osaftest', si'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820103+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33695"] 262:amf/amfd/sg_2n_fsm.cc:303 TR act_found'0',
quisced_found'0', std_found'1'
<143>1 2018-09-19T21:46:26.820109+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33696"] 262:amf/amfd/sg_2n_fsm.cc:446 TR si'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820115+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33697"] 262:amf/amfd/sg_2n_fsm.cc:451 TR
su'safSu=1,safSg=1,safApp=osaftest', si'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820121+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33698"] 262:amf/amfd/sg_2n_fsm.cc:466 TR
su'safSu=2,safSg=1,safApp=osaftest', si'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820128+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33699"] 262:amf/amfd/sg_2n_fsm.cc:468 TR Act
su'safSu=1,safSg=1,safApp=osaftest', si'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820134+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33700"] 262:amf/amfd/sg_2n_fsm.cc:470 TR Std
su'safSu=2,safSg=1,safApp=osaftest', si'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.82014+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33701"] 262:amf/amfd/sg_2n_fsm.cc:480 TR 3. Act
su'safSu=1,safSg=1,safApp=osaftest', si'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820146+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33702"] 262:amf/amfd/sg_2n_fsm.cc:482 TR 3. Std
su'safSu=2,safSg=1,safApp=osaftest', si'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820151+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33703"] 262:amf/amfd/sg_2n_fsm.cc:493 << su_assigned_susi_find: act
su: 'safSu=1,safSg=1,safApp=osaftest', stdby su:
'safSu=2,safSg=1,safApp=osaftest', si: 'safSi=A,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820156+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33704"] 262:amf/amfd/sg_2n_fsm.cc:4127 >> avd_su_state_determine:
SU 'safSu=1,safSg=1,safApp=osaftest'
<143>1 2018-09-19T21:46:26.82016+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33705"] 262:amf/amfd/sg_2n_fsm.cc:4152 TR Assigned
su'safSu=1,safSg=1,safApp=osaftest', si'safSi=A,safApp=osaftest', state'1'
<143>1 2018-09-19T21:46:26.820163+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33706"] 262:amf/amfd/sg_2n_fsm.cc:4158 TR act_found'0',
quisced_found'0', quiscing_found'0'
<143>1 2018-09-19T21:46:26.820167+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33707"] 262:amf/amfd/sg_2n_fsm.cc:4175 << avd_su_state_determine:
state '1'
<143>1 2018-09-19T21:46:26.82017+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33708"] 262:amf/amfd/sg_2n_fsm.cc:640 << avd_sg_2n_act_susi: act:
'safSu=1,safSg=1,safApp=osaftest', stdby: 'safSu=2,safSg=1,safApp=osaftest'
<143>1 2018-09-19T21:46:26.820173+10:00 SC-1 osafamfd 262 osafamfd [meta
sequenceId="33709"] 262:amf/amfd/sg_2n_fsm.cc:801 << avd_sg_2n_su_chose_asgn:
'(null)'
~~~
---
Sent from sourceforge.net because opensaf-tickets@lists.sourceforge.net is
subscribed to https://sourceforge.net/p/opensaf/tickets/
To unsubscribe from further messages, a project admin can change settings at
https://sourceforge.net/p/opensaf/admin/tickets/options. Or, if this is a
mailing list, you can unsubscribe from the mailing list.
_______________________________________________
Opensaf-tickets mailing list
Opensaf-tickets@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/opensaf-tickets
