- Description has changed:
Diff:
~~~~
--- old
+++ new
@@ -1,6 +1,13 @@
**Problem Description**
During headless recovery in a multi-SC cluster, amfnd on a payload node can be
rebooted by the new active amfd with the error:
-avd_msg_sanity_chk: invalid msg id X, msg type Y, from <node> should be Z
+~~~
+2026-11-10 19:34:01.753 SC-2 osafamfd[238]: WA avd_msg_sanity_chk: invalid msg
id 2, msg type 5, from 2050f should be 3
+2026-11-10 19:34:01.753 SC-2 osafamfd[238]: WA Sending node reboot order
+2026-11-10 19:34:01.754 SC-2 osafamfd[238]: WA avd_msg_sanity_chk: invalid msg
id 2, msg type 8, from 20a0f should be 6
+2026-11-10 19:34:01.756 SC-2 osafamfd[238]: WA avd_msg_sanity_chk: invalid msg
id 3, msg type 8, from 20a0f should be 6
+2026-11-10 19:34:01.759 SC-2 osafamfd[238]: WA avd_msg_sanity_chk: invalid msg
id 4, msg type 8, from 20a0f should be 6
+2026-11-10 19:34:01.761 SC-2 osafamfd[238]: WA avd_msg_sanity_chk: invalid msg
id 5, msg type 8, from 20a0f should be 6
+~~~
Root Cause
Two related issues in src/amf/amfnd/di.cc:
~~~~
---
**[tickets:#3410] amf: unexpected reboot during split-brain roaming SC test**
**Status:** unassigned
**Milestone:** 5.27.01
**Created:** Fri Nov 13, 2026 08:48 AM UTC by Thang Duc Nguyen
**Last Updated:** Fri Nov 13, 2026 08:48 AM UTC
**Owner:** nobody
**Problem Description**
During headless recovery in a multi-SC cluster, amfnd on a payload node can be
rebooted by the new active amfd with the error:
~~~
2026-11-10 19:34:01.753 SC-2 osafamfd[238]: WA avd_msg_sanity_chk: invalid msg
id 2, msg type 5, from 2050f should be 3
2026-11-10 19:34:01.753 SC-2 osafamfd[238]: WA Sending node reboot order
2026-11-10 19:34:01.754 SC-2 osafamfd[238]: WA avd_msg_sanity_chk: invalid msg
id 2, msg type 8, from 20a0f should be 6
2026-11-10 19:34:01.756 SC-2 osafamfd[238]: WA avd_msg_sanity_chk: invalid msg
id 3, msg type 8, from 20a0f should be 6
2026-11-10 19:34:01.759 SC-2 osafamfd[238]: WA avd_msg_sanity_chk: invalid msg
id 4, msg type 8, from 20a0f should be 6
2026-11-10 19:34:01.761 SC-2 osafamfd[238]: WA avd_msg_sanity_chk: invalid msg
id 5, msg type 8, from 20a0f should be 6
~~~
Root Cause
Two related issues in src/amf/amfnd/di.cc:
**Issue #1** — Already-sent records re-armed and retransmitted as duplicates to
the new active
When the active amfd goes down, avnd_diq_rec_check_buffered_msg resets msg_id=0
on any AVSV_N2D_INFO_SU_SI_ASSIGN_MSG or AVSV_N2D_OPERATION_STATE_MSG record in
dnd_list — including records that were already delivered to the (now-down)
active and awaiting ACK (msg_id != 0). On recovery, these records are correctly
sent once to the new active via avnd_diq_rec_send_buffered_msg (triggered by
NODE_UP ACK). However, they remain in dnd_list awaiting ACK. When SET_LEDS
subsequently arrives, avnd_evt_avd_set_leds_evh triggers
avnd_diq_rec_send_buffered_msg again, retransmitting the same record as a
duplicate to the same new active. The new active's rcv_msg_id has already
advanced past this msg_id, so avd_msg_sanity_chk detects the mismatch → reboot
order.
Two sub-problems combine to cause this:
* avnd_diq_rec_check_buffered_msg incorrectly re-arms already-delivered records
by resetting msg_id=0
* avnd_diq_rec_send_buffered_msg triggered by SET_LEDS retransmits the
lingering record a second time to the same new active
**Issue #2** — Stale buffered object updates with out-of-sequence msg_id
During headless, avnd_di_object_upd_send buffers object update messages
(AVSV_N2D_DATA_REQUEST_MSG) for attributes such as comp oper state, presence
state, restart count, and SU oper state. These attributes are already carried
by avnd_sync_csicomp/avnd_sync_sisu sent to the new active before node_up. When
the buffered copies are later resent, they carry low msg_id values that fall
behind the new active's already-advanced rcv_msg_id → mismatch → can cause
reboot.
---
Sent from sourceforge.net because [email protected] is
subscribed to https://sourceforge.net/p/opensaf/tickets/
To unsubscribe from further messages, a project admin can change settings at
https://sourceforge.net/p/opensaf/admin/tickets/options. Or, if this is a
mailing list, you can unsubscribe from the mailing list.
_______________________________________________
Opensaf-tickets mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/opensaf-tickets