- **status**: unassigned --> accepted
- **assigned_to**: Nagendra Kumar
- **Version**: --> 5.2 FC
- **Comment**:
This is reproducible on eb8089acf533+ (opensaf-5.1.x) 5.1.GA/5.1.0 release also.
Reproducible steps:
1. The following code changes were done for reproducing on standby controller:
diff --git a/osaf/services/saf/amf/amfd/svctype.cc
b/osaf/services/saf/amf/amfd/svctype.cc
--- a/osaf/services/saf/amf/amfd/svctype.cc
+++ b/osaf/services/saf/amf/amfd/svctype.cc
@@ -230,6 +230,9 @@ SaAisErrorT avd_svctype_config_get(void)
searchParam.searchOneAttr.attrName =
const_cast<SaImmAttrNameT>("SaImmAttrClassName");
searchParam.searchOneAttr.attrValueType = SA_IMM_ATTR_SASTRINGT;
searchParam.searchOneAttr.attrValue = &className;
+ LOG_ER("1. Sleeping .........................");
+ sleep(1);
+ LOG_ER("2. Sleeping .........................");
if (immutil_saImmOmSearchInitialize_2(avd_cb->immOmHandle, nullptr,
SA_IMM_SUBTREE,
SA_IMM_SEARCH_ONE_ATTR | SA_IMM_SEARCH_GET_ALL_ATTR,
&searchParam,
2. Start Act(SC-1) and Standby(SC-2) controller.
3. Kill immnd on SC-2 and when when following errors comes again kill Immnd:
"1. Sleeping ........................."
4. Amfd exists:
Mar 14 13:02:58 PM_SC-2 osafimmd[1586]: NO SBY: New Epoch for IMMND process at
node 2010f old epoch: 26 new epoch:27
Mar 14 13:02:58 PM_SC-2 osafimmd[1586]: NO IMMND coord at 2010f
Mar 14 13:02:58 PM_SC-2 osafamfd[1637]: ER No objects found (1)
Mar 14 13:02:58 PM_SC-2 osafamfd[1637]: ER Failed to read configuration, AMF
will not start
Mar 14 13:02:58 PM_SC-2 osafamfd[1637]: ER avd_imm_config_get FAILED
Mar 14 13:02:58 PM_SC-2 osafamfnd[1647]: WA AMF director unexpectedly crashed
Mar 14 13:02:58 PM_SC-2 osafamfnd[1647]: Rebooting OpenSAF NodeId = 131599 EE
Name = , Reason: local AVD down(Adest) or both AVD down(Vdest) received,
OwnNodeId = 131599, SupervisionTime = 60
Mar 14 13:02:58 PM_SC-2 opensaf_reboot: Rebooting local node; timeout=60
---
** [tickets:#2361] AMFD: amfd crashed with healthCheckcallbackTimeout causing
both controllers to reboot**
**Status:** accepted
**Milestone:** 5.0.2
**Created:** Fri Mar 10, 2017 09:08 AM UTC by Chani Srivastava
**Last Updated:** Fri Mar 10, 2017 10:29 AM UTC
**Owner:** Nagendra Kumar
**Environment details**
OS : Suse 64bit
Changeset : 8634 ( 5.2.FC)
Setup : 4 nodes ( 2 controllers and 2 payloads with 1PBE enabled )
**Step**
1. Bringu opensaf on four nodes and create a load of 1 lakh objects
2. Imm test cases running on standby controller
SC-1 syslog
Mar 7 19:45:58 OSAF-SC1 osafamfnd[4720]: NO
'safComp=IMMD,safSu=SC-1,safSg=2N,safApp=OpenSAF' recovery action escalated
from 'componentFailover' to 'suFailover'
Mar 7 19:45:58 OSAF-SC1 osafamfnd[4720]: NO
'safComp=IMMD,safSu=SC-1,safSg=2N,safApp=OpenSAF' faulted due to
'healthCheckcallbackTimeout' : Recovery is 'suFailover'
**Mar 7 19:45:58 OSAF-SC1 osafamfnd[4720]: ER
safComp=IMMD,safSu=SC-1,safSg=2N,safApp=OpenSAF Faulted due
to:healthCheckcallbackTimeout Recovery is:suFailover
Mar 7 19:45:58 OSAF-SC1 osafamfnd[4720]: Rebooting OpenSAF NodeId = 131343 EE
Name = , Reason: Component faulted: recovery is node failfast, OwnNodeId =
131343, SupervisionTime = 60**
Mar 7 19:45:58 OSAF-SC1 opensaf_reboot: Rebooting local node; timeout=60
SC-2 syslog
Mar 7 19:41:00 OSAF-SC2 osafamfd[4339]: ER Failed to read configuration, AMF
will not start
Mar 7 19:41:00 OSAF-SC2 osafamfd[4339]: ER avd_imm_config_get FAILED
**Mar 7 19:41:00 OSAF-SC2 osafamfnd[4349]: ER AMFD has unexpectedly crashed.
Rebooting node**
Mar 7 19:41:00 OSAF-SC2 osafamfnd[4349]: Rebooting OpenSAF NodeId = 131599 EE
Name = , Reason: AMFD has unexpectedly crashed. Rebooting node, OwnNodeId =
131599, SupervisionTime = 60
Mar 7 19:41:00 OSAF-SC2 opensaf_reboot: Rebooting local node; timeout=60
amfd, immnd and immd traces are shared seperately as those are huge in size
---
Sent from sourceforge.net because opensaf-tickets@lists.sourceforge.net is
subscribed to https://sourceforge.net/p/opensaf/tickets/
To unsubscribe from further messages, a project admin can change settings at
https://sourceforge.net/p/opensaf/admin/tickets/options. Or, if this is a
mailing list, you can unsubscribe from the mailing list.------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Opensaf-tickets mailing list
Opensaf-tickets@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/opensaf-tickets
