[tickets] [opensaf:tickets] #2388 imm: active node rebooted due immd assertion failure
- **status**: assigned --> invalid - **Comment**: 4.7 is not supported from OpenSAF perspective. closing this defect as invalid --- ** [tickets:#2388] imm: active node rebooted due immd assertion failure** **Status:** invalid **Milestone:** 5.2.0 **Created:** Tue Mar 21, 2017 07:18 AM UTC by M Chandrasekhar **Last Updated:** Fri Mar 24, 2017 11:38 AM UTC **Owner:** Neelakanta Reddy **Attachments:** - [logs.tar](https://sourceforge.net/p/opensaf/tickets/2388/attachment/logs.tar) (38.0 MB; application/octet-stream) ###Environment details OS : Suse 64bit Setup : 4 nodes ( 2 controllers and 2 payloads with 1PBE enabled ) SC-1 and PL-3 installed with 4.7GA SC-2 and PL-4 installed with 5.2RC1 ###Summary Active controller got rebooted due to immd got assertion failure after few immnd restarts. steps followed: 1. bring up SC-1 and PL-3 with 4.7GA version 2. bring up SC-2 and PL-4 with 5.2RC version 3. do si-swap, and make SC-2 active 3. run few regression tests and immnd restarts and issue was noticed. Mar 20 23:38:02 fos2 osafimmnd[27544]: NO NODE STATE-> IMM_NODE_FULLY_AVAILABLE 2927 Mar 20 23:38:02 fos2 osafimmd[17384]: NO ACT: New Epoch for IMMND process at node 2010f old epoch: 29 new epoch:30 Mar 20 23:38:02 fos2 osafimmnd[27544]: NO RepositoryInitModeT is SA_IMM_KEEP_REPOSITORY Mar 20 23:38:02 fos2 osafimmnd[27544]: WA IMM Access Control mode is DISABLED! Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Epoch set to 30 in ImmModel Mar 20 23:38:02 fos2 test_immsv: IN Received PROC_STALE_CLIENTS Mar 20 23:38:02 fos2 osafimmnd[27544]: NO SERVER STATE: IMM_SERVER_SYNC_CLIENT --> IMM_SERVER_READY Mar 20 23:38:02 fos2 osafimmnd[27544]: NO ImmModel received scAbsenceAllowed 0 Mar 20 23:38:02 fos2 osafimmd[17384]: NO ACT: New Epoch for IMMND process at node 2030f old epoch: 29 new epoch:30 Mar 20 23:38:02 fos2 osafimmd[17384]: NO ACT: New Epoch for IMMND process at node 2040f old epoch: 29 new epoch:30 Mar 20 23:38:02 fos2 osafimmd[17384]: NO ACT: New Epoch for IMMND process at node 2020f old epoch: 0 new epoch:30 Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 944 (safSmfService) <315, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 945 (safEvtService) <123, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 946 (safLogService) <127, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 947 (safCheckPointService) <134, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 948 (safClmService) <131, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 949 (safLckService) <135, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 950 (MsgQueueService131599) <12777, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 951 (safAmfService) <129, 2020f> Mar 20 23:38:03 fos2 osafimmnd[27544]: NO Implementer (applier) connected: 952 (@OpenSafImmReplicatorB) <13770, 2020f> Mar 20 23:38:03 fos2 osafntfimcnd[27526]: NO Started Mar 20 23:38:03 fos2 osafimmnd[27544]: NO PBE-OI established on other SC. Dumping incrementally to file imm.db Mar 20 23:38:08 fos2 sudo: tet : TTY=unknown ; PWD=/tmp/26815aa ; USER=root ; COMMAND=/bin/kill -9 27544 Mar 20 23:38:08 fos2 osafimmd[17384]: NO MDS event from svc_id 25 (change:4, dest:565217221926950) Mar 20 23:38:08 fos2 osafamfnd[17445]: NO Restarting a component of 'safSu=SC-2,safSg=NoRed,safApp=OpenSAF' (comp restart count: 10) Mar 20 23:38:08 fos2 osafamfnd[17445]: NO 'safComp=IMMND,safSu=SC-2,safSg=NoRed,safApp=OpenSAF' faulted due to 'avaDown' : Recovery is 'componentRestart' Mar 20 23:38:08 fos2 osafntfimcnd[27526]: NO saImmOiDispatch() Fail SA_AIS_ERR_BAD_HANDLE (9) Mar 20 23:38:08 fos2 osafimmnd[27586]: mkfifo already exists: /var/lib/opensaf/osafimmnd.fifo File exists Mar 20 23:38:08 fos2 osafimmnd[27586]: Started Mar 20 23:38:08 fos2 osafimmnd[27586]: NO Persistent Back-End capability configured, Pbe file:imm.db (suffix may get added) Mar 20 23:38:08 fos2 osafimmd[17384]: NO MDS event from svc_id 25 (change:3, dest:565217221935144) Mar 20 23:38:08 fos2 osafimmnd[27586]: NO IMMD service is UP ... ScAbsenseAllowed?:0 introduced?:0 Mar 20 23:38:08 fos2 osafimmnd[27586]: NO SERVER STATE: IMM_SERVER_ANONYMOUS --> IMM_SERVER_CLUSTER_WAITING Mar 20 23:38:08 fos2 osafimmnd[27586]: NO Fevs count adjusted to 64649 preLoadPid: 0 Mar 20 23:38:08 fos2 osafimmnd[27586]: src/imm/immnd/immnd_evt.c:9125: immnd_evt_proc_fevs_rcv: Assertion '!reply_dest || (reply_dest == cb->immnd_mdest_id) || isObjSync' failed. Mar 20 23:38:08 fos2 osafimmd[17384]: NO MDS event from svc_id 25 (change:4, dest:565217221935144) Mar 20 23:38:08 fos2 osafamfnd[17445]: NO Restarting a component of 'safSu=SC-2,safSg=NoRed,safApp=OpenSAF' (comp restart count: 11) Mar 20 23:38:08 fos2 osafamfnd[17445]: NO 'safComp=IMMND,safSu=SC-2,safSg=NoRed,safApp=OpenSAF' faulted due to 'avaDown' : Recovery is 'componentRestart'
[tickets] [opensaf:tickets] #2388 imm: active node rebooted due immd assertion failure
Please, try to reproduce again: share the following logs from all the nodes : 1) IMMD traces from starting of the service 2) syslog 3) IMMND traces (if they are big, share from the time of immnd tests) --- ** [tickets:#2388] imm: active node rebooted due immd assertion failure** **Status:** assigned **Milestone:** 5.2.RC2 **Created:** Tue Mar 21, 2017 07:18 AM UTC by M Chandrasekhar **Last Updated:** Fri Mar 24, 2017 10:58 AM UTC **Owner:** Neelakanta Reddy **Attachments:** - [logs.tar](https://sourceforge.net/p/opensaf/tickets/2388/attachment/logs.tar) (38.0 MB; application/octet-stream) ###Environment details OS : Suse 64bit Setup : 4 nodes ( 2 controllers and 2 payloads with 1PBE enabled ) SC-1 and PL-3 installed with 4.7GA SC-2 and PL-4 installed with 5.2RC1 ###Summary Active controller got rebooted due to immd got assertion failure after few immnd restarts. steps followed: 1. bring up SC-1 and PL-3 with 4.7GA version 2. bring up SC-2 and PL-4 with 5.2RC version 3. do si-swap, and make SC-2 active 3. run few regression tests and immnd restarts and issue was noticed. Mar 20 23:38:02 fos2 osafimmnd[27544]: NO NODE STATE-> IMM_NODE_FULLY_AVAILABLE 2927 Mar 20 23:38:02 fos2 osafimmd[17384]: NO ACT: New Epoch for IMMND process at node 2010f old epoch: 29 new epoch:30 Mar 20 23:38:02 fos2 osafimmnd[27544]: NO RepositoryInitModeT is SA_IMM_KEEP_REPOSITORY Mar 20 23:38:02 fos2 osafimmnd[27544]: WA IMM Access Control mode is DISABLED! Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Epoch set to 30 in ImmModel Mar 20 23:38:02 fos2 test_immsv: IN Received PROC_STALE_CLIENTS Mar 20 23:38:02 fos2 osafimmnd[27544]: NO SERVER STATE: IMM_SERVER_SYNC_CLIENT --> IMM_SERVER_READY Mar 20 23:38:02 fos2 osafimmnd[27544]: NO ImmModel received scAbsenceAllowed 0 Mar 20 23:38:02 fos2 osafimmd[17384]: NO ACT: New Epoch for IMMND process at node 2030f old epoch: 29 new epoch:30 Mar 20 23:38:02 fos2 osafimmd[17384]: NO ACT: New Epoch for IMMND process at node 2040f old epoch: 29 new epoch:30 Mar 20 23:38:02 fos2 osafimmd[17384]: NO ACT: New Epoch for IMMND process at node 2020f old epoch: 0 new epoch:30 Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 944 (safSmfService) <315, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 945 (safEvtService) <123, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 946 (safLogService) <127, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 947 (safCheckPointService) <134, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 948 (safClmService) <131, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 949 (safLckService) <135, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 950 (MsgQueueService131599) <12777, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 951 (safAmfService) <129, 2020f> Mar 20 23:38:03 fos2 osafimmnd[27544]: NO Implementer (applier) connected: 952 (@OpenSafImmReplicatorB) <13770, 2020f> Mar 20 23:38:03 fos2 osafntfimcnd[27526]: NO Started Mar 20 23:38:03 fos2 osafimmnd[27544]: NO PBE-OI established on other SC. Dumping incrementally to file imm.db Mar 20 23:38:08 fos2 sudo: tet : TTY=unknown ; PWD=/tmp/26815aa ; USER=root ; COMMAND=/bin/kill -9 27544 Mar 20 23:38:08 fos2 osafimmd[17384]: NO MDS event from svc_id 25 (change:4, dest:565217221926950) Mar 20 23:38:08 fos2 osafamfnd[17445]: NO Restarting a component of 'safSu=SC-2,safSg=NoRed,safApp=OpenSAF' (comp restart count: 10) Mar 20 23:38:08 fos2 osafamfnd[17445]: NO 'safComp=IMMND,safSu=SC-2,safSg=NoRed,safApp=OpenSAF' faulted due to 'avaDown' : Recovery is 'componentRestart' Mar 20 23:38:08 fos2 osafntfimcnd[27526]: NO saImmOiDispatch() Fail SA_AIS_ERR_BAD_HANDLE (9) Mar 20 23:38:08 fos2 osafimmnd[27586]: mkfifo already exists: /var/lib/opensaf/osafimmnd.fifo File exists Mar 20 23:38:08 fos2 osafimmnd[27586]: Started Mar 20 23:38:08 fos2 osafimmnd[27586]: NO Persistent Back-End capability configured, Pbe file:imm.db (suffix may get added) Mar 20 23:38:08 fos2 osafimmd[17384]: NO MDS event from svc_id 25 (change:3, dest:565217221935144) Mar 20 23:38:08 fos2 osafimmnd[27586]: NO IMMD service is UP ... ScAbsenseAllowed?:0 introduced?:0 Mar 20 23:38:08 fos2 osafimmnd[27586]: NO SERVER STATE: IMM_SERVER_ANONYMOUS --> IMM_SERVER_CLUSTER_WAITING Mar 20 23:38:08 fos2 osafimmnd[27586]: NO Fevs count adjusted to 64649 preLoadPid: 0 Mar 20 23:38:08 fos2 osafimmnd[27586]: src/imm/immnd/immnd_evt.c:9125: immnd_evt_proc_fevs_rcv: Assertion '!reply_dest || (reply_dest == cb->immnd_mdest_id) || isObjSync' failed. Mar 20 23:38:08 fos2 osafimmd[17384]: NO MDS event from svc_id 25 (change:4, dest:565217221935144) Mar 20 23:38:08 fos2 osafamfnd[17445]: NO Restarting a component of 'safSu=SC-2,safSg=NoRed,safApp=OpenSAF' (comp restart count: 11) Mar 20 23:38:08 fos2 osafamfnd[17445]: NO
[tickets] [opensaf:tickets] #2388 imm: active node rebooted due immd assertion failure
it happened after 13th immnd restart, i was started immsv tests at timestamp Mar 20 23:13:56. earlier logs attachment has +/- 1hr logs, let me know if u need more logs. will try to reproduce again. --- ** [tickets:#2388] imm: active node rebooted due immd assertion failure** **Status:** assigned **Milestone:** 5.2.RC2 **Created:** Tue Mar 21, 2017 07:18 AM UTC by M Chandrasekhar **Last Updated:** Thu Mar 23, 2017 12:46 PM UTC **Owner:** Neelakanta Reddy **Attachments:** - [logs.tar](https://sourceforge.net/p/opensaf/tickets/2388/attachment/logs.tar) (38.0 MB; application/octet-stream) ###Environment details OS : Suse 64bit Setup : 4 nodes ( 2 controllers and 2 payloads with 1PBE enabled ) SC-1 and PL-3 installed with 4.7GA SC-2 and PL-4 installed with 5.2RC1 ###Summary Active controller got rebooted due to immd got assertion failure after few immnd restarts. steps followed: 1. bring up SC-1 and PL-3 with 4.7GA version 2. bring up SC-2 and PL-4 with 5.2RC version 3. do si-swap, and make SC-2 active 3. run few regression tests and immnd restarts and issue was noticed. Mar 20 23:38:02 fos2 osafimmnd[27544]: NO NODE STATE-> IMM_NODE_FULLY_AVAILABLE 2927 Mar 20 23:38:02 fos2 osafimmd[17384]: NO ACT: New Epoch for IMMND process at node 2010f old epoch: 29 new epoch:30 Mar 20 23:38:02 fos2 osafimmnd[27544]: NO RepositoryInitModeT is SA_IMM_KEEP_REPOSITORY Mar 20 23:38:02 fos2 osafimmnd[27544]: WA IMM Access Control mode is DISABLED! Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Epoch set to 30 in ImmModel Mar 20 23:38:02 fos2 test_immsv: IN Received PROC_STALE_CLIENTS Mar 20 23:38:02 fos2 osafimmnd[27544]: NO SERVER STATE: IMM_SERVER_SYNC_CLIENT --> IMM_SERVER_READY Mar 20 23:38:02 fos2 osafimmnd[27544]: NO ImmModel received scAbsenceAllowed 0 Mar 20 23:38:02 fos2 osafimmd[17384]: NO ACT: New Epoch for IMMND process at node 2030f old epoch: 29 new epoch:30 Mar 20 23:38:02 fos2 osafimmd[17384]: NO ACT: New Epoch for IMMND process at node 2040f old epoch: 29 new epoch:30 Mar 20 23:38:02 fos2 osafimmd[17384]: NO ACT: New Epoch for IMMND process at node 2020f old epoch: 0 new epoch:30 Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 944 (safSmfService) <315, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 945 (safEvtService) <123, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 946 (safLogService) <127, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 947 (safCheckPointService) <134, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 948 (safClmService) <131, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 949 (safLckService) <135, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 950 (MsgQueueService131599) <12777, 2020f> Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 951 (safAmfService) <129, 2020f> Mar 20 23:38:03 fos2 osafimmnd[27544]: NO Implementer (applier) connected: 952 (@OpenSafImmReplicatorB) <13770, 2020f> Mar 20 23:38:03 fos2 osafntfimcnd[27526]: NO Started Mar 20 23:38:03 fos2 osafimmnd[27544]: NO PBE-OI established on other SC. Dumping incrementally to file imm.db Mar 20 23:38:08 fos2 sudo: tet : TTY=unknown ; PWD=/tmp/26815aa ; USER=root ; COMMAND=/bin/kill -9 27544 Mar 20 23:38:08 fos2 osafimmd[17384]: NO MDS event from svc_id 25 (change:4, dest:565217221926950) Mar 20 23:38:08 fos2 osafamfnd[17445]: NO Restarting a component of 'safSu=SC-2,safSg=NoRed,safApp=OpenSAF' (comp restart count: 10) Mar 20 23:38:08 fos2 osafamfnd[17445]: NO 'safComp=IMMND,safSu=SC-2,safSg=NoRed,safApp=OpenSAF' faulted due to 'avaDown' : Recovery is 'componentRestart' Mar 20 23:38:08 fos2 osafntfimcnd[27526]: NO saImmOiDispatch() Fail SA_AIS_ERR_BAD_HANDLE (9) Mar 20 23:38:08 fos2 osafimmnd[27586]: mkfifo already exists: /var/lib/opensaf/osafimmnd.fifo File exists Mar 20 23:38:08 fos2 osafimmnd[27586]: Started Mar 20 23:38:08 fos2 osafimmnd[27586]: NO Persistent Back-End capability configured, Pbe file:imm.db (suffix may get added) Mar 20 23:38:08 fos2 osafimmd[17384]: NO MDS event from svc_id 25 (change:3, dest:565217221935144) Mar 20 23:38:08 fos2 osafimmnd[27586]: NO IMMD service is UP ... ScAbsenseAllowed?:0 introduced?:0 Mar 20 23:38:08 fos2 osafimmnd[27586]: NO SERVER STATE: IMM_SERVER_ANONYMOUS --> IMM_SERVER_CLUSTER_WAITING Mar 20 23:38:08 fos2 osafimmnd[27586]: NO Fevs count adjusted to 64649 preLoadPid: 0 Mar 20 23:38:08 fos2 osafimmnd[27586]: src/imm/immnd/immnd_evt.c:9125: immnd_evt_proc_fevs_rcv: Assertion '!reply_dest || (reply_dest == cb->immnd_mdest_id) || isObjSync' failed. Mar 20 23:38:08 fos2 osafimmd[17384]: NO MDS event from svc_id 25 (change:4, dest:565217221935144) Mar 20 23:38:08 fos2 osafamfnd[17445]: NO Restarting a component of 'safSu=SC-2,safSg=NoRed,safApp=OpenSAF' (comp restart count: 11) Mar 20 23:38:08 fos2 osafamfnd[17445]: NO
[tickets] [opensaf:tickets] #2388 imm: active node rebooted due immd assertion failure
- **status**: unassigned --> assigned
- **assigned_to**: Neelakanta Reddy
- **Comment**:
The Assertion src/imm/immd/immd_evt.c:813: immd_accept_node: Assertion
'node_info->immnd_key != cb->node_id' failed.
1. The cb->node_id will be updated at immd_mds_register
The cb->node_id has to be 2020f, since it is 2020f
2. The node arrived is :
LOG_NO("Extended intro from node %x", node_info->immnd_key);
Mar 20 23:38:09.111387 osafimmd [17384:src/imm/immd/immd_evt.c:1563] NO
Extended intro from node 2020f
Means "node_info->immnd_key is 2020f from above trace print"
There should be some memory corruption problem over here.
For how many IMMND restarts does you observe this, the shared immd logs are not
from starting.
---
** [tickets:#2388] imm: active node rebooted due immd assertion failure**
**Status:** assigned
**Milestone:** 5.2.RC2
**Created:** Tue Mar 21, 2017 07:18 AM UTC by M Chandrasekhar
**Last Updated:** Tue Mar 21, 2017 07:18 AM UTC
**Owner:** Neelakanta Reddy
**Attachments:**
-
[logs.tar](https://sourceforge.net/p/opensaf/tickets/2388/attachment/logs.tar)
(38.0 MB; application/octet-stream)
###Environment details
OS : Suse 64bit
Setup : 4 nodes ( 2 controllers and 2 payloads with 1PBE enabled )
SC-1 and PL-3 installed with 4.7GA
SC-2 and PL-4 installed with 5.2RC1
###Summary
Active controller got rebooted due to immd got assertion failure after few
immnd restarts.
steps followed:
1. bring up SC-1 and PL-3 with 4.7GA version
2. bring up SC-2 and PL-4 with 5.2RC version
3. do si-swap, and make SC-2 active
3. run few regression tests and immnd restarts and issue was noticed.
Mar 20 23:38:02 fos2 osafimmnd[27544]: NO NODE STATE-> IMM_NODE_FULLY_AVAILABLE
2927
Mar 20 23:38:02 fos2 osafimmd[17384]: NO ACT: New Epoch for IMMND process at
node 2010f old epoch: 29 new epoch:30
Mar 20 23:38:02 fos2 osafimmnd[27544]: NO RepositoryInitModeT is
SA_IMM_KEEP_REPOSITORY
Mar 20 23:38:02 fos2 osafimmnd[27544]: WA IMM Access Control mode is DISABLED!
Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Epoch set to 30 in ImmModel
Mar 20 23:38:02 fos2 test_immsv: IN Received PROC_STALE_CLIENTS
Mar 20 23:38:02 fos2 osafimmnd[27544]: NO SERVER STATE: IMM_SERVER_SYNC_CLIENT
--> IMM_SERVER_READY
Mar 20 23:38:02 fos2 osafimmnd[27544]: NO ImmModel received scAbsenceAllowed 0
Mar 20 23:38:02 fos2 osafimmd[17384]: NO ACT: New Epoch for IMMND process at
node 2030f old epoch: 29 new epoch:30
Mar 20 23:38:02 fos2 osafimmd[17384]: NO ACT: New Epoch for IMMND process at
node 2040f old epoch: 29 new epoch:30
Mar 20 23:38:02 fos2 osafimmd[17384]: NO ACT: New Epoch for IMMND process at
node 2020f old epoch: 0 new epoch:30
Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 944
(safSmfService) <315, 2020f>
Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 945
(safEvtService) <123, 2020f>
Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 946
(safLogService) <127, 2020f>
Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 947
(safCheckPointService) <134, 2020f>
Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 948
(safClmService) <131, 2020f>
Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 949
(safLckService) <135, 2020f>
Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 950
(MsgQueueService131599) <12777, 2020f>
Mar 20 23:38:02 fos2 osafimmnd[27544]: NO Implementer connected: 951
(safAmfService) <129, 2020f>
Mar 20 23:38:03 fos2 osafimmnd[27544]: NO Implementer (applier) connected: 952
(@OpenSafImmReplicatorB) <13770, 2020f>
Mar 20 23:38:03 fos2 osafntfimcnd[27526]: NO Started
Mar 20 23:38:03 fos2 osafimmnd[27544]: NO PBE-OI established on other SC.
Dumping incrementally to file imm.db
Mar 20 23:38:08 fos2 sudo: tet : TTY=unknown ; PWD=/tmp/26815aa ;
USER=root ; COMMAND=/bin/kill -9 27544
Mar 20 23:38:08 fos2 osafimmd[17384]: NO MDS event from svc_id 25 (change:4,
dest:565217221926950)
Mar 20 23:38:08 fos2 osafamfnd[17445]: NO Restarting a component of
'safSu=SC-2,safSg=NoRed,safApp=OpenSAF' (comp restart count: 10)
Mar 20 23:38:08 fos2 osafamfnd[17445]: NO
'safComp=IMMND,safSu=SC-2,safSg=NoRed,safApp=OpenSAF' faulted due to 'avaDown'
: Recovery is 'componentRestart'
Mar 20 23:38:08 fos2 osafntfimcnd[27526]: NO saImmOiDispatch() Fail
SA_AIS_ERR_BAD_HANDLE (9)
Mar 20 23:38:08 fos2 osafimmnd[27586]: mkfifo already exists:
/var/lib/opensaf/osafimmnd.fifo File exists
Mar 20 23:38:08 fos2 osafimmnd[27586]: Started
Mar 20 23:38:08 fos2 osafimmnd[27586]: NO Persistent Back-End capability
configured, Pbe file:imm.db (suffix may get added)
Mar 20 23:38:08 fos2 osafimmd[17384]: NO MDS event from svc_id 25 (change:3,
dest:565217221935144)
Mar 20 23:38:08 fos2 osafimmnd[27586]: NO IMMD service is UP ...
ScAbsenseAllowed?:0 introduced?:0
Mar 20 23:38:08 fos2 osafimmnd[27586]: NO SERVER STATE: IMM_SERVER_ANONYMOUS
--> IMM_SERVER_CLUSTER_WAITING
Mar 20 23:38:08 fos2 osafimmnd[27586]: NO Fevs count
