[tickets] [opensaf:tickets] #2142 imm: Compile the IMM library using the C++ compiler
- **status**: review --> fixed - **Comment**: default (5.2) [staging:3bd4e5] changeset: 8432:3bd4e5b7a96d user:Hung Nguyen date:Wed Nov 02 11:23:40 2016 +0700 summary: imm: Compile the IMM library using the C++ compiler [#2142] [staging:c94117] changeset: 8433:c9411767b601 user:Hung Nguyen date:Thu Dec 08 10:51:46 2016 +0700 summary: imm: Fix "crosses initialization" errors [#2142] [staging:4b9cd9] changeset: 8434:4b9cd9530600 user:Hung Nguyen date:Thu Dec 08 10:57:16 2016 +0700 summary: imm: Fix "invalid conversion" errors. [#2142] [staging:b0317b] changeset: 8435:b0317ba353eb user:Hung Nguyen date:Thu Dec 08 10:57:39 2016 +0700 summary: imm: Fix "comparison between signed and unsigned integer" errors [#2142] [staging:b358f6] changeset: 8436:b358f65db262 user:Hung Nguyen date:Thu Dec 08 10:59:06 2016 +0700 summary: imm: Fix linkage errors [#2142] --- ** [tickets:#2142] imm: Compile the IMM library using the C++ compiler** **Status:** fixed **Milestone:** 5.2.FC **Created:** Thu Oct 27, 2016 04:31 AM UTC by Hung Nguyen **Last Updated:** Thu Nov 03, 2016 11:02 AM UTC **Owner:** Hung Nguyen Compile the IMM library using the C++ compiler and fix all errors that C++ compiler complains. --- Sent from sourceforge.net because opensaf-tickets@lists.sourceforge.net is subscribed to https://sourceforge.net/p/opensaf/tickets/ To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/opensaf/admin/tickets/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.-- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today.http://sdm.link/xeonphi___ Opensaf-tickets mailing list Opensaf-tickets@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-tickets
[tickets] [opensaf:tickets] #2219 ntfd: circular dependency with osafntfimcnd
Hi Praveen, Below is my understanding regarding your concerns: > We need gaurantee that thread will surely terminated IMCN IMCN component after getting unblocked, i think, it will go to `handle_sigterm_event()` for SIGTERM signal handling and exit gracefully. Other thing is, when IMCN is blocking at saImmOiDispatch(), if any IMM configuration changes, all CCB callbacks to the blocking applier may get TIME_OUT, I guess. I see, IMCN sets `IMMA_SYNCR_TIMEOUT` IMM sync timeout to 01 second. > One more thread will get spawned for termination if doing another switchver This concern is valid. We should have one flag to prevent this happens. /Vu --- ** [tickets:#2219] ntfd: circular dependency with osafntfimcnd** **Status:** assigned **Milestone:** 5.0.2 **Created:** Thu Dec 08, 2016 05:14 AM UTC by Gary Lee **Last Updated:** Mon Dec 12, 2016 06:38 AM UTC **Owner:** Praveen A circular dependency can be seen when performing a si-swap of safSi=SC-2N,safApp=OpenSAF: 1. Active NTFD is trying to sync with Standby using MBC 2. Standby NTFD is the process of terminating its local osafntfimcnd. It is stuck in timedwait_imcn_exit() and cannot reply to the Active. 3. osafntfimcnd [on standby] is trying to send a notfication to Active NTFD So we have (1) depending on (2) depending on (3) depending on (1) This results in a temporary deadlock that dramatically slows down NTFD's ability to process its main dispatch loop. The deadlock only lasts for approx. 1 second, when mbcsv_mds_send_msg() times out. But since there could be lots of MBC messages to send, sometimes osafntfimcnd is killed with SIGABRT generating a coredump. The si-swap operation will also timeout. steps to reproduce - Run loop of ntftest 32 root@SC-1:~# for i in {1..10}; do ntftest 32; done - On another terminal, keep swapping 2N Opensaf SI, got coredump after couples of swaps root@SC-1:~# amf-adm si-swap safSi=SC-2N,safApp=OpenSAF ... root@SC-1:~# amf-adm si-swap safSi=SC-2N,safApp=OpenSAF ~~~ SC-2 (active) There are a lot of send failures. Each taking approx. 1 second to timeout. During these 1 second timeouts, NTFD cannot process the main dispatch loop. Dec 7 11:01:37.531772 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:37.531781 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:38.537307 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:38.537758 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:38.537766 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:39.543180 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:39.543695 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:39.543698 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:40.545252 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:40.545719 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:40.545726 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:41.551328 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:41.551971 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:41.551979 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:42.557594 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:42.558171 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:42.558179 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:43.564051 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:43.564874 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:43.564883 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:44.572407 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:44.573262 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:44.573271 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:45.575091 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:47.083548 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e ~~~ ~~~ SC-1 (standby) NTFD is trying to terminate osafntfimcnd. While it is doing that, it cannot reply to NTFD on SC-2. Meanwhile, osafntfimcnd is sending NTF notifications to NTFD on SC-1. Dec 7 11:01:35.453151 osafntfd [464:ntfs_imcnutil.c:0316] TR handle_state_ntfimcn: Terminating osafntfimcnd process Dec 7 11:01:45.474313 osafntfd [464:ntfs_imcnutil.c:0124] TR Termination timeout Dec
[tickets] [opensaf:tickets] #2219 ntfd: circular dependency with osafntfimcnd
Hi Vu, That was expected behaviour that IMCN after getting unblocked will process sig_term event. For this to happen, it has to come out of IMMDispatch(DISPATCH_ALL). I think IMMND just sends CCB callbacks for atleast create/delete/modify and complete on one go. For CCB Apply IMMND will have to wait for completed response which creates a dependency on IMCN. But I this not synschronous lilke callbacks in other services. So in our test application which is only one client making conf changes, no problem will occur. However in case of multiple clients are making CCB changes then IMMND will be sending these to IMCNand these will be piling up. Anyways, I do not see much risk but still I would like Minh to clarify the problem enviornment whether configuration changes are done much faster and by multiple clients. I have seen in some IMM tickets where reporter was creating thousands of objects Yes, by maintaining the flag spawning of mulitple threads can be avoided. But we need to remember here that maintaining this flag does not block a user to trigger swtichvers. So per switchover, restart of IMCN may get missed. Thanks, Praveen --- ** [tickets:#2219] ntfd: circular dependency with osafntfimcnd** **Status:** assigned **Milestone:** 5.0.2 **Created:** Thu Dec 08, 2016 05:14 AM UTC by Gary Lee **Last Updated:** Mon Dec 12, 2016 08:47 AM UTC **Owner:** Praveen A circular dependency can be seen when performing a si-swap of safSi=SC-2N,safApp=OpenSAF: 1. Active NTFD is trying to sync with Standby using MBC 2. Standby NTFD is the process of terminating its local osafntfimcnd. It is stuck in timedwait_imcn_exit() and cannot reply to the Active. 3. osafntfimcnd [on standby] is trying to send a notfication to Active NTFD So we have (1) depending on (2) depending on (3) depending on (1) This results in a temporary deadlock that dramatically slows down NTFD's ability to process its main dispatch loop. The deadlock only lasts for approx. 1 second, when mbcsv_mds_send_msg() times out. But since there could be lots of MBC messages to send, sometimes osafntfimcnd is killed with SIGABRT generating a coredump. The si-swap operation will also timeout. steps to reproduce - Run loop of ntftest 32 root@SC-1:~# for i in {1..10}; do ntftest 32; done - On another terminal, keep swapping 2N Opensaf SI, got coredump after couples of swaps root@SC-1:~# amf-adm si-swap safSi=SC-2N,safApp=OpenSAF ... root@SC-1:~# amf-adm si-swap safSi=SC-2N,safApp=OpenSAF ~~~ SC-2 (active) There are a lot of send failures. Each taking approx. 1 second to timeout. During these 1 second timeouts, NTFD cannot process the main dispatch loop. Dec 7 11:01:37.531772 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:37.531781 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:38.537307 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:38.537758 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:38.537766 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:39.543180 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:39.543695 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:39.543698 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:40.545252 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:40.545719 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:40.545726 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:41.551328 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:41.551971 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:41.551979 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:42.557594 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:42.558171 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:42.558179 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:43.564051 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:43.564874 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:43.564883 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:44.572407 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:44.573262 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:44.573271 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:45.575091 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:47.083548
[tickets] [opensaf:tickets] #2219 ntfd: circular dependency with osafntfimcnd
Hi The original specificatinon for IMCN said that it should be able to handle HA state changes. A first increment of IMCN without any HA handling except the possibility to inform the IMCN process if it shall run as active or standby when started. To handle a HA state change the IMCN process must be terminated and restarted given the correct state. In the "IMCN NTF notification protocol" a special notification was defined (See Error Report Notification in the OpenSAF Extensions PR document) to be sent if there is a risk that any IMM changes may have been missed. This notification is sent every time IMCN is started as active. If a client receives such a notification it has to read all relevant IMM data in order to synchronize itself. After that it can rely on notifications from IMCN until the next Error report notification is received. The intention was to make IMCN able to change role without beeing restarted and without loosing track of IMM changes (ticket #157). However the first increment was considered good enough so #157 was never implemented. This means that IMCN can be terminated at any time also if executing an IMM callback function. It does not matter if any IMM change is lost and never reported. Since IMCN is a process and it is the process that is terminated there is no problem with memory leeks or resource leeks. IMM will detect that the client is gone and will finalize all resources. In NTF the IMCN surveillance thread will also detect that the IMCN process is gone and start a new one and give it the correct state (which may no longer be active) This means that any solution for immediate termination is Ok e.g. to handle the termination in a separate thread to avoid the described deadlock. NOTE: SInce there is no HA handling implemented it is actually not needed to start any IMCN process on any other node than the active. That this is done is also remainings from the original intention. MUST BE FIXED! I saw that the init_ntfimcn() has been move from the main initialize() to initialize_for_assignment(). This is NOT OK since initialize_for_assignment() is called more than once meaning that also init_ntfimcn() is called more than once. The init_ntfimcn() function starts the IMCN surveillance thread and that cannot be done more than once! Other init functions that are called from initialize_for_assignment() has an "is_initialized flag" in order to prevent initialization more than once. This could be a solution to implement for init_ntfimcn() as well. Thanks Lennart --- ** [tickets:#2219] ntfd: circular dependency with osafntfimcnd** **Status:** assigned **Milestone:** 5.0.2 **Created:** Thu Dec 08, 2016 05:14 AM UTC by Gary Lee **Last Updated:** Mon Dec 12, 2016 09:33 AM UTC **Owner:** Praveen A circular dependency can be seen when performing a si-swap of safSi=SC-2N,safApp=OpenSAF: 1. Active NTFD is trying to sync with Standby using MBC 2. Standby NTFD is the process of terminating its local osafntfimcnd. It is stuck in timedwait_imcn_exit() and cannot reply to the Active. 3. osafntfimcnd [on standby] is trying to send a notfication to Active NTFD So we have (1) depending on (2) depending on (3) depending on (1) This results in a temporary deadlock that dramatically slows down NTFD's ability to process its main dispatch loop. The deadlock only lasts for approx. 1 second, when mbcsv_mds_send_msg() times out. But since there could be lots of MBC messages to send, sometimes osafntfimcnd is killed with SIGABRT generating a coredump. The si-swap operation will also timeout. steps to reproduce - Run loop of ntftest 32 root@SC-1:~# for i in {1..10}; do ntftest 32; done - On another terminal, keep swapping 2N Opensaf SI, got coredump after couples of swaps root@SC-1:~# amf-adm si-swap safSi=SC-2N,safApp=OpenSAF ... root@SC-1:~# amf-adm si-swap safSi=SC-2N,safApp=OpenSAF ~~~ SC-2 (active) There are a lot of send failures. Each taking approx. 1 second to timeout. During these 1 second timeouts, NTFD cannot process the main dispatch loop. Dec 7 11:01:37.531772 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:37.531781 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:38.537307 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:38.537758 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:38.537766 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:39.543180 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:39.543695 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:39.543698 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:40.545252 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:40.545719 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_
[tickets] [opensaf:tickets] #2219 ntfd: circular dependency with osafntfimcnd
Hi Lennart, Seems the global flag `fully_initialized` in control block is used to ensure that `initialize_for_assignment` is called once. Another thing I would like to bring up here, that is TIMEOUT for all NTF API is set to 10 seconds (`#define NTFS_WAIT_TIME 1000`). Then, if calling to one of them e.g `saNtfNotificationSend()`, in worse case, the calling thread will be blocked up to 10 seconds and could lead to unexpected result as below syslog when perfoming si-swap: > osafsmfd[9785]: NO Fail to invoke admin operation, rc=SA_AIS_ERR_TIMEOUT (5). > dn=[safSi=SC-2N,safApp=OpenSAF], opId=[7] > osafsmfd[9785]: NO Admin op SA_AMF_ADMIN_SI_SWAP fail [rc = 5] I am not sure if that setting time is ok or not. /Vu --- ** [tickets:#2219] ntfd: circular dependency with osafntfimcnd** **Status:** assigned **Milestone:** 5.0.2 **Created:** Thu Dec 08, 2016 05:14 AM UTC by Gary Lee **Last Updated:** Mon Dec 12, 2016 09:53 AM UTC **Owner:** Praveen A circular dependency can be seen when performing a si-swap of safSi=SC-2N,safApp=OpenSAF: 1. Active NTFD is trying to sync with Standby using MBC 2. Standby NTFD is the process of terminating its local osafntfimcnd. It is stuck in timedwait_imcn_exit() and cannot reply to the Active. 3. osafntfimcnd [on standby] is trying to send a notfication to Active NTFD So we have (1) depending on (2) depending on (3) depending on (1) This results in a temporary deadlock that dramatically slows down NTFD's ability to process its main dispatch loop. The deadlock only lasts for approx. 1 second, when mbcsv_mds_send_msg() times out. But since there could be lots of MBC messages to send, sometimes osafntfimcnd is killed with SIGABRT generating a coredump. The si-swap operation will also timeout. steps to reproduce - Run loop of ntftest 32 root@SC-1:~# for i in {1..10}; do ntftest 32; done - On another terminal, keep swapping 2N Opensaf SI, got coredump after couples of swaps root@SC-1:~# amf-adm si-swap safSi=SC-2N,safApp=OpenSAF ... root@SC-1:~# amf-adm si-swap safSi=SC-2N,safApp=OpenSAF ~~~ SC-2 (active) There are a lot of send failures. Each taking approx. 1 second to timeout. During these 1 second timeouts, NTFD cannot process the main dispatch loop. Dec 7 11:01:37.531772 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:37.531781 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:38.537307 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:38.537758 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:38.537766 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:39.543180 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:39.543695 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:39.543698 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:40.545252 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:40.545719 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:40.545726 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:41.551328 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:41.551971 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:41.551979 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:42.557594 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:42.558171 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:42.558179 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:43.564051 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:43.564874 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:43.564883 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:44.572407 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:44.573262 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:44.573271 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:45.575091 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:47.083548 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e ~~~ ~~~ SC-1 (standby) NTFD is trying to terminate osafntfimcnd. While it is doing that, it cannot reply to NTFD on SC-2. Meanwhile, osafntfimcnd is sending NTF notifications to NTFD on SC-1. Dec 7 11:01:35.453151 osafntfd [464:ntfs_imcnutil.c:0316] TR handle_state_ntfimcn: Terminating osafntfimcnd process Dec 7 11:01:45.4743
[tickets] [opensaf:tickets] #2221 build: OpenSAF fails to build outside the source tree
- **status**: review --> fixed - **Comment**: default(5.2) : changeset: 8437:1a479e952a20 tag: tip user:Zoran Milinkovic date:Thu Dec 08 16:42:57 2016 +0100 summary: build: add top build directory to include path [#2221] --- ** [tickets:#2221] build: OpenSAF fails to build outside the source tree** **Status:** fixed **Milestone:** 5.2.FC **Created:** Thu Dec 08, 2016 03:29 PM UTC by Zoran Milinkovic **Last Updated:** Thu Dec 08, 2016 03:52 PM UTC **Owner:** Zoran Milinkovic ~~~ build for SLES12 with OpenSAF 5.2 CS8417 outside the source tree fails make[9]: Leaving directory `/home/user/git/build/opensaf/osaf/libs/core/cplusplus/base/tests' make[9]: Entering directory `/home/user/git/build/opensaf/osaf/libs/core/cplusplus/base' CXX libbase_la-file_notify.lo CXX libbase_la-log_message.lo CXX libbase_la-getenv.lo CXX libbase_la-process.lo CXX libbase_la-mutex.lo CXX libbase_la-condition_variable.lo CXX libbase_la-unix_client_socket.lo CXX libbase_la-unix_server_socket.lo CXX libbase_la-unix_socket.lo ../../../../../../../coremw/opensaf/osaf/libs/core/cplusplus/base/mutex.cc:19:22: fatal error: ./config.h: No such file or directory #include "./config.h" ^ compilation terminated. make[9]: *** [libbase_la-mutex.lo] Error 1 make[9]: *** Waiting for unfinished jobs make[9]: Leaving directory `/home/user/git/build/opensaf/osaf/libs/core/cplusplus/base' ~~~ After ./configure, config.h is created in the build directory. The root of build directory needs to be added to the include path list. --- Sent from sourceforge.net because opensaf-tickets@lists.sourceforge.net is subscribed to https://sourceforge.net/p/opensaf/tickets/ To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/opensaf/admin/tickets/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.-- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today.http://sdm.link/xeonphi___ Opensaf-tickets mailing list Opensaf-tickets@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-tickets
[tickets] [opensaf:tickets] #2226 smf: failed one-step upgrade between opensaf versions
--- ** [tickets:#2226] smf: failed one-step upgrade between opensaf versions** **Status:** unassigned **Milestone:** 5.1.1 **Created:** Mon Dec 12, 2016 03:00 PM UTC by Rafael **Last Updated:** Mon Dec 12, 2016 03:00 PM UTC **Owner:** Rafael When upgrading with one-step mode from a version before SmfExecControlHdl.cc was introduced to a version using it upgrade fails. Reason for the failure is that SMF tries to read "procExecMode" from "openSafSmfExecControl=SmfHdlCopy" but this object is only created during init in a version using SmfExecControlHdl.cc. Solution is to read in the original value of procExecMode if the copy is missing. Drawback of this solution is that changing procExecMode during a running campaign but after init phase will cause errors. --- Sent from sourceforge.net because opensaf-tickets@lists.sourceforge.net is subscribed to https://sourceforge.net/p/opensaf/tickets/ To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/opensaf/admin/tickets/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.-- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today.http://sdm.link/xeonphi___ Opensaf-tickets mailing list Opensaf-tickets@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-tickets
[tickets] [opensaf:tickets] #2134 AMF: Update RTA saAmfSISUHAState to IMM
Hi Praveen, Although part 2 #1725 is pushed, there is a chance that both avd_sg_su_si_mod_snd() and avd_susi_mod_send() are updating this attribute differently. To avoid backward incompatibility in future, I would like to document that reading saAmfSISUState while assignment is ongoing not recommended/supported. User should only read this attribute if admin operation/failover completes. Do you agree? Thanks, Minh --- ** [tickets:#2134] AMF: Update RTA saAmfSISUHAState to IMM** **Status:** wontfix **Milestone:** never **Created:** Thu Oct 20, 2016 07:58 PM UTC by Minh Hon Chau **Last Updated:** Thu Nov 24, 2016 05:25 AM UTC **Owner:** nobody In scenario of 2N Si-swap, when AMFD sends QUIESCED su_si assignment msg (for example) to AMFND that changes the HA State of SUSI assignment, AMFD updates its local state AVD_SU_SI_REL::state, checkpoint this change to standby AMFD. However, AMFD does not updates saAmfSISUHAState untill receiving su_si assignment response. Question: (1). Whether AMFD should update the runtime attribute saAmfSISUHAState to IMM as long as local @state gets updated in implementer; to make IMM, active AMFD, standby AMFD all are synced (2). Or AMFD updates saAmfSISUHAState to IMM only if AMFD receives su_si assignment from AMFND, as it has been implemented currently for some reason (not expose the change of saAmfSISUHAState to user too early?) grep "avd_susi_update" which updates saAmfSISUHAState to IMM, there is also an inconsistency in usage. For avd_susi_mod_send() sends su_si msg and also updates saAmfSISUHAState immediately, while avd_sg_su_si_mod_snd does otherwise. Since the headless recovery relies on IMM to restore the state. If saAmfSISUHAState is not updated punctually and the node is reboot during headless stage, so after headless saAmfSISUHAState read from IMM does not fit with many other states (SG fsm, SUSI fsm, saAmfSISUHAState of the other SUSIs). My question is if doing (1) will cause any problem for normal cluster? Pending patches #1725 part 2 currently implement (1). --- Sent from sourceforge.net because opensaf-tickets@lists.sourceforge.net is subscribed to https://sourceforge.net/p/opensaf/tickets/ To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/opensaf/admin/tickets/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.-- Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot___ Opensaf-tickets mailing list Opensaf-tickets@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-tickets
[tickets] [opensaf:tickets] #2223 NTFD: Increment iterator out of range NotificationMap
- **status**: assigned --> review --- ** [tickets:#2223] NTFD: Increment iterator out of range NotificationMap** **Status:** review **Milestone:** 5.0.2 **Created:** Mon Dec 12, 2016 12:37 AM UTC by Minh Hon Chau **Last Updated:** Mon Dec 12, 2016 12:37 AM UTC **Owner:** Minh Hon Chau In NtfAdmin::checkNotificationList(), when deleteConfirmedNotification() deletes the last item, the NotificationMap is reduced, posNot++ will be out of range of NotificationMap, not equal NotificationMap.end() Got ntfd coredump while switch over ntfd coredump bt ~~~ (gdb) bt full #0 NtfNotification::getNextSubscription (this=0x978930, subId=...) at NtfNotification.cc:165 No locals. #1 0x0041658d in NtfAdmin::checkNotificationList (this=0x974240) at NtfAdmin.cc:661 uSubId = {clientId = 1277, subscriptionId = 1} notification = {> = { _M_ptr = 0x978930, _M_refcount = {_M_pi = 0x978a40}}, } __FUNCTION__ = "checkNotificationList" posNot = {_M_node = 0x978770} #2 0x0040407e in amf_csi_set_callback (invocation=4290772996, compName=, new_haState=, csiDescriptor=...) at ntfs_amf.c:255 error = prev_haState = SA_AMF_HA_STANDBY role_change = rc = __FUNCTION__ = "amf_csi_set_callback" #3 0x7ff32687e48e in ava_hdl_cbk_rec_prc (info=0x7ff32000b880, reg_cbk=reg_cbk@entry=0x7ffc7437fd00) at ava_hdl.cc:655 csi_set = 0x7ff32000b898 actv_or_stdby = rc = 1 i = __FUNCTION__ = "ava_hdl_cbk_rec_prc" #4 0x7ff32687f749 in ava_hdl_cbk_dispatch_all (hdl_rec=0x7ffc7437fda8, cb=0x7ffc7437fda0) at ava_hdl.cc:446 list_resp = 0x962698 reg_cbk = {saAmfHealthcheckCallback = 0x403ee0 , saAmfComponentTerminateCallback = 0x403f00 , saAmfCSISetCallback = 0x403fd0 , saAmfCSIRemoveCallback = 0x403e70 , ---Type to continue, or q to quit--- saAmfProtectionGroupTrackCallback = 0x0, saAmfProtectionGroupTrackCallback_4 = 0x0, saAmfProxiedComponentInstantiateCallback = 0x0, saAmfProxiedComponentCleanupCallback = 0x0, saAmfContainedComponentInstantiateCallback = 0x0, saAmfContainedComponentCleanupCallback = 0x0, osafCsiAttributeChangeCallback = 0x0} rec = 0x7ff32000c2a0 hdl = 4288675841 rc = 1 #5 ava_hdl_cbk_dispatch (cb=0x7ffc7437fda0, hdl_rec=0x7ffc7437fda8, flags=) at ava_hdl.cc:320 rc = 1 __FUNCTION__ = "ava_hdl_cbk_dispatch" #6 0x7ff3268783ea in AmfAgent::Dispatch (hdl=4288675841, flags=flags@entry=SA_DISPATCH_ALL) at amf_agent.cc:283 hdl_rec = 0x962610 pend_dis = 0 __FUNCTION__ = "Dispatch" cb = 0x961b20 rc = SA_AIS_OK pend_fin = 0 #7 0x7ff326878465 in saAmfDispatch (hdl=, flags=flags@entry=SA_DISPATCH_ALL) at amf_agent.cc:244 No locals. #8 0x004037e4 in main (argc=, argv=) at ntfs_main.c:357 mbx_fd = error = rc = fds = {{fd = 20, events = 1, revents = 0}, {fd = 16, events = 1, revents = 1}, {fd = 22, events = 1, revents = 1}, {fd = 14, events = 1, revents = 0}, {fd = 24, events = 1, ---Type to continue, or q to quit--- revents = 0}, {fd = 18, events = 1, revents = 0}} term_fd = 20 __FUNCTION__ = "main" (gdb) ~~~ ntfd trace: ~~~ Dec 9 22:39:28.984523 osafntfd [453:ntfs_amf.c:0175] >> amf_csi_set_callback Dec 9 22:39:28.984527 osafntfd [453:ntfs_main.c:0271] >> initialize_for_assignment: ha_state = 1 Dec 9 22:39:28.984531 osafntfd [453:ntfs_main.c:0292] << initialize_for_assignment: rc = 1 Dec 9 22:39:28.984535 osafntfd [453:ntfs_amf.c:0043] >> amf_active_state_handler: HA ACTIVE request Dec 9 22:39:28.984539 osafntfd [453:ntfs_amf.c:0047] << amf_active_state_handler Dec 9 22:39:28.984556 osafntfd [453:ntfs_mbcsv.c:0180] >> ntfs_mbcsv_change_HA_state Dec 9 22:39:28.984560 osafntfd [453:mbcsv_api.c:0662] >> mbcsv_process_chg_role_request: Change HA role for the checkpoint Dec 9 22:39:28.984564 osafntfd [453:mbcsv_api.c:0685] TR svc_id:44, pwe_hdl:65550 Dec 9 22:39:28.984568 osafntfd [453:mbcsv_api.c:0743] << mbcsv_process_chg_role_request: retval: 1 Dec 9 22:39:28.984571 osafntfd [453:ntfs_mbcsv.c:0194] << ntfs_mbcsv_change_HA_state Dec 9 22:39:28.984575 osafntfd [453:ntfs_amf.c:0248] TR amf_csi_set_callback NTFS changing HA role from SA_AMF_HA_STANDBY to SA_AMF_HA_ACTIVE Dec 9 22:39:28.984578 osafntfd [453:amf_agent.cc:1987] >> Response: SaAmfHandleT passed is ffa1 Dec 9 22:39:28.984582 osafntfd [453:ava_hdl.cc:0890] >> ava_hdl_pend_resp_get Dec 9 22:39:28.984585 osafntfd [453:ava_hdl.cc:0906] << ava_hdl_pend_resp_get Dec 9 22:39:28.984589 osafntfd [453:ava_mds.cc:0342] >> ava_mds_send Dec 9 22:39:28.984601 osafntfd [453:ava_mds.cc:0928] >> ava_mds_msg_async_send Dec 9 22:39:28.984606 osafntfd [453:ava_mds.cc:0181] >> ava_mds_cbk D
[tickets] [opensaf:tickets] #2219 ntfd: circular dependency with osafntfimcnd
Hi, I think both patches can solve the problem. V1 (Praveen's) removes the dependency in ntfimcnd, V3 (Vu's) removes the dependency in ntfd. Below is a summary I have collected from previous discussions: - V3 (Vu's) . There could be chances that we have no active ntfimcnd: the period of no active imcn happens when both termination threads are terminating imcnd at the same time (there should be one doing earlier), and the active imcnd is stuck in ntfInitialize() . The period of 2 active ntfimcnd could be longer: In state transition from quiesced->standby, termination thread is ongoing, the old active imcnd could be a bit longer as active one I think the difficulty in V3 is ntfd can not know things happening in both local and remote ntfimcnd - V1 (Praveen's) . When ntfimcnd gets sigterm, the job from dispatching IMM callback to sending notification could be interrupted. If we make sigterm_handler sync with IMM's thread, that could cause signal abort from ntfd due to latency . But what if loss of notification is always assumpted, and we have special notification to inform clients at ntfimcnd's start up . IMM thread needs to do synchronization with sigterm_handler on common stuffs: ie immOI handle, ... Thanks /Minh --- ** [tickets:#2219] ntfd: circular dependency with osafntfimcnd** **Status:** assigned **Milestone:** 5.0.2 **Created:** Thu Dec 08, 2016 05:14 AM UTC by Gary Lee **Last Updated:** Mon Dec 12, 2016 10:26 AM UTC **Owner:** Praveen A circular dependency can be seen when performing a si-swap of safSi=SC-2N,safApp=OpenSAF: 1. Active NTFD is trying to sync with Standby using MBC 2. Standby NTFD is the process of terminating its local osafntfimcnd. It is stuck in timedwait_imcn_exit() and cannot reply to the Active. 3. osafntfimcnd [on standby] is trying to send a notfication to Active NTFD So we have (1) depending on (2) depending on (3) depending on (1) This results in a temporary deadlock that dramatically slows down NTFD's ability to process its main dispatch loop. The deadlock only lasts for approx. 1 second, when mbcsv_mds_send_msg() times out. But since there could be lots of MBC messages to send, sometimes osafntfimcnd is killed with SIGABRT generating a coredump. The si-swap operation will also timeout. steps to reproduce - Run loop of ntftest 32 root@SC-1:~# for i in {1..10}; do ntftest 32; done - On another terminal, keep swapping 2N Opensaf SI, got coredump after couples of swaps root@SC-1:~# amf-adm si-swap safSi=SC-2N,safApp=OpenSAF ... root@SC-1:~# amf-adm si-swap safSi=SC-2N,safApp=OpenSAF ~~~ SC-2 (active) There are a lot of send failures. Each taking approx. 1 second to timeout. During these 1 second timeouts, NTFD cannot process the main dispatch loop. Dec 7 11:01:37.531772 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:37.531781 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:38.537307 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:38.537758 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:38.537766 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:39.543180 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:39.543695 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:39.543698 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:40.545252 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:40.545719 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:40.545726 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:41.551328 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:41.551971 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:41.551979 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:42.557594 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:42.558171 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:42.558179 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:43.564051 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:43.564874 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:43.564883 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:44.572407 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:44.573262 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:44.573271 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:45.575091 os
[tickets] [opensaf:tickets] Re: #2219 ntfd: circular dependency with osafntfimcnd
Hi Vu, For the NtfSend API(), 10 secs can count on saflogging at server side, writing to disk, ... so I guess we should not change for NtfSend(). For the other APIs() we could see how many critical jobs the server get involved, then we can decide to reduce this timeout, perhaps for thos API() doing finalize jobs thanks, Minh --- ** [tickets:#2219] ntfd: circular dependency with osafntfimcnd** **Status:** assigned **Milestone:** 5.0.2 **Created:** Thu Dec 08, 2016 05:14 AM UTC by Gary Lee **Last Updated:** Tue Dec 13, 2016 04:40 AM UTC **Owner:** Praveen A circular dependency can be seen when performing a si-swap of safSi=SC-2N,safApp=OpenSAF: 1. Active NTFD is trying to sync with Standby using MBC 2. Standby NTFD is the process of terminating its local osafntfimcnd. It is stuck in timedwait_imcn_exit() and cannot reply to the Active. 3. osafntfimcnd [on standby] is trying to send a notfication to Active NTFD So we have (1) depending on (2) depending on (3) depending on (1) This results in a temporary deadlock that dramatically slows down NTFD's ability to process its main dispatch loop. The deadlock only lasts for approx. 1 second, when mbcsv_mds_send_msg() times out. But since there could be lots of MBC messages to send, sometimes osafntfimcnd is killed with SIGABRT generating a coredump. The si-swap operation will also timeout. steps to reproduce - Run loop of ntftest 32 root@SC-1:~# for i in {1..10}; do ntftest 32; done - On another terminal, keep swapping 2N Opensaf SI, got coredump after couples of swaps root@SC-1:~# amf-adm si-swap safSi=SC-2N,safApp=OpenSAF ... root@SC-1:~# amf-adm si-swap safSi=SC-2N,safApp=OpenSAF ~~~ SC-2 (active) There are a lot of send failures. Each taking approx. 1 second to timeout. During these 1 second timeouts, NTFD cannot process the main dispatch loop. Dec 7 11:01:37.531772 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:37.531781 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:38.537307 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:38.537758 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:38.537766 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:39.543180 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:39.543695 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:39.543698 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:40.545252 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:40.545719 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:40.545726 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:41.551328 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:41.551971 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:41.551979 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:42.557594 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:42.558171 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:42.558179 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:43.564051 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:43.564874 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:43.564883 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:44.572407 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:44.573262 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e Dec 7 11:01:44.573271 osafntfd [452:mbcsv_mds.c:0209] TR send type MDS_SENDTYPE_REDRSP: Dec 7 11:01:45.575091 osafntfd [452:mbcsv_mds.c:0247] << mbcsv_mds_send_msg: failure Dec 7 11:01:47.083548 osafntfd [452:mbcsv_mds.c:0185] >> mbcsv_mds_send_msg: sending to vdest:e ~~~ ~~~ SC-1 (standby) NTFD is trying to terminate osafntfimcnd. While it is doing that, it cannot reply to NTFD on SC-2. Meanwhile, osafntfimcnd is sending NTF notifications to NTFD on SC-1. Dec 7 11:01:35.453151 osafntfd [464:ntfs_imcnutil.c:0316] TR handle_state_ntfimcn: Terminating osafntfimcnd process Dec 7 11:01:45.474313 osafntfd [464:ntfs_imcnutil.c:0124] TR Termination timeout Dec 7 11:01:45.474375 osafntfd [464:ntfs_imcnutil.c:0130] << wait_imcnproc_termination: rc = -1, retry_cnt = 101 Dec 7 11:01:45.474387 osafntfd [464:ntfs_imcnutil.c:0168] TR Normal termination failed. Escalate to abort Dec 7 11:01:45.574703 osafntfd [464:ntfs_imcnutil.c:0172] TR Imcn successfully aborted D
[tickets] [opensaf:tickets] #2227 smf:ONE-STEP upgrade failed due to duplicated entities in comp and SU
--- ** [tickets:#2227] smf:ONE-STEP upgrade failed due to duplicated entities in comp and SU** **Status:** unassigned **Milestone:** 5.2.FC **Created:** Tue Dec 13, 2016 06:16 AM UTC by Neelakanta Reddy **Last Updated:** Tue Dec 13, 2016 06:16 AM UTC **Owner:** nobody This Ticket is extension of #2209. In #2209 if a campaign contains both rolling and singlestep. The singlestep contains duplicated node forAddremove AU/DU present in the rolling upgrad also. This ticket is related to duplcated entities SU and comp present in forAddRemove and rolling procedure. --- Sent from sourceforge.net because opensaf-tickets@lists.sourceforge.net is subscribed to https://sourceforge.net/p/opensaf/tickets/ To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/opensaf/admin/tickets/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.-- Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot___ Opensaf-tickets mailing list Opensaf-tickets@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-tickets
[tickets] [opensaf:tickets] #2134 AMF: Update RTA saAmfSISUHAState to IMM
Hi Minh, I think we cannot use "not supported". Instead we should use the word not recommended. Suppose SG is in transition state because of assignments going on, if a user makes some query at this time then user will surely get some value of attribute but this value will be changing with each query that is made during these assignments. But when admin operation or any escalation finishes then value will remain same in each query. Thanks, Praveen --- ** [tickets:#2134] AMF: Update RTA saAmfSISUHAState to IMM** **Status:** wontfix **Milestone:** never **Created:** Thu Oct 20, 2016 07:58 PM UTC by Minh Hon Chau **Last Updated:** Mon Dec 12, 2016 11:35 PM UTC **Owner:** nobody In scenario of 2N Si-swap, when AMFD sends QUIESCED su_si assignment msg (for example) to AMFND that changes the HA State of SUSI assignment, AMFD updates its local state AVD_SU_SI_REL::state, checkpoint this change to standby AMFD. However, AMFD does not updates saAmfSISUHAState untill receiving su_si assignment response. Question: (1). Whether AMFD should update the runtime attribute saAmfSISUHAState to IMM as long as local @state gets updated in implementer; to make IMM, active AMFD, standby AMFD all are synced (2). Or AMFD updates saAmfSISUHAState to IMM only if AMFD receives su_si assignment from AMFND, as it has been implemented currently for some reason (not expose the change of saAmfSISUHAState to user too early?) grep "avd_susi_update" which updates saAmfSISUHAState to IMM, there is also an inconsistency in usage. For avd_susi_mod_send() sends su_si msg and also updates saAmfSISUHAState immediately, while avd_sg_su_si_mod_snd does otherwise. Since the headless recovery relies on IMM to restore the state. If saAmfSISUHAState is not updated punctually and the node is reboot during headless stage, so after headless saAmfSISUHAState read from IMM does not fit with many other states (SG fsm, SUSI fsm, saAmfSISUHAState of the other SUSIs). My question is if doing (1) will cause any problem for normal cluster? Pending patches #1725 part 2 currently implement (1). --- Sent from sourceforge.net because opensaf-tickets@lists.sourceforge.net is subscribed to https://sourceforge.net/p/opensaf/tickets/ To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/opensaf/admin/tickets/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.-- Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot___ Opensaf-tickets mailing list Opensaf-tickets@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-tickets