Hi Mahesh See my new comments inline [Lennart]
Thanks Lennart > -----Original Message----- > From: A V Mahesh [mailto:mahesh.va...@oracle.com] > Sent: den 3 augusti 2016 08:21 > To: Lennart Lund <lennart.l...@ericsson.com>; Vu Minh Nguyen > <vu.m.ngu...@dektech.com.au>; Anders Widell > <anders.wid...@ericsson.com> > Cc: opensaf-devel@lists.sourceforge.net > Subject: Re: [PATCH 2 of 3] lgs: director Cluster Membership (CLM) > integration [#1638] V2 > > Hi Lennart, > > I just incorporated possible comments based on the current code base, > This My fist patch for LOG :) , so I just followed how existing > functions/variables organized like. > > I will considered all other inputs while doing complete refracting code > to C++ in a separate ticket soon. > > Please see also inline response [AVM] > > -AVM > > On 8/2/2016 6:30 PM, Lennart Lund wrote: > > Hi > > > > My comments for patch 2 > > > > I have found some things that should be considered in order to get as clean > code as possible: > > 1. > > Functions for handling CLM can be found in several files. It is better to > collect all CLM handling in the lgs_clm file and just include function calls > in the > rest of the code. Also all functions that shall be globally available should > have > a prototype in the corresponding lgs_clm.h file > [AVM] I will considered all your inputs while doing complete > refracting code to C++ in a separate ticket soon. [Lennart] I still think you should do this now. It will also make it easier to do the refactoring later > > 2. > > New global variables are added to the cb structure. Do not use global > variables try at least to keep the scope of state variables, flags and other > variables within the lgs_clm file. Implement setter and getter functions if > needed or even better functions that are using the variables e.g. like the > is_client_clm_member() function. This also makes it possible to make these > functions thread safe (do not use the global cb lock mutex). which means > > that this can be handled in one place instead of all over the code > [AVM] I moved the stuff that is possible based on current code base of > LOG service , i will take care of your suggestions while doing complete > refracting code to C++ in a separate ticket soon. [Lennart] As mentioned in 1) I still think you should do this now > > 3. > > Take advantage of C++ and make simpler handling of lists. It's no longer > needed to use patricia tree handling > > Maybe even create a LgsClm class? > [AVM] Same comment was give by Anders Widell as well. > I will do all NCS_PATRICIA_TREE conversion to std::map of > Log service in one go in a separate ticket soon. [Lennart] Why not now? There will be less and simpler code > > > > See also inline comments [Lennart] > > > > Thanks > > Lennart > > > >> -----Original Message----- > >> From: mahesh.va...@oracle.com [mailto:mahesh.va...@oracle.com] > >> Sent: den 2 augusti 2016 10:18 > >> To: Vu Minh Nguyen <vu.m.ngu...@dektech.com.au>; Lennart Lund > >> <lennart.l...@ericsson.com>; Anders Widell > <anders.wid...@ericsson.com> > >> Cc: opensaf-devel@lists.sourceforge.net > >> Subject: [PATCH 2 of 3] lgs: director Cluster Membership (CLM) > integration > >> [#1638] V2 > >> > >> osaf/services/saf/logsv/lgs/Makefile.am | 7 +- > >> osaf/services/saf/logsv/lgs/lgs_cb.h | 15 +++ > >> osaf/services/saf/logsv/lgs/lgs_clm.cc | 142 > >> +++++++++++++++++++++++++++++++ > >> osaf/services/saf/logsv/lgs/lgs_clm.h | 25 +++++ > >> osaf/services/saf/logsv/lgs/lgs_evt.cc | 143 > >> ++++++++++++++++++++++++++++++++ > >> osaf/services/saf/logsv/lgs/lgs_evt.h | 2 + > >> osaf/services/saf/logsv/lgs/lgs_main.cc | 28 ++++++ > >> osaf/services/saf/logsv/lgs/lgs_mds.cc | 38 ++++++++- > >> osaf/services/saf/logsv/lgs/lgs_util.cc | 83 ++++++++++++++++++ > >> 9 files changed, 480 insertions(+), 3 deletions(-) > >> > >> > >> diff --git a/osaf/services/saf/logsv/lgs/Makefile.am > >> b/osaf/services/saf/logsv/lgs/Makefile.am > >> --- a/osaf/services/saf/logsv/lgs/Makefile.am > >> +++ b/osaf/services/saf/logsv/lgs/Makefile.am > >> @@ -37,7 +37,8 @@ noinst_HEADERS = \ > >> lgs_mbcsv_v3.h \ > >> lgs_mbcsv_v5.h \ > >> lgs_recov.h \ > >> - lgs_imm_gcfg.h > >> + lgs_imm_gcfg.h \ > >> + lgs_clm.h > >> > >> osaf_execbindir = $(pkglibdir) > >> osaf_execbin_PROGRAMS = osaflogd > >> @@ -67,7 +68,8 @@ osaflogd_SOURCES = \ > >> lgs_mbcsv_v3.cc \ > >> lgs_mbcsv_v5.cc \ > >> lgs_recov.cc \ > >> - lgs_imm_gcfg.cc > >> + lgs_imm_gcfg.cc \ > >> + lgs_clm.cc > >> > >> osaflogd_LDADD = \ > >> $(top_builddir)/osaf/tools/safimm/src/libimmutil.la \ > >> @@ -75,4 +77,5 @@ osaflogd_LDADD = \ > >> $(top_builddir)/osaf/libs/saf/libSaAmf/libSaAmf.la \ > >> $(top_builddir)/osaf/libs/saf/libSaImm/libSaImmOi.la \ > >> $(top_builddir)/osaf/libs/saf/libSaImm/libSaImmOm.la \ > >> + $(top_builddir)/osaf/libs/saf/libSaClm/libSaClm.la \ > >> $(top_builddir)/osaf/libs/agents/infrastructure/rda/librda.la > >> diff --git a/osaf/services/saf/logsv/lgs/lgs_cb.h > >> b/osaf/services/saf/logsv/lgs/lgs_cb.h > >> --- a/osaf/services/saf/logsv/lgs/lgs_cb.h > >> +++ b/osaf/services/saf/logsv/lgs/lgs_cb.h > >> @@ -21,6 +21,7 @@ > >> #include <stdbool.h> > >> #include <saLog.h> > >> #include <saImmOi.h> > >> +#include <saClm.h> > >> #include <mbcsv_papi.h> > >> #include <ncs_edu_pub.h> > >> > >> @@ -55,6 +56,11 @@ typedef struct lgs_stream_list { > >> } lgs_stream_list_t; > >> > >> typedef struct { > >> + NCS_PATRICIA_NODE patnode; > >> + NODE_ID node_id; > >> +} lgs_clm_node_t; > >> + > > [Lennart] Would it be possible to not add new patricia node handling? Since > the log service is compiled as C++ code C++ list tools could be used instead? > > lgs_clm_node_find > [AVM] Same comment was give by Anders Widell as well. > I will do all NCS_PATRICIA_TREE conversion to std::map of > Log service in one go in a separate ticket soon. > >> +typedef struct { > >> NCS_PATRICIA_NODE pat_node; > >> uint32_t client_id; > >> uint32_t client_id_net; > >> @@ -73,6 +79,7 @@ typedef struct lgs_cb { > >> MDS_DEST vaddr; /* My identification in MDS > >> */ > >> SaVersionT log_version; /* The version currently supported > >> */ > >> NCS_PATRICIA_TREE client_tree; /* LGA/Library/Client > >> instantiation pat. tree */ > >> + NCS_PATRICIA_TREE clm_node_tree; /* LGA/Library/Client > >> instantiation pat. tree */ > >> SaNameT comp_name; /* Components's name LGS */ > >> SaAmfHandleT amf_hdl; /* AMF handle, obtained thru AMF > >> init */ > >> SaSelectionObjectT amfSelectionObject; /* Selection Object to > >> wait for AMF events */ > >> @@ -80,6 +87,9 @@ typedef struct lgs_cb { > >> bool is_quiesced_set; > >> SaImmOiHandleT immOiHandle; /* IMM OI handle > >> */ > >> SaSelectionObjectT immSelectionObject; /* Selection Object to > >> wait for IMM events */ > >> + SaSelectionObjectT clmSelectionObject; /* Selection Object to wait > >> for clms events */ > >> + SaClmHandleT clm_hdl; /* CLM handle, obtained through CLM init > >> */ > >> + bool clm_initialized; //For CLM init status; > > [Lennart] Why adding new globals? At least this one is not needed. This > variable could be declared locally in lgs_clm.cc only if the is_clm_init() > function is moved from lgs_util.cc to lgs_clm.cc. Try to keep CLM handling > code in one place (could be a class) and avoid global varaibles (avoid using > the > cb structure as an all-purpose "slask" global if possible) > [AVM] clm_initialized is Agent wide information this will used future > purposes like all services concurrently starting ect.. [Lennart] Ok but it can still be placed locally in the lgs_clm.c file and be handled via functions that also implement needed thread safe handling. Please do not add new global variables > > >> SaAmfHAStateT ha_state; /* present AMF HA state of the > >> component */ > >> uint32_t last_client_id; /* Value of last client_id assigned > >> */ > >> uint32_t async_upd_cnt; /* Async Update Count for Warmsync > >> */ > >> @@ -94,6 +104,7 @@ typedef struct lgs_cb { > >> down events > >> Processing */ > >> LGA_DOWN_LIST *lga_down_list_tail; > >> > >> + NCS_SEL_OBJ clm_init_sel_obj; /* Selection object for CLM > >> initialization.*/ > >> bool nid_started; /**< true if started by NID */ > >> SaUint32T scAbsenceAllowed; /* OpenSAF global configuration for > >> recovery handling */ > >> lgs_state_t lgs_recovery_state; /* Indicate current recovery state for > >> the server */ > >> @@ -106,5 +117,9 @@ typedef struct lgs_cb { > >> extern uint32_t lgs_cb_init(lgs_cb_t *); > >> extern void lgs_process_mbx(SYSF_MBX *mbx); > >> extern uint32_t lgs_stream_add(lgs_cb_t *cb, log_stream_t *stream); > >> +extern uint32_t lgs_clm_node_del(NODE_ID node_id); > >> +extern uint32_t lgs_clm_node_add(NODE_ID node_id); > >> +extern uint32_t lgs_clm_node_find(NODE_ID node_id); > >> +extern bool is_client_clm_member(NODE_ID node_id, SaVersionT > *ver); > > [Lennart] Does the function protoypes really belongs here? The code can > be found in lgs_evt.cc and lgs_util.cc but probably belongs to the lgs_clm.cc > file and the prototypes should be in the lgs_clm.h file? > [AVM] Moved. > > > >> #endif > >> diff --git a/osaf/services/saf/logsv/lgs/lgs_clm.cc > >> b/osaf/services/saf/logsv/lgs/lgs_clm.cc > >> new file mode 100644 > >> --- /dev/null > >> +++ b/osaf/services/saf/logsv/lgs/lgs_clm.cc > >> @@ -0,0 +1,142 @@ > >> +/* -*- OpenSAF -*- > >> + * > >> + * (C) Copyright 2016 The OpenSAF Foundation > >> + * > >> + * This program is distributed in the hope that it will be useful, but > >> + * WITHOUT ANY WARRANTY; without even the implied warranty of > >> MERCHANTABILITY > >> + * or FITNESS FOR A PARTICULAR PURPOSE. This file and program are > >> licensed > >> + * under the GNU Lesser General Public License Version 2.1, February > 1999. > >> + * The complete license can be accessed from the following location: > >> + * http://opensource.org/licenses/lgpl-license.php > >> + * See the Copying file included with the OpenSAF distribution for full > >> + * licensing terms. > >> + * > >> + * Author(s): Oracle > >> + * > >> + */ > >> + > >> +#include "lgs.h" > >> +#include "lgs_clm.h" > >> + > >> +/* > >> + * @brief CLM callback for tracking node membership status. > >> + * Depending upon the membership status (joining/leaving > >> cluster) > >> + * of a node, LGS will add or remove node from its data base. > >> + * A node is added when it is part of the cluster and is removed > >> + * when it leaves the cluster membership. An update of status is > >> + * sent to the clients on that node. Based on this status LGA > >> + * will decide return code for different API calls. > >> + * > >> + */ > >> +static void lgs_clm_track_cbk(const SaClmClusterNotificationBufferT_4 > >> *notificationBuffer, > >> + SaUint32T numberOfMembers, SaInvocationT invocation, > >> + const SaNameT *rootCauseEntity, const SaNtfCorrelationIdsT > >> *correlationIds, > >> + SaClmChangeStepT step, SaTimeT timeSupervision, > >> SaAisErrorT error) > >> +{ > >> + NODE_ID node_id; > >> + SaClmClusterChangesT cluster_change; > >> + SaBoolT is_member; > >> + uint32_t i = 0; > >> + > >> + TRACE_ENTER2("'%llu' '%u' '%u'", invocation, step, error); > >> + if (error != SA_AIS_OK) { > >> + TRACE_1("Error received in ClmTrackCallback"); > >> + goto done; > >> + } > >> + lgs_cb->clm_initialized = true; > >> + > >> + for (i = 0; i < notificationBuffer->numberOfItems; i++) { > >> + switch(step) { > >> + case SA_CLM_CHANGE_COMPLETED: > >> + is_member = notificationBuffer- > >>> notification[i].clusterNode.member; > >> + node_id = notificationBuffer- > >>> notification[i].clusterNode.nodeId; > >> + if (lgs_clm_node_find(node_id) == > >> NCSCC_RC_SUCCESS) { > >> + TRACE_1("'%x' is present in LGS db", > >> node_id); > >> + if (!is_member) { > >> + TRACE("CLM Node : %x Left > >> the cluster", node_id); > >> + cluster_change = > >> SA_CLM_NODE_LEFT; > >> + if > >> (lgs_clm_node_del(node_id) == NCSCC_RC_SUCCESS){ > >> + if (lgs_cb->ha_state > >> == SA_AMF_HA_ACTIVE) > >> + > >> send_clm_node_status_change(cluster_change, node_id); > >> + } > >> + } > >> + } else { > >> + TRACE_1("'%x' is not present in LGS > >> db", node_id); > >> + if (is_member) { > >> + TRACE("CLM Node : %x > >> Joined the cluster", node_id); > >> + cluster_change = > >> SA_CLM_NODE_JOINED; > >> + > >> if(lgs_clm_node_add(node_id) == NCSCC_RC_SUCCESS) { > >> + if (lgs_cb->ha_state > >> == SA_AMF_HA_ACTIVE) > >> + > >> send_clm_node_status_change(cluster_change, node_id); > >> + } > >> + } > >> + } > >> + break; > >> + default: > >> + break; > >> + } > >> + } > >> +done: > >> + TRACE_LEAVE(); > >> + return; > >> +} > >> + > >> +static const SaClmCallbacksT_4 clm_callbacks = { > >> + 0, > >> + lgs_clm_track_cbk /*saClmClusterTrackCallback*/ > >> +}; > >> + > >> +/* > >> + * @brief Registers with the CLM service (B.04.01). > >> + * > >> + * @return SaAisErrorT > >> + */ > >> +void *lgs_clm_init_thread(void *cb) > >> +{ > >> + static SaVersionT clmVersion = { 'B', 0x04, 0x01 }; > >> + lgs_cb_t *_lgs_cb = (lgs_cb_t *) cb; > >> + SaAisErrorT rc = SA_AIS_OK; > >> + TRACE_ENTER(); > >> + rc = saClmInitialize_4(&_lgs_cb->clm_hdl, &clm_callbacks, > >> &clmVersion); > >> + if (rc != SA_AIS_OK) { > >> + LOG_ER("saClmInitialize failed with error: %d", rc); > >> + TRACE_LEAVE(); > >> + exit(EXIT_FAILURE); > >> + } > >> + rc = saClmSelectionObjectGet(_lgs_cb->clm_hdl, &lgs_cb- > >>> clmSelectionObject); > >> + if (rc != SA_AIS_OK) { > >> + LOG_ER("saClmSelectionObjectGet failed with error: %d", > >> rc); > >> + TRACE_LEAVE(); > >> + exit(EXIT_FAILURE); > >> + } > >> + //TODO:subscribe for SA_TRACK_START_STEP also. > >> + rc = saClmClusterTrack_4(_lgs_cb->clm_hdl, (SA_TRACK_CURRENT | > >> SA_TRACK_CHANGES), NULL); > >> + if (rc != SA_AIS_OK) { > >> + LOG_ER("saClmClusterTrack failed with error: %d", rc); > >> + TRACE_LEAVE(); > >> + exit(EXIT_FAILURE); > >> + } > >> + TRACE("CLM Initialization SUCCESS......"); > >> + TRACE_LEAVE(); > >> + return NULL; > >> +} > >> + > >> +/* > >> + * @brief Creates a thread to initialize with CLM. > >> + */ > >> +void lgs_init_with_clm(void) > >> +{ > >> + pthread_t thread; > >> + pthread_attr_t attr; > >> + TRACE_ENTER(); > >> + > >> + pthread_attr_init(&attr); > >> + pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED); > >> + > >> + if (pthread_create(&thread, &attr, lgs_clm_init_thread, lgs_cb) != 0) > >> { > >> + LOG_ER("pthread_create FAILED: %s", strerror(errno)); > >> + exit(EXIT_FAILURE); > >> + } > >> + pthread_attr_destroy(&attr); > >> + TRACE_LEAVE(); > >> +} > >> diff --git a/osaf/services/saf/logsv/lgs/lgs_clm.h > >> b/osaf/services/saf/logsv/lgs/lgs_clm.h > >> new file mode 100644 > >> --- /dev/null > >> +++ b/osaf/services/saf/logsv/lgs/lgs_clm.h > >> @@ -0,0 +1,25 @@ > >> +/* -*- OpenSAF -*- > >> + * > >> + * (C) Copyright 2016 The OpenSAF Foundation > >> + * > >> + * This program is distributed in the hope that it will be useful, but > >> + * WITHOUT ANY WARRANTY; without even the implied warranty of > >> MERCHANTABILITY > >> + * or FITNESS FOR A PARTICULAR PURPOSE. This file and program are > >> licensed > >> + * under the GNU Lesser General Public License Version 2.1, February > 1999. > >> + * The complete license can be accessed from the following location: > >> + * http://opensource.org/licenses/lgpl-license.php > >> + * See the Copying file included with the OpenSAF distribution for full > >> + * licensing terms. > >> + * > >> + * Author(s): Oracle > >> + * > >> + */ > >> + > >> +#include <saClm.h> > >> + > >> +#define m_LGS_GET_NODE_ID_FROM_ADEST(adest) (NODE_ID) > >> ((uint64_t)adest >> 32) > >> + > >> +/* > >> + * @brief Creates a thread to initialize with CLM. > >> + */ > >> +void lgs_init_with_clm(void); > >> diff --git a/osaf/services/saf/logsv/lgs/lgs_evt.cc > >> b/osaf/services/saf/logsv/lgs/lgs_evt.cc > >> --- a/osaf/services/saf/logsv/lgs/lgs_evt.cc > >> +++ b/osaf/services/saf/logsv/lgs/lgs_evt.cc > >> @@ -23,6 +23,7 @@ > >> #include "lgs_mbcsv_v2.h" > >> #include "lgs_recov.h" > >> #include "lgs_imm_gcfg.h" > >> +#include "lgs_clm.h" > >> > >> /* Macro to validate the version */ > >> #define m_LOG_VER_IS_VALID(ver) \ > >> @@ -56,6 +57,94 @@ LGSV_LGS_LGA_API_MSG_HANDLER lgs_lga_api > >> }; > >> > >> /** > >> + * Name : lgs_clm_node_tree_init > >> + * Description : This routine is used to initialize the clm_node_tree > >> + * Arguments : lgs_cb - pointer to the lgs Control Block > >> + * Return Values : NCSCC_RC_SUCCESS/NCSCC_RC_FAILURE > >> + * Notes : None > >> + */ > >> +uint32_t lgs_clm_node_tree_init(lgs_cb_t *lgs_cb) > >> +{ > >> + TRACE_ENTER(); > >> + NCS_PATRICIA_PARAMS param; > >> + memset(¶m, 0, sizeof(NCS_PATRICIA_PARAMS)); > >> + > >> + param.key_size = sizeof(NODE_ID); > >> + if (ncs_patricia_tree_init(&lgs_cb->clm_node_tree, ¶m) != > >> NCSCC_RC_SUCCESS) { > >> + LOG_ER("lgs patricia tee init failed for clm_node_tree"); > >> + return NCSCC_RC_FAILURE; > >> + } > >> + TRACE_LEAVE(); > >> + return NCSCC_RC_SUCCESS; > >> +} > >> + > >> +/** > >> + * Name : lgs_clm_node_find > >> + * Description : This routine finds the clm_node . > >> + * Arguments : node_id - CLM Node id > >> + * Return Values : NCSCC_RC_SUCCESS/NCSCC_RC_FAILURE > >> + * Notes : None > >> + */ > >> +uint32_t lgs_clm_node_find(NODE_ID node_id) > >> +{ > >> + uint32_t rc = NCSCC_RC_FAILURE; > >> + lgs_clm_node_t *clm_node = (lgs_clm_node_t *) > >> + ncs_patricia_tree_get(&lgs_cb->clm_node_tree, (uint8_t > >> *)&node_id); > >> + > >> + if (clm_node != NULL) > >> + rc = NCSCC_RC_SUCCESS; > >> + else > >> + TRACE("node_id find in DB failed : %x",node_id); > >> + > >> + return rc; > >> +} > >> + > >> +/** > >> + * Name : lgs_clm_node_add > >> + * Description : This routine adds the new node to clm_node_tree. > >> + * Arguments : node_id - CLM Node id > >> + * Return Values : NCSCC_RC_SUCCESS/NCSCC_RC_FAILURE > >> + * Notes : None > >> + */ > >> +uint32_t lgs_clm_node_add(NODE_ID node_id) > >> +{ > >> + TRACE_ENTER(); > >> + uint32_t rc = NCSCC_RC_FAILURE; > >> + lgs_clm_node_t *clm_node; > >> + > >> + clm_node = (lgs_clm_node_t *) malloc(sizeof(lgs_clm_node_t)); > >> + clm_node->node_id = node_id; > >> + clm_node->patnode.key_info = (uint8_t *)&clm_node->node_id; > >> + rc = ncs_patricia_tree_add(&lgs_cb->clm_node_tree, > >> (NCS_PATRICIA_NODE *)&clm_node->patnode); > >> + if (rc != NCSCC_RC_SUCCESS) > >> + TRACE("node_id add to DB failed : %x",node_id); > >> + TRACE_LEAVE(); > >> + return rc; > >> +} > >> + > >> +/** > >> + * Name : lgs_clm_node_del > >> + * Description : Function to Delete the clm_node. > >> + * Arguments : node_id - CLM Node id > >> + * Return Values : NCSCC_RC_SUCCESS/NCSCC_RC_FAILURE > >> + * Notes : None. > >> + */ > >> +uint32_t lgs_clm_node_del(NODE_ID node_id) > >> +{ > >> + TRACE_ENTER(); > >> + uint32_t rc = NCSCC_RC_FAILURE; > >> + > >> + lgs_clm_node_t *clm_node = (lgs_clm_node_t *) > >> + ncs_patricia_tree_get(&lgs_cb->clm_node_tree, (uint8_t > >> *)&node_id); > >> + if (clm_node != NULL) > >> + rc = ncs_patricia_tree_del(&lgs_cb->clm_node_tree, > >> (NCS_PATRICIA_NODE *)&clm_node->patnode); > >> + if (rc != NCSCC_RC_SUCCESS) > >> + TRACE("node_id delete to DB failed : %x",node_id); > >> + TRACE_LEAVE(); > >> + return rc; > >> +} > >> + > >> +/** > >> * Get client record from client ID > >> * @param client_id > >> * > >> @@ -583,6 +672,9 @@ uint32_t lgs_cb_init(lgs_cb_t *lgs_cb) > >> lgs_cb->amfSelectionObject = -1; > >> lgs_cb->immSelectionObject = -1; > >> lgs_cb->mbcsv_sel_obj = -1; > >> + lgs_cb->clm_hdl = 0; > >> + lgs_cb->clm_initialized = false; > >> + lgs_cb->clmSelectionObject = -1; > >> > >> /* Assign Version. Currently, hardcoded, This will change later */ > >> lgs_cb->log_version.releaseCode = LOG_RELEASE_CODE; > >> @@ -598,6 +690,12 @@ uint32_t lgs_cb_init(lgs_cb_t *lgs_cb) > >> if (NCSCC_RC_SUCCESS != ncs_patricia_tree_init(&lgs_cb- > >>> client_tree, ®_param)) > >> return NCSCC_RC_FAILURE; > >> > >> + /* Initialize patricia tree for CLM Node list */ > >> + if (NCSCC_RC_SUCCESS != lgs_clm_node_tree_init(lgs_cb)) { > >> + LOG_ER("LGS: ncs_patricia_tree_init FAILED"); > >> + return NCSCC_RC_FAILURE; > >> + } > >> + > >> done: > >> TRACE_LEAVE(); > >> return rc; > >> @@ -637,6 +735,12 @@ static uint32_t proc_initialize_msg(lgs_ > >> goto snd_rsp; > >> } > >> > >> + if (is_client_clm_member(evt->fr_node_id, version) != true){ > >> + ais_rc = SA_AIS_ERR_UNAVAILABLE; > >> + TRACE("client not a CLM member FAILED"); > >> + goto snd_rsp; > >> + } > >> + > >> if ((client = lgs_client_new(evt->fr_dest, 0, NULL)) == NULL) { > >> ais_rc = SA_AIS_ERR_NO_MEMORY; > >> goto snd_rsp; > >> @@ -1451,3 +1555,42 @@ void lgs_process_mbx(SYSF_MBX *mbx) > >> lgs_evt_destroy(msg); > >> } > >> } > >> + > >> +/** > >> + * @brief Sends CLM membership status of the node to all the clients > >> + * on the node except A11 clients. > >> + * @param cluster_change (CLM membership status of node). > >> + * @param NCS node_id. > >> + * @return NCSCC_RC_SUCCESS/NCSCC_RC_FAILURE. > >> + */ > >> +uint32_t > >> send_cluster_membership_msg_to_clients(SaClmClusterChangesT > >> cluster_change, NODE_ID node_id) { > >> + uint32_t rc = NCSCC_RC_SUCCESS; > >> + log_client_t *rp = NULL; > >> + uint32_t client_id_net; > >> + > >> + TRACE_ENTER(); > >> + TRACE_3("node_id: %x, change:%u", node_id, cluster_change); > >> + > >> + rp = reinterpret_cast<log_client_t > >> *>(ncs_patricia_tree_getnext(&lgs_cb->client_tree, NULL)); > >> + > >> + while (rp != NULL) { > >> + /** Store the client_id_net for get Next */ > >> + client_id_net = rp->client_id_net; > >> + NODE_ID tmp_node_id = > >> m_LGS_GET_NODE_ID_FROM_ADEST(rp->mds_dest); > >> + //Do not send to A11 client. Send only to specific Node > >> + if (tmp_node_id == node_id) > >> + rc = send_clm_node_status_lib(cluster_change, rp- > >>> client_id, rp->mds_dest); > >> + > >> + rp = reinterpret_cast<log_client_t > >> *>(ncs_patricia_tree_getnext( > >> + &lgs_cb->client_tree, > >> reinterpret_cast<uint8_t *>(&client_id_net))); > >> + } > >> + > >> + TRACE_LEAVE(); > >> + return rc; > >> +} > >> + > >> +uint32_t send_clm_node_status_change(SaClmClusterChangesT > >> cluster_change, NODE_ID node_id) { > >> + return (send_cluster_membership_msg_to_clients(cluster_change, > >> node_id)); > >> + > >> +} > >> + > >> diff --git a/osaf/services/saf/logsv/lgs/lgs_evt.h > >> b/osaf/services/saf/logsv/lgs/lgs_evt.h > >> --- a/osaf/services/saf/logsv/lgs/lgs_evt.h > >> +++ b/osaf/services/saf/logsv/lgs/lgs_evt.h > >> @@ -77,6 +77,8 @@ extern bool lgs_lga_entry_valid(lgs_cb_t > >> extern uint32_t lgs_remove_lga_down_rec(lgs_cb_t *cb, MDS_DEST > >> mds_dest); > >> extern void lgs_send_write_log_ack(uint32_t client_id, SaInvocationT > >> invocation, SaAisErrorT error, MDS_DEST mds_dest); > >> extern void lgs_free_write_log(const lgsv_write_log_async_req_t > *param); > >> +extern uint32_t send_clm_node_status_lib(SaClmClusterChangesT > >> cluster_change, unsigned int client_id, MDS_DEST mdsDest); > >> +extern uint32_t send_clm_node_status_change(SaClmClusterChangesT > >> cluster_change, NODE_ID node_id); > >> > >> SaAisErrorT create_new_app_stream( > >> lgsv_stream_open_req_t *open_sync_param, > >> diff --git a/osaf/services/saf/logsv/lgs/lgs_main.cc > >> b/osaf/services/saf/logsv/lgs/lgs_main.cc > >> --- a/osaf/services/saf/logsv/lgs/lgs_main.cc > >> +++ b/osaf/services/saf/logsv/lgs/lgs_main.cc > >> @@ -38,6 +38,7 @@ > >> #include "osaf_utility.h" > >> #include "lgs_recov.h" > >> #include "immutil.h" > >> +#include "lgs_clm.h" > >> > >> /* > >> > ========================================================== > >> ============== > >> * DEFINITIONS > >> @@ -49,6 +50,7 @@ enum { > >> FD_MBCSV, > >> FD_MBX, > >> FD_CLTIMER, > >> + FD_CLM, > >> FD_IMM, /* Must be the last in the fds array */ > >> FD_NUM > >> }; > >> @@ -332,6 +334,13 @@ static uint32_t log_initialize(void) > >> goto done; > >> } > >> > >> + /* Create a CLM selection object */ > >> + if ((rc = ncs_sel_obj_create(&lgs_cb->clm_init_sel_obj)) != > >> NCSCC_RC_SUCCESS) > >> + { > >> + LOG_ER("lgsv: CLM ncs_sel_obj_create failed"); > >> + goto done; > >> + } > >> + > >> /* > >> * Initialize a signal handler that will use the selection object. > >> * The signal is sent from our script when AMF does instantiate. > >> @@ -501,6 +510,8 @@ int main(int argc, char *argv[]) > >> fds[FD_IMM].fd = lgs_cb->immSelectionObject; > >> fds[FD_IMM].events = POLLIN; > >> > >> + lgs_cb->clmSelectionObject = lgs_cb->clm_init_sel_obj.rmv_obj; > >> + > >> while (1) { > >> if (cltimer_fd < 0 && log_rtobj_list_no() != 0) { > >> /* Needed only if any "lost" objects are found > >> @@ -513,6 +524,8 @@ int main(int argc, char *argv[]) > >> fds[FD_CLTIMER].events = POLLIN; > >> fds[FD_MBCSV].fd = lgs_cb->mbcsv_sel_obj; > >> fds[FD_MBCSV].events = POLLIN; > >> + fds[FD_CLM].fd = lgs_cb->clmSelectionObject; > >> + fds[FD_CLM].events = POLLIN; > >> > >> /* Protect since the reinit thread may be in the process of > >> * changing the values > >> @@ -567,6 +580,21 @@ int main(int argc, char *argv[]) > >> } > >> } > >> > >> + if (fds[FD_CLM].revents & POLLIN) { > >> + if (lgs_cb->clm_hdl != 0) { > >> + if ((error = saClmDispatch(lgs_cb->clm_hdl, > >> SA_DISPATCH_ALL)) != SA_AIS_OK) { > >> + LOG_ER("saClmDispatch failed: %u", > >> error); > >> + break; > >> + } > >> + } else { > >> + TRACE("init CLM "); > >> + ncs_sel_obj_rmv_ind(&lgs_cb- > >>> clm_init_sel_obj, true, true); > >> + ncs_sel_obj_destroy(&lgs_cb- > >>> clm_init_sel_obj); > >> + lgs_cb->clmSelectionObject = -1; > >> + lgs_init_with_clm(); > >> + } > >> + } > >> + > >> if (fds[FD_CLTIMER].revents & POLLIN) { > >> /* To avoid 'stray objects', after a timeout all runtime > >> * objects that has not been restored shall be deleted > >> diff --git a/osaf/services/saf/logsv/lgs/lgs_mds.cc > >> b/osaf/services/saf/logsv/lgs/lgs_mds.cc > >> --- a/osaf/services/saf/logsv/lgs/lgs_mds.cc > >> +++ b/osaf/services/saf/logsv/lgs/lgs_mds.cc > >> @@ -763,11 +763,19 @@ static uint32_t mds_enc(struct ncsmds_ca > >> goto err; > >> } > >> ncs_encode_32bit(&p8, msg- > >>> info.cbk_info.write_cbk.error); > >> + TRACE_8("LGSV_WRITE_LOG_CALLBACK_IND"); > >> + } else if (msg->info.cbk_info.type == > >> LGSV_CLM_NODE_STATUS_CALLBACK) { > >> + p8 = ncs_enc_reserve_space(uba, 4); > >> + if (!p8) { > >> + TRACE("ncs_enc_reserve_space failed"); > >> + goto err; > >> + } > >> + ncs_encode_32bit(&p8, msg- > >>> info.cbk_info.clm_node_status_cbk.clm_node_status); > >> + TRACE_8("LGSV_CLM_NODE_STATUS_CALLBACK"); > >> } else { > >> TRACE("unknown callback type %d", msg- > >>> info.cbk_info.type); > >> goto err; > >> } > >> - TRACE_8("LGSV_WRITE_LOG_CALLBACK_IND"); > >> } else { > >> TRACE("unknown msg type %d", msg->type); > >> goto err; > >> @@ -1167,6 +1175,16 @@ static uint32_t mds_svc_event(struct ncs > >> osafassert(rc == NCSCC_RC_SUCCESS); > >> } > >> } > >> + } else if (info->info.svc_evt.i_svc_id == NCSMDS_SVC_ID_AVD) { > >> + if (info->info.svc_evt.i_change == NCSMDS_UP) { > >> + TRACE_8("MDS UP dest: %" PRIx64 ", node ID: %x, > >> svc_id: %d", > >> + info->info.svc_evt.i_dest, info- > >>> info.svc_evt.i_node_id, info->info.svc_evt.i_svc_id); > >> + //Subscribed for only INTRA NODE, only one ADEST > >> will come. > >> + if (m_MDS_DEST_IS_AN_ADEST(info- > >>> info.svc_evt.i_dest)) { > >> + TRACE_8("AVD ADEST UP"); > >> + ncs_sel_obj_ind(&lgs_cb->clm_init_sel_obj); > >> + } > >> + } > >> } > >> > >> done: > >> @@ -1334,6 +1352,24 @@ uint32_t lgs_mds_init(lgs_cb_t *cb, SaAm > >> return rc; > >> } > >> > >> + > >> + svc = NCSMDS_SVC_ID_AVD; > >> + /* Now subscribe for AVD events in MDS. This will be > >> + used for CLM registration.*/ > >> + memset(&mds_info, '\0', sizeof(NCSMDS_INFO)); > >> + mds_info.i_mds_hdl = cb->mds_hdl; > >> + mds_info.i_svc_id = NCSMDS_SVC_ID_LGS; > >> + mds_info.i_op = MDS_SUBSCRIBE; > >> + mds_info.info.svc_subscribe.i_scope = > NCSMDS_SCOPE_INTRANODE; > >> + mds_info.info.svc_subscribe.i_num_svcs = 1; > >> + mds_info.info.svc_subscribe.i_svc_ids = &svc; > >> + > >> + rc = ncsmds_api(&mds_info); > >> + if (rc != NCSCC_RC_SUCCESS) { > >> + LOG_ER("MDS for AVD subscribe FAILED"); > >> + return rc; > >> + } > >> + > >> TRACE_LEAVE(); > >> return rc; > >> } > >> diff --git a/osaf/services/saf/logsv/lgs/lgs_util.cc > >> b/osaf/services/saf/logsv/lgs/lgs_util.cc > >> --- a/osaf/services/saf/logsv/lgs/lgs_util.cc > >> +++ b/osaf/services/saf/logsv/lgs/lgs_util.cc > >> @@ -415,6 +415,89 @@ void lgs_send_write_log_ack(uint32_t cli > >> } > >> > >> /** > >> + * @brief Send Membership status of node to a lib on that node. > >> + * > >> + * @param SaClmClusterChangesT (CLM status of node) > >> + * @param client_id > >> + * @param mdsDest of client > >> + * > >> + * @return NCSCC_RC_SUCCESS/NCSCC_RC_FAILURE. > >> + */ > >> +uint32_t send_clm_node_status_lib(SaClmClusterChangesT > >> cluster_change, unsigned int client_id, MDS_DEST mdsDest) > >> +{ > >> + uint32_t rc = NCSCC_RC_SUCCESS; > >> + NCSMDS_INFO mds_info = {0}; > >> + lgsv_msg_t msg; > >> + > >> + TRACE_ENTER(); > >> + TRACE_3("change:%u, client_id: %u", cluster_change, client_id); > >> + > >> + memset(&msg, 0, sizeof(lgsv_msg_t)); > >> + msg.type = LGSV_LGS_CBK_MSG; > >> + msg.info.cbk_info.type = LGSV_CLM_NODE_STATUS_CALLBACK; > >> + msg.info.cbk_info.lgs_client_id = client_id; > >> + msg.info.cbk_info.inv = 0; > >> + msg.info.cbk_info.clm_node_status_cbk.clm_node_status = > >> cluster_change; > >> + > >> + mds_info.i_mds_hdl = lgs_cb->mds_hdl; > >> + mds_info.i_svc_id = NCSMDS_SVC_ID_LGS; > >> + mds_info.i_op = MDS_SEND; > >> + mds_info.info.svc_send.i_msg = &msg; > >> + mds_info.info.svc_send.i_to_svc = NCSMDS_SVC_ID_LGA; > >> + mds_info.info.svc_send.i_priority = MDS_SEND_PRIORITY_HIGH; > >> + mds_info.info.svc_send.i_sendtype = MDS_SENDTYPE_SND; > >> + mds_info.info.svc_send.info.snd.i_to_dest = mdsDest; > >> + > >> + rc = ncsmds_api(&mds_info); > >> + if (rc != NCSCC_RC_SUCCESS) > >> + LOG_NO("Failed (%u) to send of WRITE ack to: %" PRIx64, rc, > >> mdsDest); > >> + > >> + TRACE_LEAVE(); > >> + return rc; > >> +} > >> + > >> +/** > >> + * @brief Checks if LGSV has already initialized with CLM service. > >> + * > >> + * @return true/false. > >> + */ > >> +bool is_clm_init() > >> +{ > >> + return (((lgs_cb->clm_hdl != 0) && (lgs_cb->clm_initialized == > >> true)) > ? > >> true : false); > >> +} > >> + > >> +/** > >> + * @brief Checks CLM membership status of a client. > >> + * A.02.01 clients are always CLM member. > >> + * @param Client MDS_DEST > >> + * @param Client saf version. > >> + * @return NCSCC_RC_SUCCESS/NCSCC_RC_FAILURE. > >> + */ > >> +bool is_client_clm_member(NODE_ID node_id, SaVersionT *ver) { > >> + > >> + //Before CLM init all clients are clm member. > >> + if (is_clm_init() == false) > >> + return true; > >> + > >> + TRACE("client Version: %d.%d.%d", ver->releaseCode, ver- > >majorVersion, > >> ver->minorVersion); > >> + //CLM integration is supported from A.02.02. So old clients A.02.01 are > >> always clm member. > >> + if ((ver->releaseCode == LOG_RELEASE_CODE_0) && > >> + (ver->majorVersion == LOG_MAJOR_VERSION_0) && > >> + (ver->minorVersion == LOG_MINOR_VERSION_0)) > >> + return true; > >> + /* > >> + It means CLM initialization is successful and this is atleast a > >> A.02.02 > client. > >> + So check CLM membership status of client's node. > >> + */ > >> + if (lgs_clm_node_find(node_id) != NCSCC_RC_SUCCESS) > >> + return false; > >> + else > >> + return true; > >> +} > >> + > >> + > >> + > >> +/** > >> * Free all dynamically allocated memory for a WRITE > >> * @param param > >> */ ------------------------------------------------------------------------------ _______________________________________________ Opensaf-devel mailing list Opensaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-devel
