Dear Anders and Mahesh, Thank you very much for your comments.
I would like to clarify the backward compatible satisfaction in this case. We have `small format shm` that is used from beginning (version 1). We also have `big format shm` that is introduced with Long DN feature (changeset 7949:815c56c74d18) (version 2). The question is: Do we need to make modification that compatible with both 2 kind of shm or just the original small one and complete remove the big one? As you might know, the 2 SHMs approach comes before this SHM swapping ideal but I postpone implementing it because of the complicated logic for working with both 2 old versions. If we don't need to care about version 2, I might introduce cleaner code solution. Thank you and best regards, Hoang -----Original Message----- From: A V Mahesh [mailto:mahesh.va...@oracle.com] Sent: Tuesday, October 18, 2016 3:51 PM To: Anders Widell <anders.wid...@ericsson.com>; Vo Minh Hoang <hoang.m...@dektech.com.au> Cc: opensaf-devel@lists.sourceforge.net Subject: Re: [PATCH 1 of 1] cpnd: use shared memory based on ckpt name length [#2108] Hi Hoang, Swamping SHM looks more complex logic to me while accessing old & new shm in transit , and it will create more issue and will take some time to stabilize. Let us explore other options like Anders Widell suggested or any other, which can be simple and avoids the SHM swap. I assessed these issues and I was reluctant to have Long DN for Ckpt service , where their is no much piratical use case. -AVM On 10/18/2016 1:57 PM, Anders Widell wrote: > Maybe it is better to create a second shm containing a fixed-size > record of the remaining 2048-256 bytes of the DN? Then you will not > have to convert the shm format and creating a new shm segment should > be very quick. > > regards, > > Anders Widell > > > On 10/18/2016 09:29 AM, Vo Minh Hoang wrote: >> Dear Mahesh, >> >> Sorry I miss-sending incomplete email. >> This is full version. >> ------ >> I would like to send my answer to 2 of your concerning points in >> compound. >> >> Based on my understand, a client command affects shared mem by following >> behavior: >> >> Client --> CPA ==> CPND (1) ==> CPD (active) ==> CPND (has replica) (2) >> ----> update shm (3) >> >> When: >> --> Synchronous >> ==> Asynchronous >> (1) and (2) has same behavior to update shm and store pointer to shm >> (3) The modification only take place here include swapping shm and >> update >> pointers >> >> So even there are multiple call from multiple client at a time, CPND >> update >> shm in sequence. So just after the first request swaps shm, the second >> request could access shm. There is not case that 2 requests access >> shm at >> the same time. >> When shm already storing data, in swapping, CPND will update pointer >> so the >> next request that accesses old data can still work with updated >> pointer with >> same behavior. >> >> Thank you and best regards, >> Hoang >> >> -----Original Message----- >> From: Vo Minh Hoang [mailto:hoang.m...@dektech.com.au] >> Sent: Tuesday, October 18, 2016 2:15 PM >> To: 'A V Mahesh' <mahesh.va...@oracle.com> >> Cc: 'anders.wid...@ericsson.com' <anders.wid...@ericsson.com>; >> 'opensaf-devel@lists.sourceforge.net' >> <opensaf-devel@lists.sourceforge.net> >> Subject: RE: [PATCH 1 of 1] cpnd: use shared memory based on ckpt name >> length [#2108] >> >> Dear Mahesh, >> >> I would like to send my answer to 2 of your concerning points in >> compound. >> >> Based on my understand, a client command affects shared mem by following >> behavior: >> >> Client --> CPA ==> CPND (1) ==> CPD (active) ==> CPND (has replica) (2) >> ----> update shm (3) >> >> When: >> --> >> >> >> >> -----Original Message----- >> From: A V Mahesh [mailto:mahesh.va...@oracle.com] >> Sent: Tuesday, October 18, 2016 1:10 PM >> To: Vo Minh Hoang <hoang.m...@dektech.com.au> >> Cc: anders.wid...@ericsson.com; opensaf-devel@lists.sourceforge.net >> Subject: Re: [PATCH 1 of 1] cpnd: use shared memory based on ckpt name >> length [#2108] >> >> Hi Hoang, >> >> >> On 10/18/2016 11:24 AM, Vo Minh Hoang wrote: >>> Dear Mahesh, >>> >>>>> [AVM] A non-collated Ckpt will have two replicas on both Active and >>> standby. >>> Each node will receive one CPND_EVT_D2ND_CKPT_CREATE message so it >>> handles >>> swapping itself and does not affect each other nor another. >> [AVM] I was taking about existing, swapping of existing `small >> format shm` >> not not new create request , where the ckpt is already opened >> multiple nodes with ALL option. >>>>> [AVM] piratically we can have large size data & transit time, if ckt >> pat >>> has large data sham is file I/O operation >>>>> not middle-ware controlled activity , swap time will >>>>> vary >>> depending on system. >>> I am agree that this modification affects performance of create/open >>> function so it need performance acceptance verification. >>> Fortunately, shared mem is on memory so it is not heavily depend on >>> OS or >>> file system (unless on swap memory area). >>> Maybe I am not understand your ideal here but I have not found a clear >>> reason of handling try-again. >> [AVM] say for example an application is writing in a loop to old `small >> format shm`, >> at that moment you started conversation of old `small format shm` to >> new `big format` >> >> -AVM >> >>> Thank you and best regards, >>> Hoang >>> >>> -----Original Message----- >>> From: A V Mahesh [mailto:mahesh.va...@oracle.com] >>> Sent: Tuesday, October 18, 2016 12:14 PM >>> To: Vo Minh Hoang <hoang.m...@dektech.com.au> >>> Cc: anders.wid...@ericsson.com; opensaf-devel@lists.sourceforge.net >>> Subject: Re: [PATCH 1 of 1] cpnd: use shared memory based on ckpt name >>> length [#2108] >>> >>> Hi Hoan, >>> >>> >>> On 10/18/2016 9:59 AM, Vo Minh Hoang wrote: >>>> Dear Mahesh, >>>> >>>> Thank you very much for your comments. >>>> >>>> I would like to explain my understanding and reason for this solution. >>>> Please correct me if I am wrong. >>>> >>>> - This memory swapping works on single node alone, it will occur >>>> maximum once per node in open/create checkpoint process. >>>> - This swapping action just takes place in nodes that meet condition >>>> and does not affect other node. >>> [AVM] A non-collated Ckpt will have two replicas on both Active and >> standby >>> . >>>> - CPND handles open/create processes atomically in sequence in one >>>> thread only. >>>> >>>> Because of that I think it is unnecessary to implement thread >>>> synchronizing or `try-again` handling. >>> [AVM] piratically we can have large size data & transit time, if >>> ckt pat >>> has large data sham is file I/O operation >>> not middle-ware controlled activity , swap time will >>> vary >>> depending on system. >>>> Sincerely, >>>> Hoang >>>> >>>> -----Original Message----- >>>> From: A V Mahesh [mailto:mahesh.va...@oracle.com] >>>> Sent: Tuesday, October 18, 2016 10:48 AM >>>> To: Vo Minh Hoang <hoang.m...@dektech.com.au> >>>> Cc: anders.wid...@ericsson.com; opensaf-devel@lists.sourceforge.net >>>> Subject: Re: [PATCH 1 of 1] cpnd: use shared memory based on ckpt name >>>> length [#2108] >>>> >>>> Hi Hoang, >>>> >>>> On 10/13/2016 12:44 PM, Vo Minh Hoang wrote: >>>>> No, old checkpoint data is converted to `big format`. >>>>> So all of them will be stored in `big format`. >>>> [AVM] This approach is introducing NEW transit , so far application >>>> are aware of switch-over & fail-over transit and TRY-AGAIN is >>>> expected only in those case , now this solution is introducing a new >>>> transit for the application which are accessioning the old (by the >>>> way this patch didn't implemented TRY-AGAIN when shared memory >>>> swapping action occurring) >>>> >>>> `small format shm`, up on some application creating `big format` ( >>>> application impacting the HA behavior ) >>>> not sure about the solution approach need to discussed ! >>>> >>>> -AVM >>>> >>>> On 10/13/2016 12:44 PM, Vo Minh Hoang wrote: >>>>> Dear Mahesh, >>>>> >>>>> Because of keeping the consistent working behavior of existing >>>>> function, only 1 shared memory at a time. If shared memory swapping >>>>> action occurs, a new shared memory will replace old one. >>>>> >>>>> Here is the detailed answers to your questions: >>>>>>> -The existing `small format shm` will continue to be small , is >>>>>>> that >>>>> right ? >>>>>>> -Only newly created longDN checkpoint will be in `big format shm`, >>>>>>> is >>>>> that right ? >>>>> No, old checkpoint data is converted to `big format`. >>>>> So all of them will be stored in `big format`. >>>>> >>>>>>> - what will be the format of newly joined the PL-5 opens an >>>>>>> existing >>>>> `small format shm` >>>>> PL-5 still use `small format`. >>>>> Only when a long DN replica is added in this node, the shared memory >>>>> is converted to `big format`. >>>>>>> the what will be the new replica on new node `small format >>>>>>> shm` >>>>>>> or `big >>>>> format shm` ? >>>>> This implementation only affect the `header` shared memory >>>>> (opensaf_CPND_CHECKPOINT_INFO_nodeid). It do not change replica >>>>> shared memory (opensaf_ckptname_nodeid_n). >>>>> >>>>> About testing, because of above specification, I tested: >>>>> - start new node >>>>> - restart ckptnd with existing small shm >>>>> - restart ckptnd with existing big shm >>>>> - create first long dn (check all node) >>>>> >>>>> Thank you and best regards, >>>>> Hoang >>>>> >>>>> -----Original Message----- >>>>> From: A V Mahesh [mailto:mahesh.va...@oracle.com] >>>>> Sent: Thursday, October 13, 2016 1:33 PM >>>>> To: Hoang Vo <hoang.m...@dektech.com.au>; anders.wid...@ericsson.com >>>>> Cc: opensaf-devel@lists.sourceforge.net >>>>> Subject: Re: [PATCH 1 of 1] cpnd: use shared memory based on ckpt >>>>> name length [#2108] >>>>> >>>>> Hi Hoang, >>>>> >>>>> >> - Run time cpnd keep using small format shm until first >>>>> longDN >>>>> checkpoint is created. >>>>> >> After that cpnd use big format shm. >>>>> >>>>> While reviewing I am assuming following please confirm : >>>>> >>>>> -The existing `small format shm` will continue to be small , is >>>>> that right ? >>>>> -Only newly created longDN checkpoint will be in `big format shm`, is >>>>> that right ? >>>>> - what will be the format of newly joined the PL-5 opens an existing >>>>> `small format shm` >>>>> the what will be the new replica on new node `small format >>>>> shm` >>>>> or `big format shm` ? >>>>> >>>>> >>>>> I hope you tested following : >>>>> ========================== >>>>> - combination of some `small format shm` and some `big format shm` >>>>> ckpts >>>>> - Joined a New node ( say PL-5) and then opened the existing `small >>>>> format shm` ckpt from the new Node >>>>> - Restating controller which has combination of `small format shm` >>>>> and `big format shm` and how the restored non-collocated ckpt`s >>>>> >>>>> -AVM >>>>> >>>>> On 10/11/2016 1:15 PM, Hoang Vo wrote: >>>>>> osaf/libs/common/cpsv/include/cpsv_shm.h | 9 +- >>>>>> osaf/services/saf/cpsv/cpnd/cpnd_res.c | 565 >>>>> ++++++++++++++++++++++++++++-- >>>>>> 2 files changed, 536 insertions(+), 38 deletions(-) >>>>>> >>>>>> >>>>>> problem: In the case of CKPT osafckptnd increased 3,5Mb - 240 >>>>>> percent on all nodes CKPT_INFO size inscrease when support longDN >>>>>> lead to total >>>>> size increase. >>>>>> solution: >>>>>> - From start, cpnd use small format shm. >>>>>> - Run time cpnd keep using small format shm until first longDN >>>>>> checkpoint >>>>> is created. >>>>>> After that cpnd use big format shm. >>>>>> >>>>>> diff --git a/osaf/libs/common/cpsv/include/cpsv_shm.h >>>>>> b/osaf/libs/common/cpsv/include/cpsv_shm.h >>>>>> --- a/osaf/libs/common/cpsv/include/cpsv_shm.h >>>>>> +++ b/osaf/libs/common/cpsv/include/cpsv_shm.h >>>>>> @@ -27,7 +27,8 @@ >>>>>> #define SHM_NEXT -3 >>>>>> #define SHM_INIT -1 >>>>>> -#define CPSV_CPND_SHM_VERSION 1 >>>>>> +#define CPSV_CPND_SHM_VERSION_SHORT_DN 0 >>>>>> +#define CPSV_CPND_SHM_VERSION_LONG_DN 1 >>>>>> typedef struct cpsv_ckpt_hdr { >>>>>> SaCkptCheckpointHandleT ckpt_id; /* Index for >>> identifying the >>>>> checkpoint */ >>>>>> @@ -134,4 +135,10 @@ typedef enum cpnd_type_info { >>>>>> CPND_CKPT_INFO >>>>>> } CPND_TYPE_INFO; >>>>>> +#define cpsv_cpnd_shm_size(x) x == >>>>>> CPSV_CPND_SHM_VERSION_LONG_DN ? >> \ >>>>>> + sizeof(CLIENT_HDR) + (MAX_CLIENTS * sizeof(CLIENT_INFO)) + >>>>> \ >>>>>> + sizeof(CKPT_HDR) + (MAX_CKPTS * sizeof(CKPT_INFO)) : >>>>> \ >>>>>> + sizeof(CLIENT_HDR) + (MAX_CLIENTS * sizeof(CLIENT_INFO)) + >>>>> \ >>>>>> + sizeof(CKPT_HDR) + (MAX_CKPTS * sizeof(CKPT_INFO_V0)) >>>>> \ >>>>>> + >>>>>> #endif >>>>>> diff --git a/osaf/services/saf/cpsv/cpnd/cpnd_res.c >>>>>> b/osaf/services/saf/cpsv/cpnd/cpnd_res.c >>>>>> --- a/osaf/services/saf/cpsv/cpnd/cpnd_res.c >>>>>> +++ b/osaf/services/saf/cpsv/cpnd/cpnd_res.c >>>>>> @@ -44,20 +44,34 @@ >>>>>> #define m_CPND_CKPTINFO_UPDATE(addr,ckpt_info,offset) >>>>>> memcpy(addr+offset,&ckpt_info,sizeof(CKPT_INFO)) >>>>>> +#define m_CPND_CKPTINFO_V0_UPDATE(addr,ckpt_info,offset) >>>>>> +memcpy(addr+offset,&ckpt_info,sizeof(CKPT_INFO_V0)) >>>>>> + >>>>>> #define m_CPND_CKPTHDR_UPDATE(ckpt_hdr,offset) >>>>>> memcpy(offset,&ckpt_hdr,sizeof(CKPT_HDR)) >>>>>> +void *cpnd_restart_shm(NCS_OS_POSIX_SHM_REQ_INFO >>>>>> *cpnd_open_req, >>>>>> +CPND_CB *cb, SaClmNodeIdT nodeid); uint32_t >>>>>> +cpnd_update_ckpt_with_clienthdl_v1(CPND_CB *cb, CPND_CKPT_NODE >>>>>> +*cp_node, SaCkptHandleT client_hdl); uint32_t >>>>>> +cpnd_update_ckpt_with_clienthdl_v0(CPND_CB *cb, CPND_CKPT_NODE >>>>>> +*cp_node, SaCkptHandleT client_hdl); uint32_t >>>>>> +cpnd_write_ckpt_info_v1(CPND_CB *cb, CPND_CKPT_NODE *cp_node, >>>>>> +int32_t offset, SaCkptHandleT client_hdl); uint32_t >>>>>> +cpnd_write_ckpt_info_v0(CPND_CB *cb, CPND_CKPT_NODE *cp_node, >>>>>> +int32_t offset, SaCkptHandleT client_hdl); >>>>>> + >>>>>> static uint32_t cpnd_res_ckpt_sec_add(CPND_CKPT_SECTION_INFO >>>>>> *pSecPtr, >>>>> CPND_CKPT_NODE *cp_node); >>>>>> static bool cpnd_find_exact_ckptinfo(CPND_CB *cb, CKPT_INFO >>>>>> *ckpt_info, >>>>> uint32_t bitmap_offset, >>>>>> uint32_t *offset, uint32_t >>>>> *prev_offset); >>>>>> +static bool cpnd_find_exact_ckptinfo_v0(CPND_CB *cb, CKPT_INFO_V0 >>>>> *ckpt_info, uint32_t bitmap_offset, >>>>>> + uint32_t *offset, uint32_t >>>>> *prev_offset); >>>>>> static void cpnd_clear_ckpt_info(CPND_CB *cb, CPND_CKPT_NODE >>>>>> *cp_node, >>>>> uint32_t curr_offset, uint32_t prev_offset); >>>>>> static uint32_t cpnd_restore_client_info(CPND_CB *cb, uint8_t >>>>> *cli_addr); >>>>>> static uint32_t cpnd_restore_ckpt_info_v1(CPND_CB *cb, uint8_t >>>>> *ckpt_addr, SaClmNodeIdT nodeid); >>>>>> static uint32_t cpnd_restore_ckpt_info_v0(CPND_CB *cb, uint8_t >>>>>> *ckpt_addr, SaClmNodeIdT nodeid); >>>>>> +static void cpnd_restart_client_reset_v1(CPND_CB *cb, >>>>>> +CPND_CKPT_NODE *cp_node, CPND_CKPT_CLIENT_NODE *cl_node); static >>>>>> +void cpnd_restart_client_reset_v0(CPND_CB *cb, CPND_CKPT_NODE >>>>>> +*cp_node, CPND_CKPT_CLIENT_NODE *cl_node); >>>>>> static void >>>>>> cpnd_destroy_shm_cpnd_cp_info(NCS_OS_POSIX_SHM_REQ_OPEN_INFO >>>>> *open_req); >>>>>> static void >>>>>> *cpnd_create_shm_cpnd_cp_info(NCS_OS_POSIX_SHM_REQ_INFO >>>>> *req_info); >>>>>> static void cpnd_update_shm_cpnd_cp_info(CPND_CB *cb); >>>>>> static void cpnd_convert_cp_info_v0(CKPT_INFO_V0 *cp_info_v0, >>>>>> CKPT_INFO *cp_info); >>>>>> +static CPND_SHM_VERSION cpnd_shm_version; >>>>>> + >>>>>> >>>>> /******************************************************************** >>>>> * >>>>> ****** >>>>> **** * >>>>>> * Name : cpnd_client_extract_bits >>>>>> * >>>>>> @@ -296,12 +310,21 @@ void cpnd_restart_update_timer(CPND_CB * >>>>>> CKPT_INFO ckpt_info; >>>>>> memset(&ckpt_info, '\0', sizeof(CKPT_INFO)); >>>>>> - if (cp_node->offset >= 0) { >>>>>> + if (cp_node->offset < 0) { >>>>>> + return; >>>>>> + } >>>>>> + if (cpnd_shm_version.shm_version == >>>>>> CPSV_CPND_SHM_VERSION_LONG_DN) >>>>>> +{ >>>>>> m_CPND_CKPTINFO_READ(ckpt_info, (char >>>>> *)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR), >>>>>> cp_node->offset * >>> sizeof(CKPT_INFO)); >>>>>> ckpt_info.close_time = closetime; >>>>>> m_CPND_CKPTINFO_UPDATE((char >>> *)cb->shm_addr.ckpt_addr + >>>>> sizeof(CKPT_HDR), >>>>>> ckpt_info, cp_node->offset * >>>>> sizeof(CKPT_INFO)); >>>>>> + } else { >>>>>> + m_CPND_CKPTINFO_V0_READ(ckpt_info, (char >>>>> *)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR), >>>>>> + cp_node->offset * >>>>> sizeof(CKPT_INFO_V0)); >>>>>> + ((CKPT_INFO_V0 *)(&ckpt_info))->close_time = closetime; >>>>>> + m_CPND_CKPTINFO_V0_UPDATE((char *)cb->shm_addr.ckpt_addr + >>>>> sizeof(CKPT_HDR), >>>>>> + ckpt_info, cp_node->offset * >>>>> sizeof(CKPT_INFO_V0)); >>>>>> } >>>>>> return; >>>>>> } >>>>>> @@ -321,25 +344,52 @@ void cpnd_restart_update_timer(CPND_CB * >>>>>> shared memory and update the shared memory >>>>>> as and >>>>> when the database gets updated >>>>>> * TBD : TO CHECK THE ERROR CONDITIONS >>>>>> ******************************************************************** >>>>>> * >>>>>> * >>>>>> ******************************/ >>>>>> +void *cpnd_restart_shm_create(NCS_OS_POSIX_SHM_REQ_INFO >>>>>> +*cpnd_open_req, CPND_CB *cb, SaClmNodeIdT nodeid) { >>>>>> + TRACE_ENTER(); >>>>>> -void *cpnd_restart_shm_create(NCS_OS_POSIX_SHM_REQ_INFO >>>>>> *cpnd_open_req, CPND_CB *cb, SaClmNodeIdT nodeid) >>>>>> + /* Initializing shared memory version */ >>>>>> + memset(&cpnd_shm_version, '\0', sizeof(cpnd_shm_version)); >>>>>> + /* From start always init shm for short DN version */ >>>>>> + cpnd_shm_version.shm_version = CPSV_CPND_SHM_VERSION_SHORT_DN; >>>>>> + >>>>>> + void *ret = cpnd_restart_shm(cpnd_open_req, cb, nodeid); >>>>>> + >>>>>> + TRACE_LEAVE(); >>>>>> + >>>>>> + return ret; >>>>>> +} >>>>>> + >>>>>> >>>>> +/******************************************************************* >>>>> +* >>>>> +****** >>>>> ************************** >>>>>> + * Name : cpnd_restart_shm >>>>>> + * >>>>>> + * Description : To create the shared memory for CPND restart >>>>>> + * >>>>>> + * Arguments : NCS_OS_POSIX_SHM_REQ_INFO *open_req - Shared >> Memory >>>>> Request Info pointer >>>>>> + * CPND_CB *cb - CPND CB pointer >>>>>> + * >>>>>> + * Return Values : void * - Returns the starting address of the >>>>>> + shared >>>>> memory >>>>>> + * Notes : If the shared memory is present - CPND has >>> restarted >>>>> , so CPND will update its database by >>>>>> + reading the information from the shared memory >>>>>> + If the shared memory is not present - CPND is >>>>>> + coming >>>>> up for the first time , so create a new >>>>>> + shared memory and update the shared memory as >>>>>> + and >>>>> when the database gets updated >>>>>> + * TBD : TO CHECK THE ERROR CONDITIONS >>>>>> +******************************************************************* >>>>>> +* >>>>>> +* *******************************/ void >>>>>> +*cpnd_restart_shm(NCS_OS_POSIX_SHM_REQ_INFO *cpnd_open_req, CPND_CB >>>>>> +*cb, SaClmNodeIdT nodeid) >>>>>> { >>>>>> uint32_t rc = NCSCC_RC_SUCCESS; >>>>>> char *buf = NULL, *buffer = NULL; >>>>>> uint8_t size = 0, total_length; >>>>>> GBL_SHM_PTR gbl_shm_addr = {0, 0, 0, 0, 0}; >>>>>> - CPND_SHM_VERSION cpnd_shm_version; >>>>>> TRACE_ENTER(); >>>>>> - /* Initializing shared memory version */ >>>>>> - memset(&cpnd_shm_version, '\0', sizeof(cpnd_shm_version)); >>>>>> - cpnd_shm_version.shm_version = CPSV_CPND_SHM_VERSION; >>>>>> size = strlen("CPND_CHECKPOINT_INFO"); >>>>>> total_length = size + sizeof(nodeid) + 5; >>>>>> buffer = m_MMGR_ALLOC_CPND_DEFAULT(total_length); >>>>>> if (buffer == NULL) { >>>>>> LOG_ER("cpnd default memory allocation failed in >>> cpnd_open >>>>> in >>>>>> resart shm create"); >>>>>> + TRACE_LEAVE(); >>>>>> return NULL; >>>>>> } >>>>>> cb->cpnd_res_shm_name = (uint8_t*)buffer; @@ -349,9 +399,7 >>> @@ >>>>>> void *cpnd_restart_shm_create(NCS_OS_POS >>>>>> /* 1. FIRST TRYING TO OPEN IN RDWR MODE */ >>>>>> cpnd_open_req->type = NCS_OS_POSIX_SHM_REQ_OPEN; >>>>>> - cpnd_open_req->info.open.i_size = >>>>>> - sizeof(CLIENT_HDR) + (MAX_CLIENTS * sizeof(CLIENT_INFO)) + >>>>> sizeof(CKPT_HDR) + >>>>>> - (MAX_CKPTS * sizeof(CKPT_INFO)); >>>>>> + cpnd_open_req->info.open.i_size = >>>>>> +cpsv_cpnd_shm_size(cpnd_shm_version.shm_version); >>>>>> if (cb->shm_alloc_guaranteed == true) >>>>>> cpnd_open_req->info.open.ensures_space = true; >>>>>> else >>>>>> @@ -370,12 +418,14 @@ void *cpnd_restart_shm_create(NCS_OS_POS >>>>>> if (NULL == >>> cpnd_create_shm_cpnd_cp_info(cpnd_open_req)) { >>>>>> LOG_ER("cpnd open request fail for RDWR mode >>>>> %s",buf); >>>>>> m_MMGR_FREE_CPND_DEFAULT(buffer); >>>>>> + TRACE_LEAVE(); >>>>>> return NULL; >>>>>> } >>>>>> >>>>>> cb->cpnd_first_time = true; >>>>>> TRACE_1("cpnd new shm create request success"); >>>>>> + TRACE_LEAVE(); >>>>>> return cpnd_open_req->info.open.o_addr; >>>>>> } >>>>>> @@ -390,7 +440,7 @@ void *cpnd_restart_shm_create(NCS_OS_POS >>>>>> /* Already the shared memory exists */ >>>>>> else { >>>>>> - CPND_SHM_VERSION shm_version; >>>>>> + CPND_SHM_VERSION shm_version; // Shm version of existing >>>>> memory >>>>>> TRACE_1("cpnd restart already shared memory exits"); >>>>>> gbl_shm_addr.cli_addr = >>> cpnd_open_req->info.open.o_addr + >>>>> sizeof(cpnd_shm_version); /* Starting address of the shared >>>>> memory */ >>>>>> gbl_shm_addr.ckpt_addr = (void *)((char >>>>> *)gbl_shm_addr.cli_addr + >>>>>> sizeof(CLIENT_HDR) + @@ -403,10 +453,15 @@ void >>>>> *cpnd_restart_shm_create(NCS_OS_POS >>>>>> TRACE("CPND IS RESTARTING WITH VERSION %d", >>>>> shm_version.shm_version); >>>>>> cpnd_restore_client_info(cb, gbl_shm_addr.cli_addr); >>>>>> switch (shm_version.shm_version) { >>>>>> - case 0: >>>>>> + case CPSV_CPND_SHM_VERSION_SHORT_DN: >>>>>> { >>>>>> cpnd_restore_ckpt_info_v0(cb, >>>>> gbl_shm_addr.ckpt_addr, nodeid); >>>>>> + /* Keep v0 format if both have same version */ >>>>>> + if (cpnd_shm_version.shm_version == >>>>> shm_version.shm_version) { >>>>>> + break; >>>>>> + } >>>>>> + >>>>>> /* Destroy the CPND_CHECKPOINT_INFO version >>> 0 */ >>>>> cpnd_destroy_shm_cpnd_cp_info(&cpnd_open_req->info.open); >>>>>> @@ -414,6 +469,7 @@ void *cpnd_restart_shm_create(NCS_OS_POS >>>>>> if (NULL == >>>>> cpnd_create_shm_cpnd_cp_info(cpnd_open_req)) { >>>>>> LOG_ER("cpnd open request fail for >>> RDWR mode >>>>> %s",buf); >>>>>> m_MMGR_FREE_CPND_DEFAULT(buffer); >>>>>> + TRACE_LEAVE(); >>>>>> return NULL; >>>>>> } >>>>>> @@ -426,7 +482,8 @@ void *cpnd_restart_shm_create(NCS_OS_POS >>>>>> break; >>>>>> } >>>>>> - case 1: >>>>>> + case CPSV_CPND_SHM_VERSION_LONG_DN: >>>>>> + shm_version.shm_version = >>>>> CPSV_CPND_SHM_VERSION_LONG_DN; >>>>>> cpnd_restore_ckpt_info_v1(cb, >>>>> gbl_shm_addr.ckpt_addr, nodeid); >>>>>> break; >>>>>> default: >>>>>> @@ -505,8 +562,13 @@ int32_t cpnd_find_free_loc(CPND_CB *cb, >>>>>> } >>>>>> read_req.type = NCS_OS_POSIX_SHM_REQ_READ; >>>>>> read_req.info.read.i_addr = (void *)((char >>>>> *)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR)); >>>>>> - read_req.info.read.i_read_size = sizeof(CKPT_INFO); >>>>>> - read_req.info.read.i_offset = counter * >>>>> sizeof(CKPT_INFO); >>>>>> + if (cpnd_shm_version.shm_version == >>>>> CPSV_CPND_SHM_VERSION_LONG_DN) { >>>>>> + read_req.info.read.i_read_size = >>>>> sizeof(CKPT_INFO); >>>>>> + read_req.info.read.i_offset = counter * >>>>> sizeof(CKPT_INFO); >>>>>> + } else { >>>>>> + read_req.info.read.i_read_size = >>>>> sizeof(CKPT_INFO_V0); >>>>>> + read_req.info.read.i_offset = counter * >>>>> sizeof(CKPT_INFO_V0); >>>>>> + } >>>>>> read_req.info.read.i_to_buff = (CKPT_INFO >>>>> *)&ckpt_info; >>>>>> rc = ncs_os_posix_shm(&read_req); >>>>>> if (rc != NCSCC_RC_SUCCESS) { @@ -515,7 >>> +577,8 @@ int32_t >>>>>> cpnd_find_free_loc(CPND_CB *cb, >>>>>> return -2; >>>>>> } >>>>>> - if (1 == ((CKPT_INFO >>>>> *)read_req.info.read.i_to_buff)->is_valid) { >>>>>> + if ((cpnd_shm_version.shm_version == >>>>> CPSV_CPND_SHM_VERSION_LONG_DN && 1 == ckpt_info.is_valid) || >>>>>> + (cpnd_shm_version.shm_version == >>>>> CPSV_CPND_SHM_VERSION_SHORT_DN >>>>>> +&& 1 == ((CKPT_INFO_V0 *)(&ckpt_info))->is_valid)) { >>>>>> counter++; >>>>>> memset(&ckpt_info, '\0', >>> sizeof(CKPT_INFO)); >>>>>> if (counter == MAX_CKPTS) { >>>>>> @@ -738,25 +801,95 @@ bool cpnd_find_exact_ckptinfo(CPND_CB *c >>>>>> } >>>>>> >>>>> +/******************************************************************* >>>>> +* >>>>> +****** >>>>> ********************** >>>>>> + * Name : cpnd_find_exact_ckptinfo_v0 >>>>>> + * >>>>>> + * Description : find if the checkpoint info v0 exists in the >>>>>> shared >>>>> memory , >>>>>> + then findout the matching bm_offset offset value >>>>>> + * >>>>>> + * Arguments : CPND_CKPT_NODE - ckpt node >>>>>> + * >>>>>> + * Return Values : The offset( if same bm_pffset present) /prev >>>>>> + offset( >>>>> if same bm_offset is not present ) >>>>>> + where this checkpoint info is present >>>>>> + * >>>>>> +******************************************************************* >>>>>> +* >>>>>> +* >>>>>> +*****************************/ >>>>>> + >>>>>> +bool cpnd_find_exact_ckptinfo_v0(CPND_CB *cb, CKPT_INFO_V0 >>>>>> +*ckpt_info, >>>>> uint32_t bitmap_offset, uint32_t *offset, >>>>>> + uint32_t *prev_offset) >>>>>> +{ >>>>>> + int32_t next; >>>>>> + CKPT_INFO_V0 prev_ckpt_info; >>>>>> + uint64_t i_offset; >>>>>> + bool found = false; >>>>>> + >>>>>> + TRACE_ENTER(); >>>>>> + memset(&prev_ckpt_info, 0, sizeof(CKPT_INFO_V0)); >>>>>> + memcpy(&prev_ckpt_info, ckpt_info, sizeof(CKPT_INFO_V0)); >>>>>> + next = ckpt_info->offset; >>>>>> + *prev_offset = prev_ckpt_info.offset; >>>>>> + >>>>>> + while (next >= 0) { >>>>>> + memset(&prev_ckpt_info, 0, sizeof(CKPT_INFO_V0)); >>>>>> + if ((next * sizeof(CKPT_INFO_V0)) > UINTMAX_MAX) { >>>>>> + LOG_ER("cpnd exact_ckptinf failed,exceeded the write >>>>> limits(UINT64_MAX) "); >>>>>> + } >>>>>> + i_offset = next * sizeof(CKPT_INFO_V0); >>>>>> + m_CPND_CKPTINFO_V0_READ(prev_ckpt_info, (char >>>>> *)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR), i_offset); >>>>>> + if (prev_ckpt_info.bm_offset == bitmap_offset) { >>>>>> + found = true; >>>>>> + *offset = prev_ckpt_info.offset; >>>>>> + break; >>>>>> + } >>>>>> + next = prev_ckpt_info.next; >>>>>> + *prev_offset = prev_ckpt_info.offset; >>>>>> + } >>>>>> + TRACE_LEAVE(); >>>>>> + return found; >>>>>> + >>>>>> +} >>>>>> + >>>>>> >>>>> /******************************************************************** >>>>> * >>>>> ****** >>>>> **************************** >>>>>> * Name : cpnd_update_ckpt_with_clienthdl >>>>>> * >>>>>> * Description : To write the checkpoint data, which already >>> exists >>>> in >>>>> the shared memory, here we just change the bitmap >>>>>> * >>>>>> - * Arguments : CPND_CKPT_NODE - ckpt node , offset - to >>>>>> update the >>>>> respective ckpt , client_hdl >>>>>> + * Arguments : CPND_CKPT_NODE - ckpt node , offset - to >>>>>> update the >>>>> respective ckpt , client_hdl >>>>>> * >>>>>> * Return Values : Success / Error >>>>>> - * >>>>>> + * >>>>>> * Notes : None >>>>>> ******************************************************************** >>>>>> * >>>>>> * >>>>>> ********************************/ >>>>>> uint32_t cpnd_update_ckpt_with_clienthdl(CPND_CB *cb, >>>>>> CPND_CKPT_NODE >>>>> *cp_node, SaCkptHandleT client_hdl) >>>>>> { >>>>>> + uint32_t rc = NCSCC_RC_SUCCESS; >>>>>> + if (cpnd_shm_version.shm_version == >>>>>> CPSV_CPND_SHM_VERSION_LONG_DN) { >>>>>> + rc = cpnd_update_ckpt_with_clienthdl_v1(cb, cp_node, >>>>> client_hdl); >>>>>> + } else { >>>>>> + rc = cpnd_update_ckpt_with_clienthdl_v0(cb, cp_node, >>>>> client_hdl); >>>>>> + } >>>>>> + return rc; >>>>>> +} >>>>>> + >>>>>> >>>>> +/******************************************************************* >>>>> +* >>>>> +****** >>>>> ***************************** >>>>>> + * Name : cpnd_update_ckpt_with_clienthdl_v1 >>>>>> + * >>>>>> + * Description : To write the checkpoint data, which already >>>>>> exists >>> in >>>>> the shared memory, here we just change the bitmap >>>>>> + * >>>>>> + * Arguments : CPND_CKPT_NODE - ckpt node , offset - to >>>>>> update the >>>>> respective ckpt , client_hdl >>>>>> + * >>>>>> + * Return Values : Success / Error >>>>>> + * >>>>>> + * Notes : None >>>>>> + >>>>>> +******************************************************************* >>>>>> +* >>>>>> +* >>>>>> +*********************************/ >>>>>> + >>>>>> +uint32_t cpnd_update_ckpt_with_clienthdl_v1(CPND_CB *cb, >>>>>> +CPND_CKPT_NODE *cp_node, SaCkptHandleT client_hdl) { >>>>>> CKPT_INFO ckpt_info, prev_ckpt_info, new_ckpt_info; >>>>>> uint32_t bitmap_offset = 0, bitmap_value = 0, prev_offset, >>>>>> offset, >>>>> rc = NCSCC_RC_SUCCESS; >>>>>> bool found = false; >>>>>> - uint64_t i_offset; >>>>>> - >>>>>> + uint64_t i_offset; >>>>>> + >>>>>> TRACE_ENTER(); >>>>>> memset(&ckpt_info, '\0', sizeof(CKPT_INFO)); >>>>>> memset(&prev_ckpt_info, '\0', sizeof(CKPT_INFO)); @@ >>>>>> -837,20 >>>>>> +970,139 @@ uint32_t cpnd_update_ckpt_with_clienthdl >>>>>> return rc; >>>>>> } >>>>>> -/****************************************************************** >>>>>> * >>>>>> * >>>>>> **************************************** >>>>>> >>>>> +/******************************************************************* >>>>> +* >>>>> +****** >>>>> ***************************** >>>>>> + * Name : cpnd_update_ckpt_with_clienthdl_v0 >>>>>> + * >>>>>> + * Description : To write the checkpoint data, which already >>>>>> exists >>> in >>>>> the shared memory, here we just change the bitmap >>>>>> + * >>>>>> + * Arguments : CPND_CKPT_NODE - ckpt node , offset - to >>>>>> update the >>>>> respective ckpt , client_hdl >>>>>> + * >>>>>> + * Return Values : Success / Error >>>>>> + * >>>>>> + * Notes : None >>>>>> + >>>>>> +******************************************************************* >>>>>> +* >>>>>> +* >>>>>> +*********************************/ >>>>>> + >>>>>> +uint32_t cpnd_update_ckpt_with_clienthdl_v0(CPND_CB *cb, >>>>>> +CPND_CKPT_NODE *cp_node, SaCkptHandleT client_hdl) { >>>>>> + CKPT_INFO_V0 ckpt_info, prev_ckpt_info, new_ckpt_info; >>>>>> + uint32_t bitmap_offset = 0, bitmap_value = 0, prev_offset, >>>>>> offset, >>>>> rc = NCSCC_RC_SUCCESS; >>>>>> + bool found = false; >>>>>> + uint64_t i_offset; >>>>>> + >>>>>> + TRACE_ENTER(); >>>>>> + memset(&ckpt_info, '\0', sizeof(CKPT_INFO_V0)); >>>>>> + memset(&prev_ckpt_info, '\0', sizeof(CKPT_INFO_V0)); >>>>>> + memset(&new_ckpt_info, '\0', sizeof(CKPT_INFO_V0)); >>>>>> + >>>>>> + /* Read the starting shared memory entry for this cp_node */ >>>>>> + prev_offset = cp_node->offset; >>>>>> + i_offset = prev_offset * sizeof(CKPT_INFO_V0); >>>>>> + m_CPND_CKPTINFO_V0_READ(ckpt_info, (char >>>>>> *)cb->shm_addr.ckpt_addr >>>>>> ++ sizeof(CKPT_HDR), i_offset); >>>>>> + >>>>>> + /* Findout the bitmap offset and bitmap value for the input >>>>>> client >>>>> handle */ >>>>>> + bitmap_offset = client_hdl / 32; >>>>>> + >>>>>> + bitmap_value = cpnd_client_bitmap_set(client_hdl % 32); >>>>>> + >>>>>> + /*findout the ckpt_info with the exact bitmap_offset or findout >>>>>> +prev >>>>> ckpt_info if exact one not found */ >>>>>> + found = cpnd_find_exact_ckptinfo_v0(cb, &ckpt_info, >>>>>> bitmap_offset, >>>>>> +&offset, &prev_offset); >>>>>> + >>>>>> + if (!found) { >>>>>> + CKPT_HDR ckpt_hdr; >>>>>> + uint32_t no_ckpts = 0; >>>>>> + /* Update the Next Location in the previous >>>>> prev_ckpt_info.next as we have to find a new ckpt_info */ >>>>>> + if ((prev_offset * sizeof(CKPT_INFO_V0)) > UINTMAX_MAX) { >>>>>> + LOG_ER("cpnd update clienthdl failed,exceeded the >>>>> write limits(UINT64_MAX) "); >>>>>> + } >>>>>> + i_offset = prev_offset * sizeof(CKPT_INFO_V0); >>>>>> + m_CPND_CKPTINFO_V0_READ(prev_ckpt_info, (char >>>>>> +*)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR), i_offset); >>>>>> + >>>>>> + prev_ckpt_info.next = cpnd_find_free_loc(cb, >>>>> CPND_CKPT_INFO); >>>>>> + if (prev_ckpt_info.next == -1) { >>>>>> + TRACE_4("cpnd client free block failed "); >>>>>> + TRACE_LEAVE(); >>>>>> + return NCSCC_RC_FAILURE; >>>>>> + } >>>>>> + if (prev_ckpt_info.next == -2) { >>>>>> + TRACE_LEAVE(); >>>>>> + return NCSCC_RC_FAILURE; >>>>>> + /* SHARED MEMORY READ ERROR */ >>>>>> + } >>>>>> + >>>>>> + /* Update the Header with incremented number of >>>>>> ckpt_info 's >>>>> */ >>>>>> + memset(&ckpt_hdr, '\0', sizeof(CKPT_HDR)); >>>>>> + m_CPND_CKPTHDR_READ(ckpt_hdr, (char >>>>> *)cb->shm_addr.ckpt_addr, 0); >>>>>> + no_ckpts = ++(ckpt_hdr.num_ckpts); >>>>>> + >>>>>> + if (no_ckpts >= MAX_CKPTS) >>>>>> + { >>>>>> + TRACE_LEAVE(); >>>>>> + return NCSCC_RC_FAILURE; >>>>>> + } >>>>>> + >>>>>> + /* write the checkpoint info (number of ckpts)in the >>>>>> header >>>>> */ >>>>>> + cpnd_ckpt_write_header(cb, no_ckpts); >>>>>> + >>>>>> + m_CPND_CKPTINFO_V0_UPDATE((char *)cb->shm_addr.ckpt_addr + >>>>>> +sizeof(CKPT_HDR), prev_ckpt_info, i_offset); >>>>>> + >>>>>> + /* Allocate New ckpt_info information */ >>>>>> + offset = prev_ckpt_info.next; >>>>>> + /* bitmap_value = >>>>>> cpnd_client_bitmap_set((client_hdl%32)+1); >>>>> */ >>>>>> + memcpy(&new_ckpt_info, &prev_ckpt_info, >>>>> sizeof(CKPT_INFO_V0)); >>>>>> + new_ckpt_info.offset = offset; >>>>>> + new_ckpt_info.client_bitmap = bitmap_value; >>>>>> + new_ckpt_info.bm_offset = bitmap_offset; >>>>>> + new_ckpt_info.is_valid = 1; >>>>>> + new_ckpt_info.is_first = false; >>>>>> + new_ckpt_info.next = SHM_NEXT; >>>>>> + >>>>>> + i_offset = offset * sizeof(CKPT_INFO_V0); >>>>>> + m_CPND_CKPTINFO_V0_UPDATE((char *)cb->shm_addr.ckpt_addr + >>>>>> +sizeof(CKPT_HDR), new_ckpt_info, i_offset); >>>>>> + >>>>>> + } else { >>>>>> + i_offset = offset * sizeof(CKPT_INFO_V0); >>>>>> + m_CPND_CKPTINFO_V0_READ(prev_ckpt_info, (char >>>>> *)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR), i_offset); >>>>>> + prev_ckpt_info.client_bitmap = prev_ckpt_info.client_bitmap >>>>> | bitmap_value; >>>>>> + m_CPND_CKPTINFO_V0_UPDATE((char *)cb->shm_addr.ckpt_addr + >>>>> sizeof(CKPT_HDR), prev_ckpt_info, i_offset); >>>>>> + } >>>>>> + TRACE_LEAVE(); >>>>>> + return rc; >>>>>> +}/***************************************************************** >>>>>> +* >>>>>> +* >>>>>> +***************************************** >>>>>> * Name : cpnd_write_ckpt_info >>>>>> * >>>>>> - * Description : To write checkpoint info >>>>>> + * Description : To write checkpoint info >>>>>> * >>>>>> * Arguments : CPND_CKPT_NODE - ckpt node , offset - to >>>>>> write >> to >>>>> corresponding ckpt , client_hdl >>>>>> * >>>>>> * Return Values : Success / Error >>>>>> * >>>>>> - * Notes : Check if the offset is less than 31 , if so then update >>>>>> the information in the corresponding offset >>>>>> + * Notes : Check if the offset is less than 31 , if so then update >>>>>> + the information in the corresponding offset >>>>>> else find the next free location and there >>>>>> update the >>>>> checkpoint information >>>>>> ******************************************************************** >>>>>> * >>>>>> * **************************************/ >>>>>> +uint32_t cpnd_write_ckpt_info(CPND_CB *cb, CPND_CKPT_NODE *cp_node, >>>>>> +int32_t offset, SaCkptHandleT client_hdl) { >>>>>> + uint32_t rc = NCSCC_RC_SUCCESS; >>>>>> + if (cpnd_shm_version.shm_version == >>>>>> CPSV_CPND_SHM_VERSION_LONG_DN) { >>>>>> + rc = cpnd_write_ckpt_info_v1(cb, cp_node, offset, >>>>> client_hdl); >>>>>> + } else { >>>>>> + rc = cpnd_write_ckpt_info_v0(cb, cp_node, offset, >>>>> client_hdl); >>>>>> + } >>>>>> + return rc; >>>>>> +} >>>>>> -uint32_t cpnd_write_ckpt_info(CPND_CB *cb, CPND_CKPT_NODE >>>>>> *cp_node, >>>>>> int32_t offset, SaCkptHandleT client_hdl) >>>>>> >>>>> +/******************************************************************* >>>>> +* >>>>> +****** >>>>> ********************************** >>>>>> + * Name : cpnd_write_ckpt_info_v1 >>>>>> + * >>>>>> + * Description : To write checkpoint info >>>>>> + * >>>>>> + * Arguments : CPND_CKPT_NODE - ckpt node , offset - to >>>>>> write to >>>>> corresponding ckpt , client_hdl >>>>>> + * >>>>>> + * Return Values : Success / Error >>>>>> + * >>>>>> + * Notes : Check if the offset is less than 31 , if so then update >>>>>> + the >>>>> information in the corresponding offset >>>>>> + else find the next free location and there update the >>>>>> +checkpoint information >>>>>> +******************************************************************* >>>>>> +* >>>>>> +* ***************************************/ >>>>>> +uint32_t cpnd_write_ckpt_info_v1(CPND_CB *cb, CPND_CKPT_NODE >>>>>> +*cp_node, int32_t offset, SaCkptHandleT client_hdl) >>>>>> { >>>>>> CKPT_INFO ckpt_info; >>>>>> uint32_t rc = NCSCC_RC_SUCCESS; @@ -885,6 +1137,54 @@ >>> uint32_t >>>>>> cpnd_write_ckpt_info(CPND_CB *c >>>>>> } >>>>>> >>>>> +/******************************************************************* >>>>> +* >>>>> +****** >>>>> ********************************** >>>>>> + * Name : cpnd_write_ckpt_info_v0 >>>>>> + * >>>>>> + * Description : To write checkpoint info v0 >>>>>> + * >>>>>> + * Arguments : CPND_CKPT_NODE - ckpt node , offset - to >>>>>> write to >>>>> corresponding ckpt , client_hdl >>>>>> + * >>>>>> + * Return Values : Success / Error >>>>>> + * >>>>>> + * Notes : Check if the offset is less than 31 , if so then update >>>>>> + the >>>>> information in the corresponding offset >>>>>> + else find the next free location and there update the >>>>>> +checkpoint information >>>>>> +******************************************************************* >>>>>> +* >>>>>> +* ***************************************/ >>>>>> + >>>>>> +uint32_t cpnd_write_ckpt_info_v0(CPND_CB *cb, CPND_CKPT_NODE >>>>>> +*cp_node, int32_t offset, SaCkptHandleT client_hdl) { >>>>>> + CKPT_INFO_V0 ckpt_info; >>>>>> + uint32_t rc = NCSCC_RC_SUCCESS; >>>>>> + uint64_t i_offset; >>>>>> + >>>>>> + TRACE_ENTER(); >>>>>> + memset(&ckpt_info, 0, sizeof(CKPT_INFO_V0)); >>>>>> + osaf_extended_name_lend(cp_node->ckpt_name, >>>>>> &ckpt_info.ckpt_name); >>>>>> + ckpt_info.ckpt_id = cp_node->ckpt_id; >>>>>> + ckpt_info.maxSections = cp_node->create_attrib.maxSections; >>>>>> + ckpt_info.maxSecSize = cp_node->create_attrib.maxSectionSize; >>>>>> + ckpt_info.cpnd_rep_create = cp_node->cpnd_rep_create; >>>>>> + ckpt_info.offset = offset; >>>>>> + ckpt_info.node_id = cb->nodeid; >>>>>> + ckpt_info.is_first = true; >>>>>> + >>>>>> + if (client_hdl) { >>>>>> + ckpt_info.bm_offset = client_hdl / 32; >>>>>> + ckpt_info.client_bitmap = cpnd_client_bitmap_set(client_hdl >>>>> % 32); >>>>>> + } >>>>>> + ckpt_info.is_valid = 1; >>>>>> + ckpt_info.next = SHM_NEXT; >>>>>> + >>>>>> + if ((offset * sizeof(CKPT_INFO_V0)) > UINTMAX_MAX) { >>>>>> + LOG_ER("cpnd write ckpt info failed,exceeded the write >>>>> limits(UINT64_MAX) "); >>>>>> + } >>>>>> + i_offset = offset * sizeof(CKPT_INFO_V0); >>>>>> + m_CPND_CKPTINFO_UPDATE((char *)cb->shm_addr.ckpt_addr + >>>>>> +sizeof(CKPT_HDR), ckpt_info, i_offset); >>>>>> + >>>>>> + TRACE_LEAVE2("cpnd ckpt info write success >>>>> ckpt_id:%llx",cp_node->ckpt_id); >>>>>> + return rc; >>>>>> + >>>>>> +} >>>>>> + >>>>>> >>> /******************************************************************** >>>>>> * Name : cpnd_restart_shm_client_update >>>>>> * >>>>>> @@ -1062,11 +1362,18 @@ uint32_t cpnd_restart_shm_ckpt_free(CPND >>>>>> if (((cp_node->offset) * sizeof(CKPT_INFO)) > >>>>>> UINTMAX_MAX) { >>>>>> LOG_ER("cpnd ckpt free failed,exceeded the write >>>>> limits(UINT64_MAX) "); >>>>>> } >>>>>> - i_offset = (cp_node->offset) * sizeof(CKPT_INFO); >>>>>> + if (cpnd_shm_version.shm_version == >>>>>> CPSV_CPND_SHM_VERSION_LONG_DN) { >>>>>> + i_offset = (cp_node->offset) * sizeof(CKPT_INFO); >>>>>> + >>>>>> + /*Update the prev & curr shared memory segments with the >>>>>> new >>>>> data */ >>>>>> + m_CPND_CKPTINFO_UPDATE((char *)cb->shm_addr.ckpt_addr + >>>>> sizeof(CKPT_HDR), ckpt_info, i_offset); >>>>>> + } else { >>>>>> + i_offset = (cp_node->offset) * sizeof(CKPT_INFO_V0); >>>>>> + >>>>>> + /*Update the prev & curr shared memory segments with the >>>>>> new >>>>> data */ >>>>>> + m_CPND_CKPTINFO_V0_UPDATE((char *)cb->shm_addr.ckpt_addr + >>>>> sizeof(CKPT_HDR), ckpt_info, i_offset); >>>>>> + } >>>>>> cp_node->offset = SHM_INIT; >>>>>> - >>>>>> - /*Update the prev & curr shared memory segments with the new >>>>>> data */ >>>>>> - m_CPND_CKPTINFO_UPDATE((char *)cb->shm_addr.ckpt_addr + >>>>> sizeof(CKPT_HDR), ckpt_info, i_offset); >>>>>> >>>>>> TRACE_LEAVE(); >>>>>> @@ -1088,12 +1395,21 @@ void >>>>>> cpnd_restart_ckpt_name_length_reset >>>>>> CKPT_INFO ckpt_info; >>>>>> memset(&ckpt_info, '\0', sizeof(CKPT_INFO)); >>>>>> - if (cp_node->offset >= 0) { >>>>>> + if (cp_node->offset < 0) { >>>>>> + return; >>>>>> + } >>>>>> + if (cpnd_shm_version.shm_version == >>>>>> CPSV_CPND_SHM_VERSION_LONG_DN) >>>>>> +{ >>>>>> m_CPND_CKPTINFO_READ(ckpt_info, (char >>>>> *)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR), >>>>>> cp_node->offset * >>> sizeof(CKPT_INFO)); >>>>>> ckpt_info.is_unlink = true; >>>>>> m_CPND_CKPTINFO_UPDATE((char >>> *)cb->shm_addr.ckpt_addr + >>>>> sizeof(CKPT_HDR), ckpt_info, >>>>>> cp_node->offset * >>> sizeof(CKPT_INFO)); >>>>>> + } else { >>>>>> + m_CPND_CKPTINFO_V0_READ(ckpt_info, (char >>>>> *)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR), >>>>>> + cp_node->offset * >>>>> sizeof(CKPT_INFO_V0)); >>>>>> + ((CKPT_INFO_V0 *)(&ckpt_info))->is_unlink = true; >>>>>> + m_CPND_CKPTINFO_V0_UPDATE((char *)cb->shm_addr.ckpt_addr + >>>>> sizeof(CKPT_HDR), ckpt_info, >>>>>> + cp_node->offset * >>>>> sizeof(CKPT_INFO_V0)); >>>>>> } >>>>>> return; >>>>>> } >>>>>> @@ -1112,12 +1428,21 @@ void cpnd_restart_set_close_flag(CPND_CB >>>>>> CKPT_INFO ckpt_info; >>>>>> memset(&ckpt_info, '\0', sizeof(CKPT_INFO)); >>>>>> - if (cp_node->offset >= 0) { >>>>>> + if (cp_node->offset < 0) { >>>>>> + return; >>>>>> + } >>>>>> + if (cpnd_shm_version.shm_version == >>>>>> CPSV_CPND_SHM_VERSION_LONG_DN) >>>>>> +{ >>>>>> m_CPND_CKPTINFO_READ(ckpt_info, (char >>>>> *)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR), >>>>>> cp_node->offset * >>> sizeof(CKPT_INFO)); >>>>>> ckpt_info.is_close = true; >>>>>> m_CPND_CKPTINFO_UPDATE((char >>> *)cb->shm_addr.ckpt_addr + >>>>> sizeof(CKPT_HDR), ckpt_info, >>>>>> cp_node->offset * >>> sizeof(CKPT_INFO)); >>>>>> + } else { >>>>>> + m_CPND_CKPTINFO_V0_READ(ckpt_info, (char >>>>> *)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR), >>>>>> + cp_node->offset * >>>>> sizeof(CKPT_INFO_V0)); >>>>>> + ((CKPT_INFO_V0 *)(&ckpt_info))->is_close = true; >>>>>> + m_CPND_CKPTINFO_V0_UPDATE((char *)cb->shm_addr.ckpt_addr + >>>>> sizeof(CKPT_HDR), ckpt_info, >>>>>> + cp_node->offset * >>>>> sizeof(CKPT_INFO_V0)); >>>>>> } >>>>>> return; >>>>>> } >>>>>> @@ -1136,12 +1461,21 @@ void cpnd_restart_reset_close_flag(CPND_ >>>>>> CKPT_INFO ckpt_info; >>>>>> memset(&ckpt_info, '\0', sizeof(CKPT_INFO)); >>>>>> - if (cp_node->offset >= 0) { >>>>>> + if (cp_node->offset < 0) { >>>>>> + return; >>>>>> + } >>>>>> + if (cpnd_shm_version.shm_version == >>>>>> CPSV_CPND_SHM_VERSION_LONG_DN) >>>>>> +{ >>>>>> m_CPND_CKPTINFO_READ(ckpt_info, (char >>>>> *)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR), >>>>>> cp_node->offset * >>> sizeof(CKPT_INFO)); >>>>>> ckpt_info.is_close = false; >>>>>> m_CPND_CKPTINFO_UPDATE((char >>> *)cb->shm_addr.ckpt_addr + >>>>> sizeof(CKPT_HDR), >>>>>> ckpt_info, cp_node->offset * >>>>> sizeof(CKPT_INFO)); >>>>>> + } else { >>>>>> + m_CPND_CKPTINFO_V0_READ(ckpt_info, (char >>>>> *)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR), >>>>>> + cp_node->offset * >>>>> sizeof(CKPT_INFO_V0)); >>>>>> + ((CKPT_INFO_V0 *)(&ckpt_info))->is_close = false; >>>>>> + m_CPND_CKPTINFO_V0_UPDATE((char *)cb->shm_addr.ckpt_addr + >>>>> sizeof(CKPT_HDR), >>>>>> + ckpt_info, cp_node->offset * >>>>> sizeof(CKPT_INFO_V0)); >>>>>> } >>>>>> return; >>>>>> } >>>>>> @@ -1235,6 +1569,94 @@ void cpnd_clear_ckpt_info(CPND_CB *cb, C >>>>>> } >>>>>> /******************************************************************* >>>>>> * >>>>>> * >>>>>> *************************************** >>>>>> + * Name : cpnd_clear_ckpt_info_v0 >>>>>> + * >>>>>> + * Description : To start the timer and to reset the client >>>>>> +information >>>>>> + * >>>>>> + * Arguments : >>>>>> + * >>>>>> + * Return Values: >>>>>> +******************************************************************* >>>>>> +* >>>>>> +* ***************************************/ >>>>>> +void cpnd_clear_ckpt_info_v0(CPND_CB *cb, CPND_CKPT_NODE *cp_node, >>>>>> +uint32_t curr_offset, uint32_t prev_offset) { >>>>>> + CKPT_INFO_V0 prev_ckpt_info, curr_ckpt_info, next_ckpt_info; >>>>>> + uint64_t i_offset, no_ckpts; >>>>>> + CKPT_HDR ckpt_hdr; >>>>>> + >>>>>> + TRACE_ENTER(); >>>>>> + memset(&prev_ckpt_info, '\0', sizeof(CKPT_INFO_V0)); >>>>>> + memset(&curr_ckpt_info, '\0', sizeof(CKPT_INFO_V0)); >>>>>> + memset(&next_ckpt_info, '\0', sizeof(CKPT_INFO_V0)); >>>>>> + >>>>>> + if (((prev_offset * sizeof(CKPT_INFO_V0)) > UINTMAX_MAX) || >>>>> ((curr_offset * sizeof(CKPT_INFO_V0)) > UINTMAX_MAX)){ >>>>>> + LOG_ER("cpnd clear ckpt info failed,exceeded the write >>>>> limits(UINT64_MAX) "); >>>>>> + } >>>>>> + i_offset = prev_offset * sizeof(CKPT_INFO_V0); >>>>>> + m_CPND_CKPTINFO_V0_READ(prev_ckpt_info, (char >>>>>> +*)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR), i_offset); >>>>>> + >>>>>> + i_offset = curr_offset * sizeof(CKPT_INFO_V0); >>>>>> + m_CPND_CKPTINFO_V0_READ(curr_ckpt_info, (char >>>>>> +*)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR), i_offset); >>>>>> + >>>>>> + /* Update the Next Location in the previous >>>>>> prev_ckpt_info.next as >>>>> we have to clear the curr ckpt_info */ >>>>>> + if (cp_node->offset != curr_offset) { >>>>>> + memset(&ckpt_hdr, '\0', sizeof(CKPT_HDR)); >>>>>> + m_CPND_CKPTHDR_READ(ckpt_hdr, (char >>>>> *)cb->shm_addr.ckpt_addr, 0); >>>>>> + no_ckpts = --(ckpt_hdr.num_ckpts); >>>>>> + /* write the checkpoint info (number of ckpts)in the >>>>>> header >>>>> */ >>>>>> + cpnd_ckpt_write_header(cb, no_ckpts); >>>>>> + >>>>>> + prev_ckpt_info.next = curr_ckpt_info.next; >>>>>> + /*Update the prev & curr shared memory segments with the >>>>>> new >>>>> data */ >>>>>> + i_offset = prev_offset * sizeof(CKPT_INFO_V0); >>>>>> + m_CPND_CKPTINFO_V0_UPDATE((char *)cb->shm_addr.ckpt_addr + >>>>>> +sizeof(CKPT_HDR), prev_ckpt_info, i_offset); >>>>>> + >>>>>> + memset(&curr_ckpt_info, '\0', sizeof(CKPT_INFO_V0)); >>>>>> + i_offset = curr_offset * sizeof(CKPT_INFO_V0); >>>>>> + m_CPND_CKPTINFO_V0_UPDATE((char *)cb->shm_addr.ckpt_addr + >>>>> sizeof(CKPT_HDR), curr_ckpt_info, i_offset); >>>>>> + } else { /* This is the starting entry for this >>>>> cp_node so update accordingly */ >>>>>> + >>>>>> + if (curr_ckpt_info.next >= 0) { >>>>>> + memset(&ckpt_hdr, '\0', sizeof(CKPT_HDR)); >>>>>> + m_CPND_CKPTHDR_READ(ckpt_hdr, (char >>>>> *)cb->shm_addr.ckpt_addr, 0); >>>>>> + no_ckpts = --(ckpt_hdr.num_ckpts); >>>>>> + /* write the checkpoint info (number of ckpts)in the >>>>> header */ >>>>>> + cpnd_ckpt_write_header(cb, no_ckpts); >>>>>> + >>>>>> + cp_node->offset = curr_ckpt_info.next; >>>>>> + >>>>>> + i_offset = (curr_ckpt_info.next) * >>>>> sizeof(CKPT_INFO_V0); >>>>>> + if (((curr_ckpt_info.next) * sizeof(CKPT_INFO_V0)) > >>>>> UINTMAX_MAX) { >>>>>> + LOG_ER("cpnd clear ckpt info failed,exceeded >>>>> the write limits(UINT64_MAX) "); >>>>>> + } >>>>>> + i_offset = (curr_ckpt_info.next) * >>>>> sizeof(CKPT_INFO_V0); >>>>>> + m_CPND_CKPTINFO_V0_READ(next_ckpt_info, (char >>>>> *)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR), >>>>>> + i_offset); >>>>>> + >>>>>> + next_ckpt_info.is_close = curr_ckpt_info.is_close; >>>>>> + next_ckpt_info.is_unlink = curr_ckpt_info.is_unlink; >>>>>> + next_ckpt_info.close_time = >>>>> curr_ckpt_info.close_time; >>>>>> + next_ckpt_info.is_first = true; >>>>>> + m_CPND_CKPTINFO_V0_UPDATE((char >>>>> *)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR), next_ckpt_info, >>>>>> + i_offset); >>>>>> + >>>>>> + if (((curr_ckpt_info.offset) * sizeof(CKPT_INFO_V0)) >>>>>> UINTMAX_MAX) { >>>>>> + LOG_ER("cpnd clear ckpt info failed,exceeded >>>>> the write limits(UINT64_MAX) "); >>>>>> + } >>>>>> + i_offset = (curr_ckpt_info.offset) * >>>>> sizeof(CKPT_INFO_V0); >>>>>> + memset(&curr_ckpt_info, '\0', sizeof(CKPT_INFO_V0)); >>>>>> + >>>>>> + m_CPND_CKPTINFO_V0_UPDATE((char >>>>> *)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR), curr_ckpt_info, >>>>>> + i_offset); >>>>>> + >>>>>> + } else { >>>>>> + /* There is only one ckpt_info is there for this >>>>> cp_node so no need to delete this node as part of close >>>>>> + This ckpt_info gets deleted as part of unlink & >>>>> lcl_ref_cnt of cp_node == 0 / lcl_ref_cnt == 0 & ret_tmr expires */ >>>>>> + } >>>>>> + } >>>>>> + TRACE_LEAVE(); >>>>>> +} >>>>>> + >>>>>> +/****************************************************************** >>>>>> +* >>>>>> +* >>>>>> +**************************************** >>>>>> * Name : cpnd_restart_client_reset >>>>>> * >>>>>> * Description : To start the timer and to reset the client >>>>>> information @@ -1245,6 +1667,23 @@ void cpnd_clear_ckpt_info(CPND_CB >>>>>> *cb, >>>>> C >>>>> ********************************************************************* >>>>> * >>>>> ****** >>>>> ********************************/ >>>>>> void cpnd_restart_client_reset(CPND_CB *cb, CPND_CKPT_NODE >>>>>> *cp_node, >>>>> CPND_CKPT_CLIENT_NODE *cl_node) >>>>>> { >>>>>> + if (cpnd_shm_version.shm_version == >>>>>> CPSV_CPND_SHM_VERSION_LONG_DN) { >>>>>> + cpnd_restart_client_reset_v1(cb, cp_node, cl_node); >>>>>> + } else { >>>>>> + cpnd_restart_client_reset_v0(cb, cp_node, cl_node); >>>>>> + } >>>>>> +} >>>>>> +/****************************************************************** >>>>>> +* >>>>>> +* >>>>>> +**************************************** >>>>>> + * Name : cpnd_restart_client_reset_v1 >>>>>> + * >>>>>> + * Description : To start the timer and to reset the client >>>>>> +information >>>>>> + * >>>>>> + * Arguments : >>>>>> + * >>>>>> + * Return Values: >>>>>> +******************************************************************* >>>>>> +* >>>>>> +* ***************************************/ >>>>>> +void cpnd_restart_client_reset_v1(CPND_CB *cb, CPND_CKPT_NODE >>>>>> +*cp_node, CPND_CKPT_CLIENT_NODE *cl_node) { >>>>>> CKPT_INFO ckpt_info; >>>>>> uint32_t bitmap_offset = 0, num_bitset = 0; >>>>>> bool found = false; >>>>>> @@ -1281,6 +1720,53 @@ void cpnd_restart_client_reset(CPND_CB * >>>>>> return; >>>>>> } >>>>>> +/****************************************************************** >>>>>> +* >>>>>> +* >>>>>> +**************************************** >>>>>> + * Name : cpnd_restart_client_reset_v0 >>>>>> + * >>>>>> + * Description : To start the timer and to reset the client >>>>>> +information >>>>>> + * >>>>>> + * Arguments : >>>>>> + * >>>>>> + * Return Values: >>>>>> +******************************************************************* >>>>>> +* >>>>>> +* ***************************************/ >>>>>> +void cpnd_restart_client_reset_v0(CPND_CB *cb, CPND_CKPT_NODE >>>>>> +*cp_node, CPND_CKPT_CLIENT_NODE *cl_node) { >>>>>> + CKPT_INFO_V0 ckpt_info; >>>>>> + uint32_t bitmap_offset = 0, num_bitset = 0; >>>>>> + bool found = false; >>>>>> + uint32_t offset, prev_offset; >>>>>> + SaCkptHandleT client_hdl = cl_node->ckpt_app_hdl; >>>>>> + >>>>>> + >>>>>> + TRACE_ENTER(); >>>>>> + bitmap_offset = client_hdl / 32; >>>>>> + >>>>>> + memset(&ckpt_info, '\0', sizeof(CKPT_INFO_V0)); >>>>>> + >>>>>> + if (cp_node->offset >= 0) { >>>>>> + m_CPND_CKPTINFO_V0_READ(ckpt_info, (char >>>>> *)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR), >>>>>> + cp_node->offset * >>>>> sizeof(CKPT_INFO_V0)); >>>>>> + /* findour the exact ckpt_info matching the client_hdl */ >>>>>> + found = cpnd_find_exact_ckptinfo_v0(cb, &ckpt_info, >>>>> bitmap_offset, &offset, &prev_offset); >>>>>> + if (found) { >>>>>> + memset(&ckpt_info, '\0', sizeof(CKPT_INFO_V0)); >>>>>> + m_CPND_CKPTINFO_V0_READ(ckpt_info, (char >>>>> *)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR), >>>>>> + offset * sizeof(CKPT_INFO_V0)); >>>>>> + client_bitmap_reset(&ckpt_info.client_bitmap, >>>>> (client_hdl % 32)); >>>>>> + m_CPND_CKPTINFO_V0_UPDATE((char >>>>> *)cb->shm_addr.ckpt_addr + sizeof(CKPT_HDR), >>>>>> + ckpt_info, offset * >>>>> sizeof(CKPT_INFO_V0)); >>>>>> + >>>>>> + /* Delete the ckpt_info from shared memory if this >>>>> ckpt_info's all 31 refs are closed */ >>>>>> + num_bitset = >>>>> client_bitmap_isset(ckpt_info.client_bitmap); >>>>>> + if (!num_bitset) >>>>>> + cpnd_clear_ckpt_info_v0(cb, cp_node, offset, >>>>> prev_offset); >>>>>> + >>>>>> + } >>>>>> + } >>>>>> + TRACE_LEAVE(); >>>>>> + return; >>>>>> +} >>>>>> + >>>>>> >>>>> /******************************************************************** >>>>> * >>>>> ****** >>>>> ****************** >>>>>> * Name : cpnd_restart_shm_ckpt_update >>>>>> * >>>>>> @@ -1304,6 +1790,14 @@ uint32_t cpnd_restart_shm_ckpt_update(CP >>>>>> CKPT_HDR ckpt_hdr; >>>>>> TRACE_ENTER(); >>>>>> + /* Check ckpt name input versus shm version and restart shm if >>>>>> +need >>>>> */ >>>>>> + if (strlen(cp_node->ckpt_name) > >>>>>> SA_MAX_UNEXTENDED_NAME_LENGTH && >>>>> cpnd_shm_version.shm_version != CPSV_CPND_SHM_VERSION_LONG_DN) { >>>>>> + cpnd_shm_version.shm_version = >>>>> CPSV_CPND_SHM_VERSION_LONG_DN; >>>>>> + NCS_OS_POSIX_SHM_REQ_INFO cpnd_open_req; >>>>>> + memset(&cpnd_open_req, 0, sizeof(cpnd_open_req)); >>>>>> + cpnd_restart_shm(&cpnd_open_req, cb, cb->nodeid); >>>>>> + } >>>>>> + >>>>>> /* check if the ckpt already exists */ >>>>>> if (cp_node->offset == SHM_INIT) { /* if it is not >>> there then >>>>> find the free place to fit into */ >>>>>> /* now find the free shm for placing the checkpoint >>> info */ >>>>> @@ >>>>>> -1372,6 +1866,9 @@ static uint32_t cpnd_restore_client_info >>>>>> n_clients = cli_hdr.num_clients; >>>>>> TRACE_1("cpnd num clients read - n_clients = %d", >>> n_clients); >>>>>> + /* Clean up before loading data*/ >>>>>> + cpnd_client_node_tree_cleanup(cb); >>>>>> + >>>>>> /* ( DO - WHILE )- READ THE CLIENT INFO AND FILL THE >>> DATABASE >>>>>> OF >>>>> CLIENT INFO */ >>>>>> if (n_clients != 0) { >>>>>> while (counter < MAX_CLIENTS) { @@ -1839,12 +2336,8 >>> @@ static >>>>>> void cpnd_destroy_shm_cpnd_cp_inf >>>>>> static void >>>>>> *cpnd_create_shm_cpnd_cp_info(NCS_OS_POSIX_SHM_REQ_INFO >>>>> *req_info) >>>>>> { >>>>>> uint32_t rc = NCSCC_RC_SUCCESS; >>>>>> - CPND_SHM_VERSION cpnd_shm_version; >>>>>> TRACE_ENTER(); >>>>>> - /* Initializing shared memory version */ >>>>>> - memset(&cpnd_shm_version, '\0', sizeof(cpnd_shm_version)); >>>>>> - cpnd_shm_version.shm_version = CPSV_CPND_SHM_VERSION; >>>>>> /* Create the shared memory */ >>>>>> req_info->info.open.i_flags = O_CREAT | O_RDWR; @@ -1855,9 >>>>>> +2348,7 @@ static void *cpnd_create_shm_cpnd_cp_inf >>>>>> } >>>>>> /* Initialize memory and set version */ >>>>>> - memset(req_info->info.open.o_addr, 0, >>>>>> - sizeof(CLIENT_HDR) + (MAX_CLIENTS * >>>>>> sizeof(CLIENT_INFO)) + >>>>> sizeof(CKPT_HDR) + >>>>>> - (MAX_CKPTS * sizeof(CKPT_INFO))); >>>>>> + memset(req_info->info.open.o_addr, 0, >>>>>> +cpsv_cpnd_shm_size(cpnd_shm_version.shm_version)); >>>>>> memcpy(req_info->info.open.o_addr, &cpnd_shm_version, >>>>>> sizeof(cpnd_shm_version)); >>>>>> TRACE_LEAVE(); >>>> >>> >>> >> > ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ Opensaf-devel mailing list Opensaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-devel
