On Wednesday 05 February 2014 05:09 PM, Hans Feldt wrote: > Great to see the performance improvement but seriously... > > This patch does the same thing my previous patch did in a more intrusive, > more complex and non scalable way. [Surya] This includes TCP as well, which your patch was missing. > > Compare: > 4 files changed, 51 insertions(+), 25 deletions(-) > > With: > 11 files changed, 374 insertions(+), 29 deletions(-) > > And for scalability, publishing an extra cluster wide port just for the > purpose of understanding peer capabilities? [Surya] Sort of easy way to understand/implement, but yes, as said can be optimized. > I had solved that in a much more elegant way. I never got any feedback for > that patch. [Surya] I had given you the feedback(atleast on the personal mail). That was sort of a Work Around, not as per the bits which it should be used for. Your patch was not considering the 32bit and 64bit combination of application and opensaf. > > I can not ack this patch. I propose you go back and test and review my > original patch. > > Thanks, > Hans > > > On 01/29/2014 12:28 PM, [email protected] wrote: >> osaf/libs/core/include/mds_papi.h | 4 +- >> osaf/libs/core/include/ncsusrbuf.h | 17 ++- >> osaf/libs/core/mds/include/mds_core.h | 25 +++ >> osaf/libs/core/mds/include/mds_dt.h | 25 +++- >> osaf/libs/core/mds/include/mds_dt2c.h | 1 + >> osaf/libs/core/mds/include/mds_dt_tipc.h | 2 + >> osaf/libs/core/mds/mds_c_api.c | 20 +++ >> osaf/libs/core/mds/mds_c_db.c | 83 ++++++++++++ >> osaf/libs/core/mds/mds_c_sndrcv.c | 2 +- >> osaf/libs/core/mds/mds_dt_tipc.c | 206 >> ++++++++++++++++++++++++++++-- >> osaf/libs/core/mds/mds_dt_trans.c | 18 +- >> 11 files changed, 374 insertions(+), 29 deletions(-) >> >> >> 1) MDS fragmentation value changed from 1400 to MDS_DIRECT_BUF_MAXSIZE >> (65535 maximum packet size)-(56 MDS header) , >> #defined MDTM_NORMAL_MSG_FRAG_SIZE_MAX MDS_DIRECT_BUF_MAXSIZE . >> >> 2) MDS MDS_DIRECT_BUF_MAXSIZE value changed from 8000 to (65535 maximum >> packet size)-(56 MDS header) , >> #defined MDTM_NORMAL_MSG_FRAG_SIZE_MAX (65535 maximum packet size)-(56 MDS >> header) >> >> 3) This PAYLOAD_BUF_SIZE value is suppose to be equal to >> MDS_DIRECT_BUF_MAXSIZE >> (65535 maximum packet size)-(56 MDS header) ,but in the previous releases of >> Opensaf the value of TIPC inbuf size of MDTM_RECV_BUFFER_SIZE (mds_dt_tipc.c) >> TIPC was limited to(8000+MDS header ) , we have in-service Upgrade issue, >> so it is not possible to send the new MDS_DIRECT_BUF_MAXSIZE (65535 maximum >> packet size)-(56 MDS header) >> value previous Opensaf version nodes , so for the current release it is >> limited to 8000, >> possibly adjust this in the future Opensaf releases to >> MDS_DIRECT_BUF_MAXSIZE. >> >> >> 4) To support in-service MDS TIPC Upgrade issue MDTM_NORMAL_MSG_FRAG_SIZE_MIN >> is provide, and this fragment size is used to send to previous version of >> Opensaf Node >> while in-service Upgrade. >> >> In the previous releases of Opensaf the value of TIPC inbuf size of >> MDTM_RECV_BUFFER_SIZE (mds_dt_tipc.c) TIPC was limited to(8000+MDS header) , >> we have in-service Upgrade issue, so it is not possible to send the new >> MDS_DIRECT_BUF_MAXSIZE (65535 maximum packet size)-(56 MDS header) >> value previous Opensaf version nodes , so for the current release it is >> limited to 8000, >> possibly this will be removed in the future Opensaf releases . >> >> 5) To handle in-service Upgrade of fragmentation changes , New node do >> install & subscribe its Node Mds version. >> the MDTM_NORMAL_MSG_FRAG_SIZE_MIN fragment size is used to send to previous >> version of Opensaf >> Node while in-service Upgrade ,and the old nodes will not publishes so we >> need to fragment to MDTM_NORMAL_MSG_FRAG_SIZE_MIN >> size.mds_mdtm_node_info_install_tipc() & mds_mdtm_node_info_subscribe_tipc() >> >> diff --git a/osaf/libs/core/include/mds_papi.h >> b/osaf/libs/core/include/mds_papi.h >> --- a/osaf/libs/core/include/mds_papi.h >> +++ b/osaf/libs/core/include/mds_papi.h >> @@ -57,6 +57,8 @@ extern "C" { >> 1) Basic data type >> \**********************************************************************/ >> >> +/* The NODE version of MDS */ >> +#define MDS_NODE_VERSION 1 >> /* The version of MDS that this header ships with */ >> #define MDS_MAJOR_VERSION 2 >> #define MDS_MINOR_VERSION 1 >> @@ -208,7 +210,7 @@ extern "C" { >> #define NCSMDS_MAX_VDEST 32767 >> #define NCSMDS_MAX_SVCS 1023 >> >> -#define MDS_DIRECT_BUF_MAXSIZE 8000 >> +#define MDS_DIRECT_BUF_MAXSIZE (65535 - 56) /*(65535 maximum packet >> size)-(56 MDS header)*/ >> typedef uint8_t *MDS_DIRECT_BUFF; >> #define m_MDS_ALLOC_DIRECT_BUFF(size) mds_alloc_direct_buff(size) >> #define m_MDS_FREE_DIRECT_BUFF(x) mds_free_direct_buff(x) >> diff --git a/osaf/libs/core/include/ncsusrbuf.h >> b/osaf/libs/core/include/ncsusrbuf.h >> --- a/osaf/libs/core/include/ncsusrbuf.h >> +++ b/osaf/libs/core/include/ncsusrbuf.h >> @@ -47,8 +47,23 @@ >> #ifdef __cplusplus >> extern "C" { >> #endif >> + /* This PAYLOAD_BUF_SIZE value is suppose to be equal to >> MDS_DIRECT_BUF_MAXSIZE , >> + but in the previous releases of Opensaf the value of TIPC inbuf >> size of >> + MDTM_RECV_BUFFER_SIZE (mds_dt_tipc.c) TIPC was limited to(8000+MDS >> header ) , >> + we have in-service Upgrade issue, so it is not possible to send the >> new >> + MDS_DIRECT_BUF_MAXSIZE (65535 maximum packet size)-(56 MDS header) >> + value previous Opensaf version nodes , so for the current release >> it is limited to 8000, >> + possibly adjust this in the future Opensaf releases to >> MDS_DIRECT_BUF_MAXSIZE. >> >> -#define PAYLOAD_BUF_SIZE 1400 /* default size of packet_data bufrs */ >> + IMP TCP Note : Their is NO issue with TCP in-service Upgrade ,The >> value of >> + SO_RCVBUF & SO_SNDBUF of TCP sockets was set to 64000 ( >> MDS_SND_RCV_SIZE) >> + using setsockopt() ,the kernel doubles this value ,and this doubled >> value is returned by >> + getsockopt(). >> + Replace PAYLOAD_BUF_SIZE 8000 value with MDS_DIRECT_BUF_MAXSIZE >> (65535 maximum packet size)-(56 MDS header) , >> + after one cycle of in-service upgrade (below 4.4.FC Opensaf rpms ) >> #define PAYLOAD_BUF_SIZE (65535 - 56) >> + */ >> + >> +#define PAYLOAD_BUF_SIZE 8000 >> >> >> /***************************************************************************** >> >> diff --git a/osaf/libs/core/mds/include/mds_core.h >> b/osaf/libs/core/mds/include/mds_core.h >> --- a/osaf/libs/core/mds/include/mds_core.h >> +++ b/osaf/libs/core/mds/include/mds_core.h >> @@ -240,6 +240,19 @@ typedef struct mds_vdest_info { >> >> } MDS_VDEST_INFO; >> >> +typedef struct mds_node_info { >> + >> + /* Indexing info */ >> + NCS_PATRICIA_NODE node; >> + >> + /* Node info */ >> + NODE_ID node_id; /* Serves as Node ID. Key for Patricia node */ >> + uint16_t node_mds_ver; /* Node MDS version */ >> + /* PWE and service list */ >> + MDS_PWE_INFO *pwe_list; >> + >> +} MDS_NODE_INFO; >> + >> typedef struct mds_svc_info { >> >> /* Indexing info */ >> @@ -284,6 +297,7 @@ typedef struct mds_mcm_cb { >> NCS_PATRICIA_TREE subtn_results; >> NCS_PATRICIA_TREE svc_list; /* Tree of MDS_SVC_INFO information */ >> NCS_PATRICIA_TREE vdest_list; /* Tree of MDS_VDEST_INFO information */ >> + NCS_PATRICIA_TREE node_list; /* Tree of MDS_NODE_INFO information */ >> } MDS_MCM_CB; >> >> /* Global MDSCB */ >> @@ -376,6 +390,11 @@ extern uint32_t mds_vdest_tbl_get_subtn_ >> extern uint32_t mds_vdest_tbl_get_vdest_info_cb(MDS_VDEST_ID vdest_id, >> MDS_VDEST_INFO **vdest_info); >> extern uint32_t mds_vdest_tbl_cleanup(void); >> >> +/* NODE TABLE Operations */ >> +extern uint32_t mds_node_tbl_add(NODE_ID node_id, uint16_t node_mds_ver); >> +extern uint32_t mds_node_tbl_del(NODE_ID node_id); >> +extern uint32_t mds_node_tbl_query(NODE_ID node_id); >> + >> /* PWE TABLE Operations */ >> >> extern uint32_t mds_pwe_tbl_add(MDS_VDEST_HDL vdest_hdl, PW_ENV_ID >> pwe_id, MDS_PWE_HDL *pwe_hdl); >> @@ -512,6 +531,12 @@ extern uint32_t mds_mcm_free_msg_uba_sta >> #define m_MMGR_FREE_VDEST_INFO(p) m_NCS_MEM_FREE(p, >> NCS_MEM_REGION_TRANSIENT, \ >> NCS_SERVICE_ID_MDS, >> MDS_MEM_VDEST_INFO) >> >> +#define m_MMGR_ALLOC_NODE_INFO (MDS_NODE_INFO >> *)m_NCS_MEM_ALLOC(sizeof(MDS_NODE_INFO), \ >> + NCS_MEM_REGION_TRANSIENT, NCS_SERVICE_ID_MDS, MDS_MEM_NODE_INFO) >> + >> +#define m_MMGR_FREE_NODE_INFO(p) m_NCS_MEM_FREE(p, >> NCS_MEM_REGION_TRANSIENT, \ >> + NCS_SERVICE_ID_MDS, MDS_MEM_NODE_INFO) >> + >> #define m_MMGR_ALLOC_PWE_INFO (MDS_PWE_INFO >> *)m_NCS_MEM_ALLOC(sizeof(MDS_PWE_INFO), \ >> NCS_MEM_REGION_TRANSIENT, \ >> NCS_SERVICE_ID_MDS, MDS_MEM_PWE_INFO) >> diff --git a/osaf/libs/core/mds/include/mds_dt.h >> b/osaf/libs/core/mds/include/mds_dt.h >> --- a/osaf/libs/core/mds/include/mds_dt.h >> +++ b/osaf/libs/core/mds/include/mds_dt.h >> @@ -109,10 +109,28 @@ typedef struct mdtm_reassembly_queue { >> >> #define MDTM_MAX_DIRECT_BUFF_SIZE MDTM_MAX_SEGMENT_SIZE >> >> -#define MDTM_NORMAL_MSG_FRAG_SIZE 1400 >> +/* To support in-service Upgrade issue MDTM_NORMAL_MSG_FRAG_SIZE_MIN >> + is provide, and this fragment size is used to send to previous version >> of Opensaf Node >> + while in-service Upgrade. >> >> -#define MDTM_RECV_BUFFER_SIZE >> ((MDS_DIRECT_BUF_MAXSIZE>MDTM_NORMAL_MSG_FRAG_SIZE)? \ >> - >> (MDS_DIRECT_BUF_MAXSIZE+SUM_MDS_HDR_PLUS_MDTM_HDR_PLUS_LEN):(MDTM_NORMAL_MSG_FRAG_SIZE+SUM_MDS_HDR_PLUS_MDTM_HDR_PLUS_LEN)) >> + In the previous releases of Opensaf the value of TIPC inbuf size of >> + MDTM_RECV_BUFFER_SIZE (mds_dt_tipc.c) TIPC was limited to(8000+MDS >> header) , >> + we have in-service Upgrade issue, so it is not possible to send the new >> + MDS_DIRECT_BUF_MAXSIZE (65535 maximum packet size)-(56 MDS header) >> + value previous Opensaf version nodes , so for the current release it is >> limited to 8000, >> + possibly adjust this in the future Opensaf releases to >> MDS_DIRECT_BUF_MAXSIZE. >> + >> + IMP TCP Note : Their is NO issue with TCP in-service Upgrade ,The value >> of >> + SO_RCVBUF & SO_SNDBUF of TCP sockets was set to 64000 ( >> MDS_SND_RCV_SIZE) >> + using setsockopt() ,the kernel doubles this value ,and this doubled >> value is returned by >> + getsockopt(). So MDTM_NORMAL_MSG_FRAG_SIZE_MAX size is used to send for >> + for both previous & current versitions of Opensaf nodes which are using >> MDS TCP as transport*/ >> +#define MDTM_NORMAL_MSG_FRAG_SIZE_MIN 8000 >> + >> +#define MDTM_NORMAL_MSG_FRAG_SIZE_MAX MDS_DIRECT_BUF_MAXSIZE >> + >> +#define MDTM_RECV_BUFFER_SIZE >> ((MDS_DIRECT_BUF_MAXSIZE>MDTM_NORMAL_MSG_FRAG_SIZE_MAX)? \ >> + >> (MDS_DIRECT_BUF_MAXSIZE+SUM_MDS_HDR_PLUS_MDTM_HDR_PLUS_LEN):(MDTM_NORMAL_MSG_FRAG_SIZE_MAX+SUM_MDS_HDR_PLUS_MDTM_HDR_PLUS_LEN)) >> >> /* Prefixes and defines regarding to the MDS_TIPC*/ >> >> @@ -124,6 +142,7 @@ typedef enum { >> MDS_PCON_INST_TYPE = 0x00030000, >> MDS_NODE_INST_TYPE = 0x00000000, >> MDS_PROCESS_INST_TYPE = 0x00050000, >> + MDS_NODE_INFO_TYPE = 0x00060000, >> } MDS_MDTM_INST_TYPES; >> >> /* Common to TCP and TIPC */ >> diff --git a/osaf/libs/core/mds/include/mds_dt2c.h >> b/osaf/libs/core/mds/include/mds_dt2c.h >> --- a/osaf/libs/core/mds/include/mds_dt2c.h >> +++ b/osaf/libs/core/mds/include/mds_dt2c.h >> @@ -223,6 +223,7 @@ typedef enum { >> MDS_MEM_HDL_LIST, >> MDS_MEM_CACHED_EVENTS_LIST, >> MDS_MEM_BCAST_BUFF_LIST, >> + MDS_MEM_NODE_INFO, >> } MDS_MEM_SUB_ID; >> >> /* ******************************************** */ >> diff --git a/osaf/libs/core/mds/include/mds_dt_tipc.h >> b/osaf/libs/core/mds/include/mds_dt_tipc.h >> --- a/osaf/libs/core/mds/include/mds_dt_tipc.h >> +++ b/osaf/libs/core/mds/include/mds_dt_tipc.h >> @@ -92,6 +92,8 @@ extern uint32_t mds_mdtm_vdest_subscribe >> extern uint32_t mds_mdtm_vdest_unsubscribe_tipc(MDS_VDEST_ID vdest_id, >> MDS_SUBTN_REF_VAL subtn_ref_val); >> extern uint32_t mds_mdtm_tx_hdl_register_tipc(MDS_DEST adest); >> extern uint32_t mds_mdtm_tx_hdl_unregister_tipc(MDS_DEST adest); >> +extern uint32_t mds_mdtm_node_info_install_tipc(); >> +extern uint32_t mds_mdtm_node_info_subscribe_tipc(MDS_SUBTN_REF_VAL >> *subtn_ref_val); >> >> extern uint32_t mds_mdtm_send_tipc(MDTM_SEND_REQ *req); >> >> diff --git a/osaf/libs/core/mds/mds_c_api.c b/osaf/libs/core/mds/mds_c_api.c >> --- a/osaf/libs/core/mds/mds_c_api.c >> +++ b/osaf/libs/core/mds/mds_c_api.c >> @@ -3782,6 +3782,23 @@ uint32_t mds_mcm_init(void) >> return NCSCC_RC_FAILURE; >> } >> >> + /* NODE TREE */ >> + memset(&pat_tree_params, 0, sizeof(NCS_PATRICIA_PARAMS)); >> + pat_tree_params.key_size = sizeof(NODE_ID); >> + if (NCSCC_RC_SUCCESS != >> ncs_patricia_tree_init(&gl_mds_mcm_cb->node_list, &pat_tree_params)) { >> + m_MDS_LOG_ERR("MCM_API : patricia_tree_init: subscription: >> failure, L mds_mcm_init"); >> + if (NCSCC_RC_SUCCESS != >> ncs_patricia_tree_destroy(&gl_mds_mcm_cb->svc_list)) { >> + m_MDS_LOG_ERR("MCM_API : patricia_tree_destroy: service >> :failure, L mds_mcm_init"); >> + } >> + if (NCSCC_RC_SUCCESS != >> ncs_patricia_tree_destroy(&gl_mds_mcm_cb->vdest_list)) { >> + m_MDS_LOG_ERR("MCM_API : patricia_tree_destroy: vdest >> :failure, L mds_mcm_init"); >> + } >> + if (NCSCC_RC_SUCCESS != >> ncs_patricia_tree_destroy(&gl_mds_mcm_cb->subtn_results)) { >> + m_MDS_LOG_ERR("MCM_API : patricia_tree_destroy: >> subtn_results :failure, L mds_mcm_init"); >> + } >> + return NCSCC_RC_FAILURE; >> + } >> + >> /* Add VDEST for ADEST entry in tree */ >> vdest_for_adest_node = m_MMGR_ALLOC_VDEST_INFO; >> memset(vdest_for_adest_node, 0, sizeof(MDS_VDEST_INFO)); >> @@ -3824,6 +3841,9 @@ uint32_t mds_mcm_destroy(void) >> /* VDEST TREE */ >> ncs_patricia_tree_destroy(&gl_mds_mcm_cb->vdest_list); >> >> + /* NODE TREE */ >> + ncs_patricia_tree_destroy(&gl_mds_mcm_cb->node_list); >> + >> /* Free MCM control block */ >> m_MMGR_FREE_MCM_CB(gl_mds_mcm_cb); >> >> diff --git a/osaf/libs/core/mds/mds_c_db.c b/osaf/libs/core/mds/mds_c_db.c >> --- a/osaf/libs/core/mds/mds_c_db.c >> +++ b/osaf/libs/core/mds/mds_c_db.c >> @@ -376,6 +376,89 @@ uint32_t mds_vdest_tbl_cleanup(void) >> >> /* ******************************************** */ >> /* ******************************************** */ >> +/* NODE TABLE Operations */ >> +/* ******************************************** */ >> +/* ******************************************** */ >> +/********************************************************* >> + Function NAME: mds_node_tbl_add >> + *********************************************************/ >> +uint32_t mds_node_tbl_add(NODE_ID node_id, uint16_t node_mds_ver) >> +{ >> + MDS_NODE_INFO *node_info; >> + >> + m_MDS_LOG_DBG("MCM_DB : Entering : mds_node_tbl_add"); >> + >> + /* Check if node is not already created */ >> + if (ncs_patricia_tree_get(&gl_mds_mcm_cb->node_list, (uint8_t >> *)&node_id) != NULL) { >> + m_MDS_LOG_DBG("MCM_DB : Leaving : F : mds_node_tbl_add : VDEST >> already present"); >> + return NCSCC_RC_FAILURE; >> + } >> + >> + >> + node_info = m_MMGR_ALLOC_NODE_INFO; >> + memset(node_info, 0, sizeof(MDS_NODE_INFO)); >> + >> + node_info->node_id = node_id; >> + node_info->node_mds_ver = node_mds_ver; >> + >> + >> + m_MDS_LOG_DBG("node_id=%d node_mds_ver=%d", node_info->node_id, >> node_info->node_mds_ver); >> + node_info->node.key_info = (uint8_t *)&node_info->node_id; >> + >> + ncs_patricia_tree_add(&gl_mds_mcm_cb->node_list, (NCS_PATRICIA_NODE >> *)&node_info->node); >> + >> + m_MDS_LOG_DBG("MCM_DB : Leaving : S : mds_node_tbl_added node_id : >> %d",node_id); >> + return NCSCC_RC_SUCCESS; >> +} >> + >> +/********************************************************* >> + Function NAME: mds_node_tbl_del >> + *********************************************************/ >> +uint32_t mds_node_tbl_del(NODE_ID node_id) >> +{ >> + MDS_NODE_INFO *node_info = NULL; >> + >> + m_MDS_LOG_DBG("MCM_DB : Entering : mds_node_tbl_del"); >> + >> + /* Check if node is already created */ >> + node_info = (MDS_NODE_INFO >> *)ncs_patricia_tree_get(&gl_mds_mcm_cb->node_list, (uint8_t *)&node_id); >> + if (node_info == NULL) { >> + m_MDS_LOG_DBG("MCM_DB : Leaving : F : mds_node_tbl_del : >> NODE_ID not present"); >> + return NCSCC_RC_FAILURE; >> + } else { >> + >> + ncs_patricia_tree_del(&gl_mds_mcm_cb->node_list, >> (NCS_PATRICIA_NODE *)node_info); >> + >> + /* Free memory of NODE_INFO */ >> + m_MMGR_FREE_NODE_INFO(node_info); >> + } >> + m_MDS_LOG_DBG("MCM_DB : Leaving : S : mds_node_tbl_del"); >> + return NCSCC_RC_SUCCESS; >> +} >> + >> +/********************************************************* >> + Function NAME: mds_node_tbl_query >> + *********************************************************/ >> +uint32_t mds_node_tbl_query(NODE_ID node_id) >> +{ >> + MDS_NODE_INFO *node_info = NULL; >> + >> + m_MDS_LOG_DBG("MCM_DB : Entering : mds_node_tbl_query"); >> + >> + /* Check if node is already created */ >> + node_info = (MDS_NODE_INFO >> *)ncs_patricia_tree_get(&gl_mds_mcm_cb->node_list, (uint8_t *)&node_id); >> + if (node_info == NULL) { >> + m_MDS_LOG_DBG("MCM_DB : Leaving : F : mds_node_tbl_query : >> NODE_ID not present"); >> + return NCSCC_RC_FAILURE; >> + } else { >> + m_MDS_LOG_DBG("MCM_DB : Leaving : S : mds_node_tbl_query"); >> + return NCSCC_RC_SUCCESS; >> + } >> + >> +} >> + >> +/* ******************************************** */ >> +/* ******************************************** */ >> /* PWE TABLE Operations */ >> /* ******************************************** */ >> /* ******************************************** */ >> diff --git a/osaf/libs/core/mds/mds_c_sndrcv.c >> b/osaf/libs/core/mds/mds_c_sndrcv.c >> --- a/osaf/libs/core/mds/mds_c_sndrcv.c >> +++ b/osaf/libs/core/mds/mds_c_sndrcv.c >> @@ -34,7 +34,7 @@ >> 2. Direct send in which the user provides MDS with a flat buffer >> pointer and length >> of the buffer. Memory will be allocated to the flat buffer by >> using the MDS api >> m_MDS_ALLOC_DIRECT_BUFF(size) (where size is the number of >> bytes of memory to be >> - allocated and is limited to 8000 bytes). As buffer pointer is >> allocated by MDS, in all the >> + allocated and is limited to MDS_DIRECT_BUF_MAXSIZE bytes). As >> buffer pointer is allocated by MDS, in all the >> cases of success and failure of DIRECT sends, memory will be >> freed by the MDS and application >> should not free the memory. >> >> diff --git a/osaf/libs/core/mds/mds_dt_tipc.c >> b/osaf/libs/core/mds/mds_dt_tipc.c >> --- a/osaf/libs/core/mds/mds_dt_tipc.c >> +++ b/osaf/libs/core/mds/mds_dt_tipc.c >> @@ -85,6 +85,8 @@ uint32_t mds_mdtm_vdest_install_tipc(MDS >> uint32_t mds_mdtm_vdest_uninstall_tipc(MDS_VDEST_ID vdest_id); >> uint32_t mds_mdtm_vdest_subscribe_tipc(MDS_VDEST_ID vdest_id, >> MDS_SUBTN_REF_VAL *subtn_ref_val); >> uint32_t mds_mdtm_vdest_unsubscribe_tipc(MDS_VDEST_ID vdest_id, >> MDS_SUBTN_REF_VAL subtn_ref_val); >> +uint32_t mds_mdtm_node_info_install_tipc(); >> +uint32_t mds_mdtm_node_info_subscribe_tipc(MDS_SUBTN_REF_VAL >> *subtn_ref_val); >> uint32_t mds_mdtm_tx_hdl_register_tipc(MDS_DEST adest); >> uint32_t mds_mdtm_tx_hdl_unregister_tipc(MDS_DEST adest); >> >> @@ -128,6 +130,10 @@ static MDS_SUBTN_REF_VAL handle; >> static uint16_t num_subscriptions; >> >> uint32_t mdtm_global_frag_num; >> +#define MDTM_MAX_SEND_PKT_SIZE >> (MDTM_NORMAL_MSG_FRAG_SIZE_MAX+SUM_MDS_HDR_PLUS_MDTM_HDR_PLUS_LEN) /* >> Includes the 30 header bytes(2+8+20) */ >> +static uint32_t msg_frag_size = MDTM_NORMAL_MSG_FRAG_SIZE_MAX; >> +static uint32_t max_send_pkt_size = MDTM_MAX_SEND_PKT_SIZE; >> + >> /********************************************************* >> >> Function NAME: mdtm_tipc_init >> @@ -158,6 +164,7 @@ uint32_t mdtm_tipc_init(NODE_ID nodeid, >> num_subscriptions = 0; >> handle = 0; >> mdtm_global_frag_num = 0; >> + MDS_SUBTN_REF_VAL subtn_ref_ptr; >> >> /* REASSEMBLY TREE */ >> memset(&pat_tree_params, 0, sizeof(pat_tree_params)); >> @@ -307,6 +314,21 @@ uint32_t mdtm_tipc_init(NODE_ID nodeid, >> } else { >> m_MDS_LOG_INFO("MDTM: Successfully set default socket >> option TIPC_IMP = %d", TIPCIMPORTANCE); >> } >> + /* To handle in-service Upgrade of fragmentation changes , >> theMDTM_NORMAL_MSG_FRAG_SIZE_MIN >> + fragment size is used to send to previous version of Opensaf Node >> while in-service Upgrade , >> + so the only new node do install & subscribe its Node Mds version >> + ( old nodes will not publishes so we need to fragment to >> MDTM_NORMAL_MSG_FRAG_SIZE_MIN size. >> + >> + In the previous releases of Opensaf the value of TIPC inbuf size of >> + MDTM_RECV_BUFFER_SIZE (mds_dt_tipc.c) TIPC was limited to(8000+MDS >> header) */ >> + if (mds_mdtm_node_info_install_tipc() != NCSCC_RC_SUCCESS) { >> + LOG_ER("MDTM: Can't mds_mdtm_node_info_install_tipc TIPC_IMP >> err \n"); >> + osafassert(0); >> + } >> + if (mds_mdtm_node_info_subscribe_tipc(&subtn_ref_ptr) != >> NCSCC_RC_SUCCESS) { >> + LOG_ER("MDTM: Can't mds_mdtm_node_info_subscribe_tipc TIPC_IMP >> err \n"); >> + osafassert(0); >> + } >> >> return NCSCC_RC_SUCCESS; >> } >> @@ -1016,11 +1038,43 @@ static uint32_t mdtm_process_discovery_e >> >> } >> break; >> - >> - default: >> + case MDS_NODE_INFO_TYPE: >> + { >> + /* To handle in-service Upgrade of fragmentation >> changes , >> + theMDTM_NORMAL_MSG_FRAG_SIZE_MIN fragment size is >> used to send >> + to previous version of Opensaf Node while >> in-service Upgrade , >> + so the only new node publishes its Node Mds version >> + ( old nodes will not publishes so we need to >> fragment to >> + MDTM_NORMAL_MSG_FRAG_SIZE_MIN size. >> + >> + In the previous releases of Opensaf the value of >> TIPC inbuf size of >> + MDTM_RECV_BUFFER_SIZE (mds_dt_tipc.c) TIPC was >> limited to(8000+MDS header) */ >> + >> + uint32_t node_status = 0; >> + NODE_ID node_id = 0; >> + uint32_t node_ver = 0; >> + >> + node_ver = (uint32_t)lower; >> + >> + m_MDS_LOG_INFO("MDTM: Received NODE event"); >> + >> + node_status = m_MDS_CHECK_TIPC_NODE_ID_RANGE(node); >> + >> + if (NCSCC_RC_SUCCESS == node_status) { >> + node_id = >> ((NODE_ID)(m_MDS_GET_NCS_NODE_ID_FROM_TIPC_NODE_ID(node))); >> + >> + } else { >> + LOG_ER("MDTM: Dropping the node event,as the >> TIPC NODE_ID is not in the prescribed range=0x%08x, Event type=%d", >> + node, discovery_event); >> + return NCSCC_RC_FAILURE; >> + } >> + mds_node_tbl_add(node_id,node_ver); >> + } >> + break; >> + default: >> { >> m_MDS_LOG_ERR >> - ("MDTM: TIPC EVENT UNSUPPORTED (default). If this >> case comes this should assert as there no other events being processed"); >> + ("MDTM: TIPC EVENT UNSUPPORTED (default). If >> this case comes this should assert as there no other events being >> processed"); >> return NCSCC_RC_FAILURE; >> } >> break; >> @@ -1530,6 +1584,107 @@ uint32_t mds_mdtm_svc_subscribe_tipc(PW_ >> } >> >> /********************************************************* >> + Function NAME: mds_mdtm_node_info_install_tipc >> + >> + DESCRIPTION: >> + >> + ARGUMENTS: >> + >> + RETURNS: 1 - NCSCC_RC_SUCCESS >> + 2 - NCSCC_RC_FAILURE >> + >> +*********************************************************/ >> +uint32_t mds_mdtm_node_info_install_tipc() >> +{ >> + /* >> + STEP 1: Bind to socket BSRSock with Tipc name sequence >> + TIPC Name: >> + <MDS-prefix, MDS_NODE_INFO_TYPE, 0> >> + TIPC Range: >> + <0,ROLE=0,POLICY=0,NODE_VERSION > to >> + <0,ROLE=0,POLICY=0,NODE_VERSION > >> + */ >> + struct sockaddr_tipc server_addr; >> + uint32_t server_type = 0, server_inst = 0; >> + >> + memset(&server_addr, 0, sizeof(server_addr)); >> + server_addr.family = AF_TIPC; >> + >> + server_addr.addrtype = TIPC_ADDR_NAMESEQ; >> + >> + server_type = server_type | MDS_TIPC_PREFIX | MDS_NODE_INFO_TYPE; >> + server_inst |= MDS_NODE_VERSION; >> + >> + server_addr.addr.nameseq.type = server_type; >> + server_addr.addr.nameseq.lower = server_inst; >> + server_addr.addr.nameseq.upper = server_inst; >> + server_addr.scope = TIPC_CLUSTER_SCOPE; >> + >> + if (0 != bind(tipc_cb.BSRsock, (struct sockaddr *)&server_addr, >> sizeof(server_addr))) { >> + m_MDS_LOG_ERR("MDTM: MDS-NODE-INFO-INSTALL Failure err :%s\n", >> strerror(errno)); >> + return NCSCC_RC_FAILURE; >> + } >> + return NCSCC_RC_SUCCESS; >> +} >> + >> +/********************************************************* >> + >> + Function NAME: mds_mdtm_node_info_subscribe_tipc >> + >> + DESCRIPTION: >> + >> + ARGUMENTS: >> + >> + RETURNS: 1 - NCSCC_RC_SUCCESS >> + 2 - NCSCC_RC_FAILURE >> + >> +*********************************************************/ >> +uint32_t mds_mdtm_node_info_subscribe_tipc(MDS_SUBTN_REF_VAL *subtn_ref_val) >> +{ >> + /* >> + STEP 1: Subscribe to socket DSock with Tipc name sequence >> + TIPC Name: >> + <MDS-prefix, VDEST_INST_TYPE, 0> >> + TIPC Range: >> + <0,ROLE=0,POLICY=0,VDEST_ID > to >> + <0,ROLE=0,POLICY=0,VDEST_ID > >> + */ >> + >> + uint32_t inst = 0, server_type = 0; >> + struct tipc_subscr subscr; >> + >> + if (num_subscriptions > MAX_SUBSCRIPTIONS) { >> + m_MDS_LOG_ERR("MDTM: SYSTEM CRITICAL Crossing =%d >> subscriptions\n", num_subscriptions); >> + if (num_subscriptions > MAX_SUBSCRIPTIONS_RETURN_ERROR) { >> + m_MDS_LOG_ERR >> + ("MDTM: SYSTEM has crossed the max =%d >> subscriptions , Returning failure to the user", >> + MAX_SUBSCRIPTIONS_RETURN_ERROR); >> + return NCSCC_RC_FAILURE; >> + } >> + } >> + >> + server_type = server_type | MDS_TIPC_PREFIX | MDS_NODE_INFO_TYPE; >> + inst |= MDS_NODE_VERSION; >> + memset(&subscr, 0, sizeof(subscr)); >> + subscr.seq.type = HTONL(server_type); >> + subscr.seq.lower = HTONL(inst); >> + subscr.seq.upper = HTONL(inst); >> + subscr.timeout = HTONL(FOREVER); >> + subscr.filter = HTONL(TIPC_SUB_PORTS); >> + *subtn_ref_val = 0; >> + *subtn_ref_val = ++handle; >> + *((uint64_t *)subscr.usr_handle) = *subtn_ref_val; >> + >> + if (send(tipc_cb.Dsock, &subscr, sizeof(subscr), 0) != sizeof(subscr)) { >> + m_MDS_LOG_ERR("MDTM: MDS-NODE-INFO-SUBSCRIBE Failure\n"); >> + return NCSCC_RC_FAILURE; >> + } >> + ++num_subscriptions; >> + >> + return NCSCC_RC_SUCCESS; >> +} >> + >> +/********************************************************* >> >> Function NAME: mds_mdtm_node_subscribe_tipc >> >> @@ -2034,12 +2189,32 @@ uint32_t mds_mdtm_send_tipc(MDTM_SEND_RE >> { >> uint32_t len = 0; >> len = m_MMGR_LINK_DATA_LEN(usrbuf); /* >> Getting total len */ >> - >> - m_MDS_LOG_INFO("MDTM: User Sending Data >> lenght=%d Fr_svc=%d to_svc=%d\n", len, >> - req->src_svc_id, >> req->dest_svc_id); >> - >> - if (len > MDTM_NORMAL_MSG_FRAG_SIZE) { >> - /* Packet needs to be fragmented and >> send */ >> + /* To support in-service Upgrade issue >> MDTM_NORMAL_MSG_FRAG_SIZE_MIN >> + is provide, and this fragment size is used >> to send to previous version of Opensaf Node >> + while in-service Upgrade. >> + >> + In the previous releases of Opensaf the >> value of TIPC inbuf size of >> + MDTM_RECV_BUFFER_SIZE (mds_dt_tipc.c) TIPC >> was limited to(8000+MDS header) , >> + we have in-service Upgrade issue, so it is >> not possible to send the new >> + MDS_DIRECT_BUF_MAXSIZE (65535 maximum packet >> size)-(56 MDS header) >> + value previous Opensaf version nodes , so >> for the current release it is limited to 8000, >> + possibly adjust this in the future Opensaf >> releases to MDS_DIRECT_BUF_MAXSIZE. >> + >> + IMP TCP Note : Their is NO issue with TCP >> in-service Upgrade ,The value of >> + SO_RCVBUF & SO_SNDBUF of TCP sockets was >> set to 64000 ( MDS_SND_RCV_SIZE) >> + using setsockopt() ,the kernel doubles this >> value ,and this doubled value is returned by >> + getsockopt(). So >> MDTM_NORMAL_MSG_FRAG_SIZE_MAX size is used to send for >> + for both previous & current versitions of >> Opensaf nodes which are using MDS TCP as transport*/ >> + if (len > MDTM_NORMAL_MSG_FRAG_SIZE_MIN) { >> + if >> (mds_node_tbl_query(m_MDS_GET_NODE_ID_FROM_ADEST(req->adest)) != >> NCSCC_RC_SUCCESS) { >> + msg_frag_size = >> MDTM_NORMAL_MSG_FRAG_SIZE_MIN; >> + max_send_pkt_size = >> (MDTM_NORMAL_MSG_FRAG_SIZE_MIN+SUM_MDS_HDR_PLUS_MDTM_HDR_PLUS_LEN); >> + m_MDS_LOG_DBG("MDTM:Sending >> message TO node_id=%d with msg_frag_size = %d max_send_pkt_size = %d", >> + >> m_MDS_GET_NODE_ID_FROM_ADEST(req->adest), msg_frag_size, max_send_pkt_size); >> + } >> + } >> + >> + if (len > msg_frag_size) { >> status = mdtm_frag_and_send(req, >> frag_seq_num, tipc_id); >> return status; >> >> @@ -2155,7 +2330,6 @@ uint32_t mds_mdtm_send_tipc(MDTM_SEND_RE >> 2 - NCSCC_RC_FAILURE >> >> *********************************************************/ >> -#define MDTM_MAX_SEND_PKT_SIZE >> (MDTM_NORMAL_MSG_FRAG_SIZE+SUM_MDS_HDR_PLUS_MDTM_HDR_PLUS_LEN) /* >> Includes the 30 header bytes(2+8+20) */ >> >> #ifdef MDS_CHECKSUM_ENABLE_FLAG >> #define MDTM_FRAG_HDR_PLUS_LEN_2 13 >> @@ -2187,22 +2361,22 @@ uint32_t mdtm_frag_and_send(MDTM_SEND_RE >> >> len = m_MMGR_LINK_DATA_LEN(usrbuf); /* Getting total len */ >> >> - if (len > (32767 * MDTM_NORMAL_MSG_FRAG_SIZE)) { /* We have 15 >> bits for frag number so 2( pow 15) -1=32767 */ >> + if (len > (32767 * MDTM_NORMAL_MSG_FRAG_SIZE_MAX)) { /* We have 15 >> bits for frag number so 2( pow 15) -1=32767 */ >> m_MDS_LOG_CRITICAL >> ("MDTM: App. is trying to send data more than MDTM Can >> fragment and send, Max size is =%d\n", >> - 32767 * MDTM_NORMAL_MSG_FRAG_SIZE); >> + 32767 * MDTM_NORMAL_MSG_FRAG_SIZE_MAX); >> m_MMGR_FREE_BUFR_LIST(usrbuf); >> return NCSCC_RC_FAILURE; >> } >> >> while (len != 0) { >> - if (len > MDTM_NORMAL_MSG_FRAG_SIZE) { >> + if (len > msg_frag_size) { >> if (i == 1) { >> - len_buf = MDTM_MAX_SEND_PKT_SIZE; >> + len_buf = max_send_pkt_size; >> frag_val = MORE_FRAG_BIT | i; >> } else { >> - if ((len + MDTM_FRAG_HDR_PLUS_LEN_2) > >> MDTM_MAX_SEND_PKT_SIZE) { >> - len_buf = MDTM_MAX_SEND_PKT_SIZE; >> + if ((len + MDTM_FRAG_HDR_PLUS_LEN_2) > >> max_send_pkt_size) { >> + len_buf = max_send_pkt_size; >> frag_val = MORE_FRAG_BIT | i; >> } else { >> len_buf = len + >> MDTM_FRAG_HDR_PLUS_LEN_2; >> diff --git a/osaf/libs/core/mds/mds_dt_trans.c >> b/osaf/libs/core/mds/mds_dt_trans.c >> --- a/osaf/libs/core/mds/mds_dt_trans.c >> +++ b/osaf/libs/core/mds/mds_dt_trans.c >> @@ -41,7 +41,7 @@ >> >> #define SUM_MDS_HDR_PLUS_MDTM_HDR_PLUS_LEN_TCP ((2 + >> MDS_SEND_ADDRINFO_TCP + MDTM_FRAG_HDR_LEN_TCP + MDS_HDR_LEN_TCP)) >> >> -#define MDTM_MAX_SEND_PKT_SIZE_TCP >> (MDTM_NORMAL_MSG_FRAG_SIZE+SUM_MDS_HDR_PLUS_MDTM_HDR_PLUS_LEN_TCP) /* >> Includes the 30 header bytes(2+8+20) */ >> +#define MDTM_MAX_SEND_PKT_SIZE_TCP >> (MDTM_NORMAL_MSG_FRAG_SIZE_MAX+SUM_MDS_HDR_PLUS_MDTM_HDR_PLUS_LEN_TCP) /* >> Includes the 30 header bytes(2+8+20) */ >> >> uint32_t mdtm_global_frag_num_tcp; >> extern struct pollfd pfd[2]; >> @@ -363,16 +363,16 @@ static uint32_t mdtm_frag_and_send_tcp(M >> >> len = m_MMGR_LINK_DATA_LEN(usrbuf); /* Getting total len */ >> >> - if (len > (32767 * MDTM_NORMAL_MSG_FRAG_SIZE)) { /* We have 15 >> bits for frag number so 2( pow 15) -1=32767 */ >> + if (len > (32767 * MDTM_NORMAL_MSG_FRAG_SIZE_MAX)) { /* We have 15 >> bits for frag number so 2( pow 15) -1=32767 */ >> m_MDS_LOG_CRITICAL >> ("MDTM: App. is trying to send data more than MDTM Can >> fragment and send, Max size is =%d\n", >> - 32767 * MDTM_NORMAL_MSG_FRAG_SIZE); >> + 32767 * MDTM_NORMAL_MSG_FRAG_SIZE_MAX); >> m_MMGR_FREE_BUFR_LIST(usrbuf); >> return NCSCC_RC_FAILURE; >> } >> >> while (len != 0) { >> - if (len > MDTM_NORMAL_MSG_FRAG_SIZE) { >> + if (len > MDTM_NORMAL_MSG_FRAG_SIZE_MAX) { >> if (i == 1) { >> len_buf = MDTM_MAX_SEND_PKT_SIZE_TCP; >> frag_val = MORE_FRAG_BIT | i; >> @@ -551,8 +551,12 @@ uint32_t mds_mdtm_send_tcp(MDTM_SEND_REQ >> >> m_MDS_LOG_INFO("MDTM: User Sending Data >> lenght=%d Fr_svc=%d to_svc=%d\n", len, >> req->src_svc_id, >> req->dest_svc_id); >> - >> - if (len > MDTM_NORMAL_MSG_FRAG_SIZE) { >> + /*Their is NO implication on TCP in-service >> Upgrade because of >> + MDTM_NORMAL_MSG_FRAG_SIZE_MAX (65535 maximum >> packet size)-(56 MDS header) change , >> + The value of SO_RCVBUF & SO_SNDBUF of TCP >> sockets was set to 64000 ( MDS_SND_RCV_SIZE) >> + using setsockopt() ,the kernel doubles this >> value ,and this doubled value is returned by >> + getsockopt().*/ >> + if (len > MDTM_NORMAL_MSG_FRAG_SIZE_MAX) { >> /* Packet needs to be fragmented and >> send */ >> status = mdtm_frag_and_send_tcp(req, >> frag_seq_num, id); >> return status; >> @@ -678,7 +682,7 @@ void mdtm_process_poll_recv_data_tcp(voi >> /* Receive all incoming data on this socket */ >> >> /*******************************************************/ >> >> - recd_bytes = recv(tcp_cb->DBSRsock, tcp_cb->len_buff, >> 2, 0); >> + recd_bytes = recv(tcp_cb->DBSRsock, tcp_cb->len_buff, >> 2, MSG_NOSIGNAL); >> if (0 == recd_bytes) { >> LOG_ER("MDTM:socket_recv() = %d, conn lost with >> dh server, exiting library err :%s", recd_bytes, strerror(errno)); >> close(tcp_cb->DBSRsock); >> >> > ------------------------------------------------------------------------------ > Managing the Performance of Cloud-Based Applications > Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. > Read the Whitepaper. > http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk > _______________________________________________ > Opensaf-devel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/opensaf-devel
------------------------------------------------------------------------------ Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk _______________________________________________ Opensaf-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/opensaf-devel
