On Tuesday 20 May 2014 11:52 AM, Hans Feldt wrote: > >> -----Original Message----- >> From: SuryaNarayana Garlapati [mailto:[email protected]] >> Sent: den 20 maj 2014 08:17 >> To: Hans Feldt; [email protected]; [email protected] >> Cc: [email protected] >> Subject: Re: [devel] [PATCH 1 of 3] dtm: upgrade the fragment size to >> MDS_DIRECT_BUF_MAXSIZE [#654] >> >> >> On Tuesday 20 May 2014 12:56 AM, Hans Feldt wrote: >>> - I guess the commit message prefix should be "mds: " >>> - I miss the use case / benefit of the proposed change in the commit message >>> >>> I don't understand the need for setting a default of rcvbuf and sndbuf for >>> unix domain sockets. This makes it impossible todo system >> wide tuning the standard way using sysctl. >>> I think setting rcvbuf and sndbuf should only be done if configured with >>> the environment variable. Otherwise system defaults >> should be used. In the code it says " Increase the socket buffer size" which >> is wrong on most modern Linux systems it will actuallt >> decrease it! On my Ubuntu system rmem_max and default is 212992. >> [Surya] If the system default size is less than the size of a MDS >> message, then it should be set to the value equal/just higher than the >> MDS message. For example, in MDS presently we are sending a max size of >> 66000. So the socket size should be at least 66000 or higher. > Then I guess the patch should do getsockopt first and then only if the value > is lesser do setsockopt. [Surya] Yep, This should be followed through out the MDS/DTM. > /Hans > >>> Thanks, >>> Hans >>> >>>> -----Original Message----- >>>> From: [email protected] [mailto:[email protected]] >>>> Sent: den 19 maj 2014 06:41 >>>> To: [email protected]; Hans Feldt >>>> Cc: [email protected] >>>> Subject: [PATCH 1 of 3] dtm: upgrade the fragment size to >>>> MDS_DIRECT_BUF_MAXSIZE [#654] >>>> >>>> osaf/libs/core/mds/include/mds_dt_tcp_disc.h | 1 - >>>> osaf/libs/core/mds/mds_dt_tcp.c | 20 ++++++++++++++++++-- >>>> osaf/libs/core/mds/mds_dt_trans.c | 12 ++++++------ >>>> 3 files changed, 24 insertions(+), 9 deletions(-) >>>> >>>> >>>> Upgrade the fragment size to MDS_DIRECT_BUF_MAXSIZE and their is NO issue >>>> with MDS TCP in-service Upgrade , >>>> In the current code SO_RCVBUF & SO_SNDBUF of TCP sockets was already >>>> set to 64000 (MDS_SND_RCV_SIZE) >>>> using setsockopt() ,the kernel doubles this value ,and this doubled value >>>> is returned by getsockopt(). >>>> So MDS_DIRECT_BUF_MAXSIZE size is used to send for for both previous & >>>> current versions of >>>> Opensaf nodes which are using MDS TCP as transport. >>>> >>>> Note :This path is on top of Hans Feldt published on 4-29-2014 10:58 AM. >>>> >>>> diff --git a/osaf/libs/core/mds/include/mds_dt_tcp_disc.h >>>> b/osaf/libs/core/mds/include/mds_dt_tcp_disc.h >>>> --- a/osaf/libs/core/mds/include/mds_dt_tcp_disc.h >>>> +++ b/osaf/libs/core/mds/include/mds_dt_tcp_disc.h >>>> @@ -70,7 +70,6 @@ server_instance_upper + sub_ref_val + su >>>> #define MDS_SND_VERSION 1 >>>> #define MDS_RCV_IDENTIFIRE 0x56123456 >>>> #define MDS_RCV_VERSION 1 >>>> -#define DTM_INTRANODE_MSG_SIZE 1500 >>>> #define DTM_INTRANODE_UNSENT_MSG 200 >>>> >>>> #define MDS_MDTM_LOWER_INSTANCE 0x00000000 >>>> diff --git a/osaf/libs/core/mds/mds_dt_tcp.c >>>> b/osaf/libs/core/mds/mds_dt_tcp.c >>>> --- a/osaf/libs/core/mds/mds_dt_tcp.c >>>> +++ b/osaf/libs/core/mds/mds_dt_tcp.c >>>> @@ -38,7 +38,6 @@ >>>> #include <configmake.h> >>>> >>>> #define MDS_MDTM_SUN_PATH 255 >>>> -#define MDS_SND_RCV_SIZE 64000 >>>> #define MDS_MDTM_CONNECT_PATH PKGLOCALSTATEDIR "/osaf_dtm_intra_server" >>>> >>>> #ifndef MDS_PORT_NUMBER >>>> @@ -53,6 +52,9 @@ >>>> /* Send_buffer_size + MDS_MDTM_DTM_PID_BUFFER_SIZE */ >>>> #define MDS_MDTM_DTM_PID_BUFFER_SIZE (2 + MDS_MDTM_DTM_PID_SIZE) >>>> >>>> +/* The default value is set by the rmem_default/wmem_default */ >>>> +#define MDS_SOCK_SND_RCV_BUF_SIZE 126976 >>>> + >>>> extern uint32_t mdtm_num_subscriptions; >>>> extern MDS_SUBTN_REF_VAL mdtm_handle; >>>> extern uint32_t mdtm_global_frag_num_tcp; >>>> @@ -79,11 +81,12 @@ uint32_t mdtm_process_recv_events_tcp(vo >>>> uint32_t mds_mdtm_init_tcp(NODE_ID nodeid, uint32_t *mds_tcp_ref) >>>> { >>>> uint32_t flags; >>>> - uint32_t size = MDS_SND_RCV_SIZE; >>>> + uint32_t size = MDS_SOCK_SND_RCV_BUF_SIZE; >>>> struct sockaddr_un server_addr_un, dhserver_addr_un; >>>> struct sockaddr_in server_addr_in; >>>> struct sockaddr_in6 server_addr_in6; >>>> uint8_t buffer[MDS_MDTM_DTM_PID_BUFFER_SIZE]; >>>> + char *ptr; >>>> >>>> mdtm_pid = getpid(); >>>> >>>> @@ -116,6 +119,19 @@ uint32_t mds_mdtm_init_tcp(NODE_ID nodei >>>> syslog(LOG_ERR, "MDS:MDTM:TCP ncs_patricia_tree_init >>>> failed MDTM_INIT\n"); >>>> return NCSCC_RC_FAILURE; >>>> } >>>> + >>>> + /* setting MDS_SOCK_SND_RCV_BUF_SIZE from environment variable if >>>> given. >>>> + The default value is set to MDS_SOCK_SND_RCV_BUF_SIZE (126976). >>>> + based on application requirements user need to export >>>> MDS_SOCK_SND_RCV_BUF_SIZE >>>> + varible. >>>> + >>>> + If MDS_SOCK_SND_RCV_BUF_SIZE exported to new value >>>> + it is also mandatory to change `DTM_SOCK_SND_RCV_BUF_SIZE=` with >>>> the same value of >>>> + for example if we export MDS_SOCK_SND_RCV_BUF_SIZE=16777216 >>>> + DTM_SOCK_SND_RCV_BUF_SIZE=16777216 also need to be changed in >>>> /etc/opensaf/dtm.conf */ >>>> + if ((ptr = getenv("MDS_SOCK_SND_RCV_BUF_SIZE")) != NULL) { >>>> + size = atoi(ptr); >>>> + } >>>> >>>> /* Create the sockets required for Binding, Send, receive and >>>> Discovery */ >>>> >>>> diff --git a/osaf/libs/core/mds/mds_dt_trans.c >>>> b/osaf/libs/core/mds/mds_dt_trans.c >>>> --- a/osaf/libs/core/mds/mds_dt_trans.c >>>> +++ b/osaf/libs/core/mds/mds_dt_trans.c >>>> @@ -41,7 +41,7 @@ >>>> >>>> #define SUM_MDS_HDR_PLUS_MDTM_HDR_PLUS_LEN_TCP ((2 + >>>> MDS_SEND_ADDRINFO_TCP + MDTM_FRAG_HDR_LEN_TCP + >>>> MDS_HDR_LEN_TCP)) >>>> >>>> -#define MDTM_MAX_SEND_PKT_SIZE_TCP >>>> (MDTM_NORMAL_MSG_FRAG_SIZE+SUM_MDS_HDR_PLUS_MDTM_HDR_PLUS_LEN_TCP) /* >>>> Includes the 30 header >>>> bytes(2+8+20) */ >>>> +#define MDTM_MAX_SEND_PKT_SIZE_TCP >> (MDS_DIRECT_BUF_MAXSIZE+SUM_MDS_HDR_PLUS_MDTM_HDR_PLUS_LEN_TCP) >>>> /* Includes the 30 header bytes(2+8+20) */ >>>> >>>> uint32_t mdtm_global_frag_num_tcp; >>>> extern struct pollfd pfd[2]; >>>> @@ -363,16 +363,16 @@ static uint32_t mdtm_frag_and_send_tcp(M >>>> >>>> len = m_MMGR_LINK_DATA_LEN(usrbuf); /* Getting total len */ >>>> >>>> - if (len > (32767 * MDTM_NORMAL_MSG_FRAG_SIZE)) { /* We have 15 >>>> bits for frag number so 2( pow 15) -1=32767 */ >>>> + if (len > (32767 * MDS_DIRECT_BUF_MAXSIZE)) { /* We have 15 bits for >>>> frag number so 2( pow 15) -1=32767 */ >>>> m_MDS_LOG_CRITICAL >>>> ("MDTM: App. is trying to send data more than MDTM >>>> Can fragment and send, Max size is =%d\n", >>>> - 32767 * MDTM_NORMAL_MSG_FRAG_SIZE); >>>> + 32767 * MDS_DIRECT_BUF_MAXSIZE); >>>> m_MMGR_FREE_BUFR_LIST(usrbuf); >>>> return NCSCC_RC_FAILURE; >>>> } >>>> >>>> while (len != 0) { >>>> - if (len > MDTM_NORMAL_MSG_FRAG_SIZE) { >>>> + if (len > MDS_DIRECT_BUF_MAXSIZE) { >>>> if (i == 1) { >>>> len_buf = MDTM_MAX_SEND_PKT_SIZE_TCP; >>>> frag_val = MORE_FRAG_BIT | i; >>>> @@ -552,7 +552,7 @@ uint32_t mds_mdtm_send_tcp(MDTM_SEND_REQ >>>> m_MDS_LOG_INFO("MDTM: User Sending Data >>>> lenght=%d Fr_svc=%d to_svc=%d\n", len, >>>> req->src_svc_id, >>>> req->dest_svc_id); >>>> >>>> - if (len > MDTM_NORMAL_MSG_FRAG_SIZE) { >>>> + if (len > MDS_DIRECT_BUF_MAXSIZE) { >>>> /* Packet needs to be >>>> fragmented and send */ >>>> status = >>>> mdtm_frag_and_send_tcp(req, frag_seq_num, id); >>>> return status; >>>> @@ -678,7 +678,7 @@ void mdtm_process_poll_recv_data_tcp(voi >>>> /* Receive all incoming data on this socket */ >>>> >>>> /*******************************************************/ >>>> >>>> - recd_bytes = recv(tcp_cb->DBSRsock, tcp_cb->len_buff, >>>> 2, 0); >>>> + recd_bytes = recv(tcp_cb->DBSRsock, tcp_cb->len_buff, >>>> 2, MSG_NOSIGNAL); >>>> if (0 == recd_bytes) { >>>> LOG_ER("MDTM:socket_recv() = %d, conn >>>> lost with dh server, exiting library err :%s", recd_bytes, >>>> strerror(errno)); >>>> close(tcp_cb->DBSRsock); >>> ------------------------------------------------------------------------------ >>> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE >>> Instantly run your Selenium tests across 300+ browser/OS combos. >>> Get unparalleled scalability from the best Selenium testing platform >>> available >>> Simple to use. Nothing to install. Get started now for free." >>> http://p.sf.net/sfu/SauceLabs >>> _______________________________________________ >>> Opensaf-devel mailing list >>> [email protected] >>> https://lists.sourceforge.net/lists/listinfo/opensaf-devel
------------------------------------------------------------------------------ "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability from the best Selenium testing platform available Simple to use. Nothing to install. Get started now for free." http://p.sf.net/sfu/SauceLabs _______________________________________________ Opensaf-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/opensaf-devel
