Hi All,
I re-verified the behavior of OpenSAF
MDS_ENC_TYPE_FULL/MDS_ENC_TYPE_FLAT messaging
and its optimization , following is my observation and option:
1) In the default OpenSAF configuration (` mds_arch=0` ) , messaging
across the node is ENC_TYPE_FULL and
messaging with-in the node is ENC_TYPE_FLAT .
2) Even though Opensaf is configured with default `mds_arch=0`, if
their is difference of 32--bit application communicating to 64-bit
middle-ware
( mixed 32/64 bit clients on the same Node), messaging with-in
the node is also ENC_TYPE_FULL.
3) when OpenSAF is configured with `mds_arch=1... to 7` messaging
across & with-in the node is ENC_TYPE_FLAT ,
hear we have advantage of message optimization , only when
explicitly enabling this configuration .
4) Unfortunate the same arc message optimization feature got broken in
few services such as RDE & PLM
doesn't support ENC_TYPE_FLAT , we need fix these issue along with
in-service upgrade to make it work properly .
5) One more important note is that the `same arc message optimization`
feature was not exposed to OpenSAF user till now.
6) This current patch Honors the default Opensaf configuration `
mds_arch=0` behavior ( point 1 & 2 ) and it removed
hidden `same arc message optimization` feature related code and
used those 3bits/variable for the purpose of MDS version.
So I agree with Hans and let us not regressed for Un exposed feature ,
and we will use the ARCHTYPE 3 bits for
MDS version unless we get alternate bits/variables used for MDS
version. I also NOT much in favor of little bit expensive
`new Node install & subscribe its Mds version` (My previously published
approach)
So following are current options :
- Use ARCHTYPE 3 bits for MDS version and discard un exposed feature
( `mds_arch` config option)
- Find out alter native bits for MDS version fix all the
ENC_TYPE_FLAT issue with in-service upgrade
and expose `same arc message optimization` feature to OpenSAF users.
- Use `new Node install & subscribe` option for Mds version (My
previously published approach) and fix
all the ENC_TYPE_FLAT issue with in-service upgrade and expose
`same arc message optimization`
feature to OpenSAF users.
-AVM
On 5/1/2014 2:45 PM, SuryaNarayana Garlapati wrote:
>
> On Thursday 01 May 2014 02:30 PM, Hans Feldt wrote:
>> Let's be clear that MDS a pure internal opensaf service since quite a
>> long time. There are no "applications" to consider at all. This means
>> the only thing to care about is in-service performance and making
>> sure characteristics are improved, not regressed.
> [Surya] I am saying in the context of internal messaging between the
> services which use MDS. Yes, thats what even i am trying to.
>>
>> I am not sure we have to maintain some legacy requirement of mixed
>> 32/64 bit clients on the same system. I don't see why we should.
> [Surya] This is not limited to even 32/64 bit. Its how the encoding is
> handled for the destinations.
>>
>> Thanks,
>> Hans
>>
>>> -----Original Message-----
>>> From: SuryaNarayana Garlapati
>>> [mailto:suryanarayana.garlap...@oracle.com]
>>> Sent: den 1 maj 2014 10:54
>>> To: A V Mahesh
>>> Cc: opensaf-devel@lists.sourceforge.net
>>> Subject: Re: [devel] [PATCH 3 of 4] mds: use TIPC
>>> segmentation/reassembly [#654]
>>>
>>>
>>> On Thursday 01 May 2014 01:04 PM, A V Mahesh wrote:
>>>> On 5/1/2014 10:16 AM, SuryaNarayana Garlapati wrote:
>>>>> The archtype is not a legacy one, it was introduced to have the
>>>>> optimization for the message encoding by the applications.
>>>> Passing compile flags `mds_arch=1` `mds_arch=2` (mds_arch, [The
>>>> arch-type input to MDS - valid values 1-7]) , is not a standard
>>>> Open-Source ( enterprise things for legacy product ) option for
>>>> compilations.
>>> [Surya] Wrong, if you see any standard linux configure options for any
>>> linux application code distributions, they do provide the options for
>>> supporting the options to provide such powerpc, intel, 32/64bit etc. So
>>> similar is the configure options for opensaf as well. For example you
>>> can see the configure options for the net-snmp as well.
>>>> Open source user not need to Understand/get-Educate about this
>>>> internal options ,
>>> [Surya] This is not a internal option, configure option which is
>>> exposed
>>> to user as well.
>>>> we are ok loose advantage of flat encode optimization ,
>>>> from now we want to send full_encode to different node.
>>> [Surya] This is not only done for optimization, this internally effects
>>> the message performance between the opensaf services which do huge
>>> messaging.
>>> I am on way of thinking the solution for this. Lets wait.
>>>>
>>>> We are also raising enhancement ticket for code clean.
>>>>
>>>> -AVM
>>>>
>>>>
>>>> On 5/1/2014 10:16 AM, SuryaNarayana Garlapati wrote:
>>>>> On Thursday 01 May 2014 09:56 AM, A V Mahesh wrote:
>>>>>> Hi Surya,
>>>>>>
>>>>>> On 5/1/2014 7:28 AM, Suryanarayana Garlapati wrote:
>>>>>>> Nack.
>>>>>>> MDS ARCHWORD is used for deciding the factor whether the message
>>>>>>> which is being sent to the destination should be encoded
>>>>>>> full/flat/copied. This
>>>>>>> bits were introduced to do a optimization in the way the callbacks
>>>>>>> encode full/flat are called. For example, if there are two
>>>>>>> processes(can be present in the same node or different node) and
>>>>>>> there architecture and bit size is same, so in this case, the
>>>>>>> encode flat can be called (or simply a flat copy of the message can
>>>>>>> be done in the UB). If these bits were not used, earlier the
>>>>>>> messages was being full encoded when the messages were sent to
>>>>>>> another node. At the same time, the arch field is given in the
>>>>>>> configure such that this can be used for different architectures
>>>>>>> such as the powerpc, etc.
>>>>>> [AVM]
>>>>>> “ARCHTYPE” is getting only used while cross-compiling, and to
>>>>>> distinguish architecture flavors: x86,
>>>>>> PowerPC, m68k ,ect .
>>>>>> In the code "arch_word" is used while encoding whether to do
>>>>>> MDS_ENC_TYPE_FLAT/MDS_ENC_TYPE_FULL.
>>>>>>
>>>>>> Keeping “ARCHTYPE” doesn't have any significance in current
>>>>>> Open-sourced Opnesaf code ,
>>>>> [Surya] This is provided to accommodate different architectures and
>>>>> is a build option which optimizes the applications way of message
>>>>> encoding(full/flat) and corresponding decode(full/flat).
>>>>> So that means, you want to remove this support and introduce the over
>>>>> head again. One more thing, with the new changes if you have two
>>>>> nodes one with powerpc and intel, both will not be able to
>>>>> communicate
>>>>> each other as both will result in flat encode/flat decode(as the
>>>>> value is one with the latest changes).
>>>>>> why because In the current MDS messaging we do full encoded when
>>>>>> the messages were sent to another process/node, irrelevant of
>>>>>> “ARCHTYPE”
>>>>> [Surya] Wrong, the decision is done always on the archword only.
>>>>>> I already verified cross 32/64 middle-ware & application
>>>>>> combination , all the fowls are working fine .
>>>>> [Surya] As you said this might work, but you are loosing the
>>>>> optimization and would always result in full encode/full decode of
>>>>> application messages which is an overhead.
>>>>>>> I am thinking of a solution for this and will get back to you.
>>>>>> [AVM] we already have an alternative option of new node do install
>>>>>> & subscribe its Node Mds version, to handle in-service Upgrade ,
>>>>>> but we used legacy “ARCHTYPE” 3 bits for Mds version
>>>>>> which is optimized solution then install & subscribe.
>>>>> [Surya] Thats what we had discussed before as well. The archtype is
>>>>> not a legacy one, it was introduced to have the optimization for the
>>>>> message encoding by the applications.
>>>>>> -AVM
>>>>>>>
>>>>>>> Regards
>>>>>>> Surya
>>>>>>>
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>> From: osafde...@gmail.com
>>>>>>> To: mahesh.va...@oracle.com
>>>>>>> Cc: opensaf-devel@lists.sourceforge.net
>>>>>>> Sent: Tuesday, April 29, 2014 10:58:53 AM GMT +05:30 Chennai,
>>>>>>> Kolkata, Mumbai, New Delhi
>>>>>>> Subject: [devel] [PATCH 3 of 4] mds: use TIPC
>>>>>>> segmentation/reassembly [#654]
>>>>>>>
>>>>>>> configure.ac | 21
>>>>>>> ---------------------
>>>>>>> osaf/libs/core/mds/Makefile.am | 1 -
>>>>>>> osaf/libs/core/mds/include/mds_dt2c.h | 17 ++++++++---------
>>>>>>> osaf/libs/core/mds/mds_c_sndrcv.c | 14 +++++---------
>>>>>>> osaf/libs/core/mds/mds_dt_tipc.c | 11 ++++++++++-
>>>>>>> osaf/libs/core/mds/mds_log.c | 4 ++--
>>>>>>> 6 files changed, 25 insertions(+), 43 deletions(-)
>>>>>>>
>>>>>>>
>>>>>>> TIPC supports sending message with a maximum length of 66000 bytes.
>>>>>>> MDS supports
>>>>>>> sending even bigger messages, upto ~46Mb. MDS has used an internal
>>>>>>> fragmentation
>>>>>>> of just 1400 bytes (MDTM_NORMAL_MSG_FRAG_SIZE) despite that the
>>>>>>> receive buffer
>>>>>>> is 8000 bytes (MDS_DIRECT_BUF_MAXSIZE)
>>>>>>>
>>>>>>> This patch changes MDS to use TIPC segmentation/reassembly. This
>>>>>>> gives benefits
>>>>>>> such as secure transmission (TIPC does retransmission). TIPC send
>>>>>>> congestion is
>>>>>>> reduced since TIPC counts messages not bytes. Less processing done
>>>>>>> in user
>>>>>>> space in the sending opensaf or client process.
>>>>>>>
>>>>>>> Since the receive buffer of older MDS cores is limited to 8000
>>>>>>> bytes we have be
>>>>>>> sure not to send larger messages than that to older cores.
>>>>>>> Otherwise we have
>>>>>>> problems with in-service upgradeability.
>>>>>>>
>>>>>>> Each MDS TIPC port is bound to a functional address - a port name.
>>>>>>> The port name
>>>>>>> consists of 32 bit type and 32 bit instance. The 4 msb of the
>>>>>>> instance word
>>>>>>> contains the so called "archword" field. The msb indicates if this
>>>>>>> MDS core is
>>>>>>> 32 or 64 bit. The remaining 3 bits is now used as a version field.
>>>>>>> Version 0 is
>>>>>>> used by older versions of MDS, version 1 is now used to indicate
>>>>>>> the capability
>>>>>>> for longer fragmentation.
>>>>>>>
>>>>>>> This information is part of the published port name thus a
>>>>>>> subscriber will
>>>>>>> get them using the TIPC discovery service. These bits are ignored
>>>>>>> by old MDS
>>>>>>> cores, so it can receive messages from a new MDS core. When sending
>>>>>>> a message
>>>>>>> this bit is used to understand if the peer uses TIPC
>>>>>>> segmentation/reassembly
>>>>>>> (or not) and adapt the limit accordingly.
>>>>>>>
>>>>>>> The possibility to set mds_arch using configure has been removed.
>>>>>>>
>>>>>>> The change is in-service compliant and can be upgraded into a
>>>>>>> system.
>>>>>>>
>>>>>>> diff --git a/configure.ac b/configure.ac
>>>>>>> --- a/configure.ac
>>>>>>> +++ b/configure.ac
>>>>>>> @@ -470,27 +470,6 @@ if test "$enable_java" = no; then
>>>>>>> fi
>>>>>>> #############################################
>>>>>>> -# MDS ARCH TYPE: MDS advises message encode/decode
>>>>>>> -# optimization to its clients if the sender and receiver
>>>>>>> -# are of "compatible-type". For this optimization to work,
>>>>>>> -# the sender's platform and receiver's platform should be
>>>>>>> -# similar (for e.g. both PowerPC) and use an libncs_core built
>>>>>>> -# with the same value for "mds_arch". An "mds_arch" value of 0
>>>>>>> -# is the default value and stands for a platform which is NOT
>>>>>>> -# compatible with any platform.
>>>>>>> -#############################################
>>>>>>> -
>>>>>>> -AC_ARG_VAR([mds_arch], [The arch-type input to MDS - valid values
>>>>>>> 1-7])
>>>>>>> -
>>>>>>> -if test "$mds_arch" = ""; then
>>>>>>> - mdsarchtype=0
>>>>>>> -else
>>>>>>> - mdsarchtype=$mds_arch
>>>>>>> -fi
>>>>>>> -
>>>>>>> -AC_SUBST([MDS_WORD_ARCH_FLAGS], [" -DMDS_ARCH_TYPE=$mdsarchtype"])
>>>>>>> -
>>>>>>> -#############################################
>>>>>>> # Checks for libraries.
>>>>>>> #############################################
>>>>>>> PKG_CHECK_MODULES([XML2], [libxml-2.0])
>>>>>>> diff --git a/osaf/libs/core/mds/Makefile.am
>>>>>>> b/osaf/libs/core/mds/Makefile.am
>>>>>>> --- a/osaf/libs/core/mds/Makefile.am
>>>>>>> +++ b/osaf/libs/core/mds/Makefile.am
>>>>>>> @@ -23,7 +23,6 @@ SUBDIRS = include
>>>>>>> noinst_LTLIBRARIES = libmds.la
>>>>>>> libmds_la_CPPFLAGS = \
>>>>>>> - @MDS_WORD_ARCH_FLAGS@ \
>>>>>>> $(AM_CPPFLAGS)
>>>>>>> libmds_la_LDFLAGS = -static
>>>>>>> diff --git a/osaf/libs/core/mds/include/mds_dt2c.h
>>>>>>> b/osaf/libs/core/mds/include/mds_dt2c.h
>>>>>>> --- a/osaf/libs/core/mds/include/mds_dt2c.h
>>>>>>> +++ b/osaf/libs/core/mds/include/mds_dt2c.h
>>>>>>> @@ -33,17 +33,16 @@
>>>>>>> typedef uint8_t MDS_SVC_ARCHWORD_TYPE; /*MDS app-svc arch
>>>>>>> and word_size combination */
>>>>>>> -/* MDS_WORD_SIZE_TYPE and MDS_ARCH_TYPE are compile-time
>>>>>>> macros */
>>>>>>> -
>>>>>>> -#ifndef MDS_ARCH_TYPE
>>>>>>> -#define MDS_ARCH_TYPE 0 /* Stands for unspecified
>>>>>>> architecture type */
>>>>>>> -#elif (MDS_ARCH_TYPE > 7)
>>>>>>> -#error MDS_ARCH_TYPE should be in the range 0 to 7.
>>>>>>> -#endif
>>>>>>> -
>>>>>>> #define MDS_WORD_SIZE_TYPE ((sizeof(long)/4) - 1) /* 0 for
>>>>>>> 32-bit, 1 for 64-bit */
>>>>>>> -#define MDS_SELF_ARCHWORD ((MDS_SVC_ARCHWORD_TYPE)
>>>>>>> ((MDS_WORD_SIZE_TYPE<<3) | MDS_ARCH_TYPE))
>>>>>>> +/*
>>>>>>> + * 4 bit ARCHWORD usage:
>>>>>>> + * Bit 3 is wordsize
>>>>>>> + * Bit 2:0 is a version field indicating capabilities.
>>>>>>> + * Version 0 uses 1400 bytes fragmentation.
>>>>>>> + * Version 1 uses TIPC max msg (66000 bytes) fragmentation.
>>>>>>> + */
>>>>>>> +#define MDS_SELF_ARCHWORD ((MDS_SVC_ARCHWORD_TYPE)
>>>>>>> ((MDS_WORD_SIZE_TYPE<<3) | 1))
>>>>>>> typedef enum {
>>>>>>> MDS_VIEW_NORMAL,
>>>>>>> diff --git a/osaf/libs/core/mds/mds_c_sndrcv.c
>>>>>>> b/osaf/libs/core/mds/mds_c_sndrcv.c
>>>>>>> --- a/osaf/libs/core/mds/mds_c_sndrcv.c
>>>>>>> +++ b/osaf/libs/core/mds/mds_c_sndrcv.c
>>>>>>> @@ -1483,10 +1483,10 @@ static uint32_t mcm_msg_encode_full_or_f
>>>>>>> msg_send.dest_pwe_id =
>>>>>>> m_MDS_GET_PWE_ID_FROM_SVC_HDL(svc_cb->svc_hdl);
>>>>>>> msg_send.dest_vdest_id = dest_vdest_id;
>>>>>>> msg_send.src_svc_sub_part_ver = svc_cb->svc_sub_part_ver;
>>>>>>> + msg_send.msg_arch_word = to_msg->rem_svc_arch_word;
>>>>>>> if (msg_send.msg.encoding == MDS_ENC_TYPE_FULL) {
>>>>>>> if (NULL == bcast_ptr) {
>>>>>>> - msg_send.msg_fmt_ver =
>>>>>>> cbinfo.info.enc.o_msg_fmt_ver; /* archword will be filled in
>>>>>>> next label */
>>>>>>> - msg_send.msg_arch_word = to_msg->rem_svc_arch_word;
>>>>>>> + msg_send.msg_fmt_ver = cbinfo.info.enc.o_msg_fmt_ver;
>>>>>>> }
>>>>>>> } else {
>>>>>>> if (NULL == bcast_ptr) {
>>>>>>> @@ -6502,14 +6502,10 @@ static uint32_t mcm_query_for_node_dest_
>>>>>>> if (m_MDS_GET_ADEST == adest) {
>>>>>>> *to = DESTINATION_SAME_PROCESS;
>>>>>>> } else if (MDS_SELF_ARCHWORD == arch_word) {
>>>>>>> - if ((0 == (MDS_SELF_ARCHWORD & 0x7) && (0 == (arch_word &
>>>>>>> 0x7)))) {
>>>>>>> - if (m_MDS_GET_NODE_ID_FROM_ADEST(m_MDS_GET_ADEST) ==
>>>>>>> m_MDS_GET_NODE_ID_FROM_ADEST(adest)) {
>>>>>>> - *to = DESTINATION_ON_NODE; /* This hash define
>>>>>>> may give a wrong impression, but actually it means to do
>>>>>>> flat_enc */
>>>>>>> - } else {
>>>>>>> - *to = DESTINATION_OFF_NODE;
>>>>>>> - }
>>>>>>> + if (m_MDS_GET_NODE_ID_FROM_ADEST(m_MDS_GET_ADEST) ==
>>>>>>> m_MDS_GET_NODE_ID_FROM_ADEST(adest)) {
>>>>>>> + *to = DESTINATION_ON_NODE; /* This hash define may
>>>>>>> give a wrong impression, but actually it means to do flat_enc */
>>>>>>> } else {
>>>>>>> - *to = DESTINATION_ON_NODE;
>>>>>>> + *to = DESTINATION_OFF_NODE;
>>>>>>> }
>>>>>>> } else {
>>>>>>> *to = DESTINATION_OFF_NODE;
>>>>>>> diff --git a/osaf/libs/core/mds/mds_dt_tipc.c
>>>>>>> b/osaf/libs/core/mds/mds_dt_tipc.c
>>>>>>> --- a/osaf/libs/core/mds/mds_dt_tipc.c
>>>>>>> +++ b/osaf/libs/core/mds/mds_dt_tipc.c
>>>>>>> @@ -2035,7 +2035,16 @@ uint32_t mds_mdtm_send_tipc(MDTM_SEND_RE
>>>>>>> m_MDS_LOG_INFO("MDTM: User Sending Data
>>>>>>> lenght=%d
>>>>>>> Fr_svc=%d to_svc=%d\n", len,
>>>>>>> req->src_svc_id, req->dest_svc_id);
>>>>>>> - int frag_size = MDTM_NORMAL_MSG_FRAG_SIZE;
>>>>>>> + // determine fragment limit using a bit in
>>>>>>> destination archword
>>>>>>> + int frag_size;
>>>>>>> + int version = req->msg_arch_word & 0x7;
>>>>>>> + if (version > 0) {
>>>>>>> + // normal mode, use TIPC fragmentation
>>>>>>> + frag_size = TIPC_MAX_USER_MSG_SIZE;
>>>>>>> + } else {
>>>>>>> + // old mode, completely skip TIPC
>>>>>>> fragmentation
>>>>>>> + frag_size = MDTM_NORMAL_MSG_FRAG_SIZE;
>>>>>>> + }
>>>>>>> if (len > frag_size) {
>>>>>>> /* Packet needs to be fragmented and send */
>>>>>>> diff --git a/osaf/libs/core/mds/mds_log.c
>>>>>>> b/osaf/libs/core/mds/mds_log.c
>>>>>>> --- a/osaf/libs/core/mds/mds_log.c
>>>>>>> +++ b/osaf/libs/core/mds/mds_log.c
>>>>>>> @@ -64,8 +64,8 @@ uint32_t mds_log_init(char *log_file_nam
>>>>>>> if ((fh = fopen(lf, "a+")) != NULL) {
>>>>>>> fclose(fh);
>>>>>>> - log_mds_notify("BEGIN MDS LOGGING|
>>>>>>> PID=%d|ARCH=%d|64bit=%ld\n",
>>>>>>> - process_id, MDS_ARCH_TYPE,
>>>>>>> (long)MDS_WORD_SIZE_TYPE);
>>>>>>> + log_mds_notify("BEGIN MDS LOGGING|
>>>>>>> PID=%d|ARCHW=%x|64bit=%ld\n",
>>>>>>> + process_id, MDS_SELF_ARCHWORD,
>>>>>>> (long)MDS_WORD_SIZE_TYPE);
>>>>>>> }
>>>>>>> return NCSCC_RC_SUCCESS;
>>>>>>>
>>>>>>> ------------------------------------------------------------------------------
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> "Accelerate Dev Cycles with Automated Cross-Browser Testing -
>>>>>>> For FREE
>>>>>>> Instantly run your Selenium tests across 300+ browser/OS
>>>>>>> combos. Get
>>>>>>> unparalleled scalability from the best Selenium testing platform
>>>>>>> available.
>>>>>>> Simple to use. Nothing to install. Get started now for free."
>>>>>>> http://p.sf.net/sfu/SauceLabs
>>>>>>> _______________________________________________
>>>>>>> Opensaf-devel mailing list
>>>>>>> Opensaf-devel@lists.sourceforge.net
>>>>>>> https://lists.sourceforge.net/lists/listinfo/opensaf-devel
>>>
>>> ------------------------------------------------------------------------------
>>>
>>>
>>> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
>>> Instantly run your Selenium tests across 300+ browser/OS combos. Get
>>> unparalleled scalability from the best Selenium testing platform
>>> available.
>>> Simple to use. Nothing to install. Get started now for free."
>>> http://p.sf.net/sfu/SauceLabs
>>> _______________________________________________
>>> Opensaf-devel mailing list
>>> Opensaf-devel@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/opensaf-devel
>
------------------------------------------------------------------------------
Is your legacy SCM system holding you back? Join Perforce May 7 to find out:
• 3 signs your SCM is hindering your productivity
• Requirements for releasing software faster
• Expert tips and advice for migrating your SCM now
http://p.sf.net/sfu/perforce
_______________________________________________
Opensaf-devel mailing list
Opensaf-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/opensaf-devel
