Re: [devel] [PATCH 3 of 4] mds: use TIPC segmentation/reassembly [#654]
Hi, This is a series of 4 patches. It is all or nothing. I sent out v2 for patch 1 after Surya’s comments. So is this an ack for the complete series? Thanks, Hans From: A V Mahesh [mailto:mahesh.va...@oracle.com] Sent: den 16 maj 2014 03:51 To: Hans Feldt; Hans Feldt Cc: opensaf-devel@lists.sourceforge.net; Suryanarayana Garlapati Subject: Re: [devel] [PATCH 3 of 4] mds: use TIPC segmentation/reassembly [#654] Importance: High Hi Hans , ACK from me for your published patch , please go-ahead and push the patch as it is. I discussion/syncdup with Surya, following are the summary points : By default MDS does FULL encode/decode between different nodes (inter-node communication), also the behavior is same when the MDS messaging happens between 32-bit and 64-bit processes. your published Patch: #654 is not impacting any of these default functionality . Even though MDS is providing a compile time flag called mds_arch, when this flag is set explicitly to the same value on different nodes, then only MDS messaging across nodes will happen with FLAT encode/decode routines, but most of the OpenSAF services (mds application) are still doing FULL encode/decode even when the MDS is triggered FLAT encode/decode callbacks ( Middle ware applications are not using advantage of mds_arch ). Till now none of the OpenSAF user are using the 'mds_arch' field and all are going with the MDS default settings. So they are NO impacts of deprecating mds_arch and considering that for versions. -AVM On 5/7/2014 12:20 PM, SuryaNarayana Garlapati wrote: OK, Here goes the solution. There is a 8 bit field in each message that is being exchanged between the services (which contains the message priority, mds prot_ver(mds protocol and mds version)) when message send is attempted. 6 bits are allocated for the mds prot_ver, present value for this field is 0xA8. This field can be used for versioning(and intended for this type of changes in MDS). This version can be dynamically learnt for a destination, when the first message is received from that destination and is stored in the Subscription result table. MDS prot_ver value will be changed in this release. So we can identify the old and new ones. If the destination is an old one, we will send with the old logic and if its a new one we will send with the bigger message. There is other solution, but this includes quite code changes. For example, In each node bind a new type say X and in each process subscribe for this X. When we get up this means, it indicates a new node and messaging is with bigger size else the normal one. Only a single extra bind is done for a node. There is one more solution, but it breaks the inservice upgradabilty, where in which the vdest id range is decreased. Regards Surya On Wednesday 07 May 2014 08:44 AM, A V Mahesh wrote: Surya, Thank for reiterating arch_word of the MDS feature ,we all in sync. On 5/6/2014 3:55 PM, SuryaNarayana Garlapati wrote: MDS version unless we get alternate bits/variables used for MDS version. [Surya] Thats the reason i am asking for some time. [AVM] If we get some alternate bits/variables used for MDS version this will the best Option and we can retain arch_word MDS feature. On 5/6/2014 3:55 PM, SuryaNarayana Garlapati wrote: 6) This current patch Honors the default Opensaf configuration ` mds_arch=0` behavior ( point 1 2 ) and it removed hidden `same arc message optimization` feature related code and used those 3bits/variable for the purpose of MDS version. [Surya] Current patch has still problems. I am repeating the same, but for your convenience, they are listed below. 1. Its taking out MDS optimization feature for the performance enhancement of the messaging. [AVM] we all in sync on this , now we are at the point whether we should expose this or not to OpenSAF users? 2. 64 bit and 32 bit combination will not work on the local node as it is doing flat encode which is wrong. [AVM] The current patch do ENC_TYPE_FULL for mixed 32/64 bit clients on the same Node. -AVM On 5/6/2014 3:55 PM, SuryaNarayana Garlapati wrote: Before going ahead, Following is the explanation for the arch_word of the MDS. Arch word(4bits) is combination of architecture and bit size of the machine. 3 bits are allocated for architecture and 1 bit is allocated for bit size. architecture of value 0 means unspecified. Message encoding is done as follows: 1. If the source and destination of the message is same process, then copy callback is called. 2. If source and destination arch_word are same, then encode flat is called. But there are some exceptions here, Say there are two nodes, A(Intel) and B(Powerpc) communicating with each other and are 64 bit each and opensaf code was compiled without giving any mds_arch input. In this case mds_arch is set to default 0. So as per the rule, encode flat callback should be called. But in this case if the flat encode is called
Re: [devel] [PATCH 3 of 4] mds: use TIPC segmentation/reassembly [#654]
I am getting an IMM sync problem when testing with different controller versions. When second controller is part of IMM loading it works, when it syncs it does not work: May 16 12:01:01 SC-2 local0.notice osafimmnd[427]: Started May 16 12:01:01 SC-2 local0.notice osafimmnd[427]: NO SERVER STATE: IMM_SERVER_ANONYMOUS -- IMM_SERVER_CLUSTER_WAITING May 16 12:01:01 SC-2 local0.notice osafimmnd[427]: NO SERVER STATE: IMM_SERVER_CLUSTER_WAITING -- IMM_SERVER_LOADING_PENDING May 16 12:01:01 SC-2 local0.notice osafimmnd[427]: NO SERVER STATE: IMM_SERVER_LOADING_PENDING -- IMM_SERVER_SYNC_PENDING May 16 12:01:01 SC-2 local0.notice osafimmnd[427]: NO NODE STATE- IMM_NODE_ISOLATED May 16 12:01:02 SC-2 local0.notice osafimmd[389]: NO SBY: Ruling epoch noted as:4 May 16 12:01:02 SC-2 local0.notice osafimmd[389]: NO IMMND coord at 2010f May 16 12:01:02 SC-2 local0.notice osafimmnd[427]: NO NODE STATE- IMM_NODE_W_AVAILABLE May 16 12:01:02 SC-2 local0.notice osafimmnd[427]: NO SERVER STATE: IMM_SERVER_SYNC_PENDING -- IMM_SERVER_SYNC_CLIENT May 16 12:01:03 SC-2 local0.notice osafimmd[389]: NO SBY: New Epoch for IMMND process at node 2010f old epoch: 3 new epoch:4 May 16 12:01:03 SC-2 local0.notice osafimmd[389]: NO IMMND coord at 2010f May 16 12:01:03 SC-2 local0.err osafimmnd[427]: ER Can not sync Ccb that is active May 16 12:01:03 SC-2 local0.err opensafd[337]: ER Could Not RESPAWN IMMND and trace: May 16 12:01:20.395920 osafimmnd [455:ImmModel.cc:15360] finalizeSync May 16 12:01:20.396115 osafimmnd [455:ImmModel.cc:15643] IN finalizeSync message contains 0 admin-owners May 16 12:01:20.396218 osafimmnd [455:ImmModel.cc:15748] T5 Immnd sync client synced implementer id:6 name:safSmfService size:13 May 16 12:01:20.396522 osafimmnd [455:ImmModel.cc:15748] T5 Immnd sync client synced implementer id:5 name:safCheckPointService size:20 May 16 12:01:20.396615 osafimmnd [455:ImmModel.cc:15748] T5 Immnd sync client synced implementer id:0 name:@safAmfService2020f size:19 May 16 12:01:20.396706 osafimmnd [455:ImmModel.cc:15748] T5 Immnd sync client synced implementer id:3 name:safAmfService size:13 May 16 12:01:20.396798 osafimmnd [455:ImmModel.cc:15748] T5 Immnd sync client synced implementer id:2 name:safClmService size:13 May 16 12:01:20.396887 osafimmnd [455:ImmModel.cc:15748] T5 Immnd sync client synced implementer id:1 name:safLogService size:13 May 16 12:01:20.397029 osafimmnd [455:ImmModel.cc:15752] IN finalizeSync message contains 6 implementers May 16 12:01:20.397293 osafimmnd [455:ImmModel.cc:15828] IN Synced 58 classes May 16 12:01:20.397404 osafimmnd [455:ImmModel.cc:15859] T5 CCB 1 state OTHER May 16 12:01:20.397838 osafimmnd [455:ImmModel.cc:15861] ER Can not sync Ccb that is active May 16 12:01:20.397938 osafimmnd [455:ImmModel.cc:16173] finalizeSync May 16 12:01:20.406856 osafimmnd [455:immnd_evt.c:8607] ER Unexpected local error 21 in finalizeSync for sync client - aborting On 16 May 2014 10:20, A V Mahesh mahesh.va...@oracle.com wrote: Hi, So is this an ack for the complete series? Yes. -AVM On 5/16/2014 12:51 PM, Hans Feldt wrote: Hi, This is a series of 4 patches. It is all or nothing. I sent out v2 for patch 1 after Surya’s comments. So is this an ack for the complete series? Thanks, Hans *From:*A V Mahesh [mailto:mahesh.va...@oracle.com] *Sent:* den 16 maj 2014 03:51 *To:* Hans Feldt; Hans Feldt *Cc:* opensaf-devel@lists.sourceforge.net; Suryanarayana Garlapati *Subject:* Re: [devel] [PATCH 3 of 4] mds: use TIPC segmentation/reassembly [#654] *Importance:* High Hi Hans , *ACK from me for your published patch , please go-ahead and push the patch as it is.* I discussion/syncdup with Surya, following are the summary points : By *default *MDS does FULL encode/decode between different nodes (inter-node communication), also the behavior is same when the MDS messaging happens between 32-bit and 64-bit processes. your published Patch: #654 is not impacting any of these default functionality**. Even though MDS is providing a compile time flag called mds_arch, when this flag is set explicitly to the same value on different nodes, then only MDS messaging across nodes will happen with FLAT encode/decode routines, but most of the OpenSAF services (mds application) are still doing FULL encode/decode even when the MDS is triggered FLAT encode/decode callbacks ( Middle ware applications are not using advantage of mds_arch ). Till now none of the OpenSAF user are using the 'mds_arch' field and all are going with the MDS default settings. So they are NO impacts of deprecating mds_arch and considering that for versions. -AVM On 5/7/2014 12:20 PM, SuryaNarayana Garlapati wrote: OK, Here goes the solution. There is a 8 bit field in each message that is being exchanged between the services (which contains the message priority, mds prot_ver(mds protocol and mds version)) when message send is attempted. 6 bits are allocated
Re: [devel] [PATCH 3 of 4] mds: use TIPC segmentation/reassembly [#654]
. There are no applications to consider at all. This means the only thing to care about is in-service performance and making sure characteristics are improved, not regressed. [Surya] I am saying in the context of internal messaging between the services which use MDS. Yes, thats what even i am trying to. I am not sure we have to maintain some legacy requirement of mixed 32/64 bit clients on the same system. I don't see why we should. [Surya] This is not limited to even 32/64 bit. Its how the encoding is handled for the destinations. Thanks, Hans -Original Message- From: SuryaNarayana Garlapati [mailto:suryanarayana.garlap...@oracle.com] Sent: den 1 maj 2014 10:54 To: A V Mahesh Cc: opensaf-devel@lists.sourceforge.net Subject: Re: [devel] [PATCH 3 of 4] mds: use TIPC segmentation/reassembly [#654] On Thursday 01 May 2014 01:04 PM, A V Mahesh wrote: On 5/1/2014 10:16 AM, SuryaNarayana Garlapati wrote: The archtype is not a legacy one, it was introduced to have the optimization for the message encoding by the applications. Passing compile flags `mds_arch=1` `mds_arch=2` (mds_arch, [The arch-type input to MDS - valid values 1-7]) , is not a standard Open-Source ( enterprise things for legacy product ) option for compilations. [Surya] Wrong, if you see any standard linux configure options for any linux application code distributions, they do provide the options for supporting the options to provide such powerpc, intel, 32/64bit etc. So similar is the configure options for opensaf as well. For example you can see the configure options for the net-snmp as well. Open source user not need to Understand/get-Educate about this internal options , [Surya] This is not a internal option, configure option which is exposed to user as well. we are ok loose advantage of flat encode optimization , from now we want to send full_encode to different node. [Surya] This is not only done for optimization, this internally effects the message performance between the opensaf services which do huge messaging. I am on way of thinking the solution for this. Lets wait. We are also raising enhancement ticket for code clean. -AVM On 5/1/2014 10:16 AM, SuryaNarayana Garlapati wrote: On Thursday 01 May 2014 09:56 AM, A V Mahesh wrote: Hi Surya, On 5/1/2014 7:28 AM, Suryanarayana Garlapati wrote: Nack. MDS ARCHWORD is used for deciding the factor whether the message which is being sent to the destination should be encoded full/flat/copied. This bits were introduced to do a optimization in the way the callbacks encode full/flat are called. For example, if there are two processes(can be present in the same node or different node) and there architecture and bit size is same, so in this case, the encode flat can be called (or simply a flat copy of the message can be done in the UB). If these bits were not used, earlier the messages was being full encoded when the messages were sent to another node. At the same time, the arch field is given in the configure such that this can be used for different architectures such as the powerpc, etc. [AVM] “ARCHTYPE” is getting only used while cross-compiling, and to distinguish architecture flavors: x86, PowerPC, m68k ,ect . In the code arch_word is used while encoding whether to do MDS_ENC_TYPE_FLAT/MDS_ENC_TYPE_FULL. Keeping “ARCHTYPE” doesn't have any significance in current Open-sourced Opnesaf code , [Surya] This is provided to accommodate different architectures and is a build option which optimizes the applications way of message encoding(full/flat) and corresponding decode(full/flat). So that means, you want to remove this support and introduce the over head again. One more thing, with the new changes if you have two nodes one with powerpc and intel, both will not be able to communicate each other as both will result in flat encode/flat decode(as the value is one with the latest changes). why because In the current MDS messaging we do full encoded when the messages were sent to another process/node, irrelevant of “ARCHTYPE” [Surya] Wrong, the decision is done always on the archword only. I already verified cross 32/64 middle-ware application combination , all the fowls are working fine . [Surya] As you said this might work, but you are loosing the optimization and would always result in full encode/full decode of application messages which is an overhead. I am thinking of a solution for this and will get back to you. [AVM] we already have an alternative option of new node do install subscribe its Node Mds version, to handle in-service Upgrade , but we used legacy “ARCHTYPE” 3 bits for Mds version which is optimized solution then install subscribe. [Surya] Thats what we had discussed before as well. The archtype is not a legacy one, it was introduced to have the optimization for the message encoding by the applications. -AVM Regards Surya
Re: [devel] [PATCH 3 of 4] mds: use TIPC segmentation/reassembly [#654]
enhancement of the messaging. 2. 64 bit and 32 bit combination will not work on the local node as it is doing flat encode which is wrong. So I agree with Hans and let us not regressed for Un exposed feature , [Surya] Feature is present, the only thing is it was not used. and we will use the ARCHTYPE 3 bits for MDS version unless we get alternate bits/variables used for MDS version. [Surya] Thats the reason i am asking for some time. I also NOT much in favor of little bit expensive `new Node install subscribe its Mds version` (My previously published approach) So following are current options : - Use ARCHTYPE 3 bits for MDS version and discard un exposed feature ( `mds_arch` config option) - Find out alter native bits for MDS version fix all the ENC_TYPE_FLAT issue with in-service upgrade and expose `same arc message optimization` feature to OpenSAF users. - Use `new Node install subscribe` option for Mds version (My previously published approach) and fix all the ENC_TYPE_FLAT issue with in-service upgrade and expose `same arc message optimization` feature to OpenSAF users. -AVM On 5/1/2014 2:45 PM, SuryaNarayana Garlapati wrote: On Thursday 01 May 2014 02:30 PM, Hans Feldt wrote: Let's be clear that MDS a pure internal opensaf service since quite a long time. There are no applications to consider at all. This means the only thing to care about is in-service performance and making sure characteristics are improved, not regressed. [Surya] I am saying in the context of internal messaging between the services which use MDS. Yes, thats what even i am trying to. I am not sure we have to maintain some legacy requirement of mixed 32/64 bit clients on the same system. I don't see why we should. [Surya] This is not limited to even 32/64 bit. Its how the encoding is handled for the destinations. Thanks, Hans -Original Message- From: SuryaNarayana Garlapati [mailto:suryanarayana.garlap...@oracle.com] Sent: den 1 maj 2014 10:54 To: A V Mahesh Cc: opensaf-devel@lists.sourceforge.net Subject: Re: [devel] [PATCH 3 of 4] mds: use TIPC segmentation/reassembly [#654] On Thursday 01 May 2014 01:04 PM, A V Mahesh wrote: On 5/1/2014 10:16 AM, SuryaNarayana Garlapati wrote: The archtype is not a legacy one, it was introduced to have the optimization for the message encoding by the applications. Passing compile flags `mds_arch=1` `mds_arch=2` (mds_arch, [The arch-type input to MDS - valid values 1-7]) , is not a standard Open-Source ( enterprise things for legacy product ) option for compilations. [Surya] Wrong, if you see any standard linux configure options for any linux application code distributions, they do provide the options for supporting the options to provide such powerpc, intel, 32/64bit etc. So similar is the configure options for opensaf as well. For example you can see the configure options for the net-snmp as well. Open source user not need to Understand/get-Educate about this internal options , [Surya] This is not a internal option, configure option which is exposed to user as well. we are ok loose advantage of flat encode optimization , from now we want to send full_encode to different node. [Surya] This is not only done for optimization, this internally effects the message performance between the opensaf services which do huge messaging. I am on way of thinking the solution for this. Lets wait. We are also raising enhancement ticket for code clean. -AVM On 5/1/2014 10:16 AM, SuryaNarayana Garlapati wrote: On Thursday 01 May 2014 09:56 AM, A V Mahesh wrote: Hi Surya, On 5/1/2014 7:28 AM, Suryanarayana Garlapati wrote: Nack. MDS ARCHWORD is used for deciding the factor whether the message which is being sent to the destination should be encoded full/flat/copied. This bits were introduced to do a optimization in the way the callbacks encode full/flat are called. For example, if there are two processes(can be present in the same node or different node) and there architecture and bit size is same, so in this case, the encode flat can be called (or simply a flat copy of the message can be done in the UB). If these bits were not used, earlier the messages was being full encoded when the messages were sent to another node. At the same time, the arch field is given in the configure such that this can be used for different architectures such as the powerpc, etc. [AVM] “ARCHTYPE” is getting only used while cross-compiling, and to distinguish architecture flavors: x86, PowerPC, m68k ,ect . In the code arch_word is used while encoding whether to do MDS_ENC_TYPE_FLAT/MDS_ENC_TYPE_FULL. Keeping “ARCHTYPE” doesn't have any significance in current Open-sourced Opnesaf code , [Surya] This is provided to accommodate different architectures and is a build option which optimizes the applications way of message encoding(full/flat
Re: [devel] [PATCH 3 of 4] mds: use TIPC segmentation/reassembly [#654]
Hi All, I re-verified the behavior of OpenSAF MDS_ENC_TYPE_FULL/MDS_ENC_TYPE_FLAT messaging and its optimization , following is my observation and option: 1) In the default OpenSAF configuration (` mds_arch=0` ) , messaging across the node is ENC_TYPE_FULL and messaging with-in the node is ENC_TYPE_FLAT . 2) Even though Opensaf is configured with default `mds_arch=0`, if their is difference of 32--bit application communicating to 64-bit middle-ware ( mixed 32/64 bit clients on the same Node), messaging with-in the node is also ENC_TYPE_FULL. 3) when OpenSAF is configured with `mds_arch=1... to 7` messaging across with-in the node is ENC_TYPE_FLAT , hear we have advantage of message optimization , only when explicitly enabling this configuration . 4) Unfortunate the same arc message optimization feature got broken in few services such as RDE PLM doesn't support ENC_TYPE_FLAT , we need fix these issue along with in-service upgrade to make it work properly . 5) One more important note is that the `same arc message optimization` feature was not exposed to OpenSAF user till now. 6) This current patch Honors the default Opensaf configuration ` mds_arch=0` behavior ( point 1 2 ) and it removed hidden `same arc message optimization` feature related code and used those 3bits/variable for the purpose of MDS version. So I agree with Hans and let us not regressed for Un exposed feature , and we will use the ARCHTYPE 3 bits for MDS version unless we get alternate bits/variables used for MDS version. I also NOT much in favor of little bit expensive `new Node install subscribe its Mds version` (My previously published approach) So following are current options : - Use ARCHTYPE 3 bits for MDS version and discard un exposed feature ( `mds_arch` config option) - Find out alter native bits for MDS version fix all the ENC_TYPE_FLAT issue with in-service upgrade and expose `same arc message optimization` feature to OpenSAF users. - Use `new Node install subscribe` option for Mds version (My previously published approach) and fix all the ENC_TYPE_FLAT issue with in-service upgrade and expose `same arc message optimization` feature to OpenSAF users. -AVM On 5/1/2014 2:45 PM, SuryaNarayana Garlapati wrote: On Thursday 01 May 2014 02:30 PM, Hans Feldt wrote: Let's be clear that MDS a pure internal opensaf service since quite a long time. There are no applications to consider at all. This means the only thing to care about is in-service performance and making sure characteristics are improved, not regressed. [Surya] I am saying in the context of internal messaging between the services which use MDS. Yes, thats what even i am trying to. I am not sure we have to maintain some legacy requirement of mixed 32/64 bit clients on the same system. I don't see why we should. [Surya] This is not limited to even 32/64 bit. Its how the encoding is handled for the destinations. Thanks, Hans -Original Message- From: SuryaNarayana Garlapati [mailto:suryanarayana.garlap...@oracle.com] Sent: den 1 maj 2014 10:54 To: A V Mahesh Cc: opensaf-devel@lists.sourceforge.net Subject: Re: [devel] [PATCH 3 of 4] mds: use TIPC segmentation/reassembly [#654] On Thursday 01 May 2014 01:04 PM, A V Mahesh wrote: On 5/1/2014 10:16 AM, SuryaNarayana Garlapati wrote: The archtype is not a legacy one, it was introduced to have the optimization for the message encoding by the applications. Passing compile flags `mds_arch=1` `mds_arch=2` (mds_arch, [The arch-type input to MDS - valid values 1-7]) , is not a standard Open-Source ( enterprise things for legacy product ) option for compilations. [Surya] Wrong, if you see any standard linux configure options for any linux application code distributions, they do provide the options for supporting the options to provide such powerpc, intel, 32/64bit etc. So similar is the configure options for opensaf as well. For example you can see the configure options for the net-snmp as well. Open source user not need to Understand/get-Educate about this internal options , [Surya] This is not a internal option, configure option which is exposed to user as well. we are ok loose advantage of flat encode optimization , from now we want to send full_encode to different node. [Surya] This is not only done for optimization, this internally effects the message performance between the opensaf services which do huge messaging. I am on way of thinking the solution for this. Lets wait. We are also raising enhancement ticket for code clean. -AVM On 5/1/2014 10:16 AM, SuryaNarayana Garlapati wrote: On Thursday 01 May 2014 09:56 AM, A V Mahesh wrote: Hi Surya, On 5/1/2014 7:28 AM, Suryanarayana Garlapati wrote: Nack. MDS ARCHWORD is used for deciding the factor whether the message which is being sent to the destination should
Re: [devel] [PATCH 3 of 4] mds: use TIPC segmentation/reassembly [#654]
On 5/1/2014 10:16 AM, SuryaNarayana Garlapati wrote: The archtype is not a legacy one, it was introduced to have the optimization for the message encoding by the applications. Passing compile flags `mds_arch=1` `mds_arch=2` (mds_arch, [The arch-type input to MDS - valid values 1-7]) , is not a standard Open-Source ( enterprise things for legacy product ) option for compilations. Open source user not need to Understand/get-Educate about this internal options , we are ok loose advantage of flat encode optimization , from now we want to send full_encode to different node. We are also raising enhancement ticket for code clean. -AVM On 5/1/2014 10:16 AM, SuryaNarayana Garlapati wrote: On Thursday 01 May 2014 09:56 AM, A V Mahesh wrote: Hi Surya, On 5/1/2014 7:28 AM, Suryanarayana Garlapati wrote: Nack. MDS ARCHWORD is used for deciding the factor whether the message which is being sent to the destination should be encoded full/flat/copied. This bits were introduced to do a optimization in the way the callbacks encode full/flat are called. For example, if there are two processes(can be present in the same node or different node) and there architecture and bit size is same, so in this case, the encode flat can be called (or simply a flat copy of the message can be done in the UB). If these bits were not used, earlier the messages was being full encoded when the messages were sent to another node. At the same time, the arch field is given in the configure such that this can be used for different architectures such as the powerpc, etc. [AVM] “ARCHTYPE” is getting only used while cross-compiling, and to distinguish architecture flavors: x86, PowerPC, m68k ,ect . In the code arch_word is used while encoding whether to do MDS_ENC_TYPE_FLAT/MDS_ENC_TYPE_FULL. Keeping “ARCHTYPE” doesn't have any significance in current Open-sourced Opnesaf code , [Surya] This is provided to accommodate different architectures and is a build option which optimizes the applications way of message encoding(full/flat) and corresponding decode(full/flat). So that means, you want to remove this support and introduce the over head again. One more thing, with the new changes if you have two nodes one with powerpc and intel, both will not be able to communicate each other as both will result in flat encode/flat decode(as the value is one with the latest changes). why because In the current MDS messaging we do full encoded when the messages were sent to another process/node, irrelevant of “ARCHTYPE” [Surya] Wrong, the decision is done always on the archword only. I already verified cross 32/64 middle-ware application combination , all the fowls are working fine . [Surya] As you said this might work, but you are loosing the optimization and would always result in full encode/full decode of application messages which is an overhead. I am thinking of a solution for this and will get back to you. [AVM] we already have an alternative option of new node do install subscribe its Node Mds version, to handle in-service Upgrade , but we used legacy “ARCHTYPE” 3 bits for Mds version which is optimized solution then install subscribe. [Surya] Thats what we had discussed before as well. The archtype is not a legacy one, it was introduced to have the optimization for the message encoding by the applications. -AVM Regards Surya - Original Message - From: osafde...@gmail.com To: mahesh.va...@oracle.com Cc: opensaf-devel@lists.sourceforge.net Sent: Tuesday, April 29, 2014 10:58:53 AM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: [devel] [PATCH 3 of 4] mds: use TIPC segmentation/reassembly [#654] configure.ac | 21 - osaf/libs/core/mds/Makefile.am| 1 - osaf/libs/core/mds/include/mds_dt2c.h | 17 - osaf/libs/core/mds/mds_c_sndrcv.c | 14 +- osaf/libs/core/mds/mds_dt_tipc.c | 11 ++- osaf/libs/core/mds/mds_log.c | 4 ++-- 6 files changed, 25 insertions(+), 43 deletions(-) TIPC supports sending message with a maximum length of 66000 bytes. MDS supports sending even bigger messages, upto ~46Mb. MDS has used an internal fragmentation of just 1400 bytes (MDTM_NORMAL_MSG_FRAG_SIZE) despite that the receive buffer is 8000 bytes (MDS_DIRECT_BUF_MAXSIZE) This patch changes MDS to use TIPC segmentation/reassembly. This gives benefits such as secure transmission (TIPC does retransmission). TIPC send congestion is reduced since TIPC counts messages not bytes. Less processing done in user space in the sending opensaf or client process. Since the receive buffer of older MDS cores is limited to 8000 bytes we have be sure not to send larger messages than that to older cores. Otherwise we have problems with in-service upgradeability
Re: [devel] [PATCH 3 of 4] mds: use TIPC segmentation/reassembly [#654]
Let's be clear that MDS a pure internal opensaf service since quite a long time. There are no applications to consider at all. This means the only thing to care about is in-service performance and making sure characteristics are improved, not regressed. I am not sure we have to maintain some legacy requirement of mixed 32/64 bit clients on the same system. I don't see why we should. Thanks, Hans -Original Message- From: SuryaNarayana Garlapati [mailto:suryanarayana.garlap...@oracle.com] Sent: den 1 maj 2014 10:54 To: A V Mahesh Cc: opensaf-devel@lists.sourceforge.net Subject: Re: [devel] [PATCH 3 of 4] mds: use TIPC segmentation/reassembly [#654] On Thursday 01 May 2014 01:04 PM, A V Mahesh wrote: On 5/1/2014 10:16 AM, SuryaNarayana Garlapati wrote: The archtype is not a legacy one, it was introduced to have the optimization for the message encoding by the applications. Passing compile flags `mds_arch=1` `mds_arch=2` (mds_arch, [The arch-type input to MDS - valid values 1-7]) , is not a standard Open-Source ( enterprise things for legacy product ) option for compilations. [Surya] Wrong, if you see any standard linux configure options for any linux application code distributions, they do provide the options for supporting the options to provide such powerpc, intel, 32/64bit etc. So similar is the configure options for opensaf as well. For example you can see the configure options for the net-snmp as well. Open source user not need to Understand/get-Educate about this internal options , [Surya] This is not a internal option, configure option which is exposed to user as well. we are ok loose advantage of flat encode optimization , from now we want to send full_encode to different node. [Surya] This is not only done for optimization, this internally effects the message performance between the opensaf services which do huge messaging. I am on way of thinking the solution for this. Lets wait. We are also raising enhancement ticket for code clean. -AVM On 5/1/2014 10:16 AM, SuryaNarayana Garlapati wrote: On Thursday 01 May 2014 09:56 AM, A V Mahesh wrote: Hi Surya, On 5/1/2014 7:28 AM, Suryanarayana Garlapati wrote: Nack. MDS ARCHWORD is used for deciding the factor whether the message which is being sent to the destination should be encoded full/flat/copied. This bits were introduced to do a optimization in the way the callbacks encode full/flat are called. For example, if there are two processes(can be present in the same node or different node) and there architecture and bit size is same, so in this case, the encode flat can be called (or simply a flat copy of the message can be done in the UB). If these bits were not used, earlier the messages was being full encoded when the messages were sent to another node. At the same time, the arch field is given in the configure such that this can be used for different architectures such as the powerpc, etc. [AVM] “ARCHTYPE” is getting only used while cross-compiling, and to distinguish architecture flavors: x86, PowerPC, m68k ,ect . In the code arch_word is used while encoding whether to do MDS_ENC_TYPE_FLAT/MDS_ENC_TYPE_FULL. Keeping “ARCHTYPE” doesn't have any significance in current Open-sourced Opnesaf code , [Surya] This is provided to accommodate different architectures and is a build option which optimizes the applications way of message encoding(full/flat) and corresponding decode(full/flat). So that means, you want to remove this support and introduce the over head again. One more thing, with the new changes if you have two nodes one with powerpc and intel, both will not be able to communicate each other as both will result in flat encode/flat decode(as the value is one with the latest changes). why because In the current MDS messaging we do full encoded when the messages were sent to another process/node, irrelevant of “ARCHTYPE” [Surya] Wrong, the decision is done always on the archword only. I already verified cross 32/64 middle-ware application combination , all the fowls are working fine . [Surya] As you said this might work, but you are loosing the optimization and would always result in full encode/full decode of application messages which is an overhead. I am thinking of a solution for this and will get back to you. [AVM] we already have an alternative option of new node do install subscribe its Node Mds version, to handle in-service Upgrade , but we used legacy “ARCHTYPE” 3 bits for Mds version which is optimized solution then install subscribe. [Surya] Thats what we had discussed before as well. The archtype is not a legacy one, it was introduced to have the optimization for the message encoding by the applications. -AVM Regards Surya - Original Message - From: osafde
Re: [devel] [PATCH 3 of 4] mds: use TIPC segmentation/reassembly [#654]
On Thursday 01 May 2014 02:30 PM, Hans Feldt wrote: Let's be clear that MDS a pure internal opensaf service since quite a long time. There are no applications to consider at all. This means the only thing to care about is in-service performance and making sure characteristics are improved, not regressed. [Surya] I am saying in the context of internal messaging between the services which use MDS. Yes, thats what even i am trying to. I am not sure we have to maintain some legacy requirement of mixed 32/64 bit clients on the same system. I don't see why we should. [Surya] This is not limited to even 32/64 bit. Its how the encoding is handled for the destinations. Thanks, Hans -Original Message- From: SuryaNarayana Garlapati [mailto:suryanarayana.garlap...@oracle.com] Sent: den 1 maj 2014 10:54 To: A V Mahesh Cc: opensaf-devel@lists.sourceforge.net Subject: Re: [devel] [PATCH 3 of 4] mds: use TIPC segmentation/reassembly [#654] On Thursday 01 May 2014 01:04 PM, A V Mahesh wrote: On 5/1/2014 10:16 AM, SuryaNarayana Garlapati wrote: The archtype is not a legacy one, it was introduced to have the optimization for the message encoding by the applications. Passing compile flags `mds_arch=1` `mds_arch=2` (mds_arch, [The arch-type input to MDS - valid values 1-7]) , is not a standard Open-Source ( enterprise things for legacy product ) option for compilations. [Surya] Wrong, if you see any standard linux configure options for any linux application code distributions, they do provide the options for supporting the options to provide such powerpc, intel, 32/64bit etc. So similar is the configure options for opensaf as well. For example you can see the configure options for the net-snmp as well. Open source user not need to Understand/get-Educate about this internal options , [Surya] This is not a internal option, configure option which is exposed to user as well. we are ok loose advantage of flat encode optimization , from now we want to send full_encode to different node. [Surya] This is not only done for optimization, this internally effects the message performance between the opensaf services which do huge messaging. I am on way of thinking the solution for this. Lets wait. We are also raising enhancement ticket for code clean. -AVM On 5/1/2014 10:16 AM, SuryaNarayana Garlapati wrote: On Thursday 01 May 2014 09:56 AM, A V Mahesh wrote: Hi Surya, On 5/1/2014 7:28 AM, Suryanarayana Garlapati wrote: Nack. MDS ARCHWORD is used for deciding the factor whether the message which is being sent to the destination should be encoded full/flat/copied. This bits were introduced to do a optimization in the way the callbacks encode full/flat are called. For example, if there are two processes(can be present in the same node or different node) and there architecture and bit size is same, so in this case, the encode flat can be called (or simply a flat copy of the message can be done in the UB). If these bits were not used, earlier the messages was being full encoded when the messages were sent to another node. At the same time, the arch field is given in the configure such that this can be used for different architectures such as the powerpc, etc. [AVM] “ARCHTYPE” is getting only used while cross-compiling, and to distinguish architecture flavors: x86, PowerPC, m68k ,ect . In the code arch_word is used while encoding whether to do MDS_ENC_TYPE_FLAT/MDS_ENC_TYPE_FULL. Keeping “ARCHTYPE” doesn't have any significance in current Open-sourced Opnesaf code , [Surya] This is provided to accommodate different architectures and is a build option which optimizes the applications way of message encoding(full/flat) and corresponding decode(full/flat). So that means, you want to remove this support and introduce the over head again. One more thing, with the new changes if you have two nodes one with powerpc and intel, both will not be able to communicate each other as both will result in flat encode/flat decode(as the value is one with the latest changes). why because In the current MDS messaging we do full encoded when the messages were sent to another process/node, irrelevant of “ARCHTYPE” [Surya] Wrong, the decision is done always on the archword only. I already verified cross 32/64 middle-ware application combination , all the fowls are working fine . [Surya] As you said this might work, but you are loosing the optimization and would always result in full encode/full decode of application messages which is an overhead. I am thinking of a solution for this and will get back to you. [AVM] we already have an alternative option of new node do install subscribe its Node Mds version, to handle in-service Upgrade , but we used legacy “ARCHTYPE” 3 bits for Mds version which is optimized solution then install subscribe. [Surya] Thats what we had discussed before as well
[devel] [PATCH 3 of 4] mds: use TIPC segmentation/reassembly [#654]
configure.ac | 21 -
osaf/libs/core/mds/Makefile.am| 1 -
osaf/libs/core/mds/include/mds_dt2c.h | 17 -
osaf/libs/core/mds/mds_c_sndrcv.c | 14 +-
osaf/libs/core/mds/mds_dt_tipc.c | 11 ++-
osaf/libs/core/mds/mds_log.c | 4 ++--
6 files changed, 25 insertions(+), 43 deletions(-)
TIPC supports sending message with a maximum length of 66000 bytes. MDS supports
sending even bigger messages, upto ~46Mb. MDS has used an internal fragmentation
of just 1400 bytes (MDTM_NORMAL_MSG_FRAG_SIZE) despite that the receive buffer
is 8000 bytes (MDS_DIRECT_BUF_MAXSIZE)
This patch changes MDS to use TIPC segmentation/reassembly. This gives benefits
such as secure transmission (TIPC does retransmission). TIPC send congestion is
reduced since TIPC counts messages not bytes. Less processing done in user
space in the sending opensaf or client process.
Since the receive buffer of older MDS cores is limited to 8000 bytes we have be
sure not to send larger messages than that to older cores. Otherwise we have
problems with in-service upgradeability.
Each MDS TIPC port is bound to a functional address - a port name. The port name
consists of 32 bit type and 32 bit instance. The 4 msb of the instance word
contains the so called archword field. The msb indicates if this MDS core is
32 or 64 bit. The remaining 3 bits is now used as a version field. Version 0 is
used by older versions of MDS, version 1 is now used to indicate the capability
for longer fragmentation.
This information is part of the published port name thus a subscriber will
get them using the TIPC discovery service. These bits are ignored by old MDS
cores, so it can receive messages from a new MDS core. When sending a message
this bit is used to understand if the peer uses TIPC segmentation/reassembly
(or not) and adapt the limit accordingly.
The possibility to set mds_arch using configure has been removed.
The change is in-service compliant and can be upgraded into a system.
diff --git a/configure.ac b/configure.ac
--- a/configure.ac
+++ b/configure.ac
@@ -470,27 +470,6 @@ if test $enable_java = no; then
fi
#
-# MDS ARCH TYPE: MDS advises message encode/decode
-# optimization to its clients if the sender and receiver
-# are of compatible-type. For this optimization to work,
-# the sender's platform and receiver's platform should be
-# similar (for e.g. both PowerPC) and use an libncs_core built
-# with the same value for mds_arch. An mds_arch value of 0
-# is the default value and stands for a platform which is NOT
-# compatible with any platform.
-#
-
-AC_ARG_VAR([mds_arch], [The arch-type input to MDS - valid values 1-7])
-
-if test $mds_arch = ; then
-mdsarchtype=0
-else
-mdsarchtype=$mds_arch
-fi
-
-AC_SUBST([MDS_WORD_ARCH_FLAGS], [ -DMDS_ARCH_TYPE=$mdsarchtype])
-
-#
# Checks for libraries.
#
PKG_CHECK_MODULES([XML2], [libxml-2.0])
diff --git a/osaf/libs/core/mds/Makefile.am b/osaf/libs/core/mds/Makefile.am
--- a/osaf/libs/core/mds/Makefile.am
+++ b/osaf/libs/core/mds/Makefile.am
@@ -23,7 +23,6 @@ SUBDIRS = include
noinst_LTLIBRARIES = libmds.la
libmds_la_CPPFLAGS = \
- @MDS_WORD_ARCH_FLAGS@ \
$(AM_CPPFLAGS)
libmds_la_LDFLAGS = -static
diff --git a/osaf/libs/core/mds/include/mds_dt2c.h
b/osaf/libs/core/mds/include/mds_dt2c.h
--- a/osaf/libs/core/mds/include/mds_dt2c.h
+++ b/osaf/libs/core/mds/include/mds_dt2c.h
@@ -33,17 +33,16 @@
typedef uint8_t MDS_SVC_ARCHWORD_TYPE; /*MDS app-svc arch and word_size
combination */
-/* MDS_WORD_SIZE_TYPE and MDS_ARCH_TYPE are compile-time macros */
-
-#ifndef MDS_ARCH_TYPE
-#define MDS_ARCH_TYPE 0/* Stands for unspecified architecture
type */
-#elif (MDS_ARCH_TYPE 7)
-#error MDS_ARCH_TYPE should be in the range 0 to 7.
-#endif
-
#define MDS_WORD_SIZE_TYPE ((sizeof(long)/4) - 1) /* 0 for 32-bit, 1 for
64-bit */
-#define MDS_SELF_ARCHWORD((MDS_SVC_ARCHWORD_TYPE) ((MDS_WORD_SIZE_TYPE3)
| MDS_ARCH_TYPE))
+/*
+ * 4 bit ARCHWORD usage:
+ * Bit 3 is wordsize
+ * Bit 2:0 is a version field indicating capabilities.
+ *Version 0 uses 1400 bytes fragmentation.
+ *Version 1 uses TIPC max msg (66000 bytes) fragmentation.
+ */
+#define MDS_SELF_ARCHWORD ((MDS_SVC_ARCHWORD_TYPE) ((MDS_WORD_SIZE_TYPE3) |
1))
typedef enum {
MDS_VIEW_NORMAL,
diff --git a/osaf/libs/core/mds/mds_c_sndrcv.c
b/osaf/libs/core/mds/mds_c_sndrcv.c
--- a/osaf/libs/core/mds/mds_c_sndrcv.c
+++ b/osaf/libs/core/mds/mds_c_sndrcv.c
@@ -1483,10 +1483,10 @@ static uint32_t mcm_msg_encode_full_or_f
msg_send.dest_pwe_id = m_MDS_GET_PWE_ID_FROM_SVC_HDL(svc_cb-svc_hdl);
msg_send.dest_vdest_id = dest_vdest_id;
msg_send.src_svc_sub_part_ver =
