[
https://issues.apache.org/jira/browse/MAPREDUCE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated MAPREDUCE-7290:
---------------------------------------
Target Version/s: 3.3.1 (was: 3.3.0)
> ShuffleHeader should be compatible between client&server when rolling update
> ----------------------------------------------------------------------------
>
> Key: MAPREDUCE-7290
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7290
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 2.6.0
> Reporter: chaoli
> Priority: Major
> Fix For: 2.9.0, 3.3.0
>
> Attachments:
> 0001-MAPREDUCE-7290-ShuffleHeader-should-be-compatible-be.patch,
> 0001-MAPREDUCE-7290-ShuffleHeader-should-be-compatible-be.patch
>
>
> Recently, we need to add a property in ShuffleHeader class. And there were a
> problem that client&server may not compatible because of
> serialization/deserialization when we were rolling update NodeManager
> instance.
> To solve the problem, we made an improvement to help shuffle header version
> upgrade smoothly.
> In HeaderVersion class, shuffle client uses *targetVersion(eg 1.2.0)* to
> negotiate with shuffle server. In that case, shuffle server has three cases:
> # shuffle server does not have the newest code which is able to support
> version chosen, and we have to use *defaultVersion(eg 1.0.0)* instead*.*
> # shuffle server can support older version(eg 1.1.0), then it can reply to
> shuffle client with *compatibleVersion(eg 1.1.0),* and finally, client and
> server can use the same version.
> # shuffle server can support older version(eg 1.3.0), then it can reply to
> shuffle client with *compatibleVersion(eg 1.2.0),* and finally, client and
> server can use the same version.
> Then, client and server can serialize&deserialize property using the
> available version.
>
> {code:java}
> /**
> * unified hard code header version for new properties.
> * @throws IOException
> */
> private void writeByVersion(DataOutput out) throws IOException {
> if (headerVersion == null ||
> headerVersion.compareTo(DEFAULT_HEADER_VERSION_INSTANCE) == 0) {
> return;
> }
> // if current version larger then target version,
> // we should serialize the properties owned by the version in order.
> if (headerVersion.compareTo(HEADER_VERSION_INSTANCE_V1_1) >= 0) {
> // here write version properties
> }
> }
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
