[
https://issues.apache.org/jira/browse/MAPREDUCE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18044309#comment-18044309
]
ASF GitHub Bot commented on MAPREDUCE-7290:
-------------------------------------------
github-actions[bot] commented on PR #2200:
URL: https://github.com/apache/hadoop/pull/2200#issuecomment-3639488764
We're closing this stale PR because it has been open for 100 days with no
activity. This isn't a judgement on the merit of the PR in any way. It's just a
way of keeping the PR queue manageable.
If you feel like this was a mistake, or you would like to continue working
on it, please feel free to re-open it and ask for a committer to remove the
stale tag and review again.
Thanks all for your contribution.
> ShuffleHeader should be compatible between client&server when rolling update
> ----------------------------------------------------------------------------
>
> Key: MAPREDUCE-7290
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7290
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 2.6.0
> Reporter: chaoli
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.9.0, 3.3.0
>
> Attachments:
> 0001-MAPREDUCE-7290-ShuffleHeader-should-be-compatible-be.patch,
> 0001-MAPREDUCE-7290-ShuffleHeader-should-be-compatible-be.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Recently, we need to add a property in ShuffleHeader class. And there were a
> problem that client&server may not compatible because of
> serialization/deserialization when we were rolling update NodeManager
> instance.
> To solve the problem, we made an improvement to help shuffle header version
> upgrade smoothly.
> In HeaderVersion class, shuffle client uses *targetVersion(eg 1.2.0)* to
> negotiate with shuffle server. In that case, shuffle server has three cases:
> # shuffle server does not have the newest code which is able to support
> version chosen, and we have to use *defaultVersion(eg 1.0.0)* instead*.*
> # shuffle server can support older version(eg 1.1.0), then it can reply to
> shuffle client with *compatibleVersion(eg 1.1.0),* and finally, client and
> server can use the same version.
> # shuffle server can support older version(eg 1.3.0), then it can reply to
> shuffle client with *compatibleVersion(eg 1.2.0),* and finally, client and
> server can use the same version.
> Then, client and server can serialize&deserialize property using the
> available version.
>
> {code:java}
> /**
> * unified hard code header version for new properties.
> * @throws IOException
> */
> private void writeByVersion(DataOutput out) throws IOException {
> if (headerVersion == null ||
> headerVersion.compareTo(DEFAULT_HEADER_VERSION_INSTANCE) == 0) {
> return;
> }
> // if current version larger then target version,
> // we should serialize the properties owned by the version in order.
> if (headerVersion.compareTo(HEADER_VERSION_INSTANCE_V1_1) >= 0) {
> // here write version properties
> }
> }
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
