[
https://issues.apache.org/jira/browse/MAPREDUCE-5485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15000789#comment-15000789
]
Bikas Saha commented on MAPREDUCE-5485:
---------------------------------------
Sorry I missed that the test committer derives from FileOutputCommitter and
throws the file not found exception.
About the overall test. The main overall change is to allow the retry AM to
continue after seeing an in-progress commit from the previous AM. It seems
incomplete to not have a test for that. However if you think that we dont have
existing infra for that code path then we should create a follow up jira to add
that infra and relevant tests. I have not followed the MR AM code changes for a
while and so I cannot recall of the top of my head about any existing test
cases. Maybe other committers may have some ideas.
With that caveat, the latest patch looks good to me. Thanks for your patience
through the reviews :)
> Allow repeating job commit by extending OutputCommitter API
> -----------------------------------------------------------
>
> Key: MAPREDUCE-5485
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5485
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 2.1.0-beta
> Reporter: Nemon Lou
> Assignee: Junping Du
> Priority: Critical
> Attachments: MAPREDUCE-5485-demo-2.patch, MAPREDUCE-5485-demo.patch,
> MAPREDUCE-5485-v1.patch, MAPREDUCE-5485-v2.patch, MAPREDUCE-5485-v3.1.patch,
> MAPREDUCE-5485-v3.patch, MAPREDUCE-5485-v4.1.patch, MAPREDUCE-5485-v4.patch,
> MAPREDUCE-5485-v5.patch
>
>
> There are chances MRAppMaster crush during job committing,or NodeManager
> restart cause the committing AM exit due to container expire.In these cases
> ,the job will fail.
> However,some jobs can redo commit so failing the job becomes unnecessary.
> Let clients tell AM to allow redo commit or not is a better choice.
> This idea comes from Jason Lowe's comments in MAPREDUCE-4819
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
