[
https://issues.apache.org/jira/browse/MAPREDUCE-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13543425#comment-13543425
]
Jason Lowe commented on MAPREDUCE-4819:
---------------------------------------
bq. If we change this code to create new file or fail then AM knows when it has
lost its race to commit. Does this provide a simpler fix for MAPREDUCE-4832?
If an app attempt sees the file, how does it even know whether there's an
active race that was lost? The other AM could have simply crashed mid-commit.
The losing AM could just assume that's the case and unregister from the RM with
a FAILED status assuming job commit failed. (Or maybe wait for some
configurable timeout "just in case".)
However this would only cover job commit, and two racing app attempts could
still commit output for tasks simultaneously. MAPREDUCE-4832 prevents two
racing app attempts from committing the same task output, as at most one will
be "active" and allowed to commit. That could be bad if the old attempt is
re-committing output for a fetch-failure map task while the second attempt is
trying to recover, for example. Task output could be lost in that case.
> AM can rerun job after reporting final job status to the client
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-4819
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4819
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am
> Affects Versions: 0.23.3, 2.0.1-alpha
> Reporter: Jason Lowe
> Assignee: Bikas Saha
> Priority: Critical
> Attachments: MAPREDUCE-4819.1.patch, MAPREDUCE-4819.2.patch,
> MAPREDUCE-4819.3.patch, MR-4819-bobby-trunk.txt, MR-4819-bobby-trunk.txt,
> MR-4819-bobby-trunk.txt, MR-4819-bobby-trunk.txt, MR-4819-bobby-trunk.txt
>
>
> If the AM reports final job status to the client but then crashes before
> unregistering with the RM then the RM can run another AM attempt. Currently
> AM re-attempts assume that the previous attempts did not reach a final job
> state, and that causes the job to rerun (from scratch, if the output format
> doesn't support recovery).
> Re-running the job when we've already told the client the final status of the
> job is bad for a number of reasons. If the job failed, it's confusing at
> best since the client was already told the job failed but the subsequent
> attempt could succeed. If the job succeeded there could be data loss, as a
> subsequent job launched by the client tries to consume the job's output as
> input just as the re-attempt starts removing output files in preparation for
> the output commit.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
