[
https://issues.apache.org/jira/browse/MAPREDUCE-3711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196508#comment-13196508
]
Robert Joseph Evans commented on MAPREDUCE-3711:
------------------------------------------------
OK thinking about it more adding in recover job adds in some potential race
conditions. It is possible that a task was in the process of committing its
output when the RM dies. That task could finish committing successfully, but
the jhist file will have no record of that happening. recoverJob would be able
to recover that output no problem, but we would still rerun the task. It is
possible for some task to try and use a random number or something to produce a
unique file name, this would result in multiple copies of the tasks output
showing up. I know that this is really a corner case, and it would mean that
someone somewhere is doing something that is not that wise to do, but I think
it is better to have the JHist file be the single source of truth and force the
HDFS file layout to support it. I will work on that patch instead.
> AppMaster recovery for Medium to large jobs take long time
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3711
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3711
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Robert Joseph Evans
> Priority: Blocker
>
> Reported by [~karams]
> yarn.resourcemanager.am.max-retries=2
> Ran test cases with sort job on 350 scale having 16800 maps and 680 reduces -:
> 1. After 70 secs of Job Sumbission Am is killed using kill -9, around 3900
> maps were completed and 680 reduces were
> scheduled, Second AM got restart. Job got completed in 980 secs. AM took very
> less time to recover.
> 2. After 150 secs of Job Sumbission AM is killed using kill -9, around 90%
> maps were completed and 680 reduces were
> scheduled , Second AM got restart Job got completed in 1000 secs. AM got
> revocer.
> 3. After 150 secs of Job Sumbission AM as killed using kill -9, almost all
> maps were completed and only 680 reduces
> were running, Recovery was too slow, AM was still revocering after 1hr :40
> mis when I killed the run.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
