[ 
https://issues.apache.org/jira/browse/MAPREDUCE-4951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13560736#comment-13560736
 ] 

Jason Lowe commented on MAPREDUCE-4951:
---------------------------------------

bq. having the RM ask the AM to kill the container in case of preemption would 
likely not work as the AM cannot be trusted.

Agreed, I was thinking of exactly the alternative you propose where preemption 
has potentially two phases, a "please AM, preempt that container you have" with 
a watchdog timer to have the RM kill it forcefully if the AM does not comply in 
a reasonable amount of time.  This eliminates the race where the container can 
fail because of the preemption and provides a way for the AM to potentially 
checkpoint the state of the container for faster recovery.  However it does 
mean the meantime latency for container availability would be higher since the 
AM will have a grace period before relinquishing the resources.
                
> Container preemption interpreted as task failure
> ------------------------------------------------
>
>                 Key: MAPREDUCE-4951
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4951
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: applicationmaster, mr-am, mrv2
>    Affects Versions: 2.0.2-alpha
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>         Attachments: MAPREDUCE-4951-1.patch, MAPREDUCE-4951-2.patch, 
> MAPREDUCE-4951.patch
>
>
> When YARN reports a completed container to the MR AM, it always interprets it 
> as a failure.  This can lead to a job failing because too many of its tasks 
> failed, when in fact they only failed because the scheduler preempted them.
> MR needs to recognize the special exit code value of -100 and interpret it as 
> a container being killed instead of a container failure.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Reply via email to