[
https://issues.apache.org/jira/browse/MAPREDUCE-5841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13977881#comment-13977881
]
Sangjin Lee commented on MAPREDUCE-5841:
----------------------------------------
Updated the patch.
I added code that interrupts the task attempt for which a container clean-up is
being requested via Future.cancel(). System.exit() was replaced with
ExitUtil.terminate(). Also added a unit test.
I ran some -kill-task scenarios to confirm the patch works for them. For
example, for a 2-mapper (no reducer) uber job, if I kill any of the tasks, it
gets interrupted promptly (provided the task responds to interruption), a new
attempt is scheduled, and the job completes successfully.
I also tried a 2-mapper, 1-reducer uber job. I killed a mapper task, and
confirmed the running task is killed promptly. In this case the job eventually
fails because the reducer attempt detects that not all the mappers ran
successfully. I suspect this is somewhat of an expected behavior.
Regarding the aforementioned unit test timeouts, it turned out to be a fluke.
It may have been the state my machine was in when I ran the test network-wise.
I can no longer reproduce the timeouts under a stable condition.
> uber job doesn't terminate on getting mapred job kill
> -----------------------------------------------------
>
> Key: MAPREDUCE-5841
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5841
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.3.0
> Reporter: Sangjin Lee
> Assignee: Sangjin Lee
> Attachments: mapreduce-5841.patch, mapreduce-5841.patch
>
>
> If you issue a "mapred job -kill" against a uberized job, the job (and the
> yarn application) state transitions to KILLED, but the application master
> process continues to run. The job actually runs to completion despite the
> killed status.
> This can be easily reproduced by running a sleep job:
> {noformat}
> hadoop jar hadoop-mapreduce-client-jobclient-2.3.0-tests.jar sleep -m 1 -r 0
> -mt 300000
> {noformat}
> Issue a kill with "mapred job -kill \[job-id\]". The UI will show the job
> (app) is in the KILLED state. However, you can see the application master is
> still running.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
