[
https://issues.apache.org/jira/browse/MAPREDUCE-5502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13771891#comment-13771891
]
Jason Lowe commented on MAPREDUCE-5502:
---------------------------------------
bq. From our investigation, it appears that the client's kill sends a KILL to
the App Master as well as to the RM for this App.
Were you actually seeing the AM shutdown due to the SIGTERM it would receive as
part of the YARN kill, or did you see "Kill job received from client ...." in
the AM logs as well? I see in YARNRunner.killJob that it can send a kill to
the AM and later to the RM if for 10 seconds the AM doesn't end up in the
KILLED state. That, too, seems to be a bug, since it really should be checking
not for state != KILLED but rather for state not in a terminal state, i.e.:
FAILED, KILLED, SUCCEEDED. Otherwise there's a race where the AM can enter a
terminal state on its own but the code later tries to kill it via YARN anyway.
bq. Similar to the patch in MAPREDUCE-5497, in YarnRunner's killJob function,
we added a sleep for a few seconds before the (2nd) call to
resMgrDelegate.killApplication where status.getState() != JobStatus.State.KILLED
In general I'm not a fan of sleeps as a "fix" since they're just masking a race
window rather than resolving the underlying condition. Sleeps also slow down
the process in general, and it would be better to solve it without them if
possible. Also MAPREDUCE-5497 didn't add a sleep, rather it moved an existing
sleep to later in the AM shutdown process. That sleep is simply there for the
AM to linger around for clients to fetch the final job status rather than
redirect to the history server. I'm not sure it's necessary anymore, actually.
I think a better fix would be to change YARNRunner.killJob to avoid sending a
kill to the RM if the reported job state is terminal rather than just checking
for KILLED.
> History link in resource manager is broken for KILLED jobs
> ----------------------------------------------------------
>
> Key: MAPREDUCE-5502
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5502
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: resourcemanager
> Affects Versions: 2.0.5-alpha
> Reporter: Vrushali C
> Assignee: Vrushali C
> Labels: ui
>
> History link in resource manager is broken for KILLED jobs.
> Seems to happen with jobs with State 'KILLED' and FinalStatus 'KILLED'. If
> the State is 'FINISHED' and FinalStatus is 'KILLED', then the "History" link
> is fine.
> It isn't easy to reproduce the problem since the time at which the app is
> killed determines the state it ends up in, which is hard to guess. these
> particular jobs seem to get a Diagnostics message of "Application killed by
> user." where as the other killed jobs get " Kill Job received from client
> job_1378766187901_0002
> Job received Kill while in RUNNING state. "
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira