[
https://issues.apache.org/jira/browse/MAPREDUCE-5502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13772604#comment-13772604
]
Vrushali C commented on MAPREDUCE-5502:
---------------------------------------
Right. So the fix seems to be in two places in YARNRunner.
I think that the first call to resMgrDelegate.killApplication should occur when
the job is in PREP state, (as opposed to !RUNNING in the current code). At this
time, there is no RUNNING/FAILED/SUCCEEDED/KILLED status since it probably has
not even started running. Hence the kill to RM and return would make sense. In
this case, the application ends up in KILLED/KILLED which is correct according
to me. The "Tracking URL: History" on the cluster/app/application_number page
points to itself, which is also correct in this case I think.
The second call to resMgrDelegate.killApplication should occur when the
JobStatus is in any of the terminal states - KILLED/SUCCEEDED/FAILED. In this
case, the application ends up in FINISHED/KILLED , FINISHED/SUCCEEDED (I
haven't yet experimented with FAILED). The tracking URL on the application page
is also updated correctly.
The changes are:
---
a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
+++
b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
@@ -550,8 +550,12 @@ public JobStatus getJobStatus(JobID jobID) throws
IOException,
public void killJob(JobID arg0) throws IOException, InterruptedException {
/* check if the status is not running, if not send kill to RM */
JobStatus status = clientCache.getClient(arg0).getJobStatus(arg0);
- if (status.getState() != JobStatus.State.RUNNING) {
+ if ( (status.getState() == JobStatus.State.PREP) ) {
resMgrDelegate.killApplication(TypeConverter.toYarn(arg0).getAppId());
return;
}
@@ -574,7 +578,11 @@ public void killJob(JobID arg0) throws IOException,
InterruptedException {
} catch(IOException io) {
LOG.debug("Error when checking for application status", io);
}
- if (status.getState() != JobStatus.State.KILLED) {
+ if ( (status.getState() != JobStatus.State.KILLED) &&
+ (status.getState() != JobStatus.State.FAILED) &&
+ (status.getState() != JobStatus.State.SUCCEEDED) ){
resMgrDelegate.killApplication(TypeConverter.toYarn(arg0).getAppId());
}
What do you think?
> History link in resource manager is broken for KILLED jobs
> ----------------------------------------------------------
>
> Key: MAPREDUCE-5502
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5502
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: resourcemanager
> Affects Versions: 2.0.5-alpha
> Reporter: Vrushali C
> Assignee: Vrushali C
> Labels: ui
>
> History link in resource manager is broken for KILLED jobs.
> Seems to happen with jobs with State 'KILLED' and FinalStatus 'KILLED'. If
> the State is 'FINISHED' and FinalStatus is 'KILLED', then the "History" link
> is fine.
> It isn't easy to reproduce the problem since the time at which the app is
> killed determines the state it ends up in, which is hard to guess. these
> particular jobs seem to get a Diagnostics message of "Application killed by
> user." where as the other killed jobs get " Kill Job received from client
> job_1378766187901_0002
> Job received Kill while in RUNNING state. "
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira