[
https://issues.apache.org/jira/browse/MAPREDUCE-4367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400604#comment-13400604
]
Jason Lowe commented on MAPREDUCE-4367:
---------------------------------------
If the history server isn't running or there are issues connecting to the
history server, the kill command produces many retry messages. For example:
{noformat}
$ mapred job -kill job_1340642510012_0003
2012-06-25 16:42:26,626 INFO mapred.ClientServiceDelegate
(ClientServiceDelegate.java:getProxy(254)) - Application state is completed.
FinalApplicationStatus=FAILED. Redirecting to job history server
2012-06-25 16:42:27,629 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 0 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:28,630 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 1 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:29,631 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 2 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:30,632 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 3 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:31,633 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 4 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:32,633 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 5 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:33,634 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 6 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:34,635 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 7 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:35,636 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 8 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:36,637 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 9 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:36,642 INFO mapred.ClientServiceDelegate
(ClientServiceDelegate.java:getProxy(254)) - Application state is completed.
FinalApplicationStatus=FAILED. Redirecting to job history server
2012-06-25 16:42:37,643 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 0 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:38,644 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 1 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:39,644 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 2 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:40,645 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 3 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:41,646 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 4 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:42,647 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 5 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:43,648 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 6 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:44,649 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 7 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:45,649 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 8 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:46,650 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 9 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:46,660 INFO mapred.ResourceMgrDelegate
(ResourceMgrDelegate.java:killApplication(329)) - Killing application
application_1340642510012_0003
Killed job job_1340642510012_0003
{noformat}
It can eventually fail with a max retries exception in the case where the job
is no longer running. For example, here's what happens when trying to kill the
same job again:
{noformat}
$ mapred job -kill job_1340642510012_0003
2012-06-25 16:42:57,268 INFO mapred.ClientServiceDelegate
(ClientServiceDelegate.java:getProxy(254)) - Application state is completed.
FinalApplicationStatus=FAILED. Redirecting to job history server
2012-06-25 16:42:58,273 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 0 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:42:59,274 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 1 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:00,274 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 2 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:01,275 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 3 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:02,276 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 4 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:03,277 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 5 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:04,278 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 6 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:05,279 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 7 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:06,279 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 8 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:07,280 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 9 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:07,287 INFO mapred.ClientServiceDelegate
(ClientServiceDelegate.java:getProxy(254)) - Application state is completed.
FinalApplicationStatus=FAILED. Redirecting to job history server
2012-06-25 16:43:08,288 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 0 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:09,289 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 1 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:10,289 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 2 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:11,290 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 3 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:12,291 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 4 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:13,292 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 5 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:14,293 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 6 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:15,294 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 7 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:16,294 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 8 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:17,295 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 9 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:17,300 INFO mapred.ClientServiceDelegate
(ClientServiceDelegate.java:getProxy(254)) - Application state is completed.
FinalApplicationStatus=FAILED. Redirecting to job history server
2012-06-25 16:43:18,301 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 0 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:19,302 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 1 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:20,302 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 2 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:21,303 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 3 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:22,304 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 4 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:23,305 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 5 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:24,306 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 6 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:25,307 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 7 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:26,307 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 8 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2012-06-25 16:43:27,308 INFO ipc.Client
(Client.java:handleConnectionFailure(714)) - Retrying connect to server:
xx:10020. Already tried 9 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
Exception in thread "main" java.io.IOException
at
org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:315)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:383)
at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:481)
at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:184)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:280)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1244)
{noformat}
> mapred job -kill tries to connect to history server
> ---------------------------------------------------
>
> Key: MAPREDUCE-4367
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4367
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client, mrv2
> Affects Versions: 0.23.3
> Reporter: Jason Lowe
> Priority: Minor
>
> The {{mapred job -kill}} command attempts to connect to the history server,
> even though it is unrelated to the process of killing a job.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
