@ Harsh -
Yeah, mapred.task.timeout is the valid option. but for some reasons, its
not happening the way it should be.. I am not sure what could be the
cause.Thing is my jobs are running fine, its just that they are slow at
shuffling phase, sometimes.. not everytime.. so I was thinking as an
Dear Praveenesh
I think there are only two ways to kill a job:
1- kill command, (not perfect way cause you should know the job id)
2- mapred.task.timeout (in bin/hadoop jar command using
{-Dmapred.task.timeout=} set your desired value in msec)
sometimes for me its happened too, not in all
Is there anyway through which we can kill hadoop jobs that are taking
enough time to execute ?
What I want to achieve is - If some job is running more than
_some_predefined_timeout_limit, it should be killed automatically.
Is it possible to achieve this, through shell scripts or any other way ?
You might want to take a look at the kill command : hadoop job -kill
jobid.
Prashant
On Sun, Jan 29, 2012 at 11:06 PM, praveenesh kumar praveen...@gmail.comwrote:
Is there anyway through which we can kill hadoop jobs that are taking
enough time to execute ?
What I want to achieve is - If
Yeah, I am aware of that, but it needs you to explicity monitor the job and
look for jobid and then hadoop job -kill command.
What I want to know - Is there anyway to do all this automatically by
providing some timer or something -- that if my job is taking more than
some predefined time, it would
In the current stables, this is available at the task level with a
default fo 10m of non-responsiveness per task. Controlled per-job via
mapred.task.timeout.
There is no built-in feature that lets you monitor and set a timeout
on the job execution itself, however (but should be easy to do) -- How
