Hi,
may I know where should I find the sources related to speculative
scheduling happens ? and also how do we discard the output coming from
already completed mapper ?
I am actually trying to do something similar, like spawn map tasks
redundantly but not for speculative reasons but for each mappe
Thanks Mahesh & Harsh.
On 07-Dec-2012, at 7:42 AM, Mahesh Balija wrote:
To simply, if you turn-off the speculative execution then the system will never
bother about slow running jobs unless they won't report beyond specified time
(10 minutes).
If you have set speculative execution to true the
To simply, if you turn-off the speculative execution then the system will
never bother about slow running jobs unless they won't report beyond
specified time (10 minutes).
If you have set speculative execution to true then the system may spawn
another instance of mapper and consider the output of t
Given that Speculative Execution *is* the answer to such scenarios,
I'd say the answer to your question without it, is *nothing*.
If a task does not report status for over 10 minutes (default), it is
killed and retried. If it does report status changes (such as
counters, task status, etc.) but is
Hi,
What is the behavior of jobTracker if speculative execution is off and a task
on data node is running extremely slow?
Will the jobTracker simply wait till the slow running task finishes or it will
try to heal the situation? Assuming that heartbeat from the node running slow
task are regular
