Support for per-phase speculative execution
-------------------------------------------
Key: MAPREDUCE-718
URL: https://issues.apache.org/jira/browse/MAPREDUCE-718
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobtracker
Affects Versions: 0.21.0
Reporter: Devaraj Das
Fix For: 0.21.0
It would be good to have support for per-phase speculative execution where the
algorithm looks at the current phase of a task, and compares with the other
tasks in the same phase before deciding to launch a speculative task. That
would have the following benefits:
1) Support for jobs where map tasks progresses jumps from 0% to 100%. This is
true for some jobs like randomwriter. Today, we would launch speculative tasks
for such jobs (assuming that the tasks are not making progress). But most of
them would be unnecessary.
2) In reality, for reduces, the three phases are quite different from each
other, and they take different times too. We should see better results when we
look at per-phase speculation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
