[
https://issues.apache.org/jira/browse/MAPREDUCE-718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12730049#action_12730049
]
Zhang Zhaoning commented on MAPREDUCE-718:
------------------------------------------
What about the detailed mechanism about this?
Is that when each phase hit its threshold then migrate the entire task?
If some reduce tasks will process the shuffling fast but sorting slow for the
network performance, when the sorting phase hit the sorting threshold and then
the task is migrated and re-run it on a poor network performance node.
> Support for per-phase speculative execution
> -------------------------------------------
>
> Key: MAPREDUCE-718
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-718
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.21.0
> Reporter: Devaraj Das
> Fix For: 0.21.0
>
>
> It would be good to have support for per-phase speculative execution where
> the algorithm looks at the current phase of a task, and compares with the
> other tasks in the same phase before deciding to launch a speculative task.
> That would have the following benefits:
> 1) Support for jobs where map tasks progresses jumps from 0% to 100%. This is
> true for some jobs like randomwriter. Today, we would launch speculative
> tasks for such jobs (assuming that the tasks are not making progress). But
> most of them would be unnecessary.
> 2) In reality, for reduces, the three phases are quite different from each
> other, and they take different times too. We should see better results when
> we look at per-phase speculation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
