[
https://issues.apache.org/jira/browse/MAPREDUCE-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13801439#comment-13801439
]
zhaoyunjiong commented on MAPREDUCE-5541:
-----------------------------------------
Since I only have few instance to set up a test cluster, I can only test
functionality.
I rewrite sleep job to let map/reduce sleep random time, and set
mapred.job.speculative.progress and mapred.job.speculative.factor with
different value to check whether it start speculative task as expectation.
I'll share more when we push this patch to our product cluster this week, I
believe it will be more better than before at most case.
> Improved algorithm for whether need speculative task
> ----------------------------------------------------
>
> Key: MAPREDUCE-5541
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5541
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv1
> Affects Versions: 1.2.1
> Reporter: zhaoyunjiong
> Assignee: zhaoyunjiong
> Fix For: 1.2.2
>
> Attachments: MAPREDUCE-5541-branch-1.2.patch,
> MAPREDUCE-5541-branch-1.2.patch
>
>
> Most of time, tasks won't start running at same time.
> In this case hasSpeculativeTask in TaskInProgress not working very well.
> Some times, some tasks just start running, and scheduler already decide it
> need speculative task to run.
> And this waste a lot of resource.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
