[
https://issues.apache.org/jira/browse/MAPREDUCE-262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer resolved MAPREDUCE-262.
----------------------------------------
Resolution: Fixed
Spec exec got optimized somewhere along the way. Closing.
> Optimize finding of speculative tasks
> -------------------------------------
>
> Key: MAPREDUCE-262
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-262
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Amar Kamat
>
> Assuming HADOOP-2119 provides better data structures for handling running
> TIPs, finding new speculative tasks can be further optimized. Two of which
> could be
> 1) {{conf.getMapSpeculativeExecution()}} and
> {{conf.getReduceSpeculativeExecution()}} should be moved to
> {{JobInProgress}}. A simple check for this boolean can prove useful before
> checking for speculative tasks. This will be useful for jobs with large maps
> and reducers where scanning all the TIPs can be costly.
> 2) Since the progress of a TIP changes only when
> {{TaskInProgress.recomputeProgress()}} is invoked, it makes more sense to
> check for speculation in {{JobInProgress.updateTaskStatus()}} and move the
> TIPs that can be speculated to the front of the running queue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
