[
https://issues.apache.org/jira/browse/HADOOP-2131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amareshwari Sri Ramadasu updated HADOOP-2131:
---------------------------------------------
Status: Patch Available (was: Open)
Submiting after deprecating mapred.speculative.execution
> Speculative execution should be allowed for reducers only
> ---------------------------------------------------------
>
> Key: HADOOP-2131
> URL: https://issues.apache.org/jira/browse/HADOOP-2131
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Environment: Hadoop job, map fetches data from external systems
> Reporter: Srikanth Kakani
> Assignee: Amareshwari Sri Ramadasu
> Priority: Critical
> Fix For: 0.16.0
>
> Attachments: patch-2131.txt, patch-2131.txt
>
>
> Consider hadoop jobs where maps fetch data from external systems, and emit
> the data. The reducers in this are identity reducers. The data processed by
> these jobs is huge. There could be slow nodes in this cluster and some of the
> reducers run twice as slow as their counterparts. This could result in a long
> tail. Speculative execution would help greatly in such cases. However given
> the current hadoop, we have to select speculative execution for both maps and
> reducers. In this case hurting the map performance as they are fetching data
> from external systems thereby overloading the external systems.
> Speculative execution only on reducers would be a great way to solve this
> problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
