[
https://issues.apache.org/jira/browse/HADOOP-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12531939
]
Doug Cutting commented on HADOOP-1965:
--------------------------------------
> We need to benchmark anything we implement for this issue.
+1
> improving latency will help regardless, no?
Improving task latency alone will not improve job latency much in most cases.
If we run more tasks per node than there are CPU cores, and there are
significantly more input splits than task slots (as there normally should be)
then job latency might not be improved much.
I'm not arguing that this won't help at all, rather that it might not help
much. But then again, it might. It's certainly worth a try.
> Handle map output buffers better
> --------------------------------
>
> Key: HADOOP-1965
> URL: https://issues.apache.org/jira/browse/HADOOP-1965
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Devaraj Das
> Assignee: Amar Kamat
>
> Today, the map task stops calling the map method while sort/spill is using
> the (single instance of) map output buffer. One improvement that can be done
> to improve performance of the map task is to have another buffer for writing
> the map outputs to, while sort/spill is using the first buffer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
