map outputs should not be renamed between partitions
----------------------------------------------------
Key: HADOOP-3443
URL: https://issues.apache.org/jira/browse/HADOOP-3443
Project: Hadoop Core
Issue Type: Bug
Components: mapred
Affects Versions: 0.17.0
Reporter: Owen O'Malley
Assignee: Owen O'Malley
Priority: Critical
If a map finishes with out having to spill its data buffer, the map outputs are
sorted and written to disk. However, no care is taken to make sure that the
same partition is used to write it out before it is renamed. On nodes with
multiple disks assigned to the task trackers, this will likely cause an
addition read/write cycle to disk that is very expensive.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
