Jason Lowe created MAPREDUCE-5211:
-------------------------------------
Summary: Reducer intermediate files can collide during merge
Key: MAPREDUCE-5211
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5211
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.7, 2.0.3-alpha
Reporter: Jason Lowe
Assignee: Jason Lowe
Priority: Critical
The OnDiskMerger.merge method constructs an output path that is not unique to a
reduce attempt, and as a result can result in a file collision with other
reducers from the same app that are running on the same node. In addition the
name of the output file is based on MapOutput.toString which may not be unique
in light of multi-pass merges on disk since the mapId will be null and the
basename ends up as "MapOutput(null, DISK)"
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
