[ http://issues.apache.org/jira/browse/HADOOP-591?page=all ]
Owen O'Malley resolved HADOOP-591.
----------------------------------
Resolution: Fixed
I believe this was already fixed as part of HADOOP-611.
> Reducer sort should even out the pass factors in merging different pass
> -----------------------------------------------------------------------
>
> Key: HADOOP-591
> URL: http://issues.apache.org/jira/browse/HADOOP-591
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Runping Qi
> Assigned To: Devaraj Das
>
> When multiple pass merging is needed during sort, the current sort
> implementation in SequenceFile class uses a simple "greedy" way to select
> pass factors, resulting uneven pass factor in different passes. For example,
> if the factor pass is 100 (the default), and there are 101 segments to be
> merged. The current implementation will first merge the first 100 segments
> into one and then merge the big output file with the last segment with pass
> factor 2. It will be better off to use pass factors 11 in the first pass and
> pass factor 10 in the second pass.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
