[
https://issues.apache.org/jira/browse/MAPREDUCE-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13701043#comment-13701043
]
Karthik Kambatla commented on MAPREDUCE-5221:
---------------------------------------------
The patch is easier to understand now. The patch can be summarized as follows
(correct me if I am wrong): Use ReduceTask.CombinerRunner to run the combiner.
That takes care of retrieving the combiner-class for the API being used and run
the combiner accordingly. So, the current combine code (combineAndSpill) can be
done away with.
Comment:
# From the looks of it, the {{Reporter}} passed to {{MergeManagerImpl}}
constructor can only be a TaskReporter. For this patch, it should be okay to
typecast it unconditionally. With the current if condition, it is easy to
wonder how the combine works for Reporters which are not TaskReporter.
# Consequently, may be in a different JIRA, we should use Reporter/TaskReporter
uniform everywhere - ShufflePlugin, MergeManagerImpl, and CombinerRunner.
Other than that, the patch looks good to me. Should we close this other JIRAs
as duplicates?
> Reduce side Combiner is not used when using the new API
> -------------------------------------------------------
>
> Key: MAPREDUCE-5221
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5221
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 2.0.4-alpha
> Reporter: Siddharth Seth
> Assignee: Tsuyoshi OZAWA
> Attachments: MAPREDUCE-5221.1.patch, MAPREDUCE-5221.2.patch,
> MAPREDUCE-5221.3.patch, MAPREDUCE-5221.4.patch, MAPREDUCE-5221.5.patch
>
>
> If a combiner is specified using o.a.h.mapreduce.Job.setCombinerClass - this
> will silently ignored on the reduce side since the reduce side usage is only
> aware of the old api combiner.
> This doesn't fail the job - since the new combiner key does not deprecate the
> old key.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
