[ https://issues.apache.org/jira/browse/MAPREDUCE-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-5221: ----------------------------------------------- Status: Open (was: Patch Available) Thanks for taking this up, Tsuyoshi! Comments on your patch: - In ReduceTask.getCombinerClass(), there is a lot of implicit type-casting. I am not sure if this will even work. getCombinerClass(), the variable clazz pointing to shuffle-plugin and ShuffleConsumerPlugin.Context all take in a Class<? extends Reducer> which won't be true for the new combiner. Did you test this at all? Can you please run a job with a new combiner with the patch? I doubt that it works. - Please fix the typos "Cominer" in the tests. - May be also add another simple test to verify if both new and old combiners are set. - Move this test code to the existing mapred.TestReduceTask. > Reduce side Combiner is not used when using the new API > ------------------------------------------------------- > > Key: MAPREDUCE-5221 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5221 > Project: Hadoop Map/Reduce > Issue Type: Bug > Affects Versions: 2.0.4-alpha > Reporter: Siddharth Seth > Assignee: Tsuyoshi OZAWA > Attachments: MAPREDUCE-5221.1.patch, MAPREDUCE-5221.2.patch, > MAPREDUCE-5221.3.patch > > > If a combiner is specified using o.a.h.mapreduce.Job.setCombinerClass - this > will silently ignored on the reduce side since the reduce side usage is only > aware of the old api combiner. > This doesn't fail the job - since the new combiner key does not deprecate the > old key. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira