[
https://issues.apache.org/jira/browse/MAPREDUCE-1574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Leshem updated MAPREDUCE-1574:
------------------------------------
Affects Version/s: 0.20.3
0.20.2
Fix Version/s: 0.20.1
0.20.2
> Combiners should implement a specialized "Combiner" interface, not the
> generic "Reducer" interface
> --------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1574
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1574
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 0.20.1, 0.20.2, 0.20.3
> Reporter: Danny Leshem
> Priority: Minor
> Fix For: 0.20.1, 0.20.2
>
>
> I just spent 30 minutes trying to figure out why my job throws
> "java.io.IOException: wrong key class" when I pass my Reducer class to
> Job.setCombinerClass. Finally, I understood that a Reducer can act as
> Combiner only if its output key/value are the same as its input key/value.
> So yes, this is documented. But you can make life easier for users by
> defining a Combiner interface (that Job.setCombinerClass will accept) to
> force this at compile time. The new interface should implement the Reducer
> interface and specialize it (is it even possible with generics?).
> Alternatively, you can call this interface "SimpleReducer".
> If the generics-trick suggested above is impossible to implement, for the
> (common?) case of having the same class acting as Combiner and Reducer you
> can do one of either:
> 1) Thin Combiner implementation that wraps a given Reducer.
> 2) Add a new method, say Job.setCombinerClassToReducer (that accepts a
> Reducer), acting similarly to the new Job.setCombinerClass - but here the
> name should alert the user she's doing something special.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
