[
https://issues.apache.org/jira/browse/MAPREDUCE-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tom White updated MAPREDUCE-1411:
---------------------------------
Attachment: MAPREDUCE-1411.patch
Here's a patch for discussion which introspects the Reducer for its output key
and value types. It's only a demonstration of how we might do this - it needs
more testing and cleaning up, but it show the general approach.
It works using generics-introspection code from
http://www.artima.com/weblogs/viewpost.jsp?thread=208860, although it might be
worth using the (Apache 2.0 licensed) http://code.google.com/p/gentyref/
library which has the same functionality (and handles extra cases).
With this change the following lines are removed from word count:
{code}
- job.setOutputKeyClass(Text.class);
- job.setOutputValueClass(IntWritable.class);
{code}
> Infer MapReduce types where possible
> ------------------------------------
>
> Key: MAPREDUCE-1411
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1411
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: client
> Reporter: Tom White
> Assignee: Tom White
> Attachments: MAPREDUCE-1411.patch
>
>
> Currently users must specify the output types of the map and reduce if they
> are different to the default (LongWritable keys, Text values). In many cases
> this information is available to the system from the user-supplied mapper and
> reducer, so it would be nice if the user didn't have to specify them in two
> places.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
