Hello, I am working on a Hadoop application that produces different
(key,value) types after the map and reduce phases so I'm aware that I need
to use "JobConf.setMapOutputKeyClass" and "JobConf.setMapOutputValueClass".
However, I still keep getting the following runtime error when I run my
application:
java.io.IOException: wrong value class: org.apache.hadoop.io.FloatWritable
is not class org.apache.hadoop.io.IntWritable
at
org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:938)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer$1.collect(MapTask.java:414)
at
test.DistributionCreator$Reduce.reduce(DistributionCreator.java:104)
at
test.DistributionCreator$Reduce.reduce(DistributionCreator.java:85)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.combineAndSpill(MapTask.java:439)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpillToDisk(MapTask.java:418)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:604)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:193)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1804)
My mapper class goes like:
public static class MapClass extends MapReduceBase
implements Mapper<LongWritable, Text, IntWritable, IntWritable> {
(...)
public void map(LongWritable key, Text value,
OutputCollector<IntWritable, IntWritable> output,
Reporter reporter) throws IOException {
(...)
}
}
and my Reducer goes like:
public static class Reduce extends MapReduceBase
implements Reducer<IntWritable, IntWritable, IntWritable, FloatWritable>
{
(...)
public void reduce(IntWritable key, Iterator<IntWritable> values,
OutputCollector<IntWritable, FloatWritable> output,
Reporter reporter) throws IOException {
float sum = 0;
(...)
output.collect(key, new FloatWritable(sum));
}
}
and the corresponding part of my configuration goes as follows:
conf.setMapOutputValueClass(IntWritable.class);
conf.setMapOutputKeyClass(IntWritable.class);
conf.setOutputKeyClass(IntWritable.class);
conf.setOutputValueClass(FloatWritable.class);
which I believe is consistent with the mapper and the reducer classes.
Can you please let me know what I'm missing here?
Thanks in advance,
Jim