Hi, I'm trying to refine my map reduce algorithm to run faster, but I ran into a little bit of trouble.
In my main, I have the following parameters set for my conf: conf.setOutputKeyClass(Text.class); conf.setOutputValueClass(LongWritable.class); originally, this was quite ok, since my input to my map phase was key = LongWritable value = Text, and the output of my map phase was key = Text, value = LongWritable. Similarily, the input to my reduce phase was key = Text, value = iterator<LongWritable> and the output was key = Text, value = LongWritable. I just wrote a combiner class to try and speed things up. However, now I want to do something like the following: ==map phase== input: key = LongWritable value = Text, output: key = Text, value = Longwritable ==combiner== input: key = Text, value = iterator<LongWritable> output: key = Text, value = Text ==reduce phase== input: key = Text, value = iterator<Text> output: key = Text, value = LongWritable According to the documentation I saw, I shouldn't have to change what I have set in main(), since the key type of the output of the job is Text, and the value type is still LongWritable. However, when I compile and try to run the code, I get the following: 08/09/23 18:35:45 INFO mapred.JobClient: Task Id : task_200809222115_0004_m_000000_0, Status : FAILED java.io.IOException: wrong value class: org.apache.hadoop.io.Text is not class org.apache.hadoop.io.LongWritable This tells me that Hadop doesn't like the fact that my output at the end of the combiner is of type Text, which also tells me that I mustn't have set the proper confs or something along those lines. However, I don't see anything in the JobConf class that would let me set the output of the combiner. I see a setMapOutputValueClass, but no setCombinerOutputValueClass function. Could someone point me in the right direction please? Thanks in advance, -SM
