I am trying to run my MR task in a local machine. I am doing this because
this is my first foray into MapReduce, and want to debug completely before I
move it to a multi-node cluser. The initial hitch that I faced was in
running it from Eclipse. Thanks to this
response<http://www.mail-archive.com/[EMAIL PROTECTED]/msg01368.html>,
I managed to get that working.
But now I am facing a strange problem.
My Mapper takes <IntWritable, Vector> as input, and returns <IntWritable,
Vector>.
I have a Combiner that takes the <IntWritable, Iterator<Vector>> and
computes the intermediate output of <IntWritable, InterVector>. This
IntVector basically stores the sum of the Vectors present in the iterator,
along with other information.
Now to the problem. If I run this in Eclipse, after the Mapper is done, I
straightaway get this exception:
Iteration 0
java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:831)
at
com.company.analytics.clustering.mr.core.KMeansDriver.runIteration(KMeansDriver.java:144)
at
com.company.analytics.clustering.mr.core.KMeansDriver.runJob(KMeansDriver.java:91)
at
com.company.analytics.clustering.mr.core.KMeansDriver.main(KMeansDriver.java:37)
Whereas if I run this from the cygwin command prompt, the execution manages
to go to the Combiner, but after that it gives this exception:
java.io.IOException: wrong value class:
com.company.analytics.clustering.feature.
InterVector is not class com.company.analytics.clustering.feature.Vector
at
org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:938)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer$1.collect(MapTask.java:414)
at
com.company.analytics.clustering.mr.core.KMeansCombiner.reduce(KMeansCombiner.java:57)
at
com.company.analytics.clustering.mr.core.KMeansCombiner.reduce(KMeansCombiner.java:1)
Any idea what could be the problem. Both the cygwin and Eclipse take the
same input folder.
I would be glad if I get any assistance.
Thanks,
Sridhar