I managed to fix both the problems. 1) The one in Eclipse was happening because of the df task which wasn't possible in Windows. Once I added cygwin/bin to the Path, this started working.
2) This one occurred because the output value class of my Combiner and Reducer was different. On Wed, Apr 30, 2008 at 6:49 PM, Sridhar Raman <[EMAIL PROTECTED]> wrote: > I am trying to run my MR task in a local machine. I am doing this because > this is my first foray into MapReduce, and want to debug completely before I > move it to a multi-node cluser. The initial hitch that I faced was in > running it from Eclipse. Thanks to this > response<http://www.mail-archive.com/[EMAIL PROTECTED]/msg01368.html>, > I managed to get that working. > > But now I am facing a strange problem. > > My Mapper takes <IntWritable, Vector> as input, and returns <IntWritable, > Vector>. > I have a Combiner that takes the <IntWritable, Iterator<Vector>> and > computes the intermediate output of <IntWritable, InterVector>. This > IntVector basically stores the sum of the Vectors present in the iterator, > along with other information. > > Now to the problem. If I run this in Eclipse, after the Mapper is done, I > straightaway get this exception: > Iteration 0 > java.io.IOException: Job failed! > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:831) > at > com.company.analytics.clustering.mr.core.KMeansDriver.runIteration(KMeansDriver.java:144) > at > com.company.analytics.clustering.mr.core.KMeansDriver.runJob(KMeansDriver.java:91) > at > com.company.analytics.clustering.mr.core.KMeansDriver.main(KMeansDriver.java:37) > > Whereas if I run this from the cygwin command prompt, the execution > manages to go to the Combiner, but after that it gives this exception: > java.io.IOException: wrong value class: > com.company.analytics.clustering.feature. > InterVector is not class com.company.analytics.clustering.feature.Vector > at > org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:938) > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer$1.collect(MapTask.java:414) > at > com.company.analytics.clustering.mr.core.KMeansCombiner.reduce(KMeansCombiner.java:57) > at > com.company.analytics.clustering.mr.core.KMeansCombiner.reduce(KMeansCombiner.java:1) > > Any idea what could be the problem. Both the cygwin and Eclipse take the > same input folder. > I would be glad if I get any assistance. > > Thanks, > Sridhar >
