How to debug a MapReduce application

[Pedro Vivancos]

Dear friends,

I am new at Hadoop and at MapReduce techniques. I've developed my first
map-reduce application using hadoop but I can't manage to make it work. I
get the following error at the very beginning of the execution:

java.lang.NullPointerException
    at
org.apache.hadoop.io.serializer.SerializationFactory.getSerializer(SerializationFactory.java:73)
    at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:504)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:295)
    at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:138)
16-ene-2009 18:29:30 es.vocali.intro.tools.memo.MemoAnnotationMerging main
GRAVE: Se ha producido un error
java.io.IOException: Job failed!
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1217)
    at
es.vocali.intro.tools.memo.MemoAnnotationMerging.main(MemoAnnotationMerging.java:160)
java.io.IOException: Job failed!
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1217)
    at
es.vocali.intro.tools.memo.MemoAnnotationMerging.main(MemoAnnotationMerging.java:160)

Sorry if I don't give you more information but I don't know where to start
to find the error. My app is quite simple. It just gets some rows from a
postgresql database and try to see which ones can be deleted.

Here you have the configuration I am using:

MemoAnnotationMerging memo = new MemoAnnotationMerging();

        Map<String, String> parametros = memo.checkParams(args);

        memo.initDataStore(parametros.get(DATASTORE_URL));

        JobConf conf = new JobConf(MemoAnnotationMerging.class);
        conf.setJobName("memo - annotation merging");

        conf.setMapperClass(MemoAnnotationMapper.class);
        conf.setCombinerClass(MemoAnnotationReducer.class);
        conf.setReducerClass(MemoAnnotationReducer.class);

        DBConfiguration.configureDB(conf, DRIVER_CLASS,
parametros.get(DATASTORE_URL));

        // ???
        //conf.setInputFormat(DBInputFormat.class);
        //conf.setOutputFormat(TextOutputFormat.class);

        conf.setMapOutputKeyClass(LongWritable.class);
        conf.setMapOutputValueClass(Annotation.class);


        //conf.setOutputKeyClass(Annotation.class);
        //conf.setOutputValueClass(BooleanWritable.class);

        DBInputFormat.setInput(conf, MemoAnnotationDBWritable.class,
GET_ANNOTATIONS_QUERY, COUNT_ANNOTATIONS_QUERY);

        FileOutputFormat.setOutputPath(conf, new Path("eliminar.txt"));

        // ejecutamos el algoritmo map-reduce para mezclar anotaciones
        try {
            JobClient.runJob(conf);

        } catch (IOException e) {
            e.printStackTrace();
            System.exit(-1);
        }

Thanks in advance.

 Pedro Vivancos Vicente
Vócali Sistemas Inteligentes S.L. <http://www.vocali.net>
Edificio CEEIM, Campus de Espinardo
30100, Espinardo, Murcia, Spain
Tel. +34 902 929 644  <http://www.vocali.net>