Varene, You might want to check out MRUnit. It's a unit test harness that contains mock objects for the context & other associated classes, and works with JUnit.
It's included in the (unreleased) Hadoop 0.21, as well as Cloudera's Distribution for Hadoop. See http://archive.cloudera.com/docs/mrunit/index.html for MRUnit documentation. Cheers, - Aaron On Thu, May 27, 2010 at 9:13 AM, Varene Olivier <[email protected]> wrote: > FYI > I am on Hadoop 0.20.2, JUnit 4 > > > Varene Olivier a écrit : > > Hello to all, >> >> first of all many thanks for this great piece of software you are all >> contributing to :) >> >> I am actually creating a program in Hadoop MapReduce, but I intend to >> massively use JUnit Tests to validate my code. >> >> To test a Mapper, I mock a Mapper.Context object and run the Mapper.map >> function with it, but inside my Mapper.map function, I am using Counters >> to maintain internal statistics. The update of those counters make the test >> fails with a null pointer (JavaNullPointerException). >> There is an issue "Counters" of variable visibility from my test ... >> >> Do you have any idea how to mock/validate/overcome this issue ? >> >> >> >> Thanks a lots >> Best Regards >> Olivier Varene >> >> >> : Code : >> -------- >> >> private static enum Counters {NBLINES}; >> private static IntWritable one = new IntWritable(1); >> private static LongWritable oneL = new LongWritable(1); >> >> >> // Mapper >> /** >> * outputs 1 for each line encountered >> */ >> public static class LCMapper extends >> Mapper<LongWritable, Text, IntWritable, LongWritable> >> { >> private final static IntWritable one = new IntWritable(1); >> private final static LongWritable oneL = new LongWritable(1); >> /** >> * for each line encountered outputs 1 for the key 1 >> */ >> public void map(LongWritable p_key, Text p_inputs, Context p_context) >> throws IOException, InterruptedException >> { >> p_context.getCounter(Counters.NBLINES).increment(1); >> p_context.write(one,oneL); } // end map >> } // end Mapper >> >> : Test : >> -------- >> >> @Test >> public void testLCMapper() throws IOException, InterruptedException >> { LCMapper mapper = new LCMapper(); >> Text value = new Text("ligneA\nlignB\n"); >> // mock the map job execution context >> Context context = mock(Context.class); >> try { >> mapper.map(null, value, context); >> } catch (NullPointerException e) >> { >> // exception raised by context.getCounter(...) null pointer >> // HOW TO MOCK ??? >> // Counter is a static enum from the englobing class ... >> // issue with variable visibility (from map function) in this test >> } >> // verify that the Mapper issued (one,oneL) pair >> verify(context).write(one,oneL); >> } // end testLCMapper >> >> >>
