FYI
MRUnit is the way to go and it is compatible (as far as I am aware of)
with the 0.20.2
Cheers
Olivier
Varene Olivier a écrit :
Thanks a lot Aaron,
I was looking into it, I just hope, it will be compliant with 0.20.2 ...
Otherwise, I am stuck :(
Cheers
Olivier
Aaron Kimball a écrit :
Varene,
You might want to check out MRUnit. It's a unit test harness that
contains
mock objects for the context & other associated classes, and works with
JUnit.
It's included in the (unreleased) Hadoop 0.21, as well as Cloudera's
Distribution for Hadoop. See
http://archive.cloudera.com/docs/mrunit/index.html for MRUnit
documentation.
Cheers,
- Aaron
On Thu, May 27, 2010 at 9:13 AM, Varene Olivier <[email protected]> wrote:
FYI
I am on Hadoop 0.20.2, JUnit 4
Varene Olivier a écrit :
Hello to all,
first of all many thanks for this great piece of software you are all
contributing to :)
I am actually creating a program in Hadoop MapReduce, but I intend to
massively use JUnit Tests to validate my code.
To test a Mapper, I mock a Mapper.Context object and run the Mapper.map
function with it, but inside my Mapper.map function, I am using
Counters
to maintain internal statistics. The update of those counters make
the test
fails with a null pointer (JavaNullPointerException).
There is an issue "Counters" of variable visibility from my test ...
Do you have any idea how to mock/validate/overcome this issue ?
Thanks a lots
Best Regards
Olivier Varene
: Code :
--------
private static enum Counters {NBLINES};
private static IntWritable one = new IntWritable(1);
private static LongWritable oneL = new LongWritable(1);
// Mapper
/**
* outputs 1 for each line encountered
*/
public static class LCMapper extends
Mapper<LongWritable, Text, IntWritable, LongWritable>
{
private final static IntWritable one = new IntWritable(1);
private final static LongWritable oneL = new LongWritable(1);
/**
* for each line encountered outputs 1 for the key 1
*/
public void map(LongWritable p_key, Text p_inputs, Context p_context)
throws IOException, InterruptedException
{
p_context.getCounter(Counters.NBLINES).increment(1);
p_context.write(one,oneL); } // end map
} // end Mapper
: Test :
--------
@Test
public void testLCMapper() throws IOException, InterruptedException
{ LCMapper mapper = new LCMapper();
Text value = new Text("ligneA\nlignB\n");
// mock the map job execution context
Context context = mock(Context.class);
try {
mapper.map(null, value, context);
} catch (NullPointerException e)
{
// exception raised by context.getCounter(...) null pointer
// HOW TO MOCK ???
// Counter is a static enum from the englobing class ...
// issue with variable visibility (from map function) in this test
}
// verify that the Mapper issued (one,oneL) pair
verify(context).write(one,oneL);
} // end testLCMapper
