This question is pertaining to hadoop with core version 1.2.1 and hbase 1.2.3.
I wrote a simple map/reduce job that looks like this: - The input to the mapper is whole HDFS files at a time, via a custom InputFormat - The output of the mapper is <LongWriteable, Text> The job is configured like this: Configuration conf = getConf(); JobConf cfg = new JobConf(conf, FindMissionTimeJob.class); /* Set up mapper */ Path inputPath = new Path(args[0]); WholeFileInputFormat.setInputPaths(cfg, inputPath); cfg.setNumMapTasks(1); cfg.setMapperClass(TimeMapper.class); cfg.setInputFormat(WholeFileInputFormat.class); cfg.setMapOutputKeyClass(LongWritable.class); cfg.setMapOutputValueClass(Text.class); cfg.setNumReduceTasks(1); TableMapReduceUtil.initTableReduceJob(tableName, TimeReducer.class, cfg); When I run it, I get an NPE here: 16/10/07 15:33:55 INFO mapreduce.Job: Task Id : attempt_1475608557171_0021_r_000000_2, Status : FAILED Error: java.lang.NullPointerException at org.apache.hadoop.mapred.Task.getFsStatistics(Task.java:373) at org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter.<init>(ReduceTask.java:478) at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:414) Tracing the source of that, it really comes down to this bit of code in ReduceTask.java that is failing: if (job.getOutputFormat() instanceof FileOutputFormat) { matchedStats = getFsStatistics(FileOutputFormat.getOutputPath(job), job); } getFsStatistics() throws an NPE becaue there is no output path. The problem is, we shoud not even get there, because initTableReduceJob() already set the output format to this:: job.setOutputFormat(TableOutputFormat.class); Looking at the source for that guy: public class TableOutputFormat extends FileOutputFormat<ImmutableBytesWritable, Put> Ahh hah. There's the problem. That extends FileOutputFormat, which causes everything to blow up because it's a table reducer. It's not really a File output, but getFsStatistics is invoked and subsequently blows up. It seems likely that getFsStatistics should check for a NULL here, and perhaps that TableOutputFormat probably should not extend FileOutputFormat - I'm not sure. The way I got around this was to create my own OutputFormat called MyTableOutputFormat that was essentially a copy of the real TableOutputFormat but instead of extending FIleOutputFormat I did this: public class MyTableOutputFormat implements OutputFormat<ImmutableBytesWritable, Put> { ... } I don't know if that's a correct or incorrect solution... just that it runs an empty hbase-outputting reducer without blowing up on an NPE. I would appreciate any comments/feedback on the problem as well as my workaround, and whether or not anybody else has encountered this. --Chris