I'm not sure why my Mapper and Reducer have no output. The logic behind my code is, given a file of UUIDs (new line separated), I want to use `globStatus` to display all the paths to all potential files that the UUID might be in. Open and read the file. Each file contains 1-n lines of JSON. The UUID is in `event_header.event_id` in the JSON.
Right now the MapReduce job runs without errors. However, something is wrong because I dont have any output. I'm not sure how to debug MapReduce jobs as well. If someone could provide me a source that would be awesome! The expected output from this program should be UUID_1 1 UUID_2 1 UUID_3 1 UUID_4 1 ... ... UUID_n 1 In my logic, the output file should be the UUIDs with a 1 next to them because upon found, 1 is written, if not found 0 is written. They should be all 1's because I pulled the UUIDs from the source. My Reducer currently does not do anything except I just wanted to see if I could get some simple logic working. There are most likely bugs in my code as I dont know have a easy way to debug MapReduce jobs Driver: public class SearchUUID { public static void main(String[] args) throws Exception { Configuration conf = new Configuration(); Job job = Job.getInstance(conf, "UUID Search"); job.getConfiguration().set("mapred.job.queue.name", "exp_dsa"); job.setJarByClass(SearchUUID.class); job.setMapperClass(UUIDMapper.class); job.setReducerClass(UUIDReducer.class); job.setOutputKeyClass(Text.class); job.setOutputValueClass(Text.class); FileInputFormat.addInputPath(job, new Path(args[0])); FileOutputFormat.setOutputPath(job, new Path(args[1])); System.exit(job.waitForCompletion(true) ? 0 : 1); } } UUIDMapper: public class UUIDMapper extends Mapper<Object, Text, Text, Text> { public void map(Object key, Text value, Context context) throws IOException, InterruptedException { try { Text one = new Text("1"); Text zero = new Text("0"); FileSystem fs = FileSystem.get(new Configuration()); FileStatus[] paths = fs.globStatus(new Path("/data/path/to/file/d_20150330-1650")); for (FileStatus path : paths) { BufferedReader br = new BufferedReader(new InputStreamReader(fs.open(path.getPath()))); String json_string = br.readLine(); while (json_string != null) { JsonElement jelement = new JsonParser().parse(json_string); JsonObject jsonObject = jelement.getAsJsonObject(); jsonObject = jsonObject.getAsJsonObject("event_header"); jsonObject = jsonObject.getAsJsonObject("event_id"); if (value.toString().equals(jsonObject.getAsString())) { System.out.println(value.toString() + "slkdjfksajflkjsfdkljsadfk;ljasklfjklasjfklsadl;sjdf"); context.write(value, one); } else { context.write(value, zero); } json_string = br.readLine(); } } } catch (IOException failed) { } } } Reducer: public class UUIDReducer extends Reducer<Text, Text, Text, Text>{ public void reduce(Text key, Text value, Context context) throws IOException, InterruptedException{ context.write(key, value); } }