What is the reason of using the queue? "job.getConfiguration().set("mapred.job.queue.name", "exp_dsa");"
Is your mapper or reducer even been called? Try adding the override annotation to the map/reduce methods as below: @Override public void map(Object key, Text value, Context context) throws IOException, InterruptedException { Regards, Shahab On Tue, Mar 31, 2015 at 3:26 AM, bradford li <bradfor...@gmail.com> wrote: > I'm not sure why my Mapper and Reducer have no output. The logic behind my > code is, given a file of UUIDs (new line separated), I want to use > `globStatus` to display all the paths to all potential files that the UUID > might be in. Open and read the file. Each file contains 1-n lines of JSON. > The UUID is in `event_header.event_id` in the JSON. > > Right now the MapReduce job runs without errors. However, something is > wrong because I dont have any output. I'm not sure how to debug MapReduce > jobs as well. If someone could provide me a source that would be awesome! > The expected output from this program should be > > UUID_1 1 > UUID_2 1 > UUID_3 1 > UUID_4 1 > ... > ... > UUID_n 1 > > In my logic, the output file should be the UUIDs with a 1 next to them > because upon found, 1 is written, if not found 0 is written. They should be > all 1's because I pulled the UUIDs from the source. > > My Reducer currently does not do anything except I just wanted to see if I > could get some simple logic working. There are most likely bugs in my code > as I dont know have a easy way to debug MapReduce jobs > > Driver: > > public class SearchUUID { > > public static void main(String[] args) throws Exception { > Configuration conf = new Configuration(); > Job job = Job.getInstance(conf, "UUID Search"); > job.getConfiguration().set("mapred.job.queue.name", > "exp_dsa"); > job.setJarByClass(SearchUUID.class); > job.setMapperClass(UUIDMapper.class); > job.setReducerClass(UUIDReducer.class); > job.setOutputKeyClass(Text.class); > job.setOutputValueClass(Text.class); > FileInputFormat.addInputPath(job, new Path(args[0])); > FileOutputFormat.setOutputPath(job, new Path(args[1])); > System.exit(job.waitForCompletion(true) ? 0 : 1); > } > } > > > UUIDMapper: > > public class UUIDMapper extends Mapper<Object, Text, Text, Text> { > public void map(Object key, Text value, Context context) throws > IOException, InterruptedException { > > try { > Text one = new Text("1"); > Text zero = new Text("0"); > > FileSystem fs = FileSystem.get(new Configuration()); > FileStatus[] paths = fs.globStatus(new > Path("/data/path/to/file/d_20150330-1650")); > for (FileStatus path : paths) { > BufferedReader br = new BufferedReader(new > InputStreamReader(fs.open(path.getPath()))); > String json_string = br.readLine(); > while (json_string != null) { > JsonElement jelement = new > JsonParser().parse(json_string); > JsonObject jsonObject = jelement.getAsJsonObject(); > jsonObject = > jsonObject.getAsJsonObject("event_header"); > jsonObject = > jsonObject.getAsJsonObject("event_id"); > > if > (value.toString().equals(jsonObject.getAsString())) { > System.out.println(value.toString() + > "slkdjfksajflkjsfdkljsadfk;ljasklfjklasjfklsadl;sjdf"); > context.write(value, one); > } else { > context.write(value, zero); > } > > json_string = br.readLine(); > } > } > } catch (IOException failed) { > } > } > } > > > Reducer: > > public class UUIDReducer extends Reducer<Text, Text, Text, Text>{ > > public void reduce(Text key, Text value, Context context) throws > IOException, InterruptedException{ > context.write(key, value); > } > } > >