Hi,all..How can I run a MR job though my own program instead of using console to submit a job to a real Hadoop env? I write code like this, this program works fine but i don't think it ran in my Hadoop env,since nothing was produced in hadoop logs folder. public int run(String[] args) throws Exception {
Configuration config = new Configuration(); config.set("fs.default.name", "hdfs://localhost:54310"); config.set("mapred.job.tracker","localhost:54311"); JobConf conf = new JobConf(config,CustomMR.class); conf.setJobName("custom aggregation"); conf.setMapOutputKeyClass(Text.class); conf.setMapOutputValueClass(IntWritable.class); conf.setOutputFormat(HiveIgnoreKeyTextOutputFormat.class); conf.setMapperClass(MapClass.class); conf.setReducerClass(Reduce.class); TextInputFormat.setInputPaths(conf,args[0]); HiveIgnoreKeyTextOutputFormat.setOutputPath(conf, new Path(args[1])); JobClient.runJob(conf); //conf return 0; } public static void main(String[] args) throws Exception { args = new String[]{"/user/hive/warehouse/ptest","/testout"}; int res = ToolRunner.run(new Configuration(), new CustomMR(), args); System.exit(res); } Could some one give me an example?Thanks!