Hi there,
I think you got the run(String[] args) method right but in the main method you
are not calling your run method but ToolRunner.run. You need to invoke your
method in order to point to localhost:54310 otherwise it will read those
properties from the default hadoop conf.
Praveen
________________________________
From: ext Felix.徐 [ygnhz...@gmail.com]
Sent: Tuesday, May 17, 2011 6:47 AM
To: mapreduce-user@hadoop.apache.org
Subject: How to run or submit MapReduce Job to Hadoop in my own program?
Hi,all..How can I run a MR job though my own program instead of using console
to submit a job to a real Hadoop env?
I write code like this, this program works fine but i don't think it ran in my
Hadoop env,since nothing was produced in hadoop logs folder.
public int run(String[] args) throws Exception {
Configuration config = new Configuration();
config.set("fs.default.name<http://fs.default.name>",
"hdfs://localhost:54310");
config.set("mapred.job.tracker","localhost:54311");
JobConf conf = new JobConf(config,CustomMR.class);
conf.setJobName("custom aggregation");
conf.setMapOutputKeyClass(Text.class);
conf.setMapOutputValueClass(IntWritable.class);
conf.setOutputFormat(HiveIgnoreKeyTextOutputFormat.class);
conf.setMapperClass(MapClass.class);
conf.setReducerClass(Reduce.class);
TextInputFormat.setInputPaths(conf,args[0]);
HiveIgnoreKeyTextOutputFormat.setOutputPath(conf, new
Path(args[1]));
JobClient.runJob(conf);
//conf
return 0;
}
public static void main(String[] args) throws Exception {
args = new String[]{"/user/hive/warehouse/ptest","/testout"};
int res = ToolRunner.run(new Configuration(), new CustomMR(), args);
System.exit(res);
}
Could some one give me an example?Thanks!
