Adding the directory to my classpath worked! Thank you both for the prompt assistance.
On Fri, Jul 11, 2008 at 12:33 PM, Matt Kent <[EMAIL PROTECTED]> wrote: > The best way to configure all that stuff is in hadoop-site.xml, which > lives in the hadoop conf directory. Make sure that directory is on the > classpath of your application. > > On Fri, 2008-07-11 at 11:55 -0400, Larry Compton wrote: > > Thanks. Is this the correct syntax? > > > > conf.set("mapred.job.tracker", "localhost:54311"); > > > > It does appear to be communicating with the JobTracker now, but I get the > > following stack trace. Is there anything else that needs to be done to > > configure the job? > > > > Exception in thread "main" org.apache.hadoop.ipc.RemoteException: > > java.io.IOException: > > /home/larry/pkg/hadoop/hdfs/mapred/system/job_200807111114_0001/job.xml: > No > > such file or directory > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:215) > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:149) > > at > org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1155) > > at > org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1136) > > at > org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:175) > > at > org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:1755) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:585) > > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446) > > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896) > > > > at org.apache.hadoop.ipc.Client.call(Client.java:557) > > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212) > > at $Proxy0.submitJob(Unknown Source) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:585) > > at > > > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82) > > at > > > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59) > > at $Proxy0.submitJob(Unknown Source) > > at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:758) > > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:973) > > at jobclient.MaterializeMain.main(MaterializeMain.java:44) > > > > > > On Fri, Jul 11, 2008 at 11:41 AM, Shengkai Zhu <[EMAIL PROTECTED]> > wrote: > > > > > You should provide JobTracker address and port through configuration. > > > > > > > > > On 7/11/08, Larry Compton <[EMAIL PROTECTED]> wrote: > > > > > > > > I'm coming up to speed on the Hadoop APIs. I need to be able to > invoke a > > > > job > > > > from within a Java application (as opposed to running from the > > > command-line > > > > "hadoop" executable). The JobConf and JobClient appear to support > this > > > and > > > > I've written a test program to configure and run a job. However, the > job > > > > doesn't appear to be submitted to the JobTracker. Here's a code > excerpt > > > > from > > > > my client... > > > > > > > > String rdfInputPath = args[0]; > > > > String outputPath = args[1]; > > > > String uriInputPath = args[2]; > > > > String jarPath = args[3]; > > > > > > > > JobConf conf = new JobConf(MaterializeMap.class); > > > > conf.setJobName("materialize"); > > > > > > > > conf.setOutputKeyClass(Text.class); > > > > conf.setOutputValueClass(Text.class); > > > > > > > > conf.setMapperClass(MaterializeMapper.class); > > > > conf.setCombinerClass(MaterializeReducer.class); > > > > conf.setReducerClass(MaterializeReducer.class); > > > > conf.setJar(jarPath); > > > > > > > > DistributedCache.addCacheFile(new Path(uriInputPath).toUri(), > > > conf); > > > > > > > > FileInputFormat.setInputPaths(conf, new Path(rdfInputPath)); > > > > FileOutputFormat.setOutputPath(conf, new Path(outputPath)); > > > > > > > > conf.setInputFormat(TextInputFormat.class); > > > > conf.setOutputFormat(TextOutputFormat.class); > > > > > > > > JobClient.runJob(conf); > > > > > > > > It seems like I should be providing a URL to the JobTracker > somewhere, > > > but > > > > I > > > > can't figure out where to provide the information. > > > > > > > > -- > > > > Larry Compton > > > > > > > > > > > > > > > > -- > > > > > > 朱盛凯 > > > > > > Jash Zhu > > > > > > 复旦大学软件学院 > > > > > > Software School, Fudan University > > > > > > > > > > > -- Larry Compton