I figure out the input directory part: I just need to add the
$HADOOP_HOME/conf directory into the classpath in eclipse.
However, now I ran into a new problem: now the program complains that it
cannot find the class files for my mapper and reducer! The error message is
as follows:
08/10/07 10:21:52 WARN mapred.JobClient: No job jar file set. User classes
may not be found. See JobConf(Class) or JobConf#setJar(String).
08/10/07 10:21:52 INFO mapred.FileInputFormat: Total input paths to process
: 1
08/10/07 10:21:54 INFO mapred.JobClient: Running job: job_200810071020_0001
08/10/07 10:21:55 INFO mapred.JobClient: map 0% reduce 0%
08/10/07 10:22:01 INFO mapred.JobClient: Task Id :
task_200810071020_0001_m_00_0, Status : FAILED
java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.ClassNotFoundException:
org.apache.hadoop.examples.WordCount$Reduce
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:633)
at org.apache.hadoop.mapred.JobConf.getCombinerClass(JobConf.java:768)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:383)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:185)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2122)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.hadoop.examples.WordCount$Reduce
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:601)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:625)
... 4 more
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.examples.WordCount$Reduce
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:581)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:599)
... 5 more
I don't know how to fix this. It seems everyone is using the hadoop jar
command line for running the program. however, I don't understand why this
won't work since I am using the jobclient interface for interacting with
hadoop...
Would really appreciate if anybody can share some experience on this!
Thank you!
On Mon, Oct 6, 2008 at 10:48 AM, Ski Gh3 <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> I have a weird problem regarding running the wordcount example from
> eclipse.
>
> I was able to run the wordcount example from the command line like:
> $ ...MyHadoop/bin/hadoop jar ../MyHadoop/hadoop-xx-examples.jar wordcount
> myinputdir myoutputdir
>
> However, if I try to run the wordcount program from Eclipse (supplying same
> two args: myinputdir myoutputdir)
> I got the following error messsage:
>
> Exception in thread "main" java.lang.RuntimeException: java.io.IOException:
> No FileSystem for scheme: file
> at
> org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:356)
> at
> org.apache.hadoop.mapred.FileInputFormat.setInputPaths(FileInputFormat.java:331)
> at
> org.apache.hadoop.mapred.FileInputFormat.setInputPaths(FileInputFormat.java:304)
> at org.apache.hadoop.examples.WordCount.run(WordCount.java:149)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.examples.WordCount.main(WordCount.java:161)
> Caused by: java.io.IOException: No FileSystem for scheme: file
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1277)
> at org.apache.hadoop.fs.FileSystem.access$1(FileSystem.java:1273)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1291)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:203)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:108)
> at
> org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:352)
> ... 5 more
>
> It seems from within Eclipse, the program does not know how to interpret
> the myinputdir as a hadoop path?
>
> Can someone please tell me how I can fix this?
>
> Thanks a lot!!!
>
