You need to put the library jar on your classpath (eg using
HADOOP_CLASSPATH) as well. The -libjars will ship it to the cluster
and put it on the classpath of your task, but not the classpath of
your "driver" code.
-Todd
On Thu, Dec 9, 2010 at 10:29 PM, Vipul Pandey <vipan...@gmail.com> wrote:
> disclaimer : a newbie!!!
> Howdy?
> Got a quick question. -libjars option doesn't seem to work for me in -
> prettymuch - my first (or mayby second) mapreduce job.
> Here's what i'm doing :
> $bin/hadoop jar sherlock.jar somepkg.FindSchoolsJob -libjars
> HStats-1A18.jar input output
>
> sherlock.jar has my main class (ofcourse) FindSchoolsJob, which runs just
> fine by itself till I add a dependency on a class in HStats-1A18.jar.
> When I run the above command with -libjars specified - it fails to find my
> classes that 'are' inside HStats jar file.
> Exception in thread "main" java.lang.NoClassDefFoundError: com/*****/HAgent
> at com.*****.FindSchoolsJob.run(FindSchoolsJob.java:46)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at com.******.FindSchoolsJob.main(FindSchoolsJob.java:101)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.lang.ClassNotFoundException:com/*****/HAgent
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> ... 8 more
>
> My main class is defined as below :
> public class FindSchoolsJob extends Configured implements Tool {
> :
> public int run(String[] args) throws Exception {
> :
> :
> }
> :
> public static void main(String[] args) throws Exception {
> int res = ToolRunner.run(new Configuration(), new FindSchoolsJob(),
> args);
> System.exit(res);
> }
> }
> Any hint would be highly appreciated.
> Thank You!
> ~V
--
Todd Lipcon
Software Engineer, Cloudera
