When you set up the Job object, do you call job.setJarByClass(Map.class)?
That will tell Hadoop which jar file to ship with the job and to use for
classloading in your code.
- Aaron
On Thu, Nov 26, 2009 at 11:56 PM, <aa...@buffalo.edu> wrote:
> Hi,
> I am running the job from command line. The job runs fine in the local
> mode
> but something happens when I try to run the job in the distributed mode.
>
>
> Abhishek Agrawal
>
> SUNY- Buffalo
> (716-435-7122)
>
> On Fri 11/27/09 2:31 AM , Jeff Zhang zjf...@gmail.com sent:
> > Do you run the map reduce job in command line or IDE? in map reduce
> > mode, you should put the jar containing the map and reduce class in
> > your classpath
> > Jeff Zhang
> > On Fri, Nov 27, 2009 at 2:19 PM, wrote:
> > Hello Everybody,
> > I have a doubt in Haddop and was wondering if
> > anybody has faced a
> > similar problem. I have a package called test. Inside that I have
> > class called
> > A.java, Map.java, Reduce.java. In A.java I have the main method
> > where I am trying
> > to initialize the jobConf object. I have written
> > jobConf.setMapperClass(Map.class) and similarly for the reduce class
> > as well. The
> > code works correctly when I run the code locally via
> > jobConf.set("mapred.job.tracker","local") but I get an exception
> > when I try to
> > run this code on my cluster. The stack trace of the exception is as
> > under. I
> > cannot understand the problem. Any help would be appreciated.
> > java.lang.RuntimeException: java.lang.RuntimeException:
> > java.lang.ClassNotFoundException: test.Map
> > at
> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:752)
> > at
> > org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:690)
> > at
> > org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> > at
> > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
> > at
> >
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:338)
> > at org.apache.hadoop.mapred.Child.main(Child.java:158)
> > Caused by: java.lang.RuntimeException:
> > java.lang.ClassNotFoundException:
> > Markowitz.covarMatrixMap
> > at
> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:720)
> > at
> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:744)
> > ... 6 more
> > Caused by: java.lang.ClassNotFoundException: test.Map
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
> > at java.security.AccessController.doPrivileged(Native
> > Method)
> > at
> > java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> > at
> > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
> > at
> > java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
> > at java.lang.Class.forName0(Native Method)
> > at java.lang.Class.forName(Class.java:247)
> > at
> >
> >
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:673)
> > at
> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:718)
> > ... 7 more
> > Thank You
> > Abhishek Agrawal
> > SUNY- Buffalo
> > (716-435-7122)
> >
> >
>
>
