You need to send a jar to the cluster so it can run your code there. Hadoop
doesn't magically know which jar is the one containing your main class, or
that of your mapper/reducer -- so you need to tell it via that call so it
knows which jar file to upload.
- Aaron
On Sun, Nov 29, 2009 at 7:54 AM, <aa...@buffalo.edu> wrote:
> Hi,
> Actually, I just made the change suggested by Aaron and my code worked.
> But I
> still would like to know why does the setJarbyClass() method have to be
> called
> when the Main class and the Map and Reduce classes are in the same package
> ?
>
> Thank You
>
> Abhishek Agrawal
>
> SUNY- Buffalo
> (716-435-7122)
>
> On Sun 11/29/09 10:39 AM , aa...@buffalo.edu sent:
> > Hi,
> > I dont set job.setJarByClass(Map.class). But my main class, the map class
> > andthe reduce class are all in the same package. Does this make any
> difference
> > at ordo I still have to call
> >
> > Thank You
> >
> > Abhishek Agrawal
> >
> > SUNY- Buffalo
> > (716-435-7122)
> >
> > On Fri 11/27/09 1:42 PM , Aaron Kimball aa...@clou
> > dera.com sent:> When you set up the Job object, do you
> > call> job.setJarByClass(Map.class)? That will tell
> > Hadoop which jar file to> ship with the job and to use for classloading
> in
> > your code.>
> > > - Aaron
> > > On Thu, Nov 26, 2009 at 11:56 PM,
> > wrote:> Hi,
> > > Â I am running the job from command
> > line. The job runs fine in the> local mode
> > > but something happens when I try to run the job
> > in the distributed> mode.
> > > Abhishek Agrawal
> > > SUNY- Buffalo
> > > (716-435-7122)
> > > On Fri 11/27/09 Â 2:31 AM , Jeff
> > Zhang sent:> > Do you run the map reduce job in command
> > line or IDE? in map> reduce
> > > > mode, you should put the jar containing the
> > map and reduce class> in
> > > > your classpath
> > > > Jeff Zhang
> > > > On Fri, Nov 27, 2009 at 2:19 PM,
> > Â wrote:> > Hello Everybody,
> > > > Â Â Â
> > Â Â Â Â Â I have
> > a doubt in Haddop and was wondering> if
> > > > anybody has faced a
> > > > similar problem. I have a package called
> > test. Inside that I have> > class called
> > > > A.java, Map.java, Reduce.java. In A.java I
> > have the main method> > where I am trying
> > > > to initialize the jobConf object. I have
> > written> > jobConf.setMapperClass(Map.class) and
> > similarly for the reduce> class
> > > > as well. The
> > > > code works correctly when I run the code
> > locally via> >
> > jobConf.set("mapred.job.tracker","local") but I get an
> > exception> > when I try to
> > > > run this code on my cluster. The stack
> > trace of the exception is> as
> > > > under. I
> > > > cannot understand the problem. Any help
> > would be appreciated.> > java.lang.RuntimeException:
> > java.lang.RuntimeException:> > java.lang.ClassNotFoundException:
> > test.Map> > Â Â Â
> > Â at> >
> > >
> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:752)> >
> Â Â Â
> > Â at> >
> > org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:690)> > Â
> Â Â
> > Â at> >
> > org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)> > Â Â
> Â
> > Â at> >
> > >
> > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)>
> > Â Â Â
> > Â at> >
> > > >
> > >
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
> > > > Â Â Â
> > Â at>
> > org.apache.hadoop.mapred.MapTask.run(MapTask.java:338)> > Â Â Â
> > Â at
> > org.apache.hadoop.mapred.Child.main(Child.java:158)> > Caused by:
> > java.lang.RuntimeException:> >
> > java.lang.ClassNotFoundException:> > Markowitz.covarMatrixMap
> > > > Â Â Â
> > Â at> >
> > >
> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:720)> >
> Â Â Â
> > Â at> >
> > >
> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:744)> >
> Â Â Â
> > Â ... 6 more> > Caused by:
> > java.lang.ClassNotFoundException: test.Map> > Â Â Â
> > Â at>
> > java.net.URLClassLoader$1.run(URLClassLoader.java:200)> > Â Â Â
> > Â at
> > java.security.AccessController.doPrivileged(Native> > Method)
> > > > Â Â Â
> > Â at> >
> > java.net.URLClassLoader.findClass(URLClassLoader.java:188)> > Â Â Â
> > Â at>
> > java.lang.ClassLoader.loadClass(ClassLoader.java:306)> > Â Â Â
> > Â at> >
> > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)> > Â Â Â
> > Â at>
> > java.lang.ClassLoader.loadClass(ClassLoader.java:251)> > Â Â Â
> > Â at> >
> > java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)> > Â Â Â
> > Â at java.lang.Class.forName0(Native Method)> > Â Â Â
> > Â at java.lang.Class.forName(Class.java:247)> > Â Â Â
> > Â at> >
> > > >
> > >
> >
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:673)
> > > > Â Â Â
> > Â at> >
> > >
> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:718)> >
> Â Â Â
> > Â ... 7 more> > Thank You
> > > > Abhishek Agrawal
> > > > SUNY- Buffalo
> > > > (716-435-7122)
> > > >
> > > >
> > >
> > >
> >
> >
> >
> >
> >
>
>
