Re: Re: Re: Re: Doubt in Hadoop

[aa225]

Hi,
   Actually, I just made the change suggested by Aaron and my code worked. But I
still would like to know why does the setJarbyClass() method have to be called
when the Main class and the Map and Reduce classes are in the same package ?

Thank You

Abhishek Agrawal

SUNY- Buffalo
(716-435-7122)

On Sun 11/29/09 10:39 AM , aa...@buffalo.edu sent:
> Hi,
> I dont set job.setJarByClass(Map.class). But my main class, the map class
> andthe reduce class are all in the same package. Does this make any difference
> at ordo I still have to call 
> 
> Thank You
> 
> Abhishek Agrawal
> 
> SUNY- Buffalo
> (716-435-7122)
> 
> On Fri 11/27/09  1:42 PM , Aaron Kimball aa...@clou
> dera.com sent:> When you set up the Job object, do you
> call> job.setJarByClass(Map.class)? That will tell
> Hadoop which jar file to> ship with the job and to use for classloading in
> your code.> 
> > - Aaron
> > On Thu, Nov 26, 2009 at 11:56 PM,  
> wrote:> Hi,
> >   I am running the job from command
> line. The job runs fine in the> local mode
> > but something happens when I try to run the job
> in the distributed> mode.
> > Abhishek Agrawal
> > SUNY- Buffalo
> > (716-435-7122)
> > On Fri 11/27/09  2:31 AM , Jeff
> Zhang  sent:> > Do you run the map reduce job in command
> line or IDE?  in map> reduce
> > > mode, you should put the jar containing the
> map and reduce class> in
> > > your classpath
> > > Jeff Zhang
> > > On Fri, Nov 27, 2009 at 2:19 PM,
>   wrote:> > Hello Everybody,
> > >      
>          I have
> a doubt in Haddop and was wondering> if
> > > anybody has faced a
> > > similar problem. I have a package called
> test. Inside that I have> > class called
> > > A.java, Map.java, Reduce.java. In A.java I
> have the main method> > where I am trying
> > > to initialize the jobConf object. I have
> written> > jobConf.setMapperClass(Map.class) and
> similarly for the reduce> class
> > > as well. The
> > > code works correctly when I run the code
> locally via> >
> jobConf.set("mapred.job.tracker","local") but I get an
> exception> > when I try to
> > > run this code on my cluster. The stack
> trace of the exception is> as
> > > under. I
> > > cannot understand the problem. Any help
> would be appreciated.> > java.lang.RuntimeException:
> java.lang.RuntimeException:> > java.lang.ClassNotFoundException:
> test.Map> >      
>  at> >
> >
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:752)> >   
>    
>  at> >
> org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:690)> >      
>  at> >
> org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)> >      
>  at> >
> >
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)> >   
>    
>  at> >
> > >
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
> > >      
>  at>
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:338)> >      
>  at
> org.apache.hadoop.mapred.Child.main(Child.java:158)> > Caused by:
> java.lang.RuntimeException:> >
> java.lang.ClassNotFoundException:> > Markowitz.covarMatrixMap
> > >      
>  at> >
> >
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:720)> >   
>    
>  at> >
> >
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:744)> >   
>    
>  ... 6 more> > Caused by:
> java.lang.ClassNotFoundException: test.Map> >      
>  at>
> java.net.URLClassLoader$1.run(URLClassLoader.java:200)> >      
>  at
> java.security.AccessController.doPrivileged(Native> > Method)
> > >      
>  at> >
> java.net.URLClassLoader.findClass(URLClassLoader.java:188)> >      
>  at>
> java.lang.ClassLoader.loadClass(ClassLoader.java:306)> >      
>  at> >
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)> >      
>  at>
> java.lang.ClassLoader.loadClass(ClassLoader.java:251)> >      
>  at> >
> java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)> >      
>  at java.lang.Class.forName0(Native Method)> >      
>  at java.lang.Class.forName(Class.java:247)> >      
>  at> >
> > >
> >
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:673)
> > >      
>  at> >
> >
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:718)> >   
>    
>  ... 7 more> > Thank You
> > > Abhishek Agrawal
> > > SUNY- Buffalo
> > > (716-435-7122)
> > >
> > >
> > 
> > 
> 
> 
> 
> 
>