RE: Specifying external jars in the classpath for Hadoop

2007-08-14 Thread Avinash Lakshman
jars in the classpath for Hadoop That was what Doug recommended last time we talked. On 8/14/07 8:33 AM, "Joydeep Sen Sarma" <[EMAIL PROTECTED]> wrote: > i found depositing required jars into the lib directory works just great > (all those jars are prepended to the classpat

Re: Specifying external jars in the classpath for Hadoop

2007-08-14 Thread Doug Cutting
Eyal Oren wrote: As far as I understand (that's what we do anyway), you have to submit one jar that contains all your dependencies (except for dependencies on hadoop libs), including external jars. The easiest is probably to build maven/ant to build such "big" jar externally with all its depend

Re: Specifying external jars in the classpath for Hadoop

2007-08-14 Thread Ted Dunning
That was what Doug recommended last time we talked. On 8/14/07 8:33 AM, "Joydeep Sen Sarma" <[EMAIL PROTECTED]> wrote: > i found depositing required jars into the lib directory works just great > (all those jars are prepended to the classpath by the hadoop script). > > Any flaws doing it this

Re: Specifying external jars in the classpath for Hadoop

2007-08-14 Thread Dennis Kubes
this way? -Original Message- From: Eyal Oren [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 14, 2007 12:45 AM To: hadoop-user@lucene.apache.org Subject: Re: Specifying external jars in the classpath for Hadoop On 08/13/07/08/07 16:49 -0700, Phantom wrote: Hi I have a map/reduce job

RE: Specifying external jars in the classpath for Hadoop

2007-08-14 Thread Joydeep Sen Sarma
-user@lucene.apache.org Subject: Re: Specifying external jars in the classpath for Hadoop On 08/13/07/08/07 16:49 -0700, Phantom wrote: >Hi > >I have a map/reduce job that uses external jar files. How do I specify those >jars in the classpath when submitting the mapred job using

Re: Specifying external jars in the classpath for Hadoop

2007-08-14 Thread Dennis Kubes
HADOOP-1622 fixes this to allow multiple resources, including jars, to be submitted for a single mapreduce job. There is currently a patch that "works" but still needs a little fixing. I should be able to get it finished in the next couple of days. Dennis Kubes Eyal Oren wrote: On 08/13/07

Re: Specifying external jars in the classpath for Hadoop

2007-08-14 Thread Eyal Oren
On 08/13/07/08/07 16:49 -0700, Phantom wrote: Hi I have a map/reduce job that uses external jar files. How do I specify those jars in the classpath when submitting the mapred job using ./hadoop jar ? Suppose my map job relies on API in some external jar how do I pass this jar file as part o

Specifying external jars in the classpath for Hadoop

2007-08-13 Thread Phantom
Hi I have a map/reduce job that uses external jar files. How do I specify those jars in the classpath when submitting the mapred job using ./hadoop jar ? Suppose my map job relies on API in some external jar how do I pass this jar file as part of my job submission. Thanks A