Thanks for the reply. I appreciate your helpfulness. I created Jars by
following instructions at "http://blog.mafr.de/2010/07/24/maven-hadoop-job/";.
So external Jars are stored in lib/ folder within a jar.
Am I summarizing this correctly:
1. If hadoop version = 0.20.203 or lower - then, there is not possible for me
to use an external jar such as "commons-lang" from apache in my application.
Any external jars packaged within my jar under "lib" directory are not
captured. This appears like a huge limitation to me?
2. If hadoop version > 0.20.204 to 1.0.x - then use
"HADOOP_USER_CLASSPATH_FIRST=true" environment variable before launching
"hadoop jar" might help. I tried this for version 0.20.205 but it didnt work.
3. If hadoop version > 2.x or formerly 0.23.x - then this can be set via API?
Is there a working version of testable jar that has these dependencies that I
can try to figure out if its my way of packaging jar or something else??
Thx
> From: ha...@cloudera.com
> Date: Mon, 9 Apr 2012 13:50:37 +0530
> Subject: Re: How do I include the newer version of Commons-lang in my jar?
> To: common-user@hadoop.apache.org
>
> Answer is a bit messy.
>
> Perhaps you can set the environment variable "export
> HADOOP_USER_CLASSPATH_FIRST=true" before you do a "hadoop jar …" to
> launch your job. However, although this approach is present in
> 0.20.204+ (0.20.205, and 1.0.x), am not sure if it makes an impact on
> the tasks as well. I don't see it changing anything but for the driver
> CP. I've not tested it - please let us know if it works in your
> environment.
>
> In higher versions (2.x or formerly 0.23.x), this is doable from
> within your job if you set "mapreduce.job.user.classpath.first" to
> true inside your job, and ship your replacement jars along.
>
> Some versions would also let you set this via
> "JobConf/Job.setUserClassesTakesPrecedence(true/false)" API calls.
>
> On Mon, Apr 9, 2012 at 11:14 AM, Sky <sky...@hotmail.com> wrote:
> > Hi.
> >
> > I am new to Hadoop and I am working on project on AWS Elastic MapReduce.
> >
> > The problem I am facing is:
> > * org.apache.commons.lang.time.DateUtils: parseDate() works OK but
> > parseDateStrictly() fails.
> > I think parseDateStrictly might be new in lang 2.5. I thought I included all
> > dependencies. However, for some reason, during runtime, my app is not
> > picking up the newer commons-lang.
> >
> > Would love some help.
> >
> > Thx
> > - sky
> >
> >
>
>
>
> --
> Harsh J
