Have you tried setting 'mapreduce.user.classpath.first'? It allows user jars to be put in the classpath before hadoop jars.
-----Original Message----- From: Sky USC <sky...@hotmail.com> Reply-To: "common-user@hadoop.apache.org" <common-user@hadoop.apache.org> Date: Mon, 9 Apr 2012 15:46:52 -0500 To: "common-user@hadoop.apache.org" <common-user@hadoop.apache.org> Subject: RE: How do I include the newer version of Commons-lang in my jar? > > > > >Thanks for the reply. I appreciate your helpfulness. I created Jars by >following instructions at >"http://blog.mafr.de/2010/07/24/maven-hadoop-job/";. So external Jars are >stored in lib/ folder within a jar. > >Am I summarizing this correctly: >1. If hadoop version = 0.20.203 or lower - then, there is not possible >for me to use an external jar such as "commons-lang" from apache in my >application. Any external jars packaged within my jar under "lib" >directory are not captured. This appears like a huge limitation to me? >2. If hadoop version > 0.20.204 to 1.0.x - then use >"HADOOP_USER_CLASSPATH_FIRST=true" environment variable before launching >"hadoop jar" might help. I tried this for version 0.20.205 but it didnt >work. >3. If hadoop version > 2.x or formerly 0.23.x - then this can be set via >API? > >Is there a working version of testable jar that has these dependencies >that I can try to figure out if its my way of packaging jar or something >else?? > >Thx > >> From: ha...@cloudera.com >> Date: Mon, 9 Apr 2012 13:50:37 +0530 >> Subject: Re: How do I include the newer version of Commons-lang in my >>jar? >> To: common-user@hadoop.apache.org >> >> Answer is a bit messy. >> >> Perhaps you can set the environment variable "export >> HADOOP_USER_CLASSPATH_FIRST=true" before you do a "hadoop jar Š" to >> launch your job. However, although this approach is present in >> 0.20.204+ (0.20.205, and 1.0.x), am not sure if it makes an impact on >> the tasks as well. I don't see it changing anything but for the driver >> CP. I've not tested it - please let us know if it works in your >> environment. >> >> In higher versions (2.x or formerly 0.23.x), this is doable from >> within your job if you set "mapreduce.job.user.classpath.first" to >> true inside your job, and ship your replacement jars along. >> >> Some versions would also let you set this via >> "JobConf/Job.setUserClassesTakesPrecedence(true/false)" API calls. >> >> On Mon, Apr 9, 2012 at 11:14 AM, Sky <sky...@hotmail.com> wrote: >> > Hi. >> > >> > I am new to Hadoop and I am working on project on AWS Elastic >>MapReduce. >> > >> > The problem I am facing is: >> > * org.apache.commons.lang.time.DateUtils: parseDate() works OK but >> > parseDateStrictly() fails. >> > I think parseDateStrictly might be new in lang 2.5. I thought I >>included all >> > dependencies. However, for some reason, during runtime, my app is not >> > picking up the newer commons-lang. >> > >> > Would love some help. >> > >> > Thx >> > - sky >> > >> > >> >> >> >> -- >> Harsh J > >
