I was using 0.23 and was adding files using the -libjars flag (wanted to upload some jars which were dependencies for my project) but for some reason I could never find it in the DistributedCache or would always keep on getting ClassNotFound on the other side. I took the snippet of code which does that work when you invoke the hadoop jar command and put it in my class and it all worked fine. I don't know if the problem was with my code or if it was a bug. Given that -files is so extensively used, I felt it could be an issue on my side. In the end I started using 1.1 hadoop and so completely forgot about diving more deeper but I can definitely revive it and dig in more.
Hopefully you can try using -libjars too and see if that also is facing a similar issue since they both are command line switches which should have almost similar behavior. Thanks, Viral On Tue, Feb 19, 2013 at 10:33 AM, Robert Evans <ev...@yahoo-inc.com> wrote: > Yes if you can trace this down I would be very interested. We are running > 0.23.6 without any issues, but that does not mean that there is not some > bug in the code that is causing this to happen in your situation. > > --Bobby > > From: Sebastiano Vigna <vi...@di.unimi.it> > Reply-To: "user@hadoop.apache.org" <user@hadoop.apache.org> > Date: Saturday, February 16, 2013 8:39 AM > To: "user@hadoop.apache.org" <user@hadoop.apache.org> > Subject: Re: Correct way to unzip locally an archive in Yarn > > I will as soon as I can understand what happens on the cluster (no access > from home). DistributedCache.getLocalCacheFiles() returns in both cases a > local name for the zip file uploaded with -files, but locally my unzip code > works, on the cluster it throws a FileNotFoundException. > > > On 16 February 2013 15:22, Arun C Murthy <a...@hortonworks.com> wrote: > >> This could be a bug, mind opening a jira? Thanks. >> >> On Feb 16, 2013, at 2:34 AM, Sebastiano Vigna wrote: >> >> On 15 February 2013 16:57, Robert Evans <ev...@yahoo-inc.com> wrote: >> >>> Are you trying to run a Map/Reduce job or are you writing a new YARN >>> application? If it is a MR job, then it should work mostly the same as >>> before (on 1.x). If you are writing a new YARN application then there is >>> a >>> separate Map in the ContainerLaunchContext that you need to fill in. >> >> >> It's a MapReduce job (0.23.6). After two days of useless trials, I'm >> uploading the zip with -files and I wrote a stub to unzip it manually. I >> was positively unable to get the archive unzipped *to a local directory* in >> any way. >> >> Unfortunately it works in local but not on the cluster. I have still to >> discover why. :( >> >> Ciao, >> >> >> >> >> -- >> Arun C. Murthy >> Hortonworks Inc. >> http://hortonworks.com/ >> >> >> >
