Re: Packaging for Hadoop - what about the Hadoop libraries?

2011-02-25 Thread James Seigel
The ones that are present. It is a little tricky for the other ones however, well not really once you “get it” -libjars on the commandline will ship the “supporting” jars out with the job to the map reducers, however if you, for some reason need them in the job submission they won’t be presen

Re: Packaging for Hadoop - what about the Hadoop libraries?

2011-02-25 Thread praveen.peddi
Just package the libraries that you MR jobs are dependent on. No need to package hadoop libraries. But make sure you hadoop client version matches with server version. Praveen On Feb 25, 2011, at 8:07 AM, "ext Mark Kerzner" wrote: > Hi, > > when packaging additional libraries for an MR job,

Packaging for Hadoop - what about the Hadoop libraries?

2011-02-25 Thread Mark Kerzner
Hi, when packaging additional libraries for an MR job, I can use a script or a Maven Hadoop plugin, but what about the Hadoop libraries themselves? Should I package them in, or should I rely on those jars that are already present in the Hadoop installation where the code will be running? What is t