You could use the DistributedCache to put multiple jar's into the classpath. Of cause you would have to write your own job-submission logic for that....

Johannes

On Dec 3, 2008, at 3:19 PM, Scott Whitecross wrote:

What's the best way to use third party libraries with Hadoop? For example, I want to run a job with both a jar file containing the mob, and also extra libraries. I noticed a couple solutions with a search, but I'm hoping for something better:

- Merge the third party jar libraries into the job jar
- Distribute the third party libraries across the cluster in the local boxes classpath.

What I'd really like is a way to add an extra option to the hadoop jar command, such as hadoop/bin/hadoop jar myJar.jar myJobClass - classpath thirdpartyjar1.jar:jar2.jar:etc args

Anything exist like this?


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101tec GmbH
Halle (Saale), Saxony-Anhalt, Germany
http://www.101tec.com

Reply via email to