On Mon, Sep 8, 2014 at 11:15 PM, Sean Owen so...@cloudera.com wrote:
This structure is not specific to Hadoop, but in theory works in any
JAR file. You can put JARs in JARs and refer to them with Class-Path
entries in META-INF/MANIFEST.MF.
Funny that you mention that, since someone internally
Hm, so it is:
http://docs.oracle.com/javase/tutorial/deployment/jar/downman.html
I'm sure I've done this before though and thought is was this mechanism. It
must be something custom.
What's the Hadoop jar structure in question then? Is it something special
like a WAR file? I confess I had never
In modern projects there are a bazillion dependencies - when I use Hadoop I
just put them in a lib directory in the jar - If I have a project that
depends on 50 jars I need a way to deliver them to Spark - maybe wordcount
can be written without dependencies but real projects need to deliver
On Wed, Sep 10, 2014 at 3:44 PM, Sean Owen so...@cloudera.com wrote:
What's the Hadoop jar structure in question then? Is it something special
like a WAR file? I confess I had never heard of this so thought this was
about generic JAR stuff.
What I've been told (and Steve's e-mail alludes to)
This structure is not specific to Hadoop, but in theory works in any
JAR file. You can put JARs in JARs and refer to them with Class-Path
entries in META-INF/MANIFEST.MF.
It works but I have found it can cause trouble with programs that
query the JARs on the classpath to find other classes. When
In a Hadoop jar there is a directory called lib and all non-provided third
party jars go there and are included in the class path of the code. Do jars
for Spark have the same structure - another way to ask the question is if I
have code to execute Spark and a jar build for Hadoop can I simply use
