Hey Stephen, In case these libraries exist on the client as a form of maven library, you can use --packages to ship the library and all it's dependencies, without building an uber jar.
Best, Burak On Tue, Jul 28, 2015 at 10:23 AM, Marcelo Vanzin <van...@cloudera.com> wrote: > Hi Stephen, > > There is no such directory currently. If you want to add an existing jar > to every app's classpath, you need to modify two config values: > spark.driver.extraClassPath and spark.executor.extraClassPath. > > On Mon, Jul 27, 2015 at 10:22 PM, Stephen Boesch <java...@gmail.com> > wrote: > >> when using spark-submit: which directory contains third party libraries >> that will be loaded on each of the slaves? I would like to scp one or more >> libraries to each of the slaves instead of shipping the contents in the >> application uber-jar. >> >> Note: I did try adding to $SPARK_HOME/lib_managed/jars. But the >> spark-submit still results in a ClassNotFoundException for classes included >> in the added library. >> >> > > > -- > Marcelo >
