RE: Extra libs in executor classpath

om> Sent: Wednesday, March 16, 2016 7:22 AM To: user@spark.apache.org<mailto:user@spark.apache.org> Subject: Extra libs in executor classpath Hello, guys! I’ve been developing a kind of framework on top of spark, and my idea is to bundle the framework jars and some extra configs with t

Fwd: Extra libs in executor classpath

It makes no sense for worker, the issue is with executor classpath, not the driver classpath. Please, answer actual question that is not in "P.S." - that one it's just a note about driver Thanks, Leonid On Wed, Mar 16, 2016 at 6:21 PM, Ted Yu wrote: > For your last point,

Re: Extra libs in executor classpath

For your last point, spark-submit has: if [ -z "${SPARK_HOME}" ]; then export SPARK_HOME="$(cd "`dirname "$0"`"/..; pwd)" fi Meaning the script would determine the proper SPARK_HOME variable. FYI On Wed, Mar 16, 2016 at 4:22 AM, Леонид Поляков wrote: > Hello, guys! > > >

Re: Extra libs in executor classpath

t; > > > > > > *From: *Леонид Поляков <owis...@gmail.com> > *Sent: *Wednesday, March 16, 2016 7:22 AM > *To: *user@spark.apache.org > *Subject: *Extra libs in executor classpath > > > > Hello, guys! > > > > I’ve been developing a kind of framework

Extra libs in executor classpath

Hello, guys! I’ve been developing a kind of framework on top of spark, and my idea is to bundle the framework jars and some extra configs with the spark and pass it to other developers for their needs. So that devs can use this bundle and run usual spark stuff but with extra flavor that