Re: SPARK_SUBMIT_CLASSPATH question

2014-10-15 Thread Marcelo Vanzin
Hi Greg, I'm not sure exactly what it is that you're trying to achieve, but I'm pretty sure those variables are not supposed to be set by users. You should take a look at the documentation for "spark.driver.extraClassPath" and "spark.driver.extraLibraryPath", and the equivalent options for executo

Re: SPARK_SUBMIT_CLASSPATH question

2014-10-15 Thread Greg Hill
explore other features and the classpath string is going to take up a whole screen. Greg From: Greg mailto:greg.h...@rackspace.com>> Date: Tuesday, October 14, 2014 1:57 PM To: "user@spark.apache.org<mailto:user@spark.apache.org>" mailto:user@spark.apache.org>>

SPARK_SUBMIT_CLASSPATH question

2014-10-14 Thread Greg Hill
It seems to me that SPARK_SUBMIT_CLASSPATH does not follow the same ability as other tools to put wildcards in the paths you add. For some reason it doesn't pick up the classpath information from yarn-site.xml either, it seems, when running on YARN. I'm having to manually add every single depe