Re: yarn-client: SparkSubmitDriverBootstrapper not found in yarn client mode (1.6.0)

2016-01-13 Thread Kevin Mellott
Lin - if you add "--verbose" to your original *spark-submit* command, it will let you know the location in which Spark is running. As Marcelo pointed out, this will likely indicate version 1.3, which may help you confirm if this is your problem. On Wed, Jan 13, 2016 at 12:06 PM, Marcelo Vanzin

yarn-client: SparkSubmitDriverBootstrapper not found in yarn client mode (1.6.0)

2016-01-13 Thread Lin Zhao
My job runs fine in yarn cluster mode but I have reason to use client mode instead. But I'm hitting this error when submitting: > spark-submit --class com.exabeam.martini.scripts.SparkStreamingTest --master > yarn --deploy-mode client --executor-memory 90G --num-executors 3 > --executor-cores

Re: yarn-client: SparkSubmitDriverBootstrapper not found in yarn client mode (1.6.0)

2016-01-13 Thread Ted Yu
Can you show the complete stack trace for the error ? I searched 1.6.0 code base but didn't find the class SparkSubmitDriverBootstrapper Thanks On Wed, Jan 13, 2016 at 9:31 AM, Lin Zhao wrote: > My job runs fine in yarn cluster mode but I have reason to use client mode >

Re: yarn-client: SparkSubmitDriverBootstrapper not found in yarn client mode (1.6.0)

2016-01-13 Thread Jeff Zhang
I also didn't find SparkSubmitDriverBootstrapper, which version of spark are you using ? On Wed, Jan 13, 2016 at 9:36 AM, Ted Yu wrote: > Can you show the complete stack trace for the error ? > > I searched 1.6.0 code base but didn't find the > class

Re: yarn-client: SparkSubmitDriverBootstrapper not found in yarn client mode (1.6.0)

2016-01-13 Thread Marcelo Vanzin
SparkSubmitDriverBootstrapper was removed back in Spark 1.4, so it seems you have a mixbag of 1.3 / 1.6 in your path / classpath and things are failing because of that. On Wed, Jan 13, 2016 at 9:31 AM, Lin Zhao wrote: > My job runs fine in yarn cluster mode but I have reason to