I'm seeing something very similar in my own Mesos/Spark Cluster. High level summary: When I use `--deploy-mode cluster`, java properties that I pass to my driver via `spark.driver.extraJavaOptions` are not available to the driver. I've confirmed this by inspecting the output of `System.getProperties` and the environment variables from within the driver.
I also see SPARK_EXECUTOR_OPTS showing the values that I wish were available as java system properties. Did you find anything that helped you either understand or resolve this issue? I'm still stuck. What version of spark + mesos are you using? 1.6.1 and 1.0.1-2.0.93.ubuntu1404 respectively over here. The only helpful bit I've found is that if I set the properties I care about in spark-defaults.conf on all the spark workers, they appear as desired in the drivers java properties. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Submit-job-with-driver-options-in-Mesos-Cluster-mode-tp27853p27972.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
