It would probably be best to retain support for SPARK_JAVA_OPTS in ClientBase though..for developers that may have been using it.
On Wed, Apr 23, 2014 at 6:26 PM, Nishkam Ravi <nr...@cloudera.com> wrote: > Bit of a race condition here it seems. Patrick made a few changes > yesterday around the same time as I did (in ClientBase.scala): > > for ((k, v) <- sys.props.filterKeys(_.startsWith("spark"))) > { JAVA_OPTS += "-D" + k + "=" + "\\\"" + v + "\\\"" } > > This would allow JAVA_OPTS to be passed on the command line to the > ApplicationMaster, and accomplishes the same things as creation of a new > command line flag --spark-java-opts. > > > Mridul, the use of SPARK_JAVA_OPTS has been intentionally suppressed. > > > On Wed, Apr 23, 2014 at 10:54 AM, Mridul Muralidharan <mri...@gmail.com>wrote: > >> Sorry, I misread - I meant SPARK_JAVA_OPTS - not JAVA_OPTS. >> See here : https://issues.apache.org/jira/browse/SPARK-1588 >> >> Regards, >> Mridul >> >> On Wed, Apr 23, 2014 at 6:37 PM, Mridul Muralidharan <mri...@gmail.com> >> wrote: >> > This breaks all existing jobs which are not using spark-submit. >> > The consensus was not to break compatibility unless there was an >> overriding >> > reason to do so >> > >> > On Apr 23, 2014 6:32 PM, "Thomas Graves (JIRA)" <j...@apache.org> >> wrote: >> >> >> >> >> >> [ >> >> >> https://issues.apache.org/jira/browse/SPARK-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13978164#comment-13978164 >> >> ] >> >> >> >> Thomas Graves commented on SPARK-1576: >> >> -------------------------------------- >> >> >> >> Is this meant for the driver or the executors? The spark-submit script >> >> has a command line option for the driver: --driver-java-options. >> >> I believe the intent of https://github.com/apache/spark/pull/299 was >> to >> >> not expose SPARK_JAVA_OPTS to the user anymore. >> >> >> >> > Passing of JAVA_OPTS to YARN on command line >> >> > -------------------------------------------- >> >> > >> >> > Key: SPARK-1576 >> >> > URL: >> https://issues.apache.org/jira/browse/SPARK-1576 >> >> > Project: Spark >> >> > Issue Type: Improvement >> >> > Affects Versions: 0.9.0, 1.0.0, 0.9.1 >> >> > Reporter: Nishkam Ravi >> >> > Fix For: 0.9.0, 1.0.0, 0.9.1 >> >> > >> >> > Attachments: SPARK-1576.patch >> >> > >> >> > >> >> > JAVA_OPTS can be passed by using either env variables (i.e., >> >> > SPARK_JAVA_OPTS) or as config vars (after Patrick's recent change). >> It would >> >> > be good to allow the user to pass them on command line as well to >> restrict >> >> > scope to single application invocation. >> >> >> >> >> >> >> >> -- >> >> This message was sent by Atlassian JIRA >> >> (v6.2#6252) >> > >