It would probably be best to retain support for SPARK_JAVA_OPTS in
ClientBase though..for developers that may have been using it.
On Wed, Apr 23, 2014 at 6:26 PM, Nishkam Ravi <nr...@cloudera.com> wrote:
> Bit of a race condition here it seems. Patrick made a few changes
> yesterday around the same time as I did (in ClientBase.scala):
>
> for ((k, v) <- sys.props.filterKeys(_.startsWith("spark")))
> { JAVA_OPTS += "-D" + k + "=" + "\\\"" + v + "\\\"" }
>
> This would allow JAVA_OPTS to be passed on the command line to the
> ApplicationMaster, and accomplishes the same things as creation of a new
> command line flag --spark-java-opts.
>
>
> Mridul, the use of SPARK_JAVA_OPTS has been intentionally suppressed.
>
>
> On Wed, Apr 23, 2014 at 10:54 AM, Mridul Muralidharan <mri...@gmail.com>wrote:
>
>> Sorry, I misread - I meant SPARK_JAVA_OPTS - not JAVA_OPTS.
>> See here : https://issues.apache.org/jira/browse/SPARK-1588
>>
>> Regards,
>> Mridul
>>
>> On Wed, Apr 23, 2014 at 6:37 PM, Mridul Muralidharan <mri...@gmail.com>
>> wrote:
>> > This breaks all existing jobs which are not using spark-submit.
>> > The consensus was not to break compatibility unless there was an
>> overriding
>> > reason to do so
>> >
>> > On Apr 23, 2014 6:32 PM, "Thomas Graves (JIRA)" <j...@apache.org>
>> wrote:
>> >>
>> >>
>> >> [
>> >>
>> https://issues.apache.org/jira/browse/SPARK-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13978164#comment-13978164
>> >> ]
>> >>
>> >> Thomas Graves commented on SPARK-1576:
>> >> --------------------------------------
>> >>
>> >> Is this meant for the driver or the executors? The spark-submit script
>> >> has a command line option for the driver: --driver-java-options.
>> >> I believe the intent of https://github.com/apache/spark/pull/299 was
>> to
>> >> not expose SPARK_JAVA_OPTS to the user anymore.
>> >>
>> >> > Passing of JAVA_OPTS to YARN on command line
>> >> > --------------------------------------------
>> >> >
>> >> > Key: SPARK-1576
>> >> > URL:
>> https://issues.apache.org/jira/browse/SPARK-1576
>> >> > Project: Spark
>> >> > Issue Type: Improvement
>> >> > Affects Versions: 0.9.0, 1.0.0, 0.9.1
>> >> > Reporter: Nishkam Ravi
>> >> > Fix For: 0.9.0, 1.0.0, 0.9.1
>> >> >
>> >> > Attachments: SPARK-1576.patch
>> >> >
>> >> >
>> >> > JAVA_OPTS can be passed by using either env variables (i.e.,
>> >> > SPARK_JAVA_OPTS) or as config vars (after Patrick's recent change).
>> It would
>> >> > be good to allow the user to pass them on command line as well to
>> restrict
>> >> > scope to single application invocation.
>> >>
>> >>
>> >>
>> >> --
>> >> This message was sent by Atlassian JIRA
>> >> (v6.2#6252)
>>
>
>
