[jira] [Commented] (SPARK-1588) SPARK_JAVA_OPTS is not getting propagated
[ https://issues.apache.org/jira/browse/SPARK-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13978827#comment-13978827 ] Mridul Muralidharan commented on SPARK-1588: Apparently, SPARK_YARN_USER_ENV is also broken > SPARK_JAVA_OPTS is not getting propagated > - > > Key: SPARK-1588 > URL: https://issues.apache.org/jira/browse/SPARK-1588 > Project: Spark > Issue Type: Bug > Components: YARN >Affects Versions: 1.0.0 >Reporter: Mridul Muralidharan >Priority: Blocker > > We could use SPARK_JAVA_OPTS to pass JAVA_OPTS to be used in the master. > This is no longer working in current master. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (SPARK-1588) SPARK_JAVA_OPTS is not getting propagated
[ https://issues.apache.org/jira/browse/SPARK-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13978703#comment-13978703 ] Patrick Wendell commented on SPARK-1588: Btw - SPARK_JAVA_OPTS is officially dropped in 1.0 and there are loud warnings printed if it's used. Unfortunately this was a brute force mechanism used all over the place that had tons of corner cases and bugs. But I agree we should do our best to backwards support the old mechanisms. It's currently backwards-supported in the park-submit script but not the yarn client. > SPARK_JAVA_OPTS is not getting propagated > - > > Key: SPARK-1588 > URL: https://issues.apache.org/jira/browse/SPARK-1588 > Project: Spark > Issue Type: Bug > Components: YARN >Affects Versions: 1.0.0 >Reporter: Mridul Muralidharan >Priority: Blocker > > We could use SPARK_JAVA_OPTS to pass JAVA_OPTS to be used in the master. > This is no longer working in current master. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (SPARK-1588) SPARK_JAVA_OPTS is not getting propagated
[ https://issues.apache.org/jira/browse/SPARK-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13978612#comment-13978612 ] Patrick Wendell commented on SPARK-1588: The approach in spark-submit is to capture SPARK_JAVA_OPTS and convert them into spark.executor.javaOptions and spark.driver.javaOptions - I think we can just do the same thing in the yarn client to provide backwards compatiblity. > SPARK_JAVA_OPTS is not getting propagated > - > > Key: SPARK-1588 > URL: https://issues.apache.org/jira/browse/SPARK-1588 > Project: Spark > Issue Type: Bug > Components: YARN >Affects Versions: 1.0.0 >Reporter: Mridul Muralidharan >Priority: Blocker > > We could use SPARK_JAVA_OPTS to pass JAVA_OPTS to be used in the master. > This is no longer working in current master. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (SPARK-1588) SPARK_JAVA_OPTS is not getting propagated
[ https://issues.apache.org/jira/browse/SPARK-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13978095#comment-13978095 ] Mridul Muralidharan commented on SPARK-1588: Noticed this specifically in yarn-alpha - we use SPARK_JAVA_OPTS to propagate system properties : this is not working anymore in the master. > SPARK_JAVA_OPTS is not getting propagated > - > > Key: SPARK-1588 > URL: https://issues.apache.org/jira/browse/SPARK-1588 > Project: Spark > Issue Type: Bug > Components: YARN >Affects Versions: 1.0.0 >Reporter: Mridul Muralidharan >Priority: Blocker > > We could use SPARK_JAVA_OPTS to pass JAVA_OPTS to be used in the master. > This is no longer working in current master. -- This message was sent by Atlassian JIRA (v6.2#6252)
