[jira] [Commented] (SPARK-1588) SPARK_JAVA_OPTS is not getting propagated
[ https://issues.apache.org/jira/browse/SPARK-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13978612#comment-13978612 ] Patrick Wendell commented on SPARK-1588: The approach in spark-submit is to capture SPARK_JAVA_OPTS and convert them into spark.executor.javaOptions and spark.driver.javaOptions - I think we can just do the same thing in the yarn client to provide backwards compatiblity. SPARK_JAVA_OPTS is not getting propagated - Key: SPARK-1588 URL: https://issues.apache.org/jira/browse/SPARK-1588 Project: Spark Issue Type: Bug Components: YARN Affects Versions: 1.0.0 Reporter: Mridul Muralidharan Priority: Blocker We could use SPARK_JAVA_OPTS to pass JAVA_OPTS to be used in the master. This is no longer working in current master. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (SPARK-1588) SPARK_JAVA_OPTS is not getting propagated
[ https://issues.apache.org/jira/browse/SPARK-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13978703#comment-13978703 ] Patrick Wendell commented on SPARK-1588: Btw - SPARK_JAVA_OPTS is officially dropped in 1.0 and there are loud warnings printed if it's used. Unfortunately this was a brute force mechanism used all over the place that had tons of corner cases and bugs. But I agree we should do our best to backwards support the old mechanisms. It's currently backwards-supported in the park-submit script but not the yarn client. SPARK_JAVA_OPTS is not getting propagated - Key: SPARK-1588 URL: https://issues.apache.org/jira/browse/SPARK-1588 Project: Spark Issue Type: Bug Components: YARN Affects Versions: 1.0.0 Reporter: Mridul Muralidharan Priority: Blocker We could use SPARK_JAVA_OPTS to pass JAVA_OPTS to be used in the master. This is no longer working in current master. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (SPARK-1588) SPARK_JAVA_OPTS is not getting propagated
[ https://issues.apache.org/jira/browse/SPARK-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13978827#comment-13978827 ] Mridul Muralidharan commented on SPARK-1588: Apparently, SPARK_YARN_USER_ENV is also broken SPARK_JAVA_OPTS is not getting propagated - Key: SPARK-1588 URL: https://issues.apache.org/jira/browse/SPARK-1588 Project: Spark Issue Type: Bug Components: YARN Affects Versions: 1.0.0 Reporter: Mridul Muralidharan Priority: Blocker We could use SPARK_JAVA_OPTS to pass JAVA_OPTS to be used in the master. This is no longer working in current master. -- This message was sent by Atlassian JIRA (v6.2#6252)
