GitHub user andreweduffy opened a pull request:
https://github.com/apache/spark/pull/13953
[SPARK-16265][SUBMIT] Addition of --driver-jre Flag to SparkSubmit
## What changes were proposed in this pull request?
1. Addition of `--driver-jre` flag to `SparkSubmitArguments`
2. Now when `--driver-jre` is specified to `SparkSubmit` and it is
connecting to a YARN master, the driver will zip up a copy of its in-use JRE
and ship it to the YARN cluster as part of `spark.yarn.dist.archives`. Then by
setting `spark.executorEnv.JAVA_HOME` and `spark.appMasterEnv.JAVA_HOME` to
point to their local copies, use of specific versions of Java can be had even
on YARN clusters that do not have those versions pre-installed.
## How was this patch tested?
Compiled the Spark examples with Java 8, did a Java 8 `spark-submit
--driver-jre` of `SparkPi` to a Java 7 YARN cluster, executors and AM used
Java 8 without issue. Running without shipping the JRE results in errors due to
Java version differences.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/andreweduffy/spark master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/13953.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13953
----
commit abd80d1bacd66eea6dfeb33fde3caa34bd7c9c7d
Author: Andrew Duffy <r...@aduffy.org>
Date: 2016-06-28T10:44:46Z
Addition of --driver-jre Flag to SparkSubmit
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
