GitHub user zjffdu opened a pull request:
https://github.com/apache/zeppelin/pull/2577
ZEPPELIN-2898. Support Yarn-Cluster for Spark Interpreter
### What is this PR for?
This is the first version for supporting yarn-cluster of
`SparkInterpreter`. I just delegate all the function to `spark-submit` as
yarn-cluster is natively supported by spark, we don't need to reinvent the
wheel. But there's still improvement to be done in future, such as I put some
spark specific login in `InterpreterSetting` which is not a good practise. I
plan to improve it when I refactor the `Interpreter` class (ZEPPELIN-2685).
Besides that, I also add `MiniHadoopCluster` & `MiniZeppelin` which help
for the integration test of yarn-client & yarn-cluster mode, otherwise I have
to manually verify yarn-client & yarn-cluster mode which would easily cause
regression issue in future.
### What type of PR is it?
[Feature]
### Todos
* [ ] - Task
### What is the Jira issue?
https://github.com/zjffdu/zeppelin/tree/ZEPPELIN-2898
### How should this be tested?
System test is added in `SparkInterpreterIT`.
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-2898
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/2577.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2577
----
commit 9ec46d931149eddc0a227cec00e1bee76180176d
Author: Jeff Zhang <[email protected]>
Date: 2017-09-04T13:54:56Z
ZEPPELIN-2898. Support Yarn-Cluster for Spark Interpreter
----
---
