Repository: zeppelin Updated Branches: refs/heads/master 8b40268d1 -> b619699d1
[ZEPPELIN-1258] Add Spark packages support to Livy interpreter ### What is this PR for? Adding extra libraries to livy interpreter which isn't exist by default. ### What type of PR is it? [ Improvement ] ### Todos * [Test case ] - Task ### What is the Jira issue? * [ZEPPELIN-1258] ### How should this be tested? - Create new livy interpreter or modify the default. - Set `livy.spark.jars.packages` to list of maven coordinates of jars. The format for the coordinates should be groupId:artifactId:version. ### Screenshots (if appropriate)   ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: mahmoudelgamal <mahmoudf.elga...@gmail.com> Author: mfelgamal <mahmoudf.elga...@gmail.com> Author: Fouad <fuad.assay...@gmail.com> Closes #1255 from mfelgamal/feature/modifying-livy-doc and squashes the following commits: 5411a2f [mahmoudelgamal] changeing default to example d7f6818 [mahmoudelgamal] Adding livy.spark.jars.packages property 98090ab [mahmoudelgamal] Adding external libraries to livy 03a1e80 [mfelgamal] Merge pull request #4 from apache/master 2586651 [Fouad] Merge pull request #2 from apache/master Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/b619699d Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/b619699d Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/b619699d Branch: refs/heads/master Commit: b619699d186fba6a8d81896d7212b4cfeac50024 Parents: 8b40268 Author: mahmoudelgamal <mahmoudf.elga...@gmail.com> Authored: Mon Aug 8 10:42:31 2016 +0200 Committer: Felix Cheung <felixche...@apache.org> Committed: Mon Aug 15 13:53:56 2016 -0700 ---------------------------------------------------------------------- docs/interpreter/livy.md | 25 +++++++++++++++++++- .../src/main/resources/interpreter-setting.json | 5 ++++ 2 files changed, 29 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/b619699d/docs/interpreter/livy.md ---------------------------------------------------------------------- diff --git a/docs/interpreter/livy.md b/docs/interpreter/livy.md index a30ba7b..1c040b9 100644 --- a/docs/interpreter/livy.md +++ b/docs/interpreter/livy.md @@ -43,7 +43,7 @@ We added some common configurations for spark, and you can set any configuration This link contains all spark configurations: http://spark.apache.org/docs/latest/configuration.html#available-properties. And instead of starting property with `spark.` it should be replaced with `livy.spark.`. Example: `spark.master` to `livy.spark.master` - + <table class="table-configuration"> <tr> <th>Property</th> @@ -115,8 +115,31 @@ Example: `spark.master` to `livy.spark.master` <td></td> <td>Upper bound for the number of executors.</td> </tr> + <tr> + <td>livy.spark.jars.packages</td> + <td></td> + <td>Adding extra libraries to livy interpreter</td> + </tr> </table> +## Adding External libraries +You can load dynamic library to livy interpreter by set `livy.spark.jars.packages` property to comma-separated list of maven coordinates of jars to include on the driver and executor classpaths. The format for the coordinates should be groupId:artifactId:version. + +Example + +<table class="table-configuration"> + <tr> + <th>Property</th> + <th>Example</th> + <th>Description</th> + </tr> + <tr> + <td>livy.spark.jars.packages</td> + <td>io.spray:spray-json_2.10:1.3.1</td> + <td>Adding extra libraries to livy interpreter</td> + </tr> + </table> + ## How to use Basically, you can use http://git-wip-us.apache.org/repos/asf/zeppelin/blob/b619699d/livy/src/main/resources/interpreter-setting.json ---------------------------------------------------------------------- diff --git a/livy/src/main/resources/interpreter-setting.json b/livy/src/main/resources/interpreter-setting.json index 28fb280..bc5a275 100644 --- a/livy/src/main/resources/interpreter-setting.json +++ b/livy/src/main/resources/interpreter-setting.json @@ -81,6 +81,11 @@ "propertyName": "zeppelin.livy.keytab", "defaultValue": "", "description": "Kerberos keytab to authenticate livy" + }, + "livy.spark.jars.packages": { + "propertyName": "livy.spark.jars.packages", + "defaultValue": "", + "description": "Adding extra libraries to livy interpreter" } } },
