Repository: incubator-zeppelin Updated Branches: refs/heads/master cd031aad8 -> 0cfc84fa6
[Zeppelin 546](documentation) Enables interpreter library loading from maven repository ### What is this PR for? This PR is related to #590. This documentation may explain the overall process of **Dynamic Interpreter Loading** in [Helium Proposal](https://issues.apache.org/jira/browse/ZEPPELIN-533). Moreover, at the last this documentation, Zeppelin users can get the basic information about step by step of interpreter setting & configuring & binding. Since I assumed that they are novice at Zeppelin. ### What type of PR is it? Documentation ### Todos * [x] - Add docs image under `/docs/assets/themes/zeppelin/img/docs-img/` * [x] - Add dynamicinterpreter.md * [x] - Add dynamicinterpreter.html location to `docs/_includes/themes/zeppelin/_navigation.html` ### Is there a relevant Jira issue? 1. [ZEPPELIN-533 Helium](https://issues.apache.org/jira/browse/ZEPPELIN-533) 2. [ZEPPELIN-546](https://issues.apache.org/jira/browse/ZEPPELIN-546) ### How should this be tested? I add a link of this documentation to `interpreter tab` in Zeppelin web page.  ### Screenshots (if appropriate) Hopefully, below image helps you to understand this process : )  ### Questions: * Does the licenses files need update? No. * Is there breaking changes for older versions? No. * Does this needs documentation? No. Author: Ryu Ah young <[email protected]> Closes #609 from AhyoungRyu/ZEPPELIN-546-docs and squashes the following commits: d0675e0 [Ryu Ah young] ZEPPELIN-546-docs: Fix some sentences 266dac0 [Ryu Ah young] ZEPPELIN-546-docs: Add content for licence e180a8e [Ryu Ah young] ZEPPELIN-546-docs: Fix typo again d3cc50f [Ryu Ah young] ZEPPELIN-546-docs: Fix typo a9ec7d7 [Ryu Ah young] ZEPPELIN-546-docs: Add dynamicinterpreterload.html location to _navigation.html 16b46f4 [Ryu Ah young] ZEPPELIN-546-docs: Add dynamicinterpreterload.md 3067025 [Ryu Ah young] ZEPPELIN-546-docs: Add docs image Project: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/commit/0cfc84fa Tree: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/tree/0cfc84fa Diff: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/diff/0cfc84fa Branch: refs/heads/master Commit: 0cfc84fa6fea886d64e9cdf5c02850a209f23138 Parents: cd031aa Author: Ryu Ah young <[email protected]> Authored: Tue Jan 12 22:00:48 2016 -0800 Committer: Damien CORNEAU <[email protected]> Committed: Mon Jan 18 10:17:24 2016 +0900 ---------------------------------------------------------------------- docs/_includes/themes/zeppelin/_navigation.html | 1 + .../img/docs-img/interpreter_binding_1.png | Bin 0 -> 329451 bytes .../img/docs-img/interpreter_binding_2.png | Bin 0 -> 305356 bytes .../img/docs-img/interpreter_setting_1.png | Bin 0 -> 405396 bytes .../img/docs-img/interpreter_setting_2.png | Bin 0 -> 149752 bytes .../img/docs-img/interpreter_setting_3.png | Bin 0 -> 263085 bytes .../zeppelin/img/docs-img/zeppelin_user.png | Bin 0 -> 332805 bytes docs/manual/dynamicinterpreterload.md | 124 +++++++++++++++++++ 8 files changed, 125 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/0cfc84fa/docs/_includes/themes/zeppelin/_navigation.html ---------------------------------------------------------------------- diff --git a/docs/_includes/themes/zeppelin/_navigation.html b/docs/_includes/themes/zeppelin/_navigation.html index f3cd406..62db605 100644 --- a/docs/_includes/themes/zeppelin/_navigation.html +++ b/docs/_includes/themes/zeppelin/_navigation.html @@ -37,6 +37,7 @@ <a href="#" data-toggle="dropdown" class="dropdown-toggle">Interpreter <b class="caret"></b></a> <ul class="dropdown-menu"> <li><a href="{{BASE_PATH}}/manual/interpreters.html">Overview</a></li> + <li><a href="{{BASE_PATH}}/manual/dynamicinterpreterload.html">Dynamic Interpreter Loading</a></li> <li role="separator" class="divider"></li> <li><a href="{{BASE_PATH}}/interpreter/cassandra.html">Cassandra</a></li> <li><a href="{{BASE_PATH}}/interpreter/elasticsearch.html">Elasticsearch</a></li> http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/0cfc84fa/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_1.png ---------------------------------------------------------------------- diff --git a/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_1.png b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_1.png new file mode 100644 index 0000000..ce092a5 Binary files /dev/null and b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_1.png differ http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/0cfc84fa/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_2.png ---------------------------------------------------------------------- diff --git a/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_2.png b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_2.png new file mode 100644 index 0000000..0c6b993 Binary files /dev/null and b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_2.png differ http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/0cfc84fa/docs/assets/themes/zeppelin/img/docs-img/interpreter_setting_1.png ---------------------------------------------------------------------- diff --git a/docs/assets/themes/zeppelin/img/docs-img/interpreter_setting_1.png b/docs/assets/themes/zeppelin/img/docs-img/interpreter_setting_1.png new file mode 100644 index 0000000..d60d0ef Binary files /dev/null and b/docs/assets/themes/zeppelin/img/docs-img/interpreter_setting_1.png differ http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/0cfc84fa/docs/assets/themes/zeppelin/img/docs-img/interpreter_setting_2.png ---------------------------------------------------------------------- diff --git a/docs/assets/themes/zeppelin/img/docs-img/interpreter_setting_2.png b/docs/assets/themes/zeppelin/img/docs-img/interpreter_setting_2.png new file mode 100644 index 0000000..5c583c3 Binary files /dev/null and b/docs/assets/themes/zeppelin/img/docs-img/interpreter_setting_2.png differ http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/0cfc84fa/docs/assets/themes/zeppelin/img/docs-img/interpreter_setting_3.png ---------------------------------------------------------------------- diff --git a/docs/assets/themes/zeppelin/img/docs-img/interpreter_setting_3.png b/docs/assets/themes/zeppelin/img/docs-img/interpreter_setting_3.png new file mode 100644 index 0000000..bb6752d Binary files /dev/null and b/docs/assets/themes/zeppelin/img/docs-img/interpreter_setting_3.png differ http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/0cfc84fa/docs/assets/themes/zeppelin/img/docs-img/zeppelin_user.png ---------------------------------------------------------------------- diff --git a/docs/assets/themes/zeppelin/img/docs-img/zeppelin_user.png b/docs/assets/themes/zeppelin/img/docs-img/zeppelin_user.png new file mode 100644 index 0000000..7ea9e05 Binary files /dev/null and b/docs/assets/themes/zeppelin/img/docs-img/zeppelin_user.png differ http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/0cfc84fa/docs/manual/dynamicinterpreterload.md ---------------------------------------------------------------------- diff --git a/docs/manual/dynamicinterpreterload.md b/docs/manual/dynamicinterpreterload.md new file mode 100644 index 0000000..5ed55cf --- /dev/null +++ b/docs/manual/dynamicinterpreterload.md @@ -0,0 +1,124 @@ +--- +layout: page +title: "Dynamic Interpreter Loading" +description: "" +group: manual +--- +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> +{% include JB/setup %} + +## Dynamic Interpreter Loading using REST API + +Zeppelin provides pluggable interpreter architecture which results in a wide and variety of the supported backend system. In this section, we will introduce **Dynamic interpreter loading** using **REST API**. This concept actually comes from [Zeppelin Helium Proposal](https://cwiki.apache.org/confluence/display/ZEPPELIN/Helium+proposal). +Before we start, if you are not familiar with the concept of **Zeppelin interpreter**, you can check out [Overview of Zeppelin interpreter](../manual/interpreters.html) first. + +<br/> +## Overview +In the past, Zeppelin was loading interpreter binaries from `/interpreter/[interpreter_name]` directory. They were configured by `zeppelin.interpreters` property in `conf/zeppelin-site.xml` or `ZEPPELIN_INTERPRETERS` env variables in `conf/zeppelin-env.sh`. They were loaded on Zeppelin server startup and stayed alive until the server was stopped. +In order to simplify using 3rd party interpreters, we changed this way to **dynamically** load interpreters from **Maven Repository** using **REST API**. Hopefully, the picture below will help you to understand the process. +<center><img src="../assets/themes/zeppelin/img/docs-img/zeppelin_user.png" height="85%" width="85%"></center> + +## Load & Unload Interpreters Using REST API + +### 1. Load +You can **load** interpreters located in Maven repository using REST API, like this: + +( Maybe, you are unfamiliar with `[interpreter_group_name]` or `[interpreter_name]`. If so, please checkout [Interpreters in Zeppelin](../manual/interpreter.html) again. ) + +``` +http://[zeppelin-server]:[zeppelin-port]/api/interpreter/load/[interpreter_group_name]/[interpreter_name] +``` +The Restful method will be <code>**POST**</code>. And the parameters you need are: + + 1. **Artifact:** Maven artifact ( groupId:artifactId:version ) + + 2. **Class Name:** Package name + Interpreter class name + + 3. **Repository ( optional ):** Additional maven repository address + +For example, if you want to load `markdown` interpreter to your Zeppelin, the parameters and URL you need may look like: + +``` +http://127.0.0.1:8080/api/interpreter/load/md/markdown +``` + +``` +{ + "artifact": "org.apache.zeppelin:zeppelin-markdown:0.6.0-incubating-SNAPSHOT", + "className": "org.apache.zeppelin.markdown.Markdown", + "repository": { + "url": "http://dl.bintray.com/spark-packages/maven";, + "snapshot": false + } +} +``` + + +The meaning of each parameters is: + + 1. **Artifact** + - groupId: org.apache.zeppelin + - artifactId: zeppelin-markdown + - version: 0.6.0-incubating-SNAPSHOT + + 2. **Class Name** + - Package Name: org.apache.zeppelin + - Interpreter Class Name: markdown.Markdown + + 3. **Repository ( optional )** + - Url: http://dl.bintray.com/spark-packages/maven + - Snapshot: false + +> <b>Please note: </b>The interpreters you downloaded need to be **reload**, when your Zeppelin server is down. + +### 2. Unload +If you want to **unload** the interpreters using REST API, + +``` +http://[zeppelin-server]:[zeppelin-port]/api/interpreter/unload/[interpreter_group_name]/[interpreter_name] +``` +In this case, the Restful method will be <code>**DELETE**</code>. + +<br/> +## What is the next step after Loading ? + +### Q1. Where is the location of interpreters you downloaded ? + +Actually, the answer about this question is in the above picture. Once the REST API is called, the `.jar` files of interpreters you get are saved under `ZEPPELIN_HOME/local-repo` first. Then, they will be copied to `ZEPPELIN_HOME/interpreter` directory. So, please checkout your `ZEPPELIN_HOME/interpreter`. + +### Q2. Then, how can I use this interpreter ? + +After loading an interpreter, you can use it by creating and configuring it in Zeppelin's **Interpreter tab**. + +Oh, you don't need to restart your Zeppelin server. Because it is **Dynamic Loading**, you can configure and load it **at runtime** ! + +1. After Zeppelin server up, browse Zeppelin home and click **Interpreter tab**. +<center><img src="../assets/themes/zeppelin/img/docs-img/interpreter_setting_1.png" height="85%" width="85%"></center> + +2. At the **Interpreter** section, click **+Create** button. +<center><img src="../assets/themes/zeppelin/img/docs-img/interpreter_setting_2.png" height="85%" width="85%"></center> + +3. Then, you can verify the interpreter list that you loaded. +<center><img src="../assets/themes/zeppelin/img/docs-img/interpreter_setting_3.png" height="85%" width="85%"></center> + +4. After choosing an interpreter, you can configure and use it. Don't forget to save it. + +5. Create a new notebook in the **Notebook** section, then you can bind the interpreters from your interpreter list. Just drag and drop ! +<center><img src="../assets/themes/zeppelin/img/docs-img/interpreter_binding_1.png" height="85%" width="85%"></center> +<center><img src="../assets/themes/zeppelin/img/docs-img/interpreter_binding_2.png" height="85%" width="85%"></center> + +6. At last, you can use your interpreter ! + +If you want to get the specific information about respective interpreters, please checkout each interpreter documentation.
