Repository: zeppelin Updated Branches: refs/heads/master 86ee813e4 -> 6c623d323
Copy-editing overview.md Extensive grammar revisions for zeppelin/docs/usage/interpreter/overview.md ### What is this PR for? The documentation at zeppelin/docs/usage/interpreter/overview.md contained numerous instances of bad grammar. This PR improves it. ### What type of PR is it? Documentation Author: Gerard de Melo <g...@demelo.org> Author: User <g...@demelo.org> Closes #3185 from gdemelo/patch-1 and squashes the following commits: b16ddf2 [User] Minor changes to wording 8b7773b [Gerard de Melo] Copy-editing overview.md Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/6c623d32 Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/6c623d32 Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/6c623d32 Branch: refs/heads/master Commit: 6c623d323fa04350d89b076cd2b3a760433370fc Parents: 86ee813 Author: Gerard de Melo <g...@demelo.org> Authored: Sun Sep 23 20:37:51 2018 -0400 Committer: Felix Cheung <felixche...@apache.org> Committed: Mon Sep 24 17:59:52 2018 -0700 ---------------------------------------------------------------------- docs/usage/interpreter/overview.md | 80 ++++++++++++++++----------------- 1 file changed, 39 insertions(+), 41 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/6c623d32/docs/usage/interpreter/overview.md ---------------------------------------------------------------------- diff --git a/docs/usage/interpreter/overview.md b/docs/usage/interpreter/overview.md index 5b567c7..6580a59 100644 --- a/docs/usage/interpreter/overview.md +++ b/docs/usage/interpreter/overview.md @@ -1,7 +1,7 @@ --- layout: page title: "Interpreter in Apache Zeppelin" -description: "This document explains about the role of interpreters, interpreters group and interpreter settings in Apache Zeppelin. The concept of Zeppelin interpreter allows any language/data-processing-backend to be plugged into Zeppelin." +description: "This document explains the role of interpreters, interpreter groups and interpreter settings in Apache Zeppelin. The concept of Zeppelin interpreters allows any language or data-processing backend to be plugged into Zeppelin." group: usage/interpreter --- <!-- @@ -25,25 +25,25 @@ limitations under the License. ## Overview -In this section, we will explain about the role of interpreters, interpreters group and interpreter settings in Zeppelin. -The concept of Zeppelin interpreter allows any language/data-processing-backend to be plugged into Zeppelin. -Currently, Zeppelin supports many interpreters such as Scala ( with Apache Spark ), Python ( with Apache Spark ), Spark SQL, JDBC, Markdown, Shell and so on. +In this section, we will explain the role of interpreters, interpreter groups and interpreter settings in Zeppelin. +The concept of Zeppelin interpreters allows any language or data-processing backend to be plugged into Zeppelin. +Currently, Zeppelin supports many interpreters such as Scala (with Apache Spark), Python (with Apache Spark), Spark SQL, JDBC, Markdown, Shell and so on. -## What is Zeppelin interpreter? -Zeppelin Interpreter is a plug-in which enables Zeppelin users to use a specific language/data-processing-backend. For example, to use Scala code in Zeppelin, you need `%spark` interpreter. +## What are Zeppelin interpreters? +A Zeppelin interpreter is a plug-in which enables Zeppelin users to use a specific language/data-processing-backend. For example, to use Scala code in Zeppelin, you would use the `%spark` interpreter. -When you click the ```+Create``` button in the interpreter page, the interpreter drop-down list box will show all the available interpreters on your server. +When you click the ```+Create``` button on the interpreter page, the interpreter drop-down list box will show all the available interpreters on your server. <img src="{{BASE_PATH}}/assets/themes/zeppelin/img/screenshots/interpreter_create.png" width="280px"> -## What is interpreter setting? -Zeppelin interpreter setting is the configuration of a given interpreter on Zeppelin server. For example, the properties are required for hive JDBC interpreter to connect to the Hive server. +## What are the interpreter settings? +The interpreter settings are the configuration of a given interpreter on the Zeppelin server. For example, certain properties need to be set for the Apache Hive JDBC interpreter to connect to the Hive server. <img src="{{BASE_PATH}}/assets/themes/zeppelin/img/screenshots/interpreter_setting.png" width="500px"> -Properties are exported as environment variables when property name is consisted of upper characters, numbers and underscore ([A-Z_0-9]). Otherwise set properties as JVM property. +Properties are exported as environment variables on the system if the property name consists of upper-case characters, numbers or underscores ([A-Z_0-9]). Otherwise, the property is set as a JVM property. -You may use parameters from the context of interpreter by add #{contextParameterName} in value, parameter can be of the following types: string, number, boolean. +You may use parameters from the context of the interpreter by adding #{contextParameterName} in the value. The parameter can be of the following types: string, number, boolean. ###### Context parameters <table class="table-configuration"> @@ -69,41 +69,41 @@ You may use parameters from the context of interpreter by add #{contextParameter </tr> </table> -If context parameter is null then replaced by empty string. +If the context parameter is null, then it is replaced by an empty string. <img src="{{BASE_PATH}}/assets/themes/zeppelin/img/screenshots/interpreter_setting_with_context_parameters.png" width="800px"> <br> -Each notebook can be bound to multiple Interpreter Settings using setting icon on upper right corner of the notebook. +Each notebook can be bound to multiple Interpreter Settings using the setting icon in the upper right corner of the notebook. <img src="{{BASE_PATH}}/assets/themes/zeppelin/img/screenshots/interpreter_binding.png" width="800px"> -## What is interpreter group? -Every Interpreter is belonged to an **Interpreter Group**. Interpreter Group is a unit of start/stop interpreter. -By default, every interpreter is belonged to a single group, but the group might contain more interpreters. For example, Spark interpreter group is including Spark support, pySpark, Spark SQL and the dependency loader. +## What are interpreter groups? +Every interpreter belongs to an **Interpreter Group**. Interpreter Groups are units of interpreters that can be started/stopped together. +By default, every interpreter belongs to a separate group, but the group might contain more interpreters. For example, the Spark interpreter group includes Spark support, pySpark, Spark SQL and the dependency loader. -Technically, Zeppelin interpreters from the same group are running in the same JVM. For more information about this, please checkout [here](../development/writing_zeppelin_interpreter.html). +Technically, Zeppelin interpreters from the same group run within the same JVM. For more information about this, please consult [the documentation on writing interpreters](../development/writing_zeppelin_interpreter.html). -Each interpreters is belonged to a single group and registered together. All of their properties are listed in the interpreter setting like below image. +Each interpreter belongs to a single group and is registered together. All relevant properties are listed in the interpreter setting as in the below example. <img src="{{BASE_PATH}}/assets/themes/zeppelin/img/screenshots/interpreter_setting_spark.png" width="500px"> ## Interpreter binding mode -Each Interpreter Setting can choose one of 'shared', 'scoped', 'isolated' interpreter binding mode. -In 'shared' mode, every notebook bound to the Interpreter Setting will share the single Interpreter instance. In 'scoped' mode, each notebook will create new Interpreter instance in the same interpreter process. In 'isolated' mode, each notebook will create new Interpreter process. +In the Interpreter Settings, one can choose one of the 'shared', 'scoped', or 'isolated' interpreter binding modes. +In 'shared' mode, every notebook bound to the Interpreter Setting will share a single Interpreter instance. In 'scoped' mode, each notebook will create a new interpreter instance in the same interpreter process. In 'isolated' mode, each notebook will create new a interpreter process. -For more information, check [Interpreter Binding Mode](./interpreter_binding_mode.html). +For more information, please consult [Interpreter Binding Mode](./interpreter_binding_mode.html). <img src="{{BASE_PATH}}/assets/themes/zeppelin/img/screenshots/interpreter_persession.png" width="400px"> ## Connecting to the existing remote interpreter -Zeppelin users can start interpreter thread embedded in their service. This will provide flexibility to user to start interpreter on remote host. To start interpreter along with your service you have to create an instance of ``RemoteInterpreterServer`` and start it as follows: +Zeppelin users can start interpreter threads embedded in their service. This provides users with the flexibility of starting interpreters on a remote host. To start an interpreter along with your service you have to create an instance of ``RemoteInterpreterServer`` and start it as follows: ```java RemoteInterpreterServer interpreter=new RemoteInterpreterServer(3678); @@ -112,43 +112,41 @@ interpreter.start(); ``` -The above code will start interpreter thread inside your process. Once the interpreter is started you can configure zeppelin to connect to RemoteInterpreter by checking **Connect to existing process** checkbox and then provide **Host** and **Port** on which interpreter process is listening as shown in the image below: +The above code will start an interpreter thread inside your process. Once the interpreter has started, you can configure Zeppelin to connect to RemoteInterpreter by checking the **Connect to existing process** checkbox and then provide the **Host** and **Port** on which interpreter process is listening, as shown in the image below: <img src="{{BASE_PATH}}/assets/themes/zeppelin/img/screenshots/existing_interpreter.png" width="450px"> ## Precode -Snippet of code (language of interpreter) that executes after initialization of the interpreter depends on [Binding mode](#interpreter-binding-mode). To configure add parameter with class of interpreter (`zeppelin.<ClassName>.precode`) except JDBCInterpreter ([JDBC precode](../../interpreter/jdbc.html#usage-precode)). +Snippet of code (language of interpreter) that executes after initialization of the interpreter depends on [Binding mode](#interpreter-binding-mode). To configure, add a parameter with the class of the interpreter (`zeppelin.<ClassName>.precode`) except JDBCInterpreter ([JDBC precode](../../interpreter/jdbc.html#usage-precode)). <img src="{{BASE_PATH}}/assets/themes/zeppelin/img/screenshots/interpreter_precode.png" width="800px"> ## Interpreter Lifecycle Management -Before 0.8.0, Zeppelin don't have lifecycle management on interpreter. User have to shutdown interpreters explicitly via UI. Starting from 0.8.0, Zeppelin provides a new interface -`LifecycleManager` to control the lifecycle of interpreters. For now, there're 2 implementations: `NullLifecycleManager` and `TimeoutLifecycleManager` which is default. +Before 0.8.0, Zeppelin didn't have lifecycle management for interpreters. Users had to shut down interpreters explicitly via the UI. Starting from 0.8.0, Zeppelin provides a new interface +`LifecycleManager` to control the lifecycle of interpreters. For now, there are two implementations: `NullLifecycleManager` and `TimeoutLifecycleManager`, which is the default. -`NullLifecycleManager` will do nothing, -user need to control the lifecycle of interpreter by themselves as before. `TimeoutLifecycleManager` will shutdown interpreters after interpreter idle for a while. By default, the idle threshold is 1 hour. -User can change it via `zeppelin.interpreter.lifecyclemanager.timeout.threshold`. `TimeoutLifecycleManager` is the default lifecycle manager, user can change it via `zeppelin.interpreter.lifecyclemanager.class`. +`NullLifecycleManager` will do nothing, i.e., the user needs to control the lifecycle of interpreter by themselves as before. `TimeoutLifecycleManager` will shut down interpreters after an interpreter remains idle for a while. By default, the idle threshold is 1 hour. +Users can change this threshold via the `zeppelin.interpreter.lifecyclemanager.timeout.threshold` setting. `TimeoutLifecycleManager` is the default lifecycle manager, and users can change it via `zeppelin.interpreter.lifecyclemanager.class`. ## Generic ConfInterpreter -Zeppelin's interpreter setting is shared by all users and notes, if you want to have different setting you have to create new interpreter, e.g. you can create `spark_jar1` for running spark with dependency jar1 and `spark_jar2` for running spark with dependency jar2. -This approach works, but not so convenient. `ConfInterpreter` can provide more fine-grained control on interpreter setting and more flexibility. +Zeppelin's interpreter setting is shared by all users and notes, if you want to have different settings, you have to create a new interpreter, e.g. you can create `spark_jar1` for running Spark with dependency jar1 and `spark_jar2` for running Spark with dependency jar2. +This approach works, but is not particularly convenient. `ConfInterpreter` can provide more fine-grained control on interpreter settings and more flexibility. -`ConfInterpreter` is a generic interpreter that could be used by any interpreters. The input format should be property file format. -It can be used to make custom setting for any interpreter. But it requires to run before interpreter process launched. And when interpreter process is launched is determined by interpreter mode setting. -So users needs to understand the ([interpreter mode setting ](../usage/interpreter/interpreter_bindings_mode.html) of Zeppelin and be aware when interpreter process is launched. E.g. If we set spark interpreter setting as isolated per note. Under this setting, each note will launch one interpreter process. -In this scenario, user need to put `ConfInterpreter` as the first paragraph as the below example. Otherwise the customized setting can not be applied (Actually it would report ERROR) -<img src="{{BASE_PATH}}/assets/themes/zeppelin/img/screenshots/conf_interpreter.png" width="500px"> +`ConfInterpreter` is a generic interpreter that can be used by any interpreter. The input format should be the property file format. +It can be used to make custom settings for any interpreter. However, `ConfInterpreter` needs to be run before that interpreter process is launched. When that interpreter process is launched is determined by the interpreter mode setting. +So users need to understand the ([interpreter mode setting ](../usage/interpreter/interpreter_bindings_mode.html) of Zeppelin and be aware of when the interpreter process is launched. E.g., if we set the Spark interpreter setting as isolated per note, then, under this setting, each note will launch one interpreter process. +In this scenario, users need to put `ConfInterpreter` as the first paragraph as in the below example. Otherwise, the customized setting cannot be applied (actually it would report ERROR) +<img src="{{BASE_PATH}}/assets/themes/zeppelin/img/screenshots/conf_interpreter.png" width="700px"> ## Interpreter Process Recovery -Before 0.8.0, shutting down Zeppelin also mean to shutdown all the running interpreter processes. Usually admin will shutdown Zeppelin server for maintenance or upgrade, but don't want to shut down the running interpreter processes. -In such cases, interpreter process recovery is necessary. Starting from 0.8.0, user can enable interpreter process recovering via setting `zeppelin.recovery.storage.class` as -`org.apache.zeppelin.interpreter.recovery.FileSystemRecoveryStorage` or other implementations if available in future, by default it is `org.apache.zeppelin.interpreter.recovery.NullRecoveryStorage` - which means recovery is not enabled. Enable recover means shutting down Zeppelin would not terminating interpreter process, -and when Zeppelin is restarted, it would try to reconnect to the existing running interpreter processes. If you want to kill all the interpreter processes after terminating Zeppelin even when recovery is enabled, you can run `bin/stop-interpreter.sh` +Before 0.8.0, shutting down Zeppelin also meant to shutdown all the running interpreter processes. Usually, an administrator will shutdown the Zeppelin server for maintenance or upgrades, but would not want to shut down the running interpreter processes. +In such cases, interpreter process recovery is necessary. Starting from 0.8.0, users can enable interpreter process recovery via the setting `zeppelin.recovery.storage.class` as +`org.apache.zeppelin.interpreter.recovery.FileSystemRecoveryStorage` or other implementations if available in the future. By default it is `org.apache.zeppelin.interpreter.recovery.NullRecoveryStorage`, + which means recovery is not enabled. Enabling recovery means shutting down Zeppelin would not terminate interpreter processes, and when Zeppelin is restarted, it would try to reconnect to the existing running interpreter processes. If you want to kill all the interpreter processes after terminating Zeppelin even when recovery is enabled, you can run `bin/stop-interpreter.sh`
