Repository: incubator-zeppelin Updated Branches: refs/heads/master 98758d1cc -> e4ff4c035
ZEPPELIN-448 Update documentation for Spark configuration and BASE_PATH change https://issues.apache.org/jira/browse/ZEPPELIN-448 Addresses update documentation for Spark configuration. And BASE_PATH change after #430, #431 is deployed. (currently deployed website has this change applied) Author: Lee moon soo <[email protected]> Closes #448 from Leemoonsoo/update_install and squashes the following commits: 54e2edf [Lee moon soo] add newline cbd95d6 [Lee moon soo] Add description for included version of spark when SPARK_HOME is not set 00099ad [Lee moon soo] Add description for bumping up version cf99212 [Lee moon soo] Fix typo ab6d267 [Lee moon soo] Update BASE_PATH 73a737c [Lee moon soo] Remove instruction for 0.5.0 as zeppelin has documentation per version 4369f3e [Lee moon soo] Fix link dcdaa74 [Lee moon soo] Add more configurations possible 8c9361d [Lee moon soo] Change font style de74e6b [Lee moon soo] Update install and configuring spark Project: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/commit/e4ff4c03 Tree: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/tree/e4ff4c03 Diff: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/diff/e4ff4c03 Branch: refs/heads/master Commit: e4ff4c03536999d824c2f3b60859c156dc592a85 Parents: 98758d1 Author: Lee moon soo <[email protected]> Authored: Fri Nov 20 17:37:23 2015 +0900 Committer: Lee moon soo <[email protected]> Committed: Sat Nov 21 15:25:14 2015 +0900 ---------------------------------------------------------------------- docs/README.md | 11 +- docs/_config.yml | 2 +- docs/development/writingzeppelininterpreter.md | 2 +- docs/displaysystem/angular.md | 8 +- docs/displaysystem/display.md | 6 +- docs/displaysystem/table.md | 6 +- docs/docs.md | 8 +- docs/index.md | 19 +-- docs/install/install.md | 178 ++++++++++++++------ docs/interpreter/cassandra.md | 18 +- docs/interpreter/ignite.md | 10 +- docs/interpreter/lens.md | 10 +- docs/interpreter/spark.md | 60 +++++-- docs/manual/dynamicform.md | 14 +- docs/manual/interpreters.md | 6 +- docs/manual/notebookashomepage.md | 4 +- 16 files changed, 242 insertions(+), 120 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/e4ff4c03/docs/README.md ---------------------------------------------------------------------- diff --git a/docs/README.md b/docs/README.md index 100aacf..82a752f 100644 --- a/docs/README.md +++ b/docs/README.md @@ -3,7 +3,7 @@ This readme will walk you through building the Zeppelin documentation, which is included here with the Zeppelin source code. -## Build website +## Build documentation See https://help.github.com/articles/using-jekyll-with-pages#installing-jekyll **tl;dr version:** @@ -36,3 +36,12 @@ See https://help.github.com/articles/using-jekyll-with-pages#installing-jekyll ## Adding a new page rake page name="new-page.md" + + + +## Bumping up version + + * `BASE_PATH` property in _config.yml + * `ZEPPELIN <small>([VERSION])</small>` in _includes/themes/zeppelin/_navigation.html + +need to be updated http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/e4ff4c03/docs/_config.yml ---------------------------------------------------------------------- diff --git a/docs/_config.yml b/docs/_config.yml index ee0e727..bdba7a0 100644 --- a/docs/_config.yml +++ b/docs/_config.yml @@ -55,7 +55,7 @@ JB : # - Only the following values are falsy: ["", null, false] # - When setting BASE_PATH it must be a valid url. # This means always setting the protocol (http|https) or prefixing with "/" - BASE_PATH : false + BASE_PATH : /docs/0.6.0-incubating-SNAPSHOT # By default, the asset_path is automatically defined relative to BASE_PATH plus the enabled theme. # ex: [BASE_PATH]/assets/themes/[THEME-NAME] http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/e4ff4c03/docs/development/writingzeppelininterpreter.md ---------------------------------------------------------------------- diff --git a/docs/development/writingzeppelininterpreter.md b/docs/development/writingzeppelininterpreter.md index 4bb69b4..f64f092 100644 --- a/docs/development/writingzeppelininterpreter.md +++ b/docs/development/writingzeppelininterpreter.md @@ -25,7 +25,7 @@ Zeppelin Interpreter is a language backend. For example to use scala code in Zep Every Interpreter belongs to an InterpreterGroup. InterpreterGroup is a unit of start/stop interpreter. Interpreters in the same InterpreterGroup can reference each other. For example, SparkSqlInterpreter can reference SparkInterpreter to get SparkContext from it while they're in the same group. -<img class="img-responsive" style="width:50%; border: 1px solid #ecf0f1;" height="auto" src="../../assets/themes/zeppelin/img/interpreter.png" /> +<img class="img-responsive" style="width:50%; border: 1px solid #ecf0f1;" height="auto" src="/assets/themes/zeppelin/img/interpreter.png" /> All Interpreters in the same interpreter group are launched in a single, separate JVM process. The Interpreter communicates with Zeppelin engine via thrift. http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/e4ff4c03/docs/displaysystem/angular.md ---------------------------------------------------------------------- diff --git a/docs/displaysystem/angular.md b/docs/displaysystem/angular.md index 32e8253..c8a0ddc 100644 --- a/docs/displaysystem/angular.md +++ b/docs/displaysystem/angular.md @@ -32,7 +32,7 @@ Therefore, you can not only update scope variable from your interpreter but also #### Print AngularJS view To use angular display system, your output should starts with "%angular". -<img src="../../assets/themes/zeppelin/img/screenshots/display_angular.png" width=600px /> +<img src="/assets/themes/zeppelin/img/screenshots/display_angular.png" width=600px /> Note that display system is backend independent. @@ -62,7 +62,7 @@ z.angularUnbindGlobal(String name) In the example, let's bind "world" variable 'name'. Then you can see AngularJs view are updated immediately. -<img src="../../assets/themes/zeppelin/img/screenshots/display_angular1.png" width=600px /> +<img src="/assets/themes/zeppelin/img/screenshots/display_angular1.png" width=600px /> <br /> @@ -91,8 +91,8 @@ z.angularUnwatchGlobal(String name) Let's make an button, that increment 'run' variable by 1 when it is clicked. z.angularBind("run", 0) will initialize 'run' to zero. And then register watcher of 'run'. -<img src="../../assets/themes/zeppelin/img/screenshots/display_angular2.png" width=600px /> +<img src="/assets/themes/zeppelin/img/screenshots/display_angular2.png" width=600px /> After clicked button, you'll see both 'run' and numWatched are increased by 1 -<img src="../../assets/themes/zeppelin/img/screenshots/display_angular3.png" width=600px /> +<img src="/assets/themes/zeppelin/img/screenshots/display_angular3.png" width=600px /> http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/e4ff4c03/docs/displaysystem/display.md ---------------------------------------------------------------------- diff --git a/docs/displaysystem/display.md b/docs/displaysystem/display.md index 132e356..63cc1a7 100644 --- a/docs/displaysystem/display.md +++ b/docs/displaysystem/display.md @@ -27,11 +27,11 @@ limitations under the License. Zeppelin prints output of language backend in text, by default. -<img src="../../assets/themes/zeppelin/img/screenshots/display_text.png" /> +<img src="/assets/themes/zeppelin/img/screenshots/display_text.png" /> You can explicitly say you're using text display system. -<img src="../../assets/themes/zeppelin/img/screenshots/display_text1.png" /> +<img src="/assets/themes/zeppelin/img/screenshots/display_text1.png" /> Note that display system is backend independent. @@ -42,4 +42,4 @@ Note that display system is backend independent. With '%html' directive, Zeppelin treats your output as html -<img src="../../assets/themes/zeppelin/img/screenshots/display_html.png" /> +<img src="/assets/themes/zeppelin/img/screenshots/display_html.png" /> http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/e4ff4c03/docs/displaysystem/table.md ---------------------------------------------------------------------- diff --git a/docs/displaysystem/table.md b/docs/displaysystem/table.md index b1fe2af..085c857 100644 --- a/docs/displaysystem/table.md +++ b/docs/displaysystem/table.md @@ -24,14 +24,14 @@ limitations under the License. If you have data that row seprated by '\n' (newline) and column separated by '\t' (tab) with first row as header row, for example -<img src="../../assets/themes/zeppelin/img/screenshots/display_table.png" /> +<img src="/assets/themes/zeppelin/img/screenshots/display_table.png" /> You can simply use %table display system to leverage Zeppelin's built in visualization. -<img src="../../assets/themes/zeppelin/img/screenshots/display_table1.png" /> +<img src="/assets/themes/zeppelin/img/screenshots/display_table1.png" /> Note that display system is backend independent. If table contents start with %html, it is interpreted as an HTML. -<img src="../../assets/themes/zeppelin/img/screenshots/display_table_html.png" /> +<img src="/assets/themes/zeppelin/img/screenshots/display_table_html.png" /> http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/e4ff4c03/docs/docs.md ---------------------------------------------------------------------- diff --git a/docs/docs.md b/docs/docs.md index 9678641..28ee73a 100644 --- a/docs/docs.md +++ b/docs/docs.md @@ -35,14 +35,14 @@ limitations under the License. * [cassandra](./interpreter/cassandra.html) * [flink](./interpreter/flink.html) * [geode](./interpreter/geode.html) -* [hive](../pleasecontribute.html) +* [hive](./pleasecontribute.html) * [ignite](./interpreter/ignite.html) * [lens](./interpreter/lens.html) -* [md](../pleasecontribute.html) +* [md](./pleasecontribute.html) * [postgresql, hawq](./interpreter/postgresql.html) -* [sh](../pleasecontribute.html) +* [sh](./pleasecontribute.html) * [spark](./interpreter/spark.html) -* [tajo](../pleasecontribute.html) +* [tajo](./pleasecontribute.html) ### Storage * [S3 Storage](./storage/storage.html) http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/e4ff4c03/docs/index.md ---------------------------------------------------------------------- diff --git a/docs/index.md b/docs/index.md index 4343c64..6919c5d 100644 --- a/docs/index.md +++ b/docs/index.md @@ -34,7 +34,7 @@ limitations under the License. </ul> </div> - <div class="col-md-7"><img class="img-responsive" style="border: 1px solid #ecf0f1;" height="auto" src="assets/themes/zeppelin/img/notebook.png" /></div> + <div class="col-md-7"><img class="img-responsive" style="border: 1px solid #ecf0f1;" height="auto" src="/assets/themes/zeppelin/img/notebook.png" /></div> </div> @@ -44,17 +44,16 @@ limitations under the License. Zeppelin interpreter concept allows any language/data-processing-backend to be plugged into Zeppelin. Currently Zeppelin supports many interpreters such as Scala(with Apache Spark), Python(with Apache Spark), SparkSQL, Hive, Markdown and Shell. -<img class="img-responsive" src="assets/themes/zeppelin/img/screenshots/multiple_language_backend.png" /> +<img class="img-responsive" src="/assets/themes/zeppelin/img/screenshots/multiple_language_backend.png" /> Adding new language-backend is really simple. Learn [how to write a zeppelin interpreter](./development/writingzeppelininterpreter.html). - <br /> ### Apache Spark integration Zeppelin provides built-in Apache Spark integration. You don't need to build a separate module, plugin or library for it. -<img src="assets/themes/zeppelin/img/spark_logo.jpg" width="80px" /> +<img src="/assets/themes/zeppelin/img/spark_logo.jpg" width="80px" /> Zeppelin's Spark integration provides @@ -69,10 +68,10 @@ Some basic charts are already included in Zeppelin. Visualizations are not limit <div class="row"> <div class="col-md-6"> - <img class="img-responsive" src="./assets/themes/zeppelin/img/graph1.png" /> + <img class="img-responsive" src="/assets/themes/zeppelin/img/graph1.png" /> </div> <div class="col-md-6"> - <img class="img-responsive" src="./assets/themes/zeppelin/img/graph2.png" /> + <img class="img-responsive" src="/assets/themes/zeppelin/img/graph2.png" /> </div> </div> @@ -82,7 +81,7 @@ With simple drag and drop Zeppelin aggeregates the values and display them in pi <div class="row"> <div class="col-md-8"> - <img class="img-responsive" src="./assets/themes/zeppelin/img/screenshots/pivot.png" /> + <img class="img-responsive" src="/assets/themes/zeppelin/img/screenshots/pivot.png" /> </div> </div> Learn more about Zeppelin's Display system. ( [text](./displaysystem/display.html), [html](./displaysystem/display.html#html), [table](./displaysystem/table.html), [angular](./displaysystem/angular.html) ) @@ -93,7 +92,7 @@ Learn more about Zeppelin's Display system. ( [text](./displaysystem/display.htm Zeppelin can dynamically create some input forms into your notebook. -<img class="img-responsive" src="./assets/themes/zeppelin/img/screenshots/form_input.png" /> +<img class="img-responsive" src="/assets/themes/zeppelin/img/screenshots/form_input.png" /> Learn more about [Dynamic Forms](./manual/dynamicform.html). @@ -103,7 +102,7 @@ Learn more about [Dynamic Forms](./manual/dynamicform.html). Notebook URL can be shared among collaborators. Zeppelin can then broadcast any changes in realtime, just like the collaboration in Google docs. -<img src="./assets/themes/zeppelin/img/screenshots/collaboration.png" /> +<img src="/assets/themes/zeppelin/img/screenshots/collaboration.png" /> <br /> ### Publish @@ -112,7 +111,7 @@ Notebook URL can be shared among collaborators. Zeppelin can then broadcast any This way, you can easily embed it as an iframe inside of your website.</p> <div class="row"> - <img class="img-responsive center-block" src="./assets/themes/zeppelin/img/screenshots/publish.png" /> + <img class="img-responsive center-block" src="/assets/themes/zeppelin/img/screenshots/publish.png" /> </div> <br /> http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/e4ff4c03/docs/install/install.md ---------------------------------------------------------------------- diff --git a/docs/install/install.md b/docs/install/install.md index a4b3336..e1dc5d1 100644 --- a/docs/install/install.md +++ b/docs/install/install.md @@ -21,46 +21,14 @@ limitations under the License. -## Build +## From binary package -#### Prerequisites + Download latest binary package from [Download](../download.html). - * Java 1.7 - * None root account - * Apache Maven -Build tested on OSX, CentOS 6. - -Checkout source code from [https://github.com/apache/incubator-zeppelin](https://github.com/apache/incubator-zeppelin) - -#### Local mode - -``` -mvn install -DskipTests -``` - -#### Cluster mode - -``` -mvn install -DskipTests -Dspark.version=1.1.0 -Dhadoop.version=2.2.0 -``` - -Change spark.version and hadoop.version to your cluster's one. - -#### Custom built Spark - -Note that is you uses custom build spark, you need build Zeppelin with custome built spark artifact. To do that, deploy spark artifact to local maven repository using - -``` -sbt/sbt publish-local -``` - -and then build Zeppelin with your custom built Spark - -``` -mvn install -DskipTests -Dspark.version=1.1.0-Custom -Dhadoop.version=2.2.0 -``` +## Build from source + Check instructions in [README](https://github.com/apache/incubator-zeppelin/blob/master/README.md) to build from source. @@ -80,19 +48,135 @@ Configuration can be done by both environment variable(conf/zeppelin-env.sh) and <td>ZEPPELIN_PORT</td> <td>zeppelin.server.port</td> <td>8080</td> - <td>Zeppelin server port. Note that port+1 is used for web socket</td> + <td>Zeppelin server port.</td> + </tr> + <tr> + <td>ZEPPELIN_MEM</td> + <td>N/A</td> + <td>-Xmx1024m -XX:MaxPermSize=512m</td> + <td>JVM mem options</td> + </tr> + <tr> + <td>ZEPPELIN_INTP_MEM</td> + <td>N/A</td> + <td>ZEPPELIN_MEM</td> + <td>JVM mem options for interpreter process</td> + </tr> + <tr> + <td>ZEPPELIN_JAVA_OPTS</td> + <td>N/A</td> + <td></td> + <td>JVM Options</td> + </tr> + <tr> + <td>ZEPPELIN_ALLOWED_ORIGINS</td> + <td>zeppelin.server.allowed.origins</td> + <td>*</td> + <td>Allows a way to specify a ',' separated list of allowed origins for rest and websockets. i.e. http://localhost:8080</td> + </tr> + <tr> + <td>ZEPPELIN_SSL</td> + <td>zeppelin.ssl</td> + <td>false</td> + <td></td> + </tr> + <tr> + <td>ZEPPELIN_SSL_CLIENT_AUTH</td> + <td>zeppelin.ssl.client.auth</td> + <td>false</td> + <td></td> + </tr> + <tr> + <td>ZEPPELIN_SSL_KEYSTORE_PATH</td> + <td>zeppelin.ssl.keystore.path</td> + <td>keystore</td> + <td></td> + </tr> + <tr> + <td>ZEPPELIN_SSL_KEYSTORE_TYPE</td> + <td>zeppelin.ssl.keystore.type</td> + <td>JKS</td> + <td></td> + </tr> + <tr> + <td>ZEPPELIN_SSL_KEYSTORE_PASSWORD</td> + <td>zeppelin.ssl.keystore.password</td> + <td></td> + <td></td> + </tr> + <tr> + <td>ZEPPELIN_SSL_KEY_MANAGER_PASSWORD</td> + <td>zeppelin.ssl.key.manager.password</td> + <td></td> + <td></td> + </tr> + <tr> + <td>ZEPPELIN_SSL_TRUSTSTORE_PATH</td> + <td>zeppelin.ssl.truststore.path</td> + <td></td> + <td></td> + </tr> + <tr> + <td>ZEPPELIN_SSL_TRUSTSTORE_TYPE</td> + <td>zeppelin.ssl.truststore.type</td> + <td></td> + <td></td> + </tr> + <tr> + <td>ZEPPELIN_SSL_TRUSTSTORE_PASSWORD</td> + <td>zeppelin.ssl.truststore.password</td> + <td></td> + <td></td> + </tr> + <tr> + <td>ZEPPELIN_NOTEBOOK_HOMESCREEN</td> + <td>zeppelin.notebook.homescreen</td> + <td></td> + <td>Id of notebook to be displayed in homescreen ex) 2A94M5J1Z</td> </tr> <tr> + <td>ZEPPELIN_NOTEBOOK_HOMESCREEN_HIDE</td> + <td>zeppelin.notebook.homescreen.hide</td> + <td>false</td> + <td>hide homescreen notebook from list when this value set to "true"</td> + </tr> + <tr> <td>ZEPPELIN_NOTEBOOK_DIR</td> <td>zeppelin.notebook.dir</td> <td>notebook</td> <td>Where notebook file is saved</td> </tr> <tr> + <td>ZEPPELIN_NOTEBOOK_S3_BUCKET</td> + <td>zeppelin.notebook.s3.bucket</td> + <td>zeppelin</td> + <td>Bucket where notebook saved</td> + </tr> + <tr> + <td>ZEPPELIN_NOTEBOOK_S3_USER</td> + <td>zeppelin.notebook.s3.user</td> + <td>user</td> + <td>User in bucket where notebook saved. For example bucket/user/notebook/2A94M5J1Z/note.json</td> + </tr> + <tr> + <td>ZEPPELIN_NOTEBOOK_STORAGE</td> + <td>zeppelin.notebook.storage</td> + <td>org.apache.zeppelin.notebook.repo.VFSNotebookRepo</td> + <td>Comma separated list of notebook storage</td> + </tr> + <tr> + <td>ZEPPELIN_NOTEBOOK_RELOAD_FROM_STORAGE</td> + <td>zeppelin.notebook.reloadAllNotesFromStorage</td> + <td>false</td> + <td>Notebook list and contents will be always loaded from repository if set true. If set false, modified notebooks or new notebooks added on file system level won't be reflected on Zeppelin till user restarts Zeppelin.</td> + </tr> + <tr> <td>ZEPPELIN_INTERPRETERS</td> <td>zeppelin.interpreters</td> <description></description> - <td>org.apache.zeppelin.spark.SparkInterpreter,<br />org.apache.zeppelin.spark.PySparkInterpreter,<br />org.apache.zeppelin.spark.SparkSqlInterpreter,<br />org.apache.zeppelin.spark.DepInterpreter,<br />org.apache.zeppelin.markdown.Markdown,<br />org.apache.zeppelin.shell.ShellInterpreter,<br />org.apache.zeppelin.hive.HiveInterpreter</td> + <td>org.apache.zeppelin.spark.SparkInterpreter,<br />org.apache.zeppelin.spark.PySparkInterpreter,<br />org.apache.zeppelin.spark.SparkSqlInterpreter,<br />org.apache.zeppelin.spark.DepInterpreter,<br />org.apache.zeppelin.markdown.Markdown,<br />org.apache.zeppelin.shell.ShellInterpreter,<br />org.apache.zeppelin.hive.HiveInterpreter<br /> + ... + </td> <td>Comma separated interpreter configurations [Class]. First interpreter become a default</td> </tr> <tr> @@ -101,19 +185,14 @@ Configuration can be done by both environment variable(conf/zeppelin-env.sh) and <td>interpreter</td> <td>Zeppelin interpreter directory</td> </tr> - <tr> - <td>MASTER</td> - <td></td> - <td>N/A</td> - <td>Spark master url. eg. spark://master_addr:7077. Leave empty if you want to use local mode</td> - </tr> - <tr> - <td>ZEPPELIN_JAVA_OPTS</td> - <td></td> - <td>N/A</td> - <td>JVM Options</td> </table> +<br /> +You'll also need to configure individual interpreter. Information can be found in 'Interpreter' section in this documentation. + +For example [Spark](../interpreter/spark.html). + +<br /> ## Start/Stop #### Start Zeppelin @@ -121,7 +200,6 @@ Configuration can be done by both environment variable(conf/zeppelin-env.sh) and bin/zeppelin-daemon.sh start ``` After successful start, visit http://localhost:8080 with your web browser. -Note that port **8081** also need to be accessible for websocket connection. #### Stop Zeppelin http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/e4ff4c03/docs/interpreter/cassandra.md ---------------------------------------------------------------------- diff --git a/docs/interpreter/cassandra.md b/docs/interpreter/cassandra.md index b53295c..eded545 100644 --- a/docs/interpreter/cassandra.md +++ b/docs/interpreter/cassandra.md @@ -30,9 +30,9 @@ group: manual In a notebook, to enable the **Cassandra** interpreter, click on the **Gear** icon and select **Cassandra** <center> -  - -  +  + +  </center> <hr/> @@ -44,7 +44,7 @@ group: manual To access the interactive help, type **HELP;** <center> -  +  </center> <hr/> @@ -283,7 +283,7 @@ There is a drop-down menu on the top left corner to expand objects details. On t <br/> <center> -  +  </center> <hr/> @@ -551,7 +551,7 @@ For this, first go to the **Interpreter** menu and click on the **Create** butto <br/> <br/> <center> -  +  </center> In the interpreter creation form, put **cass-instance2** as **Name** and select the **cassandra** @@ -559,7 +559,7 @@ in the interpreter drop-down list <br/> <br/> <center> -  +  </center> Click on **Save** to create the new interpreter instance. Now you should be able to see it in the interpreter list. @@ -567,7 +567,7 @@ in the interpreter drop-down list <br/> <br/> <center> -  +  </center> Go back to your notebook and click on the **Gear** icon to configure interpreter bindings. @@ -577,7 +577,7 @@ interpreter list instead of the standard **cassandra** instance. <br/> <br/> <center> -  +  </center> <hr/> http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/e4ff4c03/docs/interpreter/ignite.md ---------------------------------------------------------------------- diff --git a/docs/interpreter/ignite.md b/docs/interpreter/ignite.md index 02fc587..963c7d9 100644 --- a/docs/interpreter/ignite.md +++ b/docs/interpreter/ignite.md @@ -11,7 +11,7 @@ group: manual ### Overview [Apache Ignite](https://ignite.apache.org/) In-Memory Data Fabric is a high-performance, integrated and distributed in-memory platform for computing and transacting on large-scale data sets in real-time, orders of magnitude faster than possible with traditional disk-based or flash technologies. - + You can use Zeppelin to retrieve distributed data from cache using Ignite SQL interpreter. Moreover, Ignite interpreter allows you to execute any Scala code in cases when SQL doesn't fit to your requirements. For example, you can populate data into your caches or execute distributed computations. @@ -68,12 +68,12 @@ At the "Interpreters" menu, you may edit Ignite interpreter or create new one. Z </tr> </table> - + ### Interpreter Binding for Zeppelin Notebook After configuring Ignite interpreter, create your own notebook. Then you can bind interpreters like below image. - + For more interpreter binding information see [here](http://zeppelin.incubator.apache.org/docs/manual/interpreters.html). @@ -87,7 +87,7 @@ For example, you can select top 10 words in the words cache using the following select _val, count(_val) as cnt from String group by _val order by cnt desc limit 10 ``` -  +  As long as your Ignite version and Zeppelin Ignite version is same, you can also use scala code. Please check the Zeppelin Ignite version before you download your own Ignite. @@ -109,7 +109,7 @@ As long as your Ignite version and Zeppelin Ignite version is same, you can also collectionAsScalaIterable(res).foreach(println _) ``` -  +  Apache Ignite also provides a guide docs for Zeppelin ["Ignite with Apache Zeppelin"](https://apacheignite.readme.io/docs/data-analysis-with-apache-zeppelin) http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/e4ff4c03/docs/interpreter/lens.md ---------------------------------------------------------------------- diff --git a/docs/interpreter/lens.md b/docs/interpreter/lens.md index 903df7e..a3eb284 100644 --- a/docs/interpreter/lens.md +++ b/docs/interpreter/lens.md @@ -11,7 +11,7 @@ group: manual ### Overview [Apache Lens](https://lens.apache.org/) provides an Unified Analytics interface. Lens aims to cut the Data Analytics silos by providing a single view of data across multiple tiered data stores and optimal execution environment for the analytical query. It seamlessly integrates Hadoop with traditional data warehouses to appear like one. - + ### Installing and Running Lens In order to use Lens interpreters, you may install Apache Lens in some simple steps: @@ -75,11 +75,11 @@ At the "Interpreters" menu, you can to edit Lens interpreter or create new one. </tr> </table> - + ### Interpreter Bindging for Zeppelin Notebook After configuring Lens interpreter, create your own notebook, then you can bind interpreters like below image. - + For more interpreter binding information see [here](http://zeppelin.incubator.apache.org/docs/manual/interpreters.html). @@ -159,14 +159,14 @@ As you can see in this video, they are using Lens Client Shell(./bin/lens-cli.sh ``` -  +  These are just examples that provided in advance by Lens. If you want to explore whole tutorials of Lens, see the [tutorial video](https://cwiki.apache.org/confluence/display/LENS/2015/07/13/20+Minute+video+demo+of+Apache+Lens+through+examples). ### Lens UI Service Lens also provides web UI service. Once the server starts up, you can open the service on http://serverhost:19999/index.html and browse. You may also check the structure that you made and use query easily here. -  +  http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/e4ff4c03/docs/interpreter/spark.md ---------------------------------------------------------------------- diff --git a/docs/interpreter/spark.md b/docs/interpreter/spark.md index 58fce0b..20be7f8 100644 --- a/docs/interpreter/spark.md +++ b/docs/interpreter/spark.md @@ -7,7 +7,7 @@ group: manual {% include JB/setup %} -## Spark +## Spark Interpreter [Apache Spark](http://spark.apache.org) is supported in Zeppelin with Spark Interpreter group, which consisted of 4 interpreters. @@ -41,10 +41,54 @@ Spark Interpreter group, which consisted of 4 interpreters. </table> +<br /><br /> + +### Configuration +<hr /> + +Without any configuration, Spark interpreter works out of box in local mode. But if you want to connect to your Spark cluster, you'll need following two simple steps. + + +#### 1. export SPARK_HOME + +In **conf/zeppelin-env.sh**, export SPARK_HOME environment variable with your Spark installation path. + +for example + +```bash +export SPARK_HOME=/usr/lib/spark +``` + +You can optionally export HADOOP\_CONF\_DIR and SPARK\_SUBMIT\_OPTIONS + +```bash +export HADOOP_CONF_DIR=/usr/lib/hadoop +export SPARK_SUBMIT_OPTIONS="--packages com.databricks:spark-csv_2.10:1.2.0" +``` + + <br /> +#### 2. set master in Interpreter menu. +After start Zeppelin, go to **Interpreter** menu and edit **master** property in your Spark interpreter setting. The value may vary depending on your Spark cluster deployment type. +for example, + + * **local[*]** in local mode + * **spark://master:7077** in standalone cluster + * **yarn-client** in Yarn client mode + * **mesos://host:5050** in Mesos cluster + + + +<br /> +That's it. Zeppelin will work with any version of Spark and any deployment type without rebuild Zeppelin in this way. (Zeppelin 0.5.5-incubating release works up to Spark 1.5.1) + +Note that without exporting SPARK_HOME, it's running in local mode with included version of Spark. The included version may vary depending on the build profile. + +<br /> <br /> ### SparkContext, SQLContext, ZeppelinContext +<hr /> SparkContext, SQLContext, ZeppelinContext are automatically created and exposed as variable names 'sc', 'sqlContext' and 'z', respectively, both in scala and python environments. @@ -55,6 +99,7 @@ Note that scala / python environment shares the same SparkContext, SQLContext, Z <br /> <br /> ### Dependency Management +<hr /> There are two ways to load external library in spark interpreter. First is using Zeppelin's %dep interpreter and second is loading Spark properties. #### 1. Dynamic Dependency Loading via %dep interpreter @@ -141,7 +186,6 @@ Note that adding jar to pyspark is only availabe via %dep interpreter at the mom <br/> Here are few examples: -##### 0.5.5 and later * SPARK\_SUBMIT\_OPTIONS in conf/zeppelin-env.sh export SPARK_SUBMIT_OPTIONS="--packages com.databricks:spark-csv_2.10:1.2.0 --jars /path/mylib1.jar,/path/mylib2.jar --files /path/mylib1.py,/path/mylib2.zip,/path/mylib3.egg" @@ -152,18 +196,10 @@ Here are few examples: spark.jars.packages com.databricks:spark-csv_2.10:1.2.0 spark.files /path/mylib1.py,/path/mylib2.egg,/path/mylib3.zip -##### 0.5.0 -* ZEPPELIN\_JAVA\_OPTS in conf/zeppelin-env.sh - - export ZEPPELIN_JAVA_OPTS="-Dspark.jars=/path/mylib1.jar,/path/mylib2.jar -Dspark.files=/path/myfile1.dat,/path/myfile2.dat" -<br /> - - -<a name="zeppelincontext"> </a> <br /> <br /> ### ZeppelinContext - +<hr /> Zeppelin automatically injects ZeppelinContext as variable 'z' in your scala/python environment. ZeppelinContext provides some additional functions and utility. @@ -218,4 +254,4 @@ In sql environment, you can create form in simple template. select * from ${table=defaultTableName} where text like '%${search}%' ``` -To learn more about dynamic form, checkout [Dynamic Form](../dynamicform.html). +To learn more about dynamic form, checkout [Dynamic Form](../manual/dynamicform.html). http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/e4ff4c03/docs/manual/dynamicform.md ---------------------------------------------------------------------- diff --git a/docs/manual/dynamicform.md b/docs/manual/dynamicform.md index dc46a5e..68aa7a1 100644 --- a/docs/manual/dynamicform.md +++ b/docs/manual/dynamicform.md @@ -37,12 +37,12 @@ To create text input form, use _${formName}_ templates. for example -<img src="../../assets/themes/zeppelin/img/screenshots/form_input.png" /> +<img src="/assets/themes/zeppelin/img/screenshots/form_input.png" /> Also you can provide default value, using _${formName=defaultValue}_. -<img src="../../assets/themes/zeppelin/img/screenshots/form_input_default.png" /> +<img src="/assets/themes/zeppelin/img/screenshots/form_input_default.png" /> <br /> @@ -52,11 +52,11 @@ To create select form, use _${formName=defaultValue,option1|option2...}_ for example -<img src="../../assets/themes/zeppelin/img/screenshots/form_select.png" /> +<img src="/assets/themes/zeppelin/img/screenshots/form_select.png" /> Also you can separate option's display name and value, using _${formName=defaultValue,option1(DisplayName)|option2(DisplayName)...}_ -<img src="../../assets/themes/zeppelin/img/screenshots/form_select_displayname.png" /> +<img src="/assets/themes/zeppelin/img/screenshots/form_select_displayname.png" /> <br /> ### Creates Programmatically @@ -79,7 +79,7 @@ Or Python print("Hello "+z.input("name")) ``` -<img src="../../assets/themes/zeppelin/img/screenshots/form_input_prog.png" /> +<img src="/assets/themes/zeppelin/img/screenshots/form_input_prog.png" /> Text input form with default value @@ -95,7 +95,7 @@ Python print("Hello "+z.input("name", "sun")) ``` -<img src="../../assets/themes/zeppelin/img/screenshots/form_input_default_prog.png" /> +<img src="/assets/themes/zeppelin/img/screenshots/form_input_default_prog.png" /> Select form @@ -123,4 +123,4 @@ print("Hello "+z.select("day", [("1","mon"), ("7","sun")])) ``` -<img src="../../assets/themes/zeppelin/img/screenshots/form_select_prog.png" /> +<img src="/assets/themes/zeppelin/img/screenshots/form_select_prog.png" /> http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/e4ff4c03/docs/manual/interpreters.md ---------------------------------------------------------------------- diff --git a/docs/manual/interpreters.md b/docs/manual/interpreters.md index ff5bff7..48cd1a3 100644 --- a/docs/manual/interpreters.md +++ b/docs/manual/interpreters.md @@ -32,13 +32,13 @@ Zeppelin Interpreter is the plug-in which enable zeppelin user to use a specific When you click on the ```+Create``` button in the interpreter page the interpreter drop-down list box will present all the available interpreters on your server. -<img src="../../assets/themes/zeppelin/img/screenshots/interpreter_create.png"> +<img src="/assets/themes/zeppelin/img/screenshots/interpreter_create.png"> ### What is zeppelin interpreter setting? Zeppelin interpreter setting is the configuration of a given interpreter on zeppelin server. For example, the properties requried for hive JDBC interpreter to connect to the Hive server. -<img src="../../assets/themes/zeppelin/img/screenshots/interpreter_setting.png"> +<img src="/assets/themes/zeppelin/img/screenshots/interpreter_setting.png"> ### What is zeppelin interpreter group? Every Interpreter belongs to an InterpreterGroup. InterpreterGroup is a unit of start/stop interpreter. @@ -48,7 +48,7 @@ SparkSQL and the dependency loader. Technically, Zeppelin interpreters from the same group are running in the same JVM. Interpreters belong to a single group a registered together and all of their properties are listed in the interpreter setting. -<img src="../../assets/themes/zeppelin/img/screenshots/interpreter_setting_spark.png"> +<img src="/assets/themes/zeppelin/img/screenshots/interpreter_setting_spark.png"> ### Programming langages for interpreter http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/e4ff4c03/docs/manual/notebookashomepage.md ---------------------------------------------------------------------- diff --git a/docs/manual/notebookashomepage.md b/docs/manual/notebookashomepage.md index 86f1ea9..3e43676 100644 --- a/docs/manual/notebookashomepage.md +++ b/docs/manual/notebookashomepage.md @@ -51,7 +51,7 @@ The process for creating your homepage is very simple as shown below: for example - <img src="../../assets/themes/zeppelin/img/screenshots/homepage_notebook_id.png" /> + <img src="/assets/themes/zeppelin/img/screenshots/homepage_notebook_id.png" /> Set the notebook id to the ```ZEPPELIN_NOTEBOOK_HOMESCREEN``` environment variable or ```zeppelin.notebook.homescreen``` property. @@ -97,7 +97,7 @@ you need to do is use our %angular support. ``` After running the notebook you will see output similar to this one: - <img src="../../assets/themes/zeppelin/img/screenshots/homepage_notebook_list.png" /> + <img src="/assets/themes/zeppelin/img/screenshots/homepage_notebook_list.png" /> The main trick here relays in linking the ```<div>``` to the controller:
