[zeppelin] branch master updated: [ZEPPELIN-5273] Support Spark 3.1.1
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new a8f7657 [ZEPPELIN-5273] Support Spark 3.1.1
a8f7657 is described below
commit a8f7657d0f663db155b62bc75ca2fb564379c108
Author: Lee moon soo
AuthorDate: Mon Mar 22 09:01:09 2021 -0700
[ZEPPELIN-5273] Support Spark 3.1.1
### What is this PR for?
This PR makes Zeppelin supports Spark 3.1.1.
In my quick test, the current Spark interpreter seems to work fine with
Spark 3.1.1 without modification, when the version check passes. So, this PR
simply update `UNSUPPORTED_FUTURE_VERSION` to Spark `3.2.0` from `3.1.0`.
However, if anyone sees a problem with Spark 3.1.1, please comment.
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-5273
https://issues.apache.org/jira/browse/ZEPPELIN-5278
### How should this be tested?
Configure SPARK_HOME env variable to point Spark 3.1.1 binary and run Spark
APIs, such as
```
%spark
println(sc.version)
sc.parallelize(1 to 10).count
```
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #4077 from Leemoonsoo/ZEPPELIN-5273 and squashes the following
commits:
0b82eded2 [Lee moon soo] add spark*Test31 file, update spark test version
3.0.1->3.0.2, 2.4.4->2.4.7, remove spark test versions not available on spark
download page
6252d3623 [Lee moon soo] 3.0.1 -> 3.1.1
5676f0923 [Lee moon soo] update unsupported future version
---
spark/pom.xml | 2 +-
.../org/apache/zeppelin/spark/SparkVersion.java| 3 +-
spark/spark3-shims/pom.xml | 2 +-
.../integration/SparkIntegrationTest16.java| 40 --
.../integration/SparkIntegrationTest20.java| 40 --
.../integration/SparkIntegrationTest22.java| 40 --
.../integration/SparkIntegrationTest23.java| 40 --
.../integration/SparkIntegrationTest24.java| 2 +-
.../integration/SparkIntegrationTest30.java| 4 +--
...tionTest21.java => SparkIntegrationTest31.java} | 23 +++--
.../integration/ZeppelinSparkClusterTest20.java| 40 --
.../integration/ZeppelinSparkClusterTest21.java| 40 --
.../integration/ZeppelinSparkClusterTest22.java| 40 --
.../integration/ZeppelinSparkClusterTest23.java| 40 --
.../integration/ZeppelinSparkClusterTest24.java| 2 +-
.../integration/ZeppelinSparkClusterTest30.java| 4 +--
...Test16.java => ZeppelinSparkClusterTest31.java} | 8 ++---
17 files changed, 34 insertions(+), 336 deletions(-)
diff --git a/spark/pom.xml b/spark/pom.xml
index 38c2a24..ac4a2dc 100644
--- a/spark/pom.xml
+++ b/spark/pom.xml
@@ -187,7 +187,7 @@
spark-3.0
-3.0.1
+3.1.1
2.5.0
0.10.9
diff --git
a/spark/spark-shims/src/main/scala/org/apache/zeppelin/spark/SparkVersion.java
b/spark/spark-shims/src/main/scala/org/apache/zeppelin/spark/SparkVersion.java
index ec80740..0ad1a35 100644
---
a/spark/spark-shims/src/main/scala/org/apache/zeppelin/spark/SparkVersion.java
+++
b/spark/spark-shims/src/main/scala/org/apache/zeppelin/spark/SparkVersion.java
@@ -33,9 +33,10 @@ public class SparkVersion {
public static final SparkVersion SPARK_2_3_1 =
SparkVersion.fromVersionString("2.3.1");
public static final SparkVersion SPARK_2_4_0 =
SparkVersion.fromVersionString("2.4.0");
public static final SparkVersion SPARK_3_1_0 =
SparkVersion.fromVersionString("3.1.0");
+ public static final SparkVersion SPARK_3_2_0 =
SparkVersion.fromVersionString("3.2.0");
public static final SparkVersion MIN_SUPPORTED_VERSION = SPARK_1_6_0;
- public static final SparkVersion UNSUPPORTED_FUTURE_VERSION = SPARK_3_1_0;
+ public static final SparkVersion UNSUPPORTED_FUTURE_VERSION = SPARK_3_2_0;
private int version;
private int majorVersion;
diff --git a/spark/spark3-shims/pom.xml b/spark/spark3-shims/pom.xml
index c9a4f35..eb5f5b6 100644
--- a/spark/spark3-shims/pom.xml
+++ b/spark/spark3-shims/pom.xml
@@ -34,7 +34,7 @@
2.12
-3.0.1
+3.1.1
diff --git
a/zeppelin-interpreter-integration/src/test/java/org/apache/zeppelin/integration/SparkIntegrationTest16.java
b/zeppelin-interpreter-integration/src/test/java/org/apache/zeppelin/integration/SparkIntegrationTest
[zeppelin] branch master updated: [ZEPPELIN-5167] Set HOME env variable to in Dockerfile, so Helium can install package successfully
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new f1db722 [ZEPPELIN-5167] Set HOME env variable to in Dockerfile, so
Helium can install package successfully
f1db722 is described below
commit f1db72297b373c856dcb149c2c27dcc3a845959c
Author: Lee moon soo
AuthorDate: Tue Mar 9 08:45:53 2021 -0800
[ZEPPELIN-5167] Set HOME env variable to in Dockerfile, so Helium can
install package successfully
### What is this PR for?
Without HOME env variable set in Docker image, installing a helium
visualization package fails with error.
```
INFO [2021-03-04 06:40:51,674] ({qtp418304857-31}
YarnTaskExecutor.java[execute]:54) - Running 'yarn config set cache-folder
/opt/zeppelin/local-repo/helium-bundle/yarn-cache
--registry=https://registry.npmjs.org/' in
/opt/zeppelin/local-repo/helium-bundle
ERROR [2021-03-04 06:40:51,793] ({Thread-24}
InputStreamHandler.java[log]:60) -
/opt/zeppelin/local-repo/helium-bundle/node/yarn/dist/node_modules/mkdirp/index.js:90
ERROR [2021-03-04 06:40:51,795] ({Thread-24}
InputStreamHandler.java[log]:60) - throw err0;
ERROR [2021-03-04 06:40:51,795] ({Thread-24}
InputStreamHandler.java[log]:60) - ^
ERROR [2021-03-04 06:40:51,795] ({Thread-24}
InputStreamHandler.java[log]:60) -
ERROR [2021-03-04 06:40:51,796] ({Thread-24}
InputStreamHandler.java[log]:60) - Error: EACCES: permission denied, mkdir
'/.cache'
ERROR [2021-03-04 06:40:51,796] ({Thread-24}
InputStreamHandler.java[log]:60) - at Error (native)
ERROR [2021-03-04 06:40:51,796] ({Thread-24}
InputStreamHandler.java[log]:60) - at Object.fs.mkdirSync (fs.js:922:18)
ERROR [2021-03-04 06:40:51,797] ({Thread-24}
InputStreamHandler.java[log]:60) - at sync
(/opt/zeppelin/local-repo/helium-bundle/node/yarn/dist/node_modules/mkdirp/index.js:71:13)
ERROR [2021-03-04 06:40:51,797] ({Thread-24}
InputStreamHandler.java[log]:60) - at Function.sync
(/opt/zeppelin/local-repo/helium-bundle/node/yarn/dist/node_modules/mkdirp/index.js:77:24)
ERROR [2021-03-04 06:40:51,797] ({Thread-24}
InputStreamHandler.java[log]:60) - at Object.
(/opt/zeppelin/local-repo/helium-bundle/node/yarn/dist/bin/yarn.js:30:8)
ERROR [2021-03-04 06:40:51,797] ({Thread-24}
InputStreamHandler.java[log]:60) - at Module._compile (module.js:570:32)
ERROR [2021-03-04 06:40:51,798] ({Thread-24}
InputStreamHandler.java[log]:60) - at Object.Module._extensions..js
(module.js:579:10)
ERROR [2021-03-04 06:40:51,799] ({Thread-24}
InputStreamHandler.java[log]:60) - at Module.load (module.js:487:32)
ERROR [2021-03-04 06:40:51,801] ({Thread-24}
InputStreamHandler.java[log]:60) - at tryModuleLoad (module.js:446:12)
ERROR [2021-03-04 06:40:51,801] ({Thread-24}
InputStreamHandler.java[log]:60) - at Function.Module._load
(module.js:438:3)
```
This error can be solved by manually adding HOME env variable when starting
docker container, like
```
docker run -p8080:8080 --rm -e HOME=/opt/zeppelin -e
ZEPPELIN_HELIUM_REGISTRY=helium,https://s3.amazonaws.com/helium-package/helium.json
apache/zeppelin:0.9.0
```
However, the error message is not very intuitive and it may take lots of
time to find a solution.
So, I think it make sense to set the HOME env variable in Dockerfile.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-5167
### How should this be tested?
Reproduce the problem by running Zeppelin from the latest docker image
without the HOME env variable.
And try to install a visualization package from Helium menu
```
docker run -p8080:8080 --rm -e
ZEPPELIN_HELIUM_REGISTRY=helium,https://s3.amazonaws.com/helium-package/helium.json
apache/zeppelin:0.9.0
```
And then test with HOME environment variable, and verify visualization
package is installed without error.
```
docker run -p8080:8080 --rm -e HOME=/opt/zeppelin -e
ZEPPELIN_HELIUM_REGISTRY=helium,https://s3.amazonaws.com/helium-package/helium.json
apache/zeppelin:0.9.0
```
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #4070 from Leemoonsoo/ZEPPELIN-5167 and squashes the following
commits:
bc9cc8dcb [Lee moon soo] add HOME env var to zeppelin server Dockerfile
8a296cf8d [Lee moon soo] Z_HOME -> ZEPPELIN_HOME
d9a9ed245 [Lee moon soo] set HOME env variable
---
bin/interpreter.sh | 2 +-
bin/zeppelin.sh| 2 +-
docs/quickst
[zeppelin] branch branch-0.9 updated: [ZEPPELIN-5019] Enable configurable HTML addons
This is an automated email from the ASF dual-hosted git repository. moon pushed a commit to branch branch-0.9 in repository https://gitbox.apache.org/repos/asf/zeppelin.git The following commit(s) were added to refs/heads/branch-0.9 by this push: new 37ac478 [ZEPPELIN-5019] Enable configurable HTML addons 37ac478 is described below commit 37ac4789874330ba5208a84caafb3a307fba8349 Author: Andreas Weise AuthorDate: Thu Aug 27 17:05:47 2020 +0200 [ZEPPELIN-5019] Enable configurable HTML addons ### What is this PR for? Currently its hard to integrate certain plotting libraries in one single zeppelin notebook efficiently. efficiently here means in the sense of how often library code needs to be embedded into a single notebook. while few plotting libraries provide APIs to properly enable lazy loading of their JS library (e.g. bokeh), others do not (e.g. plotly), which results in large overhead in notebooks with couple of plots, where each plot embeds the library code again and again. attempts to only [...] A general solution for that problem is to enable the loading of 3rd party JS libraries within the root HTML. Hence this issue will introduce a new optional zeppelin configuration that allows administrators to add global addons to the zeppelin root HTML, so analysts can rely on that the JS library is already loaded for every paragraph. As there exist two places in HTML where initial loading may be placed, the issue will introduce the following two configuration properties: - `zeppelin.server.html.head.addon` > addon html code to be placed at the end of the html->head section - `zeppelin.server.html.body.addon` > addon html code to be placed at the end of the html->body section Each property may contain any valid HTML code, e.g. ```
[zeppelin] branch master updated: [ZEPPELIN-5019] Enable configurable HTML addons
This is an automated email from the ASF dual-hosted git repository. moon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/zeppelin.git The following commit(s) were added to refs/heads/master by this push: new cb06507 [ZEPPELIN-5019] Enable configurable HTML addons cb06507 is described below commit cb06507f6323a9bf7a338ad0ffccc1adb9464c1f Author: Andreas Weise AuthorDate: Thu Aug 27 17:05:47 2020 +0200 [ZEPPELIN-5019] Enable configurable HTML addons ### What is this PR for? Currently its hard to integrate certain plotting libraries in one single zeppelin notebook efficiently. efficiently here means in the sense of how often library code needs to be embedded into a single notebook. while few plotting libraries provide APIs to properly enable lazy loading of their JS library (e.g. bokeh), others do not (e.g. plotly), which results in large overhead in notebooks with couple of plots, where each plot embeds the library code again and again. attempts to only [...] A general solution for that problem is to enable the loading of 3rd party JS libraries within the root HTML. Hence this issue will introduce a new optional zeppelin configuration that allows administrators to add global addons to the zeppelin root HTML, so analysts can rely on that the JS library is already loaded for every paragraph. As there exist two places in HTML where initial loading may be placed, the issue will introduce the following two configuration properties: - `zeppelin.server.html.head.addon` > addon html code to be placed at the end of the html->head section - `zeppelin.server.html.body.addon` > addon html code to be placed at the end of the html->body section Each property may contain any valid HTML code, e.g. ```
[zeppelin] branch master updated: [ZEPPELIN-4985] Instant visual feedback on high latency network environment
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 1d01972 [ZEPPELIN-4985] Instant visual feedback on high latency
network environment
1d01972 is described below
commit 1d01972c0318fae43321ee74b2cb3f224f7d61fb
Author: Lee moon soo
AuthorDate: Thu Aug 6 14:07:22 2020 -0700
[ZEPPELIN-4985] Instant visual feedback on high latency network environment
### What is this PR for?
This PR provides an instant visual feedback to improve user experience on
high latency network environment.
It focuses on improving 2 frequent user action
- Run paragraph(s)
- Swich visualization
In high latency network, the user may feel a short 'freeze' between action
(click) and any visual change in notebook.
While the paragraph is updated after get a response from server.
After this change, the paragraph will get immediate update after a user
action, without waiting for server response.
Before
- Some delay to become PENDING status after click run
- Some delay after switch visualization (screenrecord is not long enough
to see actual change after delay)
After
- Instant status update to PENDING after click run
- Instant visualization switch
### What type of PR is it?
Improvement
### Todos
* [x] - Instant visual feedback on paragraph run
* [x] - Instant visual feedback on visualization switch
* [x] - Instant visual feedback on paragraph run (new ui)
* [x] - Instant visual feedback on visualization switch (new ui)
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4985
### How should this be tested?
Can simulate high latency network by adding sleep in NotebookSocket.send()
method
```
---
a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookSocket.java
+++
b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookSocket.java
-67,6 +67,9 public class NotebookSocket extends WebSocketAdapter {
}
public synchronized void send(String serializeMessage) throws
IOException {
+try {
+ Thread.sleep(1000);
+} catch (InterruptedException e) {}
connection.getRemote().sendString(serializeMessage);
}
```
Screen recordings attached are made in this way.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3873 from Leemoonsoo/ZEPPELIN-4985 and squashes the following
commits:
b064cf414 [Lee moon soo] update test
c44c1002f [Lee moon soo] implement paragraph update short circuit in new ui
5952cf77c [Lee moon soo] instant update on switching visualization
c38002972 [Lee moon soo] try implement short circuit logic in new ui
97c832e0f [Lee moon soo] skip update paragraph to READY when submit
00135e149 [Lee moon soo] short circuit status update on paragraph run
---
.../zeppelin/integration/ParagraphActionsIT.java | 2 +-
.../zeppelin/cluster/event/ClusterMessage.java | 9
.../org/apache/zeppelin/rest/NotebookRestApi.java | 4 +-
.../apache/zeppelin/socket/ConnectionManager.java | 4 +-
.../org/apache/zeppelin/socket/NotebookServer.java | 53 ++--
.../projects/zeppelin-sdk/package.json | 2 +-
.../interfaces/message-data-type-map.interface.ts | 2 +
.../src/interfaces/message-operator.interface.ts | 8
.../src/interfaces/message-paragraph.interface.ts | 5 ++
.../src/interfaces/websocket-message.interface.ts | 1 +
.../projects/zeppelin-sdk/src/message.ts | 56 ++
.../src/app/core/paragraph-base/paragraph-base.ts | 8
.../app/notebook/paragraph/paragraph.controller.js | 7 +++
.../notebook/paragraph/result/result.controller.js | 17 +++
.../websocket/websocket-event.factory.js | 18 ++-
.../websocket/websocket-message.service.js | 16 +++
.../org/apache/zeppelin/notebook/Paragraph.java| 1 -
.../apache/zeppelin/notebook/socket/Message.java | 12 +
18 files changed, 178 insertions(+), 47 deletions(-)
diff --git
a/zeppelin-integration/src/test/java/org/apache/zeppelin/integration/ParagraphActionsIT.java
b/zeppelin-integration/src/test/java/org/apache/zeppelin/integration/ParagraphActionsIT.java
index 13b96ae..c6a11eb 100644
[zeppelin] branch branch-0.9 updated: [ZEPPELIN-4985] Instant visual feedback on high latency network environment
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch branch-0.9
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/branch-0.9 by this push:
new 1fa5a20 [ZEPPELIN-4985] Instant visual feedback on high latency
network environment
1fa5a20 is described below
commit 1fa5a2032aec1c8396c4e5d436ec4ee517dea48e
Author: Lee moon soo
AuthorDate: Thu Aug 6 14:07:22 2020 -0700
[ZEPPELIN-4985] Instant visual feedback on high latency network environment
### What is this PR for?
This PR provides an instant visual feedback to improve user experience on
high latency network environment.
It focuses on improving 2 frequent user action
- Run paragraph(s)
- Swich visualization
In high latency network, the user may feel a short 'freeze' between action
(click) and any visual change in notebook.
While the paragraph is updated after get a response from server.
After this change, the paragraph will get immediate update after a user
action, without waiting for server response.
Before
- Some delay to become PENDING status after click run
- Some delay after switch visualization (screenrecord is not long enough
to see actual change after delay)
After
- Instant status update to PENDING after click run
- Instant visualization switch
### What type of PR is it?
Improvement
### Todos
* [x] - Instant visual feedback on paragraph run
* [x] - Instant visual feedback on visualization switch
* [x] - Instant visual feedback on paragraph run (new ui)
* [x] - Instant visual feedback on visualization switch (new ui)
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4985
### How should this be tested?
Can simulate high latency network by adding sleep in NotebookSocket.send()
method
```
---
a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookSocket.java
+++
b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookSocket.java
-67,6 +67,9 public class NotebookSocket extends WebSocketAdapter {
}
public synchronized void send(String serializeMessage) throws
IOException {
+try {
+ Thread.sleep(1000);
+} catch (InterruptedException e) {}
connection.getRemote().sendString(serializeMessage);
}
```
Screen recordings attached are made in this way.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3873 from Leemoonsoo/ZEPPELIN-4985 and squashes the following
commits:
b064cf414 [Lee moon soo] update test
c44c1002f [Lee moon soo] implement paragraph update short circuit in new ui
5952cf77c [Lee moon soo] instant update on switching visualization
c38002972 [Lee moon soo] try implement short circuit logic in new ui
97c832e0f [Lee moon soo] skip update paragraph to READY when submit
00135e149 [Lee moon soo] short circuit status update on paragraph run
(cherry picked from commit 1d01972c0318fae43321ee74b2cb3f224f7d61fb)
Signed-off-by: Lee moon soo
---
.../zeppelin/integration/ParagraphActionsIT.java | 2 +-
.../zeppelin/cluster/event/ClusterMessage.java | 9
.../org/apache/zeppelin/rest/NotebookRestApi.java | 4 +-
.../apache/zeppelin/socket/ConnectionManager.java | 4 +-
.../org/apache/zeppelin/socket/NotebookServer.java | 53 ++--
.../projects/zeppelin-sdk/package.json | 2 +-
.../interfaces/message-data-type-map.interface.ts | 2 +
.../src/interfaces/message-operator.interface.ts | 8
.../src/interfaces/message-paragraph.interface.ts | 5 ++
.../src/interfaces/websocket-message.interface.ts | 1 +
.../projects/zeppelin-sdk/src/message.ts | 56 ++
.../src/app/core/paragraph-base/paragraph-base.ts | 8
.../app/notebook/paragraph/paragraph.controller.js | 7 +++
.../notebook/paragraph/result/result.controller.js | 17 +++
.../websocket/websocket-event.factory.js | 18 ++-
.../websocket/websocket-message.service.js | 16 +++
.../org/apache/zeppelin/notebook/Paragraph.java| 1 -
.../apache/zeppelin/notebook/socket/Message.java | 12 +
18 files changed, 178 insertions(+), 47 deletions(-)
diff --git
a/zeppelin-integration/src/test/java/org/apache/zeppelin/integration/ParagraphActionsIT.java
b
[zeppelin] branch master updated: [ZEPPELIN-4918] Add Table of Contents to Sidebar
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 8b8a50a [ZEPPELIN-4918] Add Table of Contents to Sidebar
8b8a50a is described below
commit 8b8a50a4bd99f563a50a32e284e4cb8a2d39905d
Author: Pranav Tharoor
AuthorDate: Tue Jun 30 11:52:19 2020 +0530
[ZEPPELIN-4918] Add Table of Contents to Sidebar
### What is this PR for?
This PR adds a Table of Contents (ToC) to the new Zeppelin UI. The ToC
allows quick navigation through long notebooks. The ToC is generated based on
the presence of Heading tags in the results.
### What type of PR is it?
Improvement
### Todos
* [ ] - N/A
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4918
### How should this be tested?
Travis CI:
https://travis-ci.com/github/pranavtharoor/zeppelin/builds/173370001
Manual steps:
1. Create a notebook
2. Add different levels of Markdown headings or other inline HTML heading
tags
3. Run the paragraphs so that the result is shown
4. Click the ToC icon in the sidebar ribbon to open the ToC. The ToC should
be generated/refreshed
5. Try clicking on the ToC rows to scroll to the respective paragraph
### Screenshots (if appropriate)
https://user-images.githubusercontent.com/22600061/86060953-6f5bf100-ba83-11ea-8fff-8af2ce856da7.png;>
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Pranav Tharoor
Closes #3834 from pranavtharoor/ZEPPELIN-4918 and squashes the following
commits:
1b67911bf [Pranav Tharoor] [ZEPPELIN-4918] Fix breaking components and
sidebar refresh on adding a new notebook
c5c362982 [Pranav Tharoor] [ZEPPELIN-4918] Add Table of Contents to Sidebar
---
.../workspace/notebook/notebook.component.html | 14 ++-
.../workspace/notebook/notebook.component.less | 35 ++-
.../pages/workspace/notebook/notebook.component.ts | 6 +-
.../pages/workspace/notebook/notebook.module.ts| 4 +-
.../notebook/sidebar/sidebar.component.html| 35 +++
.../notebook/sidebar/sidebar.component.less| 67 +
.../notebook/sidebar/sidebar.component.ts | 51 ++
.../src/app/share/note-toc/note-toc.component.html | 23 +
.../src/app/share/note-toc/note-toc.component.less | 84
.../src/app/share/note-toc/note-toc.component.ts | 110 +
zeppelin-web-angular/src/app/share/share.module.ts | 10 +-
11 files changed, 397 insertions(+), 42 deletions(-)
diff --git
a/zeppelin-web-angular/src/app/pages/workspace/notebook/notebook.component.html
b/zeppelin-web-angular/src/app/pages/workspace/notebook/notebook.component.html
index 52b8c96..2c3a682 100644
---
a/zeppelin-web-angular/src/app/pages/workspace/notebook/notebook.component.html
+++
b/zeppelin-web-angular/src/app/pages/workspace/notebook/notebook.component.html
@@ -25,15 +25,13 @@
-
-
-
-
-
+[style.min-width.px]="!isSidebarOpen ? 40 : sidebarWidth">
+
diff --git
a/zeppelin-web-angular/src/app/pages/workspace/notebook/notebook.component.less
b/zeppelin-web-angular/src/app/pages/workspace/notebook/notebook.component.less
index 4c0e7cd..6938ce4 100644
---
a/zeppelin-web-angular/src/app/pages/workspace/notebook/notebook.component.less
+++
b/zeppelin-web-angular/src/app/pages/workspace/notebook/notebook.component.less
@@ -32,30 +32,15 @@
.sidebar-area {
box-shadow: 4px 0px 2px 0 rgba(0, 0, 0, 0.06);
-border-right: solid thin @component-background;;
-overflow-y: auto;
-overflow-x: hidden;
+border-right: solid thin @component-background;
+overflow: hidden;
background: @component-background;
-padding: 20px;
z-index: 25;
position: sticky;
top: 0px;
height: 100vh;
}
- .sidebar-toggle {
-width: 40px;
-position: absolute;
-top: 0;
-right: 0;
-background: none;
-border: none;
-cursor: pointer;
-padding: 10px 0;
-font-size: 20px;
-z-index: 30;
- }
-
.sidebar-resize {
height: 100%;
width: 5px;
@@ -68,24 +53,15 @@
.simple .sidebar-area {
box-shadow: none;
-
-.sidebar-toggle {
- opacity: 0;
- transition: opacity 0.2s ease;
-}
-
+
&:hover {
border-right: solid thin rgba(0, 0, 0, 0.06);
-
- .sidebar-toggle {
-opacity: 1;
- }
}
-
}
-
+
.notebook-area {
overflow: hidden;
+width: 100%;
}
.extension-area {
@@ -96,6 +72,7 @@
top: 0px;
z-index: 20;
}
+
.paragraph-area {
padding: 0 5px;
}
diff --git
a/
[zeppelin] branch branch-0.9 updated: [ZEPPELIN-4918] Add Table of Contents to Sidebar
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch branch-0.9
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/branch-0.9 by this push:
new 9af725e [ZEPPELIN-4918] Add Table of Contents to Sidebar
9af725e is described below
commit 9af725e97a8ccf4ba5a3ff54d1e31f4e2e908ad1
Author: Pranav Tharoor
AuthorDate: Tue Jun 30 11:52:19 2020 +0530
[ZEPPELIN-4918] Add Table of Contents to Sidebar
### What is this PR for?
This PR adds a Table of Contents (ToC) to the new Zeppelin UI. The ToC
allows quick navigation through long notebooks. The ToC is generated based on
the presence of Heading tags in the results.
### What type of PR is it?
Improvement
### Todos
* [ ] - N/A
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4918
### How should this be tested?
Travis CI:
https://travis-ci.com/github/pranavtharoor/zeppelin/builds/173370001
Manual steps:
1. Create a notebook
2. Add different levels of Markdown headings or other inline HTML heading
tags
3. Run the paragraphs so that the result is shown
4. Click the ToC icon in the sidebar ribbon to open the ToC. The ToC should
be generated/refreshed
5. Try clicking on the ToC rows to scroll to the respective paragraph
### Screenshots (if appropriate)
https://user-images.githubusercontent.com/22600061/86060953-6f5bf100-ba83-11ea-8fff-8af2ce856da7.png;>
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Pranav Tharoor
Closes #3834 from pranavtharoor/ZEPPELIN-4918 and squashes the following
commits:
1b67911bf [Pranav Tharoor] [ZEPPELIN-4918] Fix breaking components and
sidebar refresh on adding a new notebook
c5c362982 [Pranav Tharoor] [ZEPPELIN-4918] Add Table of Contents to Sidebar
(cherry picked from commit 8b8a50a4bd99f563a50a32e284e4cb8a2d39905d)
Signed-off-by: Lee moon soo
---
.../workspace/notebook/notebook.component.html | 14 ++-
.../workspace/notebook/notebook.component.less | 35 ++-
.../pages/workspace/notebook/notebook.component.ts | 6 +-
.../pages/workspace/notebook/notebook.module.ts| 4 +-
.../notebook/sidebar/sidebar.component.html| 35 +++
.../notebook/sidebar/sidebar.component.less| 67 +
.../notebook/sidebar/sidebar.component.ts | 51 ++
.../src/app/share/note-toc/note-toc.component.html | 23 +
.../src/app/share/note-toc/note-toc.component.less | 84
.../src/app/share/note-toc/note-toc.component.ts | 110 +
zeppelin-web-angular/src/app/share/share.module.ts | 10 +-
11 files changed, 397 insertions(+), 42 deletions(-)
diff --git
a/zeppelin-web-angular/src/app/pages/workspace/notebook/notebook.component.html
b/zeppelin-web-angular/src/app/pages/workspace/notebook/notebook.component.html
index 52b8c96..2c3a682 100644
---
a/zeppelin-web-angular/src/app/pages/workspace/notebook/notebook.component.html
+++
b/zeppelin-web-angular/src/app/pages/workspace/notebook/notebook.component.html
@@ -25,15 +25,13 @@
-
-
-
-
-
+[style.min-width.px]="!isSidebarOpen ? 40 : sidebarWidth">
+
diff --git
a/zeppelin-web-angular/src/app/pages/workspace/notebook/notebook.component.less
b/zeppelin-web-angular/src/app/pages/workspace/notebook/notebook.component.less
index 4c0e7cd..6938ce4 100644
---
a/zeppelin-web-angular/src/app/pages/workspace/notebook/notebook.component.less
+++
b/zeppelin-web-angular/src/app/pages/workspace/notebook/notebook.component.less
@@ -32,30 +32,15 @@
.sidebar-area {
box-shadow: 4px 0px 2px 0 rgba(0, 0, 0, 0.06);
-border-right: solid thin @component-background;;
-overflow-y: auto;
-overflow-x: hidden;
+border-right: solid thin @component-background;
+overflow: hidden;
background: @component-background;
-padding: 20px;
z-index: 25;
position: sticky;
top: 0px;
height: 100vh;
}
- .sidebar-toggle {
-width: 40px;
-position: absolute;
-top: 0;
-right: 0;
-background: none;
-border: none;
-cursor: pointer;
-padding: 10px 0;
-font-size: 20px;
-z-index: 30;
- }
-
.sidebar-resize {
height: 100%;
width: 5px;
@@ -68,24 +53,15 @@
.simple .sidebar-area {
box-shadow: none;
-
-.sidebar-toggle {
- opacity: 0;
- transition: opacity 0.2s ease;
-}
-
+
&:hover {
border-right: solid thin rgba(0, 0, 0, 0.06);
-
- .sidebar-toggle {
-opacity: 1;
- }
}
-
}
-
+
.notebook-area {
overflow: hidden;
+width: 100%;
}
.extension-a
[zeppelin] branch master updated: [ZEPPELIN-4799] Use spark resource configuration
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 3b7df78 [ZEPPELIN-4799] Use spark resource configuration
3b7df78 is described below
commit 3b7df786cb067e234448c943994c3fa685693069
Author: Philipp Dallig
AuthorDate: Thu May 7 09:12:20 2020 +0200
[ZEPPELIN-4799] Use spark resource configuration
### What is this PR for?
With this PR, we use spark configuration values for K8s Pod resources. A
memory limit is not set because of a potential OOM-Killer.
https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits
> If you set a memory limit of 4GiB for that Container, the kubelet (and
container runtime ) enforce the limit. The runtime prevents the container from
using more than the configured resource limit. For example: when a process in
the container tries to consume more than the allowed amount of memory, the
system kernel terminates the process that attempted the allocation, with an out
of memory (OOM) error.
zjffdu Are using a YARN cluster to schedule your Interpreters? Maybe we
should change the location of the calculation class.
### What type of PR is it?
- Improvement
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-4799
### How should this be tested?
* **Travis-CI**:
https://travis-ci.org/github/Reamer/zeppelin/builds/683269227
### Questions:
* **Maybe a higher default memory overhead?** Edit: No, because we doesn't
need it in the past.
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Philipp Dallig
Closes #3761 from Reamer/spark_k8s_resources and squashes the following
commits:
64bd91294 [Philipp Dallig] Add short comment for limits.memory
45e565fff [Philipp Dallig] Use Spark config values for K8s Interpreter Pod
resources
3218309b2 [Philipp Dallig] Some cleanup
---
k8s/interpreter/100-interpreter-spec.yaml | 9 +++
.../launcher/K8sRemoteInterpreterProcess.java | 33 +++--
.../zeppelin/interpreter/launcher/K8sUtils.java| 59
.../launcher/K8sRemoteInterpreterProcessTest.java | 78 +-
.../interpreter/launcher/K8sUtilsTest.java | 30 +
5 files changed, 200 insertions(+), 9 deletions(-)
diff --git a/k8s/interpreter/100-interpreter-spec.yaml
b/k8s/interpreter/100-interpreter-spec.yaml
index c331a99..76f1dea 100644
--- a/k8s/interpreter/100-interpreter-spec.yaml
+++ b/k8s/interpreter/100-interpreter-spec.yaml
@@ -54,6 +54,15 @@ spec:
- name: {{key}}
value: {{value}}
{% endfor %}
+ {% if zeppelin.k8s.interpreter.cores is defined and
zeppelin.k8s.interpreter.memory is defined %}
+resources:
+ requests:
+memory: "{{zeppelin.k8s.interpreter.memory}}"
+cpu: "{{zeppelin.k8s.interpreter.cores}}"
+{# limits.memory is not set because of a potential OOM-Killer.
https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits
#}
+ limits:
+cpu: "{{zeppelin.k8s.interpreter.cores}}"
+ {% endif %}
{% if zeppelin.k8s.interpreter.group.name == "spark" %}
volumeMounts:
- name: spark-home
diff --git
a/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
b/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
index c167ae7..864f660 100644
---
a/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
+++
b/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
@@ -44,6 +44,12 @@ public class K8sRemoteInterpreterProcess extends
RemoteInterpreterProcess {
private String userName;
private AtomicBoolean started = new AtomicBoolean(false);
+ private Random rand = new Random();
+
+ private static final String SPARK_DRIVER_MEMROY = "spark.driver.memory";
+ private static final String SPARK_DRIVER_MEMROY_OVERHEAD =
"spark.driver.memoryOverhead";
+ private static final String SPARK_DRIVER_CORES = "spark.driver.cores";
+ private static final String ENV_SERVICE_DOMAIN = "SERVICE_DOMAIN";
public K8sRemoteInterpreterProcess(
Kubectl kubectl,
@@ -273,7 +279,7 @@ public class K8sRemoteInterpreterProcess extends
RemoteInterpreterProcess {
}
// environment variables
-envs.put("SERVICE_DOMAIN", envs.getOrDefault("SERVIC
[zeppelin] branch branch-0.9 updated: [ZEPPELIN-4799] Use spark resource configuration
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch branch-0.9
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/branch-0.9 by this push:
new e4f6630 [ZEPPELIN-4799] Use spark resource configuration
e4f6630 is described below
commit e4f663048cff7bb18064a3e24fe4792f622dd031
Author: Philipp Dallig
AuthorDate: Thu May 7 09:12:20 2020 +0200
[ZEPPELIN-4799] Use spark resource configuration
### What is this PR for?
With this PR, we use spark configuration values for K8s Pod resources. A
memory limit is not set because of a potential OOM-Killer.
https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits
> If you set a memory limit of 4GiB for that Container, the kubelet (and
container runtime ) enforce the limit. The runtime prevents the container from
using more than the configured resource limit. For example: when a process in
the container tries to consume more than the allowed amount of memory, the
system kernel terminates the process that attempted the allocation, with an out
of memory (OOM) error.
zjffdu Are using a YARN cluster to schedule your Interpreters? Maybe we
should change the location of the calculation class.
### What type of PR is it?
- Improvement
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-4799
### How should this be tested?
* **Travis-CI**:
https://travis-ci.org/github/Reamer/zeppelin/builds/683269227
### Questions:
* **Maybe a higher default memory overhead?** Edit: No, because we doesn't
need it in the past.
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Philipp Dallig
Closes #3761 from Reamer/spark_k8s_resources and squashes the following
commits:
64bd91294 [Philipp Dallig] Add short comment for limits.memory
45e565fff [Philipp Dallig] Use Spark config values for K8s Interpreter Pod
resources
3218309b2 [Philipp Dallig] Some cleanup
(cherry picked from commit 3b7df786cb067e234448c943994c3fa685693069)
Signed-off-by: Lee moon soo
---
k8s/interpreter/100-interpreter-spec.yaml | 9 +++
.../launcher/K8sRemoteInterpreterProcess.java | 33 +++--
.../zeppelin/interpreter/launcher/K8sUtils.java| 59
.../launcher/K8sRemoteInterpreterProcessTest.java | 78 +-
.../interpreter/launcher/K8sUtilsTest.java | 30 +
5 files changed, 200 insertions(+), 9 deletions(-)
diff --git a/k8s/interpreter/100-interpreter-spec.yaml
b/k8s/interpreter/100-interpreter-spec.yaml
index c331a99..76f1dea 100644
--- a/k8s/interpreter/100-interpreter-spec.yaml
+++ b/k8s/interpreter/100-interpreter-spec.yaml
@@ -54,6 +54,15 @@ spec:
- name: {{key}}
value: {{value}}
{% endfor %}
+ {% if zeppelin.k8s.interpreter.cores is defined and
zeppelin.k8s.interpreter.memory is defined %}
+resources:
+ requests:
+memory: "{{zeppelin.k8s.interpreter.memory}}"
+cpu: "{{zeppelin.k8s.interpreter.cores}}"
+{# limits.memory is not set because of a potential OOM-Killer.
https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits
#}
+ limits:
+cpu: "{{zeppelin.k8s.interpreter.cores}}"
+ {% endif %}
{% if zeppelin.k8s.interpreter.group.name == "spark" %}
volumeMounts:
- name: spark-home
diff --git
a/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
b/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
index c167ae7..864f660 100644
---
a/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
+++
b/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
@@ -44,6 +44,12 @@ public class K8sRemoteInterpreterProcess extends
RemoteInterpreterProcess {
private String userName;
private AtomicBoolean started = new AtomicBoolean(false);
+ private Random rand = new Random();
+
+ private static final String SPARK_DRIVER_MEMROY = "spark.driver.memory";
+ private static final String SPARK_DRIVER_MEMROY_OVERHEAD =
"spark.driver.memoryOverhead";
+ private static final String SPARK_DRIVER_CORES = "spark.driver.cores";
+ private static final String ENV_SERVICE_DOMAIN = "SERVICE_DOMAIN";
public K8sRemoteInterpreterProcess(
Kubectl kubectl,
@@ -273,7 +279,7 @@ public class K8sRemoteInterpreterProcess extends
RemoteInterp
[zeppelin] branch master updated: [ZEPPELIN-4782] Correct K8s service usage in Zeppelin server
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 4359193 [ZEPPELIN-4782] Correct K8s service usage in Zeppelin server
4359193 is described below
commit 4359193e4f0b757118f929489627eaca463c8902
Author: Philipp Dallig
AuthorDate: Wed Apr 29 10:50:39 2020 +0200
[ZEPPELIN-4782] Correct K8s service usage in Zeppelin server
### What is this PR for?
With this PR, we render the K8s-Service name into the Zeppelin interpreter
start command for the initial connection.
The K8s-Service name should be variable, because the name can vary from
environment to environment.
### What type of PR is it?
- Bug Fix
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-4782
### How should this be tested?
* Travis-CI: https://travis-ci.org/github/Reamer/zeppelin/builds/680915013
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Philipp Dallig
Closes #3753 from Reamer/k8s_service and squashes the following commits:
74be24f90 [Philipp Dallig] Use K8s service to connect Zeppelin interpreter
with Zeppelin server
7ecee5d6a [Philipp Dallig] Some cleanup
84e4e3329 [Philipp Dallig] Add ".factorypath" to .gitignore it's created by
eclipse
---
.gitignore | 1 +
docs/setup/operation/configuration.md| 8 +++-
k8s/interpreter/100-interpreter-spec.yaml| 2 +-
k8s/zeppelin-server.yaml | 2 +-
.../java/org/apache/zeppelin/conf/ZeppelinConfiguration.java | 5 +
.../interpreter/launcher/K8sRemoteInterpreterProcess.java| 11 +--
.../interpreter/launcher/K8sStandardInterpreterLauncher.java | 10 +-
.../org/apache/zeppelin/interpreter/launcher/Kubectl.java| 3 ++-
.../launcher/K8sRemoteInterpreterProcessTest.java| 12 ++--
.../launcher/K8sStandardInterpreterLauncherTest.java | 6 --
10 files changed, 33 insertions(+), 27 deletions(-)
diff --git a/.gitignore b/.gitignore
index 4086a4b..618075d 100644
--- a/.gitignore
+++ b/.gitignore
@@ -94,6 +94,7 @@ Thumbs.db
**/.settings/
.project
.settings/
+.factorypath
# intelliJ IDEA project files
.idea/
diff --git a/docs/setup/operation/configuration.md
b/docs/setup/operation/configuration.md
index 65129ff..3c509b0 100644
--- a/docs/setup/operation/configuration.md
+++ b/docs/setup/operation/configuration.md
@@ -418,7 +418,13 @@ If both are defined, then the **environment variables**
will take priority.
zeppelin.k8s.template.dir
k8s
Kubernetes yaml spec files
-
+
+
+ZEPPELIN_K8S_SERVICE_NAME
+zeppelin.k8s.service.name
+zeppelin-server
+Name of the Zeppelin server service resources
+
diff --git a/k8s/interpreter/100-interpreter-spec.yaml
b/k8s/interpreter/100-interpreter-spec.yaml
index c857ff2..c331a99 100644
--- a/k8s/interpreter/100-interpreter-spec.yaml
+++ b/k8s/interpreter/100-interpreter-spec.yaml
@@ -43,7 +43,7 @@ spec:
containers:
- name: {{zeppelin.k8s.interpreter.container.name}}
image: {{zeppelin.k8s.interpreter.container.image}}
-command: ["sh", "-c", "$(ZEPPELIN_HOME)/bin/interpreter.sh -d
$(ZEPPELIN_HOME)/interpreter/{{zeppelin.k8s.interpreter.group.name}} -r
{{zeppelin.k8s.interpreter.rpc.portRange}} -c {{zeppelin.k8s.server.rpc.host}}
-p {{zeppelin.k8s.server.rpc.portRange}} -i
{{zeppelin.k8s.interpreter.group.id}} -l {{zeppelin.k8s.interpreter.localRepo}}
-g {{zeppelin.k8s.interpreter.setting.name}}"]
+command: ["sh", "-c", "$(ZEPPELIN_HOME)/bin/interpreter.sh -d
$(ZEPPELIN_HOME)/interpreter/{{zeppelin.k8s.interpreter.group.name}} -r
{{zeppelin.k8s.interpreter.rpc.portRange}} -c
{{zeppelin.k8s.server.rpc.service}} -p {{zeppelin.k8s.server.rpc.portRange}} -i
{{zeppelin.k8s.interpreter.group.id}} -l {{zeppelin.k8s.interpreter.localRepo}}
-g {{zeppelin.k8s.interpreter.setting.name}}"]
lifecycle:
preStop:
exec:
diff --git a/k8s/zeppelin-server.yaml b/k8s/zeppelin-server.yaml
index 08b3c03..aefd07c 100644
--- a/k8s/zeppelin-server.yaml
+++ b/k8s/zeppelin-server.yaml
@@ -186,7 +186,7 @@ spec:
kind: Service
apiVersion: v1
metadata:
- name: zeppelin-server# keep Service name the same to Pod name.
+ name: zeppelin-server
spec:
ports:
- name: http
diff --git
a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java
b/zeppelin-interpreter/src/main/java/org/apache/
[zeppelin] branch branch-0.9 updated: [ZEPPELIN-4782] Correct K8s service usage in Zeppelin server
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch branch-0.9
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/branch-0.9 by this push:
new 83fcb11 [ZEPPELIN-4782] Correct K8s service usage in Zeppelin server
83fcb11 is described below
commit 83fcb11407925eb8a7fdce7c8b083ebf2cf9f5b8
Author: Philipp Dallig
AuthorDate: Wed Apr 29 10:50:39 2020 +0200
[ZEPPELIN-4782] Correct K8s service usage in Zeppelin server
### What is this PR for?
With this PR, we render the K8s-Service name into the Zeppelin interpreter
start command for the initial connection.
The K8s-Service name should be variable, because the name can vary from
environment to environment.
### What type of PR is it?
- Bug Fix
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-4782
### How should this be tested?
* Travis-CI: https://travis-ci.org/github/Reamer/zeppelin/builds/680915013
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Philipp Dallig
Closes #3753 from Reamer/k8s_service and squashes the following commits:
74be24f90 [Philipp Dallig] Use K8s service to connect Zeppelin interpreter
with Zeppelin server
7ecee5d6a [Philipp Dallig] Some cleanup
84e4e3329 [Philipp Dallig] Add ".factorypath" to .gitignore it's created by
eclipse
(cherry picked from commit 4359193e4f0b757118f929489627eaca463c8902)
Signed-off-by: Lee moon soo
---
.gitignore | 1 +
docs/setup/operation/configuration.md| 8 +++-
k8s/interpreter/100-interpreter-spec.yaml| 2 +-
k8s/zeppelin-server.yaml | 2 +-
.../java/org/apache/zeppelin/conf/ZeppelinConfiguration.java | 5 +
.../interpreter/launcher/K8sRemoteInterpreterProcess.java| 11 +--
.../interpreter/launcher/K8sStandardInterpreterLauncher.java | 10 +-
.../org/apache/zeppelin/interpreter/launcher/Kubectl.java| 3 ++-
.../launcher/K8sRemoteInterpreterProcessTest.java| 12 ++--
.../launcher/K8sStandardInterpreterLauncherTest.java | 6 --
10 files changed, 33 insertions(+), 27 deletions(-)
diff --git a/.gitignore b/.gitignore
index 4086a4b..618075d 100644
--- a/.gitignore
+++ b/.gitignore
@@ -94,6 +94,7 @@ Thumbs.db
**/.settings/
.project
.settings/
+.factorypath
# intelliJ IDEA project files
.idea/
diff --git a/docs/setup/operation/configuration.md
b/docs/setup/operation/configuration.md
index 65129ff..3c509b0 100644
--- a/docs/setup/operation/configuration.md
+++ b/docs/setup/operation/configuration.md
@@ -418,7 +418,13 @@ If both are defined, then the **environment variables**
will take priority.
zeppelin.k8s.template.dir
k8s
Kubernetes yaml spec files
-
+
+
+ZEPPELIN_K8S_SERVICE_NAME
+zeppelin.k8s.service.name
+zeppelin-server
+Name of the Zeppelin server service resources
+
diff --git a/k8s/interpreter/100-interpreter-spec.yaml
b/k8s/interpreter/100-interpreter-spec.yaml
index c857ff2..c331a99 100644
--- a/k8s/interpreter/100-interpreter-spec.yaml
+++ b/k8s/interpreter/100-interpreter-spec.yaml
@@ -43,7 +43,7 @@ spec:
containers:
- name: {{zeppelin.k8s.interpreter.container.name}}
image: {{zeppelin.k8s.interpreter.container.image}}
-command: ["sh", "-c", "$(ZEPPELIN_HOME)/bin/interpreter.sh -d
$(ZEPPELIN_HOME)/interpreter/{{zeppelin.k8s.interpreter.group.name}} -r
{{zeppelin.k8s.interpreter.rpc.portRange}} -c {{zeppelin.k8s.server.rpc.host}}
-p {{zeppelin.k8s.server.rpc.portRange}} -i
{{zeppelin.k8s.interpreter.group.id}} -l {{zeppelin.k8s.interpreter.localRepo}}
-g {{zeppelin.k8s.interpreter.setting.name}}"]
+command: ["sh", "-c", "$(ZEPPELIN_HOME)/bin/interpreter.sh -d
$(ZEPPELIN_HOME)/interpreter/{{zeppelin.k8s.interpreter.group.name}} -r
{{zeppelin.k8s.interpreter.rpc.portRange}} -c
{{zeppelin.k8s.server.rpc.service}} -p {{zeppelin.k8s.server.rpc.portRange}} -i
{{zeppelin.k8s.interpreter.group.id}} -l {{zeppelin.k8s.interpreter.localRepo}}
-g {{zeppelin.k8s.interpreter.setting.name}}"]
lifecycle:
preStop:
exec:
diff --git a/k8s/zeppelin-server.yaml b/k8s/zeppelin-server.yaml
index 08b3c03..aefd07c 100644
--- a/k8s/zeppelin-server.yaml
+++ b/k8s/zeppelin-server.yaml
@@ -186,7 +186,7 @@ spec:
kind: Service
apiVersion: v1
metadata:
- name: zeppelin-server# keep Service name the same to Pod name.
+ name: zeppelin-server
spec:
ports:
- name: http
diff --git
a/zeppelin-interpreter
[zeppelin] branch master updated: [ZEPPELIN-4783] Avoid warning on PUBKEY when apt update cran.rstudio
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new e43e133 [ZEPPELIN-4783] Avoid warning on PUBKEY when apt update
cran.rstudio
e43e133 is described below
commit e43e133e7c7af5125d87dd80c089305c7f783498
Author: Yann PETIT
AuthorDate: Wed Apr 29 17:57:54 2020 +0200
[ZEPPELIN-4783] Avoid warning on PUBKEY when apt update cran.rstudio
### What is this PR for?
Solves https://issues.apache.org/jira/browse/ZEPPELIN-4783
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4783
### How should this be tested?
Run the docker build and see no warning with the new Key.
Checks that from an inherited docker image we can now apt-get update
without warnings and apt-get install succeed also, so the build of he inherited
file is OK.
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Yann PETIT
Closes #3755 from finalspy/branch-0.9 and squashes the following commits:
1cde03302 [Yann PETIT] [ZEPPELIN-4783] Avoid warning on PUBKEY when apt
update cran.rstudio
(cherry picked from commit f256c4d8207683a111ce552eb7205c981dd92395)
Signed-off-by: Lee moon soo
---
scripts/docker/zeppelin/bin/Dockerfile | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/scripts/docker/zeppelin/bin/Dockerfile
b/scripts/docker/zeppelin/bin/Dockerfile
index 592f61f..6745b46 100644
--- a/scripts/docker/zeppelin/bin/Dockerfile
+++ b/scripts/docker/zeppelin/bin/Dockerfile
@@ -81,8 +81,7 @@ RUN echo "$LOG_TAG Install R related packages" && \
echo "PATH: $PATH" && \
ls /opt/conda/bin && \
echo "deb http://cran.rstudio.com/bin/linux/ubuntu xenial/" | tee -a
/etc/apt/sources.list && \
-gpg --keyserver keyserver.ubuntu.com --recv-key E084DAB9 && \
-gpg -a --export E084DAB9 | apt-key add - && \
+apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 51716619E084DAB9
&& \
apt-get -y update && \
apt-get -y --allow-unauthenticated install r-base r-base-dev && \
R -e "install.packages('evaluate', repos = 'https://cloud.r-project.org')"
&& \
[zeppelin] branch branch-0.9 updated: [ZEPPELIN-4783] Avoid warning on PUBKEY when apt update cran.rstudio
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch branch-0.9
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/branch-0.9 by this push:
new f256c4d [ZEPPELIN-4783] Avoid warning on PUBKEY when apt update
cran.rstudio
f256c4d is described below
commit f256c4d8207683a111ce552eb7205c981dd92395
Author: Yann PETIT
AuthorDate: Wed Apr 29 17:57:54 2020 +0200
[ZEPPELIN-4783] Avoid warning on PUBKEY when apt update cran.rstudio
### What is this PR for?
Solves https://issues.apache.org/jira/browse/ZEPPELIN-4783
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4783
### How should this be tested?
Run the docker build and see no warning with the new Key.
Checks that from an inherited docker image we can now apt-get update
without warnings and apt-get install succeed also, so the build of he inherited
file is OK.
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Yann PETIT
Closes #3755 from finalspy/branch-0.9 and squashes the following commits:
1cde03302 [Yann PETIT] [ZEPPELIN-4783] Avoid warning on PUBKEY when apt
update cran.rstudio
---
scripts/docker/zeppelin/bin/Dockerfile | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/scripts/docker/zeppelin/bin/Dockerfile
b/scripts/docker/zeppelin/bin/Dockerfile
index 592f61f..6745b46 100644
--- a/scripts/docker/zeppelin/bin/Dockerfile
+++ b/scripts/docker/zeppelin/bin/Dockerfile
@@ -81,8 +81,7 @@ RUN echo "$LOG_TAG Install R related packages" && \
echo "PATH: $PATH" && \
ls /opt/conda/bin && \
echo "deb http://cran.rstudio.com/bin/linux/ubuntu xenial/" | tee -a
/etc/apt/sources.list && \
-gpg --keyserver keyserver.ubuntu.com --recv-key E084DAB9 && \
-gpg -a --export E084DAB9 | apt-key add - && \
+apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 51716619E084DAB9
&& \
apt-get -y update && \
apt-get -y --allow-unauthenticated install r-base r-base-dev && \
R -e "install.packages('evaluate', repos = 'https://cloud.r-project.org')"
&& \
svn commit: r39185 - in /release/zeppelin: zeppelin-0.8.0/ zeppelin-0.8.1/
Author: moon Date: Wed Apr 29 23:02:17 2020 New Revision: 39185 Log: Remove 0.8.0 and 0.8.1 release, since 0.8.2 is latest Removed: release/zeppelin/zeppelin-0.8.0/ release/zeppelin/zeppelin-0.8.1/
[zeppelin] branch branch-0.9 updated: [ZEPPELIN-4770] Switch to K8s deployment
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch branch-0.9
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/branch-0.9 by this push:
new cd3c644 [ZEPPELIN-4770] Switch to K8s deployment
cd3c644 is described below
commit cd3c6441777fcf34407a294d58f65ea907000dde
Author: Philipp Dallig
AuthorDate: Wed Apr 22 12:47:44 2020 +0200
[ZEPPELIN-4770] Switch to K8s deployment
### What is this PR for?
We should create a
[k8s-deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/)
for zeppelin-server.
Changes in detail:
- Switch from Pod to Deployment
- Creation of a ServiceAccount `zeppelin-server` + update RoleBinding
- Using a ConfigMap for all Environment variables. I think this ConfigMap
give a better overview over all environment variables
- Label change from `app` to
[`app.kubernetes.io/name`](https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/#labels)
### What type of PR is it?
- Improvement
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-4770
### How should this be tested?
* Install on K8s with `kubectl apply -f k8s/zeppelin-server.yml`
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Philipp Dallig
Closes #3746 from Reamer/k8s_deployment and squashes the following commits:
b8c158a87 [Philipp Dallig] Switch to K8s deployment
(cherry picked from commit d252d0b64a226cabfa6e5b514ca2aef2203c759d)
Signed-off-by: Lee moon soo
---
k8s/zeppelin-server.yaml | 203 ++-
1 file changed, 113 insertions(+), 90 deletions(-)
diff --git a/k8s/zeppelin-server.yaml b/k8s/zeppelin-server.yaml
index 348e1cd..08b3c03 100644
--- a/k8s/zeppelin-server.yaml
+++ b/k8s/zeppelin-server.yaml
@@ -17,7 +17,7 @@
apiVersion: v1
kind: ConfigMap
metadata:
- name: zeppelin-server-conf
+ name: zeppelin-server-conf-map
data:
# 'serviceDomain' is a Domain name to use for accessing Zeppelin UI.
# Should point IP address of 'zeppelin-server' service.
@@ -27,8 +27,21 @@ data:
#
# Default value is 'local.zeppelin-project.org' while it points 127.0.0.1
and `kubectl port-forward zeppelin-server` will give localhost to connects.
# If you have your ingress controller configured to connect to
`zeppelin-server` service and have a domain name for it (with wildcard
subdomain point the same address), you can replace serviceDomain field with
your own domain.
- serviceDomain: local.zeppelin-project.org:8080
- sparkContainerImage: spark:2.4.5
+ SERVICE_DOMAIN: local.zeppelin-project.org:8080
+ ZEPPELIN_K8S_SPARK_CONTAINER_IMAGE: spark:2.4.5
+ ZEPPELIN_K8S_CONTAINER_IMAGE: apache/zeppelin:0.9.0-SNAPSHOT
+ ZEPPELIN_HOME: /zeppelin
+ ZEPPELIN_SERVER_RPC_PORTRANGE: 12320:12320
+ # default value of 'master' property for spark interpreter.
+ MASTER: k8s://https://kubernetes.default.svc
+ # default value of 'SPARK_HOME' property for spark interpreter.
+ SPARK_HOME: /spark
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: zeppelin-server-conf
+data:
nginx.conf: |
daemon off;
worker_processes auto;
@@ -74,96 +87,101 @@ data:
}
}
---
-kind: Pod
-apiVersion: v1
+apiVersion: apps/v1
+kind: Deployment
metadata:
name: zeppelin-server
labels:
-app: zeppelin-server
+app.kubernetes.io/name: zeppelin-server
spec:
- automountServiceAccountToken: true
- containers:
- - name: zeppelin-server
-image: apache/zeppelin:0.9.0-SNAPSHOT
-command: ["sh", "-c", "$(ZEPPELIN_HOME)/bin/zeppelin.sh"]
-lifecycle:
- preStop:
-exec:
- # SIGTERM triggers a quick exit; gracefully terminate instead
- command: ["sh", "-c", "ps -ef | grep
org.apache.zeppelin.server.ZeppelinServer | grep -v grep | awk '{print $2}' |
xargs kill"]
-env:
-- name: ZEPPELIN_K8S_CONTAINER_IMAGE
- value: apache/zeppelin:0.9.0-SNAPSHOT
-- name: ZEPPELIN_HOME
- value: /zeppelin
-- name: ZEPPELIN_SERVER_RPC_PORTRANGE
- value: 12320:12320
-- name: POD_UID
- valueFrom:
-fieldRef:
- apiVersion: v1
- fieldPath: metadata.uid
-- name: POD_NAME
- valueFrom:
-fieldRef:
- apiVersion: v1
- fieldPath: metadata.name
-- name: ZEPPELIN_K8S_SPARK_CONTAINER_IMAGE
- valueFrom:
-configMapKeyRef:
- name: zeppelin-server-conf
- key: sparkContainerImage
-- name: SERVICE_DOMAIN
- valueFrom:
-configMapKeyRef:
+ replicas: 1
+ selector:
+matchLabels:
+ ap
[zeppelin] branch master updated: [ZEPPELIN-4770] Switch to K8s deployment
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new d252d0b [ZEPPELIN-4770] Switch to K8s deployment
d252d0b is described below
commit d252d0b64a226cabfa6e5b514ca2aef2203c759d
Author: Philipp Dallig
AuthorDate: Wed Apr 22 12:47:44 2020 +0200
[ZEPPELIN-4770] Switch to K8s deployment
### What is this PR for?
We should create a
[k8s-deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/)
for zeppelin-server.
Changes in detail:
- Switch from Pod to Deployment
- Creation of a ServiceAccount `zeppelin-server` + update RoleBinding
- Using a ConfigMap for all Environment variables. I think this ConfigMap
give a better overview over all environment variables
- Label change from `app` to
[`app.kubernetes.io/name`](https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/#labels)
### What type of PR is it?
- Improvement
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-4770
### How should this be tested?
* Install on K8s with `kubectl apply -f k8s/zeppelin-server.yml`
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Philipp Dallig
Closes #3746 from Reamer/k8s_deployment and squashes the following commits:
b8c158a87 [Philipp Dallig] Switch to K8s deployment
---
k8s/zeppelin-server.yaml | 203 ++-
1 file changed, 113 insertions(+), 90 deletions(-)
diff --git a/k8s/zeppelin-server.yaml b/k8s/zeppelin-server.yaml
index 348e1cd..08b3c03 100644
--- a/k8s/zeppelin-server.yaml
+++ b/k8s/zeppelin-server.yaml
@@ -17,7 +17,7 @@
apiVersion: v1
kind: ConfigMap
metadata:
- name: zeppelin-server-conf
+ name: zeppelin-server-conf-map
data:
# 'serviceDomain' is a Domain name to use for accessing Zeppelin UI.
# Should point IP address of 'zeppelin-server' service.
@@ -27,8 +27,21 @@ data:
#
# Default value is 'local.zeppelin-project.org' while it points 127.0.0.1
and `kubectl port-forward zeppelin-server` will give localhost to connects.
# If you have your ingress controller configured to connect to
`zeppelin-server` service and have a domain name for it (with wildcard
subdomain point the same address), you can replace serviceDomain field with
your own domain.
- serviceDomain: local.zeppelin-project.org:8080
- sparkContainerImage: spark:2.4.5
+ SERVICE_DOMAIN: local.zeppelin-project.org:8080
+ ZEPPELIN_K8S_SPARK_CONTAINER_IMAGE: spark:2.4.5
+ ZEPPELIN_K8S_CONTAINER_IMAGE: apache/zeppelin:0.9.0-SNAPSHOT
+ ZEPPELIN_HOME: /zeppelin
+ ZEPPELIN_SERVER_RPC_PORTRANGE: 12320:12320
+ # default value of 'master' property for spark interpreter.
+ MASTER: k8s://https://kubernetes.default.svc
+ # default value of 'SPARK_HOME' property for spark interpreter.
+ SPARK_HOME: /spark
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: zeppelin-server-conf
+data:
nginx.conf: |
daemon off;
worker_processes auto;
@@ -74,96 +87,101 @@ data:
}
}
---
-kind: Pod
-apiVersion: v1
+apiVersion: apps/v1
+kind: Deployment
metadata:
name: zeppelin-server
labels:
-app: zeppelin-server
+app.kubernetes.io/name: zeppelin-server
spec:
- automountServiceAccountToken: true
- containers:
- - name: zeppelin-server
-image: apache/zeppelin:0.9.0-SNAPSHOT
-command: ["sh", "-c", "$(ZEPPELIN_HOME)/bin/zeppelin.sh"]
-lifecycle:
- preStop:
-exec:
- # SIGTERM triggers a quick exit; gracefully terminate instead
- command: ["sh", "-c", "ps -ef | grep
org.apache.zeppelin.server.ZeppelinServer | grep -v grep | awk '{print $2}' |
xargs kill"]
-env:
-- name: ZEPPELIN_K8S_CONTAINER_IMAGE
- value: apache/zeppelin:0.9.0-SNAPSHOT
-- name: ZEPPELIN_HOME
- value: /zeppelin
-- name: ZEPPELIN_SERVER_RPC_PORTRANGE
- value: 12320:12320
-- name: POD_UID
- valueFrom:
-fieldRef:
- apiVersion: v1
- fieldPath: metadata.uid
-- name: POD_NAME
- valueFrom:
-fieldRef:
- apiVersion: v1
- fieldPath: metadata.name
-- name: ZEPPELIN_K8S_SPARK_CONTAINER_IMAGE
- valueFrom:
-configMapKeyRef:
- name: zeppelin-server-conf
- key: sparkContainerImage
-- name: SERVICE_DOMAIN
- valueFrom:
-configMapKeyRef:
+ replicas: 1
+ selector:
+matchLabels:
+ app.kubernetes.io/name: zeppelin-server
+ strategy:
+type: RollingUpdate
+ template:
+metadata:
+ labels
[zeppelin] branch master updated: [ZEPPELIN-4760] Add description and homepage information on github repository
This is an automated email from the ASF dual-hosted git repository. moon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/zeppelin.git The following commit(s) were added to refs/heads/master by this push: new 1e85513 [ZEPPELIN-4760] Add description and homepage information on github repository 1e85513 is described below commit 1e855136f53b9d803848da14be8b1d695b880294 Author: Lee moon soo AuthorDate: Wed Apr 22 07:35:24 2020 -0700 [ZEPPELIN-4760] Add description and homepage information on github repository ### What is this PR for? I'd like to discuss and get ideas on how we can improve Zeppelin GitHub repository description, which looks like  ### What type of PR is it? Improvement ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-4760 ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Lee moon soo Closes #3736 from Leemoonsoo/ZEPPELIN-4760 and squashes the following commits: 434408c6c [Lee moon soo] add more labels 33f59a7b9 [Lee moon soo] add .asf.yaml --- .asf.yaml | 32 1 file changed, 32 insertions(+) diff --git a/.asf.yaml b/.asf.yaml new file mode 100644 index 000..99a643f --- /dev/null +++ b/.asf.yaml @@ -0,0 +1,32 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# https://cwiki.apache.org/confluence/display/INFRA/.asf.yaml+features+for+git+repositories + +github: + description: "Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more." + homepage: https://zeppelin.apache.org/ + labels: +- spark +- flink +- database +- nosql +- scala +- java +- big-data +- zeppelin +- javascript
[zeppelin] branch branch-0.9 updated: [ZEPPELIN-4722] User Impersonation via --proxy-user for Spark Interpreter with K8s
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch branch-0.9
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/branch-0.9 by this push:
new 6225868 [ZEPPELIN-4722] User Impersonation via --proxy-user for Spark
Interpreter with K8s
6225868 is described below
commit 62258682c103732a516c36e6ac4a2a22bf800278
Author: Philipp Dallig
AuthorDate: Wed Apr 8 09:39:30 2020 +0200
[ZEPPELIN-4722] User Impersonation via --proxy-user for Spark Interpreter
with K8s
### What is this PR for?
This PullRequest fixes the impersonation issue when running Spark Zeppelin
interpreter on K8s. A general user impersonation should not be necessary and
possible on K8s, since the interpreter process runs unprivileged - without the
right to change the user via ssh.
### What type of PR is it?
* Improvement
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-4722
### How should this be tested?
* **Travic-CI**:
https://travis-ci.org/github/Reamer/zeppelin/builds/674723139
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Philipp Dallig
Closes #3733 from Reamer/proxy_user and squashes the following commits:
8c2e58bb5 [Philipp Dallig] proxy user in spark on k8s
3989c4e87 [Philipp Dallig] Some Cleanup
(cherry picked from commit 85fc206b4f1504dc29da8264b3619ed31006a3b2)
Signed-off-by: Lee moon soo
---
.../zeppelin/conf/ZeppelinConfiguration.java | 5 +
.../launcher/K8sRemoteInterpreterProcess.java | 46
.../launcher/K8sStandardInterpreterLauncher.java | 23 ++--
.../zeppelin/interpreter/launcher/Kubectl.java | 11 +-
.../launcher/K8sRemoteInterpreterProcessTest.java | 117 +++--
.../launcher/SparkInterpreterLauncher.java | 5 +-
6 files changed, 164 insertions(+), 43 deletions(-)
diff --git
a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java
b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java
index 1be243f..3deb8af 100644
---
a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java
+++
b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java
@@ -697,6 +697,10 @@ public class ZeppelinConfiguration extends
XMLConfiguration {
return getString(ConfVars.ZEPPELIN_INTERPRETER_LIFECYCLE_MANAGER_CLASS);
}
+ public boolean getZeppelinImpersonateSparkProxyUser() {
+ return getBoolean(ConfVars.ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER);
+ }
+
public String getZeppelinNotebookGitURL() {
return getString(ConfVars.ZEPPELIN_NOTEBOOK_GIT_REMOTE_URL);
}
@@ -998,6 +1002,7 @@ public class ZeppelinConfiguration extends
XMLConfiguration {
ZEPPELIN_DOCKER_CONTAINER_IMAGE("zeppelin.docker.container.image",
"apache/zeppelin:" + Util.getVersion()),
+
ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER("zeppelin.impersonate.spark.proxy.user",
true),
ZEPPELIN_NOTEBOOK_GIT_REMOTE_URL("zeppelin.notebook.git.remote.url", ""),
ZEPPELIN_NOTEBOOK_GIT_REMOTE_USERNAME("zeppelin.notebook.git.remote.username",
"token"),
ZEPPELIN_NOTEBOOK_GIT_REMOTE_ACCESS_TOKEN("zeppelin.notebook.git.remote.access-token",
""),
diff --git
a/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
b/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
index 959718d..5f872cc 100644
---
a/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
+++
b/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
@@ -13,13 +13,15 @@ import java.util.concurrent.atomic.AtomicBoolean;
import com.hubspot.jinjava.Jinjava;
import org.apache.commons.exec.ExecuteWatchdog;
import org.apache.commons.lang3.ArrayUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.zeppelin.conf.ZeppelinConfiguration;
import org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess;
import org.apache.zeppelin.interpreter.remote.RemoteInterpreterUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class K8sRemoteInterpreterProcess extends RemoteInterpreterProcess {
- private static final Logger LOGGER =
LoggerFactory.getLogger(K8sStandardInterpreterLauncher.class);
+ private static final Logger LOGGER =
LoggerFactory.getLogger(K8sRemoteInterpreterProcess.
[zeppelin] branch master updated: [ZEPPELIN-4722] User Impersonation via --proxy-user for Spark Interpreter with K8s
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 85fc206 [ZEPPELIN-4722] User Impersonation via --proxy-user for Spark
Interpreter with K8s
85fc206 is described below
commit 85fc206b4f1504dc29da8264b3619ed31006a3b2
Author: Philipp Dallig
AuthorDate: Wed Apr 8 09:39:30 2020 +0200
[ZEPPELIN-4722] User Impersonation via --proxy-user for Spark Interpreter
with K8s
### What is this PR for?
This PullRequest fixes the impersonation issue when running Spark Zeppelin
interpreter on K8s. A general user impersonation should not be necessary and
possible on K8s, since the interpreter process runs unprivileged - without the
right to change the user via ssh.
### What type of PR is it?
* Improvement
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-4722
### How should this be tested?
* **Travic-CI**:
https://travis-ci.org/github/Reamer/zeppelin/builds/674723139
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Philipp Dallig
Closes #3733 from Reamer/proxy_user and squashes the following commits:
8c2e58bb5 [Philipp Dallig] proxy user in spark on k8s
3989c4e87 [Philipp Dallig] Some Cleanup
---
.../zeppelin/conf/ZeppelinConfiguration.java | 5 +
.../launcher/K8sRemoteInterpreterProcess.java | 46
.../launcher/K8sStandardInterpreterLauncher.java | 23 ++--
.../zeppelin/interpreter/launcher/Kubectl.java | 11 +-
.../launcher/K8sRemoteInterpreterProcessTest.java | 117 +++--
.../launcher/SparkInterpreterLauncher.java | 5 +-
6 files changed, 164 insertions(+), 43 deletions(-)
diff --git
a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java
b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java
index 1be243f..3deb8af 100644
---
a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java
+++
b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java
@@ -697,6 +697,10 @@ public class ZeppelinConfiguration extends
XMLConfiguration {
return getString(ConfVars.ZEPPELIN_INTERPRETER_LIFECYCLE_MANAGER_CLASS);
}
+ public boolean getZeppelinImpersonateSparkProxyUser() {
+ return getBoolean(ConfVars.ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER);
+ }
+
public String getZeppelinNotebookGitURL() {
return getString(ConfVars.ZEPPELIN_NOTEBOOK_GIT_REMOTE_URL);
}
@@ -998,6 +1002,7 @@ public class ZeppelinConfiguration extends
XMLConfiguration {
ZEPPELIN_DOCKER_CONTAINER_IMAGE("zeppelin.docker.container.image",
"apache/zeppelin:" + Util.getVersion()),
+
ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER("zeppelin.impersonate.spark.proxy.user",
true),
ZEPPELIN_NOTEBOOK_GIT_REMOTE_URL("zeppelin.notebook.git.remote.url", ""),
ZEPPELIN_NOTEBOOK_GIT_REMOTE_USERNAME("zeppelin.notebook.git.remote.username",
"token"),
ZEPPELIN_NOTEBOOK_GIT_REMOTE_ACCESS_TOKEN("zeppelin.notebook.git.remote.access-token",
""),
diff --git
a/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
b/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
index 959718d..5f872cc 100644
---
a/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
+++
b/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
@@ -13,13 +13,15 @@ import java.util.concurrent.atomic.AtomicBoolean;
import com.hubspot.jinjava.Jinjava;
import org.apache.commons.exec.ExecuteWatchdog;
import org.apache.commons.lang3.ArrayUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.zeppelin.conf.ZeppelinConfiguration;
import org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess;
import org.apache.zeppelin.interpreter.remote.RemoteInterpreterUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class K8sRemoteInterpreterProcess extends RemoteInterpreterProcess {
- private static final Logger LOGGER =
LoggerFactory.getLogger(K8sStandardInterpreterLauncher.class);
+ private static final Logger LOGGER =
LoggerFactory.getLogger(K8sRemoteInterpreterProcess.class);
private static final int K8S_INTERPRETER_SERVICE_PORT = 12321;
private final Kubectl kubectl;
private f
[zeppelin] branch master updated: [ZEPPELIN-4748] Format Spark web ui url dynamically on Kubernetes
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new d55c857 [ZEPPELIN-4748] Format Spark web ui url dynamically on
Kubernetes
d55c857 is described below
commit d55c85741ef5833ed89e33128fd0dc42cace9d59
Author: Lee moon soo
AuthorDate: Thu Apr 9 22:46:52 2020 -0700
[ZEPPELIN-4748] Format Spark web ui url dynamically on Kubernetes
### What is this PR for?
When Zeppelin is running on Kubernetes, SparkUI URL should be dynamically
generated, while Kubernetes Service name for Spark interpreter Pod is generated
on runtime. And Ingress controller or reverse-proxy route traffic to SparkUI.
Problem is, depends on those Ingress or reverse proxy configuration,
different SparkUI url format might be required.
Currently, generated url format is hardcoded to
"//-.". And letting user set
'zeppelin.spark.uiWebUrl' with static value doesn't help at all while url is
decided on runtime.
This PR accept [jinja
template](https://jinja.palletsprojects.com/en/2.11.x/) string from
'zeppelin.spark.uiWebUrl' and bind 3 variables 'PORT', 'SERVICE_NAME',
'SERVICE_DOMAIN'. Therefore any URL pattern required by Ingress/Reverse-proxy
can be specified. Each variable has values
* PORT - spark ui port
* SERVICE_NAME -
[Service](https://kubernetes.io/docs/concepts/services-networking/service/)
name for Spark Interpreter Pod.
* SERVICE_DOMAIN - value of SERVICE_DOMAIN env variable.
For example, when spark UI is running on port '4040', Service name for
Spark interpreter pod is 'spark-wcoyqq', SERVICE_DOMAIN is my.domain.io,
```
https://port-{{PORT}}-{{SERVICE_NAME}}.{{SERVICE_DOMAIN}}
```
value on 'zeppelin.spark.uiWebUrl' property will generate Spark UI link
with address
```
https://port-4040-spark-wcoyqq.mydomain.io
```
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4748
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: Lee moon soo
Closes #3728 from Leemoonsoo/ZEPPELIN-4748 and squashes the following
commits:
0103c44c4 [Lee moon soo] update description of the property
42a51b6d5 [Lee moon soo] set classloader for jinjava to prevent
"jinjava.javax.el.ELException: Class
com.hubspot.jinjava.el.ExtendedSyntaxBuilder not found"
c931253d3 [Lee moon soo] template rendering zeppelin.spark.uiWebUrl
property value
---
docs/interpreter/spark.md | 6 +++-
.../src/main/resources/interpreter-setting.json| 2 +-
.../launcher/K8sRemoteInterpreterProcess.java | 30 ++--
.../launcher/K8sRemoteInterpreterProcessTest.java | 42 ++
4 files changed, 76 insertions(+), 4 deletions(-)
diff --git a/docs/interpreter/spark.md b/docs/interpreter/spark.md
index 3c07e01..1f0aed7 100644
--- a/docs/interpreter/spark.md
+++ b/docs/interpreter/spark.md
@@ -184,7 +184,11 @@ You can also set other Spark properties which are not
listed in the table. For a
zeppelin.spark.uiWebUrl
-Overrides Spark UI default URL. Value should be a full URL (ex:
http://{hostName}/{uniquePath}
+
+ Overrides Spark UI default URL. Value should be a full URL (ex:
http://{hostName}/{uniquePath}.
+ In Kubernetes mode, value can be Jinja template string with 3 template
variables 'PORT', 'SERVICE_NAME' and 'SERVICE_DOMAIN'.
+ (ex: http://{{PORT}}-{{SERVICE_NAME}}.{{SERVICE_DOMAIN}})
+
spark.webui.yarn.useProxy
false
diff --git a/spark/interpreter/src/main/resources/interpreter-setting.json
b/spark/interpreter/src/main/resources/interpreter-setting.json
index f879431..5505f4d 100644
--- a/spark/interpreter/src/main/resources/interpreter-setting.json
+++ b/spark/interpreter/src/main/resources/interpreter-setting.json
@@ -109,7 +109,7 @@
"envName": null,
"propertyName": "zeppelin.spark.uiWebUrl",
"defaultValue": "",
-"description": "Override Spark UI default URL",
+"description": "Override Spark UI default URL. In Kubernetes mode,
value can be Jinja template string with 3 template variables 'PORT',
'SERVICE_NAME' and 'SERVICE_DOMAIN'. (ex:
http://{{PORT}}-{{SERVICE_NAME}}.{{SERVICE_DOMAIN}})",
"type": "string"
},
"zeppelin.spark.ui.hidden": {
diff --git
a/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
b/zeppelin-plugins
[zeppelin] branch branch-0.9 updated: [ZEPPELIN-4748] Format Spark web ui url dynamically on Kubernetes
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch branch-0.9
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/branch-0.9 by this push:
new 7521a00 [ZEPPELIN-4748] Format Spark web ui url dynamically on
Kubernetes
7521a00 is described below
commit 7521a00080855d0b23d55c5344e8e6304b9631ca
Author: Lee moon soo
AuthorDate: Thu Apr 9 22:46:52 2020 -0700
[ZEPPELIN-4748] Format Spark web ui url dynamically on Kubernetes
### What is this PR for?
When Zeppelin is running on Kubernetes, SparkUI URL should be dynamically
generated, while Kubernetes Service name for Spark interpreter Pod is generated
on runtime. And Ingress controller or reverse-proxy route traffic to SparkUI.
Problem is, depends on those Ingress or reverse proxy configuration,
different SparkUI url format might be required.
Currently, generated url format is hardcoded to
"//-.". And letting user set
'zeppelin.spark.uiWebUrl' with static value doesn't help at all while url is
decided on runtime.
This PR accept [jinja
template](https://jinja.palletsprojects.com/en/2.11.x/) string from
'zeppelin.spark.uiWebUrl' and bind 3 variables 'PORT', 'SERVICE_NAME',
'SERVICE_DOMAIN'. Therefore any URL pattern required by Ingress/Reverse-proxy
can be specified. Each variable has values
* PORT - spark ui port
* SERVICE_NAME -
[Service](https://kubernetes.io/docs/concepts/services-networking/service/)
name for Spark Interpreter Pod.
* SERVICE_DOMAIN - value of SERVICE_DOMAIN env variable.
For example, when spark UI is running on port '4040', Service name for
Spark interpreter pod is 'spark-wcoyqq', SERVICE_DOMAIN is my.domain.io,
```
https://port-{{PORT}}-{{SERVICE_NAME}}.{{SERVICE_DOMAIN}}
```
value on 'zeppelin.spark.uiWebUrl' property will generate Spark UI link
with address
```
https://port-4040-spark-wcoyqq.mydomain.io
```
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4748
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: Lee moon soo
Closes #3728 from Leemoonsoo/ZEPPELIN-4748 and squashes the following
commits:
0103c44c4 [Lee moon soo] update description of the property
42a51b6d5 [Lee moon soo] set classloader for jinjava to prevent
"jinjava.javax.el.ELException: Class
com.hubspot.jinjava.el.ExtendedSyntaxBuilder not found"
c931253d3 [Lee moon soo] template rendering zeppelin.spark.uiWebUrl
property value
(cherry picked from commit d55c85741ef5833ed89e33128fd0dc42cace9d59)
Signed-off-by: Lee moon soo
---
docs/interpreter/spark.md | 6 +++-
.../src/main/resources/interpreter-setting.json| 2 +-
.../launcher/K8sRemoteInterpreterProcess.java | 30 ++--
.../launcher/K8sRemoteInterpreterProcessTest.java | 42 ++
4 files changed, 76 insertions(+), 4 deletions(-)
diff --git a/docs/interpreter/spark.md b/docs/interpreter/spark.md
index 3c07e01..1f0aed7 100644
--- a/docs/interpreter/spark.md
+++ b/docs/interpreter/spark.md
@@ -184,7 +184,11 @@ You can also set other Spark properties which are not
listed in the table. For a
zeppelin.spark.uiWebUrl
-Overrides Spark UI default URL. Value should be a full URL (ex:
http://{hostName}/{uniquePath}
+
+ Overrides Spark UI default URL. Value should be a full URL (ex:
http://{hostName}/{uniquePath}.
+ In Kubernetes mode, value can be Jinja template string with 3 template
variables 'PORT', 'SERVICE_NAME' and 'SERVICE_DOMAIN'.
+ (ex: http://{{PORT}}-{{SERVICE_NAME}}.{{SERVICE_DOMAIN}})
+
spark.webui.yarn.useProxy
false
diff --git a/spark/interpreter/src/main/resources/interpreter-setting.json
b/spark/interpreter/src/main/resources/interpreter-setting.json
index f879431..5505f4d 100644
--- a/spark/interpreter/src/main/resources/interpreter-setting.json
+++ b/spark/interpreter/src/main/resources/interpreter-setting.json
@@ -109,7 +109,7 @@
"envName": null,
"propertyName": "zeppelin.spark.uiWebUrl",
"defaultValue": "",
-"description": "Override Spark UI default URL",
+"description": "Override Spark UI default URL. In Kubernetes mode,
value can be Jinja template string with 3 template variables 'PORT',
'SERVICE_NAME' and 'SERVICE_DOMAIN'. (ex:
http://{{PORT}}-{{SERVICE_NAME}}.{{SERVICE_DOMAIN}})",
"type": "string"
},
"zeppelin.spark.ui.hidden": {
diff --git
a/zeppelin-plugins/launcher/k8s
[zeppelin] branch master updated: pip install with wrong parameter format
This is an automated email from the ASF dual-hosted git repository. moon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/zeppelin.git The following commit(s) were added to refs/heads/master by this push: new ba92ffb pip install with wrong parameter format ba92ffb is described below commit ba92ffbb1b85298f8d3b4b588ceff547db40bcaa Author: Hchen AuthorDate: Mon Mar 30 14:38:35 2020 +0800 pip install with wrong parameter format maybe misspelling(`intake-parquet=0.2.2 altair=3.2.0` should be `intake-parquet==0.2.2 altair==3.2.0`) Author: Hchen Closes #3705 from pzghost/typo and squashes the following commits: f8ea50c0c [Hchen] typo (cherry picked from commit 1df4adf8014fe0e0036b5b60817ed260114e7c4e) Signed-off-by: Lee moon soo --- scripts/docker/zeppelin/bin/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/docker/zeppelin/bin/Dockerfile b/scripts/docker/zeppelin/bin/Dockerfile index 9b2cdfc..592f61f 100644 --- a/scripts/docker/zeppelin/bin/Dockerfile +++ b/scripts/docker/zeppelin/bin/Dockerfile @@ -75,7 +75,7 @@ RUN echo "$LOG_TAG Install python related packages" && \ conda info -a && \ conda config --add channels conda-forge && \ pip install -q pycodestyle==2.5.0 && \ -pip install -q numpy==1.17.3 pandas==0.25.0 scipy==1.3.1 grpcio==1.19.0 bkzep==0.6.1 hvplot==0.5.2 protobuf==3.10.0 pandasql==0.7.3 ipython==7.8.0 matplotlib==3.0.3 ipykernel==5.1.2 jupyter_client==5.3.4 bokeh==1.3.4 panel==0.6.0 holoviews==1.12.3 seaborn==0.9.0 plotnine==0.5.1 intake==0.5.3 intake-parquet=0.2.2 altair=3.2.0 pycodestyle==2.5.0 apache_beam==2.15.0 +pip install -q numpy==1.17.3 pandas==0.25.0 scipy==1.3.1 grpcio==1.19.0 bkzep==0.6.1 hvplot==0.5.2 protobuf==3.10.0 pandasql==0.7.3 ipython==7.8.0 matplotlib==3.0.3 ipykernel==5.1.2 jupyter_client==5.3.4 bokeh==1.3.4 panel==0.6.0 holoviews==1.12.3 seaborn==0.9.0 plotnine==0.5.1 intake==0.5.3 intake-parquet==0.2.2 altair==3.2.0 pycodestyle==2.5.0 apache_beam==2.15.0 RUN echo "$LOG_TAG Install R related packages" && \ echo "PATH: $PATH" && \
[zeppelin] branch branch-0.9 updated: pip install with wrong parameter format
This is an automated email from the ASF dual-hosted git repository. moon pushed a commit to branch branch-0.9 in repository https://gitbox.apache.org/repos/asf/zeppelin.git The following commit(s) were added to refs/heads/branch-0.9 by this push: new 1df4adf pip install with wrong parameter format 1df4adf is described below commit 1df4adf8014fe0e0036b5b60817ed260114e7c4e Author: Hchen AuthorDate: Mon Mar 30 14:38:35 2020 +0800 pip install with wrong parameter format maybe misspelling(`intake-parquet=0.2.2 altair=3.2.0` should be `intake-parquet==0.2.2 altair==3.2.0`) Author: Hchen Closes #3705 from pzghost/typo and squashes the following commits: f8ea50c0c [Hchen] typo --- scripts/docker/zeppelin/bin/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/docker/zeppelin/bin/Dockerfile b/scripts/docker/zeppelin/bin/Dockerfile index 9b2cdfc..592f61f 100644 --- a/scripts/docker/zeppelin/bin/Dockerfile +++ b/scripts/docker/zeppelin/bin/Dockerfile @@ -75,7 +75,7 @@ RUN echo "$LOG_TAG Install python related packages" && \ conda info -a && \ conda config --add channels conda-forge && \ pip install -q pycodestyle==2.5.0 && \ -pip install -q numpy==1.17.3 pandas==0.25.0 scipy==1.3.1 grpcio==1.19.0 bkzep==0.6.1 hvplot==0.5.2 protobuf==3.10.0 pandasql==0.7.3 ipython==7.8.0 matplotlib==3.0.3 ipykernel==5.1.2 jupyter_client==5.3.4 bokeh==1.3.4 panel==0.6.0 holoviews==1.12.3 seaborn==0.9.0 plotnine==0.5.1 intake==0.5.3 intake-parquet=0.2.2 altair=3.2.0 pycodestyle==2.5.0 apache_beam==2.15.0 +pip install -q numpy==1.17.3 pandas==0.25.0 scipy==1.3.1 grpcio==1.19.0 bkzep==0.6.1 hvplot==0.5.2 protobuf==3.10.0 pandasql==0.7.3 ipython==7.8.0 matplotlib==3.0.3 ipykernel==5.1.2 jupyter_client==5.3.4 bokeh==1.3.4 panel==0.6.0 holoviews==1.12.3 seaborn==0.9.0 plotnine==0.5.1 intake==0.5.3 intake-parquet==0.2.2 altair==3.2.0 pycodestyle==2.5.0 apache_beam==2.15.0 RUN echo "$LOG_TAG Install R related packages" && \ echo "PATH: $PATH" && \
[zeppelin] branch master updated: [ZEPPELIN-4619] Run a note from the commandline
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 7c7d51a [ZEPPELIN-4619] Run a note from the commandline
7c7d51a is described below
commit 7c7d51a2d5876456fe1a7eba2f985454213e0e32
Author: Lee moon soo
AuthorDate: Thu Feb 27 17:34:45 2020 -0800
[ZEPPELIN-4619] Run a note from the commandline
### What is this PR for?
This PR adds a command-line option to run a single note.
The code is pickled from https://github.com/apache/zeppelin/pull/3356.
Usage is
```
bin/zeppelin.sh --run
```
The command will launch a Zeppelin server and run a given note, and
terminate on after all paragraph's successful run (with exit 0) or terminate on
any paragraph error (with exit 1).
### What type of PR is it?
Feature
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4619
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
This patch had conflicts when merged, resolved by
Committer: Lee moon soo
Closes #3654 from Leemoonsoo/ZEPPELIN-4619 and squashes the following
commits:
6a8316388 [Lee moon soo] remove useless test
a7531a49f [Lee moon soo] address comment
7743b43f6 [Lee moon soo] add log
99974f89d [Lee moon soo] overload loading runAllParagraphs()
101d1557b [Lee moon soo] add --run option
---
bin/zeppelin.sh| 34 +
.../zeppelin/conf/ZeppelinConfiguration.java | 22 ++
.../org/apache/zeppelin/server/ZeppelinServer.java | 87 +-
.../apache/zeppelin/service/NotebookService.java | 33 ++--
.../zeppelin/service/NotebookServiceTest.java | 18 +++--
5 files changed, 149 insertions(+), 45 deletions(-)
diff --git a/bin/zeppelin.sh b/bin/zeppelin.sh
index bf5aaba..9f32fb4 100755
--- a/bin/zeppelin.sh
+++ b/bin/zeppelin.sh
@@ -36,20 +36,26 @@ if [ -z "$uidentry" ] ; then
fi
fi
-USAGE="Usage: bin/zeppelin.sh [--config ]"
-
-if [[ "$1" == "--config" ]]; then
- shift
- conf_dir="$1"
- if [[ ! -d "${conf_dir}" ]]; then
-echo "ERROR : ${conf_dir} is not a directory"
-echo ${USAGE}
-exit 1
- else
-export ZEPPELIN_CONF_DIR="${conf_dir}"
- fi
- shift
-fi
+USAGE="Usage: bin/zeppelin.sh [--config ] [--run ]"
+
+POSITIONAL=()
+while [[ $# -gt 0 ]]
+do
+ key="$1"
+ case $key in
+--config)
+export ZEPPELIN_CONF_DIR="$2"
+shift # past argument
+shift # past value
+;;
+--run)
+export ZEPPELIN_NOTEBOOK_RUN_ID="$2"
+shift # past argument
+shift # past value
+;;
+ esac
+done
+set -- "${POSITIONAL[@]}" # restore positional parameters
bin=$(dirname "${BASH_SOURCE-$0}")
bin=$(cd "${bin}">/dev/null; pwd)
diff --git
a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java
b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java
index 78fdb86..43f9ec7 100644
---
a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java
+++
b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java
@@ -397,6 +397,22 @@ public class ZeppelinConfiguration extends
XMLConfiguration {
return getString(ConfVars.ZEPPELIN_NOTEBOOK_DIR);
}
+ public String getNotebookRunId() {
+return getString(ConfVars.ZEPPELIN_NOTEBOOK_RUN_ID);
+ }
+
+ public String getNotebookRunRev() {
+return getString(ConfVars.ZEPPELIN_NOTEBOOK_RUN_REV);
+ }
+
+ public String getNotebookRunServiceContext() {
+return getString(ConfVars.ZEPPELIN_NOTEBOOK_RUN_SERVICE_CONTEXT);
+ }
+
+ public boolean getNotebookRunAutoShutdown() {
+return getBoolean(ConfVars.ZEPPELIN_NOTEBOOK_RUN_AUTOSHUTDOWN);
+ }
+
public String getPluginsDir() {
return getRelativeDir(getString(ConfVars.ZEPPELIN_PLUGINS_DIR));
}
@@ -831,6 +847,12 @@ public class ZeppelinConfiguration extends
XMLConfiguration {
ZEPPELIN_INTERPRETER_OUTPUT_LIMIT("zeppelin.interpreter.output.limit",
1024 * 100),
ZEPPELIN_ENCODING("zeppelin.encoding", "UTF-8"),
ZEPPELIN_NOTEBOOK_DIR("zeppelin.notebook.dir", "notebook"),
+
+ZEPPELIN_NOTEBOOK_RUN_ID("zeppelin.notebook.run.id", null), // run
particular note id on zeppelin start
+ZEPPELIN_NOTEBOOK_RUN_REV("zeppelin.notebook.run.rev", null), // revision
id for ZEPPELIN_NOTEBOOK_RUN_ID.
+
ZEPPELIN_NOTEBOOK_RUN_SERVICE_CONTEXT("zeppelin.no
[zeppelin] branch master updated: [ZEPPELIN-4568] Fix can not update output when the data is empty
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new b0e220e [ZEPPELIN-4568] Fix can not update output when the data is
empty
b0e220e is described below
commit b0e220e6dbbf1a54cbce98f290d593b49e97bc23
Author: Hsuan Lee
AuthorDate: Mon Feb 10 17:01:24 2020 +0800
[ZEPPELIN-4568] Fix can not update output when the data is empty
### What is this PR for?
Fix can not update output when the data is empty
### What type of PR is it?
[Bug Fix]
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4568
### How should this be tested?
Try the following code:
```pythone
import sys
import time
from IPython.display import display, clear_output
for i in range(10):
time.sleep(0.25)
clear_output(wait=True)
print(i)
sys.stdout.flush()
```
Should be updated instead of append the output
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Hsuan Lee
Closes #3635 from hsuanxyz/fix/clean-out and squashes the following commits:
df9c6f8ac [Hsuan Lee] fix: can not update output when the data is empty
---
zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js| 2 +-
.../src/app/notebook/paragraph/result/result.controller.js | 7 ++-
zeppelin-web/src/app/spell/spell-result.js | 2 +-
3 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
b/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
index f0e5d7f..a303b0f 100644
--- a/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
+++ b/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
@@ -212,7 +212,7 @@ function ParagraphCtrl($scope, $rootScope, $route, $window,
$routeParams, $locat
$scope.paragraph.results.msg = [];
}
- let update = ($scope.paragraph.results.msg[data.index]) ? true : false;
+ let update = typeof $scope.paragraph.results.msg[data.index] !==
'undefined';
$scope.paragraph.results.msg[data.index] = {
data: data.data,
diff --git
a/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
index dc0fdbf..dc6b0bb 100644
--- a/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
+++ b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
@@ -216,7 +216,12 @@ function ResultCtrl($scope, $rootScope, $route, $window,
$routeParams, $location
callback(elem);
}
-$timeout(retry);
+if(isDOMLoaded(targetElemId)) {
+ const elem = angular.element(`#${targetElemId}`);
+ callback(elem);
+} else {
+ $timeout(retry);
+}
}
$scope.$on('updateResult', function(event, result, newConfig, paragraphRef,
index) {
diff --git a/zeppelin-web/src/app/spell/spell-result.js
b/zeppelin-web/src/app/spell/spell-result.js
index 52bcdc1..93d71ed 100644
--- a/zeppelin-web/src/app/spell/spell-result.js
+++ b/zeppelin-web/src/app/spell/spell-result.js
@@ -217,7 +217,7 @@ export class SpellResult {
}
static isObject(data) {
-return (data &&
+return (typeof data !== 'undefined' &&
!SpellResult.isFunction(data) &&
!SpellResult.isPromise(data));
}
[zeppelin] branch master updated: [ZEPPELIN-4618] Zeppelin on Kubernetes does not automatically configure Spark on Kubernetes
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 6d55732 [ZEPPELIN-4618] Zeppelin on Kubernetes does not automatically
configure Spark on Kubernetes
6d55732 is described below
commit 6d55732150bd36f2017ebc74896a4a5e09e3abc7
Author: Lee moon soo
AuthorDate: Thu Feb 13 16:14:39 2020 -0800
[ZEPPELIN-4618] Zeppelin on Kubernetes does not automatically configure
Spark on Kubernetes
### What is this PR for?
Since https://github.com/apache/zeppelin/pull/3577, Zeppelin on Kubernetes
does not automatically configure Spark on Kubernetes.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4618
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3648 from Leemoonsoo/fix_spark_on_kubernetes and squashes the
following commits:
5dfe0010d [Lee moon soo] fix spark on kubernetes
---
k8s/zeppelin-server.yaml | 6 --
spark/interpreter/src/main/resources/interpreter-setting.json | 2 +-
2 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/k8s/zeppelin-server.yaml b/k8s/zeppelin-server.yaml
index ecb7606..348e1cd 100644
--- a/k8s/zeppelin-server.yaml
+++ b/k8s/zeppelin-server.yaml
@@ -28,7 +28,7 @@ data:
# Default value is 'local.zeppelin-project.org' while it points 127.0.0.1
and `kubectl port-forward zeppelin-server` will give localhost to connects.
# If you have your ingress controller configured to connect to
`zeppelin-server` service and have a domain name for it (with wildcard
subdomain point the same address), you can replace serviceDomain field with
your own domain.
serviceDomain: local.zeppelin-project.org:8080
- sparkContainerImage: spark:2.4.0
+ sparkContainerImage: spark:2.4.5
nginx.conf: |
daemon off;
worker_processes auto;
@@ -118,8 +118,10 @@ spec:
configMapKeyRef:
name: zeppelin-server-conf
key: serviceDomain
-- name: MASTER # default value of master property for spark interpreter.
+- name: MASTER # default value of 'master' property for spark
interpreter.
value: k8s://https://kubernetes.default.svc
+- name: SPARK_HOME # default value of 'SPARK_HOME' property for spark
interpreter.
+ value: /spark
# volumeMounts:
# - name: zeppelin-server-notebook-volume # configure this to persist
notebook
#mountPath: /zeppelin/notebook
diff --git a/spark/interpreter/src/main/resources/interpreter-setting.json
b/spark/interpreter/src/main/resources/interpreter-setting.json
index 100c97c..d8549b3 100644
--- a/spark/interpreter/src/main/resources/interpreter-setting.json
+++ b/spark/interpreter/src/main/resources/interpreter-setting.json
@@ -13,7 +13,7 @@
"type": "string"
},
"master": {
-"envName": "",
+"envName": "MASTER",
"propertyName": "spark.master",
"defaultValue": "local[*]",
"description": "Spark master uri. ex) spark://master_host:7077",
[zeppelin] branch master updated: [ZEPPELIN-4373]. Specify ZEPPELIN_ADDR to 0.0.0.0 in Dockerfile
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new a15b8f6 [ZEPPELIN-4373]. Specify ZEPPELIN_ADDR to 0.0.0.0 in
Dockerfile
a15b8f6 is described below
commit a15b8f62c3d55197b798c581594a91e9e5220b32
Author: Jeff Zhang
AuthorDate: Thu Oct 10 21:17:26 2019 +0800
[ZEPPELIN-4373]. Specify ZEPPELIN_ADDR to 0.0.0.0 in Dockerfile
### What is this PR for?
Straightforward change to specify env `ZEPPELIN_ADDR` to `0.0.0.0` to
Dockerfile
### What type of PR is it?
[Improvement ]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-4373
### How should this be tested?
* Build docker image, and verify it manually.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang
Closes #3512 from zjffdu/ZEPPELIN-4373 and squashes the following commits:
d7a4fbd56 [Jeff Zhang] [ZEPPELIN-4373]. Specify ZEPPELIN_ADDR to 0.0.0.0 to
Dockerfile
---
scripts/docker/zeppelin/bin/Dockerfile | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/scripts/docker/zeppelin/bin/Dockerfile
b/scripts/docker/zeppelin/bin/Dockerfile
index 33623db..16186db 100644
--- a/scripts/docker/zeppelin/bin/Dockerfile
+++ b/scripts/docker/zeppelin/bin/Dockerfile
@@ -21,10 +21,12 @@ ARG ZEPPELIN_GROUP_ID=2100
# `Z_VERSION` will be updated by `dev/change_zeppelin_version.sh`
ENV Z_VERSION="0.9.0-SNAPSHOT"
+
ENV LOG_TAG="[ZEPPELIN_${Z_VERSION}]:" \
Z_HOME="/zeppelin" \
LANG=en_US.UTF-8 \
-LC_ALL=en_US.UTF-8
+LC_ALL=en_US.UTF-8 \
+ZEPPELIN_ADDR="0.0.0.0"
RUN groupadd --gid $ZEPPELIN_GROUP_ID zeppelin \
&& useradd -ms /bin/bash -d ${Z_HOME} zeppelin --uid $ZEPPELIN_USER_ID
--gid $ZEPPELIN_GROUP_ID
[zeppelin] branch master updated: [ZEPPELIN-4347]: add seperate shortcuts for run all above/below
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 54ce30f [ZEPPELIN-4347]: add seperate shortcuts for run all
above/below
54ce30f is described below
commit 54ce30f4927e228397d38cb86e616f353619f312
Author: santosh1994
AuthorDate: Wed Oct 16 10:48:25 2019 +0530
[ZEPPELIN-4347]: add seperate shortcuts for run all above/below
### What is this PR for?
Currently same shortcut is configured for run all above and run all below,
clicking on shortcut will again show popup to choose run all above or run all
below. With this additional action required from user it doesn't makes sense to
use shortcut.
### What type of PR is it?
Improvement
### Todos
* [ ] - Task
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4347
### How should this be tested?
* Ctrl + Shift + UP shortcut should run all the paragraphs above
* Ctrl + Shift + Down shortcut should run all the paragraphs below and the
current one
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update?
* Is there breaking changes for older versions?
* Does this needs documentation?
Author: santosh1994
Closes #3458 from santosh1994/ZEPPELIN-4347 and squashes the following
commits:
c9c932252 [santosh1994] [ZEPPELIN-4347]: add seperate shortcuts for run all
above/below
---
.../app/notebook/paragraph/paragraph.controller.js | 47 --
zeppelin-web/src/app/notebook/shortcut.html| 15 ++-
2 files changed, 30 insertions(+), 32 deletions(-)
diff --git a/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
b/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
index 8014fb6..c7b7f3f 100644
--- a/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
+++ b/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
@@ -509,33 +509,18 @@ function ParagraphCtrl($scope, $rootScope, $route,
$window, $routeParams, $locat
$scope.$emit('runAllBelowAndCurrent', paragraph, true);
};
- $scope.runAllToOrFromThis = function(paragraph) {
-BootstrapDialog.show({
- message: 'Run paragraphs:',
- title: '',
- buttons: [{
-label: 'Close',
-action: function(dialog) {
- dialog.close();
-},
- },
- {
-label: 'Run all above',
-cssClass: 'btn-primary',
-action: function(dialog) {
- $scope.$emit('runAllAbove', paragraph, false);
- dialog.close();
-},
- },
- {
-label: 'Run current and all below',
-cssClass: 'btn-primary',
-action: function(dialog) {
- $scope.$emit('runAllBelowAndCurrent', paragraph, false);
- dialog.close();
-},
- }],
-});
+ $scope.runAllFromThisFromShortcut = function(paragraph) {
+if ($scope.isNoteRunning) {
+ return;
+}
+$scope.$emit('runAllBelowAndCurrent', paragraph, false);
+ };
+
+ $scope.runAllToThisFromShortcut = function(paragraph) {
+if ($scope.isNoteRunning) {
+ return;
+}
+$scope.$emit('runAllAbove', paragraph, false);
};
$scope.turnOnAutoRun = function(paragraph) {
@@ -1686,9 +1671,11 @@ function ParagraphCtrl($scope, $rootScope, $route,
$window, $routeParams, $locat
$timeout(() => $scope.$emit('moveFocusToNextParagraph', paragraphId));
} else if (!keyEvent.ctrlKey && keyEvent.shiftKey && keyCode === 13) {
// Shift + Enter
$scope.runParagraphFromShortcut($scope.getEditorValue());
- } else if (keyEvent.ctrlKey && keyEvent.shiftKey && keyCode === 13) { //
Ctrl + Shift + Enter
-$scope.runAllToOrFromThis($scope.paragraph);
- } else if (keyEvent.ctrlKey && keyEvent.altKey && keyCode === 67) { //
Ctrl + Alt + c
+ } else if (keyEvent.ctrlKey && keyEvent.shiftKey && keyCode === 38) { //
Ctrl + Shift + UP
+$scope.runAllToThisFromShortcut($scope.paragraph);
+ } else if (keyEvent.ctrlKey && keyEvent.shiftKey && keyCode === 40) { //
Ctrl + Shift + Down
+$scope.runAllFromThisFromShortcut($scope.paragraph);
+ }else if (keyEvent.ctrlKey && keyEvent.altKey && keyCode === 67) { //
Ctrl + Alt + c
$scope.cancelParagraph($scope.paragraph);
} else if (keyEvent.ctrlKey && keyEvent.altKey && keyCode === 68) { //
Ctrl + Alt + d
$scope.removeParagraph($scope.paragraph);
diff --git a/zeppelin-web/src/app/notebook/shortcut.html
b/zeppelin-web/src/app/notebook/shortcut.html
index 9bc5597..cfa57f8 100644
--- a/zeppelin-web/src/app/notebook/shortcut.html
+++ b/zeppelin-web/src/app/notebook/
[zeppelin] branch master updated: [ZEPPELIN-4320] fix packages to bundle list
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new a69d284 [ZEPPELIN-4320] fix packages to bundle list
a69d284 is described below
commit a69d284e4552f961809f6ce11aca95fd732bc8af
Author: santosh1994
AuthorDate: Wed Oct 16 10:49:51 2019 +0530
[ZEPPELIN-4320] fix packages to bundle list
### What is this PR for?
Enabled Helium packages are not getting installed/bundled on zeppelin
startup. Instead getting bundled later on paragraph with bundle dependency are
loaded.
### What type of PR is it?
Bug Fix
### Todos
* [ ] - Task
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4320
### How should this be tested?
* Enable some helium packages
* Remove helium cache folder(local-repo)
* Restart zeppelin and verify, enabled packages are getting bundled on
zeppelin server start
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update?
* Is there breaking changes for older versions?
* Does this needs documentation?
Author: santosh1994
Closes #3438 from santosh1994/ZEPPELIN-4320 and squashes the following
commits:
a805d4d70 [santosh1994] [ZEPPELIN-4320] fix packages to bundle list
---
zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/Helium.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/Helium.java
b/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/Helium.java
index e0a3845..9bd3bce 100644
--- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/Helium.java
+++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/Helium.java
@@ -454,7 +454,7 @@ public class Helium {
}
}
}
-new LinkedList<>().addAll(orderedBundlePackages);
+output.addAll(orderedBundlePackages);
return output;
}
[zeppelin] branch master updated: [ZEPPELIN-4319] Fix spell interpreter parse text logic
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new d31ae2e [ZEPPELIN-4319] Fix spell interpreter parse text logic
d31ae2e is described below
commit d31ae2e8a568a14c6e518d7510955fa664c029bf
Author: santoshk_qubole
AuthorDate: Mon Oct 14 10:49:41 2019 +0530
[ZEPPELIN-4319] Fix spell interpreter parse text logic
### What is this PR for?
Spell interpreter returns incorrect results when paragraph having magic in
between text, with current approach to get text without magic
### What type of PR is it?
Bug Fix
### Todos
* [ ] - Task
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4319
### How should this be tested?
* Enable zeppelin-markdown-spell
* Run %markdown paragraph with containing %markdown as text within it
* Verify if result is valid and complete
### Screenshots (if appropriate)
Before:
After:
### Questions:
* Does the licenses files need update?
* Is there breaking changes for older versions?
* Does this needs documentation?
Author: santoshk_qubole
Closes #3437 from santosh1994/ZEPPELIN-4319 and squashes the following
commits:
416257843 [santoshk_qubole] Merge remote-tracking branch 'origin/master'
into ZEPPELIN-4319
5245ca804 [santoshk_qubole] Merge remote-tracking branch 'origin/master'
into ZEPPELIN-4319
9c61cb8f6 [santoshk_qubole] [ZEPPELIN-4319] Fix spell interpreter parse
text logic
---
zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
b/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
index 5f01c7e..8014fb6 100644
--- a/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
+++ b/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
@@ -374,9 +374,9 @@ function ParagraphCtrl($scope, $rootScope, $route, $window,
$routeParams, $locat
try {
// remove magic from paragraphText
- const splited = paragraphText.split(magic);
+ const splited = paragraphText.slice(paragraphText.indexOf(magic) +
magic.length);
// remove leading spaces
- const textWithoutMagic = splited[1].replace(/^\s+/g, '');
+ const textWithoutMagic = splited.replace(/^\s+/g, '');
if (!propagated) {
$scope.paragraph.dateStarted = $scope.getFormattedParagraphTime();
svn commit: r1868210 - /zeppelin/site/.htaccess
Author: moon Date: Wed Oct 9 23:58:26 2019 New Revision: 1868210 URL: http://svn.apache.org/viewvc?rev=1868210=rev Log: https://github.com/apache/zeppelin/pull/3470 Modified: zeppelin/site/.htaccess Modified: zeppelin/site/.htaccess URL: http://svn.apache.org/viewvc/zeppelin/site/.htaccess?rev=1868210=1868209=1868210=diff == --- zeppelin/site/.htaccess (original) +++ zeppelin/site/.htaccess Wed Oct 9 23:58:26 2019 @@ -7,7 +7,7 @@ RewriteRule ^/?docs/0.6.0-(?!SNAPSHOT).* RewriteRule ^/?docs/(0.5.[056])(?!-incubating).*/(.*) /docs/$1-incubating/$3 [R=301,L,NE] # rewrite docs/latest to latest stable release -RewriteRule ^/?docs/latest/(.*) /docs/0.8.1/$1 [PT] +RewriteRule ^/?docs/latest/(.*) /docs/0.8.2/$1 [PT] # rewrite docs/snapshot to latest snapshot version RewriteRule ^/?docs/snapshot/(.*) /docs/0.9.0-SNAPSHOT/$1 [PT]
[zeppelin] branch master updated: [ZEPPELIN-4355][ZEPPELIN-4361] Fix broken tests in master branch
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 238e032 [ZEPPELIN-4355][ZEPPELIN-4361] Fix broken tests in master
branch
238e032 is described below
commit 238e03242b5ed108f99b5624d93fa5606b008d1f
Author: Lee moon soo
AuthorDate: Tue Oct 1 10:07:56 2019 -0700
[ZEPPELIN-4355][ZEPPELIN-4361] Fix broken tests in master branch
### What is this PR for?
Fix broken test in master branch
[ZEPPELIN-4355](https://issues.apache.org/jira/projects/ZEPPELIN/issues/ZEPPELIN-4355),
[ZEPPELIN-4361](https://issues.apache.org/jira/projects/ZEPPELIN/issues/ZEPPELIN-4361).
### What type of PR is it?
Hot Fix
### What is the Jira issue?
https://issues.apache.org/jira/projects/ZEPPELIN/issues/ZEPPELIN-4355
https://issues.apache.org/jira/projects/ZEPPELIN/issues/ZEPPELIN-4361
### How should this be tested?
* All CI test should pass
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3469 from Leemoonsoo/ZEPPELIN-4361 and squashes the following
commits:
0eb8a9666 [Lee moon soo] ZEPPELIN-4361
da5755b55 [Lee moon soo] ZEPPELIN-4355
---
.../cassandra/CassandraInterpreterTest.java| 1 +
.../zeppelin/integration/PersonalizeActionsIT.java | 38 --
.../src/app/notebook/notebook.controller.js| 2 +-
3 files changed, 30 insertions(+), 11 deletions(-)
diff --git
a/cassandra/src/test/java/org/apache/zeppelin/cassandra/CassandraInterpreterTest.java
b/cassandra/src/test/java/org/apache/zeppelin/cassandra/CassandraInterpreterTest.java
index df5e4ac..0417dc9 100644
---
a/cassandra/src/test/java/org/apache/zeppelin/cassandra/CassandraInterpreterTest.java
+++
b/cassandra/src/test/java/org/apache/zeppelin/cassandra/CassandraInterpreterTest.java
@@ -773,6 +773,7 @@ public class CassandraInterpreterTest {
.replaceAll(">\\s+<", "><")
.replaceAll("(?s)data-target=\"#[a-f0-9-]+(?:_asCQL|_indices_asCQL)?\"", "")
.replaceAll("(?s)id=\"[a-f0-9-]+(?:_asCQL|_indices_asCQL)?\"", "")
+.replaceAll("AND memtable_flush_period_in_ms = 0", "")
.trim();
}
diff --git
a/zeppelin-integration/src/test/java/org/apache/zeppelin/integration/PersonalizeActionsIT.java
b/zeppelin-integration/src/test/java/org/apache/zeppelin/integration/PersonalizeActionsIT.java
index fce9466..1fc8890 100644
---
a/zeppelin-integration/src/test/java/org/apache/zeppelin/integration/PersonalizeActionsIT.java
+++
b/zeppelin-integration/src/test/java/org/apache/zeppelin/integration/PersonalizeActionsIT.java
@@ -201,10 +201,10 @@ public class PersonalizeActionsIT extends
AbstractZeppelinIT {
createNewNote();
}
String noteId =
driver.getCurrentUrl().substring(driver.getCurrentUrl().lastIndexOf("/") + 1);
- setTextOfParagraph(1, "print(s\"\"\"%table\\n" +
- "name\\tsize\\n" +
- "sun\\t100\\n" +
- "moon\\t10\"\"\")");
+ setTextOfParagraph(1, "%python print(\"%table " +
+ "nametsizen" +
+ "sunt100n" +
+ "moont10\")");
runParagraph(1);
try {
@@ -215,6 +215,11 @@ public class PersonalizeActionsIT extends
AbstractZeppelinIT {
"ERROR", CoreMatchers.equalTo("FINISHED"));
}
+ pollingWait(By.xpath("//*[@id='actionbar']" +
+ "//button[contains(@uib-tooltip, 'Switch to personal mode')]"),
MAX_BROWSER_TIMEOUT_SEC).click();
+ clickAndWait(By.xpath("//div[@class='modal-dialog'][contains(.,'Do you
want to personalize your analysis?')" +
+ "]//div[@class='modal-footer']//button[contains(.,'OK')]"));
+
pollingWait(By.xpath(getParagraphXPath(1) +
"//button[contains(@uib-tooltip, 'Bar Chart')]"),
MAX_BROWSER_TIMEOUT_SEC).click();
collector.checkThat("The output of graph mode is changed",
@@ -222,11 +227,8 @@ public class PersonalizeActionsIT extends
AbstractZeppelinIT {
"'btn btn-default btn-sm ng-binding ng-scope
active')]//i")).getAttribute("class"),
CoreMatchers.equalTo("fa fa-bar-chart"));
- pollingWait(By.xpath("//*[@id='actionbar']" +
- "//button[contains(@uib-tooltip, 'Switch to personal mode')]"),
MAX_BROWSER_TIMEOUT_SEC).click(
[zeppelin] branch master updated: [ZEPPELIN-4339] Flaky test org.apache.zeppelin.AbstractZeppelinIT#testAngularDisplay
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 4fe2e20 [ZEPPELIN-4339] Flaky test
org.apache.zeppelin.AbstractZeppelinIT#testAngularDisplay
4fe2e20 is described below
commit 4fe2e20caa6e37e353d51d46aea41320e07be76e
Author: Lee moon soo
AuthorDate: Thu Sep 19 15:54:55 2019 -0700
[ZEPPELIN-4339] Flaky test
org.apache.zeppelin.AbstractZeppelinIT#testAngularDisplay
### What is this PR for?
This PR try to fix flaky test described in
https://issues.apache.org/jira/browse/ZEPPELIN-4339
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4339
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3457 from Leemoonsoo/ZEPPELIN-4339 and squashes the following
commits:
41d5c88e8 [Lee moon soo] wait for RUN button to be rendered before click
---
.../src/test/java/org/apache/zeppelin/AbstractZeppelinIT.java | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git
a/zeppelin-integration/src/test/java/org/apache/zeppelin/AbstractZeppelinIT.java
b/zeppelin-integration/src/test/java/org/apache/zeppelin/AbstractZeppelinIT.java
index b7cf80d..3a8bf58 100644
---
a/zeppelin-integration/src/test/java/org/apache/zeppelin/AbstractZeppelinIT.java
+++
b/zeppelin-integration/src/test/java/org/apache/zeppelin/AbstractZeppelinIT.java
@@ -55,7 +55,9 @@ abstract public class AbstractZeppelinIT {
}
protected void runParagraph(int paragraphNo) {
-driver.findElement(By.xpath(getParagraphXPath(paragraphNo) +
"//span[@class='icon-control-play']")).click();
+By by = By.xpath(getParagraphXPath(paragraphNo) +
"//span[@class='icon-control-play']");
+pollingWait(by, 5);
+driver.findElement(by).click();
}
[zeppelin] branch branch-0.8 updated: [ZEPPELIN-4280] [Followup] Add comments
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch branch-0.8
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/branch-0.8 by this push:
new e2e1d19 [ZEPPELIN-4280] [Followup] Add comments
e2e1d19 is described below
commit e2e1d1968c894937ae031f330e4a25204d822906
Author: Lee moon soo
AuthorDate: Wed Sep 25 12:19:42 2019 -0700
[ZEPPELIN-4280] [Followup] Add comments
### What is this PR for?
This is followup issue for ZEPPELIN-4280, to add some comments on the code.
### What type of PR is it?
Documentation
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4280
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3461 from Leemoonsoo/ZEPPELIN-4280-followup and squashes the
following commits:
75d4995ea [Lee moon soo] add comment
(cherry picked from commit 7d74b922c20d66bf6610e18cd40ad04b5897956a)
Signed-off-by: Lee moon soo
---
zeppelin-web/src/app/notebook/paragraph/result/result.controller.js | 4
1 file changed, 4 insertions(+)
diff --git
a/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
index 1723e7e..28c74a9 100644
--- a/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
+++ b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
@@ -802,9 +802,13 @@ function ResultCtrl($scope, $rootScope, $route, $window,
$routeParams, $location
newConfig.graph.values = newConfig.graph.commonSetting.pivot.values;
delete newConfig.graph.commonSetting.pivot;
}
+
+ // don't send commitParagraphResult when config is the same.
+ // see https://issues.apache.org/jira/browse/ZEPPELIN-4280.
if (angular.equals($scope.config, newConfig)) {
return;
}
+
console.debug('committVizConfig', newConfig);
let newParams = angular.copy(paragraph.settings.params);
commitParagraphResult(paragraph.title, paragraph.text, newConfig,
newParams);
[zeppelin] branch master updated: [ZEPPELIN-4280] [Followup] Add comments
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 7d74b92 [ZEPPELIN-4280] [Followup] Add comments
7d74b92 is described below
commit 7d74b922c20d66bf6610e18cd40ad04b5897956a
Author: Lee moon soo
AuthorDate: Wed Sep 25 12:19:42 2019 -0700
[ZEPPELIN-4280] [Followup] Add comments
### What is this PR for?
This is followup issue for ZEPPELIN-4280, to add some comments on the code.
### What type of PR is it?
Documentation
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4280
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3461 from Leemoonsoo/ZEPPELIN-4280-followup and squashes the
following commits:
75d4995ea [Lee moon soo] add comment
---
zeppelin-web/src/app/notebook/paragraph/result/result.controller.js | 4
1 file changed, 4 insertions(+)
diff --git
a/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
index 97372e2..b5435ed 100644
--- a/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
+++ b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
@@ -812,9 +812,13 @@ function ResultCtrl($scope, $rootScope, $route, $window,
$routeParams, $location
newConfig.graph.values = newConfig.graph.commonSetting.pivot.values;
delete newConfig.graph.commonSetting.pivot;
}
+
+ // don't send commitParagraphResult when config is the same.
+ // see https://issues.apache.org/jira/browse/ZEPPELIN-4280.
if (angular.equals($scope.config, newConfig)) {
return;
}
+
console.debug('committVizConfig', newConfig);
let newParams = angular.copy(paragraph.settings.params);
commitParagraphResult(paragraph.title, paragraph.text, newConfig,
newParams);
[zeppelin] branch master updated: [ZEPPELIN-4351] Unittest broken after ZEPPELIN-4311
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new a70b301 [ZEPPELIN-4351] Unittest broken after ZEPPELIN-4311
a70b301 is described below
commit a70b301c1d810719e7a64f9b7fbb612804313562
Author: Lee moon soo
AuthorDate: Wed Sep 25 14:24:30 2019 -0700
[ZEPPELIN-4351] Unittest broken after ZEPPELIN-4311
### What is this PR for?
Unittest is broken after ZEPPELIN-4311.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4351
### How should this be tested?
* This should make 4th test profile in .travis.yaml pass. 7th profile is
broken with other reason.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3463 from Leemoonsoo/ZEPPELIN-4311 and squashes the following
commits:
0ab317abf [Lee moon soo] update test
---
.../src/test/java/org/apache/zeppelin/rest/InterpreterRestApiTest.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/zeppelin-server/src/test/java/org/apache/zeppelin/rest/InterpreterRestApiTest.java
b/zeppelin-server/src/test/java/org/apache/zeppelin/rest/InterpreterRestApiTest.java
index f78ee80..e873ebf 100644
---
a/zeppelin-server/src/test/java/org/apache/zeppelin/rest/InterpreterRestApiTest.java
+++
b/zeppelin-server/src/test/java/org/apache/zeppelin/rest/InterpreterRestApiTest.java
@@ -383,6 +383,6 @@ public class InterpreterRestApiTest extends
AbstractTestRestApi {
}
private static String getSimulatedMarkdownResult(String markdown) {
-return String.format("\n%s\n",
markdown);
+return String.format("\n%s\n\n",
markdown);
}
}
[zeppelin] branch branch-0.8 updated: [ZEPPELIN-4280] Fix an issue that makes an infinite loop while switching the visualizations
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch branch-0.8
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/branch-0.8 by this push:
new d1d8585 [ZEPPELIN-4280] Fix an issue that makes an infinite loop
while switching the visualizations
d1d8585 is described below
commit d1d858535f41def4660b1ffe2dbdf183fedf87af
Author: Jaewon Seo
AuthorDate: Thu Jul 25 17:00:00 2019 -0700
[ZEPPELIN-4280] Fix an issue that makes an infinite loop while switching
the visualizations
### What is this PR for?
Fix an issue that makes an infinite loop while switching the visualizations.
An infinite loop occurs because the front-end web application tries to
commit the paragraph even if there are no changes in the visualization
configuration.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4280
### How should this be tested?
If you switch quickly between different visualizations multiple times, the
infinite loops should not happen.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jaewon Seo
Closes #3414 from Byeol/ZEPPELIN-4280 and squashes the following commits:
d49d36ac1 [Jaewon Seo] Commit the paragraph only if there are changes in
the visualization configuration
(cherry picked from commit be7ace1f5e84c7a5989b4d5882cbf759de34112d)
Signed-off-by: Lee moon soo
---
zeppelin-web/src/app/notebook/paragraph/result/result.controller.js | 3 +++
1 file changed, 3 insertions(+)
diff --git
a/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
index 517fcbe..1723e7e 100644
--- a/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
+++ b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
@@ -802,6 +802,9 @@ function ResultCtrl($scope, $rootScope, $route, $window,
$routeParams, $location
newConfig.graph.values = newConfig.graph.commonSetting.pivot.values;
delete newConfig.graph.commonSetting.pivot;
}
+ if (angular.equals($scope.config, newConfig)) {
+return;
+ }
console.debug('committVizConfig', newConfig);
let newParams = angular.copy(paragraph.settings.params);
commitParagraphResult(paragraph.title, paragraph.text, newConfig,
newParams);
[zeppelin] branch master updated: [ZEPPELIN-4280] Fix an issue that makes an infinite loop while switching the visualizations
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new be7ace1 [ZEPPELIN-4280] Fix an issue that makes an infinite loop
while switching the visualizations
be7ace1 is described below
commit be7ace1f5e84c7a5989b4d5882cbf759de34112d
Author: Jaewon Seo
AuthorDate: Thu Jul 25 17:00:00 2019 -0700
[ZEPPELIN-4280] Fix an issue that makes an infinite loop while switching
the visualizations
### What is this PR for?
Fix an issue that makes an infinite loop while switching the visualizations.
An infinite loop occurs because the front-end web application tries to
commit the paragraph even if there are no changes in the visualization
configuration.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4280
### How should this be tested?
If you switch quickly between different visualizations multiple times, the
infinite loops should not happen.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jaewon Seo
Closes #3414 from Byeol/ZEPPELIN-4280 and squashes the following commits:
d49d36ac1 [Jaewon Seo] Commit the paragraph only if there are changes in
the visualization configuration
---
zeppelin-web/src/app/notebook/paragraph/result/result.controller.js | 3 +++
1 file changed, 3 insertions(+)
diff --git
a/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
index f4e5b82..97372e2 100644
--- a/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
+++ b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
@@ -812,6 +812,9 @@ function ResultCtrl($scope, $rootScope, $route, $window,
$routeParams, $location
newConfig.graph.values = newConfig.graph.commonSetting.pivot.values;
delete newConfig.graph.commonSetting.pivot;
}
+ if (angular.equals($scope.config, newConfig)) {
+return;
+ }
console.debug('committVizConfig', newConfig);
let newParams = angular.copy(paragraph.settings.params);
commitParagraphResult(paragraph.title, paragraph.text, newConfig,
newParams);
[zeppelin] branch branch-0.8 updated: [ZEPPELIN-4338] Fix docker image build error
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch branch-0.8
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/branch-0.8 by this push:
new 51091e9 [ZEPPELIN-4338] Fix docker image build error
51091e9 is described below
commit 51091e9d93d9f3f9756bd78df3332bde26dde0ff
Author: Lee moon soo
AuthorDate: Thu Sep 19 12:23:45 2019 -0700
[ZEPPELIN-4338] Fix docker image build error
### What is this PR for?
This PR fixes docker image build error using
/script/docker/zeppelin/bin/Dockerfile
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4338
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3455 from Leemoonsoo/ZEPPELIN-4338 and squashes the following
commits:
953102f32 [Lee moon soo] Add allow-unauthenticated
(cherry picked from commit 7b9356142a196b4cb8399cd399613b302c410e03)
Signed-off-by: Lee moon soo
---
scripts/docker/zeppelin/bin/Dockerfile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/docker/zeppelin/bin/Dockerfile
b/scripts/docker/zeppelin/bin/Dockerfile
index 94cd3c8..3122a34 100644
--- a/scripts/docker/zeppelin/bin/Dockerfile
+++ b/scripts/docker/zeppelin/bin/Dockerfile
@@ -79,7 +79,7 @@ RUN echo "$LOG_TAG Install R related packages" && \
gpg --keyserver keyserver.ubuntu.com --recv-key E084DAB9 && \
gpg -a --export E084DAB9 | apt-key add - && \
apt-get -y update && \
-apt-get -y install r-base r-base-dev && \
+apt-get -y --allow-unauthenticated install r-base r-base-dev && \
R -e "install.packages('knitr', repos='http://cran.us.r-project.org')" && \
R -e "install.packages('ggplot2', repos='http://cran.us.r-project.org')"
&& \
R -e "install.packages('googleVis', repos='http://cran.us.r-project.org')"
&& \
[zeppelin] branch master updated: [ZEPPELIN-4338] Fix docker image build error
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 7b93561 [ZEPPELIN-4338] Fix docker image build error
7b93561 is described below
commit 7b9356142a196b4cb8399cd399613b302c410e03
Author: Lee moon soo
AuthorDate: Thu Sep 19 12:23:45 2019 -0700
[ZEPPELIN-4338] Fix docker image build error
### What is this PR for?
This PR fixes docker image build error using
/script/docker/zeppelin/bin/Dockerfile
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4338
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3455 from Leemoonsoo/ZEPPELIN-4338 and squashes the following
commits:
953102f32 [Lee moon soo] Add allow-unauthenticated
---
scripts/docker/zeppelin/bin/Dockerfile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/docker/zeppelin/bin/Dockerfile
b/scripts/docker/zeppelin/bin/Dockerfile
index b05df71..5cf0277 100644
--- a/scripts/docker/zeppelin/bin/Dockerfile
+++ b/scripts/docker/zeppelin/bin/Dockerfile
@@ -74,7 +74,7 @@ RUN echo "$LOG_TAG Install R related packages" && \
gpg --keyserver keyserver.ubuntu.com --recv-key E084DAB9 && \
gpg -a --export E084DAB9 | apt-key add - && \
apt-get -y update && \
-apt-get -y install r-base r-base-dev && \
+apt-get -y --allow-unauthenticated install r-base r-base-dev && \
R -e "install.packages('knitr', repos='http://cran.us.r-project.org')" && \
R -e "install.packages('ggplot2', repos='http://cran.us.r-project.org')"
&& \
R -e "install.packages('googleVis', repos='http://cran.us.r-project.org')"
&& \
[zeppelin] branch master updated: [ZEPPELIN-4337] Make Zeppelin runs on kubernetes with different domain suffix
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 97dfb3e [ZEPPELIN-4337] Make Zeppelin runs on kubernetes with
different domain suffix
97dfb3e is described below
commit 97dfb3e6c0e267a4597336fc70a3908f2186da3e
Author: Lee moon soo
AuthorDate: Thu Sep 19 12:06:17 2019 -0700
[ZEPPELIN-4337] Make Zeppelin runs on kubernetes with different domain
suffix
### What is this PR for?
A couple of hardcoded dns suffix in the source code (cluster.local). Remove
them so Zeppelin have better compatibility with Kubernetes cluster with dns
name.
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4337
### How should this be tested?
Should work on Kubernetes cluster with different cluster dns name than
'cluster.local'
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3454 from Leemoonsoo/ZEPPELIN-4337 and squashes the following
commits:
ae736b1c3 [Lee moon soo] remove cluster.local suffix
---
k8s/zeppelin-server.yaml | 2 +-
.../zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java| 2 +-
.../zeppelin/interpreter/launcher/K8sStandardInterpreterLauncher.java | 4 ++--
.../interpreter/launcher/K8sRemoteInterpreterProcessTest.java | 2 +-
.../interpreter/launcher/K8sStandardInterpreterLauncherTest.java | 2 +-
5 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/k8s/zeppelin-server.yaml b/k8s/zeppelin-server.yaml
index dbee39c..ecb7606 100644
--- a/k8s/zeppelin-server.yaml
+++ b/k8s/zeppelin-server.yaml
@@ -61,7 +61,7 @@ data:
server_name "~(?[0-9]+)-(?[^.]*)\.(.*)";
location / {
resolver 127.0.0.1:53 ipv6=off;
- proxy_pass http://$svc_name.NAMESPACE.svc.cluster.local:$svc_port;
+ proxy_pass http://$svc_name.NAMESPACE.svc:$svc_port;
proxy_set_header Host $host;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
diff --git
a/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
b/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
index afa8541..63b5a3e 100644
---
a/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
+++
b/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java
@@ -321,7 +321,7 @@ public class K8sRemoteInterpreterProcess extends
RemoteInterpreterProcess {
}
private String getInterpreterPodDnsName() {
-return String.format("%s.%s.svc.cluster.local",
+return String.format("%s.%s.svc",
getPodName(), // service name and pod name is the same
kubectl.getNamespace());
}
diff --git
a/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sStandardInterpreterLauncher.java
b/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sStandardInterpreterLauncher.java
index 4f2ed91..3f2e39d 100644
---
a/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sStandardInterpreterLauncher.java
+++
b/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sStandardInterpreterLauncher.java
@@ -103,12 +103,12 @@ public class K8sStandardInterpreterLauncher extends
InterpreterLauncher {
/**
* get Zeppelin server host dns.
- * return ..svc.cluster.local
+ * return ..svc
* @throws IOException
*/
private String getZeppelinServiceHost() throws IOException {
if (isRunningOnKubernetes()) {
- return String.format("%s.%s.svc.cluster.local",
+ return String.format("%s.%s.svc",
getHostname(), // service name and pod name should be the same
getNamespace());
} else {
diff --git
a/zeppelin-plugins/launcher/k8s-standard/src/test/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcessTest.java
b/zeppelin-plugins/launcher/k8s-standard/src/test/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcessTest.java
index ce2a2cb..d08e73a 100644
---
a/zeppelin-plugins/launcher/k8s-standard/src/test/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcessTest.java
+++
b/zeppelin-plugins/launcher/k8s-standard/src/
[zeppelin] branch master updated: [ZEPPELIN-4226] Fix "View in Spark web UI" in kubernetes mode
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 60e12cd [ZEPPELIN-4226] Fix "View in Spark web UI" in kubernetes mode
60e12cd is described below
commit 60e12cdf86778fb80ad36bb721a6a373206fa12d
Author: Lee moon soo
AuthorDate: Thu Sep 19 12:10:17 2019 -0700
[ZEPPELIN-4226] Fix "View in Spark web UI" in kubernetes mode
### What is this PR for?
When Zeppelin is running in Kubernetes, "View in Spark web UI" gives
internal address, instead of address defined in SERVICE_DOMAIN.
I think this problem is side effect of
https://github.com/apache/zeppelin/pull/3375 and this PR includes fix and
updated unittest.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4226
### How should this be tested?
Run Zeppelin on kubernetes, and run spark job, click "View in Spark web UI"
button.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3451 from Leemoonsoo/ZEPPELIN-4226 and squashes the following
commits:
7e34542f0 [Lee moon soo] use StringUtils.isBlank
a33c3b223 [Lee moon soo] pickup SparkUI address from zeppelin.spark.uiWebUrl
---
.../main/java/org/apache/zeppelin/spark/SparkInterpreter.java | 11 ---
.../java/org/apache/zeppelin/spark/SparkInterpreterTest.java | 7 ---
.../org/apache/zeppelin/spark/BaseSparkScalaInterpreter.scala | 7 ++-
3 files changed, 10 insertions(+), 15 deletions(-)
diff --git
a/spark/interpreter/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
b/spark/interpreter/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
index 33769be..7bacce8 100644
---
a/spark/interpreter/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
+++
b/spark/interpreter/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
@@ -60,8 +60,6 @@ public class SparkInterpreter extends AbstractInterpreter {
private SparkVersion sparkVersion;
private boolean enableSupportedVersionCheck;
- private String sparkUrl;
-
public SparkInterpreter(Properties properties) {
super(properties);
@@ -109,11 +107,6 @@ public class SparkInterpreter extends AbstractInterpreter {
}
sqlContext = this.innerInterpreter.getSqlContext();
sparkSession = this.innerInterpreter.getSparkSession();
- sparkUrl = this.innerInterpreter.getSparkUrl();
- String sparkUrlProp = getProperty("zeppelin.spark.uiWebUrl", "");
- if (!StringUtils.isBlank(sparkUrlProp)) {
-sparkUrl = sparkUrlProp;
- }
SESSION_NUM.incrementAndGet();
} catch (Exception e) {
@@ -260,10 +253,6 @@ public class SparkInterpreter extends AbstractInterpreter {
return depFiles;
}
- public String getSparkUIUrl() {
-return sparkUrl;
- }
-
public boolean isUnsupportedSparkVersion() {
return enableSupportedVersionCheck && sparkVersion.isUnsupportedVersion();
}
diff --git
a/spark/interpreter/src/test/java/org/apache/zeppelin/spark/SparkInterpreterTest.java
b/spark/interpreter/src/test/java/org/apache/zeppelin/spark/SparkInterpreterTest.java
index bceda3a..9a3d471 100644
---
a/spark/interpreter/src/test/java/org/apache/zeppelin/spark/SparkInterpreterTest.java
+++
b/spark/interpreter/src/test/java/org/apache/zeppelin/spark/SparkInterpreterTest.java
@@ -48,6 +48,7 @@ import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.reset;
import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
public class SparkInterpreterTest {
@@ -89,8 +90,6 @@ public class SparkInterpreterTest {
interpreter.setInterpreterGroup(mock(InterpreterGroup.class));
interpreter.open();
-assertEquals("fake_spark_weburl", interpreter.getSparkUIUrl());
-
InterpreterResult result = interpreter.interpret("val a=\"hello world\"",
getInterpreterContext());
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
assertEquals("a: String = hello world\n", output);
@@ -181,7 +180,9 @@ public class SparkInterpreterTest {
assertEquals("pid_2", captorEvent.getValue().get("paraId"));
// spark job url is sent
-verify(mockRemoteEventClient).onParaInfosReceived(any(Map.class));
+ArgumentCaptor onParaInfosReceivedArg =
ArgumentCaptor.forClass(Map.class);
+
verify(mockRemoteEventClient).onParaInfosReceived(onParaInfosReceivedArg.capture());
+assertTrue(((String)
onParaInfosReceivedArg.getValue().get(&
[zeppelin] branch master updated: [ZEPPELIN-3984] Flaky test NotebookRestApiTest.testRunWithServerRestart
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new df5933d [ZEPPELIN-3984] Flaky test
NotebookRestApiTest.testRunWithServerRestart
df5933d is described below
commit df5933d7f99f6d647f23b8e1ed9294a8b5cb563e
Author: Lee moon soo
AuthorDate: Thu Sep 19 15:32:43 2019 -0700
[ZEPPELIN-3984] Flaky test NotebookRestApiTest.testRunWithServerRestart
### What is this PR for?
This PR trying to fix flaky test described in
https://issues.apache.org/jira/browse/ZEPPELIN-3984
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3984
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3456 from Leemoonsoo/ZEPPELIN-3984 and squashes the following
commits:
592ad4abb [Lee moon soo] try not make NPE
---
.../src/test/java/org/apache/zeppelin/rest/NotebookRestApiTest.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/zeppelin-server/src/test/java/org/apache/zeppelin/rest/NotebookRestApiTest.java
b/zeppelin-server/src/test/java/org/apache/zeppelin/rest/NotebookRestApiTest.java
index 079fe5f..bc3bc04 100644
---
a/zeppelin-server/src/test/java/org/apache/zeppelin/rest/NotebookRestApiTest.java
+++
b/zeppelin-server/src/test/java/org/apache/zeppelin/rest/NotebookRestApiTest.java
@@ -422,7 +422,7 @@ public class NotebookRestApiTest extends
AbstractTestRestApi {
assertEquals(resp.get("status"), "OK");
post2.releaseConnection();
- assertEquals(p1.getReturn().toString(), Job.Status.FINISHED,
p1.getStatus());
+ assertEquals(Job.Status.FINISHED, p1.getStatus());
assertEquals(p2.getReturn().toString(), Job.Status.FINISHED,
p2.getStatus());
assertNotNull(p2.getReturn());
assertEquals("abc\n", p2.getReturn().message().get(0).getData());
[zeppelin] branch master updated: [ZEPPELIN-4333] escape dialog title and message
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new a7cf49c [ZEPPELIN-4333] escape dialog title and message
a7cf49c is described below
commit a7cf49cbee7208045f1eeaca23164466dd03f099
Author: Lee moon soo
AuthorDate: Tue Sep 17 10:41:42 2019 -0700
[ZEPPELIN-4333] escape dialog title and message
### What is this PR for?
There're couple of places in the code that dialog title and message is not
being escaped
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4333
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3449 from Leemoonsoo/ZEPPELIN-4333 and squashes the following
commits:
82dab1244 [Lee moon soo] escape dialog title and message
---
zeppelin-web/src/app/helium/helium.controller.js | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/zeppelin-web/src/app/helium/helium.controller.js
b/zeppelin-web/src/app/helium/helium.controller.js
index 81e0853..e409393 100644
--- a/zeppelin-web/src/app/helium/helium.controller.js
+++ b/zeppelin-web/src/app/helium/helium.controller.js
@@ -218,7 +218,7 @@ export default function HeliumCtrl($scope, $rootScope, $sce,
confirm.close();
console.log('Failed to install an interpreter %o %o', name,
artifact);
BootstrapDialog.show({
-title: 'Error while starting to install ' + name + '
interpreter',
+title: 'Error while starting to install ' + _.escape(name) + '
interpreter',
message: _.escape(data.message),
});
});
[zeppelin] branch branch-0.8 updated: ZEPPELIN-4335 Deleting a Notebook is vulnerable to XSS attach
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch branch-0.8
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/branch-0.8 by this push:
new c8eabde ZEPPELIN-4335 Deleting a Notebook is vulnerable to XSS attach
c8eabde is described below
commit c8eabde13f226fd0647c623e6eb67170768d02d1
Author: Akhil Subhash Naik
AuthorDate: Thu Sep 19 11:25:40 2019 +0530
ZEPPELIN-4335 Deleting a Notebook is vulnerable to XSS attach
### What is this PR for?
Fix of : ZEPPELIN-4335 Deleting a Notebook is vulnerable to XSS attach
Issue reproduction steps :
1) create a notebook
2) give the permission to notebook as : alert('hi') (press
space after writing this, not enter key)
3) after this, try to delete the notebook, the BootstrapDialog that popups
stating insufficient privilages is vulnerable to XSS attack
### What type of PR is it?
BUG FIX ZEPPELIN-4335
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4335
### How should this be tested?
Test as per reproduction steps :
1) create a notebook
2) give the permission to notebook as : alert('hi') (press
space after writing this, not enter key)
3) after this, try to delete the notebook, the BootstrapDialog that popups
stating insufficient privilages is vulnerable to XSS attack
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Akhil Subhash Naik
Closes #3452 from Akhilsnaik/ZEPPELIN-4335 and squashes the following
commits:
95212d846 [Akhil Subhash Naik] ZEPPELIN-4335 Deleting a Notebook is
vulnerable to XSS attach (asnaik)
(cherry picked from commit f9e2ff8ff8316e973957041b0f213ce7651cab1c)
Signed-off-by: Lee moon soo
---
zeppelin-web/src/components/websocket/websocket-event.factory.js | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/zeppelin-web/src/components/websocket/websocket-event.factory.js
b/zeppelin-web/src/components/websocket/websocket-event.factory.js
index 8698fdf..dd4f17b 100644
--- a/zeppelin-web/src/components/websocket/websocket-event.factory.js
+++ b/zeppelin-web/src/components/websocket/websocket-event.factory.js
@@ -105,7 +105,7 @@ function WebsocketEventFactory($rootScope, $websocket,
$location, baseUrlSrv, ng
closeByBackdrop: false,
closeByKeyboard: false,
title: 'Insufficient privileges',
-message: data.info.toString(),
+message: _.escape(data.info.toString()),
buttons: btn,
});
} else if (op === 'PARAGRAPH') {
[zeppelin] branch master updated: ZEPPELIN-4335 Deleting a Notebook is vulnerable to XSS attach
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new f9e2ff8 ZEPPELIN-4335 Deleting a Notebook is vulnerable to XSS attach
f9e2ff8 is described below
commit f9e2ff8ff8316e973957041b0f213ce7651cab1c
Author: Akhil Subhash Naik
AuthorDate: Thu Sep 19 11:25:40 2019 +0530
ZEPPELIN-4335 Deleting a Notebook is vulnerable to XSS attach
### What is this PR for?
Fix of : ZEPPELIN-4335 Deleting a Notebook is vulnerable to XSS attach
Issue reproduction steps :
1) create a notebook
2) give the permission to notebook as : alert('hi') (press
space after writing this, not enter key)
3) after this, try to delete the notebook, the BootstrapDialog that popups
stating insufficient privilages is vulnerable to XSS attack
### What type of PR is it?
BUG FIX ZEPPELIN-4335
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4335
### How should this be tested?
Test as per reproduction steps :
1) create a notebook
2) give the permission to notebook as : alert('hi') (press
space after writing this, not enter key)
3) after this, try to delete the notebook, the BootstrapDialog that popups
stating insufficient privilages is vulnerable to XSS attack
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Akhil Subhash Naik
Closes #3452 from Akhilsnaik/ZEPPELIN-4335 and squashes the following
commits:
95212d846 [Akhil Subhash Naik] ZEPPELIN-4335 Deleting a Notebook is
vulnerable to XSS attach (asnaik)
---
zeppelin-web/src/components/websocket/websocket-event.factory.js | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/zeppelin-web/src/components/websocket/websocket-event.factory.js
b/zeppelin-web/src/components/websocket/websocket-event.factory.js
index ef255e7..5bcf45d 100644
--- a/zeppelin-web/src/components/websocket/websocket-event.factory.js
+++ b/zeppelin-web/src/components/websocket/websocket-event.factory.js
@@ -107,7 +107,7 @@ function WebsocketEventFactory($rootScope, $websocket,
$location, baseUrlSrv, ng
closeByBackdrop: false,
closeByKeyboard: false,
title: 'Insufficient privileges',
-message: data.info.toString(),
+message: _.escape(data.info.toString()),
buttons: btn,
});
} else if (op === 'PARAGRAPH') {
[zeppelin] branch branch-0.8 updated: [ZEPPELIN-4166] Change default binding address to 127.0.0.1 (previously 0.0.0.0)
This is an automated email from the ASF dual-hosted git repository. moon pushed a commit to branch branch-0.8 in repository https://gitbox.apache.org/repos/asf/zeppelin.git The following commit(s) were added to refs/heads/branch-0.8 by this push: new 14bbeb6 [ZEPPELIN-4166] Change default binding address to 127.0.0.1 (previously 0.0.0.0) 14bbeb6 is described below commit 14bbeb6d99a16b46545fbdc0850e3b10ae8cb703 Author: Lee moon soo AuthorDate: Tue Sep 17 14:45:29 2019 -0700 [ZEPPELIN-4166] Change default binding address to 127.0.0.1 (previously 0.0.0.0) This PR changes default Zeppelin server binding address from `0.0.0.0` to `127.0.0.1`. Also update related configuration templates and document. mprovement https://issues.apache.org/jira/browse/ZEPPELIN-4166 Build and run Zeppelin server with default configuration and see if default port is listening in address 127.0.0.1 instead of 0.0.0.0 (netstat -na | grep 8080) * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo Closes #3450 from Leemoonsoo/ZEPPELIN-4166 and squashes the following commits: 54dd81237 [Lee moon soo] update upgrade to guide default bind address change 41741d282 [Lee moon soo] update doc 34e2ca8f7 [Lee moon soo] update configuration templates 18aeb8b06 [Lee moon soo] bind 127.0.0.1 instead of 0.0.0.0 by default (cherry picked from commit 7d7628eab8559093b37b2ae05a9f262ca75b3d56) Signed-off-by: Lee moon soo --- conf/zeppelin-env.sh.template | 2 ++ conf/zeppelin-site.xml.template | 4 ++-- docs/setup/operation/configuration.md | 6 ++ docs/setup/operation/upgrading.md | 2 ++ .../main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java | 2 +- 5 files changed, 13 insertions(+), 3 deletions(-) diff --git a/conf/zeppelin-env.sh.template b/conf/zeppelin-env.sh.template index 5aab0e0..b7f4ee6 100644 --- a/conf/zeppelin-env.sh.template +++ b/conf/zeppelin-env.sh.template @@ -18,6 +18,8 @@ # export JAVA_HOME= # export MASTER= # Spark master url. eg. spark://master_addr:7077. Leave empty if you want to use local mode. +# export ZEPPELIN_ADDR # Bind address (default 127.0.0.1) +# export ZEPPELIN_PORT # port number to listen (default 8080) # export ZEPPELIN_JAVA_OPTS# Additional jvm options. for example, export ZEPPELIN_JAVA_OPTS="-Dspark.executor.memory=8g -Dspark.cores.max=16" # export ZEPPELIN_MEM # Zeppelin jvm mem options Default -Xms1024m -Xmx1024m -XX:MaxPermSize=512m # export ZEPPELIN_INTP_MEM # zeppelin interpreter process jvm mem options. Default -Xms1024m -Xmx1024m -XX:MaxPermSize=512m diff --git a/conf/zeppelin-site.xml.template b/conf/zeppelin-site.xml.template index e665a9b..0ca09c0 100755 --- a/conf/zeppelin-site.xml.template +++ b/conf/zeppelin-site.xml.template @@ -21,8 +21,8 @@ zeppelin.server.addr - 0.0.0.0 - Server address + 127.0.0.1 + Server binding address diff --git a/docs/setup/operation/configuration.md b/docs/setup/operation/configuration.md index b831918..61756c8 100644 --- a/docs/setup/operation/configuration.md +++ b/docs/setup/operation/configuration.md @@ -40,6 +40,12 @@ If both are defined, then the **environment variables** will take priority. Description +ZEPPELIN_ADDR +zeppelin.server.addr +127.0.0.1 +Zeppelin server binding address + + ZEPPELIN_PORT zeppelin.server.port 8080 diff --git a/docs/setup/operation/upgrading.md b/docs/setup/operation/upgrading.md index 8ea3697..75c2327 100644 --- a/docs/setup/operation/upgrading.md +++ b/docs/setup/operation/upgrading.md @@ -45,6 +45,8 @@ To set all your "writers" to "runners": 2. Move directory **docs/assets/themes/zeppelin/note/FixReaders** to new `notebook` directory, 3. Start the new Zeppelin and run note **System/Migrate from 0.7**. +### Upgrading from Zeppelin 0.8.1 (and before) to 0.8.2 (and later) + - From 0.8.2, Zeppelin server bind `127.0.0.1` by default instead of `0.0.0.0`. Configure `zeppelin.server.addr` property or `ZEPPELIN_ADDR` env variable to change. ### Upgrading from Zeppelin 0.7 to 0.8 diff --git a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java index 245f7b9..5b4ebc0 100644 --- a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java +++ b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java @@
[zeppelin] branch master updated: [ZEPPELIN-4166] Change default binding address to 127.0.0.1 (previously 0.0.0.0)
This is an automated email from the ASF dual-hosted git repository. moon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/zeppelin.git The following commit(s) were added to refs/heads/master by this push: new 7d7628e [ZEPPELIN-4166] Change default binding address to 127.0.0.1 (previously 0.0.0.0) 7d7628e is described below commit 7d7628eab8559093b37b2ae05a9f262ca75b3d56 Author: Lee moon soo AuthorDate: Tue Sep 17 14:45:29 2019 -0700 [ZEPPELIN-4166] Change default binding address to 127.0.0.1 (previously 0.0.0.0) ### What is this PR for? This PR changes default Zeppelin server binding address from `0.0.0.0` to `127.0.0.1`. Also update related configuration templates and document. ### What type of PR is it? mprovement ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-4166 ### How should this be tested? Build and run Zeppelin server with default configuration and see if default port is listening in address 127.0.0.1 instead of 0.0.0.0 (netstat -na | grep 8080) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo Closes #3450 from Leemoonsoo/ZEPPELIN-4166 and squashes the following commits: 54dd81237 [Lee moon soo] update upgrade to guide default bind address change 41741d282 [Lee moon soo] update doc 34e2ca8f7 [Lee moon soo] update configuration templates 18aeb8b06 [Lee moon soo] bind 127.0.0.1 instead of 0.0.0.0 by default --- conf/zeppelin-env.sh.template | 2 ++ conf/zeppelin-site.xml.template | 4 ++-- docs/setup/operation/configuration.md | 6 ++ docs/setup/operation/upgrading.md | 6 +- .../main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java | 2 +- 5 files changed, 16 insertions(+), 4 deletions(-) diff --git a/conf/zeppelin-env.sh.template b/conf/zeppelin-env.sh.template index 5aaf83d..c31dd67 100644 --- a/conf/zeppelin-env.sh.template +++ b/conf/zeppelin-env.sh.template @@ -18,6 +18,8 @@ # export JAVA_HOME= # export MASTER= # Spark master url. eg. spark://master_addr:7077. Leave empty if you want to use local mode. +# export ZEPPELIN_ADDR # Bind address (default 127.0.0.1) +# export ZEPPELIN_PORT # port number to listen (default 8080) # export ZEPPELIN_LOCAL_IP # Zeppelin's thrift server ip address, if not specified, one random IP address will be choosen. # export ZEPPELIN_JAVA_OPTS# Additional jvm options. for example, export ZEPPELIN_JAVA_OPTS="-Dspark.executor.memory=8g -Dspark.cores.max=16" # export ZEPPELIN_MEM # Zeppelin jvm mem options Default -Xms1024m -Xmx1024m -XX:MaxPermSize=512m diff --git a/conf/zeppelin-site.xml.template b/conf/zeppelin-site.xml.template index 6418340..1b88e46 100755 --- a/conf/zeppelin-site.xml.template +++ b/conf/zeppelin-site.xml.template @@ -27,8 +27,8 @@ zeppelin.server.addr - 0.0.0.0 - Server address + 127.0.0.1 + Server binding address diff --git a/docs/setup/operation/configuration.md b/docs/setup/operation/configuration.md index afa4b2a..a6a602d 100644 --- a/docs/setup/operation/configuration.md +++ b/docs/setup/operation/configuration.md @@ -40,6 +40,12 @@ If both are defined, then the **environment variables** will take priority. Description +ZEPPELIN_ADDR +zeppelin.server.addr +127.0.0.1 +Zeppelin server binding address + + ZEPPELIN_PORT zeppelin.server.port 8080 diff --git a/docs/setup/operation/upgrading.md b/docs/setup/operation/upgrading.md index 99b239b..53e644d 100644 --- a/docs/setup/operation/upgrading.md +++ b/docs/setup/operation/upgrading.md @@ -39,7 +39,11 @@ So, copying `notebook` and `conf` directory should be enough. - From 0.9, we change the notes file name structure ([ZEPPELIN-2619](https://issues.apache.org/jira/browse/ZEPPELIN-2619)) and move permissions info from `notebook-authorization.json` into note file itself [ZEPPELIN-3985](https://issues.apache.org/jira/browse/ZEPPELIN-3985). So when you upgrading zeppelin to 0.9, you need to upgrade note file. Here's steps you need to follow: 1. Backup your notes file in case the upgrade fails - 2. Call `bin/upgrade-note.sh -d` to upgrade note, `-d` option means to delete the old note file, missing this option will keep the old file. + 2. Call `bin/upgrade-note.sh -d` to upgrade note, `-d` option means to delete the old note file, missing this option will keep the old file. + - From 0.9, Zeppelin server bind `127.0.0.1` by default instead of `0.0.0.0`. Configure `zeppelin.server.addr` property or `ZEPPELIN
svn commit: r1857237 - in /zeppelin/site/docs/0.9.0-SNAPSHOT/development/contribution: how_to_contribute_code.html how_to_contribute_website.html
Author: moon Date: Wed Apr 10 08:58:45 2019 New Revision: 1857237 URL: http://svn.apache.org/viewvc?rev=1857237=rev Log: ZEPPELIN-4040 Modified: zeppelin/site/docs/0.9.0-SNAPSHOT/development/contribution/how_to_contribute_code.html zeppelin/site/docs/0.9.0-SNAPSHOT/development/contribution/how_to_contribute_website.html Modified: zeppelin/site/docs/0.9.0-SNAPSHOT/development/contribution/how_to_contribute_code.html URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.9.0-SNAPSHOT/development/contribution/how_to_contribute_code.html?rev=1857237=1857236=1857237=diff == --- zeppelin/site/docs/0.9.0-SNAPSHOT/development/contribution/how_to_contribute_code.html (original) +++ zeppelin/site/docs/0.9.0-SNAPSHOT/development/contribution/how_to_contribute_code.html Wed Apr 10 08:58:45 2019 @@ -197,6 +197,7 @@ Scalding Scio Shell +Submarine @@ -291,15 +292,15 @@ Any contributions to Zeppelin (Source co Getting the source code -First of all, you need Zeppelin source code. The official location of Zeppelin is http://git.apache.org/zeppelin.git;>http://git.apache.org/zeppelin.git. +First of all, you need Zeppelin source code. The official location of Zeppelin is https://gitbox.apache.org/repos/asf/zeppelin.git;>https://gitbox.apache.org/repos/asf/zeppelin.git. git access Get the source code on your development machine using git. -git clone git://git.apache.org/zeppelin.git zeppelin +git clone git://gitbox.apache.org/repos/asf/zeppelin.git zeppelin You may also want to develop against a specific branch. For example, for branch-0.5.6 -git clone -b branch-0.5.6 git://git.apache.org/zeppelin.git zeppelin +git clone -b branch-0.5.6 git://gitbox.apache.org/repos/asf/zeppelin.git zeppelin Apache Zeppelin follows https://github.com/sevntu-checkstyle/sevntu.checkstyle/wiki/Development-workflow-with-Git:-Fork,-Branching,-Commits,-and-Pull-Request;>Fork Pull as a source control workflow. If you want to not only build Zeppelin but also make any changes, then you need to fork https://github.com/apache/zeppelin;>Zeppelin github mirror repository and make a pull request. Modified: zeppelin/site/docs/0.9.0-SNAPSHOT/development/contribution/how_to_contribute_website.html URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.9.0-SNAPSHOT/development/contribution/how_to_contribute_website.html?rev=1857237=1857236=1857237=diff == --- zeppelin/site/docs/0.9.0-SNAPSHOT/development/contribution/how_to_contribute_website.html (original) +++ zeppelin/site/docs/0.9.0-SNAPSHOT/development/contribution/how_to_contribute_website.html Wed Apr 10 08:58:45 2019 @@ -197,6 +197,7 @@ Scalding Scio Shell +Submarine @@ -273,14 +274,14 @@ Any contributions to Zeppelin (Source co Getting the source code -First of all, you need Zeppelin source code. The official location of Zeppelin is http://git.apache.org/zeppelin.git;>http://git.apache.org/zeppelin.git. +First of all, you need Zeppelin source code. The official location of Zeppelin is https://gitbox.apache.org/repos/asf/zeppelin.git;>https://gitbox.apache.org/repos/asf/zeppelin.git. Documentation website is hosted in master branch under /docs/ dir. git access -First of all, you need the website source code. The official location of mirror for Zeppelin is http://git.apache.org/zeppelin.git;>http://git.apache.org/zeppelin.git. +First of all, you need the website source code. The official location of mirror for Zeppelin is https://gitbox.apache.org/repos/asf/zeppelin.git;>https://gitbox.apache.org/repos/asf/zeppelin.git. Get the source code on your development machine using git. -git clone git://git.apache.org/zeppelin.git +git clone git://gitbox.apache.org/repos/asf/zeppelin.git cd docs Apache Zeppelin follows https://github.com/sevntu-checkstyle/sevntu.checkstyle/wiki/Development-workflow-with-Git:-Fork,-Branching,-Commits,-and-Pull-Request;>Fork Pull as a source control workflow.
[zeppelin] branch master updated: [ZEPPELIN-4040]. Changed links in documentation to use Gitbox
This is an automated email from the ASF dual-hosted git repository. moon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/zeppelin.git The following commit(s) were added to refs/heads/master by this push: new 49dfde6 [ZEPPELIN-4040]. Changed links in documentation to use Gitbox 49dfde6 is described below commit 49dfde6c00a2064835176a5f0ce839e7cce0f247 Author: Jan Hentschel AuthorDate: Sat Apr 6 16:32:47 2019 +0200 [ZEPPELIN-4040]. Changed links in documentation to use Gitbox ### What is this PR for? Changed the broken Git links to use Gitbox ### What type of PR is it? Documentation ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-4040 ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Jan Hentschel Closes #3347 from HorizonNet/ZEPPELIN-4040 and squashes the following commits: 6c4feb2da [Jan Hentschel] ZEPPELIN-4040. Changed links in documentation to use Gitbox --- docs/development/contribution/how_to_contribute_code.md| 6 +++--- docs/development/contribution/how_to_contribute_website.md | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/development/contribution/how_to_contribute_code.md b/docs/development/contribution/how_to_contribute_code.md index e71326e..e6d1151 100644 --- a/docs/development/contribution/how_to_contribute_code.md +++ b/docs/development/contribution/how_to_contribute_code.md @@ -45,20 +45,20 @@ To build the code, install * Apache Maven ## Getting the source code -First of all, you need Zeppelin source code. The official location of Zeppelin is [http://git.apache.org/zeppelin.git](http://git.apache.org/zeppelin.git). +First of all, you need Zeppelin source code. The official location of Zeppelin is [https://gitbox.apache.org/repos/asf/zeppelin.git](https://gitbox.apache.org/repos/asf/zeppelin.git). ### git access Get the source code on your development machine using git. ```bash -git clone git://git.apache.org/zeppelin.git zeppelin +git clone git://gitbox.apache.org/repos/asf/zeppelin.git zeppelin ``` You may also want to develop against a specific branch. For example, for branch-0.5.6 ```bash -git clone -b branch-0.5.6 git://git.apache.org/zeppelin.git zeppelin +git clone -b branch-0.5.6 git://gitbox.apache.org/repos/asf/zeppelin.git zeppelin ``` Apache Zeppelin follows [Fork & Pull](https://github.com/sevntu-checkstyle/sevntu.checkstyle/wiki/Development-workflow-with-Git:-Fork,-Branching,-Commits,-and-Pull-Request) as a source control workflow. diff --git a/docs/development/contribution/how_to_contribute_website.md b/docs/development/contribution/how_to_contribute_website.md index 1b7c2d9..e6b2b00 100644 --- a/docs/development/contribution/how_to_contribute_website.md +++ b/docs/development/contribution/how_to_contribute_website.md @@ -31,16 +31,16 @@ The online documentation at [zeppelin.apache.org](https://zeppelin.apache.org/do Any contributions to Zeppelin (Source code, Documents, Image, Website) means you agree with license all your contributions as Apache2 License. ## Getting the source code -First of all, you need Zeppelin source code. The official location of Zeppelin is [http://git.apache.org/zeppelin.git](http://git.apache.org/zeppelin.git). +First of all, you need Zeppelin source code. The official location of Zeppelin is [https://gitbox.apache.org/repos/asf/zeppelin.git](https://gitbox.apache.org/repos/asf/zeppelin.git). Documentation website is hosted in 'master' branch under `/docs/` dir. ### git access -First of all, you need the website source code. The official location of mirror for Zeppelin is [http://git.apache.org/zeppelin.git](http://git.apache.org/zeppelin.git). +First of all, you need the website source code. The official location of mirror for Zeppelin is [https://gitbox.apache.org/repos/asf/zeppelin.git](https://gitbox.apache.org/repos/asf/zeppelin.git). Get the source code on your development machine using git. ```bash -git clone git://git.apache.org/zeppelin.git +git clone git://gitbox.apache.org/repos/asf/zeppelin.git cd docs ``` Apache Zeppelin follows [Fork & Pull](https://github.com/sevntu-checkstyle/sevntu.checkstyle/wiki/Development-workflow-with-Git:-Fork,-Branching,-Commits,-and-Pull-Request) as a source control workflow.
[zeppelin] branch master updated: [ZEPPELIN-4004] Fix RemoteResource.invokeMethod()
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 154ee11 [ZEPPELIN-4004] Fix RemoteResource.invokeMethod()
154ee11 is described below
commit 154ee11842a9ff874a132ab2e80f6c06dfffa190
Author: Lee moon soo
AuthorDate: Mon Feb 18 16:31:00 2019 -0800
[ZEPPELIN-4004] Fix RemoteResource.invokeMethod()
### What is this PR for?
RemoteResource is a representation of object in ResourcePool in Interpreter
running on another process.
RemoteResource provides a invokeMethod() to call method of an object
remotely, which is not working now. This PR provide a fix for that. Also
improve the test case to cover this case.
Also provide small improvement to inference parameter type on remote
invocation.
### What type of PR is it?
Bug Fix, Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-4004
### How should this be tested?
Unittest included. CI pass.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3312 from Leemoonsoo/ZEPPELIN-4004 and squashes the following
commits:
c44dcc073 [Lee moon soo] address reference to invokeMethod is ambiguous
1893ad150 [Lee moon soo] inference parameter type
fe6f5ed83 [Lee moon soo] Fix RemoteResource.invokeMethod() and update
testcase to cover it
---
.../remote/RemoteInterpreterEventClient.java | 98 +++
.../remote/RemoteInterpreterServer.java| 38 +--
.../interpreter/util/ByteBufferUtils.java} | 28 +-
.../apache/zeppelin/resource/RemoteResource.java | 5 +-
.../org/apache/zeppelin/resource/Resource.java | 313 -
.../org/apache/zeppelin/resource/ResourceId.java | 3 +-
.../zeppelin/tabledata/ProxyRowIterator.java | 4 +-
.../apache/zeppelin/tabledata/TableDataProxy.java | 4 +-
.../interpreter/util/ByteBufferUtilTest.java | 17 ++
.../org/apache/zeppelin/resource/ResourceTest.java | 36 +++
.../interpreter/RemoteInterpreterEventServer.java | 15 +-
.../remote/mock/MockInterpreterResourcePool.java | 4 +-
.../resource/DistributedResourcePoolTest.java | 5 +-
13 files changed, 446 insertions(+), 124 deletions(-)
diff --git
a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterEventClient.java
b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterEventClient.java
index b7e77c6..5ac1c0a 100644
---
a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterEventClient.java
+++
b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterEventClient.java
@@ -124,39 +124,20 @@ public class RemoteInterpreterEventClient implements
ResourcePoolConnector,
Object[] params) {
LOGGER.debug("Request Invoke method {} of Resource {}", methodName,
resourceId.getName());
-return null;
-//InvokeResourceMethodEventMessage invokeMethod = new
InvokeResourceMethodEventMessage(
-//resourceId,
-//methodName,
-//paramTypes,
-//params,
-//null);
-//
-//synchronized (getInvokeResponse) {
-// // wait for previous response consumed
-// while (getInvokeResponse.containsKey(invokeMethod)) {
-//try {
-// getInvokeResponse.wait();
-//} catch (InterruptedException e) {
-// LOGGER.warn(e.getMessage(), e);
-//}
-// }
-// // send request
-// sendEvent(new RemoteInterpreterEvent(
-// RemoteInterpreterEventType.RESOURCE_INVOKE_METHOD,
-// invokeMethod.toJson()));
-// // wait for response
-// while (!getInvokeResponse.containsKey(invokeMethod)) {
-//try {
-// getInvokeResponse.wait();
-//} catch (InterruptedException e) {
-// LOGGER.warn(e.getMessage(), e);
-//}
-// }
-// Object o = getInvokeResponse.remove(invokeMethod);
-// getInvokeResponse.notifyAll();
-// return o;
-//}
+InvokeResourceMethodEventMessage invokeMethod = new
InvokeResourceMethodEventMessage(
+resourceId,
+methodName,
+paramTypes,
+params,
+null);
+try {
+ ByteBuffer buffer = intpEventServiceClient.invokeMethod(intpGroupId,
invokeMethod.toJson());
+ Object o = Resource.deserializeObject(buffer);
+ return o;
+} catch (TException | IOException | ClassNotFoundException e) {
+ LO
svn commit: r1855071 - in /zeppelin/site: index.html technologies_deployments.md
Author: moon Date: Sat Mar 9 00:28:28 2019 New Revision: 1855071 URL: http://svn.apache.org/viewvc?rev=1855071=rev Log: fix borken link, update outdated link Modified: zeppelin/site/index.html zeppelin/site/technologies_deployments.md Modified: zeppelin/site/index.html URL: http://svn.apache.org/viewvc/zeppelin/site/index.html?rev=1855071=1855070=1855071=diff == --- zeppelin/site/index.html (original) +++ zeppelin/site/index.html Sat Mar 9 00:28:28 2019 @@ -182,7 +182,7 @@ limitations under the License. Zeppelin supports Spark, PySpark, Spark R, Spark SQL with dependency loader. -USE NOW +USE NOW @@ -214,7 +214,7 @@ limitations under the License. See more details in Zeppelin supports 20+ different interpreters. -LEARN MORE +LEARN MORE @@ -277,21 +277,21 @@ limitations under the License. Python, improved -IPython interpreter provides comparable user experience like Jupyter Notebook. For the details, click here. +IPython interpreter provides comparable user experience like Jupyter Notebook. For the details, click here. Note improvements - This release includes Note level dynamic form, note revision comparator and ability to run paragraph sequentially, instead of simultaneous paragraph execution in previous releases. + This release includes Note level dynamic form, note revision comparator and ability to run paragraph sequentially, instead of simultaneous paragraph execution in previous releases. Inline configuration - Generic ConfInterpreter provide a way configure interpreter inside each note. + Generic ConfInterpreter provide a way configure interpreter inside each note. @@ -307,20 +307,20 @@ limitations under the License. Interpreter lifecycle - Interpreter lifecycle manager automatically terminate interpreter process on idle timeout. So resources are released when they're not in use. See here for more details. + Interpreter lifecycle manager automatically terminate interpreter process on idle timeout. So resources are released when they're not in use. See here for more details. Helium online registry - This release includes online registry for Helium that can add custom visualization and more. learn more about it. + This release includes online registry for Helium that can add custom visualization and more. learn more about it. See more details in -0.8 Release Note +0.8.1 Release Note @@ -441,7 +441,7 @@ Currently Apache Zeppelin supports many -Learn more about Dynamic Forms. +Learn more about Dynamic Forms. @@ -457,7 +457,7 @@ Learn more about this page. +If you want to learn more about this feature, please visit this page. Modified: zeppelin/site/technologies_deployments.md URL: http://svn.apache.org/viewvc/zeppelin/site/technologies_deployments.md?rev=1855071=1855070=1855071=diff == --- zeppelin/site/technologies_deployments.md (original) +++ zeppelin/site/technologies_deployments.md Sat Mar 9 00:28:28 2019 @@ -25,7 +25,7 @@ limitations under the License. Zeppelin supports Spark, PySpark, Spark R, Spark SQL with dependency loader. -USE NOW +USE NOW @@ -36,7 +36,7 @@ limitations under the License. Zeppelin lets you connect any JDBC data sources seamlessly. Postgresql, Mysql, MariaDB, Redshift, Apache Hive and so on. -USE NOW +USE NOW @@ -49,7 +49,7 @@ limitations under the License. Python is supported with Matplotlib, Conda, Pandas SQL and PySpark integrations. -USE NOW +USE NOW @@ -57,7 +57,7 @@ limitations under the License. See more details in Zeppelin supports 20+ different interpreters. -LEARN MORE +LEARN MORE @@ -73,7 +73,7 @@ limitations under the License. Local Spark, 6 Built-in visualizations, Display system, Dynamic form, Multiple backends are supported. -LEARN MORE +LEARN MORE @@ -87,7 +87,7 @@ limitations under
[zeppelin] branch branch-0.8 updated: [ZEPPELIN-3988] Paragraph Text output includes \r\n is not displayed correctly.
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch branch-0.8
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/branch-0.8 by this push:
new 81f2bf2 [ZEPPELIN-3988] Paragraph Text output includes \r\n is not
displayed correctly.
81f2bf2 is described below
commit 81f2bf25deca0cab21502e8172a0de4fcf4ea42c
Author: Lee moon soo
AuthorDate: Tue Feb 12 10:43:30 2019 -0800
[ZEPPELIN-3988] Paragraph Text output includes \r\n is not displayed
correctly.
### What is this PR for?
When paragraph text output includes `\r\n`, it is not displayed correctly
in paragraph result.
Expected result
Current behavior (displays empty result)
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3988
### How should this be tested?
run
```
%python
print("Hello world\r\n")
```
and see if output is displayed correctly.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3302 from Leemoonsoo/ZEPPELIN-3988 and squashes the following
commits:
b3f02906f [Lee moon soo] bring back carriage return routine
06c5d405c [Lee moon soo] improve checkAndReplaceCarriageReturn and move it
to a separate class to add tests
(cherry picked from commit d07ae2c92fde397801fefc85fd3360830f0a9843)
Signed-off-by: Lee moon soo
---
.../notebook/paragraph/result/result.controller.js | 18 ++
.../paragraph/result/result.controller.test.js | 41 ++
.../src/app/notebook/paragraph/result/result.js| 25 +
.../app/notebook/paragraph/result/result.test.js | 11 ++
4 files changed, 79 insertions(+), 16 deletions(-)
diff --git
a/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
index 98940ab..517fcbe 100644
--- a/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
+++ b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
@@ -24,6 +24,7 @@ import ScatterchartVisualization from
'../../../visualization/builtins/visualiza
import NetworkVisualization from
'../../../visualization/builtins/visualization-d3network';
import {DefaultDisplayType, SpellResult} from '../../../spell';
import {ParagraphStatus} from '../paragraph.status';
+import Result from './result';
const AnsiUp = require('ansi_up');
const AnsiUpConverter = new AnsiUp.default; // eslint-disable-line
new-parens,new-cap
@@ -488,22 +489,7 @@ function ResultCtrl($scope, $rootScope, $route, $window,
$routeParams, $location
};
const checkAndReplaceCarriageReturn = function(str) {
-if (/\r/.test(str)) {
- let newGenerated = '';
- let strArr = str.split('\n');
- for (let str of strArr) {
-if (/\r/.test(str)) {
- let splitCR = str.split('\r');
- newGenerated += splitCR[splitCR.length - 1] + '\n';
-} else {
- newGenerated += str + '\n';
-}
- }
- // remove last "\n" character
- return newGenerated.slice(0, -1);
-} else {
- return str;
-}
+return new Result(str).checkAndReplaceCarriageReturn();
};
const renderText = function(targetElemId, data, refresh) {
diff --git
a/zeppelin-web/src/app/notebook/paragraph/result/result.controller.test.js
b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.test.js
new file mode 100644
index 000..c299973
--- /dev/null
+++ b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.test.js
@@ -0,0 +1,41 @@
+describe('Controller: ResultCtrl', function() {
+ beforeEach(angular.mock.module('zeppelinWebApp'));
+
+ let scope;
+ let controller;
+ let resultMock = {
+ };
+ let configMock = {
+ };
+ let paragraphMock = {
+id: 'p1',
+results: {
+ msg: [],
+},
+ };
+ let route = {
+current: {
+ pathParams: {
+noteId: 'noteId',
+ },
+},
+ };
+
+ beforeEach(inject(function($controller, $rootScope) {
+scope = $rootScope.$new();
+scope.$parent = $rootScope.$new(true, $rootScope);
+scope.$parent.paragraph = paragraphMock;
+
+controller = $controller('ResultCtrl', {
+ $scope: scope,
+ $route: route,
+});
+
+scope.init(resultMock, configMock, paragraphMock, 1);
+ }));
+
+ it('scope should be initialized', function() {
+expect(scope).toBeDefined();
+expect(con
[zeppelin] branch master updated: [ZEPPELIN-3988] Paragraph Text output includes \r\n is not displayed correctly.
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new d07ae2c [ZEPPELIN-3988] Paragraph Text output includes \r\n is not
displayed correctly.
d07ae2c is described below
commit d07ae2c92fde397801fefc85fd3360830f0a9843
Author: Lee moon soo
AuthorDate: Tue Feb 12 10:43:30 2019 -0800
[ZEPPELIN-3988] Paragraph Text output includes \r\n is not displayed
correctly.
### What is this PR for?
When paragraph text output includes `\r\n`, it is not displayed correctly
in paragraph result.
Expected result
Current behavior (displays empty result)
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3988
### How should this be tested?
run
```
%python
print("Hello world\r\n")
```
and see if output is displayed correctly.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3302 from Leemoonsoo/ZEPPELIN-3988 and squashes the following
commits:
b3f02906f [Lee moon soo] bring back carriage return routine
06c5d405c [Lee moon soo] improve checkAndReplaceCarriageReturn and move it
to a separate class to add tests
---
.../notebook/paragraph/result/result.controller.js | 18 ++
.../paragraph/result/result.controller.test.js | 41 ++
.../src/app/notebook/paragraph/result/result.js| 25 +
.../app/notebook/paragraph/result/result.test.js | 11 ++
4 files changed, 79 insertions(+), 16 deletions(-)
diff --git
a/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
index 86f112b..f4e5b82 100644
--- a/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
+++ b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js
@@ -24,6 +24,7 @@ import ScatterchartVisualization from
'../../../visualization/builtins/visualiza
import NetworkVisualization from
'../../../visualization/builtins/visualization-d3network';
import {DefaultDisplayType, SpellResult} from '../../../spell';
import {ParagraphStatus} from '../paragraph.status';
+import Result from './result';
const AnsiUp = require('ansi_up');
const AnsiUpConverter = new AnsiUp.default; // eslint-disable-line
new-parens,new-cap
@@ -488,22 +489,7 @@ function ResultCtrl($scope, $rootScope, $route, $window,
$routeParams, $location
};
const checkAndReplaceCarriageReturn = function(str) {
-if (/\r/.test(str)) {
- let newGenerated = '';
- let strArr = str.split('\n');
- for (let str of strArr) {
-if (/\r/.test(str)) {
- let splitCR = str.split('\r');
- newGenerated += splitCR[splitCR.length - 1] + '\n';
-} else {
- newGenerated += str + '\n';
-}
- }
- // remove last "\n" character
- return newGenerated.slice(0, -1);
-} else {
- return str;
-}
+return new Result(str).checkAndReplaceCarriageReturn();
};
const renderText = function(targetElemId, data, refresh) {
diff --git
a/zeppelin-web/src/app/notebook/paragraph/result/result.controller.test.js
b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.test.js
new file mode 100644
index 000..c299973
--- /dev/null
+++ b/zeppelin-web/src/app/notebook/paragraph/result/result.controller.test.js
@@ -0,0 +1,41 @@
+describe('Controller: ResultCtrl', function() {
+ beforeEach(angular.mock.module('zeppelinWebApp'));
+
+ let scope;
+ let controller;
+ let resultMock = {
+ };
+ let configMock = {
+ };
+ let paragraphMock = {
+id: 'p1',
+results: {
+ msg: [],
+},
+ };
+ let route = {
+current: {
+ pathParams: {
+noteId: 'noteId',
+ },
+},
+ };
+
+ beforeEach(inject(function($controller, $rootScope) {
+scope = $rootScope.$new();
+scope.$parent = $rootScope.$new(true, $rootScope);
+scope.$parent.paragraph = paragraphMock;
+
+controller = $controller('ResultCtrl', {
+ $scope: scope,
+ $route: route,
+});
+
+scope.init(resultMock, configMock, paragraphMock, 1);
+ }));
+
+ it('scope should be initialized', function() {
+expect(scope).toBeDefined();
+expect(controller).toBeDefined();
+ });
+});
diff --git a/zeppelin-web/src/app/notebook/paragraph/result/result.js
b/zeppelin-web/src/app/noteb
[zeppelin] branch master updated: [ZEPPELIN-3989] Configure IPython Interpreter in Docker image
This is an automated email from the ASF dual-hosted git repository. moon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/zeppelin.git The following commit(s) were added to refs/heads/master by this push: new 64a768a [ZEPPELIN-3989] Configure IPython Interpreter in Docker image 64a768a is described below commit 64a768aa5e8f0054e18efbbadc9ffc4ddb03151f Author: Lee moon soo AuthorDate: Mon Feb 4 09:58:28 2019 -0800 [ZEPPELIN-3989] Configure IPython Interpreter in Docker image ### What is this PR for? Configure IPython interpreter in Docker image by installing necessary dependencies. ### What type of PR is it? Improvement ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-3989 ### How should this be tested? Build docker image and see if %python interpreter works with Ipython kernel. ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Lee moon soo Closes #3303 from Leemoonsoo/ZEPPELIN-3989 and squashes the following commits: e64ac428e [Lee moon soo] install ipython interpreter dependencies --- scripts/docker/zeppelin/bin/Dockerfile | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/scripts/docker/zeppelin/bin/Dockerfile b/scripts/docker/zeppelin/bin/Dockerfile index 080b05c..b05df71 100644 --- a/scripts/docker/zeppelin/bin/Dockerfile +++ b/scripts/docker/zeppelin/bin/Dockerfile @@ -63,12 +63,11 @@ RUN echo "$LOG_TAG Install python related packages" && \ apt-get install -y gfortran && \ # numerical/algebra packages apt-get install -y libblas-dev libatlas-dev liblapack-dev && \ -# font, image for matplotlib +# font, image apt-get install -y libpng-dev libfreetype6-dev libxft-dev && \ # for tkinter apt-get install -y python-tk libxml2-dev libxslt-dev zlib1g-dev && \ -pip install numpy && \ -pip install matplotlib +pip install numpy==1.12.1 pandas==0.21.1 matplotlib==2.1.1 pandasql==0.7.3 ipython==5.4.1 jupyter_client==5.1.0 ipykernel==4.7.0 bokeh==0.12.10 ggplot==0.11.5 grpcio==1.8.2 bkzep==0.4.0 RUN echo "$LOG_TAG Install R related packages" && \ echo "deb http://cran.rstudio.com/bin/linux/ubuntu xenial/" | tee -a /etc/apt/sources.list && \
[zeppelin] branch master updated: [ZEPPELIN-3920] [FOLLOWUP] get correct hyperlink address when num job url is one
This is an automated email from the ASF dual-hosted git repository.
moon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 3fe48dd [ZEPPELIN-3920] [FOLLOWUP] get correct hyperlink address when
num job url is one
3fe48dd is described below
commit 3fe48dd1e9b30a85889ab0c0c09b7761bd5ede1c
Author: Lee moon soo
AuthorDate: Fri Feb 1 15:08:44 2019 -0800
[ZEPPELIN-3920] [FOLLOWUP] get correct hyperlink address when num job url
is one
### What is this PR for?
After change https://github.com/apache/zeppelin/pull/3273, invalid value is
assigned to the Job link when only one JobUrl exists.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3920
### How should this be tested?
Test manually
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3299 from Leemoonsoo/ZEPPELIN-3920-followup and squashes the
following commits:
6f9d3f8ce [Lee moon soo] get correct hyperlink address when num job url is
one
---
zeppelin-web/src/app/notebook/paragraph/paragraph-control.html | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/zeppelin-web/src/app/notebook/paragraph/paragraph-control.html
b/zeppelin-web/src/app/notebook/paragraph/paragraph-control.html
index b39ac08..841ca5f 100644
--- a/zeppelin-web/src/app/notebook/paragraph/paragraph-control.html
+++ b/zeppelin-web/src/app/notebook/paragraph/paragraph-control.html
@@ -15,7 +15,7 @@ limitations under the License.
-
{{paragraph.runtimeInfos.jobUrl.label}}
[zeppelin] branch master updated: [ZEPPELIN-3982] add how to setup development mode for Kubernetes support
This is an automated email from the ASF dual-hosted git repository. moon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/zeppelin.git The following commit(s) were added to refs/heads/master by this push: new 82749aa [ZEPPELIN-3982] add how to setup development mode for Kubernetes support 82749aa is described below commit 82749aa15b57370fc3a7ed93c7fb88bc1cabb651 Author: Lee moon soo AuthorDate: Fri Feb 1 14:56:11 2019 -0800 [ZEPPELIN-3982] add how to setup development mode for Kubernetes support ### What is this PR for? This pr adds docs that explains development mode for Kubernetes support. ### What type of PR is it? Documentation ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-3982 ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Lee moon soo Closes #3298 from Leemoonsoo/ZEPPELIN-3982 and squashes the following commits: 5e5cc652b [Lee moon soo] add kubernetes development mode --- docs/quickstart/kubernetes.md | 15 +++ 1 file changed, 15 insertions(+) diff --git a/docs/quickstart/kubernetes.md b/docs/quickstart/kubernetes.md index f8f6a6a..7b64e7f 100644 --- a/docs/quickstart/kubernetes.md +++ b/docs/quickstart/kubernetes.md @@ -251,3 +251,18 @@ to customize, - Blocking communication between interpreter Pod. - Spark Interpreter Pod has Role CRUD for any pod/service in the same namespace. Which should be restricted to only Spark executors Pod. - Per note interpreter mode by default when Zeppelin is running on Kubernetes + + +## Development + +Instead of build Zeppelin distribution package and docker image everytime during development, +Zeppelin can run locally (such as inside your IDE in debug mode) and able to run Interpreter using [K8sStandardInterpreterLauncher](https://github.com/apache/zeppelin/blob/master/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sStandardInterpreterLauncher.java) by configuring following environment variables. + +| Environment variable | Value | Description | +| - | - | - | +| ZEPPELIN_RUN_MODE | k8s | Make Zeppelin run interpreter on Kubernetes | +| ZEPPELIN_K8S_PORTFORWARD | true | Enable port forwarding from local Zeppelin instance to Interpreters running on Kubernetes | +| ZEPPELIN_K8S_CONTAINER_IMAGE | : | Zeppelin interpreter docker image to use | +| ZEPPELIN_K8S_SPARK_CONTAINER_IMAGE | : | Spark docker image to use | + +`kubectl` command need to be configured to connect your Kubernetes cluster.
svn commit: r1852467 - /zeppelin/site/docs/0.9.0-SNAPSHOT/asf-zeppelin/
Author: moon Date: Tue Jan 29 18:36:53 2019 New Revision: 1852467 URL: http://svn.apache.org/viewvc?rev=1852467=rev Log: delete unused dir Removed: zeppelin/site/docs/0.9.0-SNAPSHOT/asf-zeppelin/
svn commit: r1852466 [1/4] - in /zeppelin/site/docs/0.9.0-SNAPSHOT: ./ assets/themes/zeppelin/img/screenshots/ interpreter/ quickstart/
Author: moon Date: Tue Jan 29 18:35:14 2019 New Revision: 1852466 URL: http://svn.apache.org/viewvc?rev=1852466=rev Log: update 0.9.0-SNAPSHOT doc Added: zeppelin/site/docs/0.9.0-SNAPSHOT/assets/themes/zeppelin/img/screenshots/form_password.png (with props) zeppelin/site/docs/0.9.0-SNAPSHOT/assets/themes/zeppelin/img/screenshots/form_password_prog.png (with props) zeppelin/site/docs/0.9.0-SNAPSHOT/interpreter/hazelcastjet.html zeppelin/site/docs/0.9.0-SNAPSHOT/interpreter/java.html zeppelin/site/docs/0.9.0-SNAPSHOT/quickstart/kubernetes.html Modified: zeppelin/site/docs/0.9.0-SNAPSHOT/atom.xml zeppelin/site/docs/0.9.0-SNAPSHOT/interpreter/spark.html zeppelin/site/docs/0.9.0-SNAPSHOT/rss.xml zeppelin/site/docs/0.9.0-SNAPSHOT/search_data.json zeppelin/site/docs/0.9.0-SNAPSHOT/sitemap.txt Added: zeppelin/site/docs/0.9.0-SNAPSHOT/assets/themes/zeppelin/img/screenshots/form_password.png URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.9.0-SNAPSHOT/assets/themes/zeppelin/img/screenshots/form_password.png?rev=1852466=auto == Binary file - no diff available. Propchange: zeppelin/site/docs/0.9.0-SNAPSHOT/assets/themes/zeppelin/img/screenshots/form_password.png -- svn:mime-type = application/octet-stream Added: zeppelin/site/docs/0.9.0-SNAPSHOT/assets/themes/zeppelin/img/screenshots/form_password_prog.png URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.9.0-SNAPSHOT/assets/themes/zeppelin/img/screenshots/form_password_prog.png?rev=1852466=auto == Binary file - no diff available. Propchange: zeppelin/site/docs/0.9.0-SNAPSHOT/assets/themes/zeppelin/img/screenshots/form_password_prog.png -- svn:mime-type = application/octet-stream Modified: zeppelin/site/docs/0.9.0-SNAPSHOT/atom.xml URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.9.0-SNAPSHOT/atom.xml?rev=1852466=1852465=1852466=diff == --- zeppelin/site/docs/0.9.0-SNAPSHOT/atom.xml (original) +++ zeppelin/site/docs/0.9.0-SNAPSHOT/atom.xml Tue Jan 29 18:35:14 2019 @@ -4,7 +4,7 @@ Apache Zeppelin http://zeppelin.apache.org/; rel="self"/> http://zeppelin.apache.org"/> - 2019-01-23T11:53:07+08:00 + 2019-01-29T10:31:21-08:00 http://zeppelin.apache.org The Apache Software Foundation Added: zeppelin/site/docs/0.9.0-SNAPSHOT/interpreter/hazelcastjet.html URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.9.0-SNAPSHOT/interpreter/hazelcastjet.html?rev=1852466=auto == --- zeppelin/site/docs/0.9.0-SNAPSHOT/interpreter/hazelcastjet.html (added) +++ zeppelin/site/docs/0.9.0-SNAPSHOT/interpreter/hazelcastjet.html Tue Jan 29 18:35:14 2019 @@ -0,0 +1,410 @@ + + + + + +Apache Zeppelin 0.9.0-SNAPSHOT Documentation: Hazelcast Jet interpreter in Apache Zeppelin + + + + + + + + + + + + + + + + + + + +https://code.jquery.com/jquery-1.10.2.min.js"</a>;> + + + + + + + + + + + + + + + + + + +Toggle navigation + + + + + +http://zeppelin.apache.org;> + + Zeppelin + 0.9.0-SNAPSHOT + + + + + + + + Quick Start + +Getting Started +Install +Kubernetes +Explore UI +Tutorial + +Spark with Zeppelin +SQL with Zeppelin +Python with Zeppelin + + + + + Usage + +Dynamic Form +What is Dynamic Form? + +Display System +Text Display +HTML Display +Table Display +Network Display +Angular Display using Backend API +Angular Display using Frontend API + +Interpreter +Overview +Interpreter Binding Mode +User Impersonation +Dependency Management +Installing Interpreters + +Execution Hooks (Experimental) + +Other Features +Publishing Paragraphs +Personalized Mode +
svn commit: r1852466 [4/4] - in /zeppelin/site/docs/0.9.0-SNAPSHOT: ./ assets/themes/zeppelin/img/screenshots/ interpreter/ quickstart/
Modified: zeppelin/site/docs/0.9.0-SNAPSHOT/sitemap.txt URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.9.0-SNAPSHOT/sitemap.txt?rev=1852466=1852465=1852466=diff == --- zeppelin/site/docs/0.9.0-SNAPSHOT/sitemap.txt (original) +++ zeppelin/site/docs/0.9.0-SNAPSHOT/sitemap.txt Tue Jan 29 18:35:14 2019 @@ -1,99 +1,99 @@ -http://zeppelin.apache.org/assets/themes/zeppelin/js/search.js -http://zeppelin.apache.org/atom.xml -http://zeppelin.apache.org/development/contribution/how_to_contribute_code.html -http://zeppelin.apache.org/development/contribution/how_to_contribute_website.html -http://zeppelin.apache.org/development/contribution/useful_developer_tools.html -http://zeppelin.apache.org/development/helium/overview.html -http://zeppelin.apache.org/development/helium/writing_application.html -http://zeppelin.apache.org/development/helium/writing_spell.html -http://zeppelin.apache.org/development/helium/writing_visualization_basic.html -http://zeppelin.apache.org/development/helium/writing_visualization_transformation.html -http://zeppelin.apache.org/development/writing_zeppelin_interpreter.html -http://zeppelin.apache.org/index.html -http://zeppelin.apache.org/interpreter/alluxio.html -http://zeppelin.apache.org/interpreter/beam.html -http://zeppelin.apache.org/interpreter/bigquery.html -http://zeppelin.apache.org/interpreter/cassandra.html -http://zeppelin.apache.org/interpreter/elasticsearch.html -http://zeppelin.apache.org/interpreter/flink.html -http://zeppelin.apache.org/interpreter/geode.html +http://zeppelin.apache.org/interpreter/livy.html +http://zeppelin.apache.org/interpreter/pig.html +http://zeppelin.apache.org/interpreter/markdown.html +http://zeppelin.apache.org/interpreter/mahout.html +http://zeppelin.apache.org/interpreter/spark.html +http://zeppelin.apache.org/interpreter/python.html +http://zeppelin.apache.org/interpreter/hive.html +http://zeppelin.apache.org/interpreter/ignite.html http://zeppelin.apache.org/interpreter/groovy.html +http://zeppelin.apache.org/interpreter/sap.html +http://zeppelin.apache.org/interpreter/kylin.html http://zeppelin.apache.org/interpreter/hazelcastjet.html -http://zeppelin.apache.org/interpreter/hbase.html http://zeppelin.apache.org/interpreter/hdfs.html -http://zeppelin.apache.org/interpreter/hive.html -http://zeppelin.apache.org/interpreter/ignite.html +http://zeppelin.apache.org/interpreter/hbase.html +http://zeppelin.apache.org/interpreter/beam.html +http://zeppelin.apache.org/interpreter/geode.html +http://zeppelin.apache.org/interpreter/r.html http://zeppelin.apache.org/interpreter/java.html -http://zeppelin.apache.org/interpreter/jdbc.html -http://zeppelin.apache.org/interpreter/kylin.html +http://zeppelin.apache.org/interpreter/flink.html +http://zeppelin.apache.org/interpreter/postgresql.html +http://zeppelin.apache.org/interpreter/cassandra.html http://zeppelin.apache.org/interpreter/lens.html -http://zeppelin.apache.org/interpreter/livy.html -http://zeppelin.apache.org/interpreter/mahout.html -http://zeppelin.apache.org/interpreter/markdown.html +http://zeppelin.apache.org/interpreter/elasticsearch.html +http://zeppelin.apache.org/interpreter/jdbc.html http://zeppelin.apache.org/interpreter/neo4j.html -http://zeppelin.apache.org/interpreter/pig.html -http://zeppelin.apache.org/interpreter/postgresql.html -http://zeppelin.apache.org/interpreter/python.html -http://zeppelin.apache.org/interpreter/r.html -http://zeppelin.apache.org/interpreter/sap.html -http://zeppelin.apache.org/interpreter/scalding.html +http://zeppelin.apache.org/interpreter/bigquery.html http://zeppelin.apache.org/interpreter/scio.html http://zeppelin.apache.org/interpreter/shell.html -http://zeppelin.apache.org/interpreter/spark.html -http://zeppelin.apache.org/pleasecontribute.html -http://zeppelin.apache.org/quickstart/explore_ui.html -http://zeppelin.apache.org/quickstart/install.html -http://zeppelin.apache.org/quickstart/kubernetes.html -http://zeppelin.apache.org/quickstart/python_with_zeppelin.html -http://zeppelin.apache.org/quickstart/spark_with_zeppelin.html -http://zeppelin.apache.org/quickstart/sql_with_zeppelin.html -http://zeppelin.apache.org/quickstart/tutorial.html +http://zeppelin.apache.org/interpreter/scalding.html +http://zeppelin.apache.org/interpreter/alluxio.html http://zeppelin.apache.org/rss.xml -http://zeppelin.apache.org/screenshots.html http://zeppelin.apache.org/search.html +http://zeppelin.apache.org/development/helium/writing_visualization_transformation.html +http://zeppelin.apache.org/development/helium/writing_spell.html +http://zeppelin.apache.org/development/helium/writing_visualization_basic.html +http://zeppelin.apache.org/development/helium/overview.html +http://zeppelin.apache.org/development/helium/writing_application.html +http://zeppelin.apache.org/development/writing_zeppelin_interpreter.html
svn commit: r1852466 [3/4] - in /zeppelin/site/docs/0.9.0-SNAPSHOT: ./ assets/themes/zeppelin/img/screenshots/ interpreter/ quickstart/
Modified: zeppelin/site/docs/0.9.0-SNAPSHOT/search_data.json
URL:
http://svn.apache.org/viewvc/zeppelin/site/docs/0.9.0-SNAPSHOT/search_data.json?rev=1852466=1852465=1852466=diff
==
--- zeppelin/site/docs/0.9.0-SNAPSHOT/search_data.json (original)
+++ zeppelin/site/docs/0.9.0-SNAPSHOT/search_data.json Tue Jan 29 18:35:14 2019
@@ -1,169 +1,166 @@
{
-
-
-"/development/contribution/how_to_contribute_code.html": {
- "title": "Contributing to Apache Zeppelin (Code)",
- "content" : "!--Licensed under the Apache License, Version 2.0 (the
License);you may not use this file except in compliance with the
License.You may obtain a copy of the License
athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law
or agreed to in writing, softwaredistributed under the License is distributed
on an AS IS BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
either express or implied.See the License for the specific language governing
permissions andlimitations under the License.--Contributing to Apache
Zeppelin ( Code )NOTE : Apache Zeppelin is an Apache2 License Software.Any
contributions to Zeppelin (Source code, Documents, Image, Website) means you
agree with license all your contributions as Apache2 License.Setting upHere are
some tools you will need to build and test Zeppelin.Software Configuration
Management ( SCM )Since Zeppelin uses Git for it#39;s SCM system, you need
git client installed in y
our development machine.Integrated Development Environment ( IDE )You are free
to use whatever IDE you prefer, or your favorite command line editor.Build
ToolsTo build the code, installOracle Java 8Apache MavenGetting the source
codeFirst of all, you need Zeppelin source code. The official location of
Zeppelin is http://git.apache.org/zeppelin.git.git accessGet the source code on
your development machine using git.git clone git://git.apache.org/zeppelin.git
zeppelinYou may also want to develop against a specific branch. For example,
for branch-0.5.6git clone -b branch-0.5.6 git://git.apache.org/zeppelin.git
zeppelinApache Zeppelin follows Fork amp; Pull as a source control
workflow.If you want to not only build Zeppelin but also make any changes, then
you need to fork Zeppelin github mirror repository and make a pull
request.Before making a pull request, please take a look Contribution
Guidelines.Buildmvn installTo skip testmvn install -DskipTestsTo build with
specific spark /
hadoop versionmvn install -Dspark.version=x.x.x -Dhadoop.version=x.x.xFor the
further Run Zeppelin server in development modeOption 1 - Command LineCopy the
conf/zeppelin-site.xml.template to
zeppelin-server/src/main/resources/zeppelin-site.xml and change the
configurations in this file if requiredRun the following commandcd
zeppelin-serverHADOOP_HOME=YOUR_HADOOP_HOME JAVA_HOME=YOUR_JAVA_HOME mvn
exec:java
-Dexec.mainClass=quot;org.apache.zeppelin.server.ZeppelinServerquot;
-Dexec.args=quot;quot;Option 2 - Daemon ScriptNote: Make sure you
first run mvn clean install -DskipTestsin your zeppelin root directory,
otherwise your server build will fail to find the required dependencies in the
local repro.or use daemon scriptbin/zeppelin-daemon startServer will be run on
http://localhost:8080.Option 3 - IDECopy the conf/zeppelin-site.xml.template to
zeppelin-server/src/main/resources/zeppelin-site.xml and change the
configurations in this file if requiredZeppelinServer.
java Main classGenerating Thrift CodeSome portions of the Zeppelin code are
generated by Thrift. For most Zeppelin changes, you don#39;t need to worry
about this. But if you modify any of the Thrift IDL files (e.g.
zeppelin-interpreter/src/main/thrift/*.thrift), then you also need to
regenerate these files and submit their updated version as part of your
patch.To regenerate the code, install thrift-0.9.2 and then run the following
command to generate thrift code.cd
lt;zeppelin_homegt;/zeppelin-interpreter/src/main/thrift./genthrift.shRun
Selenium testZeppelin has set of integration tests using Selenium. To run
these test, first build and run Zeppelin and make sure Zeppelin is running on
port 8080. Then you can run test using following commandTEST_SELENIUM=true mvn
test -Dtest=[TEST_NAME] -DfailIfNoTests=false -pl
#39;zeppelin-interpreter,zeppelin-zengine,zeppelin-server#39;For
example, to run ParagraphActionIT,TEST_SELENIUM=true mvn test
-Dtest=ParagraphActi
onsIT -DfailIfNoTests=false -pl
#39;zeppelin-interpreter,zeppelin-zengine,zeppelin-server#39;You#39;ll
need Firefox web browser installed in your development environment. While CI
server uses Firefox 31.0 to run selenium test, it is good idea to install the
same version (disable auto update to keep the version).Where to StartYou can
find issues for beginner amp; newbieStay involvedContributors should join
the Zeppelin mailing lists@zeppelin.apache.org is
svn commit: r1852466 [2/4] - in /zeppelin/site/docs/0.9.0-SNAPSHOT: ./ assets/themes/zeppelin/img/screenshots/ interpreter/ quickstart/
Added: zeppelin/site/docs/0.9.0-SNAPSHOT/quickstart/kubernetes.html URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.9.0-SNAPSHOT/quickstart/kubernetes.html?rev=1852466=auto == --- zeppelin/site/docs/0.9.0-SNAPSHOT/quickstart/kubernetes.html (added) +++ zeppelin/site/docs/0.9.0-SNAPSHOT/quickstart/kubernetes.html Tue Jan 29 18:35:14 2019 @@ -0,0 +1,514 @@ + + + + + +Apache Zeppelin 0.9.0-SNAPSHOT Documentation: Install + + + + + + + + + + + + + + + + + + + +https://code.jquery.com/jquery-1.10.2.min.js";> + + + + + + + + + + + + + + + + + + +Toggle navigation + + + + + +http://zeppelin.apache.org;> + + Zeppelin + 0.9.0-SNAPSHOT + + + + + + + + Quick Start + +Getting Started +Install +Kubernetes +Explore UI +Tutorial + +Spark with Zeppelin +SQL with Zeppelin +Python with Zeppelin + + + + + Usage + +Dynamic Form +What is Dynamic Form? + +Display System +Text Display +HTML Display +Table Display +Network Display +Angular Display using Backend API +Angular Display using Frontend API + +Interpreter +Overview +Interpreter Binding Mode +User Impersonation +Dependency Management +Installing Interpreters + +Execution Hooks (Experimental) + +Other Features +Publishing Paragraphs +Personalized Mode +Customizing Zeppelin Homepage +Notebook Actions +Cron Scheduler +Zeppelin Context + +REST API +Interpreter API +Zeppelin Server API +Notebook API +Notebook Repository API +Configuration API +Credential API +Helium API + + + + + Setup + +Basics +How to Build Zeppelin +Multi-user Support + +Deployment + +Spark Cluster Mode: Standalone +Spark Cluster Mode: YARN +Spark Cluster Mode: Mesos +Zeppelin with Flink, Spark Cluster +Zeppelin on CDH +Zeppelin on VM: Vagrant + +Security +HTTP Basic Auth using NGINX +Shiro Authentication +Notebook Authorization +Data Source Authorization +HTTP Security Headers + +Notebook Storage +Git Storage +S3 Storage +Azure Storage +ZeppelinHub Storage +MongoDB Storage + +Operation +Configuration +Proxy Setting +Upgrading +Trouble Shooting + + + + + Interpreter + +Interpreters +Overview + +Spark +JDBC +Python + +Alluxio +Beam +BigQuery +Cassandra +Elasticsearch +Flink +Geode +Groovy +Hazelcast Jet +HBase +HDFS +Hive +Ignite +Java +Kylin +Lens +Livy +Markdown +Neo4j +Pig +Postgresql, HAWQ +R +Scalding +Scio +Shell + + + + More + +Extending Zeppelin +Writing Zeppelin Interpreter + +Helium (Experimental) +Overview +Writing Helium
[zeppelin] branch master updated: [ZEPPELIN-3840] Zeppelin on Kubernetes
This is an automated email from the ASF dual-hosted git repository. moon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/zeppelin.git The following commit(s) were added to refs/heads/master by this push: new b13651c [ZEPPELIN-3840] Zeppelin on Kubernetes b13651c is described below commit b13651cedf15cff45e823738144ec2e565dc5807 Author: Lee moon soo AuthorDate: Fri Dec 7 10:45:35 2018 -0800 [ZEPPELIN-3840] Zeppelin on Kubernetes ### What type of PR is it? This PR adds ability to run Zeppelin on Kubernetes. It aims - Zero configuration to start Zeppelin on Kubernetes. (and Spark on Kubernetes) - Run everything on Kubernetes: Zeppelin, Interpreters, Spark. - Highly customizable to adopt various user configurations and extensions. Key features are - Provides zeppelin-server.yaml file for `kubectl` to run Zeppelin server - All interpreters are automatically running as a Pod. - Spark interpreter automatically configured to use [Spark on Kubernetes](https://spark.apache.org/docs/latest/running-on-kubernetes.html) - Reverse proxy is configured to access Spark UI To do - [x] Document how reverse proxy for Spark UI works and how to configure custom domain. - [x] Document how to customize zeppelin-server and interpreter yaml. - [x] Document new configurations - [x] Document how to mount volume for notebook and configurations ### How it works Run Zeppelin Server on Kubernetes `k8s/zeppelin-server.yaml` is provided to run Zeppelin Server with few sidecars and configurations. This file is easy to publish (user can easily consume it using `curl`), highly customizable while it includes all the necessary things. K8s Interpreter launcher This PR adds new module, `launcher-k8s-standard` under `zeppelin/zeppelin-plugins/launcher/k8s-standard/` directory. This launcher is [automatically being selected](https://github.com/apache/zeppelin/pull/3240/files#diff-82fddd2ffb77aaffc4b9cf7b5b1eaa79) when Zeppelin is running on Kubernetes. The launcher both handles Spark interpreter and All other interpreters. The launcher launches interpreter as a Pod using template [k8s/interpreter/100-interpreter-pod.yaml](https://github.com/apache/zeppelin/pull/3240/files#diff-d9ce62e2c992d32f0184d7edb862f3c4). Reason filename has `100-` in prefix is because all files in the directory is consumed in alphabetical order by launcher on interpreter start/stop. User can drop more files here to extend/customize interpreter, and filename can be used to control order. The template is rendered by [jinjava](https://github.com/HubSpot/jinjava). Spark interpreter When interpreter group is `spark`, K8sRemoteInterpreterProcess [sets necessary spark configuration](https://github.com/apache/zeppelin/pull/3240/files#diff-6d1d3084f55bdd519e39ede4a619e73dR297) automatically to use [Spark on Kubernetes](https://spark.apache.org/docs/latest/running-on-kubernetes.html). User doesn't have to configure anything. It uses client mode. Spark UI We may make user manually configure port-forward or do something to access Spark UI, but that's not optimal. It is the best when Spark UI is automatically accessible when user have access to Zeppelin UI, without any extra configuration. To enable this, Zeppelin server Pod has a reverse proxy as a sidecar, and it split traffic to Zeppelin server and Spark UI running in the other Pod. It assume both `service.domain.com` and `*.service.domain.com` point the nginx proxy address. `service.domain.com` is directed to ZeppelinServer, `*.service.domain.com` is directed to interpreter Pod. `-.service.domain.com` is convention to access any application running in interpreter Pod. If Spark interpreter Pod is running with a name `spark-axefeg` and Spark UI is running on port 4040, ``` 4040-spark-axefeg.service.domain.com ``` is the address to access Spark UI. Default service domain is [local.zeppelin-project.org:8080](https://github.com/apache/zeppelin/pull/3240/files#diff-56ccb2e2c2617b27dbaae866d9431e51R22), while `local.zeppelin-project.org` and `*.local.zeppelin-project.org` point `127.0.0.1`, and it works with `kubectl port-forward`. ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-3840 ### How should this be tested? Prepare a Kubernetes cluster with enough resources (cpus > 5, mem > 6g). If you're using [minikube](https://github.com/kubernetes/minikube), check your capacity using `kubectl describe node` command before start. You'll need to build Zeppelin docker image and Spark docker image to test. Please follow guide docs/quickstart/kubernetes.md. To quickly try without building docker images, I have uploaded pre-built image on
zeppelin git commit: [MINOR] Let jenkins print some build summary on failure
Repository: zeppelin Updated Branches: refs/heads/master 6e97a527e -> 09dc9fcae [MINOR] Let jenkins print some build summary on failure ### What is this PR for? Currently actual CI build occurs in individual contributor's travis-ci account, and apache Jenkins poll the result from travis-ci and print build status on it's console output, like ``` Build https://travis-ci.org/test/zeppelin/builds/451036679 [1] OK https://travis-ci.org/test/zeppelin/jobs/451036680 [2] Running ... https://travis-ci.org/test/zeppelin/jobs/451036681 [3] Running ... https://travis-ci.org/test/zeppelin/jobs/451036682 [4] Not completed https://travis-ci.org/test/zeppelin/jobs/451036683 [5] Running ... https://travis-ci.org/test/zeppelin/jobs/451036684 [6] Running ... https://travis-ci.org/test/zeppelin/jobs/451036685 [7] Running ... https://travis-ci.org/test/zeppelin/jobs/451036686 [8] Not started https://travis-ci.org/test/zeppelin/jobs/451036687 [9] Not started https://travis-ci.org/test/zeppelin/jobs/451036688 [10] Not startedhttps://travis-ci.org/test/zeppelin/jobs/451036689 [11] Not startedhttps://travis-ci.org/test/zeppelin/jobs/451036690 1 job(s) failed, 9 job(s) running/pending ``` When there're failed job, developer has to open the travis-ci link and download the log and manually look through to see what was wrong. This PR make Jenkins not only print build status but also short summary of failure, so developer quickly identify what went wrong. After this patch, Jenkins console output will look something like ``` [2018-11-18 09:11:20] Author: test, commit: ee6d89b [2018-11-18 09:11:20] [2018-11-18 09:11:25] Get build status ... Build https://travis-ci.org/test/zeppelin/builds/451036679 [1] OK https://travis-ci.org/test/zeppelin/jobs/451036680 [2] OK https://travis-ci.org/test/zeppelin/jobs/451036681 [3] Error 1 https://travis-ci.org/test/zeppelin/jobs/451036682 Tests run: 38, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 153.857 sec <<< FAILURE! - in org.apache.zeppelin.notebook.NotebookTest Please check full log at https://api.travis-ci.org/v3/job/451036682/log.txt [4] Not completed https://travis-ci.org/test/zeppelin/jobs/451036683 Your build has been stopped. Please check full log at https://api.travis-ci.org/v3/job/451036683/log.txt [5] OK https://travis-ci.org/test/zeppelin/jobs/451036684 [6] Error 1 https://travis-ci.org/test/zeppelin/jobs/451036685 Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 14.374 sec <<< FAILURE! - in org.apache.zeppelin.spark.SparkRInterpreterTest Please check full log at https://api.travis-ci.org/v3/job/451036685/log.txt [7] Error 1 https://travis-ci.org/test/zeppelin/jobs/451036686 Tests run: 15, Failures: 15, Errors: 0, Skipped: 0, Time elapsed: 216.32 sec <<< FAILURE! - in org.apache.zeppelin.interpreter.SparkIntegrationTest Please check full log at https://api.travis-ci.org/v3/job/451036686/log.txt [8] Error 1 https://travis-ci.org/test/zeppelin/jobs/451036687 Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 17.021 sec <<< FAILURE! - in org.apache.zeppelin.spark.SparkRInterpreterTest Please check full log at https://api.travis-ci.org/v3/job/451036687/log.txt [9] Error 1 https://travis-ci.org/test/zeppelin/jobs/451036688 Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 18.169 sec <<< FAILURE! - in org.apache.zeppelin.spark.SparkRInterpreterTest Please check full log at https://api.travis-ci.org/v3/job/451036688/log.txt [10] OK https://travis-ci.org/test/zeppelin/jobs/451036689 [11] Error 1https://travis-ci.org/test/zeppelin/jobs/451036690 Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 353.585 sec <<< FAILURE! - in org.apache.zeppelin.livy.LivyInterpreterIT Please check full log at https://api.travis-ci.org/v3/job/451036690/log.txt 7 job(s) failed, 0 job(s) running/pending ``` ### What type of PR is it? Improvement ### How should this be tested? * try script manually against some build e.g. ``` python ./travis_check.py [github username] [commit hash] python ./travis_check.py NicolasRouquette ee6d89b ``` Author: Lee moon soo Closes #3231 from Leemoonsoo/ci_summary and squashes the following commits: 9f42b3226 [Lee moon soo] print build log summary Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/09dc9fca Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/09dc9fca Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/09dc9fca Branch: refs/heads/master Commit: 09dc9fcae542815728758b09a32b19456134dd89 Parents: 6e97a52 Autho
zeppelin git commit: [ZEPPELIN-3847] Duplicate results in notebooks due to WS interruption
Repository: zeppelin
Updated Branches:
refs/heads/master b31f2998e -> 6e97a527e
[ZEPPELIN-3847] Duplicate results in notebooks due to WS interruption
### What is this PR for?
Duplicate results in notebooks after ws reconnect.
The problem occurs via duplicated 'setConnectedStatus' event handler in
notebook.controller.js.
The same function exists at
https://github.com/Leemoonsoo/zeppelin/blob/dbbf0436490941d117fd4c06b9da5b6cb47697ff/zeppelin-web/src/app/notebook/notebook.controller.js#L99-L104.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3847
### How should this be tested?
Disconnect and reconnect websocket connection to browser, and check if result
is duplicated.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3230 from Leemoonsoo/ZEPPELIN-3847 and squashes the following commits:
dbbf04364 [Lee moon soo] remove duplicated setConnectedStatus event handler
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/6e97a527
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/6e97a527
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/6e97a527
Branch: refs/heads/master
Commit: 6e97a527e1a2c84846a18340ad11f05ff6432cdc
Parents: b31f299
Author: Lee moon soo
Authored: Sat Nov 17 10:06:30 2018 +0900
Committer: Lee moon soo
Committed: Mon Nov 19 07:26:32 2018 +0900
--
zeppelin-web/src/app/notebook/notebook.controller.js | 7 ---
1 file changed, 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/6e97a527/zeppelin-web/src/app/notebook/notebook.controller.js
--
diff --git a/zeppelin-web/src/app/notebook/notebook.controller.js
b/zeppelin-web/src/app/notebook/notebook.controller.js
index 047b022..2f1437d 100644
--- a/zeppelin-web/src/app/notebook/notebook.controller.js
+++ b/zeppelin-web/src/app/notebook/notebook.controller.js
@@ -1361,13 +1361,6 @@ function NotebookCtrl($scope, $route, $routeParams,
$location, $rootScope,
$scope.$broadcast('focusParagraph', paragraph.id, row + 1, col);
};
- $scope.$on('setConnectedStatus', function(event, param) {
-if (connectedOnce && param) {
- initNotebook();
-}
-connectedOnce = true;
- });
-
$scope.$on('moveParagraphUp', function(event, paragraph) {
let newIndex = -1;
for (let i = 0; i < $scope.note.paragraphs.length; i++) {
zeppelin git commit: [ZEPPELIN-3847] Duplicate results in notebooks due to WS interruption
Repository: zeppelin
Updated Branches:
refs/heads/branch-0.8 04532305e -> 30719e8c0
[ZEPPELIN-3847] Duplicate results in notebooks due to WS interruption
### What is this PR for?
Duplicate results in notebooks after ws reconnect.
The problem occurs via duplicated 'setConnectedStatus' event handler in
notebook.controller.js.
The same function exists at
https://github.com/Leemoonsoo/zeppelin/blob/dbbf0436490941d117fd4c06b9da5b6cb47697ff/zeppelin-web/src/app/notebook/notebook.controller.js#L99-L104.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3847
### How should this be tested?
Disconnect and reconnect websocket connection to browser, and check if result
is duplicated.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3230 from Leemoonsoo/ZEPPELIN-3847 and squashes the following commits:
dbbf04364 [Lee moon soo] remove duplicated setConnectedStatus event handler
(cherry picked from commit 6e97a527e1a2c84846a18340ad11f05ff6432cdc)
Signed-off-by: Lee moon soo
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/30719e8c
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/30719e8c
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/30719e8c
Branch: refs/heads/branch-0.8
Commit: 30719e8c07553fe210a70586b0b5e32bde06c645
Parents: 0453230
Author: Lee moon soo
Authored: Sat Nov 17 10:06:30 2018 +0900
Committer: Lee moon soo
Committed: Mon Nov 19 07:26:43 2018 +0900
--
zeppelin-web/src/app/notebook/notebook.controller.js | 7 ---
1 file changed, 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/30719e8c/zeppelin-web/src/app/notebook/notebook.controller.js
--
diff --git a/zeppelin-web/src/app/notebook/notebook.controller.js
b/zeppelin-web/src/app/notebook/notebook.controller.js
index 5135e1b..a03098b 100644
--- a/zeppelin-web/src/app/notebook/notebook.controller.js
+++ b/zeppelin-web/src/app/notebook/notebook.controller.js
@@ -1308,13 +1308,6 @@ function NotebookCtrl($scope, $route, $routeParams,
$location, $rootScope,
$scope.$broadcast('focusParagraph', paragraph.id, row + 1, col);
};
- $scope.$on('setConnectedStatus', function(event, param) {
-if (connectedOnce && param) {
- initNotebook();
-}
-connectedOnce = true;
- });
-
$scope.$on('moveParagraphUp', function(event, paragraph) {
let newIndex = -1;
for (let i = 0; i < $scope.note.paragraphs.length; i++) {
svn commit: r1846764 - in /zeppelin/site/docs: 0.8.0/usage/rest_api/interpreter.html 0.8.0/usage/rest_api/notebook.html 0.9.0-SNAPSHOT/usage/rest_api/interpreter.html 0.9.0-SNAPSHOT/usage/rest_api/not
Author: moon
Date: Fri Nov 16 20:29:56 2018
New Revision: 1846764
URL: http://svn.apache.org/viewvc?rev=1846764=rev
Log:
https://github.com/apache/zeppelin/pull/3224
https://github.com/apache/zeppelin/pull/3225
Modified:
zeppelin/site/docs/0.8.0/usage/rest_api/interpreter.html
zeppelin/site/docs/0.8.0/usage/rest_api/notebook.html
zeppelin/site/docs/0.9.0-SNAPSHOT/usage/rest_api/interpreter.html
zeppelin/site/docs/0.9.0-SNAPSHOT/usage/rest_api/notebook.html
Modified: zeppelin/site/docs/0.8.0/usage/rest_api/interpreter.html
URL:
http://svn.apache.org/viewvc/zeppelin/site/docs/0.8.0/usage/rest_api/interpreter.html?rev=1846764=1846763=1846764=diff
==
--- zeppelin/site/docs/0.8.0/usage/rest_api/interpreter.html (original)
+++ zeppelin/site/docs/0.8.0/usage/rest_api/interpreter.html Fri Nov 16
20:29:56 2018
@@ -556,7 +556,7 @@ Note that Apache Zeppelin REST APIs rece
{
- status: CREATED,
+ status: OK,
message: ,
body: {
id: 2AYW25ANY,
Modified: zeppelin/site/docs/0.8.0/usage/rest_api/notebook.html
URL:
http://svn.apache.org/viewvc/zeppelin/site/docs/0.8.0/usage/rest_api/notebook.html?rev=1846764=1846763=1846764=diff
==
--- zeppelin/site/docs/0.8.0/usage/rest_api/notebook.html (original)
+++ zeppelin/site/docs/0.8.0/usage/rest_api/notebook.html Fri Nov 16 20:29:56
2018
@@ -374,7 +374,7 @@ If you work with Apache Zeppelin and fin
sample JSON response
{
- status: CREATED,
+ status: OK,
message: ,
body: 2AZPHY918
}
@@ -594,7 +594,7 @@ If you work with Apache Zeppelin and fin
sample JSON response
{
- status: CREATED,
+ status: OK,
message: ,
body: 2AZPHY918
}
@@ -709,7 +709,7 @@ If you work with Apache Zeppelin and fin
sample JSON response
{
- status: CREATED,
+ status: OK,
message: ,
body: 2AZPHY918
}
@@ -897,7 +897,7 @@ If you work with Apache Zeppelin and fin
sample JSON response
{
- status: CREATED,
+ status: OK,
message: ,
body: 20151218-100330_1754029574
}
@@ -1427,7 +1427,8 @@ If you work with Apache Zeppelin and fin
Description
- This POST method adds cron job by the given note id.
+ This POST method adds cron job by the given note id.
+ Default value of releaseResource is false.
@@ -1444,7 +1445,7 @@ If you work with Apache Zeppelin and fin
sample JSON input
- {cron: cron expression of
note}
+ {cron: cron expression of note,
releaseResource: false}
sample JSON response
@@ -1490,7 +1491,7 @@ If you work with Apache Zeppelin and fin
Description
This GET method gets cron job expression of given note
id.
- The body field of the returned JSON contains the cron expression.
+ The body field of the returned JSON contains the cron expression and
releaseResource flag.
@@ -1507,7 +1508,14 @@ If you work with Apache Zeppelin and fin
sample JSON response
- {status: OK, body: *
* * * * ?}
+
+{
+ status: OK,
+ body: {
+ cron: 0 0/1 * * * ?,
+ releaseResource: true
+ }
+}
Modified: zeppelin/site/docs/0.9.0-SNAPSHOT/usage/rest_api/interpreter.html
URL:
http://svn.apache.org/viewvc/zeppelin/site/docs/0.9.0-SNAPSHOT/usage/rest_api/interpreter.html?rev=1846764=1846763=1846764=diff
==
--- zeppelin/site/docs/0.9.0-SNAPSHOT/usage/rest_api/interpreter.html (original)
+++ zeppelin/site/docs/0.9.0-SNAPSHOT/usage/rest_api/interpreter.html Fri Nov
16 20:29:56 2018
@@ -179,10 +179,12 @@
Flink
Geode
Groovy
+Hazelcast Jet
HBase
HDFS
Hive
Ignite
+Java
Kylin
Lens
Livy
@@ -556,7 +558,7 @@ Note that Apache Zeppelin REST APIs rece
{
- status: CREATED,
+ status: OK,
message: ,
body: {
id: 2AYW25ANY,
Modified: zeppelin/site/docs/0.9.0-SNAPSHOT/usage/rest_api/notebook.html
URL:
http://svn.apache.org/viewvc/zeppelin/site/docs/0.9.0-SNAPSHOT/usage/rest_api/notebook.html?rev=1846764=1846763=1846764=diff
==
--- zeppelin/site/docs/0.9.0-SNAPSHOT/usage/rest_api/notebook.html (original)
+++ zeppelin/site/docs/0.9.0-SNAPSHOT/usage/rest_api/notebook.html Fri Nov 16
20:29:56 2018
@@ -179,10 +179,12 @@
Flink
Geode
Groovy
+Hazelcast Jet
HBase
HDFS
Hive
Ignite
+Java
zeppelin git commit: docs [branch-0.8]: REST API JSON response status fix
Repository: zeppelin
Updated Branches:
refs/heads/branch-0.8 2ca8d19ba -> 04532305e
docs [branch-0.8]: REST API JSON response status fix
### What is this PR for?
Since v0.8.0, REST API JSON response `status` attribute has changed from
`CREATED` to `OK`.
Related source code:
*
https://github.com/apache/zeppelin/blob/master/zeppelin-server/src/main/java/org/apache/zeppelin/rest/NotebookRestApi.java
*
https://github.com/apache/zeppelin/blob/master/zeppelin-server/src/main/java/org/apache/zeppelin/rest/InterpreterRestApi.java
### What type of PR is it?
Documentation
### Questions:
* Shall I add tests to check the returned `status` in:
*
https://github.com/apache/zeppelin/blob/master/zeppelin-server/src/test/java/org/apache/zeppelin/rest/InterpreterRestApiTest.java
*
https://github.com/apache/zeppelin/blob/master/zeppelin-server/src/test/java/org/apache/zeppelin/rest/NotebookRestApiTest.java
Author: John Lau
Closes #3225 from jolks/branch-0.8_doc_rest_api_fix and squashes the following
commits:
139706003 [John Lau] REST API JSON response status fix
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/04532305
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/04532305
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/04532305
Branch: refs/heads/branch-0.8
Commit: 04532305e22704b05727c9098ae23f9a622aa725
Parents: 2ca8d19
Author: John Lau
Authored: Thu Nov 15 02:08:04 2018 +0900
Committer: Lee moon soo
Committed: Sat Nov 17 05:00:02 2018 +0900
--
docs/usage/rest_api/interpreter.md | 2 +-
docs/usage/rest_api/notebook.md| 8
2 files changed, 5 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/04532305/docs/usage/rest_api/interpreter.md
--
diff --git a/docs/usage/rest_api/interpreter.md
b/docs/usage/rest_api/interpreter.md
index 23a7c66..c750289 100644
--- a/docs/usage/rest_api/interpreter.md
+++ b/docs/usage/rest_api/interpreter.md
@@ -315,7 +315,7 @@ The role of registered interpreters, settings and
interpreters group are describ
{
- "status": "CREATED",
+ "status": "OK",
"message": "",
"body": {
"id": "2AYW25ANY",
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/04532305/docs/usage/rest_api/notebook.md
--
diff --git a/docs/usage/rest_api/notebook.md b/docs/usage/rest_api/notebook.md
index 3cbfc52..74d98db 100644
--- a/docs/usage/rest_api/notebook.md
+++ b/docs/usage/rest_api/notebook.md
@@ -132,7 +132,7 @@ Notebooks REST API supports the following operations: List,
Create, Get, Delete,
sample JSON response
{
- "status": "CREATED",
+ "status": "OK",
"message": "",
"body": "2AZPHY918"
}
@@ -344,7 +344,7 @@ Notebooks REST API supports the following operations: List,
Create, Get, Delete,
sample JSON response
{
- "status": "CREATED",
+ "status": "OK",
"message": "",
"body": "2AZPHY918"
}
@@ -455,7 +455,7 @@ Notebooks REST API supports the following operations: List,
Create, Get, Delete,
sample JSON response
{
- "status": "CREATED",
+ "status": "OK",
"message": "",
"body": "2AZPHY918"
}
@@ -636,7 +636,7 @@ Notebooks REST API supports the following operations: List,
Create, Get, Delete,
sample JSON response
{
- "status": "CREATED",
+ "status": "OK",
"message": "",
"body": "20151218-100330\_1754029574"
}
zeppelin git commit: docs [master]: REST API JSON response status fix
Repository: zeppelin
Updated Branches:
refs/heads/master a7f3f3c55 -> b31f2998e
docs [master]: REST API JSON response status fix
### What is this PR for?
Since v0.8.0, REST API JSON response `status` attribute has changed from
`CREATED` to `OK`.
Related source code:
*
https://github.com/apache/zeppelin/blob/master/zeppelin-server/src/main/java/org/apache/zeppelin/rest/NotebookRestApi.java
*
https://github.com/apache/zeppelin/blob/master/zeppelin-server/src/main/java/org/apache/zeppelin/rest/InterpreterRestApi.java
### What type of PR is it?
Documentation
### Questions:
* Shall I add tests to check the returned `status` in:
*
https://github.com/apache/zeppelin/blob/master/zeppelin-server/src/test/java/org/apache/zeppelin/rest/InterpreterRestApiTest.java
*
https://github.com/apache/zeppelin/blob/master/zeppelin-server/src/test/java/org/apache/zeppelin/rest/NotebookRestApiTest.java
Author: John Lau
Closes #3224 from jolks/doc_rest_api_fix and squashes the following commits:
9037b55c6 [John Lau] REST API JSON response status fix
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/b31f2998
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/b31f2998
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/b31f2998
Branch: refs/heads/master
Commit: b31f2998ef6eed49f053c90b36adaa6a8338d982
Parents: a7f3f3c
Author: John Lau
Authored: Tue Nov 13 18:04:08 2018 +0900
Committer: Lee moon soo
Committed: Sat Nov 17 04:59:25 2018 +0900
--
docs/usage/rest_api/interpreter.md | 2 +-
docs/usage/rest_api/notebook.md| 8
2 files changed, 5 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/b31f2998/docs/usage/rest_api/interpreter.md
--
diff --git a/docs/usage/rest_api/interpreter.md
b/docs/usage/rest_api/interpreter.md
index 23a7c66..c750289 100644
--- a/docs/usage/rest_api/interpreter.md
+++ b/docs/usage/rest_api/interpreter.md
@@ -315,7 +315,7 @@ The role of registered interpreters, settings and
interpreters group are describ
{
- "status": "CREATED",
+ "status": "OK",
"message": "",
"body": {
"id": "2AYW25ANY",
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/b31f2998/docs/usage/rest_api/notebook.md
--
diff --git a/docs/usage/rest_api/notebook.md b/docs/usage/rest_api/notebook.md
index 960847a..4d2b2cf 100644
--- a/docs/usage/rest_api/notebook.md
+++ b/docs/usage/rest_api/notebook.md
@@ -132,7 +132,7 @@ Notebooks REST API supports the following operations: List,
Create, Get, Delete,
sample JSON response
{
- "status": "CREATED",
+ "status": "OK",
"message": "",
"body": "2AZPHY918"
}
@@ -344,7 +344,7 @@ Notebooks REST API supports the following operations: List,
Create, Get, Delete,
sample JSON response
{
- "status": "CREATED",
+ "status": "OK",
"message": "",
"body": "2AZPHY918"
}
@@ -491,7 +491,7 @@ Notebooks REST API supports the following operations: List,
Create, Get, Delete,
sample JSON response
{
- "status": "CREATED",
+ "status": "OK",
"message": "",
"body": "2AZPHY918"
}
@@ -672,7 +672,7 @@ Notebooks REST API supports the following operations: List,
Create, Get, Delete,
sample JSON response
{
- "status": "CREATED",
+ "status": "OK",
"message": "",
"body": "20151218-100330\_1754029574"
}
svn commit: r1846730 - in /zeppelin/site: ./ contribution/ contribution/zeppelinweb/ releases/
Author: moon Date: Fri Nov 16 17:03:24 2018 New Revision: 1846730 URL: http://svn.apache.org/viewvc?rev=1846730=rev Log: ZEPPELIN-3868 Modified: zeppelin/site/assets.html zeppelin/site/atom.xml zeppelin/site/community.html zeppelin/site/contribution/contributions.html zeppelin/site/contribution/documentation.html zeppelin/site/contribution/webapplication.html zeppelin/site/contribution/zeppelinweb/goodPracticeGuide01.html zeppelin/site/contribution/zeppelinweb/goodPracticeGuide02.html zeppelin/site/contribution/zeppelinweb/goodPracticeGuide03.html zeppelin/site/contribution/zeppelinweb/goodPracticeGuide04.html zeppelin/site/documentation.html zeppelin/site/download.html zeppelin/site/helium_packages.html zeppelin/site/releases/zeppelin-release-0.5.0-incubating.html zeppelin/site/releases/zeppelin-release-0.5.5-incubating.html zeppelin/site/releases/zeppelin-release-0.5.6-incubating.html zeppelin/site/releases/zeppelin-release-0.6.0.html zeppelin/site/releases/zeppelin-release-0.6.1.html zeppelin/site/releases/zeppelin-release-0.6.2.html zeppelin/site/releases/zeppelin-release-0.7.0.html zeppelin/site/releases/zeppelin-release-0.7.1.html zeppelin/site/releases/zeppelin-release-0.7.2.html zeppelin/site/releases/zeppelin-release-0.7.3.html zeppelin/site/rss.xml zeppelin/site/screenshots.html zeppelin/site/sitemap.txt zeppelin/site/supported_interpreters.html Modified: zeppelin/site/assets.html URL: http://svn.apache.org/viewvc/zeppelin/site/assets.html?rev=1846730=1846729=1846730=diff == --- zeppelin/site/assets.html (original) +++ zeppelin/site/assets.html Fri Nov 16 17:03:24 2018 @@ -73,7 +73,7 @@ -Quick Start +Quick Start Download Modified: zeppelin/site/atom.xml URL: http://svn.apache.org/viewvc/zeppelin/site/atom.xml?rev=1846730=1846729=1846730=diff == --- zeppelin/site/atom.xml (original) +++ zeppelin/site/atom.xml Fri Nov 16 17:03:24 2018 @@ -4,7 +4,7 @@ Apache Zeppelin http://zeppelin.apache.org/; rel="self"/> http://zeppelin.apache.org"/> - 2018-06-29T08:44:38+08:00 + 2018-11-17T01:51:51+09:00 http://zeppelin.apache.org The Apache Software Foundation Modified: zeppelin/site/community.html URL: http://svn.apache.org/viewvc/zeppelin/site/community.html?rev=1846730=1846729=1846730=diff == --- zeppelin/site/community.html (original) +++ zeppelin/site/community.html Fri Nov 16 17:03:24 2018 @@ -73,7 +73,7 @@ -Quick Start +Quick Start Download Modified: zeppelin/site/contribution/contributions.html URL: http://svn.apache.org/viewvc/zeppelin/site/contribution/contributions.html?rev=1846730=1846729=1846730=diff == --- zeppelin/site/contribution/contributions.html (original) +++ zeppelin/site/contribution/contributions.html Fri Nov 16 17:03:24 2018 @@ -73,7 +73,7 @@ -Quick Start +Quick Start Download @@ -165,6 +165,10 @@ + + + + Documentation @@ -173,10 +177,6 @@ - - - - Modified: zeppelin/site/contribution/documentation.html URL: http://svn.apache.org/viewvc/zeppelin/site/contribution/documentation.html?rev=1846730=1846729=1846730=diff == --- zeppelin/site/contribution/documentation.html (original) +++ zeppelin/site/contribution/documentation.html Fri Nov 16 17:03:24 2018 @@ -73,7 +73,7 @@ -Quick Start +Quick Start Download @@ -165,6 +165,10 @@ + + + + Documentation @@ -173,10 +177,6 @@ - - - - Modified: zeppelin/site/contribution/webapplication.html URL: http://svn.apache.org/viewvc/zeppelin/site/contribution/webapplication.html?rev=1846730=1846729=1846730=diff == --- zeppelin/site/contribution/webapplication.html (original) +++ zeppelin/site/contribution/webapplication.html Fri Nov 16 17:03:24 2018 @@ -73,7 +73,7 @@ -Quick Start +Quick Start Download @@ -161,12 +161,12 @@ - - + + Modified: zeppelin/site/contribution/zeppelinweb/goodPracticeGu
[1/3] zeppelin git commit: [ZEPPELIN-3848] zeppelin-web dev mode does not work
Repository: zeppelin
Updated Branches:
refs/heads/master 31ceca797 -> d1606c248
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/d1606c24/zeppelin-web/package.json
--
diff --git a/zeppelin-web/package.json b/zeppelin-web/package.json
index 3b7d2c0..b5c5acd 100644
--- a/zeppelin-web/package.json
+++ b/zeppelin-web/package.json
@@ -38,10 +38,10 @@
"diff-match-patch": "1.0.0"
},
"devDependencies": {
-"autoprefixer": "^6.5.4",
-"babel-cli": "^6.18.0",
+"autoprefixer": "^6.7.7",
+"babel-cli": "^6.26.0",
"babel-core": "^6.26.3",
-"babel-loader": "^6.2.10",
+"babel-loader": "^7.1.4",
"babel-preset-env": "^1.7.0",
"bower": "^1.8.0",
"copy-webpack-plugin": "^4.0.1",
@@ -55,8 +55,7 @@
"eslint-plugin-standard": "^3.0.1",
"eslint-watch": "^3.1.0",
"express": "^4.14.0",
-"extract-text-webpack-plugin": "^1.0.1",
-"file-loader": "^0.9.0",
+"file-loader": "^2.0.0",
"grunt": "^0.4.1",
"grunt-cache-bust": "1.3.0",
"grunt-cli": "^0.1.13",
@@ -76,7 +75,7 @@
"grunt-svgmin": "^0.4.0",
"grunt-usemin": "^2.1.1",
"grunt-wiredep": "~2.0.0",
-"html-webpack-plugin": "^2.24.1",
+"html-webpack-plugin": "^3.2.0",
"imports-loader": "^0.7.1",
"istanbul-instrumenter-loader": "^0.2.0",
"jasmine-core": "^2.5.2",
@@ -89,17 +88,19 @@
"karma-spec-reporter": "0.0.31",
"karma-webpack": "^1.8.1",
"load-grunt-tasks": "^0.4.0",
+"mini-css-extract-plugin": "^0.4.4",
"ng-annotate-loader": "^0.2.0",
"npm-run-all": "^3.1.2",
-"postcss-loader": "^1.2.1",
+"postcss-loader": "^3.0.0",
"protractor": "^5.4.1",
"raw-loader": "^0.5.1",
"rimraf": "^2.5.4",
"string-replace-webpack-plugin": "^0.1.3",
"style-loader": "^0.13.1",
"time-grunt": "^0.3.1",
-"webpack": "^1.14.0",
-"webpack-dev-server": "^2.11.3"
+"webpack": "^4.24.0",
+"webpack-cli": "^3.1.2",
+"webpack-dev-server": "^3.1.10"
},
"repository": {
"type": "git",
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/d1606c24/zeppelin-web/webpack.config.js
--
diff --git a/zeppelin-web/webpack.config.js b/zeppelin-web/webpack.config.js
index 3c34924..ccb33df 100644
--- a/zeppelin-web/webpack.config.js
+++ b/zeppelin-web/webpack.config.js
@@ -19,9 +19,8 @@
var path = require('path');
var webpack = require('webpack');
-var autoprefixer = require('autoprefixer');
var HtmlWebpackPlugin = require('html-webpack-plugin');
-var ExtractTextPlugin = require('extract-text-webpack-plugin');
+var MiniCssExtractPlugin = require('mini-css-extract-plugin');
var CopyWebpackPlugin = require('copy-webpack-plugin');
var InsertLiveReloadPlugin = function InsertLiveReloadPlugin(options) {
this.options = options || {};
@@ -138,19 +137,18 @@ module.exports = function makeWebpackConfig () {
// Initialize module
config.module = {
-preLoaders: [],
-loaders: [{
+rules: [{
// headroom 0.9.3 doesn't work with webpack
//
https://github.com/WickyNilliams/headroom.js/issues/213#issuecomment-281106943
test: require.resolve('headroom.js'),
- loader: 'imports-loader?this=>window,define=>false,exports=>false'
+ use: 'imports-loader?this=>window,define=>false,exports=>false'
}, {
// JS LOADER
// Reference: https://github.com/babel/babel-loader
// Transpile .js files using babel-loader
// Compiles ES6 and ES7 into ES5 code
- test: /\.js$/,
- loaders: ['ng-annotate', 'babel-loader'],
+ test: /\.(js|jsx)$/,
+ use: ['ng-annotate-loader', 'babel-loader'],
exclude: /(node_modules|bower_components)/,
}, {
// CSS LOADER
@@ -159,13 +157,23 @@ module.exports = function makeWebpackConfig () {
//
// Reference: https://github.com/postcss/postcss-loader
// Postprocess your css with PostCSS plugins
- test: /\.css$/,
- // Reference: https://github.com/webpack/extract-text-webpack-plugin
+ test: /\.(sa|sc|c)ss$/,
+ // Reference: https://github.com/webpack-contrib/mini-css-extract-plugin
// Extract css files in production builds
//
- // Reference: https://github.com/webpack/style-loader
// Use style-loader in development.
- loader: ExtractTextPlugin.extract('style-loader',
'css-loader?sourceMap!postcss-loader')
+ use: [
+!isProd ? 'style-loader' : MiniCssExtractPlugin.loader,
+'css-loader',
+{
+ loader: 'postcss-loader',
+ options: {
+ident: 'postcss',
+plugins: [
+ require('autoprefixer')()
+]
+ }
+}]
}, {
// ASSET LOADER
// Reference: https://github.com/webpack/file-loader
@@ -174,20 +182,20 @@ module.exports = function makeWebpackConfig () {
// Pass
[3/3] zeppelin git commit: [ZEPPELIN-3848] zeppelin-web dev mode does not work
[ZEPPELIN-3848] zeppelin-web dev mode does not work ### What is this PR for? zeppelin-web development mode is broken. `npm run dev` throws error ``` Error: Cannot find module 'webpack/bin/config-yargs' ``` This PR fixes error by upgrading webpack from version 1.x to 4.x ### What type of PR is it? Bug Fix ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-3848 ### How should this be tested? check if `npm run dev` works under zeppelin-web dir. And able to browse localhost:9000 ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Lee moon soo Closes #3217 from Leemoonsoo/upgrade_webpack and squashes the following commits: 5b9bc0491 [Lee moon soo] http->https ff7e10f21 [Lee moon soo] update package-lock.json with sha512 instead of sha1 1fc1fff6e [Lee moon soo] update webpack config and package-lock b66768eb3 [Lee moon soo] upgrade webpack version Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/d1606c24 Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/d1606c24 Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/d1606c24 Branch: refs/heads/master Commit: d1606c248d4398d929683de284c62d630d19806c Parents: 31ceca7 Author: Lee moon soo Authored: Tue Nov 6 14:51:47 2018 -0800 Committer: Lee moon soo Committed: Wed Nov 7 14:26:45 2018 -0800 -- zeppelin-web/package-lock.json | 8738 +-- zeppelin-web/package.json | 19 +- zeppelin-web/webpack.config.js | 91 +- 3 files changed, 5376 insertions(+), 3472 deletions(-) --
[2/3] zeppelin git commit: [ZEPPELIN-3848] zeppelin-web dev mode does not work
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/d1606c24/zeppelin-web/package-lock.json
--
diff --git a/zeppelin-web/package-lock.json b/zeppelin-web/package-lock.json
index b398c9f..e3cbbaf 100644
--- a/zeppelin-web/package-lock.json
+++ b/zeppelin-web/package-lock.json
@@ -5,21 +5,205 @@
"requires": true,
"dependencies": {
"@types/node": {
- "version": "6.0.117",
- "resolved": "https://registry.npmjs.org/@types/node/-/node-6.0.117.tgz;,
- "integrity":
"sha512-sihk0SnN8PpiS5ihu5xJQ5ddnURNq4P+XPmW+nORlKkHy21CoZO/IVHK/Wq/l3G8fFW06Fkltgnqx229uPlnRg==",
+ "version": "6.14.1",
+ "resolved": "https://registry.npmjs.org/@types/node/-/node-6.14.1.tgz;,
+ "integrity":
"sha512-lfydm+Ul6buYjF6AmcenFjqVYB+tpNlGlwWLr43J1Cok4ybLQimrM8rdB1MdrjtyWdYRceZNgsAeSRZhFxNajQ==",
"dev": true
},
"@types/q": {
"version": "0.0.32",
- "resolved": "http://registry.npmjs.org/@types/q/-/q-0.0.32.tgz;,
+ "resolved": "https://registry.npmjs.org/@types/q/-/q-0.0.32.tgz;,
"integrity": "sha1-vShOV8hPEyXacCur/IKlMoGQwMU=",
"dev": true
},
"@types/selenium-webdriver": {
- "version": "3.0.10",
- "resolved":
"https://registry.npmjs.org/@types/selenium-webdriver/-/selenium-webdriver-3.0.10.tgz;,
- "integrity":
"sha512-ikB0JHv6vCR1KYUQAzTO4gi/lXLElT4Tx+6De2pc/OZwizE9LRNiTa+U8TBFKBD/nntPnr/MPSHSnOTybjhqNA==",
+ "version": "3.0.12",
+ "resolved":
"https://registry.npmjs.org/@types/selenium-webdriver/-/selenium-webdriver-3.0.12.tgz;,
+ "integrity":
"sha512-hYn+eoOehVUIdMwp5h34ZsGAO1ydja10GDup4BwyoFCdcH5MQ35nQq+AInSaBMEMopD5hEooFCyKo2Pajbe1ag==",
+ "dev": true
+},
+"@webassemblyjs/ast": {
+ "version": "1.7.11",
+ "resolved":
"https://registry.npmjs.org/@webassemblyjs/ast/-/ast-1.7.11.tgz;,
+ "integrity":
"sha512-ZEzy4vjvTzScC+SH8RBssQUawpaInUdMTYwYYLh54/s8TuT0gBLuyUnppKsVyZEi876VmmStKsUs28UxPgdvrA==",
+ "dev": true,
+ "requires": {
+"@webassemblyjs/helper-module-context": "1.7.11",
+"@webassemblyjs/helper-wasm-bytecode": "1.7.11",
+"@webassemblyjs/wast-parser": "1.7.11"
+ }
+},
+"@webassemblyjs/floating-point-hex-parser": {
+ "version": "1.7.11",
+ "resolved":
"https://registry.npmjs.org/@webassemblyjs/floating-point-hex-parser/-/floating-point-hex-parser-1.7.11.tgz;,
+ "integrity":
"sha512-zY8dSNyYcgzNRNT666/zOoAyImshm3ycKdoLsyDw/Bwo6+/uktb7p4xyApuef1dwEBo/U/SYQzbGBvV+nru2Xg==",
+ "dev": true
+},
+"@webassemblyjs/helper-api-error": {
+ "version": "1.7.11",
+ "resolved":
"https://registry.npmjs.org/@webassemblyjs/helper-api-error/-/helper-api-error-1.7.11.tgz;,
+ "integrity":
"sha512-7r1qXLmiglC+wPNkGuXCvkmalyEstKVwcueZRP2GNC2PAvxbLYwLLPr14rcdJaE4UtHxQKfFkuDFuv91ipqvXg==",
+ "dev": true
+},
+"@webassemblyjs/helper-buffer": {
+ "version": "1.7.11",
+ "resolved":
"https://registry.npmjs.org/@webassemblyjs/helper-buffer/-/helper-buffer-1.7.11.tgz;,
+ "integrity":
"sha512-MynuervdylPPh3ix+mKZloTcL06P8tenNH3sx6s0qE8SLR6DdwnfgA7Hc9NSYeob2jrW5Vql6GVlsQzKQCa13w==",
+ "dev": true
+},
+"@webassemblyjs/helper-code-frame": {
+ "version": "1.7.11",
+ "resolved":
"https://registry.npmjs.org/@webassemblyjs/helper-code-frame/-/helper-code-frame-1.7.11.tgz;,
+ "integrity":
"sha512-T8ESC9KMXFTXA5urJcyor5cn6qWeZ4/zLPyWeEXZ03hj/x9weSokGNkVCdnhSabKGYWxElSdgJ+sFa9G/RdHNw==",
+ "dev": true,
+ "requires": {
+"@webassemblyjs/wast-printer": "1.7.11"
+ }
+},
+"@webassemblyjs/helper-fsm": {
+ "version": "1.7.11",
+ "resolved":
"https://registry.npmjs.org/@webassemblyjs/helper-fsm/-/helper-fsm-1.7.11.tgz;,
+ "integrity":
"sha512-nsAQWNP1+8Z6tkzdYlXT0kxfa2Z1tRTARd8wYnc/e3Zv3VydVVnaeePgqUzFrpkGUyhUUxOl5ML7f1NuT+gC0A==",
+ "dev": true
+},
+"@webassemblyjs/helper-module-context": {
+ "version": "1.7.11",
+ "resolved":
"https://registry.npmjs.org/@webassemblyjs/helper-module-context/-/helper-module-context-1.7.11.tgz;,
+ "integrity":
"sha512-JxfD5DX8Ygq4PvXDucq0M+sbUFA7BJAv/GGl9ITovqE+idGX+J3QSzJYz+LwQmL7fC3Rs+utvWoJxDb6pmC0qg==",
+ "dev": true
+},
+"@webassemblyjs/helper-wasm-bytecode": {
+ "version": "1.7.11",
+ "resolved":
"https://registry.npmjs.org/@webassemblyjs/helper-wasm-bytecode/-/helper-wasm-bytecode-1.7.11.tgz;,
+ "integrity":
"sha512-cMXeVS9rhoXsI9LLL4tJxBgVD/KMOKXuFqYb5oCJ/opScWpkCMEz9EJtkonaNcnLv2R3K5jIeS4TRj/drde1JQ==",
+ "dev": true
+},
+"@webassemblyjs/helper-wasm-section": {
+ "version": "1.7.11",
+ "resolved":
"https://registry.npmjs.org/@webassemblyjs/helper-wasm-section/-/helper-wasm-section-1.7.11.tgz;,
+ "integrity":
"sha512-8ZRY5iZbZdtNFE5UFunB8mmBEAbSI3guwbrsCl4fWdfRiAcvqQpeqd5KHhSWLL5wuxo53zcaGZDBU64qgn4I4Q==",
+ "dev": true,
+
zeppelin git commit: [ZEPPELIN-3850] Fix run all paragraph behavior
Repository: zeppelin
Updated Branches:
refs/heads/master c82de2f3d -> 9f04c8fcb
[ZEPPELIN-3850] Fix run all paragraph behavior
### What is this PR for?
"Run all paragraphs" button runs only the first paragraph of the note.
This PR fixes the problem.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3850
### How should this be tested?
Unittest included
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3219 from Leemoonsoo/ZEPPELIN-3850 and squashes the following commits:
6cf5dd859 [Lee moon soo] add test runAllParagraphs
1ea145c22 [Lee moon soo] check condition correctly to run all paragraphs
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/9f04c8fc
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/9f04c8fc
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/9f04c8fc
Branch: refs/heads/master
Commit: 9f04c8fcba289c90b0754ce920f9a90dcd15a137
Parents: c82de2f
Author: Lee moon soo
Authored: Mon Nov 5 10:17:09 2018 -0800
Committer: Lee moon soo
Committed: Tue Nov 6 08:53:56 2018 -0800
--
.../org/apache/zeppelin/service/NotebookService.java | 2 +-
.../apache/zeppelin/service/NotebookServiceTest.java | 13 +
2 files changed, 14 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/9f04c8fc/zeppelin-server/src/main/java/org/apache/zeppelin/service/NotebookService.java
--
diff --git
a/zeppelin-server/src/main/java/org/apache/zeppelin/service/NotebookService.java
b/zeppelin-server/src/main/java/org/apache/zeppelin/service/NotebookService.java
index c2e99d2..4cf3bf6 100644
---
a/zeppelin-server/src/main/java/org/apache/zeppelin/service/NotebookService.java
+++
b/zeppelin-server/src/main/java/org/apache/zeppelin/service/NotebookService.java
@@ -397,7 +397,7 @@ public class NotebookService {
Map params = (Map) raw.get("params");
Map config = (Map) raw.get("config");
- if (runParagraph(noteId, paragraphId, title, text, params, config,
false, true,
+ if (!runParagraph(noteId, paragraphId, title, text, params, config,
false, true,
context, callback)) {
// stop execution when one paragraph fails.
break;
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/9f04c8fc/zeppelin-server/src/test/java/org/apache/zeppelin/service/NotebookServiceTest.java
--
diff --git
a/zeppelin-server/src/test/java/org/apache/zeppelin/service/NotebookServiceTest.java
b/zeppelin-server/src/test/java/org/apache/zeppelin/service/NotebookServiceTest.java
index 0cfdcc1..9d8a735 100644
---
a/zeppelin-server/src/test/java/org/apache/zeppelin/service/NotebookServiceTest.java
+++
b/zeppelin-server/src/test/java/org/apache/zeppelin/service/NotebookServiceTest.java
@@ -19,6 +19,8 @@
package org.apache.zeppelin.service;
import com.google.common.collect.Maps;
+import com.google.gson.Gson;
+import com.google.gson.reflect.TypeToken;
import org.apache.commons.lang.StringUtils;
import org.apache.zeppelin.conf.ZeppelinConfiguration;
import org.apache.zeppelin.interpreter.Interpreter;
@@ -63,6 +65,7 @@ import static org.mockito.Matchers.eq;
import static org.mockito.Mockito.doCallRealMethod;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.reset;
+import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
@@ -75,6 +78,8 @@ public class NotebookServiceTest {
private ServiceCallback callback = mock(ServiceCallback.class);
+ private Gson gson = new Gson();
+
@Before
public void setUp() throws Exception {
@@ -320,6 +325,14 @@ public class NotebookServiceTest {
assertTrue(runStatus);
verify(callback).onSuccess(p, context);
+// run all paragraphs
+reset(callback);
+notebookService.runAllParagraphs(
+note1.getId(),
+gson.fromJson(gson.toJson(note1.getParagraphs()), new
TypeToken(){}.getType()),
+context, callback);
+verify(callback, times(2)).onSuccess(any(), any());
+
// run paragraph synchronously via invalid code
//TODO(zjffdu) must sleep for a while, otherwise will get wrong status.
This should be due to
//bug of job component.
zeppelin git commit: [ZEPPELIN-3849] display note name correctly.
Repository: zeppelin
Updated Branches:
refs/heads/master 81e7030f7 -> c82de2f3d
[ZEPPELIN-3849] display note name correctly.
### What is this PR for?
Master branch displays note id instead of note name.
I think it's side effect after change
https://github.com/apache/zeppelin/pull/3163.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3849
### How should this be tested?
See if note name is displayed instead of note id
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3218 from Leemoonsoo/ZEPPELIN-3849 and squashes the following commits:
3e91b990f [Lee moon soo] display note name
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/c82de2f3
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/c82de2f3
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/c82de2f3
Branch: refs/heads/master
Commit: c82de2f3d3b50691f7148af41668d377b861907c
Parents: 81e7030
Author: Lee moon soo
Authored: Mon Nov 5 09:52:01 2018 -0800
Committer: Lee moon soo
Committed: Tue Nov 6 08:53:03 2018 -0800
--
.../src/components/array-ordering/array-ordering.service.js | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/c82de2f3/zeppelin-web/src/components/array-ordering/array-ordering.service.js
--
diff --git
a/zeppelin-web/src/components/array-ordering/array-ordering.service.js
b/zeppelin-web/src/components/array-ordering/array-ordering.service.js
index 22f7b7a..1f275e6 100644
--- a/zeppelin-web/src/components/array-ordering/array-ordering.service.js
+++ b/zeppelin-web/src/components/array-ordering/array-ordering.service.js
@@ -27,10 +27,10 @@ function ArrayOrderingService(TRASH_FOLDER_ID) {
};
this.getNoteName = function(note) {
-if (note.path === undefined || note.path.trim() === '') {
+if (note.name === undefined || note.name.trim() === '') {
return 'Note ' + note.id;
} else {
- return note.path;
+ return note.name;
}
};
zeppelin git commit: [ZEPPELIN-3825] Allow custom service account for GCSNotebookRepo
Repository: zeppelin Updated Branches: refs/heads/master d3028c21d -> 81e7030f7 [ZEPPELIN-3825] Allow custom service account for GCSNotebookRepo ### What is this PR for? The current implementation uses the default google applications service account to establish a connection with GCS. We cannot specify a custom service account for the connection with the existing implementation. This PR enables the same. * Added ZEPPELIN_NOTEBOOK_GCS_CREDENTIALS_FILE to ConfVars * Added functionality in GCSNotebookRepo and OldGCSNotebookRepo to get credentials from CREDENTIALS_FILE * Updated doc string and documentation ### What type of PR is it? [Improvement] ### Todos * None ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-3825 ### How should this be tested? * Create a new service account that has access to write to google cloud storage. * Disable Google Storage access to the default application service account. * Build and deploy zeppelin after updating zeppelin.notebook.gcs.credentialsJsonFilePath in zeppelin-site.xml * Validate that new notes are being written to GCS bucket ### Screenshots (if appropriate) * None ### Questions: * Does the licenses files need update? -No * Is there breaking changes for older versions? - No * Does this needs documentation? - Yes. Existing documentation has been updated to reflect the same. Author: sanjaykumar Closes #3207 from 88sanjay/ZEPPELIN-3825 and squashes the following commits: 15c8aa0fb [sanjaykumar] Merge branch 'master' into ZEPPELIN-3825 84b8da4d5 [sanjaykumar] ZEPPELIN-3825. Allow custom service account for GCSNotebookRepo Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/81e7030f Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/81e7030f Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/81e7030f Branch: refs/heads/master Commit: 81e7030f7f8cdf31a95dac141265dc5499ecaf4a Parents: d3028c2 Author: sanjaykumar Authored: Mon Nov 5 11:55:55 2018 -0500 Committer: Lee moon soo Committed: Mon Nov 5 16:37:52 2018 -0800 -- conf/zeppelin-site.xml.template | 9 + docs/setup/storage/storage.md | 18 -- .../zeppelin/conf/ZeppelinConfiguration.java | 1 + .../zeppelin/notebook/repo/GCSNotebookRepo.java | 15 +-- .../notebook/repo/OldGCSNotebookRepo.java | 15 +-- 5 files changed, 52 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/81e7030f/conf/zeppelin-site.xml.template -- diff --git a/conf/zeppelin-site.xml.template b/conf/zeppelin-site.xml.template index 9d9a99f..3920fb9 100755 --- a/conf/zeppelin-site.xml.template +++ b/conf/zeppelin-site.xml.template @@ -85,6 +85,15 @@ + zeppelin.notebook.gcs.credentialsJsonFilePath + path/to/key.json + +Path to GCS credential key file for authentication with Google Storage. + + + + + zeppelin.notebook.storage org.apache.zeppelin.notebook.repo.GCSNotebookRepo notebook persistence layer implementation http://git-wip-us.apache.org/repos/asf/zeppelin/blob/81e7030f/docs/setup/storage/storage.md -- diff --git a/docs/setup/storage/storage.md b/docs/setup/storage/storage.md index e826460..6ef3453 100644 --- a/docs/setup/storage/storage.md +++ b/docs/setup/storage/storage.md @@ -317,7 +317,7 @@ Or, if you want to simultaneously use your local git storage with GCS, use the f ### Google Cloud API Authentication Note: On Google App Engine, Google Cloud Shell, and Google Compute Engine, these -steps are not necessary, as build-in credentials are used by default. +steps are not necessary if you are using the default built in service account. For more information, see [Application Default Credentials](https://cloud.google.com/docs/authentication/production) @@ -351,11 +351,25 @@ for authentication with GCS, you will need a JSON service account key file. `/path/to/my/key.json`), and give it appropriate permissions. Ensure at least the user running the zeppelin daemon can read it. -Then, point `GOOGLE_APPLICATION_CREDENTIALS` at your new key file in **zeppelin-env.sh**. For example: + If you wish to set this as your default credential file to access Google Services, + point `GOOGLE_APPLICATION_CREDENTIALS` at your new key file in **zeppelin-env.sh**. For example: ```bash export GOOGLE_APPLICATION_CREDENTIALS=/path/to/my/key.json ``` +If you do not want to use this key file as default credential file and want to specify a custom key +file for authentication with GCS, update the following property : + +```
zeppelin git commit: [ZEPPELIN-3837] update how_to_build.md
Repository: zeppelin
Updated Branches:
refs/heads/master 2b234106d -> d3028c21d
[ZEPPELIN-3837] update how_to_build.md
### What is this PR for?
how_to_build.md includes some build profiles that has removed/changed.
This PR makes how_to_build.md up to dated.
### What type of PR is it?
Documentation
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3837
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Author: Lee moon soo
Closes #3212 from Leemoonsoo/ZEPPELIN-3837 and squashes the following commits:
704b8c81f [Lee moon soo] spark 2.1 -> 2.3 profile for binary release creation
9c35656f8 [Lee moon soo] update how_to_build.md
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/d3028c21
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/d3028c21
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/d3028c21
Branch: refs/heads/master
Commit: d3028c21dfe03bf4ab7cdf4ca1c0e432a487037a
Parents: 2b23410
Author: Lee moon soo
Authored: Sun Nov 4 09:01:07 2018 -0800
Committer: Lee moon soo
Committed: Sun Nov 4 17:33:23 2018 -0800
--
dev/create_release.sh | 4 +--
docs/setup/basics/how_to_build.md | 60 +-
2 files changed, 17 insertions(+), 47 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/d3028c21/dev/create_release.sh
--
diff --git a/dev/create_release.sh b/dev/create_release.sh
index 9cb61e0..ac4c67d 100755
--- a/dev/create_release.sh
+++ b/dev/create_release.sh
@@ -106,8 +106,8 @@ function make_binary_release() {
git_clone
make_source_package
-make_binary_release all "-Pspark-2.1 -Phadoop-2.6 -Pscala-${SCALA_VERSION}"
-make_binary_release netinst "-Pspark-2.1 -Phadoop-2.6 -Pscala-${SCALA_VERSION}
-pl
zeppelin-interpreter,zeppelin-zengine,:zeppelin-display_${SCALA_VERSION},:zeppelin-spark-dependencies_${SCALA_VERSION},:zeppelin-spark_${SCALA_VERSION},zeppelin-web,zeppelin-server,zeppelin-distribution
-am"
+make_binary_release all "-Pspark-2.3 -Phadoop-2.6 -Pscala-${SCALA_VERSION}"
+make_binary_release netinst "-Pspark-2.3 -Phadoop-2.6 -Pscala-${SCALA_VERSION}
-pl
zeppelin-interpreter,zeppelin-zengine,:zeppelin-display_${SCALA_VERSION},:zeppelin-spark-dependencies_${SCALA_VERSION},:zeppelin-spark_${SCALA_VERSION},zeppelin-web,zeppelin-server,zeppelin-distribution
-am"
# remove non release files and dirs
rm -rf "${WORKING_DIR}/zeppelin"
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/d3028c21/docs/setup/basics/how_to_build.md
--
diff --git a/docs/setup/basics/how_to_build.md
b/docs/setup/basics/how_to_build.md
index 85a59ca..942ef57 100644
--- a/docs/setup/basics/how_to_build.md
+++ b/docs/setup/basics/how_to_build.md
@@ -70,7 +70,7 @@ If you're unsure about the options, use the same commands
that creates official
# update all pom.xml to use scala 2.11
./dev/change_scala_version.sh 2.11
# build zeppelin with all interpreters and include latest version of Apache
spark support for local mode.
-mvn clean package -DskipTests -Pspark-2.0 -Phadoop-2.4 -Pr -Pscala-2.11
+mvn clean package -DskipTests -Pspark-2.3 -Phadoop-2.6 -Pscala-2.11
```
3. Done
@@ -98,35 +98,25 @@ Set spark major version
Available profiles are
```
+-Pspark-2.3
+-Pspark-2.2
-Pspark-2.1
-Pspark-2.0
-Pspark-1.6
--Pspark-1.5
--Pspark-1.4
--Pcassandra-spark-1.5
--Pcassandra-spark-1.4
--Pcassandra-spark-1.3
--Pcassandra-spark-1.2
--Pcassandra-spark-1.1
```
minor version can be adjusted by `-Dspark.version=x.x.x`
-# `-Phadoop-[version]`
+# `-Phadoop[version]`
-set hadoop major version
+set hadoop major version (default hadoop2)
Available profiles are
```
--Phadoop-0.23
--Phadoop-1
--Phadoop-2.2
--Phadoop-2.3
--Phadoop-2.4
--Phadoop-2.6
--Phadoop-2.7
+-Phadoop2
+-Phadoop3
```
minor version can be adjusted by `-Dhadoop.version=x.x.x`
@@ -144,27 +134,13 @@ Available profiles are
# `-Pr` (optional)
enable [R](https://www.r-project.org/) support with
[SparkR](https://spark.apache.org/docs/latest/sparkr.html) integration.
+Note that, this enables R interpreter which is different from sparkR included
in Spark interpreter by default.
# `-Pvendor-repo` (optional)
-enable 3rd party vendor repository (cloudera)
+enable 3rd party vendor repository (Cloudera, Hortonworks)
-# `-Pmapr[version]` (optional)
-
-For the MapR Hadoop Distribution, these profiles will handle the Hadoop
version. As MapR allows different versions of Spark to be installed,
svn commit: r1845670 - /zeppelin/site/.htaccess
Author: moon Date: Sat Nov 3 16:26:47 2018 New Revision: 1845670 URL: http://svn.apache.org/viewvc?rev=1845670=rev Log: ZEPPELIN-3836 update .htaccess for 0.8.0 release Modified: zeppelin/site/.htaccess Modified: zeppelin/site/.htaccess URL: http://svn.apache.org/viewvc/zeppelin/site/.htaccess?rev=1845670=1845669=1845670=diff == --- zeppelin/site/.htaccess (original) +++ zeppelin/site/.htaccess Sat Nov 3 16:26:47 2018 @@ -7,7 +7,7 @@ RewriteRule ^/?docs/0.6.0-(?!SNAPSHOT).* RewriteRule ^/?docs/(0.5.[056])(?!-incubating).*/(.*) /docs/$1-incubating/$3 [R=301,L,NE] # rewrite docs/latest to latest stable release -RewriteRule ^/?docs/latest/(.*) /docs/0.7.3/$1 [PT] +RewriteRule ^/?docs/latest/(.*) /docs/0.8.0/$1 [PT] # rewrite docs/snapshot to latest snapshot version -RewriteRule ^/?docs/snapshot/(.*) /docs/0.8.0-SNAPSHOT/$1 [PT] +RewriteRule ^/?docs/snapshot/(.*) /docs/0.9.0-SNAPSHOT/$1 [PT]
zeppelin git commit: [ZEPPELIN-3836] update website url rewrite rule for 0.8.0 release
Repository: zeppelin Updated Branches: refs/heads/gh-pages e44ef7bed -> 8583cf501 [ZEPPELIN-3836] update website url rewrite rule for 0.8.0 release ### What is this PR for? `https://zeppelin.apache.org/docs/latest` supposed to point latest release version documentation. But it currently points 0.7.3 version doc, instead of 0.8.0. ### What type of PR is it? Fix ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-3836 ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Lee moon soo Closes #3211 from Leemoonsoo/gh-pages-rewrite_rule_for_0.8 and squashes the following commits: 9f01b45b5 [Lee moon soo] update website url rewrite rule for 0.8.0 doc Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/8583cf50 Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/8583cf50 Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/8583cf50 Branch: refs/heads/gh-pages Commit: 8583cf5015857d1f7aca5a08cf2e8c622347b2a2 Parents: e44ef7b Author: Lee moon soo Authored: Tue Oct 30 15:53:01 2018 -0700 Committer: Lee moon soo Committed: Sat Nov 3 09:21:49 2018 -0700 -- .htaccess | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/8583cf50/.htaccess -- diff --git a/.htaccess b/.htaccess index efde503..05916da 100644 --- a/.htaccess +++ b/.htaccess @@ -7,7 +7,7 @@ RewriteRule ^/?docs/0.6.0-(?!SNAPSHOT).*/(.*) /docs/0.6.0/$2 [R=301,L,NE] RewriteRule ^/?docs/(0.5.[056])(?!-incubating).*/(.*) /docs/$1-incubating/$3 [R=301,L,NE] # rewrite docs/latest to latest stable release -RewriteRule ^/?docs/latest/(.*) /docs/0.7.3/$1 [PT] +RewriteRule ^/?docs/latest/(.*) /docs/0.8.0/$1 [PT] # rewrite docs/snapshot to latest snapshot version -RewriteRule ^/?docs/snapshot/(.*) /docs/0.8.0-SNAPSHOT/$1 [PT] +RewriteRule ^/?docs/snapshot/(.*) /docs/0.9.0-SNAPSHOT/$1 [PT]
[3/3] zeppelin git commit: [ZEPPELIN-3839] Replace | to , in the filename
[ZEPPELIN-3839] Replace | to , in the filename ### What is this PR for? A tutorial notebook filename, which includes `|`, is not compatible with windows filesystem. This PR replace `|` with `,`. ### What type of PR is it? Bug Fix ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-3839 ### How should this be tested? CI pass ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Lee moon soo Closes #3213 from Leemoonsoo/ZEPPELIN-3839 and squashes the following commits: 0c1a5dda8 [Lee moon soo] Replace | to , in the filename Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/383cdc3f Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/383cdc3f Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/383cdc3f Branch: refs/heads/master Commit: 383cdc3fa4a9316dd1a2d9fd559879e2a49463e7 Parents: a2473da Author: Lee moon soo Authored: Wed Oct 31 12:02:58 2018 -0700 Committer: Lee moon soo Committed: Sat Nov 3 09:13:35 2018 -0700 -- ...Matplotlib (Python | PySpark)_2C2AUG798.zpln | 779 --- .../Matplotlib (Python, PySpark)_2C2AUG798.zpln | 779 +++ 2 files changed, 779 insertions(+), 779 deletions(-) --
[1/3] zeppelin git commit: [ZEPPELIN-3839] Replace | to , in the filename
Repository: zeppelin
Updated Branches:
refs/heads/master a2473daf8 -> 383cdc3fa
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/383cdc3f/notebook/Zeppelin
Tutorial/Matplotlib (Python, PySpark)_2C2AUG798.zpln
--
diff --git a/notebook/Zeppelin Tutorial/Matplotlib (Python,
PySpark)_2C2AUG798.zpln b/notebook/Zeppelin Tutorial/Matplotlib (Python,
PySpark)_2C2AUG798.zpln
new file mode 100644
index 000..bbf25bf
--- /dev/null
+++ b/notebook/Zeppelin Tutorial/Matplotlib (Python, PySpark)_2C2AUG798.zpln
@@ -0,0 +1,779 @@
+{
+ "paragraphs": [
+{
+ "text": "%md\n## Introduction\nIn this tutorial we will go through some
of the basic features of Zeppelin\u0027s built-in matplotlib integration.
\n\n### Prerequisites\n`matplotlib` must be installed to your local python
installation. (use `pip install matplotlib` or `conda install matplotlib` if
you have `conda`). Additionally, you will need Zeppelin\u0027s matplotlib
backend files which are usually found in `$ZEPPELIN_HOME/lib/python`. Although
Zeppelin should automatically find this directory, it might be a good idea to
add it to your `PYTHONPATH` just in case. \n\n### Interpreters\nMost of the
examples shown in this tutorial can be used interchangeably with either the
`python` or `pyspark` interpreters. Iterative plotting using the Angular
Display System is currently only available for `pyspark`, but this
functionality will eventually be added to the base `python` interpreter.
\n\n### macOS\nMake sure locale is set, to avoid `ValueError: unknown locale:
UTF-8`\n\n### virtu
alenv\nIn case you want to use virtualenv or conda env:\n - configure python
interpreter property -\u003e `absolute/path/to/venv/bin/python`\n - see
*Working with Matplotlib in Virtual environments* in the [Matplotlib
FAQ](http://matplotlib.org/faq/virtualenv_faq.html)\n \n### A simple
example\nLet\u0027s start by making a very simple line plot:",
+ "user": "anonymous",
+ "dateUpdated": "Dec 17, 2016 3:33:25 PM",
+ "config": {
+"tableHide": false,
+"colWidth": 12.0,
+"editorMode": "ace/mode/text",
+"editorHide": true,
+"enabled": true,
+"results": [
+ {
+"graph": {
+ "mode": "table",
+ "height": 300.0,
+ "optionOpen": false,
+ "keys": [],
+ "values": [],
+ "groups": [],
+ "scatter": {}
+}
+ }
+],
+"editorSetting": {
+ "language": "text",
+ "editOnDblClick": false
+}
+ },
+ "settings": {
+"params": {},
+"forms": {}
+ },
+ "apps": [],
+ "jobName": "paragraph_1478123627954_-1473548609",
+ "id": "20160614-174657_1772993700",
+ "results": {
+"code": "SUCCESS",
+"msg": [
+ {
+"type": "HTML",
+"data": "\u003cdiv
class\u003d\"markdown-body\"\u003e\n\u003ch2\u003eIntroduction\u003c/h2\u003e\n\u003cp\u003eIn
this tutorial we will go through some of the basic features of
Zeppelin\u0026rsquo;s built-in matplotlib integration.
\u003c/p\u003e\n\u003ch3\u003ePrerequisites\u003c/h3\u003e\n\u003cp\u003e\u003ccode\u003ematplotlib\u003c/code\u003e
must be installed to your local python installation. (use \u003ccode\u003epip
install matplotlib\u003c/code\u003e or \u003ccode\u003econda install
matplotlib\u003c/code\u003e if you have
\u003ccode\u003econda\u003c/code\u003e). Additionally, you will need
Zeppelin\u0026rsquo;s matplotlib backend files which are usually found in
\u003ccode\u003e$ZEPPELIN_HOME/interpreter/lib/python\u003c/code\u003e.
Although Zeppelin should automatically find this directory, it might be a good
idea to add it to your \u003ccode\u003ePYTHONPATH\u003c/code\u003e just in
case. \u003c/p\u003e\n\u003ch3\u003eInterpreters\u003c/h3\u003e\n\u003cp\u003eMo
st of the examples shown in this tutorial can be used interchangeably with
either the \u003ccode\u003epython\u003c/code\u003e or
\u003ccode\u003epyspark\u003c/code\u003e interpreters. Iterative plotting using
the Angular Display System is currently only available for
\u003ccode\u003epyspark\u003c/code\u003e, but this functionality will
eventually be added to the base \u003ccode\u003epython\u003c/code\u003e
interpreter.
\u003c/p\u003e\n\u003ch3\u003emacOS\u003c/h3\u003e\n\u003cp\u003eMake sure
locale is set, to avoid \u003ccode\u003eValueError: unknown locale:
UTF-8\u003c/code\u003e\u003c/p\u003e\n\u003ch3\u003evirtualenv\u003c/h3\u003e\n\u003cp\u003eIn
case you want to use virtualenv or conda env:\u003cbr/\u003e - configure
python interpreter property -\u0026gt;
\u003ccode\u003eabsolute/path/to/venv/bin/python\u003c/code\u003e\u003cbr/\u003e
- see \u003cem\u003eWorking with Matplotlib in Virtual
environments\u003c/em\u003e in the \u003ca
[2/3] zeppelin git commit: [ZEPPELIN-3839] Replace | to , in the filename
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/383cdc3f/notebook/Zeppelin
Tutorial/Matplotlib (Python | PySpark)_2C2AUG798.zpln
--
diff --git a/notebook/Zeppelin Tutorial/Matplotlib (Python |
PySpark)_2C2AUG798.zpln b/notebook/Zeppelin Tutorial/Matplotlib (Python |
PySpark)_2C2AUG798.zpln
deleted file mode 100644
index 49953e6..000
--- a/notebook/Zeppelin Tutorial/Matplotlib (Python | PySpark)_2C2AUG798.zpln
+++ /dev/null
@@ -1,779 +0,0 @@
-{
- "paragraphs": [
-{
- "text": "%md\n## Introduction\nIn this tutorial we will go through some
of the basic features of Zeppelin\u0027s built-in matplotlib integration.
\n\n### Prerequisites\n`matplotlib` must be installed to your local python
installation. (use `pip install matplotlib` or `conda install matplotlib` if
you have `conda`). Additionally, you will need Zeppelin\u0027s matplotlib
backend files which are usually found in `$ZEPPELIN_HOME/lib/python`. Although
Zeppelin should automatically find this directory, it might be a good idea to
add it to your `PYTHONPATH` just in case. \n\n### Interpreters\nMost of the
examples shown in this tutorial can be used interchangeably with either the
`python` or `pyspark` interpreters. Iterative plotting using the Angular
Display System is currently only available for `pyspark`, but this
functionality will eventually be added to the base `python` interpreter.
\n\n### macOS\nMake sure locale is set, to avoid `ValueError: unknown locale:
UTF-8`\n\n### virtu
alenv\nIn case you want to use virtualenv or conda env:\n - configure python
interpreter property -\u003e `absolute/path/to/venv/bin/python`\n - see
*Working with Matplotlib in Virtual environments* in the [Matplotlib
FAQ](http://matplotlib.org/faq/virtualenv_faq.html)\n \n### A simple
example\nLet\u0027s start by making a very simple line plot:",
- "user": "anonymous",
- "dateUpdated": "Dec 17, 2016 3:33:25 PM",
- "config": {
-"tableHide": false,
-"colWidth": 12.0,
-"editorMode": "ace/mode/text",
-"editorHide": true,
-"enabled": true,
-"results": [
- {
-"graph": {
- "mode": "table",
- "height": 300.0,
- "optionOpen": false,
- "keys": [],
- "values": [],
- "groups": [],
- "scatter": {}
-}
- }
-],
-"editorSetting": {
- "language": "text",
- "editOnDblClick": false
-}
- },
- "settings": {
-"params": {},
-"forms": {}
- },
- "apps": [],
- "jobName": "paragraph_1478123627954_-1473548609",
- "id": "20160614-174657_1772993700",
- "results": {
-"code": "SUCCESS",
-"msg": [
- {
-"type": "HTML",
-"data": "\u003cdiv
class\u003d\"markdown-body\"\u003e\n\u003ch2\u003eIntroduction\u003c/h2\u003e\n\u003cp\u003eIn
this tutorial we will go through some of the basic features of
Zeppelin\u0026rsquo;s built-in matplotlib integration.
\u003c/p\u003e\n\u003ch3\u003ePrerequisites\u003c/h3\u003e\n\u003cp\u003e\u003ccode\u003ematplotlib\u003c/code\u003e
must be installed to your local python installation. (use \u003ccode\u003epip
install matplotlib\u003c/code\u003e or \u003ccode\u003econda install
matplotlib\u003c/code\u003e if you have
\u003ccode\u003econda\u003c/code\u003e). Additionally, you will need
Zeppelin\u0026rsquo;s matplotlib backend files which are usually found in
\u003ccode\u003e$ZEPPELIN_HOME/interpreter/lib/python\u003c/code\u003e.
Although Zeppelin should automatically find this directory, it might be a good
idea to add it to your \u003ccode\u003ePYTHONPATH\u003c/code\u003e just in
case. \u003c/p\u003e\n\u003ch3\u003eInterpreters\u003c/h3\u003e\n\u003cp\u003eMo
st of the examples shown in this tutorial can be used interchangeably with
either the \u003ccode\u003epython\u003c/code\u003e or
\u003ccode\u003epyspark\u003c/code\u003e interpreters. Iterative plotting using
the Angular Display System is currently only available for
\u003ccode\u003epyspark\u003c/code\u003e, but this functionality will
eventually be added to the base \u003ccode\u003epython\u003c/code\u003e
interpreter.
\u003c/p\u003e\n\u003ch3\u003emacOS\u003c/h3\u003e\n\u003cp\u003eMake sure
locale is set, to avoid \u003ccode\u003eValueError: unknown locale:
UTF-8\u003c/code\u003e\u003c/p\u003e\n\u003ch3\u003evirtualenv\u003c/h3\u003e\n\u003cp\u003eIn
case you want to use virtualenv or conda env:\u003cbr/\u003e - configure
python interpreter property -\u0026gt;
\u003ccode\u003eabsolute/path/to/venv/bin/python\u003c/code\u003e\u003cbr/\u003e
- see \u003cem\u003eWorking with Matplotlib in Virtual
environments\u003c/em\u003e in the \u003ca
href\u003d\"http://matplotlib.org/faq/virtual
env_faq.html\"\u003eMatplotlib
svn commit: r1845292 - /zeppelin/site/index.html
Author: moon Date: Tue Oct 30 22:40:56 2018 New Revision: 1845292 URL: http://svn.apache.org/viewvc?rev=1845292=rev Log: [ZEPPELIN-3835] Update website for 0.8.0 release Modified: zeppelin/site/index.html Modified: zeppelin/site/index.html URL: http://svn.apache.org/viewvc/zeppelin/site/index.html?rev=1845292=1845291=1845292=diff == --- zeppelin/site/index.html (original) +++ zeppelin/site/index.html Tue Oct 30 22:40:56 2018 @@ -270,88 +270,56 @@ limitations under the License. What's new in -Apache Zeppelin 0.7 +Apache Zeppelin 0.8 -Pluggable Visualization via Helium - +Python, improved + -Load/unload Javascript -https://www.npmjs.com/; target="_blank">npm packages -like Zeppelin built-in chart using Helium framework. - - See more in DEMO - -and -Zeppelin Visualization: How it works? +IPython interpreter provides comparable user experience like Jupyter Notebook. For the details, click here. -Multi-user Support Improvement - +Note improvements - Separate interpreter running scope - Per user or - Per Note. - -See more in DEMO. - - - Also running Zeppelin interpreter process as web front end user is available now. - -See more in DEMO - - - and - Interpreter User Impersonation. + This release includes Note level dynamic form, note revision comparator and ability to run paragraph sequentially, instead of simultaneous paragraph execution in previous releases. - -New Note Mode - Personal Mode +Inline configuration - Personalize your analysis result by switching the note to Personal Mode. - (Collaboration Mode is default.) - -See more in DEMO. - - + Generic ConfInterpreter provide a way configure interpreter inside each note. -Support Spark 2.1 +Tab-key completion - The latest version of http://spark.apache.org/releases/spark-release-2-1-0.html; target="_blank">Apache Spark 2.1.0 is now available in Zeppelin. + Press `Tab` for code completion. (previous key combination `Ctrl+.` works as well) -Improvement in Python +Interpreter lifecycle + - Integrated - Matplotlib - with Python & Pyspark interpreter. And - Conda - is now available in Zeppelin. + Interpreter lifecycle manager automatically terminate interpreter process on idle timeout. So resources are released when they're not in use. See here for more details. + -New Interpreters +Helium online registry -You can use -https://beam.apache.org/; target="_blank">Apache Beam, -https://github.com/spotify/scio; target="_blank">Scio, and -https://pig.apache.org/; target="_blank">Apache Pig as backend interpreters from this release. + This release includes online registry for Helium that can add custom visualization and more. learn more about it. See more details in -0.7 Release Note +0.8 Release Note
svn commit: r1845290 - /zeppelin/site/whats_new.md
Author: moon Date: Tue Oct 30 22:35:42 2018 New Revision: 1845290 URL: http://svn.apache.org/viewvc?rev=1845290=rev Log: [ZEPPELIN-3835] Update what's new section for release 0.8 Modified: zeppelin/site/whats_new.md Modified: zeppelin/site/whats_new.md URL: http://svn.apache.org/viewvc/zeppelin/site/whats_new.md?rev=1845290=1845289=1845290=diff == --- zeppelin/site/whats_new.md (original) +++ zeppelin/site/whats_new.md Tue Oct 30 22:35:42 2018 @@ -16,88 +16,56 @@ limitations under the License. What's new in -Apache Zeppelin 0.7 +Apache Zeppelin 0.8 -Pluggable Visualization via Helium - +Python, improved + -Load/unload Javascript -https://www.npmjs.com/; target="_blank">npm packages -like Zeppelin built-in chart using Helium framework. - - See more in DEMO - -and -Zeppelin Visualization: How it works? +IPython interpreter provides comparable user experience like Jupyter Notebook. For the details, click here. -Multi-user Support Improvement - +Note improvements - Separate interpreter running scope - Per user or - Per Note. - -See more in DEMO. - - - Also running Zeppelin interpreter process as web front end user is available now. - -See more in DEMO - - - and - Interpreter User Impersonation. + This release includes Note level dynamic form, note revision comparator and ability to run paragraph sequentially, instead of simultaneous paragraph execution in previous releases. - -New Note Mode - Personal Mode +Inline configuration - Personalize your analysis result by switching the note to Personal Mode. - (Collaboration Mode is default.) - -See more in DEMO. - - + Generic ConfInterpreter provide a way configure interpreter inside each note. -Support Spark 2.1 +Tab-key completion - The latest version of http://spark.apache.org/releases/spark-release-2-1-0.html; target="_blank">Apache Spark 2.1.0 is now available in Zeppelin. + Press `Tab` for code completion. (previous key combination `Ctrl+.` works as well) -Improvement in Python +Interpreter lifecycle + - Integrated - Matplotlib - with Python & Pyspark interpreter. And - Conda - is now available in Zeppelin. + Interpreter lifecycle manager automatically terminate interpreter process on idle timeout. So resources are released when they're not in use. See here for more details. + -New Interpreters +Helium online registry -You can use -https://beam.apache.org/; target="_blank">Apache Beam, -https://github.com/spotify/scio; target="_blank">Scio, and -https://pig.apache.org/; target="_blank">Apache Pig as backend interpreters from this release. + This release includes online registry for Helium that can add custom visualization and more. learn more about it. See more details in -0.7 Release Note +0.8 Release Note
zeppelin git commit: [ZEPPELIN-3835] Update website for 0.8.0 release
Repository: zeppelin Updated Branches: refs/heads/gh-pages 27e6d004a -> e44ef7bed [ZEPPELIN-3835] Update website for 0.8.0 release ### What is this PR for? 0.8.0 has released in last June, but what's new section in website still have information of 0.7.x. This PR update information for the latest release. I compiled contents based on https://medium.com/zjffdu/zeppelin-0-8-0-new-features-ea53e8810235 ### What type of PR is it? Website update ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-3835 ### How should this be tested? * [Run website locally](https://github.com/apache/zeppelin/tree/gh-pages#run-website) ### Screenshots (if appropriate) Before  After  ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Lee moon soo Closes #3210 from Leemoonsoo/gh-pages-0.8-release and squashes the following commits: 960f9586e [Lee moon soo] Update website for 0.8.0 release Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/e44ef7be Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/e44ef7be Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/e44ef7be Branch: refs/heads/gh-pages Commit: e44ef7bedecc03d74a54f3b928d35c60ffe6511c Parents: 27e6d00 Author: Lee moon soo Authored: Mon Oct 29 13:49:23 2018 -0700 Committer: Lee moon soo Committed: Tue Oct 30 15:31:45 2018 -0700 -- whats_new.md | 66 ++- 1 file changed, 17 insertions(+), 49 deletions(-) -- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/e44ef7be/whats_new.md -- diff --git a/whats_new.md b/whats_new.md index cd53e4a..b2bbabf 100644 --- a/whats_new.md +++ b/whats_new.md @@ -16,88 +16,56 @@ limitations under the License. What's new in -Apache Zeppelin 0.7 +Apache Zeppelin 0.8 -Pluggable Visualization via Helium - +Python, improved + -Load/unload Javascript -https://www.npmjs.com/; target="_blank">npm packages -like Zeppelin built-in chart using Helium framework. - - See more in DEMO - -and -Zeppelin Visualization: How it works? +IPython interpreter provides comparable user experience like Jupyter Notebook. For the details, click here. -Multi-user Support Improvement - +Note improvements - Separate interpreter running scope - Per user or - Per Note. - -See more in DEMO. - - - Also running Zeppelin interpreter process as web front end user is available now. - -See more in DEMO - - - and - Interpreter User Impersonation. + This release includes Note level dynamic form, note revision comparator and ability to run paragraph sequentially, instead of simultaneous paragraph execution in previous releases. - -New Note Mode - Personal Mode +Inline configuration - Personalize your analysis result by switching the note to Personal Mode. - (Collaboration Mode is default.) - -See more in DEMO. - - + Generic ConfInterpreter provide a way configure interpreter inside each note. -Support Spark 2.1 +Tab-key completion - The latest version of http://spark.apache.org/releases/spark-release-2-1-0.html; target="_blank">Apache Spark 2.1.0 is now available in Zeppelin. + Press `Tab` for code completion. (previous key combination `Ctrl+.` works as well) -Improvement in Python +Interpreter lifecycle + - Integrated - Matplotlib - with Python & Pyspark interpreter. And - Conda - is now available in Zeppelin. + Interpreter lifecycle manager automatically terminate interpreter process on idle timeout. So resources are released when they're not in use. See here for more details. + -New Interpreters +Helium online registry -
zeppelin git commit: ZEPPELIN-3581. Add "type": "textarea" to "default.statementPrecode" in interpreter-setting.json
Repository: zeppelin
Updated Branches:
refs/heads/master 0ca247b1b -> d2e789714
ZEPPELIN-3581. Add "type": "textarea" to "default.statementPrecode" in
interpreter-setting.json
### What is this PR for?
Fix "interpreter-setting.json" for JDBC interpreter
### What type of PR is it?
Bug Fix
### What is the Jira issue?
[ZEPPELIN-3581](https://issues.apache.org/jira/browse/ZEPPELIN-3581)
### How should this be tested?
* Check text area near the "default.statementPrecode" setting.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Maxim Belousov
Closes #3050 from mebelousov/ZEPPELIN-3581 and squashes the following commits:
4b052ed6b [Maxim Belousov] Add "type": "textarea" to "default.statementPrecode"
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/d2e78971
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/d2e78971
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/d2e78971
Branch: refs/heads/master
Commit: d2e78971441d0f7d78c2fb9a833f84b1891171c7
Parents: 0ca247b
Author: Maxim Belousov
Authored: Tue Jul 3 20:20:22 2018 +0300
Committer: Lee moon soo
Committed: Wed Jul 4 18:12:48 2018 +0900
--
jdbc/src/main/resources/interpreter-setting.json | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/d2e78971/jdbc/src/main/resources/interpreter-setting.json
--
diff --git a/jdbc/src/main/resources/interpreter-setting.json
b/jdbc/src/main/resources/interpreter-setting.json
index 04262d9..3f3e672 100644
--- a/jdbc/src/main/resources/interpreter-setting.json
+++ b/jdbc/src/main/resources/interpreter-setting.json
@@ -57,7 +57,8 @@
"envName": null,
"propertyName": "default.statementPrecode",
"defaultValue": "",
-"description": "Runs before each run of the paragraph, in the same
connection"
+"description": "Runs before each run of the paragraph, in the same
connection",
+"type": "textarea"
},
"default.splitQueries": {
"envName": null,
svn commit: r1835044 - /zeppelin/site/index.html
Author: moon Date: Wed Jul 4 05:46:08 2018 New Revision: 1835044 URL: http://svn.apache.org/viewvc?rev=1835044=rev Log: https://github.com/apache/zeppelin/pull/3051 Modified: zeppelin/site/index.html Modified: zeppelin/site/index.html URL: http://svn.apache.org/viewvc/zeppelin/site/index.html?rev=1835044=1835043=1835044=diff == --- zeppelin/site/index.html (original) +++ zeppelin/site/index.html Wed Jul 4 05:46:08 2018 @@ -141,7 +141,7 @@ - GET STARTED
zeppelin git commit: Fix a broken link in the landing page
Repository: zeppelin Updated Branches: refs/heads/gh-pages c11983ac7 -> 27e6d004a Fix a broken link in the landing page ### What is this PR for? Fix a broken link in the landing page ### What type of PR is it? Documentation ### What is the Jira issue? * Open an issue on Jira https://issues.apache.org/jira/browse/ZEPPELIN/ * Put link here, and add [ZEPPELIN-*Jira number*] in PR title, eg. [ZEPPELIN-533] ### How should this be tested? * First time? Setup Travis CI as described on https://zeppelin.apache.org/contribution/contributions.html#continuous-integration * Strongly recommended: add automated unit tests for any new or changed behavior * Outline any manual steps to test the PR here. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jaewon Seo Closes #3051 from Byeol/fix-broken-link and squashes the following commits: b1f33cc10 [Jaewon Seo] Fix a broken link in the landing page Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/27e6d004 Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/27e6d004 Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/27e6d004 Branch: refs/heads/gh-pages Commit: 27e6d004ade0afe097a1b86e72cf2415c672c283 Parents: c11983a Author: Jaewon Seo Authored: Wed Jul 4 14:30:00 2018 +0900 Committer: Lee moon soo Committed: Wed Jul 4 14:42:14 2018 +0900 -- _includes/themes/zeppelin/_navigation.html | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/27e6d004/_includes/themes/zeppelin/_navigation.html -- diff --git a/_includes/themes/zeppelin/_navigation.html b/_includes/themes/zeppelin/_navigation.html index 87f18e5..2950a6e 100644 --- a/_includes/themes/zeppelin/_navigation.html +++ b/_includes/themes/zeppelin/_navigation.html @@ -14,7 +14,7 @@ -Quick Start +Quick Start Download @@ -82,7 +82,7 @@ - GET STARTED
zeppelin git commit: [ZEPPELIN-3554] Flaky test - InterpreterContextTest.testThreadLocal
Repository: zeppelin
Updated Branches:
refs/heads/master ebca7aed7 -> a7e43d600
[ZEPPELIN-3554] Flaky test - InterpreterContextTest.testThreadLocal
### What is this PR for?
Fix flaky test. See
[ZEPL-3554](https://issues.apache.org/jira/browse/ZEPPELIN-3554)
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3554
### How should this be tested?
* First time? Setup Travis CI as described on
https://zeppelin.apache.org/contribution/contributions.html#continuous-integration
* Strongly recommended: add automated unit tests for any new or changed behavior
* Outline any manual steps to test the PR here.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo
Closes #3036 from Leemoonsoo/ZEPPELIN-3554 and squashes the following commits:
87fa07b2e [Lee moon soo] Clear InterpreterContext before test start
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/a7e43d60
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/a7e43d60
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/a7e43d60
Branch: refs/heads/master
Commit: a7e43d600f86811ec695ad0606845d7a114f6fba
Parents: ebca7ae
Author: Lee moon soo
Authored: Wed Jun 20 10:49:37 2018 +0900
Committer: Lee moon soo
Committed: Thu Jun 21 10:15:36 2018 +0900
--
.../org/apache/zeppelin/interpreter/InterpreterContextTest.java | 1 +
1 file changed, 1 insertion(+)
--
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a7e43d60/zeppelin-interpreter/src/test/java/org/apache/zeppelin/interpreter/InterpreterContextTest.java
--
diff --git
a/zeppelin-interpreter/src/test/java/org/apache/zeppelin/interpreter/InterpreterContextTest.java
b/zeppelin-interpreter/src/test/java/org/apache/zeppelin/interpreter/InterpreterContextTest.java
index 4888f5e..62b4035 100644
---
a/zeppelin-interpreter/src/test/java/org/apache/zeppelin/interpreter/InterpreterContextTest.java
+++
b/zeppelin-interpreter/src/test/java/org/apache/zeppelin/interpreter/InterpreterContextTest.java
@@ -26,6 +26,7 @@ public class InterpreterContextTest {
@Test
public void testThreadLocal() {
+InterpreterContext.remove();
assertNull(InterpreterContext.get());
InterpreterContext.set(InterpreterContext.builder()
zeppelin git commit: [Zeppelin-3224] - Fix "ultimate" visualization
Repository: zeppelin
Updated Branches:
refs/heads/branch-0.8 b05f1fd10 -> c4e37d619
[Zeppelin-3224] - Fix "ultimate" visualization
### What is this PR for?
This PR fixes problems with incorrect data visualization using "ultimate". With
certain aggregations, the data was not displayed. Also PR improves sorting by
columns / rows, before PR it compared by Unicode encoding, but now if there are
only numeric data on the axis, sorting will be by values.
### What type of PR is it?
[Bug Fix & Improvement]
### What is the Jira issue?
[ZEPPELIN-3224](https://issues.apache.org/jira/browse/ZEPPELIN-3224)
### Screenshots (if appropriate)
**Error:**
**Now:**
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Savalek <def...@mail.ru>
Author: tinkoff-dwh <tinkoff@gmail.com>
Closes #2841 from Savalek/ZEPPELIN-3224 and squashes the following commits:
f546b457c [Savalek] fix count of arguments in parseFloat function
a56ea59d9 [Savalek] Improved number check
5c866b183 [tinkoff-dwh] small fix
e7d1cd7cc [tinkoff-dwh] Merge branch 'master' into ZEPPELIN-3224
d4b2a4913 [Savalek] add test
17d3f8d0e [Savalek] add tests for sortWithNumberSupport()
dc78f3d2b [Savalek] code style fix
ba7a6e3e4 [tinkoff-dwh] Merge branch 'master' into ZEPPELIN-3224
9583f2cce [Savalek] small fix
eab5d86d2 [Savalek] Merge branch 'master' into ZEPPELIN-3224
33676b468 [Savalek] fix XY legend sort for numbers
94026be39 [Savalek] fix graph display
(cherry picked from commit c1c1aa892dbd7621815c19a9d72b175a330ee51e)
Signed-off-by: Lee moon soo <m...@apache.org>
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/c4e37d61
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/c4e37d61
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/c4e37d61
Branch: refs/heads/branch-0.8
Commit: c4e37d6191288265995c507a37d5dc472ad311b3
Parents: b05f1fd
Author: Savalek <def...@mail.ru>
Authored: Fri Apr 6 11:58:50 2018 +0300
Committer: Lee moon soo <m...@apache.org>
Committed: Wed May 2 15:33:17 2018 -0400
--
.../tabledata/advanced-transformation-util.js | 38 +---
.../advanced-transformation-util.test.js| 37 +++
zeppelin-web/src/app/tabledata/tabledata.js | 4 +++
3 files changed, 67 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/c4e37d61/zeppelin-web/src/app/tabledata/advanced-transformation-util.js
--
diff --git a/zeppelin-web/src/app/tabledata/advanced-transformation-util.js
b/zeppelin-web/src/app/tabledata/advanced-transformation-util.js
index 97c1b2c..71ed7b8 100644
--- a/zeppelin-web/src/app/tabledata/advanced-transformation-util.js
+++ b/zeppelin-web/src/app/tabledata/advanced-transformation-util.js
@@ -771,7 +771,7 @@ export function getKGACube(rows, keyColumns, groupColumns,
aggrColumns) {
cube = {[mergedGroupColumnName]: cube};
keyNames = [mergedGroupColumnName];
} else {
-keyNames = Object.keys(cube).sort(); /** keys should be sorted */
+keyNames = sortWithNumberSupport(Object.keys(cube)); /** keys should be
sorted */
}
return {
@@ -883,7 +883,7 @@ export function getKAGCube(rows, keyColumns, groupColumns,
aggrColumns) {
cube = {[mergedGroupColumnName]: cube};
keyNames = [mergedGroupColumnName];
} else {
-keyNames = Object.keys(cube).sort(); /** keys should be sorted */
+keyNames = sortWithNumberSupport(Object.keys(cube)); /** keys should be
sorted */
}
return {
@@ -1015,8 +1015,8 @@ export function getKKGACube(rows, key1Columns,
key2Columns, groupColumns, aggrCo
} /** end loop for aggrColumns */
}
- let key1Names = Object.keys(key1NameSet).sort(); /** keys should be sorted */
- let key2Names = Object.keys(key2NameSet).sort(); /** keys should be sorted */
+ let key1Names = sortWithNumberSupport(Object.keys(key1NameSet)); /** keys
should be sorted */
+ let key2Names = sortWithNumberSupport(Object.keys(key2NameSet)); /** keys
should be sorted */
return {
cube: cube,
@@ -1072,7 +1072,7 @@ export function getNameWithIndex(names) {
export function getArrayRowsFromKKGACube(cube, schema, aggregatorColumns,
key1Names, key2Names, groupNameSet,
selectorNameWithIndex) {
- const sortedSelectors = Object.keys(selectorNameWithIndex).sort();
+ const sortedSelectors =
zeppelin git commit: [Zeppelin-3224] - Fix "ultimate" visualization
Repository: zeppelin
Updated Branches:
refs/heads/master 0c3260e91 -> c1c1aa892
[Zeppelin-3224] - Fix "ultimate" visualization
### What is this PR for?
This PR fixes problems with incorrect data visualization using "ultimate". With
certain aggregations, the data was not displayed. Also PR improves sorting by
columns / rows, before PR it compared by Unicode encoding, but now if there are
only numeric data on the axis, sorting will be by values.
### What type of PR is it?
[Bug Fix & Improvement]
### What is the Jira issue?
[ZEPPELIN-3224](https://issues.apache.org/jira/browse/ZEPPELIN-3224)
### Screenshots (if appropriate)
**Error:**
**Now:**
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Savalek <def...@mail.ru>
Author: tinkoff-dwh <tinkoff@gmail.com>
Closes #2841 from Savalek/ZEPPELIN-3224 and squashes the following commits:
f546b457c [Savalek] fix count of arguments in parseFloat function
a56ea59d9 [Savalek] Improved number check
5c866b183 [tinkoff-dwh] small fix
e7d1cd7cc [tinkoff-dwh] Merge branch 'master' into ZEPPELIN-3224
d4b2a4913 [Savalek] add test
17d3f8d0e [Savalek] add tests for sortWithNumberSupport()
dc78f3d2b [Savalek] code style fix
ba7a6e3e4 [tinkoff-dwh] Merge branch 'master' into ZEPPELIN-3224
9583f2cce [Savalek] small fix
eab5d86d2 [Savalek] Merge branch 'master' into ZEPPELIN-3224
33676b468 [Savalek] fix XY legend sort for numbers
94026be39 [Savalek] fix graph display
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/c1c1aa89
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/c1c1aa89
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/c1c1aa89
Branch: refs/heads/master
Commit: c1c1aa892dbd7621815c19a9d72b175a330ee51e
Parents: 0c3260e
Author: Savalek <def...@mail.ru>
Authored: Fri Apr 6 11:58:50 2018 +0300
Committer: Lee moon soo <m...@apache.org>
Committed: Wed May 2 15:32:53 2018 -0400
--
.../tabledata/advanced-transformation-util.js | 38 +---
.../advanced-transformation-util.test.js| 37 +++
zeppelin-web/src/app/tabledata/tabledata.js | 4 +++
3 files changed, 67 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/c1c1aa89/zeppelin-web/src/app/tabledata/advanced-transformation-util.js
--
diff --git a/zeppelin-web/src/app/tabledata/advanced-transformation-util.js
b/zeppelin-web/src/app/tabledata/advanced-transformation-util.js
index 97c1b2c..71ed7b8 100644
--- a/zeppelin-web/src/app/tabledata/advanced-transformation-util.js
+++ b/zeppelin-web/src/app/tabledata/advanced-transformation-util.js
@@ -771,7 +771,7 @@ export function getKGACube(rows, keyColumns, groupColumns,
aggrColumns) {
cube = {[mergedGroupColumnName]: cube};
keyNames = [mergedGroupColumnName];
} else {
-keyNames = Object.keys(cube).sort(); /** keys should be sorted */
+keyNames = sortWithNumberSupport(Object.keys(cube)); /** keys should be
sorted */
}
return {
@@ -883,7 +883,7 @@ export function getKAGCube(rows, keyColumns, groupColumns,
aggrColumns) {
cube = {[mergedGroupColumnName]: cube};
keyNames = [mergedGroupColumnName];
} else {
-keyNames = Object.keys(cube).sort(); /** keys should be sorted */
+keyNames = sortWithNumberSupport(Object.keys(cube)); /** keys should be
sorted */
}
return {
@@ -1015,8 +1015,8 @@ export function getKKGACube(rows, key1Columns,
key2Columns, groupColumns, aggrCo
} /** end loop for aggrColumns */
}
- let key1Names = Object.keys(key1NameSet).sort(); /** keys should be sorted */
- let key2Names = Object.keys(key2NameSet).sort(); /** keys should be sorted */
+ let key1Names = sortWithNumberSupport(Object.keys(key1NameSet)); /** keys
should be sorted */
+ let key2Names = sortWithNumberSupport(Object.keys(key2NameSet)); /** keys
should be sorted */
return {
cube: cube,
@@ -1072,7 +1072,7 @@ export function getNameWithIndex(names) {
export function getArrayRowsFromKKGACube(cube, schema, aggregatorColumns,
key1Names, key2Names, groupNameSet,
selectorNameWithIndex) {
- const sortedSelectors = Object.keys(selectorNameWithIndex).sort();
+ const sortedSelectors =
sortWithNumberSupport(Object.keys(selectorNameWithIndex));
const sortedSelectorNameWithIndex = getNameWithIndex(so
zeppelin git commit: ZEPPELIN-3332 Zeppelin login fails with NPE if ldapRealm.authorizationEnabled is not set true
Repository: zeppelin
Updated Branches:
refs/heads/branch-0.8 380a92768 -> d3de51562
ZEPPELIN-3332 Zeppelin login fails with NPE if ldapRealm.authorizationEnabled
is not set true
### What is this PR for?
Simply fixes NPE by checking for null
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3332
### Questions:
Does the licenses files need update? No
Is there breaking changes for older versions? No
Does this needs documentation? No
Author: bag_of_tricks <falb...@hortonworks.com>
Closes #2911 from felixalbani/ZEPPELIN-3332 and squashes the following commits:
c6ccf4189 [bag_of_tricks] ZEPPELIN-3332
622333eb9 [bag_of_tricks] Revert "ZEPPELIN-3332"
dd7d37dac [bag_of_tricks] ZEPPELIN-3332
(cherry picked from commit 685eb9249d1c1d821ce57f1ed0559f1539dfbe69)
Signed-off-by: Lee moon soo <m...@apache.org>
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/d3de5156
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/d3de5156
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/d3de5156
Branch: refs/heads/branch-0.8
Commit: d3de5156234cb88caa39931fd4086b29df0b9d39
Parents: 380a927
Author: bag_of_tricks <falb...@hortonworks.com>
Authored: Thu Apr 5 12:27:18 2018 -0400
Committer: Lee moon soo <m...@apache.org>
Committed: Thu Apr 5 12:17:10 2018 -0700
--
.../src/main/java/org/apache/zeppelin/utils/SecurityUtils.java | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/d3de5156/zeppelin-server/src/main/java/org/apache/zeppelin/utils/SecurityUtils.java
--
diff --git
a/zeppelin-server/src/main/java/org/apache/zeppelin/utils/SecurityUtils.java
b/zeppelin-server/src/main/java/org/apache/zeppelin/utils/SecurityUtils.java
index e24af20..50f167b 100644
--- a/zeppelin-server/src/main/java/org/apache/zeppelin/utils/SecurityUtils.java
+++ b/zeppelin-server/src/main/java/org/apache/zeppelin/utils/SecurityUtils.java
@@ -55,7 +55,7 @@ public class SecurityUtils {
private static final HashSet EMPTY_HASHSET = Sets.newHashSet();
private static boolean isEnabled = false;
private static final Logger log =
LoggerFactory.getLogger(SecurityUtils.class);
-
+
public static void setIsEnabled(boolean value) {
isEnabled = value;
}
@@ -149,7 +149,9 @@ public class SecurityUtils {
new SimplePrincipalCollection(subject.getPrincipal(),
realm.getName()),
((LdapRealm) realm).getContextFactory()
);
-roles = new HashSet<>(auth.getRoles());
+if (auth != null) {
+ roles = new HashSet<>(auth.getRoles());
+}
} catch (NamingException e) {
log.error("Can't fetch roles", e);
}
zeppelin git commit: ZEPPELIN-3332 Zeppelin login fails with NPE if ldapRealm.authorizationEnabled is not set true
Repository: zeppelin
Updated Branches:
refs/heads/master 7a9572d67 -> 685eb9249
ZEPPELIN-3332 Zeppelin login fails with NPE if ldapRealm.authorizationEnabled
is not set true
### What is this PR for?
Simply fixes NPE by checking for null
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3332
### Questions:
Does the licenses files need update? No
Is there breaking changes for older versions? No
Does this needs documentation? No
Author: bag_of_tricks <falb...@hortonworks.com>
Closes #2911 from felixalbani/ZEPPELIN-3332 and squashes the following commits:
c6ccf4189 [bag_of_tricks] ZEPPELIN-3332
622333eb9 [bag_of_tricks] Revert "ZEPPELIN-3332"
dd7d37dac [bag_of_tricks] ZEPPELIN-3332
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/685eb924
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/685eb924
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/685eb924
Branch: refs/heads/master
Commit: 685eb9249d1c1d821ce57f1ed0559f1539dfbe69
Parents: 7a9572d
Author: bag_of_tricks <falb...@hortonworks.com>
Authored: Thu Apr 5 12:27:18 2018 -0400
Committer: Lee moon soo <m...@apache.org>
Committed: Thu Apr 5 12:16:42 2018 -0700
--
.../src/main/java/org/apache/zeppelin/utils/SecurityUtils.java | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/685eb924/zeppelin-server/src/main/java/org/apache/zeppelin/utils/SecurityUtils.java
--
diff --git
a/zeppelin-server/src/main/java/org/apache/zeppelin/utils/SecurityUtils.java
b/zeppelin-server/src/main/java/org/apache/zeppelin/utils/SecurityUtils.java
index 7762384..f9f5f22 100644
--- a/zeppelin-server/src/main/java/org/apache/zeppelin/utils/SecurityUtils.java
+++ b/zeppelin-server/src/main/java/org/apache/zeppelin/utils/SecurityUtils.java
@@ -53,7 +53,7 @@ public class SecurityUtils {
private static final HashSet EMPTY_HASHSET = Sets.newHashSet();
private static boolean isEnabled = false;
private static final Logger log =
LoggerFactory.getLogger(SecurityUtils.class);
-
+
public static void setIsEnabled(boolean value) {
isEnabled = value;
}
@@ -147,7 +147,9 @@ public class SecurityUtils {
new SimplePrincipalCollection(subject.getPrincipal(),
realm.getName()),
((LdapRealm) realm).getContextFactory()
);
-roles = new HashSet<>(auth.getRoles());
+if (auth != null) {
+ roles = new HashSet<>(auth.getRoles());
+}
} catch (NamingException e) {
log.error("Can't fetch roles", e);
}
zeppelin git commit: [ZEPPELIN-3284] z.getInterpreterContext().out().clear() broken in Python interpreter
Repository: zeppelin
Updated Branches:
refs/heads/branch-0.8 23907af25 -> f1c2b5b45
[ZEPPELIN-3284] z.getInterpreterContext().out().clear() broken in Python
interpreter
### What is this PR for?
```
%python
import time
print("Hello")
time.sleep(0.5) # in case of Ipython kernel, print may not immediately
flushed and cleared.
z.getInterpreterContext().out().clear()
print("world")
```
Expected to print `world` only.
This worked in zeppelin-0.7.x and it'll be nice keep this feature in the future
version.
### What type of PR is it?
Bug fix | Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3284
### How should this be tested?
* First time? Setup Travis CI as described on
https://zeppelin.apache.org/contribution/contributions.html#continuous-integration
* Strongly recommended: add automated unit tests for any new or changed behavior
* Outline any manual steps to test the PR here.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update?
* Is there breaking changes for older versions?
* Does this needs documentation?
Author: Lee moon soo <m...@apache.org>
Closes #2829 from Leemoonsoo/ZEPPELIN-3284 and squashes the following commits:
e0dcda61f [Lee moon soo] make z.getInterpreterContext().out().clear() work in
IPythonInterprete
224a2df8e [Lee moon soo] make z.getInterpreterContext().out().clear() work in
PythonInterpreter
(cherry picked from commit d90716d4d00f5b83d2bd1f16cadc88118a112f74)
Signed-off-by: Lee moon soo <m...@apache.org>
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/f1c2b5b4
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/f1c2b5b4
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/f1c2b5b4
Branch: refs/heads/branch-0.8
Commit: f1c2b5b45e5ecc31d51fbae29eef8dee999362f9
Parents: 23907af
Author: Lee moon soo <m...@apache.org>
Authored: Fri Mar 2 17:42:41 2018 -0800
Committer: Lee moon soo <m...@apache.org>
Committed: Sun Mar 4 10:26:26 2018 -0800
--
.../java/org/apache/zeppelin/python/PythonInterpreter.java | 1 +
python/src/main/resources/grpc/python/zeppelin_python.py | 3 +++
python/src/main/resources/python/zeppelin_python.py| 2 +-
.../org/apache/zeppelin/python/IPythonInterpreterTest.java | 5 +
.../java/org/apache/zeppelin/python/PythonInterpreterTest.java | 6 ++
5 files changed, 16 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f1c2b5b4/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java
--
diff --git
a/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java
b/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java
index 028f1c6..1864409 100644
--- a/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java
+++ b/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java
@@ -388,6 +388,7 @@ public class PythonInterpreter extends Interpreter
implements ExecuteResultHandl
zeppelinContext.setGui(context.getGui());
zeppelinContext.setNoteGui(context.getNoteGui());
+zeppelinContext.setInterpreterContext(context);
if (!pythonscriptRunning) {
return new InterpreterResult(Code.ERROR, "python process not running"
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f1c2b5b4/python/src/main/resources/grpc/python/zeppelin_python.py
--
diff --git a/python/src/main/resources/grpc/python/zeppelin_python.py
b/python/src/main/resources/grpc/python/zeppelin_python.py
index 1a9e8af..6d7c33e 100644
--- a/python/src/main/resources/grpc/python/zeppelin_python.py
+++ b/python/src/main/resources/grpc/python/zeppelin_python.py
@@ -33,6 +33,9 @@ class PyZeppelinContext(object):
self.javaList = gateway.jvm.java.util.ArrayList
self.max_result = z.getMaxResult()
+ def getInterpreterContext(self):
+return self.z.getInterpreterContext()
+
def input(self, name, defaultValue=""):
return self.z.input(name, defaultValue)
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/f1c2b5b4/python/src/main/resources/python/zeppelin_python.py
--
diff --git a/python/src/main/resources/python/zeppelin_python.py
b/python/src/main/resources/python/zeppelin_python.py
index 519d1af..f4ea2c7 100644
--- a/python/src/main/resources/python/zeppelin_python.py
+++ b/python/src/main/resources/python/zeppelin_python.py
@@ -61,7 +61,7 @@ class PyZeppelinContext(object):
self._setup_matplotlib()
def getInterpreterContext(s
zeppelin git commit: [ZEPPELIN-3284] z.getInterpreterContext().out().clear() broken in Python interpreter
Repository: zeppelin
Updated Branches:
refs/heads/master 66dada66e -> d90716d4d
[ZEPPELIN-3284] z.getInterpreterContext().out().clear() broken in Python
interpreter
### What is this PR for?
```
%python
import time
print("Hello")
time.sleep(0.5) # in case of Ipython kernel, print may not immediately
flushed and cleared.
z.getInterpreterContext().out().clear()
print("world")
```
Expected to print `world` only.
This worked in zeppelin-0.7.x and it'll be nice keep this feature in the future
version.
### What type of PR is it?
Bug fix | Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3284
### How should this be tested?
* First time? Setup Travis CI as described on
https://zeppelin.apache.org/contribution/contributions.html#continuous-integration
* Strongly recommended: add automated unit tests for any new or changed behavior
* Outline any manual steps to test the PR here.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update?
* Is there breaking changes for older versions?
* Does this needs documentation?
Author: Lee moon soo <m...@apache.org>
Closes #2829 from Leemoonsoo/ZEPPELIN-3284 and squashes the following commits:
e0dcda61f [Lee moon soo] make z.getInterpreterContext().out().clear() work in
IPythonInterprete
224a2df8e [Lee moon soo] make z.getInterpreterContext().out().clear() work in
PythonInterpreter
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/d90716d4
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/d90716d4
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/d90716d4
Branch: refs/heads/master
Commit: d90716d4d00f5b83d2bd1f16cadc88118a112f74
Parents: 66dada6
Author: Lee moon soo <m...@apache.org>
Authored: Fri Mar 2 17:42:41 2018 -0800
Committer: Lee moon soo <m...@apache.org>
Committed: Sun Mar 4 10:26:18 2018 -0800
--
.../java/org/apache/zeppelin/python/PythonInterpreter.java | 1 +
python/src/main/resources/grpc/python/zeppelin_python.py | 3 +++
python/src/main/resources/python/zeppelin_python.py| 2 +-
.../org/apache/zeppelin/python/IPythonInterpreterTest.java | 5 +
.../java/org/apache/zeppelin/python/PythonInterpreterTest.java | 6 ++
5 files changed, 16 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/d90716d4/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java
--
diff --git
a/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java
b/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java
index 028f1c6..1864409 100644
--- a/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java
+++ b/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java
@@ -388,6 +388,7 @@ public class PythonInterpreter extends Interpreter
implements ExecuteResultHandl
zeppelinContext.setGui(context.getGui());
zeppelinContext.setNoteGui(context.getNoteGui());
+zeppelinContext.setInterpreterContext(context);
if (!pythonscriptRunning) {
return new InterpreterResult(Code.ERROR, "python process not running"
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/d90716d4/python/src/main/resources/grpc/python/zeppelin_python.py
--
diff --git a/python/src/main/resources/grpc/python/zeppelin_python.py
b/python/src/main/resources/grpc/python/zeppelin_python.py
index 1a9e8af..6d7c33e 100644
--- a/python/src/main/resources/grpc/python/zeppelin_python.py
+++ b/python/src/main/resources/grpc/python/zeppelin_python.py
@@ -33,6 +33,9 @@ class PyZeppelinContext(object):
self.javaList = gateway.jvm.java.util.ArrayList
self.max_result = z.getMaxResult()
+ def getInterpreterContext(self):
+return self.z.getInterpreterContext()
+
def input(self, name, defaultValue=""):
return self.z.input(name, defaultValue)
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/d90716d4/python/src/main/resources/python/zeppelin_python.py
--
diff --git a/python/src/main/resources/python/zeppelin_python.py
b/python/src/main/resources/python/zeppelin_python.py
index 519d1af..f4ea2c7 100644
--- a/python/src/main/resources/python/zeppelin_python.py
+++ b/python/src/main/resources/python/zeppelin_python.py
@@ -61,7 +61,7 @@ class PyZeppelinContext(object):
self._setup_matplotlib()
def getInterpreterContext(self):
-return self.z.getCurrentInterpreterContext()
+return self.z.getInterpreterContext()
def input(self, name, defaultValue=&quo
zeppelin git commit: [ZEPPELIN-3283] Remove redundant method InterpreterOutput.getInterpreterResultMessages()
Repository: zeppelin
Updated Branches:
refs/heads/master 63c53fcc5 -> 66dada66e
[ZEPPELIN-3283] Remove redundant method
InterpreterOutput.getInterpreterResultMessages()
### What is this PR for?
Remove Redundant method `InterpreterOutput.getInterpreterResultMessages()`
### What type of PR is it?
Refactoring
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3283
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo <m...@apache.org>
Closes #2828 from Leemoonsoo/ZEPPELIN-3283 and squashes the following commits:
12771a145 [Lee moon soo] remove unused import
82bda9c02 [Lee moon soo] Remove redundant method getInterpreterResultMessages()
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/66dada66
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/66dada66
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/66dada66
Branch: refs/heads/master
Commit: 66dada66e25a1d24cc235fd44e58a0507fe07b84
Parents: 63c53fc
Author: Lee moon soo <m...@apache.org>
Authored: Fri Mar 2 15:47:35 2018 -0800
Committer: Lee moon soo <m...@apache.org>
Committed: Sun Mar 4 10:25:01 2018 -0800
--
.../zeppelin/python/IPythonInterpreterTest.java | 48 ++--
.../zeppelin/spark/IPySparkInterpreterTest.java | 16 +++
.../zeppelin/interpreter/InterpreterOutput.java | 11 -
3 files changed, 32 insertions(+), 43 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/66dada66/python/src/test/java/org/apache/zeppelin/python/IPythonInterpreterTest.java
--
diff --git
a/python/src/test/java/org/apache/zeppelin/python/IPythonInterpreterTest.java
b/python/src/test/java/org/apache/zeppelin/python/IPythonInterpreterTest.java
index 480cae3..869d390 100644
---
a/python/src/test/java/org/apache/zeppelin/python/IPythonInterpreterTest.java
+++
b/python/src/test/java/org/apache/zeppelin/python/IPythonInterpreterTest.java
@@ -88,7 +88,7 @@ public class IPythonInterpreterTest {
InterpreterContext context = getInterpreterContext();
result = interpreter.interpret("print(1)",
context);
assertEquals(InterpreterResult.Code.ERROR, result.code());
-List interpreterResultMessages =
context.out.getInterpreterResultMessages();
+List interpreterResultMessages =
context.out.toInterpreterResultMessage();
assertEquals(1, interpreterResultMessages.size());
assertTrue(interpreterResultMessages.get(0).getData().contains("Frame size
32 exceeds maximum: 4"));
@@ -120,7 +120,7 @@ public class IPythonInterpreterTest {
result = interpreter.interpret("import sys\nprint(sys.version[0])",
context);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
Thread.sleep(100);
-List interpreterResultMessages =
context.out.getInterpreterResultMessages();
+List interpreterResultMessages =
context.out.toInterpreterResultMessage();
assertEquals(1, interpreterResultMessages.size());
boolean isPython2 =
interpreterResultMessages.get(0).getData().equals("2\n");
@@ -129,7 +129,7 @@ public class IPythonInterpreterTest {
result = interpreter.interpret("'hello world'", context);
Thread.sleep(100);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
-interpreterResultMessages = context.out.getInterpreterResultMessages();
+interpreterResultMessages = context.out.toInterpreterResultMessage();
assertEquals(1, interpreterResultMessages.size());
assertEquals("'hello world'", interpreterResultMessages.get(0).getData());
@@ -138,7 +138,7 @@ public class IPythonInterpreterTest {
result = interpreter.interpret("print(u'ä½ å¥½')", context);
Thread.sleep(100);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
-interpreterResultMessages = context.out.getInterpreterResultMessages();
+interpreterResultMessages = context.out.toInterpreterResultMessage();
assertEquals(1, interpreterResultMessages.size());
assertEquals("ä½ å¥½\n", interpreterResultMessages.get(0).getData());
@@ -147,7 +147,7 @@ public class IPythonInterpreterTest {
result = interpreter.interpret("'hello world'\n'hello world2'", context);
Thread.sleep(100);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
-interpreterResultMessages = context.out.getInterpreterResultMessages();
+interpreterResultMessages = context.out.toInterpreterResultMessage();
assertEquals(1, interpreterResultMessages.size());
assertEquals("'he
zeppelin git commit: [ZEPPELIN-3283] Remove redundant method InterpreterOutput.getInterpreterResultMessages()
Repository: zeppelin
Updated Branches:
refs/heads/branch-0.8 e27e8d4ff -> 23907af25
[ZEPPELIN-3283] Remove redundant method
InterpreterOutput.getInterpreterResultMessages()
### What is this PR for?
Remove Redundant method `InterpreterOutput.getInterpreterResultMessages()`
### What type of PR is it?
Refactoring
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3283
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo <m...@apache.org>
Closes #2828 from Leemoonsoo/ZEPPELIN-3283 and squashes the following commits:
12771a145 [Lee moon soo] remove unused import
82bda9c02 [Lee moon soo] Remove redundant method getInterpreterResultMessages()
(cherry picked from commit 66dada66e25a1d24cc235fd44e58a0507fe07b84)
Signed-off-by: Lee moon soo <m...@apache.org>
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/23907af2
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/23907af2
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/23907af2
Branch: refs/heads/branch-0.8
Commit: 23907af25cb0cb2ffcd48699b74600bc02b2d19c
Parents: e27e8d4
Author: Lee moon soo <m...@apache.org>
Authored: Fri Mar 2 15:47:35 2018 -0800
Committer: Lee moon soo <m...@apache.org>
Committed: Sun Mar 4 10:25:09 2018 -0800
--
.../zeppelin/python/IPythonInterpreterTest.java | 48 ++--
.../zeppelin/spark/IPySparkInterpreterTest.java | 16 +++
.../zeppelin/interpreter/InterpreterOutput.java | 11 -
3 files changed, 32 insertions(+), 43 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/23907af2/python/src/test/java/org/apache/zeppelin/python/IPythonInterpreterTest.java
--
diff --git
a/python/src/test/java/org/apache/zeppelin/python/IPythonInterpreterTest.java
b/python/src/test/java/org/apache/zeppelin/python/IPythonInterpreterTest.java
index 480cae3..869d390 100644
---
a/python/src/test/java/org/apache/zeppelin/python/IPythonInterpreterTest.java
+++
b/python/src/test/java/org/apache/zeppelin/python/IPythonInterpreterTest.java
@@ -88,7 +88,7 @@ public class IPythonInterpreterTest {
InterpreterContext context = getInterpreterContext();
result = interpreter.interpret("print(1)",
context);
assertEquals(InterpreterResult.Code.ERROR, result.code());
-List interpreterResultMessages =
context.out.getInterpreterResultMessages();
+List interpreterResultMessages =
context.out.toInterpreterResultMessage();
assertEquals(1, interpreterResultMessages.size());
assertTrue(interpreterResultMessages.get(0).getData().contains("Frame size
32 exceeds maximum: 4"));
@@ -120,7 +120,7 @@ public class IPythonInterpreterTest {
result = interpreter.interpret("import sys\nprint(sys.version[0])",
context);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
Thread.sleep(100);
-List interpreterResultMessages =
context.out.getInterpreterResultMessages();
+List interpreterResultMessages =
context.out.toInterpreterResultMessage();
assertEquals(1, interpreterResultMessages.size());
boolean isPython2 =
interpreterResultMessages.get(0).getData().equals("2\n");
@@ -129,7 +129,7 @@ public class IPythonInterpreterTest {
result = interpreter.interpret("'hello world'", context);
Thread.sleep(100);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
-interpreterResultMessages = context.out.getInterpreterResultMessages();
+interpreterResultMessages = context.out.toInterpreterResultMessage();
assertEquals(1, interpreterResultMessages.size());
assertEquals("'hello world'", interpreterResultMessages.get(0).getData());
@@ -138,7 +138,7 @@ public class IPythonInterpreterTest {
result = interpreter.interpret("print(u'ä½ å¥½')", context);
Thread.sleep(100);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
-interpreterResultMessages = context.out.getInterpreterResultMessages();
+interpreterResultMessages = context.out.toInterpreterResultMessage();
assertEquals(1, interpreterResultMessages.size());
assertEquals("ä½ å¥½\n", interpreterResultMessages.get(0).getData());
@@ -147,7 +147,7 @@ public class IPythonInterpreterTest {
result = interpreter.interpret("'hello world'\n'hello world2'", context);
Thread.sleep(100);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
-interpreterResultMessages = context.out.getInterpreterResultMessages();
+interpreterResultMessages
[2/2] zeppelin git commit: [ZEPPELIN-3194][NEW-INTERPRETER] SAP Universe interpreter
[ZEPPELIN-3194][NEW-INTERPRETER] SAP Universe interpreter ### What is this PR for? New interpreter, SAP BusinessObjects (Universes). See [documentation](https://github.com/masyan/zeppelin/blob/9178d1cc80253b9ae58b988443b619127f42d945/docs/interpreter/sap.md) ### What type of PR is it? [Feature] ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-3194 ### How should this be tested? see screenshots ### Screenshots (if appropriate) Autocomplete + get results  Errors  ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Maksim Reshetov <jmas...@gmail.com> Closes #2763 from masyan/ZEPPELIN-3194 and squashes the following commits: 696b7dd54 [Maksim Reshetov] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-3194 1e25fd1ee [Maksim Reshetov] [ZEPPELIN-3194] refactor parser os query. add tests 24336a855 [Maksim Reshetov] [ZEPPELIN-3194] fix parser for result objects and conditions 9178d1cc8 [Maksim Reshetov] [ZEPPELIN-3194] docs ce97551b9 [Maksim Reshetov] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-3194 0784ba030 [Maksim Reshetov] [ZEPPELIN-3194] docs e00ace2e3 [Maksim Reshetov] [ZEPPELIN-3194] completer tests 1aeb7ad0f [Maksim Reshetov] [ZEPPELIN-3194] pom license c1c09af5c [Maksim Reshetov] [ZEPPELIN-3194] docs fix + unit tests for util f6b71f9c2 [Maksim Reshetov] [ZEPPELIN-3194] is null fix (cherry picked from commit 63c53fcc5979b8c2cbd8bae13de2561078780637) Signed-off-by: Lee moon soo <m...@apache.org> Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/e27e8d4f Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/e27e8d4f Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/e27e8d4f Branch: refs/heads/branch-0.8 Commit: e27e8d4ff8c274b86ce1724172dc3a15326302df Parents: 3fd2fec Author: Maksim Reshetov <jmas...@gmail.com> Authored: Thu Mar 1 23:34:20 2018 +0500 Committer: Lee moon soo <m...@apache.org> Committed: Sun Mar 4 10:00:32 2018 -0800 -- docs/index.md | 1 + docs/interpreter/sap.md | 123 pom.xml | 1 + sap/pom.xml | 86 +++ .../zeppelin/sap/UniverseInterpreter.java | 215 ++ .../zeppelin/sap/universe/UniverseClient.java | 733 +++ .../sap/universe/UniverseCompleter.java | 344 + .../sap/universe/UniverseException.java | 38 + .../zeppelin/sap/universe/UniverseInfo.java | 60 ++ .../zeppelin/sap/universe/UniverseNodeInfo.java | 85 +++ .../sap/universe/UniverseNodeInfoCompleter.java | 183 + .../zeppelin/sap/universe/UniverseQuery.java| 53 ++ .../sap/universe/UniverseQueryPrompt.java | 110 +++ .../zeppelin/sap/universe/UniverseUtil.java | 643 sap/src/main/resources/interpreter-setting.json | 42 ++ sap/src/main/resources/universe.keywords| 1 + .../sap/universe/UniverseCompleterTest.java | 134 .../zeppelin/sap/universe/UniverseUtilTest.java | 371 ++ .../zeppelin/completer/CompletionType.java | 3 +- .../zeppelin/conf/ZeppelinConfiguration.java| 3 +- .../notebook/paragraph/paragraph.controller.js | 2 +- 21 files changed, 3228 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/e27e8d4f/docs/index.md -- diff --git a/docs/index.md b/docs/index.md index 3d42735..f00571e 100644 --- a/docs/index.md +++ b/docs/index.md @@ -150,6 +150,7 @@ limitations under the License. * [Postgresql, HAWQ](./interpreter/postgresql.html) * [Python](./interpreter/python.html) * [R](./interpreter/r.html) + * [SAP](./interpreter/sap.html) * [Scalding](./interpreter/scalding.html) * [Scio](./interpreter/scio.html) * [Shell](./interpreter/Shell.html) http://git-wip-us.apache.org/repos/asf/zeppelin/blob/e27e8d4f/docs/interpreter/sap.md -- diff --git a/docs/interpreter/sap.md b/docs/interpreter/sap.md new file mode 100644 index 000..be05aee --- /dev/null +++ b/docs/interpreter/sap.md @@ -0,0 +1,123 @@ +--- + +layout: page + +title: "SAP BusinessObjects Interpreter for Apache Zeppelin" + +description: "SAP BusinessObjects BI platform can simplify the lives of business users and IT staff. SAP BusinessObjec
[1/2] zeppelin git commit: [ZEPPELIN-3194][NEW-INTERPRETER] SAP Universe interpreter
Repository: zeppelin
Updated Branches:
refs/heads/branch-0.8 3fd2fec0d -> e27e8d4ff
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/e27e8d4f/sap/src/main/java/org/apache/zeppelin/sap/universe/UniverseUtil.java
--
diff --git
a/sap/src/main/java/org/apache/zeppelin/sap/universe/UniverseUtil.java
b/sap/src/main/java/org/apache/zeppelin/sap/universe/UniverseUtil.java
new file mode 100644
index 000..dc9099d
--- /dev/null
+++ b/sap/src/main/java/org/apache/zeppelin/sap/universe/UniverseUtil.java
@@ -0,0 +1,643 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zeppelin.sap.universe;
+
+import org.apache.commons.lang.StringUtils;
+
+import java.util.*;
+
+/**
+ * Util class for convert request from Zeppelin to SAP
+ */
+public class UniverseUtil {
+
+ private static final String COMPRASION_START_TEMPLATE = "\n";
+ private static final String COMPRASION_END_TEMPLATE =
"\n";
+ private static final String COMPARISON_FILTER = "\n";
+ private static final String CONST_OPERAND_START_TEMPLATE =
"\n";
+ private static final String CONST_OPERAND_END_TEMPLATE =
"\n";
+ private static final String CONST_OPERAND_VALUE_TEMPLATE = "\n" +
+ "%s\n\n";
+ private static final String PREDEFINED_FILTER_TEMPLATE = "\n";
+ private static final String OBJECT_OPERAND_TEMPLATE = "\n";
+ private static final String RESULT_START_TEMPLATE = "\n";
+ private static final String RESULT_END_TEMPLATE = "\n";
+ private static final String RESULT_OBJ_TEMPLATE = "\n";
+
+ private static final String MARKER_EQUAL = "#EqualTo#";
+ private static final String MARKER_LESS_EQUAL = "#LessThanOrEqualTo#";
+ private static final String MARKER_NOT_EQUAL = "#NotEqualTo#";
+ private static final String MARKER_LESS = "#LessThan#";
+ private static final String MARKER_GREATER_EQUALS = "#GreaterThanOrEqualTo#";
+ private static final String MARKER_GREATER = "#GreaterThan#";
+ private static final String MARKER_IN = "#InList#";
+ private static final String MARKER_NOT_IN = "#NotInList#";
+ private static final String MARKER_NULL = "#IsNull#";
+ private static final String MARKER_NOT_NULL = "#IsNotNull#";
+ private static final String MARKER_FILTER = "#filter#";
+ private static final String MARKER_AND = "#and#";
+ private static final String MARKER_OR = "#or#";
+ private static final String MARKER_BACKSPACE = "#backspace#";
+ private static final String MARKER_LEFT_BRACE = "#left_brace#";
+ private static final String MARKER_RIGHT_BRACE = "#right_brace#";
+
+
+ private static final String LEFT_BRACE = "(";
+ private static final String RIGHT_BRACE = ")";
+
+ public static final Map OPERATIONS;
+
+ static {
+OPERATIONS = new HashMap<>();
+OPERATIONS.put(MARKER_EQUAL, 1);
+OPERATIONS.put(MARKER_LESS_EQUAL, 1);
+OPERATIONS.put(MARKER_NOT_EQUAL, 1);
+OPERATIONS.put(MARKER_LESS, 1);
+OPERATIONS.put(MARKER_GREATER_EQUALS, 1);
+OPERATIONS.put(MARKER_GREATER, 1);
+OPERATIONS.put(MARKER_IN, 1);
+OPERATIONS.put(MARKER_NOT_IN, 1);
+OPERATIONS.put(MARKER_NULL, 1);
+OPERATIONS.put(MARKER_NOT_NULL, 1);
+OPERATIONS.put(MARKER_FILTER, 1);
+OPERATIONS.put(MARKER_AND, 2);
+OPERATIONS.put(MARKER_OR, 3);
+ }
+
+ public UniverseQuery convertQuery(String text, UniverseClient client, String
token)
+ throws UniverseException {
+StringBuilder select = new StringBuilder();
+StringBuilder universe = new StringBuilder();
+StringBuilder buf = new StringBuilder();
+StringBuilder resultObj = new StringBuilder();
+StringBuilder whereBuf = new StringBuilder();
+UniverseInfo universeInfo = null;
+String where = null;
+boolean singleQuoteClosed = true;
+boolean pathClosed = true;
+boolean universePart = false;
+boolean selectPart = false;
+boolean wherePart = false;
+boolean listOperator = false;
+boolean operatorPosition = false;
+Map nodeInfos = null;
+
+char[] array = text.toCharArray();
+for (int i = 0; i < array.length; i++) {
+ char c = array[i];
+ buf.append(c);
+ if (c == '\'') {
+if (i == 0 ||
