GitHub user astroshim opened a pull request: https://github.com/apache/zeppelin/pull/2581
Fix/jdbc connection release 0.7 ### What is this PR for? This PR fixes JDBC connection release problem. for example whenever i run not executable command like following ![image](https://user-images.githubusercontent.com/3348133/30206892-8248d1ae-94c8-11e7-9eae-a495be075892.png) new JDBC connection is made like following. ``` $ netstat -an |grep EST |grep 3306 |wc -l 1 $ netstat -an |grep EST |grep 3306 |wc -l 2 $ netstat -an |grep EST |grep 3306 |wc -l 3 ``` ### What type of PR is it? Bug Fix ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No You can merge this pull request into a Git repository by running: $ git pull https://github.com/astroshim/zeppelin fix/jdbcConnectionRelease-0.7 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/2581.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2581 ---- commit b62e2e01bbdca87ab68a68ffae5e228217da5679 Author: AhyoungRyu <fbdkdu...@hanmail.net> Date: 2017-04-04T06:33:39Z [ZEPPELIN-2341] Remove -Psparkr build profile ### What is this PR for? Currently users who build Zeppelin from source need to include `-Psparkr` to use `%r` with embedded local Spark. But it's quite inconvenient to write this build profile every time we build i think. So I removed `-Psparkr` and make `r` related libraries automatically downloaded when we build Zeppelin like I did #2213 ### What type of PR is it? Improvement ### Todos * [x] - remove the rest of `-Psparkr` build profile in `dev/create_release.sh`, `dev/publish_release.sh`, and `docs/install/build.md` after #2213 merged ### What is the Jira issue? [ZEPPELIN-2341](https://issues.apache.org/jira/browse/ZEPPELIN-2341) ### How should this be tested? 1. Apply this patch 2. Build source with below command ``` mvn clean package -DskipTests -pl 'zeppelin-interpreter, zeppelin-zengine, zeppelin-server, zeppelin-display, spark, spark-dependencies' ``` Aftr this step, there will be `R` dir under `ZEPPELIN_HOME/interpreter/spark`. Before this PR, only `dep` dir and `zeppelin-spark_2.10-0.8.0-SNAPSHOT.jar` is generated without `-Psparkr` build profile. 4. Restart Zeppelin. To make sure, run R tutorial note under `Zeppelin Tutorial` folder It should be run successfully without any error ### Screenshots (if appropriate) If we build without `-Psparkr` - before : R related properties are not activated by default in Spark interpreter ![screen shot 2017-04-03 at 4 31 49 pm](https://cloud.githubusercontent.com/assets/10060731/24599560/b952e414-188b-11e7-80db-ac649c869c02.png) - after ![after](https://cloud.githubusercontent.com/assets/10060731/24599567/bc513a94-188b-11e7-9e93-7abca3428279.png) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: AhyoungRyu <fbdkdu...@hanmail.net> Author: Ahyoung Ryu <ahyoung...@apache.org> Closes #2215 from AhyoungRyu/ZEPPELIN-2341/includeSparkRByDefault and squashes the following commits: 8db18cc [AhyoungRyu] Remove the rest of '-Psparkr' in docs & sh files f891fd4 [Ahyoung Ryu] Merge branch 'master' into ZEPPELIN-2341/includeSparkRByDefault 445be3e [AhyoungRyu] Add SPARKR env to check each test case need to download r dep or not 67af02a [AhyoungRyu] Remove -PsparkR in travis file a00466c [AhyoungRyu] Remove sparkr build profile in pom files commit 32e86d02d8ea8fdc66a2c1a39bf5cc2317922556 Author: AhyoungRyu <fbdkdu...@hanmail.net> Date: 2017-04-04T06:47:21Z [Minor] Fix wrong Python lib path in Matplotlib tutorial note ### What is this PR for? There is a wrong information about Zeppelinâs matplotlib backend files. It actually located under `ZEPPELIN_HOME/interpreter/lib/python/*` but tutorial note says it as `ZEPPELIN_HOME/lib/python/*`. So I corrected it. ### What type of PR is it? kind a typo fix ### What is the Jira issue? N/A ### Screenshots (if appropriate) - Before ![screen shot 2017-04-04 at 3 50 06 pm](https://cloud.githubusercontent.com/assets/10060731/24644505/e573d3a0-194e-11e7-826f-4022bc5e1572.png) - After ![screen shot 2017-04-04 at 3 50 56 pm](https://cloud.githubusercontent.com/assets/10060731/24644507/e6ae25c2-194e-11e7-91d7-f42b327d820e.png) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: AhyoungRyu <fbdkdu...@hanmail.net> Closes #2220 from AhyoungRyu/fix/wrongPythonLibPathInTutorialNote and squashes the following commits: b43dc44 [AhyoungRyu] Fix wrong Python lib path in tutorial note commit 2e6f14702f6fb1b7fe76fd2e07c11f512741968e Author: William Markito Oliveira <mark...@apache.org> Date: 2017-03-30T02:19:25Z [ZEPPELIN-2326] - Updating Geode dependencies and imports due to package rename ### What is this PR for? Updating Geode dependencies and imports due to package rename - ZEPPELIN-2326 ### What type of PR is it? Refactoring ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-2326 ### How should this be tested? Same tests. One test (`oqlPdxInstanceResponse`) was failing before this change and it's still failing... Looks like a problem with the mock response, but I've not looked further. All remaining tests are passing. ### Questions: * Does the licenses files need update? N/A * Is there breaking changes for older versions? N/A * Does this needs documentation? N/A Author: William Markito Oliveira <mark...@apache.org> Closes #2199 from markito/master and squashes the following commits: e2c5650 [William Markito Oliveira] Merge branch 'master' of https://github.com/markito/zeppelin 54952a2 [William Markito Oliveira] Updating Geode dependencies and imports due to package rename ededffb [William Markito Oliveira] Updating Geode dependencies and imports due to package rename commit 5381883ff5dc50c9ccb3fe3593496b59853bad8f Author: LeiWang <wanglei6...@163.com> Date: 2017-04-07T14:10:18Z add a configuration zeppelin.jdbc.auth.kerberos.proxy for kerberos proxy behavior â¦e to disable proxy behavior ### What is this PR for? A few sentences describing the overall goals of the pull request's commits. First time? Check out the contributing guide - https://zeppelin.apache.org/contribution/contributions.html ### What type of PR is it? [Improvement] * add a configuration item zeppelin.jdbc.auth.kerberos.proxy.enable to disable kerberos behaviour as we knowï¼ in current version of zeppelinï¼ if we have kerberos auth configuredï¼and using zeppelin with a user loginï¼the jdbc interpreter will do a proxy behavior with the login user automaticallyï¼but in many casesï¼ we do not want do thisï¼and we do not want bind the zeppelin user system with kerberos auth system. I think it's make senses to add a configuration item to disable this behavior. ### Todos * [ ] - Task ### What is the Jira issue? * Open an issue on Jira https://issues.apache.org/jira/browse/ZEPPELIN/ * Put link here, and add [ZEPPELIN-*Jira number*] in PR title, eg. [ZEPPELIN-533] [ZEPPELIN-2353] (https://issues.apache.org/jira/browse/ZEPPELIN-2353) ### How should this be tested? Outline the steps to test the PR here. when we have kerberos auth configured, and using zeppelin with a login user add configuration: "zeppelin.jdbc.auth.kerberos.proxy.enable=false" for jdbc iterpreter configuration, the jdbc iterpreter will not do the proxy behavior with the login user ### Screenshots (if appropriate) <img width="1405" alt="2017-04-04 9 12 03" src="https://cloud.githubusercontent.com/assets/869480/24658501/7739e680-197c-11e7-90ab-c1938e31efc7.png"> ### Questions: * Does the licenses files need update? * Is there breaking changes for older versions? * Does this needs documentation? Author: LeiWang <wanglei6...@163.com> Author: lei wang <lei1...@outlook.com> Closes #2222 from wary/main-master and squashes the following commits: 1f26bc3 [LeiWang] Update jdbc.md ba34a41 [LeiWang] Update JDBCInterpreter.java 444b6ab [LeiWang] Update JDBCInterpreter.java ea11ce4 [LeiWang] Update jdbc.md ee9cd18 [LeiWang] Update jdbc.md e04fb38 [LeiWang] Update jdbc.md 75e4b9f [LeiWang] Update jdbc.md b850a42 [LeiWang] Update JDBCInterpreter.java 067696a [lei wang] add configure zeppelin.jdbc.auth.kerberos.proxy for kerberos auth type to disable proxy behavior commit 495be1ede1c0f5fc949275153dcaa1c6de5ddcdd Author: Jeff Zhang <zjf...@apache.org> Date: 2017-04-08T00:15:14Z ZEPPELIN-2150. NoSuchMethodError: org.apache.spark.ui.SparkUI.appUIAddress() for the spark master ### What is this PR for? `SparkUI.addUIAddress` is removed in spark master which cause this error. Actually spark 2.0 introduce new api `SparkContext.uiWebUrl` (SPARK-14576) which is public in contrast `SparkUI.addUIAddress` is private. This PR would use `SparkUI.addUIAddress` for spark 1 and `SparkContext.uiWebUrl` for spark 2. ### What type of PR is it? [Improvement | Refactoring] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2150 ### How should this be tested? Manually verify it on spark 1.6.2, spark 2.1.0 and spark master ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjf...@apache.org> Closes #2231 from zjffdu/ZEPPELIN-2150 and squashes the following commits: 499b623 [Jeff Zhang] return null when sparkUIUrl is not avaialbe e285278 [Jeff Zhang] ZEPPELIN-2150. NoSuchMethodError: org.apache.spark.ui.SparkUI.appUIAddress() for the spark master commit 241fd0344da3f6fbda7ba8d0976a85a51e3c33e9 Author: 1ambda <1am...@gmail.com> Date: 2017-04-04T08:59:14Z [ZEPPELIN-2088] Bundle helium packages one by one ### What is this PR for? Bundle helium packages one by one because #### Summary Let's say you have one helium package called `ultimate-line-chart`. **1. Package Path** - **(before)** `$ZEPPELIN_HOME/local-repo/helium-bundles/node_modules/ultimate-line-chart` - **(after)** `$ZEPPELIN_HOME/local-repo/helium-bundles/bundles/ultimate-line-chart` **2. Zeppelin Local Module Path** - **(before)** `$ZEPPELIN_HOME/local-repo/helium-bundles/node_modules/zeppelin-tabledata` - **(after)** `$ZEPPELIN_HOME/local-repo/helium-bundles/local_modules/zeppelin-tabledata` **3. Bundle Cache Path** - **(before)** `$ZEPPELIN_HOME/local-repo/helium-bundles/helium.bundle.cache.js` (mixed one) - **(after)** `$ZEPPELIN_HOME/local-repo/helium-bundles/bundles/ultimate-line-chart/helium.bundle.cache.js` (for each helium package) #### Details - Bundling them in one file is not good idea because the whole bundling is broken when only 1 bundle fails - Some node packages might not installed correctly. For example, `amchart/amcharts3` will be installed in `helium-bundles/node_modules/my-helium-vis/node_modules` but `amchart-export` will be placed in `helium-bundles/node_modules` so `my-helium-vis` can't import `amchart-export`. (This is real case) - Additionally, I used yarn to install required node_modules. separated bundling requires install same dependencies multiple times. This takes more time than before and npm is too slow to install them all. - Install Zeppelin framework modules using [local path](https://docs.npmjs.com/files/package.json#local-paths). It's more safe and correct way to install local modules. ### What type of PR is it? [Improvement] ### Todos NONE ### What is the Jira issue? [ZEPPELIN-2088](https://issues.apache.org/jira/browse/ZEPPELIN-2088) ### How should this be tested? 1. Install any local helium package 2. Install any online helium package 3. Use them 4. Disable 5. Test `npm run dev:helium` ### Screenshots (if appropriate) NONE ### Questions: * Does the licenses files need update? - NO * Is there breaking changes for older versions? - NO * Does this needs documentation? - NO Author: 1ambda <1am...@gmail.com> Closes #2210 from 1ambda/ZEPPELIN-2088/evaluate-helium-bundle-one-by-one and squashes the following commits: bea6c09 [1ambda] fix: Bundling errors 53145fa [1ambda] fix: Add error handling for failed bundle 212c737 [1ambda] feat: Apply #2178 736acee [1ambda] fix: Add nodeInstallationDir 11ef0ae [1ambda] fix: HeliumBundleFactoryTest 5a2a938 [1ambda] fix: Log single bundle error to browser console a99f981 [1ambda] test: Fix APIs 743aba4 [1ambda] feat: Loading bundles 697c5e6 [1ambda] feat: enable, disable 002e66f [1ambda] feat: Build online packages 512508d [1ambda] feat: Build each bundle using yarn commit f9830a7d64921c1f67aeb6ee179b574769e0a6f9 Author: Tinkoff DWH <tinkoff....@gmail.com> Date: 2017-04-05T07:32:44Z [ZEPPELIN-2279] excluded comments from SQL ### What is this PR for? Exclusion comments (single-, multiline) from queries before execution. Comments don't need to execute query and sometimes there are errors. ### What type of PR is it? Bug Fix | Improvement ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-2279 ### How should this be tested? ``` /* ; */ select 1; -- text select 1 /* bla bla bla*/ select 1; -- text ``` ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Tinkoff DWH <tinkoff....@gmail.com> Closes #2158 from tinkoff-dwh/ZEPPELIN-2279 and squashes the following commits: 3f7496e [Tinkoff DWH] [ZEPPELIN-2279] fix conditions, common format f48f7d6 [Tinkoff DWH] [ZEPPELIN-2279] improve test, revert precode execution 2cb94fa [Tinkoff DWH] Merge remote-tracking branch 'origin/master' into ZEPPELIN-2279 6db3c46 [Tinkoff DWH] [ZEPPELIN-2279] excluded comments from SQL commit 540ceb87d86a5751a5a93d3c050ecec36382e91f Author: AhyoungRyu <fbdkdu...@hanmail.net> Date: 2017-04-07T10:44:26Z [ZEPPELIN-2373] Remove -Pyarn build profile ### What is this PR for? Currently users who build Zeppelin from source need to include `-Pyarn` build profile to enable Yarn for embedded local Spark. This PR is to remove `-Pyarn` and make Yarn related libraries can be automatically downloaded without the profile during build time. ### What type of PR is it? just removed Yarn build profile in `spark-dependencies/pom.xml` ### What is the Jira issue? [ZEPPELIN-2373](https://issues.apache.org/jira/browse/ZEPPELIN-2373) ### How should this be tested? 1. apply this patch and build Zeppelin with below command ``` mvn clean package -DskipTests -pl 'zeppelin-server, spark-dependencies, spark' --am ``` 2. check `spark-yarn_${scala.binary.version}.jar` is downloaded(or included) during build time ``` [INFO] Including org.apache.spark:spark-yarn_2.10:jar:2.1.0 in the shaded jar. ``` In current master, the above line won't be shown if you don't use `-Pyarn` build profile. But with this patch, it will be shown without the profile. ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: AhyoungRyu <fbdkdu...@hanmail.net> Closes #2232 from AhyoungRyu/ZEPPELIN-2373/includeYarnByDefault and squashes the following commits: 59eb2b5 [AhyoungRyu] Remove -Pyarn build profile commit b55231464199f4fd5598970ab2f746f0d254c0b4 Author: 1ambda <1am...@gmail.com> Date: 2017-04-06T03:11:46Z [ZEPPELIN-2357] fix: tooltip doesn't work in btn-group ### What is this PR for? tooltip provided by [angular-ui: bootstrap](https://github.com/angular-ui/bootstrap) doesn't work with `btn-group` class. ### What type of PR is it? [Bug Fix] ### Todos * [x] - fixed tooltip issue by bumping version of angular-bootstrap * [x] - added tooltip to charts as like before ### What is the Jira issue? [ZEPPELIN-2357](https://issues.apache.org/jira/browse/ZEPPELIN-2357) ### How should this be tested? 1. Open spark tutorial 2. Test tooltips in notebook controller and in paragraphs. ### Screenshots (if appropriate) #### Before ![tooltip_before](https://cloud.githubusercontent.com/assets/4968473/24699102/dc76be40-1a2d-11e7-9ae3-b550a1739ca8.gif) #### After ![tooltip_after1](https://cloud.githubusercontent.com/assets/4968473/24699105/dfaa5716-1a2d-11e7-8d8a-0576fb307200.gif) ![tooltip_after2](https://cloud.githubusercontent.com/assets/4968473/24699111/e429454a-1a2d-11e7-97cc-09bd77dcc4ca.gif) ### Questions: * Does the licenses files need update? - YES, updated * Is there breaking changes for older versions? - NO * Does this needs documentation? - NO Author: 1ambda <1am...@gmail.com> Closes #2225 from 1ambda/ZEPPELIN-2357/tooltip-doesnt-work-with-btngroup and squashes the following commits: b9365c6 [1ambda] fix: Use uib-pagination directive 2522ff5 [1ambda] fix: Use uib-tooltip in IT classes d03a70e [1ambda] fix: bin license 0b6ad26 [1ambda] feat: Tooltip for charts 1e2db4e [1ambda] fix: Use uib-tooltip instead 62784c8 [1ambda] fix: Bump up angular-bootstrap version commit 35fa9d287fd7535dc9fed9ca8cc84e7851b045b5 Author: Renjith Kamath <renjith.kam...@gmail.com> Date: 2017-04-06T15:25:38Z ZEPPELIN-2366 In zeppelin SystemUser fails to authenticate with AD, using the password set in hadoop credential store. ### What is this PR for? In zeppelin SystemUser fails to authenticate with AD, using the password set in hadoop credential store. ### What type of PR is it? Bug Fix ### Todos * [ ] - Task ### What is the Jira issue? [ZEPPELIN-2366](https://issues.apache.org/jira/browse/ZEPPELIN-2366) ### How should this be tested? Configure AD system user password using hadoop credential in shiro.ini. _(sample config in JIRA ticket)_ ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? n/a * Is there breaking changes for older versions? n/a * Does this needs documentation? n/a Author: Renjith Kamath <renjith.kam...@gmail.com> Closes #2230 from r-kamath/ZEPPELIN-2366 and squashes the following commits: 11a8ab43d [Renjith Kamath] ZEPPELIN-2366 In zeppelin SystemUser fails to authenticate with AD, using the password set in hadoop credential store. commit 902892a97723b0c4deb4ef832588067a0faa9d74 Author: LeiWang <wanglei6...@163.com> Date: 2017-04-09T11:01:46Z fix bugs for timer saver ### What is this PR for? * add saveNote action when cron schedule setting in file notebook.controller.js * case: open one noteï¼edit some text in a paragraph, then do a cron schedule setting, the content which just edited disappeared * change invoke method for the timer saver in file paragraph.controller.js * case: open one browserï¼edit some text in a paragraph, then open the note in other browserï¼you will find that the text you just edited did not be synced ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? [ZEPPELIN-2323](https://issues.apache.org/jira/browse/ZEPPELIN-2323) ### How should this be tested? Outline the steps to test the PR here. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? * Is there breaking changes for older versions? * Does this needs documentation? Author: LeiWang <wanglei6...@163.com> Author: lei wang <lei1...@outlook.com> Closes #2196 from wary/master and squashes the following commits: b2ee2f9 [LeiWang] Update notebook.controller.js 2ab58c3 [LeiWang] Update notebook.controller.js 9908227 [LeiWang] Update paragraph.controller.js 3612f56 [lei wang] fix bugs for timer saver commit 2173b4013aa1a1403eb1c44fc3d39b40f50b5424 Author: Khalid Huseynov <khalid...@gmail.com> Date: 2017-03-29T07:54:38Z [ZEPPELIN-2318] Fix proxy configuration for http client of zeppelinhub storage layer ### What is this PR for? it fixes proxy configuration for http client of zeppelinhub storage layer. currently used jetty-client have issue with `https` over proxy ([1](https://github.com/eclipse/jetty.project/issues/408), [2](https://github.com/eclipse/jetty.project/issues/827)), so new `httpcomponents` library have been used. ### What type of PR is it? Bug Fix | Improvement ### Todos * [x] - implement whole client with proxy ### What is the Jira issue? [ZEPPELIN-2318](https://issues.apache.org/jira/browse/ZEPPELIN-2318) ### How should this be tested? 1. setup proxy and make system point to it (e.g. env should contain `https_proxy`) 2. setup storage (e.g. [here](https://zeppelin.apache.org/docs/0.7.0/storage/storage.html#storage-in-zeppelinhub)) ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Khalid Huseynov <khalid...@gmail.com> Closes #2198 from khalidhuseynov/fix/ZEPPELIN-2318 and squashes the following commits: 8ea7d5d96 [Khalid Huseynov] edge case logs from error -> warn 34bb2ce32 [Khalid Huseynov] fix log 409993a8e [Khalid Huseynov] fix websocket timing 72fa4f12d [Khalid Huseynov] fix trans dependency problem db07f5b56 [Khalid Huseynov] add ssl setup 8cb94006c [Khalid Huseynov] add close routine a5711bc1d [Khalid Huseynov] jetty client relay to asyncclient when proxy on 16ccbb461 [Khalid Huseynov] add proxy client with asynclient library d2dab6c96 [Khalid Huseynov] add dependency on asyncclient commit 45cc8a9e8a23a57271dec384245d0012a0e5e608 Author: 1ambda <1am...@gmail.com> Date: 2017-04-11T14:14:46Z [ZEPPELIN-2217] AdvancedTransformation for Visualization ### What is this PR for? `AdvancedTransformation` has more detailed options while providing existing features of `PivotTransformation` and `ColumnselectorTransformation` which Zeppelin already has ![av_in_30sec](https://cloud.githubusercontent.com/assets/4968473/24037330/c9478e86-0b40-11e7-9886-1ffb85042a7a.gif) Here are some features which advanced-transformation can provide. 1. **(screenshot)** multiple sub charts 2. **(screenshot)** parameter widgets: `input`, `checkbox`, `option`, `textarea` 3. **(screenshot)** expand/fold axis and parameter panels 4. **(screenshot)** clear axis and parameter panels 5. **(screenshot)** remove duplicated columns in an axis 6. **(screenshot)** limit column count in an axis 7. configurable char axes: `valueType`, `axisType`, `description`, ... 8. configurable chart parameters 9. lazy transformation 10. parsing parameters automatically based on their type: `int`, `float`, `string`, `JSON` 11. multiple transformation methods 12. re-initialize whole configuration based on spec hash. 13. **(screenshot)** shared axis #### API Details: Spec `AdvancedTransformation` requires `spec` which includes axis and parameter details for charts. - Let's create 2 sub-charts called `simple-line` and `step-line`. - Each sub chart can have different `axis` and `parameter` depending on their requirements. ```js constructor(targetEl, config) { super(targetEl, config) const spec = { charts: { 'simple-line': { sharedAxis: true, /** set if you want to share axes between sub charts, default is `false` */ axis: { 'xAxis': { dimension: 'multiple', axisType: 'key', }, 'yAxis': { dimension: 'multiple', axisType: 'aggregator'}, 'category': { dimension: 'multiple', axisType: 'group', }, }, parameter: { 'xAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of xAxis', }, 'yAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of yAxis', }, 'dashLength': { valueType: 'int', defaultValue: 0, description: 'the length of dash', }, }, }, 'step-line': { axis: { 'xAxis': { dimension: 'single', axisType: 'unique', }, 'yAxis': { dimension: 'multiple', axisType: 'value', }, }, parameter: { 'xAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of xAxis', }, 'yAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of yAxis', }, 'noStepRisers': { valueType: 'boolean', defaultValue: false, description: 'no risers in step line', widget: 'checkbox', }, }, }, } this.transformation = new AdvancedTransformation(config, spec) } ``` #### API Details: Axis Spec | Field Name | Available Values (type) | Description | | --- | --- | --- | |`dimension` | `single` | Axis can contains only 1 column | |`dimension` | `multiple` | Axis can contains multiple columns | |`axisType` | `key` | Column(s) in this axis will be used as `key` like in `PivotTransformation`. These columns will be served in `column.key` | |`axisType` | `aggregator` | Column(s) in this axis will be used as `value` like in `PivotTransformation`. These columns will be served in `column.aggregator` | |`axisType` | `group` | Column(s) in this axis will be used as `group` like in `PivotTransformation`. These columns will be served in `column.group` | |`axisType` | (string) | Any string value can be used here. These columns will be served in `column.custom` | |`maxAxisCount` | (int) | The maximum column count that this axis can contains. (unlimited if `undefined`) | |`valueType` | (string) | Describe the value type just for annotation | Here is an example. ```js axis: { 'xAxis': { dimension: 'multiple', axisType: 'key', }, 'yAxis': { dimension: 'multiple', axisType: 'aggregator'}, 'category': { dimension: 'multiple', axisType: 'group', maxAxisCount: 2, valueType: 'string', }, }, ``` #### API Details: Parameter Spec | Field Name | Available Values (type) | Description | | --- | --- | --- | |`valueType` | `string` | Parameter which has string value | |`valueType` | `int` | Parameter which has int value | |`valueType` | `float` | Parameter which has float value | |`valueType` | `boolean` | Parameter which has boolean value used with `checkbox` widget usually | |`valueType` | `JSON` | Parameter which has JSON value used with `textarea` widget usually. `defaultValue` should be `""` (empty string). This ||`defaultValue` | (any) | Default value of this parameter. `JSON` type should have `""` (empty string) | |`description` | (string) | Description of this parameter. This value will be parsed as HTML for pretty output | |`widget` | `input` | Use [input](https://developer.mozilla.org/en/docs/Web/HTML/Element/input) widget. This is the default widget (if `widget` is undefined)| |`widget` | `checkbox` | Use [checkbox](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/checkbox) widget. | |`widget` | `textarea` | Use [textarea](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/textarea) widget. | |`widget` | `option` | Use [select + option](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/select) widget. This parameter should have `optionValues` field as well. | |`optionValues` | (Array<string>) | Available option values used with the `option` widget | Here is an example. ```js parameter: { // string type, input widget 'xAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of xAxis', }, // boolean type, checkbox widget 'inverted': { widget: 'checkbox', valueType: 'boolean', defaultValue: false, description: 'invert x and y axes', }, // string type, option widget with `optionValues` 'graphType': { widget: 'option', valueType: 'string', defaultValue: 'line', description: 'graph type', optionValues: [ 'line', 'smoothedLine', 'step', ], }, // HTML in `description` 'dateFormat': { valueType: 'string', defaultValue: '', description: 'format of date (<a href="https://docs.amcharts.com/3/javascriptcharts/AmGraph#dateFormat">doc</a>) (e.g YYYY-MM-DD)', }, // JSON type, textarea widget 'yAxisGuides': { widget: 'textarea', valueType: 'JSON', defaultValue: '', description: 'guides of yAxis ', }, ``` #### API Details: Transformer Spec `AdvancedTransformation` supports 3 transformation methods. The return value will depend on the transformation method type. ```js const spec = { charts: { 'simple': { /** default value of `transform.method` is the flatten cube. */ axis: { ... }, parameter: { ... } }, 'cube-group': { transform: { method: 'cube', }, axis: { ... }, parameter: { ... }, } 'no-group': { transform: { method: 'raw', }, axis: { ... }, parameter: { ... }, } ``` | Field Name | Available Values (type) | Description | | --- | --- | --- | |`method` | `object` | designed for [amcharts: serial](https://www.amcharts.com/demos/date-based-data/) | |`method` | `array` | designed for [highcharts: column](http://jsfiddle.net/gh/get/library/pure/highcharts/highcharts/tree/master/samples/highcharts/demo/column-basic/) | |`method` | `drill-down` | designed for [highcharts: drill-down](http://jsfiddle.net/gh/get/library/pure/highcharts/highcharts/tree/master/samples/highcharts/demo/column-drilldown/) | |`method` | `raw` | will return the original `tableData.rows` | Whatever you specified as `transform.method`, the `transformer` value will be always function for lazy computation. ```js // advanced-transformation.util#getTransformer if (transformSpec.method === 'raw') { transformer = () => { return rows; } } else if (transformSpec.method === 'array') { transformer = () => { ... return { ... } } } ``` #### Feature Details: Automatic parameter parsing Advanced transformation will parse parameter values automatically based on their type: `int`, `float`, `string`, `JSON` - See also `advanced-transformation-util.js#parseParameter` #### Feature Details: re-initialize the whole configuration based on spec hash ```js // advanced-transformation-util#initializeConfig const currentVersion = JSON.stringify(spec) if (!config.spec || !config.spec.version || config.spec.version !== currentVersion) { spec.version = currentVersion // reset config... } ``` #### Feature Details: Shared Axes If you set `sharedAxis` to `true` in chart specification, then these charts will share their axes. (default is `false`) ```js const spec = { charts: { 'column': { transform: { method: 'array', }, sharedAxis: true, axis: { ... }, parameter: { ... }, }, 'stacked': { transform: { method: 'array', }, sharedAxis: true, axis: { ... } parameter: { ... }, }, ``` ![sharedaxis](https://cloud.githubusercontent.com/assets/4968473/24207116/6999ad8a-0f63-11e7-8b61-273b712612fc.gif) #### API Details: Usage in Visualization#render() Let's assume that we want to create 2 sub-charts called `basic` and `no-group`. - https://github.com/1ambda/zeppelin-ultimate-line-chart (an practical example) ```js drawBasicChart(parameter, column, transformer) { const { ... } = transformer() } drawNoGroupChart(parameter, column, transformer) { const { ... } = transformer() } render(data) { const { chart, parameter, column, transformer, } = data if (chart === 'basic') { this.drawBasicChart(parameter, column, transformer) } else if (chart === 'no-group') { this.drawNoGroupChart(parameter, column, transformer) } } ``` ### What type of PR is it? [Feature] ### Todos NONE ### What is the Jira issue? [ZEPPELIN-2217](https://issues.apache.org/jira/browse/ZEPPELIN-2217) ### How should this be tested? 1. Clone https://github.com/1ambda/zeppelin-ultimate-line-chart 2. Create a symbolic link `ultimate-line-chart.json` into `$ZEPPELIN_HOME/helium` 3. Modify the `artifact` value to proper absolute path considering your local machine. 4. Install the above visualization in `localhost:9000/#helium` 5. Test it ### Screenshots (if appropriate) #### 1. *(screenshot)* multiple sub charts ![av_multiple_charts](https://cloud.githubusercontent.com/assets/4968473/24034638/7b84dba0-0b35-11e7-989d-059ccc87f968.gif) #### 2. *(screenshot)* parameter widgets: `input`, `checkbox`, `option`, `textarea` ![av_widgets_new](https://cloud.githubusercontent.com/assets/4968473/24034652/88679d6c-0b35-11e7-835a-3970d7124850.gif) #### 3. *(screenshot)* expand/fold axis and parameter panels ![av_fold_expand](https://cloud.githubusercontent.com/assets/4968473/24034653/8a634ddc-0b35-11e7-9851-15280a6b5fd3.gif) #### 4. *(screenshot)* clear axis and parameter panels ![av_clean_buttons](https://cloud.githubusercontent.com/assets/4968473/24034654/8d3dc14a-0b35-11e7-98c7-3aeddce6d80a.gif) #### 5. *(screenshot)* remove duplicated columns in an axis ![av_duplicated_columns](https://cloud.githubusercontent.com/assets/4968473/24034657/910f4d20-0b35-11e7-9e9b-d9e2f799a5dd.gif) #### 6. *(screenshot)* limit column count in an axis ![av_maxaxiscount](https://cloud.githubusercontent.com/assets/4968473/24034679/a5e8eb34-0b35-11e7-89cd-070f3790d511.gif) ### Questions: * Does the licenses files need update? - NO * Is there breaking changes for older versions? - NO * Does this needs documentation? - NO Author: 1ambda <1am...@gmail.com> Closes #2098 from 1ambda/ZEPPELIN-2217/advanced-transformation and squashes the following commits: 6cde7c9 [1ambda] fix reset params when spec change c75a3f2 [1ambda] fix: Reset persisted axis 6a2130a [1ambda] fix: clear config only when axis changed 5464e84 [1ambda] fix: Optimize array 2 key method 9beb1e7 [1ambda] fix: Type error 2408225 [1ambda] test: Add test for array 2key bf56761 [1ambda] feat: Add array:2-key transform method 7c6768f [1ambda] feat: Use axisSpec.desc as tooltip f98d4c9 [1ambda] fix: Remove invalid key prop 5cf2ece [1ambda] feat: Add minAxisCount 4887800 [1ambda] fix: Remove local module yarn caches 3e29572 [1ambda] refactor: copyModule func c91a033 [1ambda] fix: Set yarn cache dir in helium-bundles 04b5140 [1ambda] fix: Import a-tr 0a876cf [1ambda] docs: Update index.md 380b1af [1ambda] docs: Fix typo and add desc for existing trs 908214b [1ambda] docs: Move experimental tags a009627 [1ambda] feat: Allow dup aggr axis 3b44e92 [1ambda] fix: Remove unuse const ab6c22e [1ambda] test: Add test for drill-down method 756107a [1ambda] test: Add array transformation method d819c73 [1ambda] test: Add object method bf00fba [1ambda] test: Add MockTableData 39fe5ae [1ambda] test: Add test for getColumnsFromAxis 4c393b4 [1ambda] fix: Add polyfill for es6 funcs in test e92c787 [1ambda] test: Add test for rmDup, aplMaxAxisCount 843f45d [1ambda] test: Add test for getCurrent* funcs ae5277c [1ambda] test: Add test for initializeConfig c14a9dc7 [1ambda] test: Add tests for widget, params c510af1 [1ambda] docs: Add doc for Transformation 52db37b [1ambda] feat: Show panel menus only when opened 17ad4a4 [1ambda] feat: Support chartChanged, parameterChanged c0d33d3 [1ambda] fix: Sort selectors in drilldown method cfd6fef [1ambda] feat: sharedAxis 9af80ce [1ambda] style: Indent 79b5654 [1ambda] fix: return the same info in transform 7bee464 [1ambda] fix: Keynames ee8788e [1ambda] feat: Support drill-down 666025a [1ambda] fix: DON'T reset current chart ae1891f [1ambda] add array:key transform 4167a2e [1ambda] fix: Sort keyNames 912b5b7 [1ambda] fix: Persist initialized config f1f6b0c [1ambda] feat: Support ARRAY transform.method 812f9a2 [1ambda] fix: Set proper aggr value when 0 group 20f9437 [1ambda] fix: getCube func 25d51a9 [1ambda] DON'T display aggr.name when aggrColumns.length == 1 f37e13d [1ambda] fix: Add 'object' transform.method da2370c [1ambda] fix: Add resetAxis, Param funcs 2370682 [1ambda] fix: average is not caculated correctly dd08e38 [1ambda] fix: Set param panel height to 400 881695a [1ambda] feat: clear chart, param separately 4d0d62b [1ambda] fix: DON'T clean panel config 92676d1 [1ambda] fix: limit parameter panel height to 370 cc29060 [1ambda] feat: parse param description as HTML 9a2d227 [1ambda] fix: Stop event propagation in widgets fcc625c [1ambda] feat: Automatic param parsing b4d774c [1ambda] fix: Dont close param panel when enter 088705b [1ambda] refactor: Remove util and add Widget funcs bf88b4f [1ambda] feat: textare widget and update hook 4e73012 [1ambda] feat: widget checkbox 11b7eaa [1ambda] feat: option widget 5d3efc9 [1ambda] fix: Change panel header b1d9d31 [1ambda] feat: Save and close with enter key 53f508c [1ambda] feat: custom axisSpec 0dbc431 [1ambda] feat: Support transformer 94d837a [1ambda] feat: Automatic spec versioning 74b8b4e [1ambda] fix: Duplicated radio btn id, name 5b88f08 [1ambda] fix: Modify margin of subchart radio btns 019892c [1ambda] feat: Support transform: flatten 0484e1e [1ambda] feat: Support maxAxisCount in axisSpec 936901b [1ambda] feat: Support undefined valueType in axisSpec 7a454ff [1ambda] feat: Cube Transformation f0ed02f [1ambda] feat: Support same axis types 49985c6 [1ambda] refactor: Refine axis, param spec d89e223 [1ambda] feat: advanced-transformation-api 75569ce [1ambda] feat: Support multiple charts in UI e1fcc2e [1ambda] feat: Support multiple charts 97be629 [1ambda] fix: Add singleDimensionAggregatorChanged 676bd7e [1ambda] refactor: Refine transform API 9fb398e [1ambda] feat: Add clearConfig a8a4fb1 [1ambda] refactor: Add getAxisInSingleDimension func 9768ecf [1ambda] feat: Add groupBase axis option 91ae54d [1ambda] fix: Overflow issue in single aggr 10c80fc [1ambda] feat: AdvancedTransformation commit e5922b6bbaecacdfdcd279c4ff1f181c482210ca Author: Tinkoff DWH <tinkoff....@gmail.com> Date: 2017-04-12T04:06:07Z [ZEPPELIN-2365] button to create note into folder ### What is this PR for? Added button to create new note into folder (home page) ### What type of PR is it? Feature ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-2365 ### Screenshots (if appropriate) ![2365](https://cloud.githubusercontent.com/assets/25951039/24740236/86b31468-1ab9-11e7-8906-3f55ab2a7158.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Tinkoff DWH <tinkoff....@gmail.com> Closes #2228 from tinkoff-dwh/ZEPPELIN-2365 and squashes the following commits: e870ad4 [Tinkoff DWH] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-2365 0fe290e [Tinkoff DWH] [ZEPPELIN-2365] button to create note into folder commit 861f1d88fe2ea105df6892abef14142327c49f6f Author: Benoy Antony <be...@apache.org> Date: 2017-04-12T03:55:36Z [ZEPPELIN-2355] Livy cancel enhancements ### What is this PR for? The Cancel functionality for the Livy interpreter has few issues. One issue is because a variable is not published correctly. Second issue is observed when there is a delay in launching the application. Any cancel before application launch is ignored. The third issue is that Cancel is not correctly implemented for SparkSQLInterpreter. ### What type of PR is it? Bug Fix ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-2355 ### How should this be tested? The test cases are modified to test the changes. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Benoy Antony <be...@apache.org> Closes #2223 from benoyantony/livy-cancel-enhancement and squashes the following commits: 244e6d3 [Benoy Antony] clear the cancel requests if livy doesnt't support cancellation and modified testcase 75fe574 [Benoy Antony] added testcase for cancellation support on LivySparkSQLInterpreter and moved the removal to finally block 9fc6dbf [Benoy Antony] remove unrelated changes in imports 8673acf [Benoy Antony] ZEPPELIN-2355 Fix race conditions while cancelling a paragraph commit 5fd4ffc56658884b8f671ca40b1f78b8354f1f69 Author: Jeff Zhang <zjf...@apache.org> Date: 2017-04-16T08:38:36Z ZEPPELIN-2407. Livy Interpreter always return plain text result ### What is this PR for? It happens when zeppelin.livy.displayAppInfo is true. Straightforward fix. ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2407 ### How should this be tested? Test is added ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjf...@apache.org> Closes #2254 from zjffdu/ZEPPELIN-2407 and squashes the following commits: ba59a77 [Jeff Zhang] ZEPPELIN-2407. Livy Interpreter always return plain text result commit 74c0408d30927b11c99a8e134284be4156b19f8a Author: Jeff Zhang <zjf...@apache.org> Date: 2017-04-09T07:46:32Z ZEPPELIN-2377. Hive Support can not be enabled in spark master ### What is this PR for? The root cause is that `org.apache.spark.sql.hive.HiveSessionState` is removed in spark master. I change it to `org.apache.spark.sql.hive.execution.InsertIntoHiveTable` which is existed early in spark 1.0. ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2377 ### How should this be tested? Verify it manually in spark master, spark 2.1.0 and spark 1.6.2 ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjf...@apache.org> Closes #2237 from zjffdu/ZEPPELIN-2377 and squashes the following commits: d16196b [Jeff Zhang] ZEPPELIN-2377. Hive Support can not be enabled in spark master commit 8d03920b9bec86dd8e2fc343d32749a08f501362 Author: CloverHearts <cloverhearts...@gmail.com> Date: 2017-04-12T08:14:36Z [Zeppelin-802] Support for Zeppelin Context redefinition on Python and Pyspark ### What is this PR for? If you override the reserved word ZeppelinContext such as `z` in the python language, the whole paragraph output problem occurred. I have taken care to avoid this issue. `z` == `_zc` == `zeppelin context` ### What type of PR is it? Improvement ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-802 ### How should this be tested? The error should not occur in the following situations: ``` %python z = 1 print("Hello Zeppelin") ``` ``` %pyspark z = 1 print("Hello Zeppelin") ``` ### Screenshots (if appropriate) #### before ![replace zeppelin context-err](https://cloud.githubusercontent.com/assets/10525473/24521772/319946be-15c8-11e7-96cf-7fdf41c70a66.png) #### after ![replace zeppelin context](https://cloud.githubusercontent.com/assets/10525473/24521775/349fa7cc-15c8-11e7-8fe4-4f3f5597deff.png) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: CloverHearts <cloverhearts...@gmail.com> Author: cloverhearts <cloverhearts...@gmail.com> Closes #2207 from cloverhearts/ZEPPELIN-802-pyspark-zeppelin-context and squashes the following commits: cc986010 [CloverHearts] added completion on namespace 14695cb8 [CloverHearts] Recovering a member name that is not associated with a namespace 31af92ab [CloverHearts] fix test case _zc to __zeppelin__ 6697d677 [CloverHearts] apply to namespace and replace name _zc to __zeppelin__ ca795cff [cloverhearts] replace output 1d372df4 [cloverhearts] change name logger 4e8435ac [CloverHearts] added test case on python b6b804ad [CloverHearts] replace name zeppelin context on python 9fbf70d6 [CloverHearts] fix pyspark test case 987e2118 [CloverHearts] added test code 5da3d6ed [CloverHearts] replace name zeppelin context on pyspark commit 775607f103d1e5f6c6e209b687f11bdc038c3e71 Author: soralee <sora0...@zepl.com> Date: 2017-04-15T05:01:40Z [ZEPPELIN-2396] eliminate the 'ctrl/command+L' keyboard shortcut ### What is this PR for? This PR is for that eliminate the `Ctrl/Command+L` shortcut keyboard. This function is what the ace editor defined. ### What type of PR is it? [Improvement | Document] ### What is the Jira issue? * [ZEPPELIN-2396](https://issues.apache.org/jira/browse/ZEPPELIN-2396) ### How should this be tested? * **Improvement** - Press `Ctrl + L` key and check if showing the dialog or not * **Document** - Run document development mode and check [this document](http://localhost:4000/install/upgrade.html#upgrading-from-zeppelin-07-to-08) ### Screenshots (if appropriate) [Before] ![image](https://cloud.githubusercontent.com/assets/8110458/24990819/8529312a-2051-11e7-8849-e00803310752.png) [After] ![z_zeppelin-2396](https://cloud.githubusercontent.com/assets/8110458/24990966/9764e108-2052-11e7-9387-560f9d587782.gif) [Document] ![image](https://cloud.githubusercontent.com/assets/8110458/25040847/e772e024-2146-11e7-9ded-322c589b424b.png) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? Yes, I did. Please review [this document](http://localhost:4000/install/upgrade.html#upgrading-from-zeppelin-07-to-08) if you run document development mode (localhost:4000) Author: soralee <sora0...@zepl.com> Closes #2248 from soralee/ZEPPELIN-2396_ctrl_l and squashes the following commits: bb0f7d8 [soralee] modify document d1f966a [soralee] modify document 928f179 [soralee] add space between + and who is providing 5f30fe4 [soralee] upgrade eliminate keyboard shortcut feature and add document 4e1b846 [soralee] ZEPPELIN-2396 drop the Ctrl+L shortcut keyboard commit 4d398ef2a6471614cebd6b0177a08333114f5802 Author: Tinkoff DWH <tinkoff....@gmail.com> Date: 2017-04-03T15:53:02Z [ZEPPELIN-2297] improvements to jdbc autocompleter ### What is this PR for? PR contains some improvements for completion (JDBC Interpreter): - types of completion - display of long values - refactoring of search of completions - uniqness of completions with type `keyword` - updating data in completer by pressing `Ctrl + .` - setting the schema filter to generate completions - fix highlighting code when used not default data source ### What type of PR is it? Improvement ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-2297 ### How should this be tested? try to work with new completer ### Screenshots (if appropriate) **1. Types of completion** ![1](https://cloud.githubusercontent.com/assets/25951039/24449367/758eeeac-1490-11e7-863f-bf1b313a3f4d.png) **2. Display of long values** before ![2297_before_long_caption](https://cloud.githubusercontent.com/assets/25951039/24449397/8ecd3072-1490-11e7-8fd4-415424ef337e.gif) after ![2297_after_long_caption](https://cloud.githubusercontent.com/assets/25951039/24449413/9c7a36b6-1490-11e7-9d7c-cbbdac71cbe7.gif) **3. Refactoring of search of completions. Updating data in completer by pressing `Ctrl + .`** before ![2297_before_refactoring_search](https://cloud.githubusercontent.com/assets/25951039/24449463/c1801214-1490-11e7-84a8-25c887b68d65.gif) after ![2297_after_refactoring_search](https://cloud.githubusercontent.com/assets/25951039/24449567/1079bdc0-1491-11e7-8409-5187aeceb428.gif) **4. uniqness of completions with type keyword** before ![2297_before_uniq](https://cloud.githubusercontent.com/assets/25951039/24449615/4e20c8d0-1491-11e7-94cc-c86aab886c53.gif) after ![2297_after_uniq](https://cloud.githubusercontent.com/assets/25951039/24449635/5cf59aca-1491-11e7-8ee1-31ea3cdacb3e.gif) **5. fix highlighting code when used not default data source** before ![2297_before_inrpret_name](https://cloud.githubusercontent.com/assets/25951039/24449730/b6c8d62a-1491-11e7-8dc3-39fa6975c8c3.gif) after ![2297_after_inrpret_name](https://cloud.githubusercontent.com/assets/25951039/24449738/baf63e18-1491-11e7-8711-12557a674212.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Tinkoff DWH <tinkoff....@gmail.com> Closes #2203 from tinkoff-dwh/ZEPPELIN-2297 and squashes the following commits: b86b57a [Tinkoff DWH] [ZEPPELIN-2297] small fix to compute caption 8552049 [Tinkoff DWH] [ZEPPELIN-2297] schema filters 5308f1e [Tinkoff DWH] [ZEPPELIN-2297] updating completions ef6c9cb [Tinkoff DWH] Merge remote-tracking branch 'origin/ZEPPELIN-2297' into ZEPPELIN-2297 1e05a68 [Tinkoff DWH] [ZEPPELIN-2297] fix uniqueness keywords ec3cd3b [Tinkoff DWH] [ZEPPELIN-2297] fix uniqueness keywords 2b58cc5 [Tinkoff DWH] [ZEPPELIN-2297] refactoring search completions 7b5835d [Tinkoff DWH] [ZEPPELIN-2297] compute caption of copletion 1c74384 [Tinkoff DWH] [ZEPPELIN-2297] add type of completion commit be20201236758dbba256f6480c5c2f8b91a86ae9 Author: Jun Kim <i2r....@gmail.com> Date: 2017-04-10T10:07:40Z [DOC] Improve documents related to Helium ### What is this PR for? What I did for the documents: * Highlight codes * Follow JSON syntax * Remove white spaces And in my opinion, [here](https://zeppelin.apache.org/docs/0.8.0-SNAPSHOT/development/writingzeppelinvisualization.html#1-create-a-npm-package) is ambiguous: > "Normally, you can add any dependencies in package.json however Zeppelin Visualization package only allows two dependencies: zeppelin-vis and zeppelin-tabledata." Does it want to say "you can add any dependencies in package.json, but you must include two dependencies: zeppelin-vis and zeppelin-tabledata."? ### What type of PR is it? [Documentation] ### Questions: * Does the licenses files need update? NO * Is there breaking changes for older versions? NO * Does this needs documentation? NO Author: Jun Kim <i2r....@gmail.com> Closes #2236 from tae-jun/helium-doc and squashes the following commits: 63505e9 [Jun Kim] Fix ambiguous sentence 3e775cf [Jun Kim] Highlight codes, follow JSON syntax, and remove white spaces commit f32651e519a27d755cdbed5f008d6d5f242695a2 Author: 1ambda <1am...@gmail.com> Date: 2017-04-17T06:24:20Z [HOTFIX] Invalid method signature in GroovyInterpreter ### What is this PR for? - https://github.com/apache/zeppelin/pull/2203 was merged - but https://github.com/apache/zeppelin/pull/2203 was not rebased recently - as a result, `GroovyInterperter` has invalid method signature. ``` [ERROR] COMPILATION ERROR : [ERROR] /home/travis/build/1ambda/zeppelin/groovy/src/main/java/org/apache/zeppelin/groovy/GroovyInterpreter.java:[123,3] method does not override or implement a method from a supertype [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project zeppelin-groovy: Compilation failure [ERROR] /home/travis/build/1ambda/zeppelin/groovy/src/main/java/org/apache/zeppelin/groovy/GroovyInterpreter.java:[123,3] method does not override or implement a method from a supertype ``` You can see CI failures in - https://travis-ci.org/1ambda/zeppelin/jobs/222709988 - https://travis-ci.org/1ambda/zeppelin/jobs/222709989 ### What type of PR is it? [Hot Fix] ### Todos NONE ### What is the Jira issue? Hotfix ### How should this be tested? - CI should be green. - Build this PR locally, `mvn clean package -DskipTests;` ### Screenshots (if appropriate) NONE ### Questions: * Does the licenses files need update? - NO * Is there breaking changes for older versions? - NO * Does this needs documentation? - NO Author: 1ambda <1am...@gmail.com> Closes #2255 from 1ambda/HOTFIX/GroovyInterpreter-has-invalid-overrided-method and squashes the following commits: 29e840f [1ambda] fix: Method signature in GroovyInterpreter commit 33663941c5ca544b0922d7093c8a118d8c3fce3f Author: 1ambda <1am...@gmail.com> Date: 2017-04-17T08:08:41Z [ZEPPELIN-2315] (bug) new note.json is overwritten by old note.json (master, branch-0.7) ### What is this PR for? - `note.json` is converted every time - as a result, changes in `note.json` is overwritten by old `note.json` Occurs in **0.8.0-SNAPSHOT** and **branch-0.7** ### What type of PR is it? [Bug Fix] ### Todos NONE ### What is the Jira issue? [ZEPPELIN-2315](https://issues.apache.org/jira/browse/ZEPPELIN-2315) ### How should this be tested? 1. create a note in 0.6.0 which including graph 2. migrate it to 0.7.0+ (just copy dir) 3. open in Zeppelin 4. click other graph type rather than table (e.g `scatter chart`) 5. restart 7. should see the last change (graph) is persisted ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? - NO * Is there breaking changes for older versions? - NO * Does this needs documentation? - NO Author: 1ambda <1am...@gmail.com> Closes #2256 from 1ambda/ZEPPELIN-2315/should-convert-old-notebook-only-once and squashes the following commits: 4e77c0a8b [1ambda] fix: Convert old notebook format only once commit a7ffc1291885cc555ed84bd71f455ebcb2e9dd62 Author: Jeff Zhang <zjf...@apache.org> Date: 2017-04-11T10:09:35Z ZEPPELIN-2390. Improve returnType for z.checkbox ### What is this PR for? Currently it is not convenient to access the individual item of the return value of z.checkbox, I would propose to return `Seq` for `SparkInterpreter` and `list` for `PySparkInterpreter`. This might cause some incompatibility, but I think it is acceptable considering the benefits. Besides that, before this PR, all the items of checkbox would be checked by default in `PySparkInterpreter` which is inconsistent with `SparkInterpreter`, so I change it to nothing is selected by default in this PR. ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2390 ### How should this be tested? Unit test is added ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjf...@apache.org> Closes #2241 from zjffdu/ZEPPELIN-2390 and squashes the following commits: 75e3afc [Jeff Zhang] ZEPPELIN-2390. Improve returnType for z.checkbox commit c706d453e06a2f8c71fcd61d7e4dddc87cafc0f4 Author: Jeff Zhang <zjf...@apache.org> Date: 2017-04-11T06:21:53Z ZEPPELIN-2386. Add parameter for check running current paragraph in ZeppelinContext ### What is this PR for? I can reproduce it via the test code in ZeppelinIT.testAngularDisplay. First run the 4 paragraphs to initiate the state. Then when I click paragraph 0 the first time, paragraph 2 will run correctly. But if I click paragraph 0 again, interpreter would raise the following exception. The cause is that ZeppelinContext's interpreterContext now point to paragraph 2, because paragraph 2 is the last paragraph that runs in this interpreter. So I propose to add parameter for check running current paragraph in ZeppelinContext. * Add 2 methods in ZeppelinContext. `run(idx, checkCurrentParagraph) `, 'run(noteId, idx, context, checkCurrentParagraph)' * Also make some changes for debug logging. ### What type of PR is it? [Bug Fix | Improvement] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2386 ### How should this be tested? Tested manually and also add integration test. ### Screenshots (if appropriate) Before ![zeppelin_before](https://cloud.githubusercontent.com/assets/164491/24895529/8514826e-1ec3-11e7-915b-70baf09c297a.gif) After ![zeppelin_after](https://cloud.githubusercontent.com/assets/164491/24895532/88e87bc0-1ec3-11e7-90ee-a7ca7a3758f8.gif) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjf...@apache.org> Closes #2240 from zjffdu/ZEPPELIN-2386 and squashes the following commits: 03995a3 [Jeff Zhang] ZEPPELIN-2386. Add parameter to check running current paragraph in ZeppelinContext commit 4d6485737cf72bc156c65c419b8e6e6cc907f3ea Author: soralee <sora0...@zepl.com> Date: 2017-03-14T05:07:44Z [ZEPPELIN-1357,1892,1370][Umbrella] Text overlap in the MultiBarChart ### What is this PR for? If using the `multiChartBar`, it often occurs overlap in text. So, this PR is for preventing to overlap in text. >~~The below is the case what I updated.~~ ~~1. The xLabel generally show all of the text without rotation (less than 30).~~ ~~2. If the xLabel size is over than 30 and less than 80, the xLabel text are rotated as 90 degree and displayed.~~ ~~3. If the xLabel size is over than 80, the xLabel text are disappeared, but the tooltip is displayed.~~ >~~I have made improvements based on my thinking, so I would appreciate to give me your feedback.~~ ~~And if merged this PR, I'll update the feature so that user can use `min` and `max` variables by user on web.~~ **[Update]** To prevent overlap in xLabel text, in one way which is talked the below mentions, user who is using zeppelin could select to visualization type of xLabel such as `rotate 45 degree` or `hide`. ### What type of PR is it? [Bug Fix | Improvement ] ### What is the Jira issue? * [ZEPPELIN-1357; UI - label wrapping not done properly in charts](https://issues.apache.org/jira/browse/ZEPPELIN-1357) * [ZEPPELIN-1892; Display label vertically or horizontally smartly](https://issues.apache.org/jira/browse/ZEPPELIN-1892) * [ZEPPELIN-1370; Label overlaps - in default visualization example barchart](https://issues.apache.org/jira/browse/ZEPPELIN-1370) ### How should this be tested? 1. Run the paragraph for bank data in the `Basic Features (Spark)` notebook. 2. Execute the following query. ``` %sql select * from bank ``` 3. Use the pivot in the `Setting` toggle in the `MultiBarChart` such as the screenshots. ### Screenshots (if appropriate) **[Before]** ![z_zeppelin-1357_b](https://cloud.githubusercontent.com/assets/8110458/23898303/f80d7bbc-08f3-11e7-9b24-3248c492b8af.png) ![z_1357_b3_](https://cloud.githubusercontent.com/assets/8110458/23898435/75b3ae60-08f4-11e7-9d7c-31746f0d4edc.png) **[After]** ![peek 2017-04-17 14-58](https://cloud.githubusercontent.com/assets/8110458/25080820/a3bfc1c0-2381-11e7-9c2b-16c0aa71234e.gif) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: soralee <sora0...@zepl.com> Closes #2133 from soralee/ZEPPELIN-1357_overlap_text and squashes the following commits: 32d7e37 [soralee] rebase master commit c3fbb1ef664e7a54a8bc52eee435b91be8a6e16d Author: soralee <sora0...@zepl.com> Date: 2017-04-11T00:43:03Z [MINOR] fix typo from 'interpeter' to 'interpreter' ### What is this PR for? Fix typo from `interpeter` to `interpreter` ### What type of PR is it? [Improvement] ### What is the Jira issue? * No, it's minor issue ### How should this be tested? - N/A ### Screenshots (if appropriate) - N/A ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: soralee <sora0...@zepl.com> Closes #2244 from soralee/typo_interpreter and squashes the following commits: 863dc4f [soralee] fix typo commit 4b1b521fc34fd3c841e38ab591af0bc7df21a64e Author: 1ambda <1am...@gmail.com> Date: 2017-04-15T08:18:56Z [ZEPPELIN-2179] Clear output DOES NOT work in personalized mode (master) ### What is this PR for? `clear output` (`cmd` + `opt` + `L`) doesn't work in the personalized mode. ### What type of PR is it? [Bug Fix] ### Todos NONE ### What is the Jira issue? [ZEPPELIN-2179](https://issues.apache.org/jira/browse/ZEPPELIN-2179) ### How should this be tested? 1. Configure shiro 2. Start Zeppelin and login in 2 browsers (e.g chrome and safari) with 2 different users (**DO NOT USE `dev` mode: localhost:9000**) 3. Create a note 4. Enable personalized mode 5. Run paragraph and clear output in each browser ### Screenshots (if appropriate) NONE ### Questions: * Does the licenses files need update? - NO * Is there breaking changes for older versions? - NO * Does this needs documentation? - NO Author: 1ambda <1am...@gmail.com> Closes #2253 from 1ambda/ZEPPELIN-2179/clear-output-doesnt-work-in-person-mode-for-master and squashes the following commits: eeff440ec [1ambda] fix: Clear personalized output before running b7387849f [1ambda] fix: Clear output in personalized paragraph commit 7b585c7399f29492c308f83e342cac29b0c7ca07 Author: Jeff Zhang <zjf...@apache.org> Date: 2017-04-12T02:50:16Z ZEPPELIN-2395. Refactor Input.java to make dynamic forms extensible ### What is this PR for? Currently, zeppelin only support 3 kinds of dynamic form controls: TextBox, Select, CheckBox. All the things are in `Input.java`, this is hard to add new controls, this PR is for refactoring Input to make dynamic forms extensible. Main Changes: * Make `Input` as the base class of dynamic forms also use it as the factory class * All the concret dynamic forms extend `Input` * Add method `toJson` and `fromJson` for `GUI` for `GUI`'s serialization/deserialization. I plan to do it for other classes as well, so that we can remove duplicated serde code and also make it easy to test serialization/deserialization * Change `z.input` to `z.textbox` as I think z.input is a little misleading. But I still keep `z.input` and make `z.input` as deprecated. * Ideally the new input forms' json should be the same as the old input form json. But there's one bug in the old input form, `type` is missing if the input forms are created in frontend for textbox and select. So I keep the old input forms for compatibility. I will load the old input forms json and convert it into new input forms, and after saving, `note.json` would have the new input forms json. After this PR, user needs to do 3 things to add new ui controls * Implement its UI control classes, (refer TextBox/CheckBox/Select), and specify it in `TypeAdapterFactory` of `Input` for serde. * Add parsing logic in `Input.getInputForm` if you want to support this control in frontend. * Add display logic in `paragraph-parameterizedQueryForm.html` ### What type of PR is it? [ Improvement | Refactoring] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2395 ### How should this be tested? Test is added ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? Yes * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjf...@apache.org> Closes #2245 from zjffdu/ZEPPELIN-2395 and squashes the following commits: 16d42a8 [Jeff Zhang] ZEPPELIN-2395. Refactor Input.java to make dynamic forms extensible commit 722ed823193ed49d4e42ef17be2241cf6d48cd97 Author: Lee moon soo <m...@apache.org> Date: 2017-04-19T01:33:00Z Prevent NPE on delete local module cache ### What is this PR for? Prevent NPE on delete local module cache in HeliumBundleFactory ``` Caused by: java.lang.NullPointerException at org.apache.zeppelin.helium.HeliumBundleFactory.deleteYarnCache(HeliumBundleFactory.java:465) at org.apache.zeppelin.helium.HeliumBundleFactory.copyFrameworkModulesToInstallPath(HeliumBundleFactory.java:487) at org.apache.zeppelin.helium.HeliumBundleFactory.buildPackage(HeliumBundleFactory.java:403) at org.apache.zeppelin.helium.Helium.enable(Helium.java:314) at org.apache.zeppelin.rest.HeliumRestApi.enablePackage(HeliumRestApi.java:193) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:180) at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96) ... 50 more ``` ### What type of PR is it? Bug Fix ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Lee moon soo <m...@apache.org> Closes #2259 from Leemoonsoo/prevent_npe_helium_bundle_factory and squashes the following commits: 0934af8 [Lee moon soo] Prevent NPE on delete local module cache ---- ---