GitHub user 1ambda reopened a pull request: https://github.com/apache/zeppelin/pull/2098
[ZEPPELIN-2217] AdvancedTransformation for Visualization ### What is this PR for? `AdvancedTransformation` has more detailed options while providing existing features of `PivotTransformation` and `ColumnselectorTransformation` which Zeppelin already has ![av_in_30sec](https://cloud.githubusercontent.com/assets/4968473/24037330/c9478e86-0b40-11e7-9886-1ffb85042a7a.gif) Here are some features which advanced-transformation can provide. 1. **(screenshot)** multiple sub charts 2. **(screenshot)** parameter widgets: `input`, `checkbox`, `option`, `textarea` 3. **(screenshot)** expand/fold axis and parameter panels 4. **(screenshot)** clear axis and parameter panels 5. **(screenshot)** remove duplicated columns in an axis 6. **(screenshot)** limit column count in an axis 7. configurable char axes: `valueType`, `axisType`, `description`, ... 8. configurable chart parameters 9. lazy transformation 10. parsing parameters automatically based on their type: `int`, `float`, `string`, `JSON` 11. multiple transformation methods 12. re-initialize whole configuration based on spec hash. 13. **(screenshot)** shared axis #### API Details: Spec `AdvancedTransformation` requires `spec` which includes axis and parameter details for charts. - Let's create 2 sub-charts called `simple-line` and `step-line`. - Each sub chart can have different `axis` and `parameter` depending on their requirements. ```js constructor(targetEl, config) { super(targetEl, config) const spec = { charts: { 'simple-line': { sharedAxis: true, /** set if you want to share axes between sub charts, default is `false` */ axis: { 'xAxis': { dimension: 'multiple', axisType: 'key', }, 'yAxis': { dimension: 'multiple', axisType: 'aggregator'}, 'category': { dimension: 'multiple', axisType: 'group', }, }, parameter: { 'xAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of xAxis', }, 'yAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of yAxis', }, 'dashLength': { valueType: 'int', defaultValue: 0, description: 'the length of dash', }, }, }, 'step-line': { axis: { 'xAxis': { dimension: 'single', axisType: 'unique', }, 'yAxis': { dimension: 'multiple', axisType: 'value', }, }, parameter: { 'xAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of xAxis', }, 'yAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of yAxis', }, 'noStepRisers': { valueType: 'boolean', defaultValue: false, description: 'no risers in step line', widget: 'checkbox', }, }, }, } this.transformation = new AdvancedTransformation(config, spec) } ``` #### API Details: Axis Spec | Field Name | Available Values (type) | Description | | --- | --- | --- | |`dimension` | `single` | Axis can contains only 1 column | |`dimension` | `multiple` | Axis can contains multiple columns | |`axisType` | `key` | Column(s) in this axis will be used as `key` like in `PivotTransformation`. These columns will be served in `column.key` | |`axisType` | `aggregator` | Column(s) in this axis will be used as `value` like in `PivotTransformation`. These columns will be served in `column.aggregator` | |`axisType` | `group` | Column(s) in this axis will be used as `group` like in `PivotTransformation`. These columns will be served in `column.group` | |`axisType` | (string) | Any string value can be used here. These columns will be served in `column.custom` | |`maxAxisCount` | (int) | The maximum column count that this axis can contains. (unlimited if `undefined`) | |`valueType` | (string) | Describe the value type just for annotation | Here is an example. ```js axis: { 'xAxis': { dimension: 'multiple', axisType: 'key', }, 'yAxis': { dimension: 'multiple', axisType: 'aggregator'}, 'category': { dimension: 'multiple', axisType: 'group', maxAxisCount: 2, valueType: 'string', }, }, ``` #### API Details: Parameter Spec | Field Name | Available Values (type) | Description | | --- | --- | --- | |`valueType` | `string` | Parameter which has string value | |`valueType` | `int` | Parameter which has int value | |`valueType` | `float` | Parameter which has float value | |`valueType` | `boolean` | Parameter which has boolean value used with `checkbox` widget usually | |`valueType` | `JSON` | Parameter which has JSON value used with `textarea` widget usually. `defaultValue` should be `""` (empty string). This ||`defaultValue` | (any) | Default value of this parameter. `JSON` type should have `""` (empty string) | |`description` | (string) | Description of this parameter. This value will be parsed as HTML for pretty output | |`widget` | `input` | Use [input](https://developer.mozilla.org/en/docs/Web/HTML/Element/input) widget. This is the default widget (if `widget` is undefined)| |`widget` | `checkbox` | Use [checkbox](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/checkbox) widget. | |`widget` | `textarea` | Use [textarea](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/textarea) widget. | |`widget` | `option` | Use [select + option](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/select) widget. This parameter should have `optionValues` field as well. | |`optionValues` | (Array<string>) | Available option values used with the `option` widget | Here is an example. ```js parameter: { // string type, input widget 'xAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of xAxis', }, // boolean type, checkbox widget 'inverted': { widget: 'checkbox', valueType: 'boolean', defaultValue: false, description: 'invert x and y axes', }, // string type, option widget with `optionValues` 'graphType': { widget: 'option', valueType: 'string', defaultValue: 'line', description: 'graph type', optionValues: [ 'line', 'smoothedLine', 'step', ], }, // HTML in `description` 'dateFormat': { valueType: 'string', defaultValue: '', description: 'format of date (<a href="https://docs.amcharts.com/3/javascriptcharts/AmGraph#dateFormat">doc</a>) (e.g YYYY-MM-DD)', }, // JSON type, textarea widget 'yAxisGuides': { widget: 'textarea', valueType: 'JSON', defaultValue: '', description: 'guides of yAxis ', }, ``` #### API Details: Transformer Spec `AdvancedTransformation` supports 3 transformation methods. The return value will depend on the transformation method type. ```js const spec = { charts: { 'simple': { /** default value of `transform.method` is the flatten cube. */ axis: { ... }, parameter: { ... } }, 'cube-group': { transform: { method: 'cube', }, axis: { ... }, parameter: { ... }, } 'no-group': { transform: { method: 'raw', }, axis: { ... }, parameter: { ... }, } ``` | Field Name | Available Values (type) | Description | | --- | --- | --- | |`method` | `object` | designed for [amcharts: serial](https://www.amcharts.com/demos/date-based-data/) | |`method` | `array` | designed for [highcharts: column](http://jsfiddle.net/gh/get/library/pure/highcharts/highcharts/tree/master/samples/highcharts/demo/column-basic/) | |`method` | `drill-down` | designed for [highcharts: drill-down](http://jsfiddle.net/gh/get/library/pure/highcharts/highcharts/tree/master/samples/highcharts/demo/column-drilldown/) | |`method` | `raw` | will return the original `tableData.rows` | Whatever you specified as `transform.method`, the `transformer` value will be always function for lazy computation. ```js // advanced-transformation.util#getTransformer if (transformSpec.method === 'raw') { transformer = () => { return rows; } } else if (transformSpec.method === 'array') { transformer = () => { ... return { ... } } } ``` #### Feature Details: Automatic parameter parsing Advanced transformation will parse parameter values automatically based on their type: `int`, `float`, `string`, `JSON` - See also `advanced-transformation-util.js#parseParameter` #### Feature Details: re-initialize the whole configuration based on spec hash ```js // advanced-transformation-util#initializeConfig const currentVersion = JSON.stringify(spec) if (!config.spec || !config.spec.version || config.spec.version !== currentVersion) { spec.version = currentVersion // reset config... } ``` #### Feature Details: Shared Axes If you set `sharedAxis` to `true` in chart specification, then these charts will share their axes. (default is `false`) ```js const spec = { charts: { 'column': { transform: { method: 'array', }, sharedAxis: true, axis: { ... }, parameter: { ... }, }, 'stacked': { transform: { method: 'array', }, sharedAxis: true, axis: { ... } parameter: { ... }, }, ``` ![sharedaxis](https://cloud.githubusercontent.com/assets/4968473/24207116/6999ad8a-0f63-11e7-8b61-273b712612fc.gif) #### API Details: Usage in Visualization#render() Let's assume that we want to create 2 sub-charts called `basic` and `no-group`. - https://github.com/1ambda/zeppelin-ultimate-line-chart (an practical example) ```js drawBasicChart(parameter, column, transformer) { const { ... } = transformer() } drawNoGroupChart(parameter, column, transformer) { const { ... } = transformer() } render(data) { const { chart, parameter, column, transformer, } = data if (chart === 'basic') { this.drawBasicChart(parameter, column, transformer) } else if (chart === 'no-group') { this.drawNoGroupChart(parameter, column, transformer) } } ``` ### What type of PR is it? [Feature] ### Todos NONE ### What is the Jira issue? [ZEPPELIN-2217](https://issues.apache.org/jira/browse/ZEPPELIN-2217) ### How should this be tested? 1. Clone https://github.com/1ambda/zeppelin-ultimate-line-chart 2. Create a symbolic link `ultimate-line-chart.json` into `$ZEPPELIN_HOME/helium` 3. Modify the `artifact` value to proper absolute path considering your local machine. 4. Install the above visualization in `localhost:9000/#helium` 5. Test it ### Screenshots (if appropriate) #### 1. *(screenshot)* multiple sub charts ![av_multiple_charts](https://cloud.githubusercontent.com/assets/4968473/24034638/7b84dba0-0b35-11e7-989d-059ccc87f968.gif) #### 2. *(screenshot)* parameter widgets: `input`, `checkbox`, `option`, `textarea` ![av_widgets_new](https://cloud.githubusercontent.com/assets/4968473/24034652/88679d6c-0b35-11e7-835a-3970d7124850.gif) #### 3. *(screenshot)* expand/fold axis and parameter panels ![av_fold_expand](https://cloud.githubusercontent.com/assets/4968473/24034653/8a634ddc-0b35-11e7-9851-15280a6b5fd3.gif) #### 4. *(screenshot)* clear axis and parameter panels ![av_clean_buttons](https://cloud.githubusercontent.com/assets/4968473/24034654/8d3dc14a-0b35-11e7-98c7-3aeddce6d80a.gif) #### 5. *(screenshot)* remove duplicated columns in an axis ![av_duplicated_columns](https://cloud.githubusercontent.com/assets/4968473/24034657/910f4d20-0b35-11e7-9e9b-d9e2f799a5dd.gif) #### 6. *(screenshot)* limit column count in an axis ![av_maxaxiscount](https://cloud.githubusercontent.com/assets/4968473/24034679/a5e8eb34-0b35-11e7-89cd-070f3790d511.gif) ### Questions: * Does the licenses files need update? - NO * Is there breaking changes for older versions? - NO * Does this needs documentation? - NO You can merge this pull request into a Git repository by running: $ git pull https://github.com/1ambda/zeppelin ZEPPELIN-2217/advanced-transformation Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/2098.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2098 ---- commit 69e0757a2871cf6a2a93a0129a7a83c4339f41b3 Author: 1ambda <1am...@gmail.com> Date: 2017-03-06T05:39:48Z feat: AdvancedTransformation commit 872cac3bbfd694acb7a1a13ae4cfa32cf34272e1 Author: 1ambda <1am...@gmail.com> Date: 2017-03-11T20:48:35Z fix: Overflow issue in single aggr commit dce24e233ce5bc96ba926df6545dd97b2e4cdad2 Author: 1ambda <1am...@gmail.com> Date: 2017-03-12T07:46:59Z feat: Add groupBase axis option commit 9d4867d3440763257a73333dc461690467c9cf72 Author: 1ambda <1am...@gmail.com> Date: 2017-03-12T07:55:12Z refactor: Add getAxisInSingleDimension func commit 727e2d5bc877a4864c904b414de00de4122a67c1 Author: 1ambda <1am...@gmail.com> Date: 2017-03-12T08:16:41Z feat: Add clearConfig commit a2e2004d32bdc6237e6b79f755dc388824ac0204 Author: 1ambda <1am...@gmail.com> Date: 2017-03-12T08:59:28Z refactor: Refine transform API commit e18424177792f54c91c0afe25a6450cf41071f25 Author: 1ambda <1am...@gmail.com> Date: 2017-03-12T09:06:40Z fix: Add singleDimensionAggregatorChanged commit 45866d0c7197d68203f0180b2d48ea357ef78f0b Author: 1ambda <1am...@gmail.com> Date: 2017-03-13T09:26:40Z feat: Support multiple charts commit e09c06c87aa3fb055ae022ffa379d2fa191713e6 Author: 1ambda <1am...@gmail.com> Date: 2017-03-13T11:21:40Z feat: Support multiple charts in UI commit 61f8d28a0443f0fb4cdf45eb2261de0e068ad844 Author: 1ambda <1am...@gmail.com> Date: 2017-03-13T12:15:38Z feat: advanced-transformation-api commit 300942d51612b7e1046301a77a09c2128eb1c855 Author: 1ambda <1am...@gmail.com> Date: 2017-03-14T15:49:23Z refactor: Refine axis, param spec commit 2bac0084f729b8261b73b2dff511c8a9993848c3 Author: 1ambda <1am...@gmail.com> Date: 2017-03-14T16:17:33Z feat: Support same axis types commit 63952ecfb7f9546574db2c291e96350b309c6a0d Author: 1ambda <1am...@gmail.com> Date: 2017-03-15T12:03:10Z feat: Cube Transformation commit 334ef190f8ea6ede00a9d2b5f61e8ba22ebc3c1c Author: 1ambda <1am...@gmail.com> Date: 2017-03-15T12:48:23Z feat: Support undefined valueType in axisSpec commit 16a22b29728c5d694780b1fa462f7dedd277014b Author: 1ambda <1am...@gmail.com> Date: 2017-03-15T13:15:59Z feat: Support maxAxisCount in axisSpec commit 7b1c99cffcbce501f53bfcde73e9bf2a1a3be47f Author: 1ambda <1am...@gmail.com> Date: 2017-03-15T15:44:04Z feat: Support transform: flatten commit 03de88c00c323dce408efcc87f1e2895d98d9864 Author: 1ambda <1am...@gmail.com> Date: 2017-03-15T15:48:24Z fix: Modify margin of subchart radio btns commit 72a644effecf6dc033be2df16e32a0548118a422 Author: 1ambda <1am...@gmail.com> Date: 2017-03-15T15:52:35Z fix: Duplicated radio btn id, name commit da783eb7b9ea7ea554e4021d57699ad5a2c35d38 Author: 1ambda <1am...@gmail.com> Date: 2017-03-15T16:35:26Z feat: Automatic spec versioning commit e62a8635883c600f7c314a6a11107a2c98b4fafa Author: 1ambda <1am...@gmail.com> Date: 2017-03-15T17:11:17Z feat: Support transformer commit 43e66a6471798da1756346ea34b617f287e531ba Author: 1ambda <1am...@gmail.com> Date: 2017-03-15T18:24:20Z feat: custom axisSpec commit 5663d530cafbc22a1c77d87fa51cd5a435db0d89 Author: 1ambda <1am...@gmail.com> Date: 2017-03-15T18:35:23Z feat: Save and close with enter key commit 5b6bea9a881715abc14e2facef14a5918c3d55dc Author: 1ambda <1am...@gmail.com> Date: 2017-03-15T18:45:55Z fix: Change panel header commit d2f3fbaba69af4c9f00bdeff0005bfa9d240f400 Author: 1ambda <1am...@gmail.com> Date: 2017-03-16T10:26:38Z feat: option widget commit 74b8df099b21c53633cce3f3d12c40837eab5e25 Author: 1ambda <1am...@gmail.com> Date: 2017-03-16T10:30:15Z feat: widget checkbox commit d531b3eedc7489183dd5fe75835298db8c22e6dd Author: 1ambda <1am...@gmail.com> Date: 2017-03-16T10:39:31Z feat: textare widget and update hook commit 101e9270189847cd177885304c6fa11df8dc942e Author: 1ambda <1am...@gmail.com> Date: 2017-03-16T15:32:46Z refactor: Remove util and add Widget funcs commit 1b1417843f648c576cc89634ff609bd4ce0b45e0 Author: 1ambda <1am...@gmail.com> Date: 2017-03-16T18:03:10Z fix: Dont close param panel when enter commit f6a1d834bd9be7277267c3e37368e069c4201d77 Author: 1ambda <1am...@gmail.com> Date: 2017-03-16T18:54:57Z feat: Automatic param parsing commit 4407b91c063d7247aa2321e458e0a042fbf7a61f Author: 1ambda <1am...@gmail.com> Date: 2017-03-16T19:11:35Z fix: Stop event propagation in widgets ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---