[jira] [Commented] (KYLIN-4972) Don't allow segment merge when use dict.global.SegmentAppendTrieDictBuilder to build bitmap measure
[ https://issues.apache.org/jira/browse/KYLIN-4972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320709#comment-17320709 ] ASF GitHub Bot commented on KYLIN-4972: --- coveralls commented on pull request #1634: URL: https://github.com/apache/kylin/pull/1634#issuecomment-819240012 ## Pull Request Test Coverage Report for [Build 7071](https://coveralls.io/builds/38794473) * **0** of **21** **(0.0%)** changed or added relevant lines in **2** files are covered. * **16** unchanged lines in **4** files lost coverage. * Overall coverage decreased (**-0.01%**) to **28.082%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-cube/src/main/java/org/apache/kylin/cube/model/CubeDesc.java](https://coveralls.io/builds/38794473/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fmodel%2FCubeDesc.java#L1635) | 0 | 7 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/service/CubeService.java](https://coveralls.io/builds/38794473/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FCubeService.java#L826) | 0 | 14 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/38794473/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62) | 1 | 81.08% | | [server-base/src/main/java/org/apache/kylin/rest/util/QueryRequestLimits.java](https://coveralls.io/builds/38794473/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Futil%2FQueryRequestLimits.java#L72) | 1 | 47.62% | | [core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/MetricsSystem.java](https://coveralls.io/builds/38794473/source?filename=core-metrics%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetrics%2Flib%2Fimpl%2FMetricsSystem.java#L46) | 5 | 66.67% | | [core-cube/src/main/java/org/apache/kylin/cube/model/CubeDesc.java](https://coveralls.io/builds/38794473/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fmodel%2FCubeDesc.java#L1649) | 9 | 62.13% | | Totals | [](https://coveralls.io/builds/38794473) | | :-- | --: | | Change from base [Build 7067](https://coveralls.io/builds/38780467): | -0.01% | | Covered Lines: | 26709 | | Relevant Lines: | 95111 | --- # - [Coveralls](https://coveralls.io) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Don't allow segment merge when use dict.global.SegmentAppendTrieDictBuilder > to build bitmap measure > --- > > Key: KYLIN-4972 > URL: https://issues.apache.org/jira/browse/KYLIN-4972 > Project: Kylin > Issue Type: Improvement >Reporter: JiangYang >Assignee: JiangYang >Priority: Minor > Fix For: v3.1.2 > > > When *org.apache.kylin.dict.global.SegmentAppendTrieDictBuilder* is used for > bitmap count distinct measure, cube segment merge will lead to incorrect > bitmap merge. Therefore, we should forbid segment merge in this case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] coveralls commented on pull request #1634: [KYLIN-4972] Don't allow segment merge when use dict.global.SegmentAp…
coveralls commented on pull request #1634: URL: https://github.com/apache/kylin/pull/1634#issuecomment-819240012 ## Pull Request Test Coverage Report for [Build 7071](https://coveralls.io/builds/38794473) * **0** of **21** **(0.0%)** changed or added relevant lines in **2** files are covered. * **16** unchanged lines in **4** files lost coverage. * Overall coverage decreased (**-0.01%**) to **28.082%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-cube/src/main/java/org/apache/kylin/cube/model/CubeDesc.java](https://coveralls.io/builds/38794473/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fmodel%2FCubeDesc.java#L1635) | 0 | 7 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/service/CubeService.java](https://coveralls.io/builds/38794473/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FCubeService.java#L826) | 0 | 14 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/38794473/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62) | 1 | 81.08% | | [server-base/src/main/java/org/apache/kylin/rest/util/QueryRequestLimits.java](https://coveralls.io/builds/38794473/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Futil%2FQueryRequestLimits.java#L72) | 1 | 47.62% | | [core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/MetricsSystem.java](https://coveralls.io/builds/38794473/source?filename=core-metrics%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetrics%2Flib%2Fimpl%2FMetricsSystem.java#L46) | 5 | 66.67% | | [core-cube/src/main/java/org/apache/kylin/cube/model/CubeDesc.java](https://coveralls.io/builds/38794473/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fmodel%2FCubeDesc.java#L1649) | 9 | 62.13% | | Totals | [](https://coveralls.io/builds/38794473) | | :-- | --: | | Change from base [Build 7067](https://coveralls.io/builds/38780467): | -0.01% | | Covered Lines: | 26709 | | Relevant Lines: | 95111 | --- # - [Coveralls](https://coveralls.io) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4972) Don't allow segment merge when use dict.global.SegmentAppendTrieDictBuilder to build bitmap measure
[ https://issues.apache.org/jira/browse/KYLIN-4972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320706#comment-17320706 ] ASF GitHub Bot commented on KYLIN-4972: --- codecov-io commented on pull request #1634: URL: https://github.com/apache/kylin/pull/1634#issuecomment-819238405 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1634?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#1634](https://codecov.io/gh/apache/kylin/pull/1634?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (b6e5552) into [master](https://codecov.io/gh/apache/kylin/commit/3e12b6d621fe8e5c747a5783f64bc535618c8035?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (3e12b6d) will **decrease** coverage by `0.18%`. > The diff coverage is `2.22%`. [](https://codecov.io/gh/apache/kylin/pull/1634?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#1634 +/- ## - Coverage 25.51% 25.32% -0.19% + Complexity 6765 6760 -5 Files 1505 1509 +4 Lines 9361894237 +619 Branches 1311113211 +100 - Hits 2388623869 -17 - Misses6735667984 +628 - Partials 2376 2384 +8 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1634?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/1634/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==) | `12.42% <0.00%> (-0.14%)` | `51.00 <0.00> (ø)` | | | [...n/java/org/apache/kylin/common/util/ClassUtil.java](https://codecov.io/gh/apache/kylin/pull/1634/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi91dGlsL0NsYXNzVXRpbC5qYXZh) | `49.20% <0.00%> (-2.47%)` | `9.00 <0.00> (ø)` | | | [...ain/java/org/apache/kylin/cube/model/CubeDesc.java](https://codecov.io/gh/apache/kylin/pull/1634/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL21vZGVsL0N1YmVEZXNjLmphdmE=) | `56.57% <0.00%> (-1.79%)` | `182.00 <0.00> (ø)` | | | [.../java/org/apache/kylin/job/util/JobRestClient.java](https://codecov.io/gh/apache/kylin/pull/1634/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi91dGlsL0pvYlJlc3RDbGllbnQuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (?)` | | | [...che/kylin/storage/gtrecord/CubeTupleConverter.java](https://codecov.io/gh/apache/kylin/pull/1634/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1zdG9yYWdlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zdG9yYWdlL2d0cmVjb3JkL0N1YmVUdXBsZUNvbnZlcnRlci5qYXZh) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [...ylin/engine/spark/SparkBatchCubingJobBuilder2.java](https://codecov.io/gh/apache/kylin/pull/1634/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-ZW5naW5lLXNwYXJrL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvc3BhcmsvU3BhcmtCYXRjaEN1YmluZ0pvYkJ1aWxkZXIyLmphdmE=) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [.../apache/kylin/engine/spark/SparkCubingByLayer.java](https://codecov.io/gh/apache/kylin/pull/1634/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-ZW5naW5lLXNwYXJrL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvc3BhcmsvU3BhcmtDdWJpbmdCeUxheWVyLmphdmE=) | `0.00% <0.00%> (ø)` | `0.00 <0.00>
[GitHub] [kylin] codecov-io commented on pull request #1634: [KYLIN-4972] Don't allow segment merge when use dict.global.SegmentAp…
codecov-io commented on pull request #1634: URL: https://github.com/apache/kylin/pull/1634#issuecomment-819238405 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1634?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#1634](https://codecov.io/gh/apache/kylin/pull/1634?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (b6e5552) into [master](https://codecov.io/gh/apache/kylin/commit/3e12b6d621fe8e5c747a5783f64bc535618c8035?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (3e12b6d) will **decrease** coverage by `0.18%`. > The diff coverage is `2.22%`. [](https://codecov.io/gh/apache/kylin/pull/1634?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#1634 +/- ## - Coverage 25.51% 25.32% -0.19% + Complexity 6765 6760 -5 Files 1505 1509 +4 Lines 9361894237 +619 Branches 1311113211 +100 - Hits 2388623869 -17 - Misses6735667984 +628 - Partials 2376 2384 +8 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1634?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/1634/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==) | `12.42% <0.00%> (-0.14%)` | `51.00 <0.00> (ø)` | | | [...n/java/org/apache/kylin/common/util/ClassUtil.java](https://codecov.io/gh/apache/kylin/pull/1634/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi91dGlsL0NsYXNzVXRpbC5qYXZh) | `49.20% <0.00%> (-2.47%)` | `9.00 <0.00> (ø)` | | | [...ain/java/org/apache/kylin/cube/model/CubeDesc.java](https://codecov.io/gh/apache/kylin/pull/1634/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL21vZGVsL0N1YmVEZXNjLmphdmE=) | `56.57% <0.00%> (-1.79%)` | `182.00 <0.00> (ø)` | | | [.../java/org/apache/kylin/job/util/JobRestClient.java](https://codecov.io/gh/apache/kylin/pull/1634/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi91dGlsL0pvYlJlc3RDbGllbnQuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (?)` | | | [...che/kylin/storage/gtrecord/CubeTupleConverter.java](https://codecov.io/gh/apache/kylin/pull/1634/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1zdG9yYWdlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zdG9yYWdlL2d0cmVjb3JkL0N1YmVUdXBsZUNvbnZlcnRlci5qYXZh) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [...ylin/engine/spark/SparkBatchCubingJobBuilder2.java](https://codecov.io/gh/apache/kylin/pull/1634/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-ZW5naW5lLXNwYXJrL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvc3BhcmsvU3BhcmtCYXRjaEN1YmluZ0pvYkJ1aWxkZXIyLmphdmE=) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [.../apache/kylin/engine/spark/SparkCubingByLayer.java](https://codecov.io/gh/apache/kylin/pull/1634/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-ZW5naW5lLXNwYXJrL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvc3BhcmsvU3BhcmtDdWJpbmdCeUxheWVyLmphdmE=) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | |
[jira] [Commented] (KYLIN-4971) Add new measure bitmap_map for count distinct measure in UI
[ https://issues.apache.org/jira/browse/KYLIN-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320703#comment-17320703 ] ASF GitHub Bot commented on KYLIN-4971: --- coveralls commented on pull request #1633: URL: https://github.com/apache/kylin/pull/1633#issuecomment-819230152 ## Pull Request Test Coverage Report for [Build 7070](https://coveralls.io/builds/38793637) * **0** of **0** changed or added relevant lines in **0** files are covered. * **2** unchanged lines in **2** files lost coverage. * Overall coverage increased (+**0.001%**) to **28.094%** --- | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/38793637/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62) | 1 | 81.08% | | [server-base/src/main/java/org/apache/kylin/rest/util/QueryRequestLimits.java](https://coveralls.io/builds/38793637/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Futil%2FQueryRequestLimits.java#L72) | 1 | 47.62% | | Totals | [](https://coveralls.io/builds/38793637) | | :-- | --: | | Change from base [Build 7067](https://coveralls.io/builds/38780467): | 0.001% | | Covered Lines: | 26715 | | Relevant Lines: | 95091 | --- # - [Coveralls](https://coveralls.io) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add new measure bitmap_map for count distinct measure in UI > > > Key: KYLIN-4971 > URL: https://issues.apache.org/jira/browse/KYLIN-4971 > Project: Kylin > Issue Type: Task >Reporter: JiangYang >Assignee: JiangYang >Priority: Minor > Fix For: v3.1.2 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] coveralls commented on pull request #1633: [KYLIN-4971] Add new measure bitmap_map for count distinct at fronten…
coveralls commented on pull request #1633: URL: https://github.com/apache/kylin/pull/1633#issuecomment-819230152 ## Pull Request Test Coverage Report for [Build 7070](https://coveralls.io/builds/38793637) * **0** of **0** changed or added relevant lines in **0** files are covered. * **2** unchanged lines in **2** files lost coverage. * Overall coverage increased (+**0.001%**) to **28.094%** --- | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/38793637/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62) | 1 | 81.08% | | [server-base/src/main/java/org/apache/kylin/rest/util/QueryRequestLimits.java](https://coveralls.io/builds/38793637/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Futil%2FQueryRequestLimits.java#L72) | 1 | 47.62% | | Totals | [](https://coveralls.io/builds/38793637) | | :-- | --: | | Change from base [Build 7067](https://coveralls.io/builds/38780467): | 0.001% | | Covered Lines: | 26715 | | Relevant Lines: | 95091 | --- # - [Coveralls](https://coveralls.io) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4971) Add new measure bitmap_map for count distinct measure in UI
[ https://issues.apache.org/jira/browse/KYLIN-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320700#comment-17320700 ] ASF GitHub Bot commented on KYLIN-4971: --- Ted-Jiang commented on pull request #1633: URL: https://github.com/apache/kylin/pull/1633#issuecomment-819227599 Add bitmap_map measure.  only support segmentBuilder and auto add in Advanced Dict UI  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add new measure bitmap_map for count distinct measure in UI > > > Key: KYLIN-4971 > URL: https://issues.apache.org/jira/browse/KYLIN-4971 > Project: Kylin > Issue Type: Task >Reporter: JiangYang >Assignee: JiangYang >Priority: Minor > Fix For: v3.1.2 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] Ted-Jiang commented on pull request #1633: [KYLIN-4971] Add new measure bitmap_map for count distinct at fronten…
Ted-Jiang commented on pull request #1633: URL: https://github.com/apache/kylin/pull/1633#issuecomment-819227599 Add bitmap_map measure.  only support segmentBuilder and auto add in Advanced Dict UI  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4972) Don't allow segment merge when use dict.global.SegmentAppendTrieDictBuilder to build bitmap measure
[ https://issues.apache.org/jira/browse/KYLIN-4972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320698#comment-17320698 ] ASF GitHub Bot commented on KYLIN-4972: --- Ted-Jiang opened a new pull request #1634: URL: https://github.com/apache/kylin/pull/1634 …pendTrieDictBuilder to build bitmap measure. ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have create an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If this change need a document change, I will prepare another pr against the `document` branch - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at user@kylin or dev@kylin by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Don't allow segment merge when use dict.global.SegmentAppendTrieDictBuilder > to build bitmap measure > --- > > Key: KYLIN-4972 > URL: https://issues.apache.org/jira/browse/KYLIN-4972 > Project: Kylin > Issue Type: Improvement >Reporter: JiangYang >Assignee: JiangYang >Priority: Minor > Fix For: v3.1.2 > > > When *org.apache.kylin.dict.global.SegmentAppendTrieDictBuilder* is used for > bitmap count distinct measure, cube segment merge will lead to incorrect > bitmap merge. Therefore, we should forbid segment merge in this case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] Ted-Jiang opened a new pull request #1634: [KYLIN-4972] Don't allow segment merge when use dict.global.SegmentAp…
Ted-Jiang opened a new pull request #1634: URL: https://github.com/apache/kylin/pull/1634 …pendTrieDictBuilder to build bitmap measure. ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have create an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If this change need a document change, I will prepare another pr against the `document` branch - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at user@kylin or dev@kylin by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (KYLIN-4972) Don't allow segment merge when use dict.global.SegmentAppendTrieDictBuilder to build bitmap measure
[ https://issues.apache.org/jira/browse/KYLIN-4972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JiangYang updated KYLIN-4972: - Priority: Minor (was: Major) > Don't allow segment merge when use dict.global.SegmentAppendTrieDictBuilder > to build bitmap measure > --- > > Key: KYLIN-4972 > URL: https://issues.apache.org/jira/browse/KYLIN-4972 > Project: Kylin > Issue Type: Improvement >Reporter: JiangYang >Assignee: JiangYang >Priority: Minor > Fix For: v3.1.2 > > > When *org.apache.kylin.dict.global.SegmentAppendTrieDictBuilder* is used for > bitmap count distinct measure, cube segment merge will lead to incorrect > bitmap merge. Therefore, we should forbid segment merge in this case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4972) Don't allow segment merge when use dict.global.SegmentAppendTrieDictBuilder to build bitmap measure
[ https://issues.apache.org/jira/browse/KYLIN-4972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JiangYang updated KYLIN-4972: - Issue Type: Improvement (was: Bug) > Don't allow segment merge when use dict.global.SegmentAppendTrieDictBuilder > to build bitmap measure > --- > > Key: KYLIN-4972 > URL: https://issues.apache.org/jira/browse/KYLIN-4972 > Project: Kylin > Issue Type: Improvement >Reporter: JiangYang >Assignee: JiangYang >Priority: Major > Fix For: v3.1.2 > > > When *org.apache.kylin.dict.global.SegmentAppendTrieDictBuilder* is used for > bitmap count distinct measure, cube segment merge will lead to incorrect > bitmap merge. Therefore, we should forbid segment merge in this case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4972) Don't allow segment merge when use dict.global.SegmentAppendTrieDictBuilder to build bitmap measure
[ https://issues.apache.org/jira/browse/KYLIN-4972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JiangYang updated KYLIN-4972: - Summary: Don't allow segment merge when use dict.global.SegmentAppendTrieDictBuilder to build bitmap measure (was: Don't allow segment merge when use dict.global.SegmentAppendTrieDictBuilder to build bitmap_map measure) > Don't allow segment merge when use dict.global.SegmentAppendTrieDictBuilder > to build bitmap measure > --- > > Key: KYLIN-4972 > URL: https://issues.apache.org/jira/browse/KYLIN-4972 > Project: Kylin > Issue Type: Bug >Reporter: JiangYang >Assignee: JiangYang >Priority: Major > Fix For: v3.1.2 > > > When *org.apache.kylin.dict.global.SegmentAppendTrieDictBuilder* is used for > bitmap count distinct measure, cube segment merge will lead to incorrect > bitmap merge. Therefore, we should forbid segment merge in this case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KYLIN-4972) Don't allow segment merge when use dict.global.SegmentAppendTrieDictBuilder to build bitmap_map measure
JiangYang created KYLIN-4972: Summary: Don't allow segment merge when use dict.global.SegmentAppendTrieDictBuilder to build bitmap_map measure Key: KYLIN-4972 URL: https://issues.apache.org/jira/browse/KYLIN-4972 Project: Kylin Issue Type: Bug Reporter: JiangYang Assignee: JiangYang Fix For: v3.1.2 When *org.apache.kylin.dict.global.SegmentAppendTrieDictBuilder* is used for bitmap count distinct measure, cube segment merge will lead to incorrect bitmap merge. Therefore, we should forbid segment merge in this case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4971) Add new measure bitmap_map for count distinct measure in UI
[ https://issues.apache.org/jira/browse/KYLIN-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320685#comment-17320685 ] ASF GitHub Bot commented on KYLIN-4971: --- Ted-Jiang opened a new pull request #1633: URL: https://github.com/apache/kylin/pull/1633 …d. (#222) ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have create an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If this change need a document change, I will prepare another pr against the `document` branch - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at user@kylin or dev@kylin by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add new measure bitmap_map for count distinct measure in UI > > > Key: KYLIN-4971 > URL: https://issues.apache.org/jira/browse/KYLIN-4971 > Project: Kylin > Issue Type: Task >Reporter: JiangYang >Assignee: JiangYang >Priority: Minor > Fix For: v3.1.2 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] Ted-Jiang opened a new pull request #1633: [KYLIN-4971] Add new measure bitmap_map for count distinct at fronten…
Ted-Jiang opened a new pull request #1633: URL: https://github.com/apache/kylin/pull/1633 …d. (#222) ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have create an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If this change need a document change, I will prepare another pr against the `document` branch - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at user@kylin or dev@kylin by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4970) Fix spark.executor.extraJavaOptions args
[ https://issues.apache.org/jira/browse/KYLIN-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320662#comment-17320662 ] ASF GitHub Bot commented on KYLIN-4970: --- coveralls commented on pull request #1632: URL: https://github.com/apache/kylin/pull/1632#issuecomment-819195513 ## Pull Request Test Coverage Report for [Build 7068](https://coveralls.io/builds/38790935) * **0** of **7** **(0.0%)** changed or added relevant lines in **1** file are covered. * **6** unchanged lines in **2** files lost coverage. * Overall coverage decreased (**-0.006%**) to **28.087%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkExecutable.java](https://coveralls.io/builds/38790935/source?filename=engine-spark%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fspark%2FSparkExecutable.java#L294) | 0 | 7 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [server-base/src/main/java/org/apache/kylin/rest/util/QueryRequestLimits.java](https://coveralls.io/builds/38790935/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Futil%2FQueryRequestLimits.java#L72) | 1 | 47.62% | | [core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/MetricsSystem.java](https://coveralls.io/builds/38790935/source?filename=core-metrics%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetrics%2Flib%2Fimpl%2FMetricsSystem.java#L46) | 5 | 66.67% | | Totals | [](https://coveralls.io/builds/38790935) | | :-- | --: | | Change from base [Build 7067](https://coveralls.io/builds/38780467): | -0.006% | | Covered Lines: | 26709 | | Relevant Lines: | 95095 | --- # - [Coveralls](https://coveralls.io) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Fix spark.executor.extraJavaOptions args > > > Key: KYLIN-4970 > URL: https://issues.apache.org/jira/browse/KYLIN-4970 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Reporter: JiangYang >Assignee: JiangYang >Priority: Minor > Fix For: v3.1.2 > > > in spark doc: spark.executor.extraJavaOption need surrond with "". > !image-2021-04-09-10-18-58-294.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] coveralls commented on pull request #1632: [KYLIN-4970] fix spark.executor.extraJavaOptions args in SparkExecut…
coveralls commented on pull request #1632: URL: https://github.com/apache/kylin/pull/1632#issuecomment-819195513 ## Pull Request Test Coverage Report for [Build 7068](https://coveralls.io/builds/38790935) * **0** of **7** **(0.0%)** changed or added relevant lines in **1** file are covered. * **6** unchanged lines in **2** files lost coverage. * Overall coverage decreased (**-0.006%**) to **28.087%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkExecutable.java](https://coveralls.io/builds/38790935/source?filename=engine-spark%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fspark%2FSparkExecutable.java#L294) | 0 | 7 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [server-base/src/main/java/org/apache/kylin/rest/util/QueryRequestLimits.java](https://coveralls.io/builds/38790935/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Futil%2FQueryRequestLimits.java#L72) | 1 | 47.62% | | [core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/MetricsSystem.java](https://coveralls.io/builds/38790935/source?filename=core-metrics%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetrics%2Flib%2Fimpl%2FMetricsSystem.java#L46) | 5 | 66.67% | | Totals | [](https://coveralls.io/builds/38790935) | | :-- | --: | | Change from base [Build 7067](https://coveralls.io/builds/38780467): | -0.006% | | Covered Lines: | 26709 | | Relevant Lines: | 95095 | --- # - [Coveralls](https://coveralls.io) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4554) Validate "filter condition" on model saving
[ https://issues.apache.org/jira/browse/KYLIN-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320661#comment-17320661 ] ASF GitHub Bot commented on KYLIN-4554: --- zheniantoushipashi commented on pull request #1626: URL: https://github.com/apache/kylin/pull/1626#issuecomment-819194725 I added all the columns of all tables to the validate collection, I did verification in ut and integration tests, **test evidence**   -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Validate "filter condition" on model saving > --- > > Key: KYLIN-4554 > URL: https://issues.apache.org/jira/browse/KYLIN-4554 > Project: Kylin > Issue Type: Improvement > Components: Metadata >Reporter: Shao Feng Shi >Assignee: Sun BiaoBiao >Priority: Major > Fix For: v4.0.0-GA > > Attachments: WechatIMG397.png > > > When define a model, user can enter a filter condition (using to do data > filtering on loading). While the system doesn't validate the filtering > condition's syntax; Later when build the cube, the condition will be send to > Hive, Hive may report an error on the syntax, but at that time, it is > difficult to modify it. > > !WechatIMG397.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] zheniantoushipashi commented on pull request #1626: [KYLIN-4554] validate filter condition on model saving
zheniantoushipashi commented on pull request #1626: URL: https://github.com/apache/kylin/pull/1626#issuecomment-819194725 I added all the columns of all tables to the validate collection, I did verification in ut and integration tests, **test evidence**   -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4970) Fix spark.executor.extraJavaOptions args
[ https://issues.apache.org/jira/browse/KYLIN-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320651#comment-17320651 ] ASF GitHub Bot commented on KYLIN-4970: --- Ted-Jiang opened a new pull request #1632: URL: https://github.com/apache/kylin/pull/1632 …able. ## Proposed changes in spark doc: spark.executor.extraJavaOption need surrond with "". ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [x] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have create an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If this change need a document change, I will prepare another pr against the `document` branch - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at user@kylin or dev@kylin by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Fix spark.executor.extraJavaOptions args > > > Key: KYLIN-4970 > URL: https://issues.apache.org/jira/browse/KYLIN-4970 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Reporter: JiangYang >Assignee: JiangYang >Priority: Minor > Fix For: v3.1.2 > > > in spark doc: spark.executor.extraJavaOption need surrond with "". > !image-2021-04-09-10-18-58-294.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] Ted-Jiang opened a new pull request #1632: [KYLIN-4970] fix spark.executor.extraJavaOptions args in SparkExecut…
Ted-Jiang opened a new pull request #1632: URL: https://github.com/apache/kylin/pull/1632 …able. ## Proposed changes in spark doc: spark.executor.extraJavaOption need surrond with "". ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [x] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have create an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If this change need a document change, I will prepare another pr against the `document` branch - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at user@kylin or dev@kylin by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (KYLIN-4971) Add new measure bitmap_map for count distinct measure in UI
JiangYang created KYLIN-4971: Summary: Add new measure bitmap_map for count distinct measure in UI Key: KYLIN-4971 URL: https://issues.apache.org/jira/browse/KYLIN-4971 Project: Kylin Issue Type: Task Reporter: JiangYang Assignee: JiangYang Fix For: v3.1.2 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KYLIN-4970) Fix spark.executor.extraJavaOptions args
JiangYang created KYLIN-4970: Summary: Fix spark.executor.extraJavaOptions args Key: KYLIN-4970 URL: https://issues.apache.org/jira/browse/KYLIN-4970 Project: Kylin Issue Type: Bug Components: Spark Engine Reporter: JiangYang Assignee: JiangYang Fix For: v3.1.2 in spark doc: spark.executor.extraJavaOption need surrond with "". !image-2021-04-09-10-18-58-294.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4780) Refactor AbstractExecutable to extract the notification relevant business logic into a single component
[ https://issues.apache.org/jira/browse/KYLIN-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320623#comment-17320623 ] ASF GitHub Bot commented on KYLIN-4780: --- yanghua commented on pull request #1596: URL: https://github.com/apache/kylin/pull/1596#issuecomment-819151203 @chenjie-sau Can you check why the Java CI failed? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Refactor AbstractExecutable to extract the notification relevant business > logic into a single component > --- > > Key: KYLIN-4780 > URL: https://issues.apache.org/jira/browse/KYLIN-4780 > Project: Kylin > Issue Type: Improvement > Components: Integration >Reporter: vinoyang >Assignee: chenjie >Priority: Major > > Currently, Kylin supports mail service to know the change around the cube. > However, there are some issues about the design, e.g. > * AbstractExecutable contains an implementation of the mechanism of a > notification listener, but only support mail service, it does not belong > there; > * The current implementation is hard to scale to other notification e.g. > Ding Talk(which is a popular production supports by Alibaba) > I propose to refactor the current implementation to extract the notification > logic and redesign a new common notification/listener -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] yanghua commented on pull request #1596: [KYLIN-4780] Support dingtalk notify
yanghua commented on pull request #1596: URL: https://github.com/apache/kylin/pull/1596#issuecomment-819151203 @chenjie-sau Can you check why the Java CI failed? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4780) Refactor AbstractExecutable to extract the notification relevant business logic into a single component
[ https://issues.apache.org/jira/browse/KYLIN-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320285#comment-17320285 ] ASF GitHub Bot commented on KYLIN-4780: --- lgtm-com[bot] commented on pull request #1596: URL: https://github.com/apache/kylin/pull/1596#issuecomment-818858759 This pull request **introduces 3 alerts** when merging 77656aa2edfe2c600c7a8df29983c33ca00b8a21 into 69368b08a770f6e54c4ecf43c538331e13cb03b2 - [view on LGTM.com](https://lgtm.com/projects/g/apache/kylin/rev/pr-a74e7c0552a8d3f18d2a79eb9eec72743bf873fe) **new alerts:** * 3 for Dereferenced variable may be null -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Refactor AbstractExecutable to extract the notification relevant business > logic into a single component > --- > > Key: KYLIN-4780 > URL: https://issues.apache.org/jira/browse/KYLIN-4780 > Project: Kylin > Issue Type: Improvement > Components: Integration >Reporter: vinoyang >Assignee: chenjie >Priority: Major > > Currently, Kylin supports mail service to know the change around the cube. > However, there are some issues about the design, e.g. > * AbstractExecutable contains an implementation of the mechanism of a > notification listener, but only support mail service, it does not belong > there; > * The current implementation is hard to scale to other notification e.g. > Ding Talk(which is a popular production supports by Alibaba) > I propose to refactor the current implementation to extract the notification > logic and redesign a new common notification/listener -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] lgtm-com[bot] commented on pull request #1596: [KYLIN-4780] Support dingtalk notify
lgtm-com[bot] commented on pull request #1596: URL: https://github.com/apache/kylin/pull/1596#issuecomment-818858759 This pull request **introduces 3 alerts** when merging 77656aa2edfe2c600c7a8df29983c33ca00b8a21 into 69368b08a770f6e54c4ecf43c538331e13cb03b2 - [view on LGTM.com](https://lgtm.com/projects/g/apache/kylin/rev/pr-a74e7c0552a8d3f18d2a79eb9eec72743bf873fe) **new alerts:** * 3 for Dereferenced variable may be null -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kylin] coveralls edited a comment on pull request #1596: [KYLIN-4780] Support dingtalk notify
coveralls edited a comment on pull request #1596: URL: https://github.com/apache/kylin/pull/1596#issuecomment-791204373 ## Pull Request Test Coverage Report for [Build 7065](https://coveralls.io/builds/38770165) * **57** of **333** **(17.12%)** changed or added relevant lines in **20** files are covered. * **35** unchanged lines in **10** files lost coverage. * Overall coverage decreased (**-0.02%**) to **28.073%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-common/src/main/java/org/apache/kylin/common/notify/util/NotificationConstants.java](https://coveralls.io/builds/38770165/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fnotify%2Futil%2FNotificationConstants.java#L21) | 0 | 1 | 0.0% | [engine-mr/src/main/java/org/apache/kylin/engine/mr/LookupSnapshotBuildJob.java](https://coveralls.io/builds/38770165/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2FLookupSnapshotBuildJob.java#L69) | 0 | 2 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/constant/NotifyType.java](https://coveralls.io/builds/38770165/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fconstant%2FNotifyType.java#L21) | 0 | 2 | 0.0% | [core-cube/src/main/java/org/apache/kylin/cube/model/CubeDesc.java](https://coveralls.io/builds/38770165/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fmodel%2FCubeDesc.java#L537) | 10 | 13 | 76.92% | [core-common/src/main/java/org/apache/kylin/common/notify/util/SecretKeyUtil.java](https://coveralls.io/builds/38770165/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fnotify%2Futil%2FSecretKeyUtil.java#L30) | 8 | 12 | 66.67% | [core-common/src/main/java/org/apache/kylin/common/notify/NotifyServiceBase.java](https://coveralls.io/builds/38770165/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fnotify%2FNotifyServiceBase.java#L31) | 0 | 5 | 0.0% | [core-common/src/main/java/org/apache/kylin/common/notify/util/DingTalkNotificationUtil.java](https://coveralls.io/builds/38770165/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fnotify%2Futil%2FDingTalkNotificationUtil.java#L26) | 12 | 17 | 70.59% | [server-base/src/main/java/org/apache/kylin/rest/job/KylinHealthCheckJob.java](https://coveralls.io/builds/38770165/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fjob%2FKylinHealthCheckJob.java#L140) | 0 | 5 | 0.0% | [cube-migration/src/main/java/org/apache/kylin/rest/service/MigrationRuleSet.java](https://coveralls.io/builds/38770165/source?filename=cube-migration%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FMigrationRuleSet.java#L84) | 0 | 6 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/service/CubeService.java](https://coveralls.io/builds/38770165/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FCubeService.java#L582) | 0 | 6 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [cube-migration/src/main/java/org/apache/kylin/rest/service/MigrationService.java](https://coveralls.io/builds/38770165/source?filename=cube-migration%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FMigrationService.java#L236) | 1 | 0% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/CubingJob.java](https://coveralls.io/builds/38770165/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2FCubingJob.java#L167) | 1 | 1.48% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/LookupSnapshotBuildJob.java](https://coveralls.io/builds/38770165/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2FLookupSnapshotBuildJob.java#L71) | 1 | 0% | | [server-base/src/main/java/org/apache/kylin/rest/job/KylinHealthCheckJob.java](https://coveralls.io/builds/38770165/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fjob%2FKylinHealthCheckJob.java#L143) | 1 | 0% | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/columnar/GeneralColumnDataReader.java](https://coveralls.io/builds/38770165/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2Fcolumnar%2FGeneralColumnDataReader.java#L43) | 1 | 92.11% | | [cube-migration/src/main/java/org/apache/kylin/rest/service/MigrationRuleSet.java](https://coveralls.io/builds/38770165/source?filename=cube-migration%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FMigrationRuleSet.java#L216) | 2 | 0% | |
[jira] [Commented] (KYLIN-4780) Refactor AbstractExecutable to extract the notification relevant business logic into a single component
[ https://issues.apache.org/jira/browse/KYLIN-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320226#comment-17320226 ] ASF GitHub Bot commented on KYLIN-4780: --- coveralls edited a comment on pull request #1596: URL: https://github.com/apache/kylin/pull/1596#issuecomment-791204373 ## Pull Request Test Coverage Report for [Build 7065](https://coveralls.io/builds/38770165) * **57** of **333** **(17.12%)** changed or added relevant lines in **20** files are covered. * **35** unchanged lines in **10** files lost coverage. * Overall coverage decreased (**-0.02%**) to **28.073%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-common/src/main/java/org/apache/kylin/common/notify/util/NotificationConstants.java](https://coveralls.io/builds/38770165/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fnotify%2Futil%2FNotificationConstants.java#L21) | 0 | 1 | 0.0% | [engine-mr/src/main/java/org/apache/kylin/engine/mr/LookupSnapshotBuildJob.java](https://coveralls.io/builds/38770165/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2FLookupSnapshotBuildJob.java#L69) | 0 | 2 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/constant/NotifyType.java](https://coveralls.io/builds/38770165/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fconstant%2FNotifyType.java#L21) | 0 | 2 | 0.0% | [core-cube/src/main/java/org/apache/kylin/cube/model/CubeDesc.java](https://coveralls.io/builds/38770165/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fmodel%2FCubeDesc.java#L537) | 10 | 13 | 76.92% | [core-common/src/main/java/org/apache/kylin/common/notify/util/SecretKeyUtil.java](https://coveralls.io/builds/38770165/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fnotify%2Futil%2FSecretKeyUtil.java#L30) | 8 | 12 | 66.67% | [core-common/src/main/java/org/apache/kylin/common/notify/NotifyServiceBase.java](https://coveralls.io/builds/38770165/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fnotify%2FNotifyServiceBase.java#L31) | 0 | 5 | 0.0% | [core-common/src/main/java/org/apache/kylin/common/notify/util/DingTalkNotificationUtil.java](https://coveralls.io/builds/38770165/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fnotify%2Futil%2FDingTalkNotificationUtil.java#L26) | 12 | 17 | 70.59% | [server-base/src/main/java/org/apache/kylin/rest/job/KylinHealthCheckJob.java](https://coveralls.io/builds/38770165/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fjob%2FKylinHealthCheckJob.java#L140) | 0 | 5 | 0.0% | [cube-migration/src/main/java/org/apache/kylin/rest/service/MigrationRuleSet.java](https://coveralls.io/builds/38770165/source?filename=cube-migration%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FMigrationRuleSet.java#L84) | 0 | 6 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/service/CubeService.java](https://coveralls.io/builds/38770165/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FCubeService.java#L582) | 0 | 6 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [cube-migration/src/main/java/org/apache/kylin/rest/service/MigrationService.java](https://coveralls.io/builds/38770165/source?filename=cube-migration%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FMigrationService.java#L236) | 1 | 0% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/CubingJob.java](https://coveralls.io/builds/38770165/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2FCubingJob.java#L167) | 1 | 1.48% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/LookupSnapshotBuildJob.java](https://coveralls.io/builds/38770165/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2FLookupSnapshotBuildJob.java#L71) | 1 | 0% | | [server-base/src/main/java/org/apache/kylin/rest/job/KylinHealthCheckJob.java](https://coveralls.io/builds/38770165/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fjob%2FKylinHealthCheckJob.java#L143) | 1 | 0% | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/columnar/GeneralColumnDataReader.java](https://coveralls.io/builds/38770165/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2Fcolumnar%2FGeneralColumnDataReader.java#L43) | 1 | 92.11% | |
[GitHub] [kylin] codecov-io edited a comment on pull request #1596: [KYLIN-4780] Support dingtalk notify
codecov-io edited a comment on pull request #1596: URL: https://github.com/apache/kylin/pull/1596#issuecomment-799678944 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1596?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#1596](https://codecov.io/gh/apache/kylin/pull/1596?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (d84c139) into [master](https://codecov.io/gh/apache/kylin/commit/02500818d85955b1125340eaa19ab88a634a5181?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (0250081) will **decrease** coverage by `0.10%`. > The diff coverage is `10.01%`. [](https://codecov.io/gh/apache/kylin/pull/1596?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#1596 +/- ## - Coverage 25.42% 25.32% -0.11% - Complexity 6765 6774 +9 Files 1508 1516 +8 Lines 9391894422 +504 Branches 1315813233 +75 + Hits 2387723910 +33 - Misses6766268126 +464 - Partials 2379 2386 +7 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1596?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/1596/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==) | `12.42% <0.00%> (-0.05%)` | `51.00 <0.00> (ø)` | | | [...rg/apache/kylin/common/notify/DingTalkService.java](https://codecov.io/gh/apache/kylin/pull/1596/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9ub3RpZnkvRGluZ1RhbGtTZXJ2aWNlLmphdmE=) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (?)` | | | [...va/org/apache/kylin/common/notify/MailService.java](https://codecov.io/gh/apache/kylin/pull/1596/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9ub3RpZnkvTWFpbFNlcnZpY2UuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (?)` | | | [...pache/kylin/common/notify/NotificationContext.java](https://codecov.io/gh/apache/kylin/pull/1596/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9ub3RpZnkvTm90aWZpY2F0aW9uQ29udGV4dC5qYXZh) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (?)` | | | [...e/kylin/common/notify/NotificationTransmitter.java](https://codecov.io/gh/apache/kylin/pull/1596/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9ub3RpZnkvTm90aWZpY2F0aW9uVHJhbnNtaXR0ZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (?)` | | | [.../apache/kylin/common/notify/NotifyServiceBase.java](https://codecov.io/gh/apache/kylin/pull/1596/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9ub3RpZnkvTm90aWZ5U2VydmljZUJhc2UuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (?)` | | | [...ylin/common/notify/util/NotificationConstants.java](https://codecov.io/gh/apache/kylin/pull/1596/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9ub3RpZnkvdXRpbC9Ob3RpZmljYXRpb25Db25zdGFudHMuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (?)` | | |
[jira] [Commented] (KYLIN-4780) Refactor AbstractExecutable to extract the notification relevant business logic into a single component
[ https://issues.apache.org/jira/browse/KYLIN-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320205#comment-17320205 ] ASF GitHub Bot commented on KYLIN-4780: --- codecov-io edited a comment on pull request #1596: URL: https://github.com/apache/kylin/pull/1596#issuecomment-799678944 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1596?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#1596](https://codecov.io/gh/apache/kylin/pull/1596?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (d84c139) into [master](https://codecov.io/gh/apache/kylin/commit/02500818d85955b1125340eaa19ab88a634a5181?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (0250081) will **decrease** coverage by `0.10%`. > The diff coverage is `10.01%`. [](https://codecov.io/gh/apache/kylin/pull/1596?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#1596 +/- ## - Coverage 25.42% 25.32% -0.11% - Complexity 6765 6774 +9 Files 1508 1516 +8 Lines 9391894422 +504 Branches 1315813233 +75 + Hits 2387723910 +33 - Misses6766268126 +464 - Partials 2379 2386 +7 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1596?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/1596/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==) | `12.42% <0.00%> (-0.05%)` | `51.00 <0.00> (ø)` | | | [...rg/apache/kylin/common/notify/DingTalkService.java](https://codecov.io/gh/apache/kylin/pull/1596/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9ub3RpZnkvRGluZ1RhbGtTZXJ2aWNlLmphdmE=) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (?)` | | | [...va/org/apache/kylin/common/notify/MailService.java](https://codecov.io/gh/apache/kylin/pull/1596/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9ub3RpZnkvTWFpbFNlcnZpY2UuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (?)` | | | [...pache/kylin/common/notify/NotificationContext.java](https://codecov.io/gh/apache/kylin/pull/1596/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9ub3RpZnkvTm90aWZpY2F0aW9uQ29udGV4dC5qYXZh) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (?)` | | | [...e/kylin/common/notify/NotificationTransmitter.java](https://codecov.io/gh/apache/kylin/pull/1596/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9ub3RpZnkvTm90aWZpY2F0aW9uVHJhbnNtaXR0ZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (?)` | | | [.../apache/kylin/common/notify/NotifyServiceBase.java](https://codecov.io/gh/apache/kylin/pull/1596/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9ub3RpZnkvTm90aWZ5U2VydmljZUJhc2UuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (?)` | | | [...ylin/common/notify/util/NotificationConstants.java](https://codecov.io/gh/apache/kylin/pull/1596/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9ub3RpZnkvdXRpbC9Ob3RpZmljYXRpb25Db25zdGFudHMuamF2YQ==)
[jira] [Commented] (KYLIN-4941) Support encoding raw data to base cuboid column-by-column
[ https://issues.apache.org/jira/browse/KYLIN-4941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320204#comment-17320204 ] ASF GitHub Bot commented on KYLIN-4941: --- codecov-io commented on pull request #1631: URL: https://github.com/apache/kylin/pull/1631#issuecomment-818775584 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1631?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#1631](https://codecov.io/gh/apache/kylin/pull/1631?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (528ad97) into [master](https://codecov.io/gh/apache/kylin/commit/02500818d85955b1125340eaa19ab88a634a5181?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (0250081) will **decrease** coverage by `0.11%`. > The diff coverage is `5.21%`. [](https://codecov.io/gh/apache/kylin/pull/1631?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#1631 +/- ## - Coverage 25.42% 25.31% -0.12% + Complexity 6765 6762 -3 Files 1508 1509 +1 Lines 9391894358 +440 Branches 1315813230 +72 + Hits 2387723883 +6 - Misses6766268092 +430 - Partials 2379 2383 +4 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1631?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==) | `12.40% <0.00%> (-0.07%)` | `51.00 <0.00> (ø)` | | | [...in/java/org/apache/kylin/cube/CubeDescManager.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL0N1YmVEZXNjTWFuYWdlci5qYXZh) | `32.41% <0.00%> (-1.16%)` | `12.00 <0.00> (ø)` | | | [...c/main/java/org/apache/kylin/cube/CubeManager.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL0N1YmVNYW5hZ2VyLmphdmE=) | `35.83% <0.00%> (-0.81%)` | `44.00 <0.00> (ø)` | | | [...c/main/java/org/apache/kylin/cube/CubeSegment.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL0N1YmVTZWdtZW50LmphdmE=) | `41.21% <0.00%> (-1.07%)` | `65.00 <0.00> (ø)` | | | [...ain/java/org/apache/kylin/cube/model/CubeDesc.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL21vZGVsL0N1YmVEZXNjLmphdmE=) | `57.02% <0.00%> (-1.34%)` | `182.00 <0.00> (ø)` | | | [...n/java/org/apache/kylin/cube/model/RowKeyDesc.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL21vZGVsL1Jvd0tleURlc2MuamF2YQ==) | `71.42% <0.00%> (-11.91%)` | `17.00 <0.00> (ø)` | | | [...a/org/apache/kylin/job/common/ShellExecutable.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9jb21tb24vU2hlbGxFeGVjdXRhYmxlLmphdmE=) | `0.00% <ø> (ø)` | `0.00 <0.00> (ø)` | | |
[GitHub] [kylin] codecov-io commented on pull request #1631: KYLIN-4941 support encoding raw data to base cuboid column-by-column
codecov-io commented on pull request #1631: URL: https://github.com/apache/kylin/pull/1631#issuecomment-818775584 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1631?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) Report > Merging [#1631](https://codecov.io/gh/apache/kylin/pull/1631?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (528ad97) into [master](https://codecov.io/gh/apache/kylin/commit/02500818d85955b1125340eaa19ab88a634a5181?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) (0250081) will **decrease** coverage by `0.11%`. > The diff coverage is `5.21%`. [](https://codecov.io/gh/apache/kylin/pull/1631?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master#1631 +/- ## - Coverage 25.42% 25.31% -0.12% + Complexity 6765 6762 -3 Files 1508 1509 +1 Lines 9391894358 +440 Branches 1315813230 +72 + Hits 2387723883 +6 - Misses6766268092 +430 - Partials 2379 2383 +4 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1631?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==) | `12.40% <0.00%> (-0.07%)` | `51.00 <0.00> (ø)` | | | [...in/java/org/apache/kylin/cube/CubeDescManager.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL0N1YmVEZXNjTWFuYWdlci5qYXZh) | `32.41% <0.00%> (-1.16%)` | `12.00 <0.00> (ø)` | | | [...c/main/java/org/apache/kylin/cube/CubeManager.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL0N1YmVNYW5hZ2VyLmphdmE=) | `35.83% <0.00%> (-0.81%)` | `44.00 <0.00> (ø)` | | | [...c/main/java/org/apache/kylin/cube/CubeSegment.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL0N1YmVTZWdtZW50LmphdmE=) | `41.21% <0.00%> (-1.07%)` | `65.00 <0.00> (ø)` | | | [...ain/java/org/apache/kylin/cube/model/CubeDesc.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL21vZGVsL0N1YmVEZXNjLmphdmE=) | `57.02% <0.00%> (-1.34%)` | `182.00 <0.00> (ø)` | | | [...n/java/org/apache/kylin/cube/model/RowKeyDesc.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL21vZGVsL1Jvd0tleURlc2MuamF2YQ==) | `71.42% <0.00%> (-11.91%)` | `17.00 <0.00> (ø)` | | | [...a/org/apache/kylin/job/common/ShellExecutable.java](https://codecov.io/gh/apache/kylin/pull/1631/diff?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9jb21tb24vU2hlbGxFeGVjdXRhYmxlLmphdmE=) | `0.00% <ø> (ø)` | `0.00 <0.00> (ø)` | | |
[GitHub] [kylin] lgtm-com[bot] commented on pull request #1596: [KYLIN-4780] support dingtalk notify
lgtm-com[bot] commented on pull request #1596: URL: https://github.com/apache/kylin/pull/1596#issuecomment-818626226 This pull request **introduces 3 alerts** when merging d84c139af8b97e6e360365bf3de1ade602300735 into 69368b08a770f6e54c4ecf43c538331e13cb03b2 - [view on LGTM.com](https://lgtm.com/projects/g/apache/kylin/rev/pr-8339c1ea84ce3123a9b109f78fdfa90db3d642db) **new alerts:** * 3 for Dereferenced variable may be null -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4780) Refactor AbstractExecutable to extract the notification relevant business logic into a single component
[ https://issues.apache.org/jira/browse/KYLIN-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320068#comment-17320068 ] ASF GitHub Bot commented on KYLIN-4780: --- lgtm-com[bot] commented on pull request #1596: URL: https://github.com/apache/kylin/pull/1596#issuecomment-818626226 This pull request **introduces 3 alerts** when merging d84c139af8b97e6e360365bf3de1ade602300735 into 69368b08a770f6e54c4ecf43c538331e13cb03b2 - [view on LGTM.com](https://lgtm.com/projects/g/apache/kylin/rev/pr-8339c1ea84ce3123a9b109f78fdfa90db3d642db) **new alerts:** * 3 for Dereferenced variable may be null -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Refactor AbstractExecutable to extract the notification relevant business > logic into a single component > --- > > Key: KYLIN-4780 > URL: https://issues.apache.org/jira/browse/KYLIN-4780 > Project: Kylin > Issue Type: Improvement > Components: Integration >Reporter: vinoyang >Assignee: chenjie >Priority: Major > > Currently, Kylin supports mail service to know the change around the cube. > However, there are some issues about the design, e.g. > * AbstractExecutable contains an implementation of the mechanism of a > notification listener, but only support mail service, it does not belong > there; > * The current implementation is hard to scale to other notification e.g. > Ding Talk(which is a popular production supports by Alibaba) > I propose to refactor the current implementation to extract the notification > logic and redesign a new common notification/listener -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4941) Support encoding raw data to base cuboid column-by-column
[ https://issues.apache.org/jira/browse/KYLIN-4941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320044#comment-17320044 ] ASF GitHub Bot commented on KYLIN-4941: --- zhengshengjun opened a new pull request #1631: URL: https://github.com/apache/kylin/pull/1631 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [x] I have create an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [x] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [x] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If this change need a document change, I will prepare another pr against the `document` branch - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at user@kylin or dev@kylin by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Support encoding raw data to base cuboid column-by-column > - > > Key: KYLIN-4941 > URL: https://issues.apache.org/jira/browse/KYLIN-4941 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v3.1.1 >Reporter: ShengJun Zheng >Assignee: ShengJun Zheng >Priority: Major > Fix For: v3.2.0 > > > When building with spark engine, the first step is to encode hive table's row > to base cuboid data. > The existing implementation is encoding row by row. If the cube has several > dictionary encoded measures, it has to use all dictionaries at the same time > to encode a single row. This causes heavy memory usage, and low cache hit > ratio of dictionary cache. > We optimized this case by encoding column by column, and it did bring > significant improvement over cubes with several high cardinality > dictionaries-encoded measures. > We will refine the implementation based on KYLIN3.x and share it out. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] zhengshengjun opened a new pull request #1631: KYLIN-4941 support encoding raw data to base cuboid column-by-column
zhengshengjun opened a new pull request #1631: URL: https://github.com/apache/kylin/pull/1631 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [x] I have create an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [x] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [x] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If this change need a document change, I will prepare another pr against the `document` branch - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at user@kylin or dev@kylin by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kylin] yanghua commented on a change in pull request #1596: [KYLIN-4780] support dingtalk notify
yanghua commented on a change in pull request #1596:
URL: https://github.com/apache/kylin/pull/1596#discussion_r612261129
##
File path:
core-job/src/main/java/org/apache/kylin/job/execution/AbstractExecutable.java
##
@@ -311,76 +317,59 @@ public final void setSubmitter(String submitter) {
setParam(SUBMITTER, submitter);
}
-public final List getNotifyList() {
-final String str = getParam(NOTIFY_LIST);
+public final List getNotificationList(String key) {
+final String str = getParam(key);
if (str != null) {
return Lists.newArrayList(StringUtils.split(str, ","));
} else {
return Collections.emptyList();
}
}
-public final void setNotifyList(String notifications) {
-setParam(NOTIFY_LIST, notifications);
+public final void setNotificationList(String key, String notifications) {
+setParam(key, notifications);
}
-public final void setNotifyList(List notifications) {
-setNotifyList(StringUtils.join(notifications, ","));
+public final void setNotificationList(String key, List
notifications) {
+setNotificationList(key, StringUtils.join(notifications, ","));
}
-protected Pair formatNotifications(ExecutableContext
executableContext, ExecutableState state) {
+protected Pair>
formatNotifications(ExecutableContext executableContext, ExecutableState state)
{
return null;
}
protected final void notifyUserStatusChange(ExecutableContext context,
ExecutableState state) {
try {
-List users = getAllNofifyUsers(config);
+Map> users = getAllNofificationUsers(config);
if (users.isEmpty()) {
logger.debug(NO_NEED_TO_SEND_EMAIL_USER_LIST_IS_EMPTY);
return;
}
-final Pair email = formatNotifications(context,
state);
-doSendMail(config, users, email);
+final Pair> notification =
formatNotifications(context, state);
+doSendNotification(config, users, state.name(), notification);
} catch (Exception e) {
-logger.error("error send email", e);
+logger.error("error send notification", e);
}
}
-private List getAllNofifyUsers(KylinConfig kylinConfig) {
-List users = Lists.newArrayList();
-users.addAll(getNotifyList());
+private Map> getAllNofificationUsers(KylinConfig
kylinConfig) {
+Map> users = Maps.newHashMap();
+users.put(NotificationConstants.NOTIFY_EMAIL_LIST,
getNotificationList(NotificationConstants.NOTIFY_EMAIL_LIST));
+users.put(NotificationConstants.NOTIFY_DINGTALK_LIST,
getNotificationList(NotificationConstants.NOTIFY_DINGTALK_LIST));
final String[] adminDls = kylinConfig.getAdminDls();
if (null != adminDls) {
for (String adminDl : adminDls) {
-users.add(adminDl);
+
users.get(NotificationConstants.NOTIFY_EMAIL_LIST).add(adminDl);
}
}
return users;
}
-private void doSendMail(KylinConfig kylinConfig, List users,
Pair email) {
-if (email == null) {
-logger.warn("no need to send email, content is null");
-return;
-}
-logger.info("prepare to send email to:{}", users);
+private void doSendNotification(KylinConfig kylinConfig, Map> receivers, String state, Pair>
content) {
Review comment:
too long...
##
File path:
core-common/src/main/java/org/apache/kylin/common/notify/NotificationContext.java
##
@@ -0,0 +1,150 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.common.notify;
+
+import org.apache.kylin.common.KylinConfig;
+import org.apache.kylin.common.util.Pair;
+
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+
+/**
+ * All basic information of the notification.
+ */
+public class NotificationContext {
+private KylinConfig config;
+
+/**
+ * Notification recipient，include email and dingTalk
+ */
+private Map> receivers;
+
+/**
+ * state of job
+ */
+
[jira] [Commented] (KYLIN-4780) Refactor AbstractExecutable to extract the notification relevant business logic into a single component
[
https://issues.apache.org/jira/browse/KYLIN-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320041#comment-17320041
]
ASF GitHub Bot commented on KYLIN-4780:
---
yanghua commented on a change in pull request #1596:
URL: https://github.com/apache/kylin/pull/1596#discussion_r612261129
##
File path:
core-job/src/main/java/org/apache/kylin/job/execution/AbstractExecutable.java
##
@@ -311,76 +317,59 @@ public final void setSubmitter(String submitter) {
setParam(SUBMITTER, submitter);
}
-public final List getNotifyList() {
-final String str = getParam(NOTIFY_LIST);
+public final List getNotificationList(String key) {
+final String str = getParam(key);
if (str != null) {
return Lists.newArrayList(StringUtils.split(str, ","));
} else {
return Collections.emptyList();
}
}
-public final void setNotifyList(String notifications) {
-setParam(NOTIFY_LIST, notifications);
+public final void setNotificationList(String key, String notifications) {
+setParam(key, notifications);
}
-public final void setNotifyList(List notifications) {
-setNotifyList(StringUtils.join(notifications, ","));
+public final void setNotificationList(String key, List
notifications) {
+setNotificationList(key, StringUtils.join(notifications, ","));
}
-protected Pair formatNotifications(ExecutableContext
executableContext, ExecutableState state) {
+protected Pair>
formatNotifications(ExecutableContext executableContext, ExecutableState state)
{
return null;
}
protected final void notifyUserStatusChange(ExecutableContext context,
ExecutableState state) {
try {
-List users = getAllNofifyUsers(config);
+Map> users = getAllNofificationUsers(config);
if (users.isEmpty()) {
logger.debug(NO_NEED_TO_SEND_EMAIL_USER_LIST_IS_EMPTY);
return;
}
-final Pair email = formatNotifications(context,
state);
-doSendMail(config, users, email);
+final Pair> notification =
formatNotifications(context, state);
+doSendNotification(config, users, state.name(), notification);
} catch (Exception e) {
-logger.error("error send email", e);
+logger.error("error send notification", e);
}
}
-private List getAllNofifyUsers(KylinConfig kylinConfig) {
-List users = Lists.newArrayList();
-users.addAll(getNotifyList());
+private Map> getAllNofificationUsers(KylinConfig
kylinConfig) {
+Map> users = Maps.newHashMap();
+users.put(NotificationConstants.NOTIFY_EMAIL_LIST,
getNotificationList(NotificationConstants.NOTIFY_EMAIL_LIST));
+users.put(NotificationConstants.NOTIFY_DINGTALK_LIST,
getNotificationList(NotificationConstants.NOTIFY_DINGTALK_LIST));
final String[] adminDls = kylinConfig.getAdminDls();
if (null != adminDls) {
for (String adminDl : adminDls) {
-users.add(adminDl);
+
users.get(NotificationConstants.NOTIFY_EMAIL_LIST).add(adminDl);
}
}
return users;
}
-private void doSendMail(KylinConfig kylinConfig, List users,
Pair email) {
-if (email == null) {
-logger.warn("no need to send email, content is null");
-return;
-}
-logger.info("prepare to send email to:{}", users);
+private void doSendNotification(KylinConfig kylinConfig, Map> receivers, String state, Pair>
content) {
Review comment:
too long...
##
File path:
core-common/src/main/java/org/apache/kylin/common/notify/NotificationContext.java
##
@@ -0,0 +1,150 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.common.notify;
+
+import org.apache.kylin.common.KylinConfig;
+import org.apache.kylin.common.util.Pair;
+
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+
+/**
+ * All basic information of the
[jira] [Updated] (KYLIN-4861) Wrong way to get CubeManager instance in CubeInstance.latestCopyForWrite()
[
https://issues.apache.org/jira/browse/KYLIN-4861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoxiang Yu updated KYLIN-4861:
Fix Version/s: (was: v3.1.2)
> Wrong way to get CubeManager instance in CubeInstance.latestCopyForWrite()
> --
>
> Key: KYLIN-4861
> URL: https://issues.apache.org/jira/browse/KYLIN-4861
> Project: Kylin
> Issue Type: Bug
>Reporter: Zhong Yanghong
>Assignee: Xiaoxiang Yu
>Priority: Major
>
> Each cube can have its own KylinConfig. Then for the following code:
> {code}
> public CubeInstance latestCopyForWrite() {
> CubeManager mgr = CubeManager.getInstance(config);
> CubeInstance latest = mgr.getCube(name); // in case this object is
> out-of-date
> return mgr.copyForWrite(latest);
> }
> {code}
> Each cube can have a different CubeManager instance, which may easily cause
> map consistency issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4823) Push down having filter error when group by dynamic column
[
https://issues.apache.org/jira/browse/KYLIN-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoxiang Yu updated KYLIN-4823:
Fix Version/s: (was: v3.1.2)
> Push down having filter error when group by dynamic column
> ---
>
> Key: KYLIN-4823
> URL: https://issues.apache.org/jira/browse/KYLIN-4823
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v3.1.0
>Reporter: hcy
>Priority: Major
>
> 如果cube只有一个segment，且shard by的列存在于group by中，满足having filter push
> down的条件时，如果group by中存在动态列，并且case when then
> 中的表达是为column而不是常量时会报数组越界的错误。配置kylin.query.enable-dynamic-column=true无效，也会报错。
> 测试Cube如下:
> 模型为kylin
> example中的kylin_sales_model，cube为kylin_sales_cube，为了重现错误把BUYER_ID的rowkey设置为shard
> by
> 测试SQL如下:
> SELECT BUYER_ID,
> CASE WHEN LSTG_SITE_ID > 1 then LSTG_SITE_ID else LEAF_CATEG_ID END AS
> dyna_group,
> SUM(PRICE)
> FROM KYLIN_SALES
> GROUP BY
> BUYER_ID,
> CASE WHEN LSTG_SITE_ID > 1 then LSTG_SITE_ID else LEAF_CATEG_ID END
> HAVING SUM(PRICE)>10
> 报错如下：
> {color:#b94a48}Index: 4, Size: 1 while executing SQL: "select * from (SELECT
> BUYER_ID, CASE WHEN LSTG_SITE_ID > 1 then LSTG_SITE_ID else LEAF_CATEG_ID END
> AS dyna_group, SUM(PRICE) FROM KYLIN_SALES GROUP BY BUYER_ID, CASE WHEN
> LSTG_SITE_ID > 1 then LSTG_SITE_ID else LEAF_CATEG_ID END HAVING
> SUM(PRICE)>10) limit 5"{color}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 4, Size: 1
> at java.util.ArrayList.rangeCheck(ArrayList.java:657)
> at java.util.ArrayList.get(ArrayList.java:433)
> at
> org.apache.kylin.storage.gtrecord.GTCubeStorageQueryBase.checkHavingCanPushDown(GTCubeStorageQueryBase.java:552)
> at
> org.apache.kylin.storage.gtrecord.GTCubeStorageQueryBase.getStorageQueryRequest(GTCubeStorageQueryBase.java:189)
> at
> org.apache.kylin.storage.gtrecord.GTCubeStorageQueryBase.search(GTCubeStorageQueryBase.java:89)
> at
> org.apache.kylin.query.enumerator.OLAPEnumerator.queryStorage(OLAPEnumerator.java:117)
> at
> org.apache.kylin.query.enumerator.OLAPEnumerator.moveNext(OLAPEnumerator.java:60)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-4955) fix typo in KYLIN UI when not set dictionary for count_distinct measure
[ https://issues.apache.org/jira/browse/KYLIN-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoxiang Yu resolved KYLIN-4955. - Resolution: Fixed > fix typo in KYLIN UI when not set dictionary for count_distinct measure > --- > > Key: KYLIN-4955 > URL: https://issues.apache.org/jira/browse/KYLIN-4955 > Project: Kylin > Issue Type: Improvement > Components: Web >Reporter: JiangYang >Assignee: JiangYang >Priority: Minor > Fix For: v3.1.2 > > Attachments: image-2021-04-07-21-12-44-396.png > > > when create count_distinct measure without set dictionary: > !image-2021-04-07-21-12-44-396.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
