[jira] [Commented] (KYLIN-4939) Transform lookup table snapshot from segment level to cube level
[ https://issues.apache.org/jira/browse/KYLIN-4939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17310403#comment-17310403 ] ASF GitHub Bot commented on KYLIN-4939: --- coveralls edited a comment on pull request #1615: URL: https://github.com/apache/kylin/pull/1615#issuecomment-802724974 ## Pull Request Test Coverage Report for [Build 7010](https://coveralls.io/builds/38322517) * **0** of **84** **(0.0%)** changed or added relevant lines in **7** files are covered. * **7** unchanged lines in **2** files lost coverage. * Overall coverage decreased (**-0.03%**) to **28.098%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-cube/src/main/java/org/apache/kylin/cube/CubeSegment.java](https://coveralls.io/builds/38322517/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2FCubeSegment.java#L329) | 0 | 2 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/msg/Message.java](https://coveralls.io/builds/38322517/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fmsg%2FMessage.java#L177) | 0 | 2 | 0.0% | [core-cube/src/main/java/org/apache/kylin/cube/CubeDescManager.java](https://coveralls.io/builds/38322517/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2FCubeDescManager.java#L260) | 0 | 5 | 0.0% | [core-cube/src/main/java/org/apache/kylin/cube/CubeManager.java](https://coveralls.io/builds/38322517/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2FCubeManager.java#L380) | 0 | 14 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/controller/CubeController.java](https://coveralls.io/builds/38322517/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fcontroller%2FCubeController.java#L359) | 0 | 14 | 0.0% | [core-cube/src/main/java/org/apache/kylin/cube/model/CubeDesc.java](https://coveralls.io/builds/38322517/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fmodel%2FCubeDesc.java#L628) | 0 | 20 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/service/CubeService.java](https://coveralls.io/builds/38322517/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FCubeService.java#L946) | 0 | 27 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/CheckPointStore.java](https://coveralls.io/builds/38322517/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2FCheckPointStore.java#L98) | 2 | 73.74% | | [core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/MetricsSystem.java](https://coveralls.io/builds/38322517/source?filename=core-metrics%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetrics%2Flib%2Fimpl%2FMetricsSystem.java#L46) | 5 | 66.67% | | Totals | [](https://coveralls.io/builds/38322517) | | :-- | --: | | Change from base [Build 6998](https://coveralls.io/builds/38225952): | -0.03% | | Covered Lines: | 26702 | | Relevant Lines: | 95033 | --- # - [Coveralls](https://coveralls.io) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Transform lookup table snapshot from segment level to cube level > > > Key: KYLIN-4939 > URL: https://issues.apache.org/jira/browse/KYLIN-4939 > Project: Kylin > Issue Type: New Feature >Affects Versions: v3.1.2 >Reporter: JiangYang >Assignee: JiangYang >Priority: Major > Fix For: v3.1.2 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] coveralls edited a comment on pull request #1615: [KYLIN-4939] Transform lookup table snapshot from segment level to cu…
coveralls edited a comment on pull request #1615: URL: https://github.com/apache/kylin/pull/1615#issuecomment-802724974 ## Pull Request Test Coverage Report for [Build 7010](https://coveralls.io/builds/38322517) * **0** of **84** **(0.0%)** changed or added relevant lines in **7** files are covered. * **7** unchanged lines in **2** files lost coverage. * Overall coverage decreased (**-0.03%**) to **28.098%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-cube/src/main/java/org/apache/kylin/cube/CubeSegment.java](https://coveralls.io/builds/38322517/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2FCubeSegment.java#L329) | 0 | 2 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/msg/Message.java](https://coveralls.io/builds/38322517/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fmsg%2FMessage.java#L177) | 0 | 2 | 0.0% | [core-cube/src/main/java/org/apache/kylin/cube/CubeDescManager.java](https://coveralls.io/builds/38322517/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2FCubeDescManager.java#L260) | 0 | 5 | 0.0% | [core-cube/src/main/java/org/apache/kylin/cube/CubeManager.java](https://coveralls.io/builds/38322517/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2FCubeManager.java#L380) | 0 | 14 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/controller/CubeController.java](https://coveralls.io/builds/38322517/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fcontroller%2FCubeController.java#L359) | 0 | 14 | 0.0% | [core-cube/src/main/java/org/apache/kylin/cube/model/CubeDesc.java](https://coveralls.io/builds/38322517/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fmodel%2FCubeDesc.java#L628) | 0 | 20 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/service/CubeService.java](https://coveralls.io/builds/38322517/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FCubeService.java#L946) | 0 | 27 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/CheckPointStore.java](https://coveralls.io/builds/38322517/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2FCheckPointStore.java#L98) | 2 | 73.74% | | [core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/MetricsSystem.java](https://coveralls.io/builds/38322517/source?filename=core-metrics%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetrics%2Flib%2Fimpl%2FMetricsSystem.java#L46) | 5 | 66.67% | | Totals | [](https://coveralls.io/builds/38322517) | | :-- | --: | | Change from base [Build 6998](https://coveralls.io/builds/38225952): | -0.03% | | Covered Lines: | 26702 | | Relevant Lines: | 95033 | --- # - [Coveralls](https://coveralls.io) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4939) Transform lookup table snapshot from segment level to cube level
[ https://issues.apache.org/jira/browse/KYLIN-4939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17310396#comment-17310396 ] ASF GitHub Bot commented on KYLIN-4939: --- codecov-io commented on pull request #1615: URL: https://github.com/apache/kylin/pull/1615#issuecomment-809054507 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1615?src=pr=h1) Report > Merging [#1615](https://codecov.io/gh/apache/kylin/pull/1615?src=pr=desc) (b733593) into [master](https://codecov.io/gh/apache/kylin/commit/02500818d85955b1125340eaa19ab88a634a5181?el=desc) (0250081) will **decrease** coverage by `0.07%`. > The diff coverage is `4.80%`. [](https://codecov.io/gh/apache/kylin/pull/1615?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#1615 +/- ## - Coverage 25.42% 25.34% -0.08% + Complexity 6765 6760 -5 Files 1508 1508 Lines 9391894159 +241 Branches 1315813197 +39 - Hits 2387723865 -12 - Misses6766267913 +251 - Partials 2379 2381 +2 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1615?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==) | `12.43% <0.00%> (-0.04%)` | `51.00 <0.00> (ø)` | | | [...in/java/org/apache/kylin/cube/CubeDescManager.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL0N1YmVEZXNjTWFuYWdlci5qYXZh) | `32.41% <0.00%> (-1.16%)` | `12.00 <0.00> (ø)` | | | [...c/main/java/org/apache/kylin/cube/CubeManager.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL0N1YmVNYW5hZ2VyLmphdmE=) | `35.83% <0.00%> (-0.81%)` | `44.00 <0.00> (ø)` | | | [...c/main/java/org/apache/kylin/cube/CubeSegment.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL0N1YmVTZWdtZW50LmphdmE=) | `41.97% <0.00%> (-0.31%)` | `65.00 <0.00> (ø)` | | | [...ain/java/org/apache/kylin/cube/model/CubeDesc.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL21vZGVsL0N1YmVEZXNjLmphdmE=) | `57.02% <0.00%> (-1.34%)` | `182.00 <0.00> (ø)` | | | [...che/kylin/storage/gtrecord/CubeTupleConverter.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-Y29yZS1zdG9yYWdlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zdG9yYWdlL2d0cmVjb3JkL0N1YmVUdXBsZUNvbnZlcnRlci5qYXZh) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [...g/apache/kylin/rest/controller/CubeController.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvY29udHJvbGxlci9DdWJlQ29udHJvbGxlci5qYXZh) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [...e/kylin/rest/controller/StreamingV2Controller.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvY29udHJvbGxlci9TdHJlYW1pbmdWMkNvbnRyb2xsZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [.../apache/kylin/rest/controller/TableController.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvY29udHJvbGxlci9UYWJsZUNvbnRyb2xsZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [...apache/kylin/rest/metrics/QueryMetrics2Facade.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvbWV0cmljcy9RdWVyeU1ldHJpY3MyRmFjYWRlLmphdmE=) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | ... and [29 more](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1615?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/1615?src=pr=footer). Last update
[GitHub] [kylin] codecov-io commented on pull request #1615: [KYLIN-4939] Transform lookup table snapshot from segment level to cu…
codecov-io commented on pull request #1615: URL: https://github.com/apache/kylin/pull/1615#issuecomment-809054507 # [Codecov](https://codecov.io/gh/apache/kylin/pull/1615?src=pr=h1) Report > Merging [#1615](https://codecov.io/gh/apache/kylin/pull/1615?src=pr=desc) (b733593) into [master](https://codecov.io/gh/apache/kylin/commit/02500818d85955b1125340eaa19ab88a634a5181?el=desc) (0250081) will **decrease** coverage by `0.07%`. > The diff coverage is `4.80%`. [](https://codecov.io/gh/apache/kylin/pull/1615?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#1615 +/- ## - Coverage 25.42% 25.34% -0.08% + Complexity 6765 6760 -5 Files 1508 1508 Lines 9391894159 +241 Branches 1315813197 +39 - Hits 2387723865 -12 - Misses6766267913 +251 - Partials 2379 2381 +2 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/1615?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==) | `12.43% <0.00%> (-0.04%)` | `51.00 <0.00> (ø)` | | | [...in/java/org/apache/kylin/cube/CubeDescManager.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL0N1YmVEZXNjTWFuYWdlci5qYXZh) | `32.41% <0.00%> (-1.16%)` | `12.00 <0.00> (ø)` | | | [...c/main/java/org/apache/kylin/cube/CubeManager.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL0N1YmVNYW5hZ2VyLmphdmE=) | `35.83% <0.00%> (-0.81%)` | `44.00 <0.00> (ø)` | | | [...c/main/java/org/apache/kylin/cube/CubeSegment.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL0N1YmVTZWdtZW50LmphdmE=) | `41.97% <0.00%> (-0.31%)` | `65.00 <0.00> (ø)` | | | [...ain/java/org/apache/kylin/cube/model/CubeDesc.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL21vZGVsL0N1YmVEZXNjLmphdmE=) | `57.02% <0.00%> (-1.34%)` | `182.00 <0.00> (ø)` | | | [...che/kylin/storage/gtrecord/CubeTupleConverter.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-Y29yZS1zdG9yYWdlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zdG9yYWdlL2d0cmVjb3JkL0N1YmVUdXBsZUNvbnZlcnRlci5qYXZh) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [...g/apache/kylin/rest/controller/CubeController.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvY29udHJvbGxlci9DdWJlQ29udHJvbGxlci5qYXZh) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [...e/kylin/rest/controller/StreamingV2Controller.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvY29udHJvbGxlci9TdHJlYW1pbmdWMkNvbnRyb2xsZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [.../apache/kylin/rest/controller/TableController.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvY29udHJvbGxlci9UYWJsZUNvbnRyb2xsZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | [...apache/kylin/rest/metrics/QueryMetrics2Facade.java](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvbWV0cmljcy9RdWVyeU1ldHJpY3MyRmFjYWRlLmphdmE=) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | | | ... and [29 more](https://codecov.io/gh/apache/kylin/pull/1615/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/1615?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/1615?src=pr=footer). Last update [49d7b9b...b733593](https://codecov.io/gh/apache/kylin/pull/1615?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). -- This is an automated message from the Apache Git Service. To respond to the
[jira] [Commented] (KYLIN-4939) Transform lookup table snapshot from segment level to cube level
[
https://issues.apache.org/jira/browse/KYLIN-4939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17310382#comment-17310382
]
ASF GitHub Bot commented on KYLIN-4939:
---
Ted-Jiang commented on a change in pull request #1615:
URL: https://github.com/apache/kylin/pull/1615#discussion_r603002498
##
File path: core-cube/src/main/java/org/apache/kylin/cube/model/CubeDesc.java
##
@@ -623,6 +624,12 @@ public boolean consistentWith(CubeDesc another) {
return this.calculateSignature().equals(another.calculateSignature());
}
+public CubeDesc lastedCopyForWrite() {
Review comment:
thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Transform lookup table snapshot from segment level to cube level
>
>
> Key: KYLIN-4939
> URL: https://issues.apache.org/jira/browse/KYLIN-4939
> Project: Kylin
> Issue Type: New Feature
>Affects Versions: v3.1.2
>Reporter: JiangYang
>Assignee: JiangYang
>Priority: Major
> Fix For: v3.1.2
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] Ted-Jiang commented on a change in pull request #1615: [KYLIN-4939] Transform lookup table snapshot from segment level to cu…
Ted-Jiang commented on a change in pull request #1615:
URL: https://github.com/apache/kylin/pull/1615#discussion_r603002498
##
File path: core-cube/src/main/java/org/apache/kylin/cube/model/CubeDesc.java
##
@@ -623,6 +624,12 @@ public boolean consistentWith(CubeDesc another) {
return this.calculateSignature().equals(another.calculateSignature());
}
+public CubeDesc lastedCopyForWrite() {
Review comment:
thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (KYLIN-4939) Transform lookup table snapshot from segment level to cube level
[
https://issues.apache.org/jira/browse/KYLIN-4939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17310381#comment-17310381
]
ASF GitHub Bot commented on KYLIN-4939:
---
Ted-Jiang commented on a change in pull request #1615:
URL: https://github.com/apache/kylin/pull/1615#discussion_r603002314
##
File path:
server-base/src/main/java/org/apache/kylin/rest/controller/CubeController.java
##
@@ -346,6 +346,35 @@ public JobInstance rebuildLookupSnapshot(@PathVariable
String cubeName,
}
}
+/**
+ * Force change a cube's lookup table to be global
+ *
+ *@throws IOException
+ */
+@RequestMapping(value = "/{cubeNames}/{tableName}/change_lookup_global",
method = {
+RequestMethod.PUT }, produces = { "application/json" })
+@ResponseBody
+public List globalLookupSnapshot(@PathVariable String
cubeNames, @PathVariable String tableName) {
+
+List result = new ArrayList<>();
+
+final CubeManager cubeMgr = cubeService.getCubeManager();
+String[] changeCubes = cubeNames.toUpperCase(Locale.ROOT).split(",");
+for (String cubeName : changeCubes) {
+try {
+checkCubeExists(cubeName);
+final CubeInstance cube = cubeMgr.getCube(cubeName);
+CubeInstance cubeInstance =
cubeService.changeLookupSnapshotBeGlobal(cube, tableName);
+logger.info("cube %s change snapshotTable %s global Success",
cubeName, tableName);
Review comment:
fix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Transform lookup table snapshot from segment level to cube level
>
>
> Key: KYLIN-4939
> URL: https://issues.apache.org/jira/browse/KYLIN-4939
> Project: Kylin
> Issue Type: New Feature
>Affects Versions: v3.1.2
>Reporter: JiangYang
>Assignee: JiangYang
>Priority: Major
> Fix For: v3.1.2
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4780) Refactor AbstractExecutable to extract the notification relevant business logic into a single component
[ https://issues.apache.org/jira/browse/KYLIN-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17310379#comment-17310379 ] ASF GitHub Bot commented on KYLIN-4780: --- lgtm-com[bot] commented on pull request #1596: URL: https://github.com/apache/kylin/pull/1596#issuecomment-809049765 This pull request **introduces 3 alerts** when merging 439fb975937b9ab420cb66446f40f8b72758af23 into 5be135c61c80b9241ac7632dec41cc7209b9d23a - [view on LGTM.com](https://lgtm.com/projects/g/apache/kylin/rev/pr-5bcc02bae5a777344622193c229c2bbd00a33813) **new alerts:** * 3 for Dereferenced variable may be null -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Refactor AbstractExecutable to extract the notification relevant business > logic into a single component > --- > > Key: KYLIN-4780 > URL: https://issues.apache.org/jira/browse/KYLIN-4780 > Project: Kylin > Issue Type: Improvement > Components: Integration >Reporter: vinoyang >Assignee: chenjie >Priority: Major > > Currently, Kylin supports mail service to know the change around the cube. > However, there are some issues about the design, e.g. > * AbstractExecutable contains an implementation of the mechanism of a > notification listener, but only support mail service, it does not belong > there; > * The current implementation is hard to scale to other notification e.g. > Ding Talk(which is a popular production supports by Alibaba) > I propose to refactor the current implementation to extract the notification > logic and redesign a new common notification/listener -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] Ted-Jiang commented on a change in pull request #1615: [KYLIN-4939] Transform lookup table snapshot from segment level to cu…
Ted-Jiang commented on a change in pull request #1615:
URL: https://github.com/apache/kylin/pull/1615#discussion_r603002314
##
File path:
server-base/src/main/java/org/apache/kylin/rest/controller/CubeController.java
##
@@ -346,6 +346,35 @@ public JobInstance rebuildLookupSnapshot(@PathVariable
String cubeName,
}
}
+/**
+ * Force change a cube's lookup table to be global
+ *
+ *@throws IOException
+ */
+@RequestMapping(value = "/{cubeNames}/{tableName}/change_lookup_global",
method = {
+RequestMethod.PUT }, produces = { "application/json" })
+@ResponseBody
+public List globalLookupSnapshot(@PathVariable String
cubeNames, @PathVariable String tableName) {
+
+List result = new ArrayList<>();
+
+final CubeManager cubeMgr = cubeService.getCubeManager();
+String[] changeCubes = cubeNames.toUpperCase(Locale.ROOT).split(",");
+for (String cubeName : changeCubes) {
+try {
+checkCubeExists(cubeName);
+final CubeInstance cube = cubeMgr.getCube(cubeName);
+CubeInstance cubeInstance =
cubeService.changeLookupSnapshotBeGlobal(cube, tableName);
+logger.info("cube %s change snapshotTable %s global Success",
cubeName, tableName);
Review comment:
fix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kylin] lgtm-com[bot] commented on pull request #1596: [KYLIN-4780] support dingtalk notify
lgtm-com[bot] commented on pull request #1596: URL: https://github.com/apache/kylin/pull/1596#issuecomment-809049765 This pull request **introduces 3 alerts** when merging 439fb975937b9ab420cb66446f40f8b72758af23 into 5be135c61c80b9241ac7632dec41cc7209b9d23a - [view on LGTM.com](https://lgtm.com/projects/g/apache/kylin/rev/pr-5bcc02bae5a777344622193c229c2bbd00a33813) **new alerts:** * 3 for Dereferenced variable may be null -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4938) remove segment by UUID
[ https://issues.apache.org/jira/browse/KYLIN-4938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17310375#comment-17310375 ] ASF GitHub Bot commented on KYLIN-4938: --- coveralls edited a comment on pull request #1616: URL: https://github.com/apache/kylin/pull/1616#issuecomment-802727651 ## Pull Request Test Coverage Report for [Build 7009](https://coveralls.io/builds/38322042) * **0** of **22** **(0.0%)** changed or added relevant lines in **3** files are covered. * **11** unchanged lines in **2** files lost coverage. * Overall coverage decreased (**-0.01%**) to **28.11%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [server-base/src/main/java/org/apache/kylin/rest/msg/Message.java](https://coveralls.io/builds/38322042/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fmsg%2FMessage.java#L69) | 0 | 1 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/controller/CubeController.java](https://coveralls.io/builds/38322042/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fcontroller%2FCubeController.java#L383) | 0 | 9 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/service/CubeService.java](https://coveralls.io/builds/38322042/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FCubeService.java#L614) | 0 | 12 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/CheckPointStore.java](https://coveralls.io/builds/38322042/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2FCheckPointStore.java#L98) | 2 | 73.74% | | [core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/MetricsSystem.java](https://coveralls.io/builds/38322042/source?filename=core-metrics%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetrics%2Flib%2Fimpl%2FMetricsSystem.java#L46) | 9 | 60.0% | | Totals | [](https://coveralls.io/builds/38322042) | | :-- | --: | | Change from base [Build 6998](https://coveralls.io/builds/38225952): | -0.01% | | Covered Lines: | 26696 | | Relevant Lines: | 94971 | --- # - [Coveralls](https://coveralls.io) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > remove segment by UUID > --- > > Key: KYLIN-4938 > URL: https://issues.apache.org/jira/browse/KYLIN-4938 > Project: Kylin > Issue Type: New Feature > Components: Job Engine >Affects Versions: v3.1.2 >Reporter: JiangYang >Assignee: JiangYang >Priority: Major > Fix For: v3.1.2 > > > Sometimes two segment have the same name, need remove segment by UUID . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] coveralls edited a comment on pull request #1616: [KYLIN-4938] remove segment by UUID
coveralls edited a comment on pull request #1616: URL: https://github.com/apache/kylin/pull/1616#issuecomment-802727651 ## Pull Request Test Coverage Report for [Build 7009](https://coveralls.io/builds/38322042) * **0** of **22** **(0.0%)** changed or added relevant lines in **3** files are covered. * **11** unchanged lines in **2** files lost coverage. * Overall coverage decreased (**-0.01%**) to **28.11%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [server-base/src/main/java/org/apache/kylin/rest/msg/Message.java](https://coveralls.io/builds/38322042/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fmsg%2FMessage.java#L69) | 0 | 1 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/controller/CubeController.java](https://coveralls.io/builds/38322042/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fcontroller%2FCubeController.java#L383) | 0 | 9 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/service/CubeService.java](https://coveralls.io/builds/38322042/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FCubeService.java#L614) | 0 | 12 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [stream-core/src/main/java/org/apache/kylin/stream/core/storage/CheckPointStore.java](https://coveralls.io/builds/38322042/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fstorage%2FCheckPointStore.java#L98) | 2 | 73.74% | | [core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/MetricsSystem.java](https://coveralls.io/builds/38322042/source?filename=core-metrics%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetrics%2Flib%2Fimpl%2FMetricsSystem.java#L46) | 9 | 60.0% | | Totals | [](https://coveralls.io/builds/38322042) | | :-- | --: | | Change from base [Build 6998](https://coveralls.io/builds/38225952): | -0.01% | | Covered Lines: | 26696 | | Relevant Lines: | 94971 | --- # - [Coveralls](https://coveralls.io) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4780) Refactor AbstractExecutable to extract the notification relevant business logic into a single component
[ https://issues.apache.org/jira/browse/KYLIN-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17310367#comment-17310367 ] ASF GitHub Bot commented on KYLIN-4780: --- coveralls edited a comment on pull request #1596: URL: https://github.com/apache/kylin/pull/1596#issuecomment-791204373 ## Pull Request Test Coverage Report for [Build 7008](https://coveralls.io/builds/38321943) * **55** of **322** **(17.08%)** changed or added relevant lines in **21** files are covered. * **1106** unchanged lines in **27** files lost coverage. * Overall coverage increased (+**0.2%**) to **28.108%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-common/src/main/java/org/apache/kylin/common/notify/util/DingTalkNotificationUtil.java](https://coveralls.io/builds/38321943/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fnotify%2Futil%2FDingTalkNotificationUtil.java#L25) | 3 | 4 | 75.0% | [core-common/src/main/java/org/apache/kylin/common/notify/util/NotificationConstants.java](https://coveralls.io/builds/38321943/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fnotify%2Futil%2FNotificationConstants.java#L21) | 0 | 1 | 0.0% | [engine-mr/src/main/java/org/apache/kylin/engine/mr/LookupSnapshotBuildJob.java](https://coveralls.io/builds/38321943/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2FLookupSnapshotBuildJob.java#L69) | 0 | 2 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/constant/NotifyType.java](https://coveralls.io/builds/38321943/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fconstant%2FNotifyType.java#L21) | 0 | 2 | 0.0% | [core-cube/src/main/java/org/apache/kylin/cube/model/CubeDesc.java](https://coveralls.io/builds/38321943/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fmodel%2FCubeDesc.java#L536) | 10 | 13 | 76.92% | [core-common/src/main/java/org/apache/kylin/common/notify/util/DingTalkTemplateProvider.java](https://coveralls.io/builds/38321943/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fnotify%2Futil%2FDingTalkTemplateProvider.java#L50) | 12 | 16 | 75.0% | [core-common/src/main/java/org/apache/kylin/common/notify/util/SecretKeyUtil.java](https://coveralls.io/builds/38321943/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fnotify%2Futil%2FSecretKeyUtil.java#L30) | 8 | 12 | 66.67% | [core-common/src/main/java/org/apache/kylin/common/notify/NotifyServiceBase.java](https://coveralls.io/builds/38321943/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fnotify%2FNotifyServiceBase.java#L31) | 0 | 5 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/job/KylinHealthCheckJob.java](https://coveralls.io/builds/38321943/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fjob%2FKylinHealthCheckJob.java#L140) | 0 | 5 | 0.0% | [cube-migration/src/main/java/org/apache/kylin/rest/service/MigrationRuleSet.java](https://coveralls.io/builds/38321943/source?filename=cube-migration%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FMigrationRuleSet.java#L84) | 0 | 6 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [cube-migration/src/main/java/org/apache/kylin/rest/service/MigrationService.java](https://coveralls.io/builds/38321943/source?filename=cube-migration%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FMigrationService.java#L236) | 1 | 0% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/CubingJob.java](https://coveralls.io/builds/38321943/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2FCubingJob.java#L167) | 1 | 1.48% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/LookupSnapshotBuildJob.java](https://coveralls.io/builds/38321943/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2FLookupSnapshotBuildJob.java#L71) | 1 | 0% | | [server-base/src/main/java/org/apache/kylin/rest/job/KylinHealthCheckJob.java](https://coveralls.io/builds/38321943/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fjob%2FKylinHealthCheckJob.java#L143) | 1 | 0% | | [core-job/src/main/java/org/apache/kylin/job/impl/threadpool/DefaultScheduler.java](https://coveralls.io/builds/38321943/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fthreadpool%2FDefaultScheduler.java#L193) | 2 | 79.07% | |
[GitHub] [kylin] coveralls edited a comment on pull request #1596: [KYLIN-4780] support dingtalk notify
coveralls edited a comment on pull request #1596: URL: https://github.com/apache/kylin/pull/1596#issuecomment-791204373 ## Pull Request Test Coverage Report for [Build 7008](https://coveralls.io/builds/38321943) * **55** of **322** **(17.08%)** changed or added relevant lines in **21** files are covered. * **1106** unchanged lines in **27** files lost coverage. * Overall coverage increased (+**0.2%**) to **28.108%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-common/src/main/java/org/apache/kylin/common/notify/util/DingTalkNotificationUtil.java](https://coveralls.io/builds/38321943/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fnotify%2Futil%2FDingTalkNotificationUtil.java#L25) | 3 | 4 | 75.0% | [core-common/src/main/java/org/apache/kylin/common/notify/util/NotificationConstants.java](https://coveralls.io/builds/38321943/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fnotify%2Futil%2FNotificationConstants.java#L21) | 0 | 1 | 0.0% | [engine-mr/src/main/java/org/apache/kylin/engine/mr/LookupSnapshotBuildJob.java](https://coveralls.io/builds/38321943/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2FLookupSnapshotBuildJob.java#L69) | 0 | 2 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/constant/NotifyType.java](https://coveralls.io/builds/38321943/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fconstant%2FNotifyType.java#L21) | 0 | 2 | 0.0% | [core-cube/src/main/java/org/apache/kylin/cube/model/CubeDesc.java](https://coveralls.io/builds/38321943/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fmodel%2FCubeDesc.java#L536) | 10 | 13 | 76.92% | [core-common/src/main/java/org/apache/kylin/common/notify/util/DingTalkTemplateProvider.java](https://coveralls.io/builds/38321943/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fnotify%2Futil%2FDingTalkTemplateProvider.java#L50) | 12 | 16 | 75.0% | [core-common/src/main/java/org/apache/kylin/common/notify/util/SecretKeyUtil.java](https://coveralls.io/builds/38321943/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fnotify%2Futil%2FSecretKeyUtil.java#L30) | 8 | 12 | 66.67% | [core-common/src/main/java/org/apache/kylin/common/notify/NotifyServiceBase.java](https://coveralls.io/builds/38321943/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fnotify%2FNotifyServiceBase.java#L31) | 0 | 5 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/job/KylinHealthCheckJob.java](https://coveralls.io/builds/38321943/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fjob%2FKylinHealthCheckJob.java#L140) | 0 | 5 | 0.0% | [cube-migration/src/main/java/org/apache/kylin/rest/service/MigrationRuleSet.java](https://coveralls.io/builds/38321943/source?filename=cube-migration%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FMigrationRuleSet.java#L84) | 0 | 6 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [cube-migration/src/main/java/org/apache/kylin/rest/service/MigrationService.java](https://coveralls.io/builds/38321943/source?filename=cube-migration%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FMigrationService.java#L236) | 1 | 0% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/CubingJob.java](https://coveralls.io/builds/38321943/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2FCubingJob.java#L167) | 1 | 1.48% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/LookupSnapshotBuildJob.java](https://coveralls.io/builds/38321943/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2FLookupSnapshotBuildJob.java#L71) | 1 | 0% | | [server-base/src/main/java/org/apache/kylin/rest/job/KylinHealthCheckJob.java](https://coveralls.io/builds/38321943/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fjob%2FKylinHealthCheckJob.java#L143) | 1 | 0% | | [core-job/src/main/java/org/apache/kylin/job/impl/threadpool/DefaultScheduler.java](https://coveralls.io/builds/38321943/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fthreadpool%2FDefaultScheduler.java#L193) | 2 | 79.07% | | [cube-migration/src/main/java/org/apache/kylin/rest/service/MigrationRuleSet.java](https://coveralls.io/builds/38321943/source?filename=cube-migration%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FMigrationRuleSet.java#L216) | 2 | 0% | |
[jira] [Commented] (KYLIN-4938) remove segment by UUID
[
https://issues.apache.org/jira/browse/KYLIN-4938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17310356#comment-17310356
]
ASF GitHub Bot commented on KYLIN-4938:
---
Ted-Jiang commented on a change in pull request #1616:
URL: https://github.com/apache/kylin/pull/1616#discussion_r602995368
##
File path:
server-base/src/main/java/org/apache/kylin/rest/service/CubeService.java
##
@@ -610,6 +610,32 @@ public CubeInstance rebuildLookupSnapshot(CubeInstance
cube, String segmentName,
return cube;
}
+public CubeInstance deleteSegmentById(CubeInstance cube, String uuid)
throws IOException {
+aclEvaluate.checkProjectWritePermission(cube);
+Message msg = MsgPicker.getMsg();
+
+CubeSegment toDelete = null;
+
+toDelete = cube.getSegmentById(uuid);
+
+if (toDelete == null) {
+throw new BadRequestException(String.format(Locale.ROOT,
msg.getSEG_NOT_FOUND(), uuid));
+}
+
+if (cube.getStatus() == RealizationStatusEnum.DISABLED
+|| (cube.getStatus() == RealizationStatusEnum.READY &&
toDelete.getStatus() != SegmentStatusEnum.READY)) {
Review comment:
good catch! fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> remove segment by UUID
> ---
>
> Key: KYLIN-4938
> URL: https://issues.apache.org/jira/browse/KYLIN-4938
> Project: Kylin
> Issue Type: New Feature
> Components: Job Engine
>Affects Versions: v3.1.2
>Reporter: JiangYang
>Assignee: JiangYang
>Priority: Major
> Fix For: v3.1.2
>
>
> Sometimes two segment have the same name, need remove segment by UUID .
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4938) remove segment by UUID
[
https://issues.apache.org/jira/browse/KYLIN-4938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17310357#comment-17310357
]
ASF GitHub Bot commented on KYLIN-4938:
---
Ted-Jiang commented on a change in pull request #1616:
URL: https://github.com/apache/kylin/pull/1616#discussion_r602995448
##
File path: server-base/src/main/java/org/apache/kylin/rest/msg/Message.java
##
@@ -65,6 +65,10 @@ public String getDELETE_NOT_READY_SEG() {
return "Cannot delete segment '%s' as its status is not READY. Discard
the on-going job for it.";
}
+public String getDELETE_READY_SEG_BY_UUID() {
+return "Cannot delete segment by UUID '%s' as its status is READY or
its Cube is READY. ";
Review comment:
fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> remove segment by UUID
> ---
>
> Key: KYLIN-4938
> URL: https://issues.apache.org/jira/browse/KYLIN-4938
> Project: Kylin
> Issue Type: New Feature
> Components: Job Engine
>Affects Versions: v3.1.2
>Reporter: JiangYang
>Assignee: JiangYang
>Priority: Major
> Fix For: v3.1.2
>
>
> Sometimes two segment have the same name, need remove segment by UUID .
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] Ted-Jiang commented on a change in pull request #1616: [KYLIN-4938] remove segment by UUID
Ted-Jiang commented on a change in pull request #1616:
URL: https://github.com/apache/kylin/pull/1616#discussion_r602995448
##
File path: server-base/src/main/java/org/apache/kylin/rest/msg/Message.java
##
@@ -65,6 +65,10 @@ public String getDELETE_NOT_READY_SEG() {
return "Cannot delete segment '%s' as its status is not READY. Discard
the on-going job for it.";
}
+public String getDELETE_READY_SEG_BY_UUID() {
+return "Cannot delete segment by UUID '%s' as its status is READY or
its Cube is READY. ";
Review comment:
fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kylin] Ted-Jiang commented on a change in pull request #1616: [KYLIN-4938] remove segment by UUID
Ted-Jiang commented on a change in pull request #1616:
URL: https://github.com/apache/kylin/pull/1616#discussion_r602995368
##
File path:
server-base/src/main/java/org/apache/kylin/rest/service/CubeService.java
##
@@ -610,6 +610,32 @@ public CubeInstance rebuildLookupSnapshot(CubeInstance
cube, String segmentName,
return cube;
}
+public CubeInstance deleteSegmentById(CubeInstance cube, String uuid)
throws IOException {
+aclEvaluate.checkProjectWritePermission(cube);
+Message msg = MsgPicker.getMsg();
+
+CubeSegment toDelete = null;
+
+toDelete = cube.getSegmentById(uuid);
+
+if (toDelete == null) {
+throw new BadRequestException(String.format(Locale.ROOT,
msg.getSEG_NOT_FOUND(), uuid));
+}
+
+if (cube.getStatus() == RealizationStatusEnum.DISABLED
+|| (cube.getStatus() == RealizationStatusEnum.READY &&
toDelete.getStatus() != SegmentStatusEnum.READY)) {
Review comment:
good catch! fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (KYLIN-4936) Exactly aggregation can't transform to project
[ https://issues.apache.org/jira/browse/KYLIN-4936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17310226#comment-17310226 ] ASF GitHub Bot commented on KYLIN-4936: --- zhengshengjun commented on pull request #1619: URL: https://github.com/apache/kylin/pull/1619#issuecomment-808908852 Thanks for solving this problem we've met. I have tested and it perfectly addressed our problem. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Exactly aggregation can't transform to project > -- > > Key: KYLIN-4936 > URL: https://issues.apache.org/jira/browse/KYLIN-4936 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Affects Versions: v4.0.0-beta >Reporter: ShengJun Zheng >Assignee: Zhichao Zhang >Priority: Major > Fix For: v4.0.0-GA > > > Exactly Aggregate can't transform to project, causing unnecessary spark > shuffle ! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kylin] zhengshengjun commented on pull request #1619: KYLIN-4936 Exactly aggregation can't transform to project
zhengshengjun commented on pull request #1619: URL: https://github.com/apache/kylin/pull/1619#issuecomment-808908852 Thanks for solving this problem we've met. I have tested and it perfectly addressed our problem. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-4936) Exactly aggregation can't transform to project
[
https://issues.apache.org/jira/browse/KYLIN-4936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17310223#comment-17310223
]
ASF GitHub Bot commented on KYLIN-4936:
---
zhengshengjun commented on a change in pull request #1619:
URL: https://github.com/apache/kylin/pull/1619#discussion_r602889485
##
File path:
kylin-spark-project/kylin-spark-query/src/main/scala/org/apache/kylin/query/runtime/plans/AggregatePlan.scala
##
@@ -53,10 +53,32 @@ object AggregatePlan extends LogEx {
.map(groupId => col(schemaNames.apply(groupId)))
.toList
-if (rel.getContext != null && rel.getContext.isExactlyAggregate) {
+rel.getContext.isExactlyAggregate = isExactlyCuboidMatched(rel, groupList)
+if (rel.getContext.isExactlyAggregate) {
// exactly match, skip agg, direct project.
- val aggCols = rel.getRewriteAggCalls.asScala
-.map(call => col(schemaNames.apply(call.getArgList.get(0.toList
+ val hash = System.identityHashCode(rel).toString
+ val aggCols = rel.getRewriteAggCalls.asScala.zipWithIndex.map {
+case (call: KylinAggregateCall, index: Int)
+ if OLAPAggregateRel.getAggrFuncName(call).equals("COUNT_DISTINCT") =>
+ val dataType = call.getFunc.getReturnDataType
+ val funcName = OLAPAggregateRel.getAggrFuncName(call)
+ val argNames =
call.getArgList.asScala.map(dataFrame.schema.names.apply(_))
+ val columnName = argNames.map(col)
+ val aggName = SchemaProcessor.replaceToAggravateSchemaName(index,
funcName, hash, argNames: _*)
+ if (dataType.getName == "hllc") {
Review comment:
What about adding a 'isHLLCountDistinct' method in FunctionDesc,
sometimes I have trouble judging whether this count_distinct function is HLL or
precise count distinct ==
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Exactly aggregation can't transform to project
> --
>
> Key: KYLIN-4936
> URL: https://issues.apache.org/jira/browse/KYLIN-4936
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Affects Versions: v4.0.0-beta
>Reporter: ShengJun Zheng
>Assignee: Zhichao Zhang
>Priority: Major
> Fix For: v4.0.0-GA
>
>
> Exactly Aggregate can't transform to project, causing unnecessary spark
> shuffle !
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] zhengshengjun commented on a change in pull request #1619: KYLIN-4936 Exactly aggregation can't transform to project
zhengshengjun commented on a change in pull request #1619:
URL: https://github.com/apache/kylin/pull/1619#discussion_r602889485
##
File path:
kylin-spark-project/kylin-spark-query/src/main/scala/org/apache/kylin/query/runtime/plans/AggregatePlan.scala
##
@@ -53,10 +53,32 @@ object AggregatePlan extends LogEx {
.map(groupId => col(schemaNames.apply(groupId)))
.toList
-if (rel.getContext != null && rel.getContext.isExactlyAggregate) {
+rel.getContext.isExactlyAggregate = isExactlyCuboidMatched(rel, groupList)
+if (rel.getContext.isExactlyAggregate) {
// exactly match, skip agg, direct project.
- val aggCols = rel.getRewriteAggCalls.asScala
-.map(call => col(schemaNames.apply(call.getArgList.get(0.toList
+ val hash = System.identityHashCode(rel).toString
+ val aggCols = rel.getRewriteAggCalls.asScala.zipWithIndex.map {
+case (call: KylinAggregateCall, index: Int)
+ if OLAPAggregateRel.getAggrFuncName(call).equals("COUNT_DISTINCT") =>
+ val dataType = call.getFunc.getReturnDataType
+ val funcName = OLAPAggregateRel.getAggrFuncName(call)
+ val argNames =
call.getArgList.asScala.map(dataFrame.schema.names.apply(_))
+ val columnName = argNames.map(col)
+ val aggName = SchemaProcessor.replaceToAggravateSchemaName(index,
funcName, hash, argNames: _*)
+ if (dataType.getName == "hllc") {
Review comment:
What about adding a 'isHLLCountDistinct' method in FunctionDesc,
sometimes I have trouble judging whether this count_distinct function is HLL or
precise count distinct ==
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (KYLIN-4936) Exactly aggregation can't transform to project
[
https://issues.apache.org/jira/browse/KYLIN-4936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17310221#comment-17310221
]
ASF GitHub Bot commented on KYLIN-4936:
---
zhengshengjun commented on a change in pull request #1619:
URL: https://github.com/apache/kylin/pull/1619#discussion_r602889056
##
File path:
kylin-spark-project/kylin-spark-query/src/main/scala/org/apache/kylin/query/runtime/plans/AggregatePlan.scala
##
@@ -115,7 +137,7 @@ object AggregatePlan extends LogEx {
val aggName = SchemaProcessor.replaceToAggravateSchemaName(index,
funcName, hash, argNames: _*)
if (funcName == FunctionDesc.FUNC_COUNT_DISTINCT) {
Review comment:
should we replace "funcName == FunctionDesc.FUNC_COUNT_DISTINCT" with
call.getFunc.isCountDistinct ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Exactly aggregation can't transform to project
> --
>
> Key: KYLIN-4936
> URL: https://issues.apache.org/jira/browse/KYLIN-4936
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Affects Versions: v4.0.0-beta
>Reporter: ShengJun Zheng
>Assignee: Zhichao Zhang
>Priority: Major
> Fix For: v4.0.0-GA
>
>
> Exactly Aggregate can't transform to project, causing unnecessary spark
> shuffle !
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kylin] zhengshengjun commented on a change in pull request #1619: KYLIN-4936 Exactly aggregation can't transform to project
zhengshengjun commented on a change in pull request #1619:
URL: https://github.com/apache/kylin/pull/1619#discussion_r602889056
##
File path:
kylin-spark-project/kylin-spark-query/src/main/scala/org/apache/kylin/query/runtime/plans/AggregatePlan.scala
##
@@ -115,7 +137,7 @@ object AggregatePlan extends LogEx {
val aggName = SchemaProcessor.replaceToAggravateSchemaName(index,
funcName, hash, argNames: _*)
if (funcName == FunctionDesc.FUNC_COUNT_DISTINCT) {
Review comment:
should we replace "funcName == FunctionDesc.FUNC_COUNT_DISTINCT" with
call.getFunc.isCountDistinct ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
