[GitHub] [kylin] asfgit commented on issue #615: KYLIN-3958 MrHive-Dict support build by livy
asfgit commented on issue #615: KYLIN-3958 MrHive-Dict support build by livy URL: https://github.com/apache/kylin/pull/615#issuecomment-485341639 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3958) MrHive-Dict support build by livy
[ https://issues.apache.org/jira/browse/KYLIN-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822900#comment-16822900 ] ASF GitHub Bot commented on KYLIN-3958: --- javalife0312 commented on pull request #615: KYLIN-3958 MrHive-Dict support build by livy URL: https://github.com/apache/kylin/pull/615 https://issues.apache.org/jira/browse/KYLIN-3958?filter=-1 build steps： 1、Build Global Dict - extract distinct 2、Build Global Dict - merge to dict table 3、Build Global Dict - replace 4、Redistribute Flat Hive Table by livy if 【kylin.engine.livy-conf.livy-enabled=true】 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > MrHive-Dict support build by livy > - > > Key: KYLIN-3958 > URL: https://issues.apache.org/jira/browse/KYLIN-3958 > Project: Kylin > Issue Type: Improvement >Reporter: jinguowei >Assignee: jinguowei >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3958) MrHive-Dict support build by livy
[ https://issues.apache.org/jira/browse/KYLIN-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822899#comment-16822899 ] jinguowei commented on KYLIN-3958: -- 4、Redistribute Flat Hive Table > MrHive-Dict support build by livy > - > > Key: KYLIN-3958 > URL: https://issues.apache.org/jira/browse/KYLIN-3958 > Project: Kylin > Issue Type: Improvement >Reporter: jinguowei >Assignee: jinguowei >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] javalife0312 opened a new pull request #615: KYLIN-3958 MrHive-Dict support build by livy
javalife0312 opened a new pull request #615: KYLIN-3958 MrHive-Dict support build by livy URL: https://github.com/apache/kylin/pull/615 https://issues.apache.org/jira/browse/KYLIN-3958?filter=-1 build steps： 1、Build Global Dict - extract distinct 2、Build Global Dict - merge to dict table 3、Build Global Dict - replace 4、Redistribute Flat Hive Table by livy if 【kylin.engine.livy-conf.livy-enabled=true】 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (KYLIN-3964) segment overlapped, the status is "NEW", last_build_job_id is null, can not refresh/delete/build this segment
[ https://issues.apache.org/jira/browse/KYLIN-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hejian updated KYLIN-3964: -- Description: *Any action* *involved* *this segment can not be excute by requesting api due to no job_id provided.* !image-2019-04-22-10-07-06-737.png! was: *Any action* *involved* *this segment can not be excute by requesting api due to no job_id provided.* !image-2019-04-22-10-07-06-737.png! > segment overlapped, the status is "NEW", last_build_job_id is null, can not > refresh/delete/build this segment > -- > > Key: KYLIN-3964 > URL: https://issues.apache.org/jira/browse/KYLIN-3964 > Project: Kylin > Issue Type: Bug > Environment: kylin-2.4.0 >Reporter: hejian >Priority: Major > Attachments: image-2019-04-22-10-07-06-737.png > > > *Any action* *involved* *this segment can not be excute by requesting api due > to no job_id provided.* > !image-2019-04-22-10-07-06-737.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3964) segment overlapped, the status is "NEW", last_build_job_id is null, can not refresh/delete/build this segment
[ https://issues.apache.org/jira/browse/KYLIN-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hejian updated KYLIN-3964: -- Description: *Any action* *involved* *this segment can not be excute by requesting api due to no job_id provided.* !image-2019-04-22-10-07-06-737.png! was: *Any action* *involved* *_**_ this segment can not be excute by requesting apis due to no job_id provided.* !image-2019-04-22-10-07-06-737.png! > segment overlapped, the status is "NEW", last_build_job_id is null, can not > refresh/delete/build this segment > -- > > Key: KYLIN-3964 > URL: https://issues.apache.org/jira/browse/KYLIN-3964 > Project: Kylin > Issue Type: Bug > Environment: kylin-2.4.0 >Reporter: hejian >Priority: Major > Attachments: image-2019-04-22-10-07-06-737.png > > > *Any action* *involved* *this segment can not be excute by requesting > api due to no job_id provided.* > !image-2019-04-22-10-07-06-737.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3964) segment overlapped, the status is "NEW", last_build_job_id is null, can not refresh/delete/build this segment
[ https://issues.apache.org/jira/browse/KYLIN-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hejian updated KYLIN-3964: -- Description: *Any action* *involved* *_**_ this segment can not be excute by requesting apis due to no job_id provided.* !image-2019-04-22-10-07-06-737.png! was:!image-2019-04-22-10-07-06-737.png! > segment overlapped, the status is "NEW", last_build_job_id is null, can not > refresh/delete/build this segment > -- > > Key: KYLIN-3964 > URL: https://issues.apache.org/jira/browse/KYLIN-3964 > Project: Kylin > Issue Type: Bug > Environment: kylin-2.4.0 >Reporter: hejian >Priority: Major > Attachments: image-2019-04-22-10-07-06-737.png > > > *Any action* *involved* *_**_ this segment can not be excute by requesting > apis due to no job_id provided.* > !image-2019-04-22-10-07-06-737.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] codecov-io commented on issue #614: Fixed use customized precision
codecov-io commented on issue #614: Fixed use customized precision URL: https://github.com/apache/kylin/pull/614#issuecomment-485321187 # [Codecov](https://codecov.io/gh/apache/kylin/pull/614?src=pr&el=h1) Report > Merging [#614](https://codecov.io/gh/apache/kylin/pull/614?src=pr&el=desc) into [master](https://codecov.io/gh/apache/kylin/commit/426a22711d6e25d2b7a4c1c3c6cea4e642b7f248?src=pr&el=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/kylin/pull/614?src=pr&el=tree) ```diff @@Coverage Diff@@ ## master #614 +/- ## = Coverage 25.34% 25.34% - Complexity 5840 5841+1 = Files 1379 1379 Lines 8178181781 Branches 1145711457 = Hits 2073120731 - Misses5902859029+1 + Partials 2022 2021-1 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/614?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...he/kylin/job/impl/threadpool/DefaultScheduler.java](https://codecov.io/gh/apache/kylin/pull/614/diff?src=pr&el=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL3RocmVhZHBvb2wvRGVmYXVsdFNjaGVkdWxlci5qYXZh) | `74.41% <0%> (-2.33%)` | `12% <0%> (ø)` | | | [...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/614/diff?src=pr&el=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==) | `78.37% <0%> (+5.4%)` | `7% <0%> (+1%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/614?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/614?src=pr&el=footer). Last update [426a227...c479ba5](https://codecov.io/gh/apache/kylin/pull/614?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] coveralls commented on issue #614: Fixed use customized precision
coveralls commented on issue #614: Fixed use customized precision URL: https://github.com/apache/kylin/pull/614#issuecomment-485320584 ## Pull Request Test Coverage Report for [Build 4410](https://coveralls.io/builds/22941025) * **0** of **0** changed or added relevant lines in **0** files are covered. * **2** unchanged lines in **1** file lost coverage. * Overall coverage increased (+**0.001%**) to **27.826%** --- | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-job/src/main/java/org/apache/kylin/job/impl/threadpool/DefaultScheduler.java](https://coveralls.io/builds/22941025/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fthreadpool%2FDefaultScheduler.java#L194) | 2 | 80.23% | | Totals | [](https://coveralls.io/builds/22941025) | | :-- | --: | | Change from base [Build 4405](https://coveralls.io/builds/22907851): | 0.001% | | Covered Lines: | 22756 | | Relevant Lines: | 81781 | --- # 💛 - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3964) segment overlapped, the status is "NEW", last_build_job_id is null, can not refresh/delete/build this segment
[ https://issues.apache.org/jira/browse/KYLIN-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822857#comment-16822857 ] hejian commented on KYLIN-3964: --- As KYLIN-2849 desc, i got the same exception. and a bad segment or a failed job can be caused by many reasons(hbase connection timeout etc.), which can not be excute any action more. Now that the "NEW" job seem inevitable, we should make the actions enabled to reslove ths issue. BTW, I can not determine the final solution in KYLIN-2849. > segment overlapped, the status is "NEW", last_build_job_id is null, can not > refresh/delete/build this segment > -- > > Key: KYLIN-3964 > URL: https://issues.apache.org/jira/browse/KYLIN-3964 > Project: Kylin > Issue Type: Bug > Environment: kylin-2.4.0 >Reporter: hejian >Priority: Major > Attachments: image-2019-04-22-10-07-06-737.png > > > !image-2019-04-22-10-07-06-737.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] majic31 opened a new pull request #614: Fixed use customized precision
majic31 opened a new pull request #614: Fixed use customized precision URL: https://github.com/apache/kylin/pull/614 In the cubeMeasures.js, It will withdraw precision and scale by using Regular Expression. The scale parameter is ok, but precision use the magic number 19. So we fixed it, and we test ok when the column is decimal(38,18). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #614: Fixed use customized precision
asfgit commented on issue #614: Fixed use customized precision URL: https://github.com/apache/kylin/pull/614#issuecomment-485317229 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (KYLIN-3948) org.apache.hadoop.hive.metastore.HiveMetaStoreClint.(Lorg/apache/hadoop/conf/configuration;)V
[ https://issues.apache.org/jira/browse/KYLIN-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822853#comment-16822853 ] praveenece edited comment on KYLIN-3948 at 4/22/19 3:51 AM: Hi Chao Long, Thanks for your update, please let me know which version of kylin support hadoop3 as well as spark engine. was (Author: praveenece): Hi Chao Long, Thanks for your update, please let me know which version of kylin support hadoop3 and spark engine. > org.apache.hadoop.hive.metastore.HiveMetaStoreClint.(Lorg/apache/hadoop/conf/configuration;)V > --- > > Key: KYLIN-3948 > URL: https://issues.apache.org/jira/browse/KYLIN-3948 > Project: Kylin > Issue Type: Test > Components: Environment >Affects Versions: v2.5.2 > Environment: hadoop-3.1.1 > apache-hive-2.3.4-bin > hbase-2.1.1 > apache-kylin-2.5.2-bin-hadoop3 >Reporter: praveenece >Priority: Critical > > hadoop-3.1.1 > apache-hive-2.3.4-bin > hbase-2.1.1 > apache-kylin-2.5.2-bin-hadoop3 > Now I tried with above environment, kylin is stated but i unable to load > metadata from hbase to kylin getting this below error in kylin UI. > org.apache.hadoop.hive.metastore.HiveMetaStoreClint.(Lorg/apache/hadoop/conf/configuration;)V > 2019-04-09 15:02:44,979 DEBUG [http-nio-7070-exec-9] common.KylinConfig:327 : > KYLIN_CONF property was not set, will seek KYLIN_HOME env variable > 2019-04-09 15:02:50,071 ERROR [http-nio-7070-exec-6] > controller.TableController:190 : > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(Lorg/apache/hadoop/conf/Configuration;)V > java.lang.NoSuchMethodError: > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(Lorg/apache/hadoop/conf/Configuration;)V > at > org.apache.kylin.source.hive.CLIHiveClient.getMetaStoreClient(CLIHiveClient.java:132) > at > org.apache.kylin.source.hive.CLIHiveClient.getHiveDbNames(CLIHiveClient.java:116) > at > org.apache.kylin.source.hive.HiveMetadataExplorer.listDatabases(HiveMetadataExplorer.java:47) > at > org.apache.kylin.rest.service.TableService.getSourceDbNames(TableService.java:277) > at > org.apache.kylin.rest.controller.TableController.showHiveDatabases(TableController.java:188) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) > at > org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3948) org.apache.hadoop.hive.metastore.HiveMetaStoreClint.(Lorg/apache/hadoop/conf/configuration;)V
[ https://issues.apache.org/jira/browse/KYLIN-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822853#comment-16822853 ] praveenece commented on KYLIN-3948: --- Hi Chao Long, Thanks for your update, please let me know which version of kylin support hadoop3 and spark engine. > org.apache.hadoop.hive.metastore.HiveMetaStoreClint.(Lorg/apache/hadoop/conf/configuration;)V > --- > > Key: KYLIN-3948 > URL: https://issues.apache.org/jira/browse/KYLIN-3948 > Project: Kylin > Issue Type: Test > Components: Environment >Affects Versions: v2.5.2 > Environment: hadoop-3.1.1 > apache-hive-2.3.4-bin > hbase-2.1.1 > apache-kylin-2.5.2-bin-hadoop3 >Reporter: praveenece >Priority: Critical > > hadoop-3.1.1 > apache-hive-2.3.4-bin > hbase-2.1.1 > apache-kylin-2.5.2-bin-hadoop3 > Now I tried with above environment, kylin is stated but i unable to load > metadata from hbase to kylin getting this below error in kylin UI. > org.apache.hadoop.hive.metastore.HiveMetaStoreClint.(Lorg/apache/hadoop/conf/configuration;)V > 2019-04-09 15:02:44,979 DEBUG [http-nio-7070-exec-9] common.KylinConfig:327 : > KYLIN_CONF property was not set, will seek KYLIN_HOME env variable > 2019-04-09 15:02:50,071 ERROR [http-nio-7070-exec-6] > controller.TableController:190 : > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(Lorg/apache/hadoop/conf/Configuration;)V > java.lang.NoSuchMethodError: > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(Lorg/apache/hadoop/conf/Configuration;)V > at > org.apache.kylin.source.hive.CLIHiveClient.getMetaStoreClient(CLIHiveClient.java:132) > at > org.apache.kylin.source.hive.CLIHiveClient.getHiveDbNames(CLIHiveClient.java:116) > at > org.apache.kylin.source.hive.HiveMetadataExplorer.listDatabases(HiveMetadataExplorer.java:47) > at > org.apache.kylin.rest.service.TableService.getSourceDbNames(TableService.java:277) > at > org.apache.kylin.rest.controller.TableController.showHiveDatabases(TableController.java:188) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) > at > org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3964) segment overlapped, the status is "NEW", last_build_job_id is null, can not refresh/delete/build this segment
[ https://issues.apache.org/jira/browse/KYLIN-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822837#comment-16822837 ] Shaofeng SHI commented on KYLIN-3964: - What kind of error you got? Did you try a newer version? > segment overlapped, the status is "NEW", last_build_job_id is null, can not > refresh/delete/build this segment > -- > > Key: KYLIN-3964 > URL: https://issues.apache.org/jira/browse/KYLIN-3964 > Project: Kylin > Issue Type: Bug > Environment: kylin-2.4.0 >Reporter: hejian >Priority: Major > Attachments: image-2019-04-22-10-07-06-737.png > > > !image-2019-04-22-10-07-06-737.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-2849) duplicate segment，cannot be deleted and data cannot be refreshed and merged
[
https://issues.apache.org/jira/browse/KYLIN-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822832#comment-16822832
]
hejian commented on KYLIN-2849:
---
[~Shaofengshi] Shaofeng, as desc in KYLIN-3964
> duplicate segment，cannot be deleted and data cannot be refreshed and merged
> ---
>
> Key: KYLIN-2849
> URL: https://issues.apache.org/jira/browse/KYLIN-2849
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine, Metadata, REST Service
>Affects Versions: v2.0.0
> Environment: hadoop：hadoop-2.6.0-cdh5.8.2
> hive ：2.1.0
> hbase：0.98
>Reporter: scott.zhai
>Assignee: Dong Li
>Priority: Major
> Labels: scope
> Fix For: v2.3.0
>
> Attachments: kylin-1.png, kylin-2.png
>
>
> cube duplicate segments。
> cannot be deleted and data cannot be refreshed and merged
> {code}
> try
> curl -X DELETE
> "http://127.0.0.1:7070/kylin/api/cubes/Remain_Cube_2/segs/2017082200_2017082300";
> -H "Authorization: Basic QURNSU46S1lMSU4=" -H "Content-Type:
> application/json;charset=UTF-8"
> Cannot delete segment '2017082200_2017082300' as it is neither the
> first nor the last
> segment.","stacktrace":"org.apache.kylin.rest.exception.InternalErrorException:
> Cannot delete segment '2017082200_2017082300' as it is neither the
> first nor the last segment
> {code}
> 暂时解决办法：
> {code}
> public CubeInstance deleteSegment(CubeInstance cube, String segmentName)
> throws IOException {
> if (!segmentName.equals(cube.getSegments().get(0).getName()) &&
> !segmentName.equals(cube.getSegments().get(cube.getSegments().size() -
> 1).getName())) {
> //throw new IllegalArgumentException("Cannot delete segment '" +
> segmentName + "' as it is neither the first nor the last segment.");
> }
> CubeSegment toDelete = null;
> for (CubeSegment seg : cube.getSegments()) {
> if (seg.getName().equals(segmentName)) {
> toDelete = seg;
> }
> }
> if (toDelete == null) {
> throw new IllegalArgumentException("Cannot find segment '" +
> segmentName + "'");
> }
> if (toDelete.getStatus() != SegmentStatusEnum.READY) {
> //throw new IllegalArgumentException("Cannot delete segment '" +
> segmentName + "' as its status is not READY. Discard the on-going job for
> it.");
> }
> CubeUpdate update = new CubeUpdate(cube);
> update.setToRemoveSegs(new CubeSegment[] { toDelete });
> return CubeManager.getInstance(getConfig()).updateCube(update);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3964) segment overlapped, the status is "NEW", last_build_job_id is null, can not refresh/delete/build this segment
hejian created KYLIN-3964: - Summary: segment overlapped, the status is "NEW", last_build_job_id is null, can not refresh/delete/build this segment Key: KYLIN-3964 URL: https://issues.apache.org/jira/browse/KYLIN-3964 Project: Kylin Issue Type: Bug Environment: kylin-2.4.0 Reporter: hejian Attachments: image-2019-04-22-10-07-06-737.png !image-2019-04-22-10-07-06-737.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-2849) duplicate segment，cannot be deleted and data cannot be refreshed and merged
[
https://issues.apache.org/jira/browse/KYLIN-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822829#comment-16822829
]
hejian commented on KYLIN-2849:
---
ok thanks Shaofeng
> duplicate segment，cannot be deleted and data cannot be refreshed and merged
> ---
>
> Key: KYLIN-2849
> URL: https://issues.apache.org/jira/browse/KYLIN-2849
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine, Metadata, REST Service
>Affects Versions: v2.0.0
> Environment: hadoop：hadoop-2.6.0-cdh5.8.2
> hive ：2.1.0
> hbase：0.98
>Reporter: scott.zhai
>Assignee: Dong Li
>Priority: Major
> Labels: scope
> Fix For: v2.3.0
>
> Attachments: kylin-1.png, kylin-2.png
>
>
> cube duplicate segments。
> cannot be deleted and data cannot be refreshed and merged
> {code}
> try
> curl -X DELETE
> "http://127.0.0.1:7070/kylin/api/cubes/Remain_Cube_2/segs/2017082200_2017082300";
> -H "Authorization: Basic QURNSU46S1lMSU4=" -H "Content-Type:
> application/json;charset=UTF-8"
> Cannot delete segment '2017082200_2017082300' as it is neither the
> first nor the last
> segment.","stacktrace":"org.apache.kylin.rest.exception.InternalErrorException:
> Cannot delete segment '2017082200_2017082300' as it is neither the
> first nor the last segment
> {code}
> 暂时解决办法：
> {code}
> public CubeInstance deleteSegment(CubeInstance cube, String segmentName)
> throws IOException {
> if (!segmentName.equals(cube.getSegments().get(0).getName()) &&
> !segmentName.equals(cube.getSegments().get(cube.getSegments().size() -
> 1).getName())) {
> //throw new IllegalArgumentException("Cannot delete segment '" +
> segmentName + "' as it is neither the first nor the last segment.");
> }
> CubeSegment toDelete = null;
> for (CubeSegment seg : cube.getSegments()) {
> if (seg.getName().equals(segmentName)) {
> toDelete = seg;
> }
> }
> if (toDelete == null) {
> throw new IllegalArgumentException("Cannot find segment '" +
> segmentName + "'");
> }
> if (toDelete.getStatus() != SegmentStatusEnum.READY) {
> //throw new IllegalArgumentException("Cannot delete segment '" +
> segmentName + "' as its status is not READY. Discard the on-going job for
> it.");
> }
> CubeUpdate update = new CubeUpdate(cube);
> update.setToRemoveSegs(new CubeSegment[] { toDelete });
> return CubeManager.getInstance(getConfig()).updateCube(update);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3962) Support streaming cubing using Spark Streaming or Flink
[ https://issues.apache.org/jira/browse/KYLIN-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822820#comment-16822820 ] Shaofeng SHI commented on KYLIN-3962: - Shaohui, we're also looking for better solution if any. Suggestions are welcomed. > Support streaming cubing using Spark Streaming or Flink > --- > > Key: KYLIN-3962 > URL: https://issues.apache.org/jira/browse/KYLIN-3962 > Project: Kylin > Issue Type: Improvement >Reporter: Liu Shaohui >Priority: Major > > KYLIN-3654 introduced the Real-time Streaming, but in my opinion, the arch is > a little too complicated to handle. > As streaming frameworks like spark streaming, flink are widely used in many > companies.Can we use the streaming framework to support real time cubing in > Kylin. > This is just a proposal. More discussion and suggestions are welcomed~ > More details of this proposal will be added later. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-2849) duplicate segment，cannot be deleted and data cannot be refreshed and merged
[
https://issues.apache.org/jira/browse/KYLIN-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822816#comment-16822816
]
Shaofeng SHI commented on KYLIN-2849:
-
[~hejian999] hejian, please open a new Jira with the necessary information,
like your REST request, the error trace in kylin.log, etc. Thanks!
> duplicate segment，cannot be deleted and data cannot be refreshed and merged
> ---
>
> Key: KYLIN-2849
> URL: https://issues.apache.org/jira/browse/KYLIN-2849
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine, Metadata, REST Service
>Affects Versions: v2.0.0
> Environment: hadoop：hadoop-2.6.0-cdh5.8.2
> hive ：2.1.0
> hbase：0.98
>Reporter: scott.zhai
>Assignee: Dong Li
>Priority: Major
> Labels: scope
> Fix For: v2.3.0
>
> Attachments: kylin-1.png, kylin-2.png
>
>
> cube duplicate segments。
> cannot be deleted and data cannot be refreshed and merged
> {code}
> try
> curl -X DELETE
> "http://127.0.0.1:7070/kylin/api/cubes/Remain_Cube_2/segs/2017082200_2017082300";
> -H "Authorization: Basic QURNSU46S1lMSU4=" -H "Content-Type:
> application/json;charset=UTF-8"
> Cannot delete segment '2017082200_2017082300' as it is neither the
> first nor the last
> segment.","stacktrace":"org.apache.kylin.rest.exception.InternalErrorException:
> Cannot delete segment '2017082200_2017082300' as it is neither the
> first nor the last segment
> {code}
> 暂时解决办法：
> {code}
> public CubeInstance deleteSegment(CubeInstance cube, String segmentName)
> throws IOException {
> if (!segmentName.equals(cube.getSegments().get(0).getName()) &&
> !segmentName.equals(cube.getSegments().get(cube.getSegments().size() -
> 1).getName())) {
> //throw new IllegalArgumentException("Cannot delete segment '" +
> segmentName + "' as it is neither the first nor the last segment.");
> }
> CubeSegment toDelete = null;
> for (CubeSegment seg : cube.getSegments()) {
> if (seg.getName().equals(segmentName)) {
> toDelete = seg;
> }
> }
> if (toDelete == null) {
> throw new IllegalArgumentException("Cannot find segment '" +
> segmentName + "'");
> }
> if (toDelete.getStatus() != SegmentStatusEnum.READY) {
> //throw new IllegalArgumentException("Cannot delete segment '" +
> segmentName + "' as its status is not READY. Discard the on-going job for
> it.");
> }
> CubeUpdate update = new CubeUpdate(cube);
> update.setToRemoveSegs(new CubeSegment[] { toDelete });
> return CubeManager.getInstance(getConfig()).updateCube(update);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3957) Query system_cube get exception Cannot cast "java.math.BigDecimal" to "java.lang.Double"
[
https://issues.apache.org/jira/browse/KYLIN-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3957:
Fix Version/s: (was: Future)
v2.6.2
v3.0.0
> Query system_cube get exception Cannot cast "java.math.BigDecimal" to
> "java.lang.Double"
>
>
> Key: KYLIN-3957
> URL: https://issues.apache.org/jira/browse/KYLIN-3957
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Reporter: Chao Long
>Assignee: Chao Long
>Priority: Major
> Fix For: v3.0.0, v2.6.2
>
>
> In system cube, the return dataType of column whose real dataType is Double
> will be converted to Decimal in SUM measure.
> {code:java}
> FunctionDesc function = new FunctionDesc();
> function.setExpression(FunctionDesc.FUNC_SUM);
> function.setParameter(parameterDesc);
>
> function.setReturnType(dataType.equals(HiveTableCreator.HiveTypeEnum.HDOUBLE.toString())
> ? HiveTableCreator.HiveTypeEnum.HDECIMAL.toString()
> : dataType);
> {code}
> but query with that measure will get exception:
> {code}
> Caused by: org.codehaus.commons.compiler.CompileException: Line 108, Column
> 44: Cannot cast "java.math.BigDecimal" to "java.lang.Double"
> at
> org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:10092)
> at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:3839)
> at org.codehaus.janino.UnitCompiler.access$6400(UnitCompiler.java:183)
> at org.codehaus.janino.UnitCompiler$10.visitCast(UnitCompiler.java:3246)
> at org.codehaus.janino.Java$Cast.accept(Java.java:3802)
> at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:3278)
> at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:3845)
> at org.codehaus.janino.UnitCompiler.access$8600(UnitCompiler.java:183)
> at
> org.codehaus.janino.UnitCompiler$10.visitParenthesizedExpression(UnitCompiler.java:3274)
> at
> org.codehaus.janino.Java$ParenthesizedExpression.accept(Java.java:3830)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
