[GitHub] [kylin] nichunen merged pull request #585: KYLIN-3936 MR/Spark task will still run after the job is stopped
nichunen merged pull request #585: KYLIN-3936 MR/Spark task will still run after the job is stopped URL: https://github.com/apache/kylin/pull/585 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3936) MR/Spark task will still run after the job is stopped.
[ https://issues.apache.org/jira/browse/KYLIN-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824826#comment-16824826 ] ASF GitHub Bot commented on KYLIN-3936: --- nichunen commented on pull request #585: KYLIN-3936 MR/Spark task will still run after the job is stopped URL: https://github.com/apache/kylin/pull/585 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > MR/Spark task will still run after the job is stopped. > -- > > Key: KYLIN-3936 > URL: https://issues.apache.org/jira/browse/KYLIN-3936 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > Fix For: v2.6.2 > > > The command "pause" only sets status of the job to "stopped" and does not > reset the status of the subtask. > So, In SparkExecutable, we can't get the real status of the running task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3936) MR/Spark task will still run after the job is stopped.
[ https://issues.apache.org/jira/browse/KYLIN-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824827#comment-16824827 ] ASF subversion and git services commented on KYLIN-3936: Commit cf2de6968b0e46365a65c8c121f0552b71e22cdf in kylin's branch refs/heads/master from Guangxu Cheng [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=cf2de69 ] KYLIN-3936 MR/Spark task will still run after the job is stopped > MR/Spark task will still run after the job is stopped. > -- > > Key: KYLIN-3936 > URL: https://issues.apache.org/jira/browse/KYLIN-3936 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > Fix For: v2.6.2 > > > The command "pause" only sets status of the job to "stopped" and does not > reset the status of the subtask. > So, In SparkExecutable, we can't get the real status of the running task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] codecov-io edited a comment on issue #585: KYLIN-3936 MR/Spark task will still run after the job is stopped
codecov-io edited a comment on issue #585: KYLIN-3936 MR/Spark task will still run after the job is stopped URL: https://github.com/apache/kylin/pull/585#issuecomment-481087044 # [Codecov](https://codecov.io/gh/apache/kylin/pull/585?src=pr=h1) Report > Merging [#585](https://codecov.io/gh/apache/kylin/pull/585?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/cfabb816a54855b321d9c1ac2acdb76bb7d2652a?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `0%`. [](https://codecov.io/gh/apache/kylin/pull/585?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #585 +/- ## - Coverage 25.34% 25.33% -0.01% - Complexity 5839 5840 +1 Files 1379 1379 Lines 8178481802 +18 Branches 1145811464 +6 + Hits 2072520727 +2 - Misses5903559052 +17 + Partials 2024 2023 -1 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/585?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../apache/kylin/job/execution/ExecutableManager.java](https://codecov.io/gh/apache/kylin/pull/585/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9leGVjdXRpb24vRXhlY3V0YWJsZU1hbmFnZXIuamF2YQ==) | `32.52% <0%> (-1.19%)` | `35 <0> (ø)` | | | [...java/org/apache/kylin/rest/service/JobService.java](https://codecov.io/gh/apache/kylin/pull/585/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3Qvc2VydmljZS9Kb2JTZXJ2aWNlLmphdmE=) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...rg/apache/kylin/rest/controller/JobController.java](https://codecov.io/gh/apache/kylin/pull/585/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvY29udHJvbGxlci9Kb2JDb250cm9sbGVyLmphdmE=) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/585/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `69.3% <0%> (-0.92%)` | `7% <0%> (ø)` | | | [.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://codecov.io/gh/apache/kylin/pull/585/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9UcmVlQ3Vib2lkU2NoZWR1bGVyLmphdmE=) | `66.15% <0%> (+2.3%)` | `0% <0%> (ø)` | :arrow_down: | | [...org/apache/kylin/rest/util/QueryRequestLimits.java](https://codecov.io/gh/apache/kylin/pull/585/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvdXRpbC9RdWVyeVJlcXVlc3RMaW1pdHMuamF2YQ==) | `40.47% <0%> (+4.76%)` | `6% <0%> (+1%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/585?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/585?src=pr=footer). Last update [cfabb81...0fcfd9a](https://codecov.io/gh/apache/kylin/pull/585?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] coveralls commented on issue #585: KYLIN-3936 MR/Spark task will still run after the job is stopped
coveralls commented on issue #585: KYLIN-3936 MR/Spark task will still run after the job is stopped URL: https://github.com/apache/kylin/pull/585#issuecomment-486060953 ## Pull Request Test Coverage Report for [Build 4416](https://coveralls.io/builds/22985892) * **0** of **21** **(0.0%)** changed or added relevant lines in **3** files are covered. * **5** unchanged lines in **1** file lost coverage. * Overall coverage decreased (**-0.005%**) to **27.816%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [server-base/src/main/java/org/apache/kylin/rest/controller/JobController.java](https://coveralls.io/builds/22985892/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fcontroller%2FJobController.java#L183) | 0 | 2 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/service/JobService.java](https://coveralls.io/builds/22985892/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FJobService.java#L663) | 0 | 5 | 0.0% | [core-job/src/main/java/org/apache/kylin/job/execution/ExecutableManager.java](https://coveralls.io/builds/22985892/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fexecution%2FExecutableManager.java#L378) | 0 | 14 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/22985892/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L439) | 5 | 77.81% | | Totals | [](https://coveralls.io/builds/22985892) | | :-- | --: | | Change from base [Build 4413](https://coveralls.io/builds/22960555): | -0.005% | | Covered Lines: | 22754 | | Relevant Lines: | 81802 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] guangxuCheng commented on a change in pull request #585: KYLIN-3936 MR/Spark task will still run after the job is stopped
guangxuCheng commented on a change in pull request #585: KYLIN-3936 MR/Spark
task will still run after the job is stopped
URL: https://github.com/apache/kylin/pull/585#discussion_r277945700
##
File path:
core-job/src/main/java/org/apache/kylin/job/execution/ExecutableManager.java
##
@@ -414,6 +414,22 @@ public void pauseJob(String jobId) {
return;
}
+if (!(job.getStatus() == ExecutableState.READY
+|| job.getStatus() == ExecutableState.RUNNING)) {
+logger.warn("The status of job " + jobId + " is " +
job.getStatus().toString()
Review comment:
Add IllegalStateException
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (KYLIN-3976) Failed to find metadata store by url: kylin_metadata@hbase
[ https://issues.apache.org/jira/browse/KYLIN-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824752#comment-16824752 ] Chao Long commented on KYLIN-3976: -- Hi, Kylin-2.3.2 does not support hadoop3, and you can download Kylin binary package(2.5.2 or above) for hadoop3 on the website. > Failed to find metadata store by url: kylin_metadata@hbase > -- > > Key: KYLIN-3976 > URL: https://issues.apache.org/jira/browse/KYLIN-3976 > Project: Kylin > Issue Type: Bug > Components: Metadata > Environment: > kylin2.3.2+hadoop3.1.2+hive2.3.4+hbase1.2.0+zookeeper3.4.5 >Reporter: ZhangXihao >Priority: Blocker > Original Estimate: 48h > Remaining Estimate: 48h > > 2019-04-24 17:30:01,894 DEBUG [main] hbase.HBaseConnection:337 : Creating > HTable 'kylin_metadata' > 2019-04-24 17:30:01,936 INFO [main] Configuration.deprecation:1174 : > hadoop.native.lib is deprecated. Instead, use io.native.lib.available > 2019-04-24 17:30:02,089 DEBUG [main] util.ZookeeperDistributedLock:223 : > 7116@ambari02 trying to unlock > /kylin/kylin_metadata/create_htable/kylin_metadata/lock > 2019-04-24 17:30:02,099 INFO [main] util.ZookeeperDistributedLock:234 : > 7116@ambari02 released lock at > /kylin/kylin_metadata/create_htable/kylin_metadata/lock > Exception in thread "main" java.lang.IllegalArgumentException: Failed to find > metadata store by url: kylin_metadata@hbase > at > org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:96) > at > org.apache.kylin.common.persistence.ResourceStore.getStore(ResourceStore.java:108) > at > org.apache.kylin.rest.service.AclTableMigrationTool.checkIfNeedMigrate(AclTableMigrationTool.java:94) > at > org.apache.kylin.tool.AclTableMigrationCLI.main(AclTableMigrationCLI.java:41) > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:90) > ... 3 more > Caused by: java.lang.NoSuchMethodError: > org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)Lorg/apache/hadoop/hbase/HTableDescriptor; > at > org.apache.kylin.storage.hbase.HBaseConnection.createHTableIfNeeded(HBaseConnection.java:344) > at > org.apache.kylin.storage.hbase.HBaseResourceStore.createHTableIfNeeded(HBaseResourceStore.java:111) > at > org.apache.kylin.storage.hbase.HBaseResourceStore.(HBaseResourceStore.java:92) > ... 8 more > 2019-04-24 17:30:02,105 INFO [close-hbase-conn] hbase.HBaseConnection:137 : > Closing HBase connections... > 2019-04-24 17:30:02,107 INFO [close-hbase-conn] > client.ConnectionManager$HConnectionImplementation:1830 : Closing zookeeper > sessionid=0x16a4c1bd0c80008 > 2019-04-24 17:30:02,111 INFO [close-hbase-conn] zookeeper.ZooKeeper:684 : > Session: 0x16a4c1bd0c80008 closed > 2019-04-24 17:30:02,111 INFO [main-EventThread] zookeeper.ClientCnxn:512 : > EventThread shut down > 2019-04-24 17:30:02,114 INFO [Thread-7] zookeeper.ZooKeeper:684 : Session: > 0x16a4c1bd0c80009 closed > 2019-04-24 17:30:02,114 INFO [main-EventThread] zookeeper.ClientCnxn:512 : > EventThread shut down -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3977) Aviod mistaken deleting dicts by storage cleanup while building jobs are running
PENG Zhengshuai created KYLIN-3977: -- Summary: Aviod mistaken deleting dicts by storage cleanup while building jobs are running Key: KYLIN-3977 URL: https://issues.apache.org/jira/browse/KYLIN-3977 Project: Kylin Issue Type: Improvement Components: Metadata Reporter: PENG Zhengshuai Assignee: PENG Zhengshuai When there are cube building jobs running, storage cleanup may delete some dictionaries which the cubes are not referenced to. But the in build jobs, those dictionaries may be reused. If these reused dictionaries have been deleted by storage cleanup, the build job will be failed -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3940) Exception in thread "main" java.lang.IllegalArgumentException: Failed to find metadata store by url: kylin_metadata@hbase
[ https://issues.apache.org/jira/browse/KYLIN-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824727#comment-16824727 ] ZhangXihao commented on KYLIN-3940: --- Have you solved this problem?How？ > Exception in thread "main" java.lang.IllegalArgumentException: Failed to find > metadata store by url: kylin_metadata@hbase > - > > Key: KYLIN-3940 > URL: https://issues.apache.org/jira/browse/KYLIN-3940 > Project: Kylin > Issue Type: Task > Components: Environment >Affects Versions: v2.5.2 > Environment: apache-hive-2.3.4-bin > hbase-1.4.9 > hadoop-3.1.0 > apache-kylin-2.5.2-bin-hadoop3 >Reporter: praveenece >Priority: Critical > > I am getting the below error. This error occurred when use > apache-kylin-2.5.2-bin-hadoop3. if we use apache-kylin-2.5.2-bin-hbase1x > kylin gets started. > apache-hive-2.3.4-bin > hbase-1.4.9 > hadoop-3.1.0 > apache-kylin-2.5.2-bin-hadoop3 > Exception in thread "main" java.lang.IllegalArgumentException: Failed to find > metadata store by url: kylin_metadata@hbase > at > org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:98) > at > org.apache.kylin.common.persistence.ResourceStore.getStore(ResourceStore.java:110) > at > org.apache.kylin.rest.service.AclTableMigrationTool.checkIfNeedMigrate(AclTableMigrationTool.java:98) > at > org.apache.kylin.tool.AclTableMigrationCLI.main(AclTableMigrationCLI.java:41) > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:92) > ... 3 more > Caused by: java.lang.NoClassDefFoundError: > org/apache/hadoop/hbase/client/ColumnFamilyDescriptor > at > org.apache.kylin.storage.hbase.HBaseResourceStore.getConnection(HBaseResourceStore.java:86) > at > org.apache.kylin.storage.hbase.HBaseResourceStore.createHTableIfNeeded(HBaseResourceStore.java:112) > at > org.apache.kylin.storage.hbase.HBaseResourceStore.(HBaseResourceStore.java:93) > ... 8 more > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.hbase.client.ColumnFamilyDescriptor > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3976) Failed to find metadata store by url: kylin_metadata@hbase
ZhangXihao created KYLIN-3976: - Summary: Failed to find metadata store by url: kylin_metadata@hbase Key: KYLIN-3976 URL: https://issues.apache.org/jira/browse/KYLIN-3976 Project: Kylin Issue Type: Bug Components: Metadata Environment: kylin2.3.2+hadoop3.1.2+hive2.3.4+hbase1.2.0+zookeeper3.4.5 Reporter: ZhangXihao 2019-04-24 17:30:01,894 DEBUG [main] hbase.HBaseConnection:337 : Creating HTable 'kylin_metadata' 2019-04-24 17:30:01,936 INFO [main] Configuration.deprecation:1174 : hadoop.native.lib is deprecated. Instead, use io.native.lib.available 2019-04-24 17:30:02,089 DEBUG [main] util.ZookeeperDistributedLock:223 : 7116@ambari02 trying to unlock /kylin/kylin_metadata/create_htable/kylin_metadata/lock 2019-04-24 17:30:02,099 INFO [main] util.ZookeeperDistributedLock:234 : 7116@ambari02 released lock at /kylin/kylin_metadata/create_htable/kylin_metadata/lock Exception in thread "main" java.lang.IllegalArgumentException: Failed to find metadata store by url: kylin_metadata@hbase at org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:96) at org.apache.kylin.common.persistence.ResourceStore.getStore(ResourceStore.java:108) at org.apache.kylin.rest.service.AclTableMigrationTool.checkIfNeedMigrate(AclTableMigrationTool.java:94) at org.apache.kylin.tool.AclTableMigrationCLI.main(AclTableMigrationCLI.java:41) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:90) ... 3 more Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)Lorg/apache/hadoop/hbase/HTableDescriptor; at org.apache.kylin.storage.hbase.HBaseConnection.createHTableIfNeeded(HBaseConnection.java:344) at org.apache.kylin.storage.hbase.HBaseResourceStore.createHTableIfNeeded(HBaseResourceStore.java:111) at org.apache.kylin.storage.hbase.HBaseResourceStore.(HBaseResourceStore.java:92) ... 8 more 2019-04-24 17:30:02,105 INFO [close-hbase-conn] hbase.HBaseConnection:137 : Closing HBase connections... 2019-04-24 17:30:02,107 INFO [close-hbase-conn] client.ConnectionManager$HConnectionImplementation:1830 : Closing zookeeper sessionid=0x16a4c1bd0c80008 2019-04-24 17:30:02,111 INFO [close-hbase-conn] zookeeper.ZooKeeper:684 : Session: 0x16a4c1bd0c80008 closed 2019-04-24 17:30:02,111 INFO [main-EventThread] zookeeper.ClientCnxn:512 : EventThread shut down 2019-04-24 17:30:02,114 INFO [Thread-7] zookeeper.ZooKeeper:684 : Session: 0x16a4c1bd0c80009 closed 2019-04-24 17:30:02,114 INFO [main-EventThread] zookeeper.ClientCnxn:512 : EventThread shut down -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3975) Can kylin accelerate query speed for natural week or natural month report?
[ https://issues.apache.org/jira/browse/KYLIN-3975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824714#comment-16824714 ] Shaofeng SHI commented on KYLIN-3975: - Hi Jintao, Let me try to understand: say a cube has three segments: seg1: [2019-01-20 to 2019-02-01) seg2: [2019-02-01 to 2019-03-01) seg3: [2019-03-01 to 2019-03-10) Assume there is a query: select dim1, dim2, sum(x), count(distinct y) from fact_table where dt > '2019-01-25' and dt < '2019-03-10' For this query, Kylin will scan all the three segments by checking the partition date's max/min value, and the selected cuboid will have "dt"; If 'month'-'week'-'dt' is defined as a hierarchy, the cuboid will have all of them. In this case, the cuboid will be "dim1+dim2+month+week+dt" Here we can see, although the data in these segments have been merged to month level, they won't be used because the query condition is on "dt". So the performance is not that perfect as we expected. A potential optimization is, if a segment is totally in the partition date scope (in this case, seg2 and seg3), and the partition date is only used as a filtering condition (not in group by), Kylin can change the execution plan to use the cuboid that has no "dt". In this case it will be optimized to "dim1+dim2", whose size is much smaller than the previous one, and the query performance can be much efficient than before as the aggregation has already been done in cube. Is this what you want to discuss? or any better idea? Thanks. > Can kylin accelerate query speed for natural week or natural month report? > --- > > Key: KYLIN-3975 > URL: https://issues.apache.org/jira/browse/KYLIN-3975 > Project: Kylin > Issue Type: New Feature > Components: Job Engine, Query Engine >Reporter: zhao jintao >Priority: Major > > Hi team: > In bigdata analytics platform, we often query data of the nature week or > nature month. > For example, in Bank or Accounting reports, the query periods are often a > natural week or natural month report. > In kylin system, we can build cube to increase query speed. However, it will > query slowly if the amount of data is large and the query cycle is long > especlially using count distinct measure. > For example, We can add month dimension to the cube, then merge cube in > normal month peroid; but if the query sql has date partition, it will also > match the cube has both week dimension and date dimension, kylin need search > data from HBase and aggregate data in memory. It also slowly if the amountof > data is large. > Does anyone face the same problem? Who has a better way to solve the problems > of nature week or nature month query? > > Best regards > Thank you. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3842) kylinProperties.js Unable to get the public configuration of the first line in the front end
[
https://issues.apache.org/jira/browse/KYLIN-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824564#comment-16824564
]
Vsevolod Ostapenko commented on KYLIN-3842:
---
I created a patch (attached) that should address both the original concern and
the regression introduced by the prior bug fix attempt.
Please review, and either comment or approve.
> kylinProperties.js Unable to get the public configuration of the first line
> in the front end
>
>
> Key: KYLIN-3842
> URL: https://issues.apache.org/jira/browse/KYLIN-3842
> Project: Kylin
> Issue Type: Bug
> Components: Web
>Affects Versions: v2.5.2
>Reporter: Yuzhang QIU
>Assignee: Yuzhang QIU
>Priority: Minor
> Fix For: v2.6.2
>
> Attachments: KYLIN-3842.master.001.patch
>
>
> Hi dear team:
> I'm developing OLAP Platform based on Kylin2.5.2. During my work, I found
> that kylinProperties.js:37(getProperty(name)) can't get the property of the
> first line in the '_config' which initialized through /admin/public_config.
> For example, the public config is
> 'kylin.restclient.connection.default-max-per-route=20\nkylin.restclient.connection.max-total=200\nkylin.engine.default=2\nkylin.storage.default=2\n
> kylin.web.hive-limit=20\nkylin.web.help.length=4\n'. I expected to get 20
> but got '' when I want to get config by key
> 'kylin.restclient.connection.default-max-per-route'. This problem caused by
> 'var keyIndex = _config.indexOf('\n' + name + '=');'(at
> kylinProperties.js:37) return -1 for those names before which don't have an
> \n(at the first line).
> Then, I debug the AdminService.java, KylinConfig.java and found that the
> KylinConfig.java:517(around this line, in method
> exportToString(Collection propertyKeys)) build the public config
> string with a char '\n' after each property, which cause the first property
> don't has '\n' before it.
> Those are what I found, which will cause problem for developers.
> How do you think?
> Best regard
> yuzhang
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3842) kylinProperties.js Unable to get the public configuration of the first line in the front end
[
https://issues.apache.org/jira/browse/KYLIN-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vsevolod Ostapenko updated KYLIN-3842:
--
Attachment: KYLIN-3842.master.001.patch
> kylinProperties.js Unable to get the public configuration of the first line
> in the front end
>
>
> Key: KYLIN-3842
> URL: https://issues.apache.org/jira/browse/KYLIN-3842
> Project: Kylin
> Issue Type: Bug
> Components: Web
>Affects Versions: v2.5.2
>Reporter: Yuzhang QIU
>Assignee: Yuzhang QIU
>Priority: Minor
> Fix For: v2.6.2
>
> Attachments: KYLIN-3842.master.001.patch
>
>
> Hi dear team:
> I'm developing OLAP Platform based on Kylin2.5.2. During my work, I found
> that kylinProperties.js:37(getProperty(name)) can't get the property of the
> first line in the '_config' which initialized through /admin/public_config.
> For example, the public config is
> 'kylin.restclient.connection.default-max-per-route=20\nkylin.restclient.connection.max-total=200\nkylin.engine.default=2\nkylin.storage.default=2\n
> kylin.web.hive-limit=20\nkylin.web.help.length=4\n'. I expected to get 20
> but got '' when I want to get config by key
> 'kylin.restclient.connection.default-max-per-route'. This problem caused by
> 'var keyIndex = _config.indexOf('\n' + name + '=');'(at
> kylinProperties.js:37) return -1 for those names before which don't have an
> \n(at the first line).
> Then, I debug the AdminService.java, KylinConfig.java and found that the
> KylinConfig.java:517(around this line, in method
> exportToString(Collection propertyKeys)) build the public config
> string with a char '\n' after each property, which cause the first property
> don't has '\n' before it.
> Those are what I found, which will cause problem for developers.
> How do you think?
> Best regard
> yuzhang
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3967) sum along with case expression does not work in query
[ https://issues.apache.org/jira/browse/KYLIN-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824283#comment-16824283 ] Gladson Vas commented on KYLIN-3967: hi [~Shaofengshi] with kylin.query.enable-dynamic-column=true select sum(case when col1>0 then 0 else col2 end ) from table when the col1 is dimension i get the correct output if the operator is > but when the operator is < the output seems to be wrong. Any idea? select sum(case when col1>0 then 0 else col2 end ) from when col1 is a metric i get the following error: No realization found for OLAPContext, CUBE_UNMATCHED_DIMENSION Thanks. > sum along with case expression does not work in query > - > > Key: KYLIN-3967 > URL: https://issues.apache.org/jira/browse/KYLIN-3967 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Reporter: Gladson Vas >Priority: Blocker > > When i try to run a query with a sum case expression combination, > eg: select sum(case when col1<0 then 0 else col1 end ) from table > i get the following error: > No realization found for OLAPContext, CUBE_UNMATCHED_AGGREGATION[FunctionDesc > [expression=SUM, parameter=CASE(<($8, 0), 0, $8), returnType=null]], > rel#36838:OLAPTableScan.OLAP.[](table=[DEFAULT, table],ctx=,fields=[0, 1, 2, > 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, > 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, > 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, > 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, > 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, > 100, 101, 102, 103, 104, 105, 106]) while executing SQL: "select sum (case > when col1 <0 then 0 else col1 end ) from table LIMIT 5" > > is there any way to support this sum case expression in the query engine? > Also I get the same error when the sum operation is done on a column derived > from a case expression in a subquery. > eg: select sum(a.col1) from (select case when col1<0 then 0 else col1 end as > col1 from table) a > Thanks, > Gladson > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3975) Can kylin accelerate query speed for natural week or natural month report?
[ https://issues.apache.org/jira/browse/KYLIN-3975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhao jintao updated KYLIN-3975: --- Description: Hi team: In bigdata analytics platform, we often query data of the nature week or nature month. For example, in Bank or Accounting reports, the query periods are often a natural week or natural month report. In kylin system, we can build cube to increase query speed. However, it will query slowly if the amount of data is large and the query cycle is long especlially using count distinct measure. For example, We can add month dimension to the cube, then merge cube in normal month peroid; but if the query sql has date partition, it will also match the cube has both week dimension and date dimension, kylin need search data from HBase and aggregate data in memory. It also slowly if the amountof data is large. Does anyone face the same problem? Who has a better way to solve the problems of nature week or nature month query? Best regards Thank you. was: Hi team: In bigdata analytics platform, we often query data of the nature week or nature month. For example, in Bank or Accounting reports, the query periods are often a natural week or natural month report. In kylin system, we can build cube to increase query speed. However, it will query slowly if the amount of data is large and the query cycle is long especlially using count distinct measure. For example, We can add month dimension to the cube, then merge cube in normal month peroid; but if the query sql has date partition, it will also match the cube has both week dimension and date dimension, kylin need search data from HBase and aggregate data in memory. It also slowly if the amountof data is large. Does anyone face the same problem? Who has a better way to solve the problems of nature week or nature month query? > Can kylin accelerate query speed for natural week or natural month report? > --- > > Key: KYLIN-3975 > URL: https://issues.apache.org/jira/browse/KYLIN-3975 > Project: Kylin > Issue Type: New Feature > Components: Job Engine, Query Engine >Reporter: zhao jintao >Priority: Major > > Hi team: > In bigdata analytics platform, we often query data of the nature week or > nature month. > For example, in Bank or Accounting reports, the query periods are often a > natural week or natural month report. > In kylin system, we can build cube to increase query speed. However, it will > query slowly if the amount of data is large and the query cycle is long > especlially using count distinct measure. > For example, We can add month dimension to the cube, then merge cube in > normal month peroid; but if the query sql has date partition, it will also > match the cube has both week dimension and date dimension, kylin need search > data from HBase and aggregate data in memory. It also slowly if the amountof > data is large. > Does anyone face the same problem? Who has a better way to solve the problems > of nature week or nature month query? > > Best regards > Thank you. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3975) Can kylin accelerate query speed for natural week or natural month report?
[ https://issues.apache.org/jira/browse/KYLIN-3975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhao jintao updated KYLIN-3975: --- Summary: Can kylin accelerate query speed for natural week or natural month report? (was: Can Kylin accelerate query for natural week and natural month scenes?) > Can kylin accelerate query speed for natural week or natural month report? > --- > > Key: KYLIN-3975 > URL: https://issues.apache.org/jira/browse/KYLIN-3975 > Project: Kylin > Issue Type: New Feature > Components: Job Engine, Query Engine >Reporter: zhao jintao >Priority: Major > > Hi team: > In bigdata analytics platform, we often query data of the nature week or > nature month. > For example, in Bank or Accounting reports, the query periods are often a > natural week or natural month report. > In kylin system, we can build cube to increase query speed. However, it will > query slowly if the amount of data is large and the query cycle is long > especlially using count distinct measure. > For example, We can add month dimension to the cube, then merge cube in > normal month peroid; but if the query sql has date partition, it will also > match the cube has both week dimension and date dimension, kylin need search > data from HBase and aggregate data in memory. It also slowly if the amountof > data is large. > > Does anyone face the same problem? Who has a better way to solve the problems > of nature week or nature month query? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3975) Can Kylin accelerate query for natural week and natural month scenes?
zhao jintao created KYLIN-3975: -- Summary: Can Kylin accelerate query for natural week and natural month scenes? Key: KYLIN-3975 URL: https://issues.apache.org/jira/browse/KYLIN-3975 Project: Kylin Issue Type: New Feature Components: Job Engine, Query Engine Reporter: zhao jintao Hi team: In bigdata analytics platform, we often query data of the nature week or nature month. For example, in Bank or Accounting reports, the query periods are often a natural week or natural month report. In kylin system, we can build cube to increase query speed. However, it will query slowly if the amount of data is large and the query cycle is long especlially using count distinct measure. For example, We can add month dimension to the cube, then merge cube in normal month peroid; but if the query sql has date partition, it will also match the cube has both week dimension and date dimension, kylin need search data from HBase and aggregate data in memory. It also slowly if the amountof data is large. Does anyone face the same problem? Who has a better way to solve the problems of nature week or nature month query? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] zhaojintaozhao edited a comment on issue #612: KYLIN-3961 Optimize TopNCounter's merge function to reduce TopNCounter's error size.
zhaojintaozhao edited a comment on issue #612: KYLIN-3961 Optimize TopNCounter's merge function to reduce TopNCounter's error size. URL: https://github.com/apache/kylin/pull/612#issuecomment-485816100 > Hi jintao, I understand your change. > > The current algorithm bases the algorithm of "a parallel space saving algorithm for frequent items and the Hurwitz zeta distribution", which you can find the link from the reference list of: > > https://kylin.apache.org/blog/2016/03/19/approximate-topn-measure/ > > It will make the aggregated value bigger than the actual value, but it is to ensure the position be relatively close to actual. Hi shaofeng: I built 3 cubes and test the query performance and accuracy of TOPN. The first is normal cube without topN; The second cube has topN measure with current version code. The third cube has topN measure with my modified code. Both the topN measure of the second and third cube is top100. The amount of source data is 22million, I build cube 30 days and using the same query sql. I find that both the second and third topN cube have a few errors to the actual. But the second and third topN cube is very fast than the first normal cube. The error of the third cube is less than the second cube about position relatively to the actual. The third optimized cube has less error than the second cube about the aggregated value. I think that the third optimized cube may be better than the second current code cube. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Closed] (KYLIN-3788) Kylin's internal time zone conversions are various and confusing
[
https://issues.apache.org/jira/browse/KYLIN-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhao jintao closed KYLIN-3788.
--
Resolution: Fixed
Fix Version/s: (was: v3.0.0-alpha)
Future
modify the version of fixed is future.
> Kylin's internal time zone conversions are various and confusing
> -
>
> Key: KYLIN-3788
> URL: https://issues.apache.org/jira/browse/KYLIN-3788
> Project: Kylin
> Issue Type: Bug
> Components: REST Service, Web
>Affects Versions: v2.5.2
> Environment: Huawei FI
>Reporter: zhao jintao
>Assignee: zhao jintao
>Priority: Major
> Labels: TimeZone
> Fix For: Future
>
>
> Hi, All team
> I found that Kylin has a variety of time zone conversions inside, which is
> confusing. In some code, the time zone is "GMT" when converting time, and
> some code uses the time zone specified in the configuration file
> ("TimeZone.getTimeZone("GMT")" );
> For Example:
> in CubingJob.java, use TimeZone from configuration file:
> format.setTimeZone(TimeZone.getTimeZone(
> config.getTimeZone())),
> but in TimeUtil.java, use GMT TimeZone:private static TimeZone gmt =
> TimeZone.getTimeZone("GMT")
>
> When inport data from Kafka Streaming Data，kylin generate "Data_Start",
> "Hour_Start" uses the GMT time zone, They are some time zones later than
> the real time. Why not use the time zone specified in the configuration file?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Reopened] (KYLIN-3788) Kylin's internal time zone conversions are various and confusing
[
https://issues.apache.org/jira/browse/KYLIN-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhao jintao reopened KYLIN-3788:
修改修复的版本为： future
> Kylin's internal time zone conversions are various and confusing
> -
>
> Key: KYLIN-3788
> URL: https://issues.apache.org/jira/browse/KYLIN-3788
> Project: Kylin
> Issue Type: Bug
> Components: REST Service, Web
>Affects Versions: v2.5.2
> Environment: Huawei FI
>Reporter: zhao jintao
>Assignee: zhao jintao
>Priority: Major
> Labels: TimeZone
> Fix For: v3.0.0-alpha
>
>
> Hi, All team
> I found that Kylin has a variety of time zone conversions inside, which is
> confusing. In some code, the time zone is "GMT" when converting time, and
> some code uses the time zone specified in the configuration file
> ("TimeZone.getTimeZone("GMT")" );
> For Example:
> in CubingJob.java, use TimeZone from configuration file:
> format.setTimeZone(TimeZone.getTimeZone(
> config.getTimeZone())),
> but in TimeUtil.java, use GMT TimeZone:private static TimeZone gmt =
> TimeZone.getTimeZone("GMT")
>
> When inport data from Kafka Streaming Data，kylin generate "Data_Start",
> "Hour_Start" uses the GMT time zone, They are some time zones later than
> the real time. Why not use the time zone specified in the configuration file?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3967) sum along with case expression does not work in query
[ https://issues.apache.org/jira/browse/KYLIN-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824158#comment-16824158 ] Shaofeng SHI commented on KYLIN-3967: - Please check: 1) col1 is a dimension; 2) in kylin.properties, add "kylin.query.enable-dynamic-column=true" (default is false) > sum along with case expression does not work in query > - > > Key: KYLIN-3967 > URL: https://issues.apache.org/jira/browse/KYLIN-3967 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Reporter: Gladson Vas >Priority: Blocker > > When i try to run a query with a sum case expression combination, > eg: select sum(case when col1<0 then 0 else col1 end ) from table > i get the following error: > No realization found for OLAPContext, CUBE_UNMATCHED_AGGREGATION[FunctionDesc > [expression=SUM, parameter=CASE(<($8, 0), 0, $8), returnType=null]], > rel#36838:OLAPTableScan.OLAP.[](table=[DEFAULT, table],ctx=,fields=[0, 1, 2, > 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, > 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, > 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, > 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, > 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, > 100, 101, 102, 103, 104, 105, 106]) while executing SQL: "select sum (case > when col1 <0 then 0 else col1 end ) from table LIMIT 5" > > is there any way to support this sum case expression in the query engine? > Also I get the same error when the sum operation is done on a column derived > from a case expression in a subquery. > eg: select sum(a.col1) from (select case when col1<0 then 0 else col1 end as > col1 from table) a > Thanks, > Gladson > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] zhaojintaozhao edited a comment on issue #612: KYLIN-3961 Optimize TopNCounter's merge function to reduce TopNCounter's error size.
zhaojintaozhao edited a comment on issue #612: KYLIN-3961 Optimize TopNCounter's merge function to reduce TopNCounter's error size. URL: https://github.com/apache/kylin/pull/612#issuecomment-485816100 > Hi jintao, I understand your change. > > The current algorithm bases the algorithm of "a parallel space saving algorithm for frequent items and the Hurwitz zeta distribution", which you can find the link from the reference list of: > > https://kylin.apache.org/blog/2016/03/19/approximate-topn-measure/ > > It will make the aggregated value bigger than the actual value, but it is to ensure the position be relatively close to actual. Hi shaofeng: I built 3 cubes and test the query performance and accuracy of TOPN. The first is normal cube without topN; The second cube has topN measure with current version code. The third cube has topN measure with my modified code. Both the topN measure of the second and third cube is top100. The amount of source data is 22million, I build cube 30 days and using the same query sql. I find that both the second and third topN cube have a few errors to the actual. But the second and third topN cube is very fast than the first normal cube. The error of the third cube is less than the second cube about position relatively to the actual. The aggregated value of the second cube is larger than the actual, but the third cube doesn't have error about aggregated value. I think that the third optimized cube is better than the second current code cube. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] zhaojintaozhao commented on issue #612: KYLIN-3961 Optimize TopNCounter's merge function to reduce TopNCounter's error size.
zhaojintaozhao commented on issue #612: KYLIN-3961 Optimize TopNCounter's merge function to reduce TopNCounter's error size. URL: https://github.com/apache/kylin/pull/612#issuecomment-485816100 > Hi jintao, I understand your change. > > The current algorithm bases the algorithm of "a parallel space saving algorithm for frequent items and the Hurwitz zeta distribution", which you can find the link from the reference list of: > > https://kylin.apache.org/blog/2016/03/19/approximate-topn-measure/ > > It will make the aggregated value bigger than the actual value, but it is to ensure the position be relatively close to actual. Hi shaofeng: I built 3 cubes and test the query performance and accuracy of TOPN. The first is normal cube without topN; The second cube has topN measure with current version code. The third cube has topN measure with my modified code. Both the topN measure of the second and third cube is top100. The amount of source data is 22million, I build cube 30 days and using the same query sql. I find that both the second and third topN cube have a few errors to the actual. But the second and third topN cube is very fast than the first normal cube. The error of the third cube is less than the second cube about position relatively to the actual. The aggregated value of the second cube is larger than the actual, but the third cube doesn't have error about aggregated value. Overall, the third modified cube is better than the second current code cube. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen commented on a change in pull request #585: KYLIN-3936 MR/Spark task will still run after the job is stopped
nichunen commented on a change in pull request #585: KYLIN-3936 MR/Spark task
will still run after the job is stopped
URL: https://github.com/apache/kylin/pull/585#discussion_r277684802
##
File path:
core-job/src/main/java/org/apache/kylin/job/execution/ExecutableManager.java
##
@@ -414,6 +414,22 @@ public void pauseJob(String jobId) {
return;
}
+if (!(job.getStatus() == ExecutableState.READY
+|| job.getStatus() == ExecutableState.RUNNING)) {
+logger.warn("The status of job " + jobId + " is " +
job.getStatus().toString()
Review comment:
why not throw an exception
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (KYLIN-3967) sum along with case expression does not work in query
[ https://issues.apache.org/jira/browse/KYLIN-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824139#comment-16824139 ] Gladson Vas commented on KYLIN-3967: hi [~hejian999] but kylin says it supports it. https://issues.apache.org/jira/browse/KYLIN-3359 > sum along with case expression does not work in query > - > > Key: KYLIN-3967 > URL: https://issues.apache.org/jira/browse/KYLIN-3967 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Reporter: Gladson Vas >Priority: Blocker > > When i try to run a query with a sum case expression combination, > eg: select sum(case when col1<0 then 0 else col1 end ) from table > i get the following error: > No realization found for OLAPContext, CUBE_UNMATCHED_AGGREGATION[FunctionDesc > [expression=SUM, parameter=CASE(<($8, 0), 0, $8), returnType=null]], > rel#36838:OLAPTableScan.OLAP.[](table=[DEFAULT, table],ctx=,fields=[0, 1, 2, > 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, > 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, > 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, > 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, > 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, > 100, 101, 102, 103, 104, 105, 106]) while executing SQL: "select sum (case > when col1 <0 then 0 else col1 end ) from table LIMIT 5" > > is there any way to support this sum case expression in the query engine? > Also I get the same error when the sum operation is done on a column derived > from a case expression in a subquery. > eg: select sum(a.col1) from (select case when col1<0 then 0 else col1 end as > col1 from table) a > Thanks, > Gladson > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] nichunen merged pull request #613: KYLIN-3963 Provide a custom serializer for PercentileCounter
nichunen merged pull request #613: KYLIN-3963 Provide a custom serializer for PercentileCounter URL: https://github.com/apache/kylin/pull/613 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3963) Provide a custom serializer for PercentileCounter
[
https://issues.apache.org/jira/browse/KYLIN-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824089#comment-16824089
]
ASF subversion and git services commented on KYLIN-3963:
Commit 7a0039b1fe17275d234844cfea488ba570ac9558 in kylin's branch
refs/heads/engine-flink from yanghua
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=7a0039b ]
KYLIN-3963 Provide a custom serializer for PercentileCounter
> Provide a custom serializer for PercentileCounter
> -
>
> Key: KYLIN-3963
> URL: https://issues.apache.org/jira/browse/KYLIN-3963
> Project: Kylin
> Issue Type: Sub-task
> Components: Flink Engine
>Reporter: vinoyang
>Assignee: vinoyang
>Priority: Major
>
> {{PercentileCounter}} contains an instance field of Type {{TDigest}} which is
> not serializable, so it is marked as {{transient}}. By default, it triggers a
> NPE when use Flink cube engine:
> {code:java}
> Caused by: org.apache.flink.client.program.ProgramInvocationException: Job
> failed. (JobID: 8077ff0e0ff01a26f62b1babe90a34b7) at
> org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:268)
> at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:487)
> at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:475)
> at
> org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:62)
> at
> org.apache.kylin.engine.flink.FlinkCubingByLayer.execute(FlinkCubingByLayer.java:196)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
> ... 18 more Caused by:
> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
> at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
> at
> org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:265)
> ... 23 more
> Caused by: java.lang.NullPointerException at
> org.apache.kylin.measure.percentile.PercentileCounter.writeRegisters(PercentileCounter.java:68)
> at
> org.apache.kylin.measure.percentile.PercentileSerializer.serialize(PercentileSerializer.java:70)
> at
> org.apache.kylin.measure.percentile.PercentileSerializer.serialize(PercentileSerializer.java:26)
> at org.apache.kylin.measure.MeasureCodec.encode(MeasureCodec.java:76) at
> org.apache.kylin.measure.BufferedMeasureCodec.encode(BufferedMeasureCodec.java:93)
> at
> org.apache.kylin.engine.flink.FlinkCubingByLayer$1.map(FlinkCubingByLayer.java:234)
> at
> org.apache.kylin.engine.flink.FlinkCubingByLayer$1.map(FlinkCubingByLayer.java:218)
> at org.apache.flink.runtime.operators.MapDriver.run(MapDriver.java:103) at
> org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503) at
> org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368) at
> org.apache.flink.runtime.taskmanager.Task.run(Task.java:704) at
> java.lang.Thread.run(Thread.java:745)
> {code}
> So we should provide a customized serializer for {{PercentileCounter}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3963) Provide a custom serializer for PercentileCounter
[
https://issues.apache.org/jira/browse/KYLIN-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824088#comment-16824088
]
ASF GitHub Bot commented on KYLIN-3963:
---
nichunen commented on pull request #613: KYLIN-3963 Provide a custom serializer
for PercentileCounter
URL: https://github.com/apache/kylin/pull/613
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Provide a custom serializer for PercentileCounter
> -
>
> Key: KYLIN-3963
> URL: https://issues.apache.org/jira/browse/KYLIN-3963
> Project: Kylin
> Issue Type: Sub-task
> Components: Flink Engine
>Reporter: vinoyang
>Assignee: vinoyang
>Priority: Major
>
> {{PercentileCounter}} contains an instance field of Type {{TDigest}} which is
> not serializable, so it is marked as {{transient}}. By default, it triggers a
> NPE when use Flink cube engine:
> {code:java}
> Caused by: org.apache.flink.client.program.ProgramInvocationException: Job
> failed. (JobID: 8077ff0e0ff01a26f62b1babe90a34b7) at
> org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:268)
> at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:487)
> at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:475)
> at
> org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:62)
> at
> org.apache.kylin.engine.flink.FlinkCubingByLayer.execute(FlinkCubingByLayer.java:196)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
> ... 18 more Caused by:
> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
> at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
> at
> org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:265)
> ... 23 more
> Caused by: java.lang.NullPointerException at
> org.apache.kylin.measure.percentile.PercentileCounter.writeRegisters(PercentileCounter.java:68)
> at
> org.apache.kylin.measure.percentile.PercentileSerializer.serialize(PercentileSerializer.java:70)
> at
> org.apache.kylin.measure.percentile.PercentileSerializer.serialize(PercentileSerializer.java:26)
> at org.apache.kylin.measure.MeasureCodec.encode(MeasureCodec.java:76) at
> org.apache.kylin.measure.BufferedMeasureCodec.encode(BufferedMeasureCodec.java:93)
> at
> org.apache.kylin.engine.flink.FlinkCubingByLayer$1.map(FlinkCubingByLayer.java:234)
> at
> org.apache.kylin.engine.flink.FlinkCubingByLayer$1.map(FlinkCubingByLayer.java:218)
> at org.apache.flink.runtime.operators.MapDriver.run(MapDriver.java:103) at
> org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503) at
> org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368) at
> org.apache.flink.runtime.taskmanager.Task.run(Task.java:704) at
> java.lang.Thread.run(Thread.java:745)
> {code}
> So we should provide a customized serializer for {{PercentileCounter}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Reopened] (KYLIN-3788) Kylin's internal time zone conversions are various and confusing
[
https://issues.apache.org/jira/browse/KYLIN-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhao jintao reopened KYLIN-3788:
修改 修复的版本为3.0.0-alpha
> Kylin's internal time zone conversions are various and confusing
> -
>
> Key: KYLIN-3788
> URL: https://issues.apache.org/jira/browse/KYLIN-3788
> Project: Kylin
> Issue Type: Bug
> Components: REST Service, Web
>Affects Versions: v2.5.2
> Environment: Huawei FI
>Reporter: zhao jintao
>Assignee: zhao jintao
>Priority: Major
> Labels: TimeZone
> Fix For: v2.6.2
>
>
> Hi, All team
> I found that Kylin has a variety of time zone conversions inside, which is
> confusing. In some code, the time zone is "GMT" when converting time, and
> some code uses the time zone specified in the configuration file
> ("TimeZone.getTimeZone("GMT")" );
> For Example:
> in CubingJob.java, use TimeZone from configuration file:
> format.setTimeZone(TimeZone.getTimeZone(
> config.getTimeZone())),
> but in TimeUtil.java, use GMT TimeZone:private static TimeZone gmt =
> TimeZone.getTimeZone("GMT")
>
> When inport data from Kafka Streaming Data，kylin generate "Data_Start",
> "Hour_Start" uses the GMT time zone, They are some time zones later than
> the real time. Why not use the time zone specified in the configuration file?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (KYLIN-3788) Kylin's internal time zone conversions are various and confusing
[
https://issues.apache.org/jira/browse/KYLIN-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhao jintao closed KYLIN-3788.
--
Resolution: Fixed
Fix Version/s: (was: v2.6.2)
v3.0.0-alpha
修复的版本为 v3.0.0-alpha
> Kylin's internal time zone conversions are various and confusing
> -
>
> Key: KYLIN-3788
> URL: https://issues.apache.org/jira/browse/KYLIN-3788
> Project: Kylin
> Issue Type: Bug
> Components: REST Service, Web
>Affects Versions: v2.5.2
> Environment: Huawei FI
>Reporter: zhao jintao
>Assignee: zhao jintao
>Priority: Major
> Labels: TimeZone
> Fix For: v3.0.0-alpha
>
>
> Hi, All team
> I found that Kylin has a variety of time zone conversions inside, which is
> confusing. In some code, the time zone is "GMT" when converting time, and
> some code uses the time zone specified in the configuration file
> ("TimeZone.getTimeZone("GMT")" );
> For Example:
> in CubingJob.java, use TimeZone from configuration file:
> format.setTimeZone(TimeZone.getTimeZone(
> config.getTimeZone())),
> but in TimeUtil.java, use GMT TimeZone:private static TimeZone gmt =
> TimeZone.getTimeZone("GMT")
>
> When inport data from Kafka Streaming Data，kylin generate "Data_Start",
> "Hour_Start" uses the GMT time zone, They are some time zones later than
> the real time. Why not use the time zone specified in the configuration file?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (KYLIN-3899) Kylin throws OutOfMemory exception during the sql query
[
https://issues.apache.org/jira/browse/KYLIN-3899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhao jintao closed KYLIN-3899.
--
Resolution: Fixed
Fix Version/s: v3.0.0-alpha
v2.6.2
已在此版本修复；[https://issues.apache.org/jira/projects/KYLIN/issues/KYLIN-3797?filter=allissues]
> Kylin throws OutOfMemory exception during the sql query
>
>
> Key: KYLIN-3899
> URL: https://issues.apache.org/jira/browse/KYLIN-3899
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v2.5.2
> Environment: Huawei FusionInsight
>Reporter: zhao jintao
>Priority: Major
> Fix For: v2.6.2, v3.0.0-alpha
>
> Original Estimate: 672h
> Remaining Estimate: 672h
>
> Hi team:
> I use Tableau+Kylin in my OLAP system.
> But Kylin sometimes throws an OutOfMemory exception and then crashed.
> I find "OutOfMemoryError info" in kylin.log:
> {quote}java.lang.OutOfMemoryError: Java heap space
> Dumping heap to java_pid24217.hprof ...
> #
> # java.lang.OutOfMemoryError: Java heap space
> # -XX:OutOfMemoryError="kill -9 %p"
> # Executing /bin/sh -c "kill -9 24217" ...
> Java HotSpot(TM) 64-Bit Server VM warning:ignoring option MaxPermSize=512m;
> support was removed in 8.0
> Java HotSpot(TM) 64-Bit Server VM warning:Using incremental CMS is deprecated
> and will likely be removed in a future release.
> {quote}
> Then I use Memory Analyzer to analysis dump file. I find that all most of
> memory is occupied by a Tableau sql query thread, that query uses 23.4G
> memory, 99% of all mepory.
>
> I find that query thread has a big linkedlist uses 19.7G memory, this
> linkedlist has LogicalTupleFilter entity, contains 43497028 entries.
>
> Then I find that the crash stack as these:
> {quote}at
> org.apache.kylin.metadata.filter.LogicalTupleFilter.copy()Lorg/apache/kylin/metadata/filter/TupleFilter;
> (LogicalTupleFilter.java 55)
> at
> org.apache.kylin.metadata.filter.TupleFilter.cartesianProduct(Ljava/util/List:Lorg/apache/kylin/metadata/filter/TupleFilter:)Ljava/util/List:
> (TupleFilter.java:279)
> at
> org.apache.kylin.metadata.filter.TupleFilter.flattenInternal(Lorg/apache/kylin/metadata/filter/TupleFilter:)Lorg/apache/kylin/metadata/filter/TupleFilter:
> (TupleFilter.java:252)
> at
> org.apache.kylin.metadata.filter.TupleFilter.flatFilter()Lorg/apache/kylin/metadata/filter/TupleFilter:
> (TupleFilter.java:218)
> at
> org.apache.kylin.cube.gridtable.ScanrangePlannerBase.flattenToOrAndFilter(Lorg/apache/kylin/metadata/filter/TupleFilter:)Lorg/apache/kylin/metadata/filter/TupleFilter:
> (ScanRangePlannerBase.java:73)
> at
> org.apache.kylin.storage.gtrecord.CubeScanRangePlanner.planScanRanges()Ljava/util/List;
> (CubeScanRangePlanner.java:216)
> ..
> {quote}
>
> I find that Kylin's SQL query statement which is generated by Tableau is very
> complex. The length of SQL query statement is 54836, with "IN", "NOT IN",
> multi-level "OR", "AND" conditions and group by filtering.
>
> Have other users ever had this problem?
>
> Best Regards.
> Thanks.
>
> Zhao Jintao.
>
>
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (KYLIN-3788) Kylin's internal time zone conversions are various and confusing
[
https://issues.apache.org/jira/browse/KYLIN-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhao jintao closed KYLIN-3788.
--
Resolution: Fixed
代码已合入主干，此问题可已修复，可关闭
> Kylin's internal time zone conversions are various and confusing
> -
>
> Key: KYLIN-3788
> URL: https://issues.apache.org/jira/browse/KYLIN-3788
> Project: Kylin
> Issue Type: Bug
> Components: REST Service, Web
>Affects Versions: v2.5.2
> Environment: Huawei FI
>Reporter: zhao jintao
>Assignee: zhao jintao
>Priority: Major
> Labels: TimeZone
> Fix For: v2.6.2
>
>
> Hi, All team
> I found that Kylin has a variety of time zone conversions inside, which is
> confusing. In some code, the time zone is "GMT" when converting time, and
> some code uses the time zone specified in the configuration file
> ("TimeZone.getTimeZone("GMT")" );
> For Example:
> in CubingJob.java, use TimeZone from configuration file:
> format.setTimeZone(TimeZone.getTimeZone(
> config.getTimeZone())),
> but in TimeUtil.java, use GMT TimeZone:private static TimeZone gmt =
> TimeZone.getTimeZone("GMT")
>
> When inport data from Kafka Streaming Data，kylin generate "Data_Start",
> "Hour_Start" uses the GMT time zone, They are some time zones later than
> the real time. Why not use the time zone specified in the configuration file?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3974) confused about the megering segments forcely function, can not work when there exists gaps
[ https://issues.apache.org/jira/browse/KYLIN-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hejian updated KYLIN-3974: -- Description: forcely !image-2019-04-23-19-02-05-743.png!!image-2019-04-23-19-02-53-396.png! was:!image-2019-04-23-19-02-05-743.png!!image-2019-04-23-19-02-53-396.png! > confused about the megering segments forcely function, can not work when > there exists gaps > -- > > Key: KYLIN-3974 > URL: https://issues.apache.org/jira/browse/KYLIN-3974 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.4.0 >Reporter: hejian >Priority: Major > Attachments: image-2019-04-23-18-57-01-878.png, > image-2019-04-23-19-02-05-743.png, image-2019-04-23-19-02-53-396.png > > > forcely > !image-2019-04-23-19-02-05-743.png!!image-2019-04-23-19-02-53-396.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (KYLIN-3974) confused about the megering segments forcely function, can not work when there exists gaps
[ https://issues.apache.org/jira/browse/KYLIN-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hejian updated KYLIN-3974: -- Comment: was deleted (was: can not find the force=true usage in source code as well) > confused about the megering segments forcely function, can not work when > there exists gaps > -- > > Key: KYLIN-3974 > URL: https://issues.apache.org/jira/browse/KYLIN-3974 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.4.0 >Reporter: hejian >Priority: Major > Attachments: image-2019-04-23-18-57-01-878.png, > image-2019-04-23-19-02-05-743.png, image-2019-04-23-19-02-53-396.png > > > !image-2019-04-23-19-02-05-743.png!!image-2019-04-23-19-02-53-396.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3974) confused about the megering segments forcely function, can not work when there exists gaps
[ https://issues.apache.org/jira/browse/KYLIN-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823980#comment-16823980 ] hejian commented on KYLIN-3974: --- can not find the force=true usage in source code as well > confused about the megering segments forcely function, can not work when > there exists gaps > -- > > Key: KYLIN-3974 > URL: https://issues.apache.org/jira/browse/KYLIN-3974 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.4.0 >Reporter: hejian >Priority: Major > Attachments: image-2019-04-23-18-57-01-878.png, > image-2019-04-23-19-02-05-743.png, image-2019-04-23-19-02-53-396.png > > > !image-2019-04-23-19-02-05-743.png!!image-2019-04-23-19-02-53-396.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3974) confused about the megering segments forcely function, can not work when there exists gaps
[ https://issues.apache.org/jira/browse/KYLIN-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hejian updated KYLIN-3974: -- Affects Version/s: v2.4.0 > confused about the megering segments forcely function, can not work when > there exists gaps > -- > > Key: KYLIN-3974 > URL: https://issues.apache.org/jira/browse/KYLIN-3974 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.4.0 >Reporter: hejian >Priority: Major > Attachments: image-2019-04-23-18-57-01-878.png, > image-2019-04-23-19-02-05-743.png, image-2019-04-23-19-02-53-396.png > > > !image-2019-04-23-19-02-05-743.png!!image-2019-04-23-19-02-53-396.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3974) confused about the megering segments forcely function, can not work when there exists gaps
[ https://issues.apache.org/jira/browse/KYLIN-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hejian updated KYLIN-3974: -- Attachment: image-2019-04-23-19-02-05-743.png > confused about the megering segments forcely function, can not work when > there exists gaps > -- > > Key: KYLIN-3974 > URL: https://issues.apache.org/jira/browse/KYLIN-3974 > Project: Kylin > Issue Type: Bug >Reporter: hejian >Priority: Major > Attachments: image-2019-04-23-18-57-01-878.png, > image-2019-04-23-19-02-05-743.png, image-2019-04-23-19-02-53-396.png > > > !image-2019-04-23-18-57-01-878.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3974) confused about the megering segments forcely function, can not work when there exists gaps
[ https://issues.apache.org/jira/browse/KYLIN-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hejian updated KYLIN-3974: -- Description: !image-2019-04-23-19-02-05-743.png!!image-2019-04-23-19-02-53-396.png! (was: !image-2019-04-23-18-57-01-878.png!) > confused about the megering segments forcely function, can not work when > there exists gaps > -- > > Key: KYLIN-3974 > URL: https://issues.apache.org/jira/browse/KYLIN-3974 > Project: Kylin > Issue Type: Bug >Reporter: hejian >Priority: Major > Attachments: image-2019-04-23-18-57-01-878.png, > image-2019-04-23-19-02-05-743.png, image-2019-04-23-19-02-53-396.png > > > !image-2019-04-23-19-02-05-743.png!!image-2019-04-23-19-02-53-396.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3974) confused about the megering segments forcely function, can not work when there exists gaps
[ https://issues.apache.org/jira/browse/KYLIN-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hejian updated KYLIN-3974: -- Attachment: image-2019-04-23-19-02-53-396.png > confused about the megering segments forcely function, can not work when > there exists gaps > -- > > Key: KYLIN-3974 > URL: https://issues.apache.org/jira/browse/KYLIN-3974 > Project: Kylin > Issue Type: Bug >Reporter: hejian >Priority: Major > Attachments: image-2019-04-23-18-57-01-878.png, > image-2019-04-23-19-02-05-743.png, image-2019-04-23-19-02-53-396.png > > > !image-2019-04-23-19-02-05-743.png!!image-2019-04-23-19-02-53-396.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3974) confused about the meger segments forcely function, can not work when there exists gaps
hejian created KYLIN-3974: - Summary: confused about the meger segments forcely function, can not work when there exists gaps Key: KYLIN-3974 URL: https://issues.apache.org/jira/browse/KYLIN-3974 Project: Kylin Issue Type: Bug Reporter: hejian Attachments: image-2019-04-23-18-57-01-878.png !image-2019-04-23-18-57-01-878.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3974) confused about the megering segments forcely function, can not work when there exists gaps
[ https://issues.apache.org/jira/browse/KYLIN-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hejian updated KYLIN-3974: -- Summary: confused about the megering segments forcely function, can not work when there exists gaps (was: confused about the meger segments forcely function, can not work when there exists gaps) > confused about the megering segments forcely function, can not work when > there exists gaps > -- > > Key: KYLIN-3974 > URL: https://issues.apache.org/jira/browse/KYLIN-3974 > Project: Kylin > Issue Type: Bug >Reporter: hejian >Priority: Major > Attachments: image-2019-04-23-18-57-01-878.png > > > !image-2019-04-23-18-57-01-878.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] coveralls commented on issue #618: KYLIN-3969 Fix Beeline meta data parser on partition information
coveralls commented on issue #618: KYLIN-3969 Fix Beeline meta data parser on partition information URL: https://github.com/apache/kylin/pull/618#issuecomment-485737837 ## Pull Request Test Coverage Report for [Build 4414](https://coveralls.io/builds/22966028) * **0** of **7** **(0.0%)** changed or added relevant lines in **1** file are covered. * **1** unchanged line in **1** file lost coverage. * Overall coverage increased (+**0.002%**) to **27.823%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [source-hive/src/main/java/org/apache/kylin/source/hive/BeelineHiveClient.java](https://coveralls.io/builds/22966028/source?filename=source-hive%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fhive%2FBeelineHiveClient.java#L206) | 0 | 7 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [source-hive/src/main/java/org/apache/kylin/source/hive/BeelineHiveClient.java](https://coveralls.io/builds/22966028/source?filename=source-hive%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fhive%2FBeelineHiveClient.java#L215) | 1 | 5.44% | | Totals | [](https://coveralls.io/builds/22966028) | | :-- | --: | | Change from base [Build 4413](https://coveralls.io/builds/22960555): | 0.002% | | Covered Lines: | 22755 | | Relevant Lines: | 81785 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] codecov-io commented on issue #618: KYLIN-3969 Fix Beeline meta data parser on partition information
codecov-io commented on issue #618: KYLIN-3969 Fix Beeline meta data parser on partition information URL: https://github.com/apache/kylin/pull/618#issuecomment-485736896 # [Codecov](https://codecov.io/gh/apache/kylin/pull/618?src=pr=h1) Report > Merging [#618](https://codecov.io/gh/apache/kylin/pull/618?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/cfabb816a54855b321d9c1ac2acdb76bb7d2652a?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `0%`. [](https://codecov.io/gh/apache/kylin/pull/618?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #618 +/- ## + Coverage 25.34% 25.34% +<.01% Complexity 5839 5839 Files 1379 1379 Lines 8178481785 +1 Branches 1145811459 +1 + Hits 2072520728 +3 + Misses5903559034 -1 + Partials 2024 2023 -1 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/618?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...rg/apache/kylin/source/hive/BeelineHiveClient.java](https://codecov.io/gh/apache/kylin/pull/618/diff?src=pr=tree#diff-c291cmNlLWhpdmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9oaXZlL0JlZWxpbmVIaXZlQ2xpZW50LmphdmE=) | `4.08% <0%> (-0.03%)` | `5 <0> (ø)` | | | [.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://codecov.io/gh/apache/kylin/pull/618/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9UcmVlQ3Vib2lkU2NoZWR1bGVyLmphdmE=) | `66.15% <0%> (+2.3%)` | `0% <0%> (ø)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/618?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/618?src=pr=footer). Last update [cfabb81...bbe554c](https://codecov.io/gh/apache/kylin/pull/618?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3971) InternalErrorException thrown when querying cube with TOP_N measure
[
https://issues.apache.org/jira/browse/KYLIN-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823904#comment-16823904
]
Grzegorz Kołakowski commented on KYLIN-3971:
It works if I add any other measure.
> InternalErrorException thrown when querying cube with TOP_N measure
> ---
>
> Key: KYLIN-3971
> URL: https://issues.apache.org/jira/browse/KYLIN-3971
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.6.1
>Reporter: Grzegorz Kołakowski
>Priority: Major
> Attachments: top_n_measure.png
>
>
> I have created a cube with only one measure TOP_N (except mandatory
> `_COUNT_`) and several dimensions.
> !top_n_measure.png!
> When I query it:
> {noformat}
> SELECT
> "DIC_ARTICLE"."SUPPLIER_NAME" AS "SUPPLIER_NAME",
> SUM("KYLIN_TICKETS"."SALE_VALUE") AS "X__alias__0"
> FROM
> "REPORT_SALES"."KYLIN_TICKETS" "KYLIN_TICKETS"
> LEFT JOIN
> "REPORT_SALES"."DIC_ARTICLE" "DIC_ARTICLE" ON
> ("KYLIN_TICKETS"."ID_ARTICLE" = "DIC_ARTICLE"."ID_ARTICLE")
> LEFT JOIN
> "REPORT_SALES"."KYLIN_DIC_STORE" "KYLIN_DIC_STORE" ON
> ("KYLIN_TICKETS"."ID_STORE" = "KYLIN_DIC_STORE"."ID_STORE")
> WHERE
> "KYLIN_TICKETS"."WORKING_DATE" = '2019-01-02'
> GROUP BY
> "DIC_ARTICLE"."SUPPLIER_NAME"
> ORDER BY
> "X__alias__0" DESC
> {noformat}
> I get the following exception:
> {noformat}
> 2019-04-23 09:10:46,618 ERROR [http-nio-7070-exec-3]
> controller.BasicController:63 :
> org.apache.kylin.rest.exception.InternalErrorException: RowType=91,
> ColumnRowType=90
> while executing SQL: "SELECT "DIC_ARTICLE"."SUPPLIER_NAME" AS
> "SUPPLIER_NAME", SUM("KYLIN_TICKETS"."SALE_VALUE") AS
> "X__alias__0"FROM "REPORT_SALES"."KYLIN_TICKETS"
> "KYLIN_TICKETS" LEFT JOIN "REPORT_SALES"."DIC_ARTICLE" "DIC_ARTICLE"
> ON ("KYLIN_TICKETS"."ID_ARTICLE" = "DIC_ARTICLE"."ID_ARTICLE") LEFT
> JOIN "REPORT_SALES"."KYLIN_DIC_STORE" "KYLIN_DIC_STORE" ON
> ("KYLIN_TICKETS"."ID_STORE" = "KYLIN_DIC_STORE"."ID_STORE")WHERE
>"KYLIN_TICKETS"."WORKING_DATE" = '2019-01-02' GROUP BY
> "DIC_ARTICLE"."SUPPLIER_NAME" ORDER BY "X__alias__0" DESC LIMIT 5"
> at
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:441)
> at
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:367)
> at
> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:87)
> at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
> at
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
> at
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
> at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
> at
> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:661)
> at
> org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at
> org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at
>
[jira] [Created] (KYLIN-3973) InvalidProtocolBufferException: Protocol message was too large. May be malicious.
Grzegorz Kołakowski created KYLIN-3973:
--
Summary: InvalidProtocolBufferException: Protocol message was too
large. May be malicious.
Key: KYLIN-3973
URL: https://issues.apache.org/jira/browse/KYLIN-3973
Project: Kylin
Issue Type: Bug
Affects Versions: v2.6.1
Reporter: Grzegorz Kołakowski
For many queries I receive the following exception.
{noformat}
2019-04-23 11:33:15,576 WARN [kylin-coproc--pool6-t17]
client.SyncCoprocessorRpcChannel:54 : Call failed on IOException
com.google.protobuf.InvalidProtocolBufferException: Protocol message was too
large. May be malicious. Use CodedInputStream.setSizeLimit() to increase the
size limit.
at
com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
at
com.google.protobuf.CodedInputStream.refillBuffer(CodedInputStream.java:755)
at
com.google.protobuf.CodedInputStream.isAtEnd(CodedInputStream.java:701)
at
com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:99)
at
org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse.(CubeVisitProtos.java:2307)
at
org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse.(CubeVisitProtos.java:2271)
at
org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$1.parsePartialFrom(CubeVisitProtos.java:2380)
at
org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$1.parsePartialFrom(CubeVisitProtos.java:2375)
at
org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$Builder.mergeFrom(CubeVisitProtos.java:5101)
at
org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitResponse$Builder.mergeFrom(CubeVisitProtos.java:4949)
at
com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:337)
at
com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
at
com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:210)
at
com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:904)
at
com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
at
org.apache.hadoop.hbase.ipc.CoprocessorRpcUtils.getResponse(CoprocessorRpcUtils.java:141)
at
org.apache.hadoop.hbase.client.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:94)
at
org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callMethod(SyncCoprocessorRpcChannel.java:52)
at
org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService$Stub.visitCube(CubeVisitProtos.java:5616)
at
org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:246)
at
org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1$1.call(CubeHBaseEndpointRPC.java:242)
at org.apache.hadoop.hbase.client.HTable$12.call(HTable.java:1012)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}
I use lz4 compression algorithm in HBase.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3964) segment overlapped, the status is "NEW", last_build_job_id is null, can not refresh/delete/build this segment
[ https://issues.apache.org/jira/browse/KYLIN-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823869#comment-16823869 ] hejian edited comment on KYLIN-3964 at 4/23/19 9:27 AM: hi [~Shaofengshi], hard to repro in test environment, even try the updated version. In recent days, this issue occurs frequently in formal environment with kylin-2.4.0, about 1 of 160 segments every day(the hbase connection timeout etc.) . I wonder know how can i handle the bad segment through restful api rather than edit the metadata, and i think the final solution of KYLIN-2849 can help a lot but i can not find it. was (Author: hejian999): hi [~Shaofengshi], hard to repro in test environment, even try the updated version. In recent days, this issue occurs frequently in formal environment with kylin-2.4.0, about 1 of 160 segments every day(the hbase connection timeout etc.) . I wonder know how can i handle the bad segment through restful api rather than edit the metadata, and i think the final solution of KYLIN-2849 can help a lot by i can not find the it even after read the related source code. > segment overlapped, the status is "NEW", last_build_job_id is null, can not > refresh/delete/build this segment > -- > > Key: KYLIN-3964 > URL: https://issues.apache.org/jira/browse/KYLIN-3964 > Project: Kylin > Issue Type: Bug > Environment: kylin-2.4.0 >Reporter: hejian >Priority: Major > Attachments: image-2019-04-22-10-07-06-737.png, > image-2019-04-23-16-51-34-805.png, image-2019-04-23-16-52-55-756.png > > > *Any action* *involved* *this segment can not be excute by requesting api due > to no job_id provided.* > !image-2019-04-22-10-07-06-737.png! > merge/refresh/rebuild/delete does not work > !image-2019-04-23-16-51-34-805.png!!image-2019-04-23-16-52-55-756.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3969) BeelineHiveClient can't parse Hive meta data result set with partitioned table
[
https://issues.apache.org/jira/browse/KYLIN-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823877#comment-16823877
]
ASF GitHub Bot commented on KYLIN-3969:
---
sickate commented on pull request #618: KYLIN-3969 Fix Beeline meta data parser
on partition information
URL: https://github.com/apache/kylin/pull/618
https://issues.apache.org/jira/browse/KYLIN-3969
In CLI + Hive2, "desc formatted tablename" outputs following text at
"Partition Information" section:
```
# Partition Information
# col_name data_type comment
pt string
```
And the code checks if the blank line exists.
But with Beeline client and Hive 3, the output becomes:
```
| # Partition Information | NULL
| NULL |
| # col_name| data_type
| comment |
| pt| string
| |
```
The blank line is gone, which causes problem when user trying to load table.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> BeelineHiveClient can't parse Hive meta data result set with partitioned table
> --
>
> Key: KYLIN-3969
> URL: https://issues.apache.org/jira/browse/KYLIN-3969
> Project: Kylin
> Issue Type: Bug
> Components: Metadata
>Affects Versions: v2.6.1
> Environment: Hive 3.1.0, HiveServer2, Beeline（Along with Ambari
> 2.7.1.0)
> CentOS Linux release 7.6.1810
>Reporter: Tuo Zhu
>Priority: Blocker
>
> When creating a model, you can add tables without partitions, but can't add
> ones with partitions.
> Web UI pops："Oops. cannot get HiveTableMeta".
> I tried same table in Hive2 with cli, it works.
> log prints:
> {code:java}
> java.lang.RuntimeException: cannot get HiveTableMeta
> at
> org.apache.kylin.source.hive.HiveMetadataExplorer.loadTableMetadata(HiveMetadataExplorer.java:68)
> at
> org.apache.kylin.rest.service.TableService.extractHiveTableMeta(TableService.java:205)
> at
> org.apache.kylin.rest.service.TableService.loadHiveTablesToProject(TableService.java:131)
> at
> org.apache.kylin.rest.controller.TableController.loadHiveTables(TableController.java:113)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
> at
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
> at
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
> at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
> at
> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:661)
> at
> org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
>
[GitHub] [kylin] asfgit commented on issue #618: KYLIN-3969 Fix Beeline meta data parser on partition information
asfgit commented on issue #618: KYLIN-3969 Fix Beeline meta data parser on partition information URL: https://github.com/apache/kylin/pull/618#issuecomment-485722134 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] sickate opened a new pull request #618: KYLIN-3969 Fix Beeline meta data parser on partition information
sickate opened a new pull request #618: KYLIN-3969 Fix Beeline meta data parser on partition information URL: https://github.com/apache/kylin/pull/618 https://issues.apache.org/jira/browse/KYLIN-3969 In CLI + Hive2, "desc formatted tablename" outputs following text at "Partition Information" section: ``` # Partition Information # col_name data_type comment pt string ``` And the code checks if the blank line exists. But with Beeline client and Hive 3, the output becomes: ``` | # Partition Information | NULL | NULL | | # col_name| data_type | comment | | pt| string | | ``` The blank line is gone, which causes problem when user trying to load table. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3964) segment overlapped, the status is "NEW", last_build_job_id is null, can not refresh/delete/build this segment
[ https://issues.apache.org/jira/browse/KYLIN-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823869#comment-16823869 ] hejian commented on KYLIN-3964: --- hi [~Shaofengshi], hard to repro in test environment, even try the updated version. In recent days, this issue occurs frequently in formal environment with kylin-2.4.0, about 1 of 160 segments every day(the hbase connection timeout etc.) . I wonder know how can i handle the bad segment through restful api rather than edit the metadata, and i think the final solution of KYLIN-2849 can help a lot by i can not find the it even after read the related source code. > segment overlapped, the status is "NEW", last_build_job_id is null, can not > refresh/delete/build this segment > -- > > Key: KYLIN-3964 > URL: https://issues.apache.org/jira/browse/KYLIN-3964 > Project: Kylin > Issue Type: Bug > Environment: kylin-2.4.0 >Reporter: hejian >Priority: Major > Attachments: image-2019-04-22-10-07-06-737.png, > image-2019-04-23-16-51-34-805.png, image-2019-04-23-16-52-55-756.png > > > *Any action* *involved* *this segment can not be excute by requesting api due > to no job_id provided.* > !image-2019-04-22-10-07-06-737.png! > merge/refresh/rebuild/delete does not work > !image-2019-04-23-16-51-34-805.png!!image-2019-04-23-16-52-55-756.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3971) InternalErrorException thrown when querying cube with TOP_N measure
[
https://issues.apache.org/jira/browse/KYLIN-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grzegorz Kołakowski updated KYLIN-3971:
---
Description:
I have created a cube with only one measure TOP_N (except mandatory `_COUNT_`)
and several dimensions.
!top_n_measure.png!
When I query it:
{noformat}
SELECT
"DIC_ARTICLE"."SUPPLIER_NAME" AS "SUPPLIER_NAME",
SUM("KYLIN_TICKETS"."SALE_VALUE") AS "X__alias__0"
FROM
"REPORT_SALES"."KYLIN_TICKETS" "KYLIN_TICKETS"
LEFT JOIN
"REPORT_SALES"."DIC_ARTICLE" "DIC_ARTICLE" ON ("KYLIN_TICKETS"."ID_ARTICLE"
= "DIC_ARTICLE"."ID_ARTICLE")
LEFT JOIN
"REPORT_SALES"."KYLIN_DIC_STORE" "KYLIN_DIC_STORE" ON
("KYLIN_TICKETS"."ID_STORE" = "KYLIN_DIC_STORE"."ID_STORE")
WHERE
"KYLIN_TICKETS"."WORKING_DATE" = '2019-01-02'
GROUP BY
"DIC_ARTICLE"."SUPPLIER_NAME"
ORDER BY
"X__alias__0" DESC
{noformat}
I get the following exception:
{noformat}
2019-04-23 09:10:46,618 ERROR [http-nio-7070-exec-3]
controller.BasicController:63 :
org.apache.kylin.rest.exception.InternalErrorException: RowType=91,
ColumnRowType=90
while executing SQL: "SELECT "DIC_ARTICLE"."SUPPLIER_NAME" AS
"SUPPLIER_NAME", SUM("KYLIN_TICKETS"."SALE_VALUE") AS
"X__alias__0"FROM "REPORT_SALES"."KYLIN_TICKETS"
"KYLIN_TICKETS" LEFT JOIN "REPORT_SALES"."DIC_ARTICLE" "DIC_ARTICLE"
ON ("KYLIN_TICKETS"."ID_ARTICLE" = "DIC_ARTICLE"."ID_ARTICLE") LEFT
JOIN "REPORT_SALES"."KYLIN_DIC_STORE" "KYLIN_DIC_STORE" ON
("KYLIN_TICKETS"."ID_STORE" = "KYLIN_DIC_STORE"."ID_STORE")WHERE
"KYLIN_TICKETS"."WORKING_DATE" = '2019-01-02' GROUP BY
"DIC_ARTICLE"."SUPPLIER_NAME" ORDER BY "X__alias__0" DESC LIMIT 5"
at
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:441)
at
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:367)
at
org.apache.kylin.rest.controller.QueryController.query(QueryController.java:87)
at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
at
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
at
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
at
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:661)
at
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
at
[jira] [Commented] (KYLIN-3788) Kylin's internal time zone conversions are various and confusing
[
https://issues.apache.org/jira/browse/KYLIN-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823851#comment-16823851
]
ASF subversion and git services commented on KYLIN-3788:
Commit 7d085b790d05836160ace072ad7e3fade4d13fc1 in kylin's branch
refs/heads/2.6.x from zhaojintaozhao
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=7d085b7 ]
KYLIN-3788 Modify the time conversion time zone of the kafka streaming access
* change simpleformat timezone of kafka streaming timestamp, such as
hour_start, day_start ...
* Modify the time conversion logic of the kafka data access; this fix solve
the problem that the streaming task time is different from the realtime due to
the time zone in kafka streaming.
* KYLIN-3788 This commit Changes SimpleDateformat to FastDateFormat because
SimpleDateFormat is forbidden API.
> Kylin's internal time zone conversions are various and confusing
> -
>
> Key: KYLIN-3788
> URL: https://issues.apache.org/jira/browse/KYLIN-3788
> Project: Kylin
> Issue Type: Bug
> Components: REST Service, Web
>Affects Versions: v2.5.2
> Environment: Huawei FI
>Reporter: zhao jintao
>Assignee: zhao jintao
>Priority: Major
> Labels: TimeZone
> Fix For: v2.6.2
>
>
> Hi, All team
> I found that Kylin has a variety of time zone conversions inside, which is
> confusing. In some code, the time zone is "GMT" when converting time, and
> some code uses the time zone specified in the configuration file
> ("TimeZone.getTimeZone("GMT")" );
> For Example:
> in CubingJob.java, use TimeZone from configuration file:
> format.setTimeZone(TimeZone.getTimeZone(
> config.getTimeZone())),
> but in TimeUtil.java, use GMT TimeZone:private static TimeZone gmt =
> TimeZone.getTimeZone("GMT")
>
> When inport data from Kafka Streaming Data，kylin generate "Data_Start",
> "Hour_Start" uses the GMT time zone, They are some time zones later than
> the real time. Why not use the time zone specified in the configuration file?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3892) Set cubing job priority
[ https://issues.apache.org/jira/browse/KYLIN-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823848#comment-16823848 ] ASF subversion and git services commented on KYLIN-3892: Commit a99183effc9d07653a915ec91dfe785dc150dab0 in kylin's branch refs/heads/2.6.x from Temple Zhou [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=a99183e ] KYLIN-3892 Set cubing job priority > Set cubing job priority > --- > > Key: KYLIN-3892 > URL: https://issues.apache.org/jira/browse/KYLIN-3892 > Project: Kylin > Issue Type: New Feature > Components: Job Engine >Affects Versions: v2.4.0, v2.5.0, v2.6.0 >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Minor > Fix For: v2.6.2 > > > The cubing job with high priority will be delayed when there are too many > tasks running. > So I want to set the job priority for the important cubing jobs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3950) Cube planner optimize job only use inmem algorithm
[ https://issues.apache.org/jira/browse/KYLIN-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823850#comment-16823850 ] ASF subversion and git services commented on KYLIN-3950: Commit fcd896c5bb153a5af142fced1aa3d7b41c9ecab2 in kylin's branch refs/heads/2.6.x from chao long [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=fcd896c ] KYLIN-3950 Make optimize job algorithm configurable > Cube planner optimize job only use inmem algorithm > -- > > Key: KYLIN-3950 > URL: https://issues.apache.org/jira/browse/KYLIN-3950 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.6.1 >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > > Currently, cube planner optimize job only use inmem algorithm to optimize > cube, which should be configurable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3938) can't delete job OPTIMIZE CHECKPOINT
[ https://issues.apache.org/jira/browse/KYLIN-3938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823849#comment-16823849 ] ASF subversion and git services commented on KYLIN-3938: Commit c711094f338e5700a9a2c33577f2e5f5f976619a in kylin's branch refs/heads/2.6.x from chao long [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=c711094 ] KYLIN-3938 Fix can't discard OPTMIZE CHECKPOINT job > can't delete job OPTIMIZE CHECKPOINT > > > Key: KYLIN-3938 > URL: https://issues.apache.org/jira/browse/KYLIN-3938 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.6.1 >Reporter: kaishen >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > > I'm using kylin-2.6.1-cdh57 ，when I use cube planner , it will generate tow > jobs one is : OPTIMIZE CUBE another is : OPTIMIZE CHECKPOINT which depend on > job OPTIMIZE CUBE . > now the first job OPTIMIZE CUBE was failed , and I want to discard the > second job , I got error : > java.lang.ClassCastException: > org.apache.kylin.job.execution.CheckpointExecutable cannot be cast to > org.apache.kylin.engine.mr.CubingJob -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3788) Kylin's internal time zone conversions are various and confusing
[
https://issues.apache.org/jira/browse/KYLIN-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823852#comment-16823852
]
ASF subversion and git services commented on KYLIN-3788:
Commit 7d085b790d05836160ace072ad7e3fade4d13fc1 in kylin's branch
refs/heads/2.6.x from zhaojintaozhao
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=7d085b7 ]
KYLIN-3788 Modify the time conversion time zone of the kafka streaming access
* change simpleformat timezone of kafka streaming timestamp, such as
hour_start, day_start ...
* Modify the time conversion logic of the kafka data access; this fix solve
the problem that the streaming task time is different from the realtime due to
the time zone in kafka streaming.
* KYLIN-3788 This commit Changes SimpleDateformat to FastDateFormat because
SimpleDateFormat is forbidden API.
> Kylin's internal time zone conversions are various and confusing
> -
>
> Key: KYLIN-3788
> URL: https://issues.apache.org/jira/browse/KYLIN-3788
> Project: Kylin
> Issue Type: Bug
> Components: REST Service, Web
>Affects Versions: v2.5.2
> Environment: Huawei FI
>Reporter: zhao jintao
>Assignee: zhao jintao
>Priority: Major
> Labels: TimeZone
> Fix For: v2.6.2
>
>
> Hi, All team
> I found that Kylin has a variety of time zone conversions inside, which is
> confusing. In some code, the time zone is "GMT" when converting time, and
> some code uses the time zone specified in the configuration file
> ("TimeZone.getTimeZone("GMT")" );
> For Example:
> in CubingJob.java, use TimeZone from configuration file:
> format.setTimeZone(TimeZone.getTimeZone(
> config.getTimeZone())),
> but in TimeUtil.java, use GMT TimeZone:private static TimeZone gmt =
> TimeZone.getTimeZone("GMT")
>
> When inport data from Kafka Streaming Data，kylin generate "Data_Start",
> "Hour_Start" uses the GMT time zone, They are some time zones later than
> the real time. Why not use the time zone specified in the configuration file?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3960) Only update user when login in LDAP environment
[ https://issues.apache.org/jira/browse/KYLIN-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823855#comment-16823855 ] ASF subversion and git services commented on KYLIN-3960: Commit 2d7d58579ca922b3715a48729e832c579fb56945 in kylin's branch refs/heads/2.6.x from Jiatao Tao [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=2d7d585 ] KYLIN-3960, Only update user when login in LDAP environment > Only update user when login in LDAP environment > --- > > Key: KYLIN-3960 > URL: https://issues.apache.org/jira/browse/KYLIN-3960 > Project: Kylin > Issue Type: Improvement > Components: Security >Reporter: Jiatao Tao >Assignee: Jiatao Tao >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3857) Add parameter to change sql quoting character for sqoop
[
https://issues.apache.org/jira/browse/KYLIN-3857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823854#comment-16823854
]
ASF subversion and git services commented on KYLIN-3857:
Commit 87ed932af38b9e3b14ca3d6370282cd465592758 in kylin's branch
refs/heads/2.6.x from hit-lacus
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=87ed932 ]
KYLIN-3857 add configuration for quote character
> Add parameter to change sql quoting character for sqoop
> ---
>
> Key: KYLIN-3857
> URL: https://issues.apache.org/jira/browse/KYLIN-3857
> Project: Kylin
> Issue Type: Bug
>Reporter: Davide Malagoli
>Assignee: XiaoXiang Yu
>Priority: Major
> Fix For: v3.0.0, v2.6.2
>
> Attachments: image-2019-03-11-11-46-05-946.png,
> image-2019-03-11-11-46-14-305.png, image-2019-03-11-11-46-20-664.png,
> image-2019-03-11-11-46-31-966.png, sqoop-error.log, sqoop-script.sh
>
>
> I've set up Kylin to access a SQL server using JDBC and do the ETL step as
> described in
> [http://kylin.apache.org/docs/tutorial/setup_jdbc_datasource.html]
> but when the "Sqoop To Flat Hive Table" runs it terminates with an error
> shown in the attached log.
> It seems that the root cause is the presence of the '`' character in the
> query, which has no meaning for sql server.
>
> My problem seems to related to this piece of code
>
> public class FlatTableSqlQuoteUtils {
> public static final String QUOTE = "`";
> /**
> * Quote identifier by default quote `
> * @param identifier
> * @return
> */
> public static String quoteIdentifier(String identifier){
> return QUOTE + identifier + QUOTE;
> }
>
> The solution may be to add a parameter to change this character used for
> quoting.
>
> Just to be sure I made a copy of the original "sqoop" command to "sqoop.orig"
> and replaced the original with a little script of mine (attached).
> This little script removes the "`" character from the original command and
> the call "sqoop.orig" with the cleaned arguments.
> It works, but it is still an ugly workaround.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3957) Query system_cube get exception Cannot cast "java.math.BigDecimal" to "java.lang.Double"
[
https://issues.apache.org/jira/browse/KYLIN-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823853#comment-16823853
]
ASF subversion and git services commented on KYLIN-3957:
Commit cd93993245336617913a4a7d9d0e49ae815f6691 in kylin's branch
refs/heads/2.6.x from chao long
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=cd93993 ]
KYLIN-3957 Fix exception cannot cast 'java.math.BigDecimal' to
'java.lang.Double'
> Query system_cube get exception Cannot cast "java.math.BigDecimal" to
> "java.lang.Double"
>
>
> Key: KYLIN-3957
> URL: https://issues.apache.org/jira/browse/KYLIN-3957
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Reporter: Chao Long
>Assignee: Chao Long
>Priority: Major
> Fix For: v3.0.0, v2.6.2
>
>
> In system cube, the return dataType of column whose real dataType is Double
> will be converted to Decimal in SUM measure.
> {code:java}
> FunctionDesc function = new FunctionDesc();
> function.setExpression(FunctionDesc.FUNC_SUM);
> function.setParameter(parameterDesc);
>
> function.setReturnType(dataType.equals(HiveTableCreator.HiveTypeEnum.HDOUBLE.toString())
> ? HiveTableCreator.HiveTypeEnum.HDECIMAL.toString()
> : dataType);
> {code}
> but query with that measure will get exception:
> {code}
> Caused by: org.codehaus.commons.compiler.CompileException: Line 108, Column
> 44: Cannot cast "java.math.BigDecimal" to "java.lang.Double"
> at
> org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:10092)
> at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:3839)
> at org.codehaus.janino.UnitCompiler.access$6400(UnitCompiler.java:183)
> at org.codehaus.janino.UnitCompiler$10.visitCast(UnitCompiler.java:3246)
> at org.codehaus.janino.Java$Cast.accept(Java.java:3802)
> at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:3278)
> at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:3845)
> at org.codehaus.janino.UnitCompiler.access$8600(UnitCompiler.java:183)
> at
> org.codehaus.janino.UnitCompiler$10.visitParenthesizedExpression(UnitCompiler.java:3274)
> at
> org.codehaus.janino.Java$ParenthesizedExpression.accept(Java.java:3830)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3964) segment overlapped, the status is "NEW", last_build_job_id is null, can not refresh/delete/build this segment
[ https://issues.apache.org/jira/browse/KYLIN-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hejian updated KYLIN-3964: -- Description: *Any action* *involved* *this segment can not be excute by requesting api due to no job_id provided.* !image-2019-04-22-10-07-06-737.png! merge/refresh/rebuild/delete does not work !image-2019-04-23-16-51-34-805.png!!image-2019-04-23-16-52-55-756.png! was: *Any action* *involved* *this segment can not be excute by requesting api due to no job_id provided.* !image-2019-04-22-10-07-06-737.png! > segment overlapped, the status is "NEW", last_build_job_id is null, can not > refresh/delete/build this segment > -- > > Key: KYLIN-3964 > URL: https://issues.apache.org/jira/browse/KYLIN-3964 > Project: Kylin > Issue Type: Bug > Environment: kylin-2.4.0 >Reporter: hejian >Priority: Major > Attachments: image-2019-04-22-10-07-06-737.png, > image-2019-04-23-16-51-34-805.png, image-2019-04-23-16-52-55-756.png > > > *Any action* *involved* *this segment can not be excute by requesting api due > to no job_id provided.* > !image-2019-04-22-10-07-06-737.png! > merge/refresh/rebuild/delete does not work > !image-2019-04-23-16-51-34-805.png!!image-2019-04-23-16-52-55-756.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3964) segment overlapped, the status is "NEW", last_build_job_id is null, can not refresh/delete/build this segment
[ https://issues.apache.org/jira/browse/KYLIN-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hejian updated KYLIN-3964: -- Attachment: image-2019-04-23-16-52-55-756.png > segment overlapped, the status is "NEW", last_build_job_id is null, can not > refresh/delete/build this segment > -- > > Key: KYLIN-3964 > URL: https://issues.apache.org/jira/browse/KYLIN-3964 > Project: Kylin > Issue Type: Bug > Environment: kylin-2.4.0 >Reporter: hejian >Priority: Major > Attachments: image-2019-04-22-10-07-06-737.png, > image-2019-04-23-16-51-34-805.png, image-2019-04-23-16-52-55-756.png > > > *Any action* *involved* *this segment can not be excute by requesting api due > to no job_id provided.* > !image-2019-04-22-10-07-06-737.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3964) segment overlapped, the status is "NEW", last_build_job_id is null, can not refresh/delete/build this segment
[ https://issues.apache.org/jira/browse/KYLIN-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hejian updated KYLIN-3964: -- Attachment: image-2019-04-23-16-51-34-805.png > segment overlapped, the status is "NEW", last_build_job_id is null, can not > refresh/delete/build this segment > -- > > Key: KYLIN-3964 > URL: https://issues.apache.org/jira/browse/KYLIN-3964 > Project: Kylin > Issue Type: Bug > Environment: kylin-2.4.0 >Reporter: hejian >Priority: Major > Attachments: image-2019-04-22-10-07-06-737.png, > image-2019-04-23-16-51-34-805.png > > > *Any action* *involved* *this segment can not be excute by requesting api due > to no job_id provided.* > !image-2019-04-22-10-07-06-737.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3946) No cube for AVG measure after include count column
[ https://issues.apache.org/jira/browse/KYLIN-3946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3946: Fix Version/s: (was: v2.6.2) v3.0.0 > No cube for AVG measure after include count column > -- > > Key: KYLIN-3946 > URL: https://issues.apache.org/jira/browse/KYLIN-3946 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v2.6.1 >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v3.0.0 > > > Before, avg(col) will use sum(col)/count(1) to calculate. > After including count column aggregation KYLIN-3883, avg(col) will use > sum(col)/count(col) to calculate. > If there is no predefined count(col) measure, query with avg(col) will get > exception "NoRealizationFoundException: No realization found for > OLAPContext", which will effect the query on old cubes. So we should consider > compatibility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3972) Kylin chinese website top navbar always open in mobile mode
[ https://issues.apache.org/jira/browse/KYLIN-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823809#comment-16823809 ] ASF GitHub Bot commented on KYLIN-3972: --- luguosheng1314 commented on pull request #617: KYLIN-3972, fix for mobile chinese navbar issue URL: https://github.com/apache/kylin/pull/617 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Kylin chinese website top navbar always open in mobile mode > --- > > Key: KYLIN-3972 > URL: https://issues.apache.org/jira/browse/KYLIN-3972 > Project: Kylin > Issue Type: Bug > Components: Website >Reporter: Roger >Assignee: Roger >Priority: Major > Attachments: WX20190423-154740.png > > > Kylin chinese website top navbar always open in mobile mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3972) Kylin chinese website top navbar always open in mobile mode
[ https://issues.apache.org/jira/browse/KYLIN-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823810#comment-16823810 ] ASF subversion and git services commented on KYLIN-3972: Commit 195933d753c4f61434257e0cc86d58a63213 in kylin's branch refs/heads/document from Emiya0306 [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=195933d ] KYLIN-3972, fix for mobile chinese navbar issue > Kylin chinese website top navbar always open in mobile mode > --- > > Key: KYLIN-3972 > URL: https://issues.apache.org/jira/browse/KYLIN-3972 > Project: Kylin > Issue Type: Bug > Components: Website >Reporter: Roger >Assignee: Roger >Priority: Major > Attachments: WX20190423-154740.png > > > Kylin chinese website top navbar always open in mobile mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] luguosheng1314 merged pull request #617: KYLIN-3972, fix for mobile chinese navbar issue
luguosheng1314 merged pull request #617: KYLIN-3972, fix for mobile chinese navbar issue URL: https://github.com/apache/kylin/pull/617 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3972) Kylin chinese website top navbar always open in mobile mode
[ https://issues.apache.org/jira/browse/KYLIN-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823805#comment-16823805 ] ASF GitHub Bot commented on KYLIN-3972: --- Emiya0306 commented on pull request #617: KYLIN-3972, fix for mobile chinese navbar issue URL: https://github.com/apache/kylin/pull/617 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Kylin chinese website top navbar always open in mobile mode > --- > > Key: KYLIN-3972 > URL: https://issues.apache.org/jira/browse/KYLIN-3972 > Project: Kylin > Issue Type: Bug > Components: Website >Reporter: Roger >Assignee: Roger >Priority: Major > Attachments: WX20190423-154740.png > > > Kylin chinese website top navbar always open in mobile mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] asfgit commented on issue #617: KYLIN-3972, fix for mobile chinese navbar issue
asfgit commented on issue #617: KYLIN-3972, fix for mobile chinese navbar issue URL: https://github.com/apache/kylin/pull/617#issuecomment-485691662 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] Emiya0306 opened a new pull request #617: KYLIN-3972, fix for mobile chinese navbar issue
Emiya0306 opened a new pull request #617: KYLIN-3972, fix for mobile chinese navbar issue URL: https://github.com/apache/kylin/pull/617 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (KYLIN-3932) KafkaConfigOverride to take effect
[ https://issues.apache.org/jira/browse/KYLIN-3932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3932: Fix Version/s: v3.0.0 > KafkaConfigOverride to take effect > -- > > Key: KYLIN-3932 > URL: https://issues.apache.org/jira/browse/KYLIN-3932 > Project: Kylin > Issue Type: Improvement >Reporter: jinguowei >Assignee: jinguowei >Priority: Major > Fix For: v3.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KYLIN-3932) KafkaConfigOverride to take effect
[ https://issues.apache.org/jira/browse/KYLIN-3932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-3932. - Resolution: Fixed > KafkaConfigOverride to take effect > -- > > Key: KYLIN-3932 > URL: https://issues.apache.org/jira/browse/KYLIN-3932 > Project: Kylin > Issue Type: Improvement >Reporter: jinguowei >Assignee: jinguowei >Priority: Major > Fix For: v3.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (KYLIN-3932) KafkaConfigOverride to take effect
[ https://issues.apache.org/jira/browse/KYLIN-3932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen reopened KYLIN-3932: - reopen to set release version > KafkaConfigOverride to take effect > -- > > Key: KYLIN-3932 > URL: https://issues.apache.org/jira/browse/KYLIN-3932 > Project: Kylin > Issue Type: Improvement >Reporter: jinguowei >Assignee: jinguowei >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3788) Kylin's internal time zone conversions are various and confusing
[
https://issues.apache.org/jira/browse/KYLIN-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-3788:
Fix Version/s: v2.6.2
> Kylin's internal time zone conversions are various and confusing
> -
>
> Key: KYLIN-3788
> URL: https://issues.apache.org/jira/browse/KYLIN-3788
> Project: Kylin
> Issue Type: Bug
> Components: REST Service, Web
>Affects Versions: v2.5.2
> Environment: Huawei FI
>Reporter: zhao jintao
>Assignee: zhao jintao
>Priority: Major
> Labels: TimeZone
> Fix For: v2.6.2
>
>
> Hi, All team
> I found that Kylin has a variety of time zone conversions inside, which is
> confusing. In some code, the time zone is "GMT" when converting time, and
> some code uses the time zone specified in the configuration file
> ("TimeZone.getTimeZone("GMT")" );
> For Example:
> in CubingJob.java, use TimeZone from configuration file:
> format.setTimeZone(TimeZone.getTimeZone(
> config.getTimeZone())),
> but in TimeUtil.java, use GMT TimeZone:private static TimeZone gmt =
> TimeZone.getTimeZone("GMT")
>
> When inport data from Kafka Streaming Data，kylin generate "Data_Start",
> "Hour_Start" uses the GMT time zone, They are some time zones later than
> the real time. Why not use the time zone specified in the configuration file?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3972) Kylin chinese website top navbar always open in mobile mode
[ https://issues.apache.org/jira/browse/KYLIN-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger updated KYLIN-3972: - Attachment: WX20190423-154740.png > Kylin chinese website top navbar always open in mobile mode > --- > > Key: KYLIN-3972 > URL: https://issues.apache.org/jira/browse/KYLIN-3972 > Project: Kylin > Issue Type: Bug > Components: Website >Reporter: Roger >Assignee: Roger >Priority: Major > Attachments: WX20190423-154740.png > > > Kylin chinese website top navbar always open in mobile mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3972) Kylin chinese website top navbar always open in mobile mode
Roger created KYLIN-3972: Summary: Kylin chinese website top navbar always open in mobile mode Key: KYLIN-3972 URL: https://issues.apache.org/jira/browse/KYLIN-3972 Project: Kylin Issue Type: Bug Components: Website Reporter: Roger Assignee: Roger Kylin chinese website top navbar always open in mobile mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3950) Cube planner optimize job only use inmem algorithm
[ https://issues.apache.org/jira/browse/KYLIN-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3950: Affects Version/s: v2.6.1 Fix Version/s: (was: Future) v2.6.2 > Cube planner optimize job only use inmem algorithm > -- > > Key: KYLIN-3950 > URL: https://issues.apache.org/jira/browse/KYLIN-3950 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.6.1 >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > > Currently, cube planner optimize job only use inmem algorithm to optimize > cube, which should be configurable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3966) Metric multiplication and metric division not working in query
[ https://issues.apache.org/jira/browse/KYLIN-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823776#comment-16823776 ] Gladson Vas commented on KYLIN-3966: [~hejian999] hi can you elaborate on pre defined expression or give an example on how to achieve the above using a select query? > Metric multiplication and metric division not working in query > -- > > Key: KYLIN-3966 > URL: https://issues.apache.org/jira/browse/KYLIN-3966 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Reporter: Gladson Vas >Priority: Blocker > > When i try to multiply or divide 2 metric columns in a query > eg: select col1*col2 from table > select col1/col2 from table. > it throws the following errors > That both of the two sides of the BinaryTupleExpression own columns is not > supported for * while executing SQL > That both of the two sides of the BinaryTupleExpression own columns is not > supported for / while executing SQL. > But when i use the + or - operators in the query it seems to be working fine. > Is there any way to support these 2 operators as well? > Thanks, > Gladson > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3319) exceeds threshold 5000000 while executing SQL
[
https://issues.apache.org/jira/browse/KYLIN-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823775#comment-16823775
]
hejian commented on KYLIN-3319:
---
Can the excessive segments aslo result in that issue?
> exceeds threshold 500 while executing SQL
> -
>
> Key: KYLIN-3319
> URL: https://issues.apache.org/jira/browse/KYLIN-3319
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v2.3.0
>Reporter: cc
>Priority: Blocker
> Attachments: image.png, sql_limit_result.png
>
>
> {color:#d04437}hello,what is the reason of the errors?{color}
> {color:#d04437}Query returned 5008662 rows exceeds threshold 500 while
> executing SQL: "select * from USER_FREQ_DAY_TEST LIMIT 5"{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3966) Metric multiplication and metric division not working in query
[ https://issues.apache.org/jira/browse/KYLIN-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823771#comment-16823771 ] hejian commented on KYLIN-3966: --- you should use the pre-defined expression on the measure > Metric multiplication and metric division not working in query > -- > > Key: KYLIN-3966 > URL: https://issues.apache.org/jira/browse/KYLIN-3966 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Reporter: Gladson Vas >Priority: Blocker > > When i try to multiply or divide 2 metric columns in a query > eg: select col1*col2 from table > select col1/col2 from table. > it throws the following errors > That both of the two sides of the BinaryTupleExpression own columns is not > supported for * while executing SQL > That both of the two sides of the BinaryTupleExpression own columns is not > supported for / while executing SQL. > But when i use the + or - operators in the query it seems to be working fine. > Is there any way to support these 2 operators as well? > Thanks, > Gladson > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (KYLIN-3966) Metric multiplication and metric division not working in query
[ https://issues.apache.org/jira/browse/KYLIN-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hejian updated KYLIN-3966: -- Comment: was deleted (was: make sure the two columns are both bigint) > Metric multiplication and metric division not working in query > -- > > Key: KYLIN-3966 > URL: https://issues.apache.org/jira/browse/KYLIN-3966 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Reporter: Gladson Vas >Priority: Blocker > > When i try to multiply or divide 2 metric columns in a query > eg: select col1*col2 from table > select col1/col2 from table. > it throws the following errors > That both of the two sides of the BinaryTupleExpression own columns is not > supported for * while executing SQL > That both of the two sides of the BinaryTupleExpression own columns is not > supported for / while executing SQL. > But when i use the + or - operators in the query it seems to be working fine. > Is there any way to support these 2 operators as well? > Thanks, > Gladson > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3946) No cube for AVG measure after include count column
[ https://issues.apache.org/jira/browse/KYLIN-3946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3946: Affects Version/s: v2.6.1 Fix Version/s: (was: Future) v2.6.2 > No cube for AVG measure after include count column > -- > > Key: KYLIN-3946 > URL: https://issues.apache.org/jira/browse/KYLIN-3946 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v2.6.1 >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > > Before, avg(col) will use sum(col)/count(1) to calculate. > After including count column aggregation KYLIN-3883, avg(col) will use > sum(col)/count(col) to calculate. > If there is no predefined count(col) measure, query with avg(col) will get > exception "NoRealizationFoundException: No realization found for > OLAPContext", which will effect the query on old cubes. So we should consider > compatibility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3966) Metric multiplication and metric division not working in query
[ https://issues.apache.org/jira/browse/KYLIN-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823768#comment-16823768 ] hejian commented on KYLIN-3966: --- make sure the two columns are both bigint > Metric multiplication and metric division not working in query > -- > > Key: KYLIN-3966 > URL: https://issues.apache.org/jira/browse/KYLIN-3966 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Reporter: Gladson Vas >Priority: Blocker > > When i try to multiply or divide 2 metric columns in a query > eg: select col1*col2 from table > select col1/col2 from table. > it throws the following errors > That both of the two sides of the BinaryTupleExpression own columns is not > supported for * while executing SQL > That both of the two sides of the BinaryTupleExpression own columns is not > supported for / while executing SQL. > But when i use the + or - operators in the query it seems to be working fine. > Is there any way to support these 2 operators as well? > Thanks, > Gladson > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3967) sum along with case expression does not work in query
[ https://issues.apache.org/jira/browse/KYLIN-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823765#comment-16823765 ] hejian commented on KYLIN-3967: --- i dont think those complicate scenes make any sence to pre-computation > sum along with case expression does not work in query > - > > Key: KYLIN-3967 > URL: https://issues.apache.org/jira/browse/KYLIN-3967 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Reporter: Gladson Vas >Priority: Blocker > > When i try to run a query with a sum case expression combination, > eg: select sum(case when col1<0 then 0 else col1 end ) from table > i get the following error: > No realization found for OLAPContext, CUBE_UNMATCHED_AGGREGATION[FunctionDesc > [expression=SUM, parameter=CASE(<($8, 0), 0, $8), returnType=null]], > rel#36838:OLAPTableScan.OLAP.[](table=[DEFAULT, table],ctx=,fields=[0, 1, 2, > 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, > 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, > 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, > 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, > 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, > 100, 101, 102, 103, 104, 105, 106]) while executing SQL: "select sum (case > when col1 <0 then 0 else col1 end ) from table LIMIT 5" > > is there any way to support this sum case expression in the query engine? > Also I get the same error when the sum operation is done on a column derived > from a case expression in a subquery. > eg: select sum(a.col1) from (select case when col1<0 then 0 else col1 end as > col1 from table) a > Thanks, > Gladson > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3971) InternalErrorException thrown when querying cube with TOP_N measure
Grzegorz Kołakowski created KYLIN-3971:
--
Summary: InternalErrorException thrown when querying cube with
TOP_N measure
Key: KYLIN-3971
URL: https://issues.apache.org/jira/browse/KYLIN-3971
Project: Kylin
Issue Type: Bug
Affects Versions: v2.6.1
Reporter: Grzegorz Kołakowski
Attachments: top_n_measure.png
I have created a cube with only one measure TOP_N and several dimensions.
!top_n_measure.png!
When I query it:
{noformat}
SELECT
"DIC_ARTICLE"."SUPPLIER_NAME" AS "SUPPLIER_NAME",
SUM("KYLIN_TICKETS"."SALE_VALUE") AS "X__alias__0"
FROM
"REPORT_SALES"."KYLIN_TICKETS" "KYLIN_TICKETS"
LEFT JOIN
"REPORT_SALES"."DIC_ARTICLE" "DIC_ARTICLE" ON ("KYLIN_TICKETS"."ID_ARTICLE"
= "DIC_ARTICLE"."ID_ARTICLE")
LEFT JOIN
"REPORT_SALES"."KYLIN_DIC_STORE" "KYLIN_DIC_STORE" ON
("KYLIN_TICKETS"."ID_STORE" = "KYLIN_DIC_STORE"."ID_STORE")
WHERE
"KYLIN_TICKETS"."WORKING_DATE" = '2019-01-02'
GROUP BY
"DIC_ARTICLE"."SUPPLIER_NAME"
ORDER BY
"X__alias__0" DESC
{noformat}
I get the following exception:
{noformat}
2019-04-23 09:10:46,618 ERROR [http-nio-7070-exec-3]
controller.BasicController:63 :
org.apache.kylin.rest.exception.InternalErrorException: RowType=91,
ColumnRowType=90
while executing SQL: "SELECT "DIC_ARTICLE"."SUPPLIER_NAME" AS
"SUPPLIER_NAME", SUM("KYLIN_TICKETS"."SALE_VALUE") AS
"X__alias__0"FROM "REPORT_SALES"."KYLIN_TICKETS"
"KYLIN_TICKETS" LEFT JOIN "REPORT_SALES"."DIC_ARTICLE" "DIC_ARTICLE"
ON ("KYLIN_TICKETS"."ID_ARTICLE" = "DIC_ARTICLE"."ID_ARTICLE") LEFT
JOIN "REPORT_SALES"."KYLIN_DIC_STORE" "KYLIN_DIC_STORE" ON
("KYLIN_TICKETS"."ID_STORE" = "KYLIN_DIC_STORE"."ID_STORE")WHERE
"KYLIN_TICKETS"."WORKING_DATE" = '2019-01-02' GROUP BY
"DIC_ARTICLE"."SUPPLIER_NAME" ORDER BY "X__alias__0" DESC LIMIT 5"
at
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:441)
at
org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:367)
at
org.apache.kylin.rest.controller.QueryController.query(QueryController.java:87)
at sun.reflect.GeneratedMethodAccessor186.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
at
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
at
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
at
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:661)
at
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at
[jira] [Updated] (KYLIN-3938) can't delete job OPTIMIZE CHECKPOINT
[ https://issues.apache.org/jira/browse/KYLIN-3938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3938: Fix Version/s: (was: Future) v2.6.2 > can't delete job OPTIMIZE CHECKPOINT > > > Key: KYLIN-3938 > URL: https://issues.apache.org/jira/browse/KYLIN-3938 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.6.1 >Reporter: kaishen >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > > I'm using kylin-2.6.1-cdh57 ，when I use cube planner , it will generate tow > jobs one is : OPTIMIZE CUBE another is : OPTIMIZE CHECKPOINT which depend on > job OPTIMIZE CUBE . > now the first job OPTIMIZE CUBE was failed , and I want to discard the > second job , I got error : > java.lang.ClassCastException: > org.apache.kylin.job.execution.CheckpointExecutable cannot be cast to > org.apache.kylin.engine.mr.CubingJob -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KYLIN-3912) Support cube level mapreduce queue config for BeelineHiveClient
[
https://issues.apache.org/jira/browse/KYLIN-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-3912.
-
Resolution: Fixed
> Support cube level mapreduce queue config for BeelineHiveClient
> ---
>
> Key: KYLIN-3912
> URL: https://issues.apache.org/jira/browse/KYLIN-3912
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
>Affects Versions: v2.6.1
>Reporter: Liu Shaohui
>Priority: Major
> Fix For: v3.0.0
>
>
> To support multi tenants, we set different mapreduce queue config for
> different projects and cubes, but BeelineHiveClient don't use those configs.
> So the getHiveTableRows api always run on same queue in kylin_hive_conf or
> jdbc url, which cause computing resource competition.
>
> {code:java}
> 2018-11-28 15:37:27,261 ERROR [Scheduler 1950398337 Job
> 08b3ee43-c84d-4039-84c5-a36ecb2cff18-228] execution.AbstractExecutable:383 :
> job:08b3ee43-c84d-4039-84c5-a36ecb2cff18-01 execute finished with exception
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> downstreamError is null.
> Query log:
> http://zjy-hadoop-prc-ct14.bj:28911/log?qid=a05e1629-2072-46dd-9d71-b5722d04b2aa
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:277)
> at
> org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:376)
> at
> org.apache.kylin.source.hive.BeelineHiveClient.getHiveTableRows(BeelineHiveClient.java:108)
> at
> org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiveTableStep.computeRowCount(HiveMRInput.java:304)
> at
> org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiveTableStep.doWork(HiveMRInput.java:354)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:67)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:300)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3912) Support cube level mapreduce queue config for BeelineHiveClient
[
https://issues.apache.org/jira/browse/KYLIN-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-3912:
Affects Version/s: v2.6.1
Fix Version/s: v3.0.0
Component/s: Job Engine
> Support cube level mapreduce queue config for BeelineHiveClient
> ---
>
> Key: KYLIN-3912
> URL: https://issues.apache.org/jira/browse/KYLIN-3912
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
>Affects Versions: v2.6.1
>Reporter: Liu Shaohui
>Priority: Major
> Fix For: v3.0.0
>
>
> To support multi tenants, we set different mapreduce queue config for
> different projects and cubes, but BeelineHiveClient don't use those configs.
> So the getHiveTableRows api always run on same queue in kylin_hive_conf or
> jdbc url, which cause computing resource competition.
>
> {code:java}
> 2018-11-28 15:37:27,261 ERROR [Scheduler 1950398337 Job
> 08b3ee43-c84d-4039-84c5-a36ecb2cff18-228] execution.AbstractExecutable:383 :
> job:08b3ee43-c84d-4039-84c5-a36ecb2cff18-01 execute finished with exception
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> downstreamError is null.
> Query log:
> http://zjy-hadoop-prc-ct14.bj:28911/log?qid=a05e1629-2072-46dd-9d71-b5722d04b2aa
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:277)
> at
> org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:376)
> at
> org.apache.kylin.source.hive.BeelineHiveClient.getHiveTableRows(BeelineHiveClient.java:108)
> at
> org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiveTableStep.computeRowCount(HiveMRInput.java:304)
> at
> org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiveTableStep.doWork(HiveMRInput.java:354)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:67)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:300)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3970) NRT Streaming cannot work at HDP3.0
[ https://issues.apache.org/jira/browse/KYLIN-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823735#comment-16823735 ] XiaoXiang Yu commented on KYLIN-3970: - h3. *First problem:* *Kafka jar conflict* org.apache.kylin.engine.mr.exception.MapReduceException: no counters for job job_1555653800318_0004Job Diagnostics:Task failed task_1555653800318_0004_m_00 Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0 Failure task Diagnostics: Error: org.apache.kafka.clients.consumer.ConsumerConfig.configNames()Ljava/util/Set; at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:173) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) h3. *Second problem: Hive null value* org.apache.kylin.engine.mr.exception.MapReduceException: no counters for job job_1555653800318_0007Job Diagnostics:Task failed task_1555653800318_0007_m_00 Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0 Failure task Diagnostics: Error: java.lang.IllegalArgumentException: there is no valid date pattern for:\N at org.apache.kylin.common.util.DateFormat.stringToMillis(DateFormat.java:124) at org.apache.kylin.metadata.datatype.DataTypeOrder$DateTimeOrder.toComparable(DataTypeOrder.java:154) at org.apache.kylin.metadata.datatype.DataTypeOrder$DateTimeOrder.toComparable(DataTypeOrder.java:148) at org.apache.kylin.metadata.datatype.DataTypeOrder.compare(DataTypeOrder.java:95) at org.apache.kylin.metadata.datatype.DataTypeOrder.max(DataTypeOrder.java:90) at org.apache.kylin.engine.mr.steps.FactDistinctColumnsMapper.doMap(FactDistinctColumnsMapper.java:194) at org.apache.kylin.engine.mr.KylinMapper.map(KylinMapper.java:77) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168) at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:173) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) > NRT Streaming cannot work at HDP3.0 > --- > > Key: KYLIN-3970 > URL: https://issues.apache.org/jira/browse/KYLIN-3970 > Project: Kylin > Issue Type: Bug > Components: NRT Streaming >Affects Versions: v2.6.1 >Reporter: XiaoXiang Yu >Assignee: XiaoXiang Yu >Priority: Major > Labels: HDP, Hadoop3 > Fix For: v3.0.0, v2.6.2 > > Attachments: image-2019-04-23-14-25-51-066.png > > > We found that when using > [apache-kylin-2.6.1-bin-hadoop3.tar.gz|https://www.apache.org/dyn/closer.cgi/kylin/apache-kylin-2.6.1/apache-kylin-2.6.1-bin-hadoop3.tar.gz], > NRT streaming cannot work in serval step for jar conflict problem. > > !image-2019-04-23-14-25-51-066.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3970) NRT Streaming cannot work at HDP3.0
[ https://issues.apache.org/jira/browse/KYLIN-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaoXiang Yu updated KYLIN-3970: Description: We found that when using [apache-kylin-2.6.1-bin-hadoop3.tar.gz|https://www.apache.org/dyn/closer.cgi/kylin/apache-kylin-2.6.1/apache-kylin-2.6.1-bin-hadoop3.tar.gz], NRT streaming cannot work in serval step for jar conflict problem. !image-2019-04-23-14-25-51-066.png! was:We found that when using [apache-kylin-2.6.1-bin-hadoop3.tar.gz|https://www.apache.org/dyn/closer.cgi/kylin/apache-kylin-2.6.1/apache-kylin-2.6.1-bin-hadoop3.tar.gz], NRT streaming cannot work in serval step for jar conflict problem. > NRT Streaming cannot work at HDP3.0 > --- > > Key: KYLIN-3970 > URL: https://issues.apache.org/jira/browse/KYLIN-3970 > Project: Kylin > Issue Type: Bug > Components: NRT Streaming >Affects Versions: v2.6.1 >Reporter: XiaoXiang Yu >Assignee: XiaoXiang Yu >Priority: Major > Labels: HDP, Hadoop3 > Fix For: v3.0.0, v2.6.2 > > Attachments: image-2019-04-23-14-25-51-066.png > > > We found that when using > [apache-kylin-2.6.1-bin-hadoop3.tar.gz|https://www.apache.org/dyn/closer.cgi/kylin/apache-kylin-2.6.1/apache-kylin-2.6.1-bin-hadoop3.tar.gz], > NRT streaming cannot work in serval step for jar conflict problem. > > !image-2019-04-23-14-25-51-066.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3970) NRT Streaming cannot work at HDP3.0
[ https://issues.apache.org/jira/browse/KYLIN-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaoXiang Yu updated KYLIN-3970: Attachment: image-2019-04-23-14-25-51-066.png > NRT Streaming cannot work at HDP3.0 > --- > > Key: KYLIN-3970 > URL: https://issues.apache.org/jira/browse/KYLIN-3970 > Project: Kylin > Issue Type: Bug > Components: NRT Streaming >Affects Versions: v2.6.1 >Reporter: XiaoXiang Yu >Assignee: XiaoXiang Yu >Priority: Major > Labels: HDP, Hadoop3 > Fix For: v3.0.0, v2.6.2 > > Attachments: image-2019-04-23-14-25-51-066.png > > > We found that when using > [apache-kylin-2.6.1-bin-hadoop3.tar.gz|https://www.apache.org/dyn/closer.cgi/kylin/apache-kylin-2.6.1/apache-kylin-2.6.1-bin-hadoop3.tar.gz], > NRT streaming cannot work in serval step for jar conflict problem. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3970) NRT Streaming cannot work at HDP3.0
[ https://issues.apache.org/jira/browse/KYLIN-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaoXiang Yu updated KYLIN-3970: Labels: HDP Hadoop3 (was: ) > NRT Streaming cannot work at HDP3.0 > --- > > Key: KYLIN-3970 > URL: https://issues.apache.org/jira/browse/KYLIN-3970 > Project: Kylin > Issue Type: Bug > Components: NRT Streaming >Affects Versions: v2.6.1 >Reporter: XiaoXiang Yu >Assignee: XiaoXiang Yu >Priority: Major > Labels: HDP, Hadoop3 > Fix For: v3.0.0, v2.6.2 > > > We found that when using > [apache-kylin-2.6.1-bin-hadoop3.tar.gz|https://www.apache.org/dyn/closer.cgi/kylin/apache-kylin-2.6.1/apache-kylin-2.6.1-bin-hadoop3.tar.gz], > NRT streaming cannot work in serval step for jar conflict problem. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3970) NRT Streaming cannot work at HDP3.0
XiaoXiang Yu created KYLIN-3970: --- Summary: NRT Streaming cannot work at HDP3.0 Key: KYLIN-3970 URL: https://issues.apache.org/jira/browse/KYLIN-3970 Project: Kylin Issue Type: Bug Components: NRT Streaming Affects Versions: v2.6.1 Reporter: XiaoXiang Yu Assignee: XiaoXiang Yu Fix For: v3.0.0, v2.6.2 We found that when using [apache-kylin-2.6.1-bin-hadoop3.tar.gz|https://www.apache.org/dyn/closer.cgi/kylin/apache-kylin-2.6.1/apache-kylin-2.6.1-bin-hadoop3.tar.gz], NRT streaming cannot work in serval step for jar conflict problem. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KYLIN-3608) Move dependency versions to top level pom properties
[ https://issues.apache.org/jira/browse/KYLIN-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-3608. - Resolution: Fixed > Move dependency versions to top level pom properties > > > Key: KYLIN-3608 > URL: https://issues.apache.org/jira/browse/KYLIN-3608 > Project: Kylin > Issue Type: Task > Components: Others >Affects Versions: v2.6.1 >Reporter: Ted Yu >Assignee: zhoujie >Priority: Minor > Fix For: v3.0.0 > > > There are some non-top level pom.xml files where dependency version is > referenced directly. > core-common/pom.xml is an example. > We should move all dependency versions to top level pom properties -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3608) Move dependency versions to top level pom properties
[ https://issues.apache.org/jira/browse/KYLIN-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3608: Affects Version/s: v2.6.1 Fix Version/s: v3.0.0 Component/s: Others > Move dependency versions to top level pom properties > > > Key: KYLIN-3608 > URL: https://issues.apache.org/jira/browse/KYLIN-3608 > Project: Kylin > Issue Type: Task > Components: Others >Affects Versions: v2.6.1 >Reporter: Ted Yu >Assignee: zhoujie >Priority: Minor > Fix For: v3.0.0 > > > There are some non-top level pom.xml files where dependency version is > referenced directly. > core-common/pom.xml is an example. > We should move all dependency versions to top level pom properties -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KYLIN-3841) Build Global Dict by MR/Hive
[ https://issues.apache.org/jira/browse/KYLIN-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-3841. - Resolution: Fixed > Build Global Dict by MR/Hive > > > Key: KYLIN-3841 > URL: https://issues.apache.org/jira/browse/KYLIN-3841 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v2.6.1 >Reporter: jinguowei >Assignee: jinguowei >Priority: Major > Fix For: v3.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3841) Build Global Dict by MR/Hive
[ https://issues.apache.org/jira/browse/KYLIN-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3841: Affects Version/s: v2.6.1 Fix Version/s: v3.0.0 Component/s: Job Engine > Build Global Dict by MR/Hive > > > Key: KYLIN-3841 > URL: https://issues.apache.org/jira/browse/KYLIN-3841 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v2.6.1 >Reporter: jinguowei >Assignee: jinguowei >Priority: Major > Fix For: v3.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
