[jira] [Created] (KYLIN-3941) Save query id, sql id and sql statement in hive and add to system cube.
zhao jintao created KYLIN-3941: -- Summary: Save query id, sql id and sql statement in hive and add to system cube. Key: KYLIN-3941 URL: https://issues.apache.org/jira/browse/KYLIN-3941 Project: Kylin Issue Type: Improvement Components: Metrics Affects Versions: v2.6.1 Reporter: zhao jintao Assignee: zhao jintao Hi team: I use kylin system cube to get sql query statistics. But when I use system cube, I find that sql statistics can not be accurate to a single sql, so I can't directly get slow sql for further analytics. I hope to get the sqlid and sql query statements from the system cube,so that I can analyze the slow query sql statement, guide creating kylin model and wiriting sql statement. In my kylin system, I save sql query id, sql id, sql length and sql statement in hive, and and sql id, sql length, sql statement in KYLIN-SYSTEM cube. Can I submit my improvements to the open source community. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] shaofengshi merged pull request #586: KYLIN-3608 Move dependency versions to top level pom properties
shaofengshi merged pull request #586: KYLIN-3608 Move dependency versions to top level pom properties URL: https://github.com/apache/kylin/pull/586 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3608) Move dependency versions to top level pom properties
[ https://issues.apache.org/jira/browse/KYLIN-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813163#comment-16813163 ] ASF subversion and git services commented on KYLIN-3608: Commit 578217e9e5d6c05863ed5370ae6aeaa063079ae5 in kylin's branch refs/heads/master from Weiqing Xu [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=578217e ] KYLIN-3608 Move dependency versions to top level pom properties > Move dependency versions to top level pom properties > > > Key: KYLIN-3608 > URL: https://issues.apache.org/jira/browse/KYLIN-3608 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Assignee: zhoujie >Priority: Minor > > There are some non-top level pom.xml files where dependency version is > referenced directly. > core-common/pom.xml is an example. > We should move all dependency versions to top level pom properties -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3608) Move dependency versions to top level pom properties
[ https://issues.apache.org/jira/browse/KYLIN-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813162#comment-16813162 ] ASF GitHub Bot commented on KYLIN-3608: --- shaofengshi commented on pull request #586: KYLIN-3608 Move dependency versions to top level pom properties URL: https://github.com/apache/kylin/pull/586 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Move dependency versions to top level pom properties > > > Key: KYLIN-3608 > URL: https://issues.apache.org/jira/browse/KYLIN-3608 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Assignee: zhoujie >Priority: Minor > > There are some non-top level pom.xml files where dependency version is > referenced directly. > core-common/pom.xml is an example. > We should move all dependency versions to top level pom properties -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] nichunen closed pull request #435: when dimvalue`s len > short.max_value then [Extract Fact Table Distinct Columns] err
nichunen closed pull request #435: when dimvalue`s len > short.max_value then [Extract Fact Table Distinct Columns] err URL: https://github.com/apache/kylin/pull/435 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] shaofengshi merged pull request #556: KYLIN-3912: Support cube level mapreduuce queue config for BeelineHiveClient
shaofengshi merged pull request #556: KYLIN-3912: Support cube level mapreduuce queue config for BeelineHiveClient URL: https://github.com/apache/kylin/pull/556 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen commented on issue #435: when dimvalue`s len > short.max_value then [Extract Fact Table Distinct Columns] err
nichunen commented on issue #435: when dimvalue`s len > short.max_value then [Extract Fact Table Distinct Columns] err URL: https://github.com/apache/kylin/pull/435#issuecomment-481174883 After discussed with @javalife0312 We think this is not a common case, and there is workaround way to deal with long dimensions. Close this pr This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3912) Support cube level mapreduce queue config for BeelineHiveClient
[
https://issues.apache.org/jira/browse/KYLIN-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813173#comment-16813173
]
ASF subversion and git services commented on KYLIN-3912:
Commit 49cb81505d71fa3eef12524d2760e877dd1fdbdd in kylin's branch
refs/heads/master from Liu Shaohui
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=49cb815 ]
KYLIN-3912: Support cube level mapreduuce queue config for BeelineHiveClient
> Support cube level mapreduce queue config for BeelineHiveClient
> ---
>
> Key: KYLIN-3912
> URL: https://issues.apache.org/jira/browse/KYLIN-3912
> Project: Kylin
> Issue Type: Improvement
>Reporter: Liu Shaohui
>Priority: Major
>
> To support multi tenants, we set different mapreduce queue config for
> different projects and cubes, but BeelineHiveClient don't use those configs.
> So the getHiveTableRows api always run on same queue in kylin_hive_conf or
> jdbc url, which cause computing resource competition.
>
> {code:java}
> 2018-11-28 15:37:27,261 ERROR [Scheduler 1950398337 Job
> 08b3ee43-c84d-4039-84c5-a36ecb2cff18-228] execution.AbstractExecutable:383 :
> job:08b3ee43-c84d-4039-84c5-a36ecb2cff18-01 execute finished with exception
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> downstreamError is null.
> Query log:
> http://zjy-hadoop-prc-ct14.bj:28911/log?qid=a05e1629-2072-46dd-9d71-b5722d04b2aa
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:277)
> at
> org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:376)
> at
> org.apache.kylin.source.hive.BeelineHiveClient.getHiveTableRows(BeelineHiveClient.java:108)
> at
> org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiveTableStep.computeRowCount(HiveMRInput.java:304)
> at
> org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiveTableStep.doWork(HiveMRInput.java:354)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:67)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:300)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3912) Support cube level mapreduce queue config for BeelineHiveClient
[
https://issues.apache.org/jira/browse/KYLIN-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813172#comment-16813172
]
ASF GitHub Bot commented on KYLIN-3912:
---
shaofengshi commented on pull request #556: KYLIN-3912: Support cube level
mapreduuce queue config for BeelineHiveClient
URL: https://github.com/apache/kylin/pull/556
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support cube level mapreduce queue config for BeelineHiveClient
> ---
>
> Key: KYLIN-3912
> URL: https://issues.apache.org/jira/browse/KYLIN-3912
> Project: Kylin
> Issue Type: Improvement
>Reporter: Liu Shaohui
>Priority: Major
>
> To support multi tenants, we set different mapreduce queue config for
> different projects and cubes, but BeelineHiveClient don't use those configs.
> So the getHiveTableRows api always run on same queue in kylin_hive_conf or
> jdbc url, which cause computing resource competition.
>
> {code:java}
> 2018-11-28 15:37:27,261 ERROR [Scheduler 1950398337 Job
> 08b3ee43-c84d-4039-84c5-a36ecb2cff18-228] execution.AbstractExecutable:383 :
> job:08b3ee43-c84d-4039-84c5-a36ecb2cff18-01 execute finished with exception
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> downstreamError is null.
> Query log:
> http://zjy-hadoop-prc-ct14.bj:28911/log?qid=a05e1629-2072-46dd-9d71-b5722d04b2aa
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:277)
> at
> org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:376)
> at
> org.apache.kylin.source.hive.BeelineHiveClient.getHiveTableRows(BeelineHiveClient.java:108)
> at
> org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiveTableStep.computeRowCount(HiveMRInput.java:304)
> at
> org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiveTableStep.doWork(HiveMRInput.java:354)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:67)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:300)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3940) Exception in thread "main" java.lang.IllegalArgumentException: Failed to find metadata store by url: kylin_metadata@hbase
[ https://issues.apache.org/jira/browse/KYLIN-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813180#comment-16813180 ] praveenece commented on KYLIN-3940: --- Hi Chao Long, hadoop-3.1.1 apache-hive-2.3.4-bin hbase-2.1.1 apache-kylin-2.5.2-bin-hadoop3 Now I tried with above environment, kylin is stated but i unable to load metadata from hbase to kylin getting this below error in kylin UI. org.apache.hadoop.hive.metastore.HiveMetaStoreClint.(Lorg/apache/hadoop/conf/configuration;)V 2019-04-09 15:02:44,979 DEBUG [http-nio-7070-exec-9] common.KylinConfig:327 : KYLIN_CONF property was not set, will seek KYLIN_HOME env variable 2019-04-09 15:02:50,071 ERROR [http-nio-7070-exec-6] controller.TableController:190 : org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(Lorg/apache/hadoop/conf/Configuration;)V java.lang.NoSuchMethodError: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(Lorg/apache/hadoop/conf/Configuration;)V at org.apache.kylin.source.hive.CLIHiveClient.getMetaStoreClient(CLIHiveClient.java:132) at org.apache.kylin.source.hive.CLIHiveClient.getHiveDbNames(CLIHiveClient.java:116) at org.apache.kylin.source.hive.HiveMetadataExplorer.listDatabases(HiveMetadataExplorer.java:47) at org.apache.kylin.rest.service.TableService.getSourceDbNames(TableService.java:277) at org.apache.kylin.rest.controller.TableController.showHiveDatabases(TableController.java:188) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) > Exception in thread "main" java.lang.IllegalArgumentException: Failed to find > metadata store by url: kylin_metadata@hbase > - > > Key: KYLIN-3940 > URL: https://issues.apache.org/jira/browse/KYLIN-3940 > Project: Kylin > Issue Type: Test > Components: Environment >Affects Versions: v2.5.2 > Environment: apache-hive-2.3.4-bin > hbase-1.4.9 > hadoop-3.1.0 > apache-kylin-2.5.2-bin-hadoop3 >Reporter: praveenece >Priority: Critical > > I am getting the below error. This error occurred when use > apache-kylin-2.5.2-bin-hadoop3. if we use apache-kylin-2.5.2-bin-hbase1x > kylin gets started. > apache-hive-2.3.4-bin > hbase-1.4.9 > hadoop-3.1.0 > apache-kylin-2.5.2-bin-hadoop3 > Exception in thread "main" java.lang.IllegalArgumentException: Failed to find > metadata store by url: kylin_metadata@hbase > at > org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:98) > at > org.apache.kylin.common.persistence.ResourceStore.getStore(ResourceStore.java:110) > at > org.apache.kylin.rest.service.AclTableMigrationTool.checkIfNeedMigrate(AclTableMigrationTool.java:98) > at > org.apache.kylin.tool.AclTableMigrationCLI.main(AclTableMigrationCLI.java:41) > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:92) > ... 3 more > Caused by: java.lang.NoClassDefFoundError: > org/apache/hadoop/hbase/client/ColumnFamilyDescriptor > at > org.apache.kylin.storage.hbase.HBaseResourceStore.getConnection(HBaseResourceStore.java:86) > at > org.apache.kylin.storage.hbase.HBaseResourceStore.createHTableIfNeeded(HBaseResourceStore.java:112) > at > org.apache.kylin.storage.hbase.HBaseResourceStore.(HBaseResourceStore.java:93) > ... 8 more > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.hbase.client.ColumnFamilyDescriptor > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) -- This message was sent by Atlassian JIRA
[GitHub] [kylin] nichunen merged pull request #445: Kylin-3777 Remove dict from query side
nichunen merged pull request #445: Kylin-3777 Remove dict from query side URL: https://github.com/apache/kylin/pull/445 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3777) Remove dict from query
[ https://issues.apache.org/jira/browse/KYLIN-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813190#comment-16813190 ] ASF subversion and git services commented on KYLIN-3777: Commit 768d7caec0b80d1bcd69caf6a9081adaf1b3d12f in kylin's branch refs/heads/kylin-on-parquet from chao long [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=768d7ca ] KYLIN-3777 Fix missing statistics resources > Remove dict from query > -- > > Key: KYLIN-3777 > URL: https://issues.apache.org/jira/browse/KYLIN-3777 > Project: Kylin > Issue Type: Sub-task >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > > To improve query performance, we should remove dict from query side. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3777) Remove dict from query
[ https://issues.apache.org/jira/browse/KYLIN-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813187#comment-16813187 ] ASF subversion and git services commented on KYLIN-3777: Commit 8a0d3f6cb692a388105c9658d5bebd38e003d5e4 in kylin's branch refs/heads/kylin-on-parquet from chao long [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=8a0d3f6 ] KYLIN-3777 Remove dim dict from parquet file > Remove dict from query > -- > > Key: KYLIN-3777 > URL: https://issues.apache.org/jira/browse/KYLIN-3777 > Project: Kylin > Issue Type: Sub-task >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > > To improve query performance, we should remove dict from query side. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3777) Remove dict from query
[ https://issues.apache.org/jira/browse/KYLIN-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813188#comment-16813188 ] ASF subversion and git services commented on KYLIN-3777: Commit 8f3195ec8b01c41728b123d8d2370cf631227b15 in kylin's branch refs/heads/kylin-on-parquet from chao long [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=8f3195e ] KYLIN-3777 Remove dict from query side > Remove dict from query > -- > > Key: KYLIN-3777 > URL: https://issues.apache.org/jira/browse/KYLIN-3777 > Project: Kylin > Issue Type: Sub-task >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > > To improve query performance, we should remove dict from query side. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3777) Remove dict from query
[ https://issues.apache.org/jira/browse/KYLIN-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813189#comment-16813189 ] ASF subversion and git services commented on KYLIN-3777: Commit 05e4bd27dc615d677fc99b00f1a65a39695eae5e in kylin's branch refs/heads/kylin-on-parquet from chao long [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=05e4bd2 ] KYLIN-3777 Pass CI > Remove dict from query > -- > > Key: KYLIN-3777 > URL: https://issues.apache.org/jira/browse/KYLIN-3777 > Project: Kylin > Issue Type: Sub-task >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > > To improve query performance, we should remove dict from query side. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] TempleZhou opened a new pull request #589: KYLIN-3892 Set cubing job priority
TempleZhou opened a new pull request #589: KYLIN-3892 Set cubing job priority URL: https://github.com/apache/kylin/pull/589 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #589: KYLIN-3892 Set cubing job priority
asfgit commented on issue #589: KYLIN-3892 Set cubing job priority URL: https://github.com/apache/kylin/pull/589#issuecomment-481181939 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3892) Set cubing job priority
[ https://issues.apache.org/jira/browse/KYLIN-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813192#comment-16813192 ] ASF GitHub Bot commented on KYLIN-3892: --- TempleZhou commented on pull request #589: KYLIN-3892 Set cubing job priority URL: https://github.com/apache/kylin/pull/589 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Set cubing job priority > --- > > Key: KYLIN-3892 > URL: https://issues.apache.org/jira/browse/KYLIN-3892 > Project: Kylin > Issue Type: New Feature > Components: Job Engine >Affects Versions: v2.4.0, v2.5.0, v2.6.0 >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Minor > Fix For: v2.6.2 > > > The cubing job with high priority will be delayed when there are too many > tasks running. > So I want to set the job priority for the important cubing jobs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3942) Rea-time OLAP don't support multi-level json event
XiaoXiang Yu created KYLIN-3942:
---
Summary: Rea-time OLAP don't support multi-level json event
Key: KYLIN-3942
URL: https://issues.apache.org/jira/browse/KYLIN-3942
Project: Kylin
Issue Type: Bug
Components: Real-time Streaming
Affects Versions: v3.0.0-alpha
Reporter: XiaoXiang Yu
Assignee: XiaoXiang Yu
Currently real-time OLAP didn't support multi-level json event.
For example,if I have a kafka multi-level json event like this:
{quote}{"country":"JAPAN","amount":13.075058425023922,"qty":8,"currency":"USD","order_time":1554801950882,"category":"ELECTRONIC","device":"Andriod","user":\{"gender":"Female","id":"7a0cfa5e-bbaa-79ef-1a38-e06f02c85fcb","first_name":"unknown","age":16}}
{quote}
Receiver will throw exception like this and discard that event:
{quote}2019-04-09 09:46:09,878 ERROR [StreamingV2Cube_channel]
kafka.TimedJsonStreamParser:107 : error
com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize
instance of `java.lang.String` out of START_OBJECT token
at [Source:
(String)"\{"country":"US","amount":14.498498222823619,"qty":1,"currency":"USD","order_time":1554803169876,"category":"Other","device":"Other","user":{"gender":"Female","id":"0736b41a-9ae7-9b4a-a124-f74436d3eb41","first_name":"unknown","age":26}}";
line: 1, column: 140] (through reference chain: java.util.HashMap["user"])
at
com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63)
at
com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1342)
at
com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1138)
at
com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1092)
at
com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:63)
at
com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:10)
at
com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:527)
at
com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
at
com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
at
com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
at
com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3030)
at
org.apache.kylin.stream.source.kafka.TimedJsonStreamParser.parse(TimedJsonStreamParser.java:79)
at
org.apache.kylin.stream.source.kafka.TimedJsonStreamParser.parse(TimedJsonStreamParser.java:54)
at
org.apache.kylin.stream.source.kafka.consumer.KafkaConnector.nextEvent(KafkaConnector.java:110)
at
org.apache.kylin.stream.core.consumer.StreamingConsumerChannel.run(StreamingConsumerChannel.java:93)
at java.lang.Thread.run(Thread.java:748)
{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3943) system-cube.sh has some problems
Chao Long created KYLIN-3943: Summary: system-cube.sh has some problems Key: KYLIN-3943 URL: https://issues.apache.org/jira/browse/KYLIN-3943 Project: Kylin Issue Type: Bug Reporter: Chao Long Assignee: Chao Long Fix For: v2.6.2 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3943) system-cube.sh has some problems
[ https://issues.apache.org/jira/browse/KYLIN-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Long updated KYLIN-3943: - Attachment: image-2019-04-09-18-50-35-111.png > system-cube.sh has some problems > > > Key: KYLIN-3943 > URL: https://issues.apache.org/jira/browse/KYLIN-3943 > Project: Kylin > Issue Type: Bug >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > Attachments: image-2019-04-09-18-50-35-111.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3943) system-cube.sh has some problems
[ https://issues.apache.org/jira/browse/KYLIN-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Long updated KYLIN-3943: - Description: 1, The name of system cube should not hard code. the suffix should get from config. !image-2019-04-09-18-50-35-111.png! 2, Use CubeBuildingCLI instead of rest api to build cube. > system-cube.sh has some problems > > > Key: KYLIN-3943 > URL: https://issues.apache.org/jira/browse/KYLIN-3943 > Project: Kylin > Issue Type: Bug >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > Attachments: image-2019-04-09-18-50-35-111.png > > > 1, The name of system cube should not hard code. the suffix should get from > config. > !image-2019-04-09-18-50-35-111.png! > 2, Use CubeBuildingCLI instead of rest api to build cube. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] nichunen commented on issue #534: KYLIN-3893: Cube build failed for wrong row key column description
nichunen commented on issue #534: KYLIN-3893: Cube build failed for wrong row key column description URL: https://github.com/apache/kylin/pull/534#issuecomment-481221559 Hi @lshmouse Seems your pr fails ut CubeControllerTest.testBasics This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] kyotoYaho commented on a change in pull request #533: kylin-3812 optimize the child CompareTupleFilter in a CompareTupleFilter
kyotoYaho commented on a change in pull request #533: kylin-3812 optimize the
child CompareTupleFilter in a CompareTupleFilter
URL: https://github.com/apache/kylin/pull/533#discussion_r273513936
##
File path:
core-metadata/src/main/java/org/apache/kylin/metadata/filter/CompareTupleFilter.java
##
@@ -283,6 +289,20 @@ public TupleFilter
acceptOptimizeTransformer(FilterOptimizeTransformer transform
return transformer.visit(this);
}
+private TupleFilter optimizeChildCompareTupleFilter(CompareTupleFilter
child) {
+FilterOptimizeTransformer transformer = new
FilterOptimizeTransformer();
+TupleFilter result = child.acceptOptimizeTransformer(transformer);
+if (result == ConstantTupleFilter.TRUE) {
+// use string instead of boolean since it's encoded as string
+result = new ConstantTupleFilter("true");
+} else if (result == ConstantTupleFilter.FALSE) {
+result = new ConstantTupleFilter("false");
+} else {
+this.isNormal = false;
Review comment:
Why do you think AlwaysTrue or AlwaysFalse cannot be pushed down? Current
CompareTupleFilter does not well support the case that both children are
CompareTupleFilter. Therefore, push down will be disabled. This behavior is not
changed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] kyotoYaho commented on a change in pull request #533: kylin-3812 optimize the child CompareTupleFilter in a CompareTupleFilter
kyotoYaho commented on a change in pull request #533: kylin-3812 optimize the
child CompareTupleFilter in a CompareTupleFilter
URL: https://github.com/apache/kylin/pull/533#discussion_r273514735
##
File path:
core-metadata/src/main/java/org/apache/kylin/metadata/filter/CompareTupleFilter.java
##
@@ -74,6 +77,9 @@ private CompareTupleFilter(CompareTupleFilter another) {
@Override
public void addChild(TupleFilter child) {
+if (child instanceof CompareTupleFilter) {
Review comment:
We need to refine our IT and add more test cases. To test this case, it's
necessary to add a column with boolean type.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] kyotoYaho commented on a change in pull request #533: kylin-3812 optimize the child CompareTupleFilter in a CompareTupleFilter
kyotoYaho commented on a change in pull request #533: kylin-3812 optimize the
child CompareTupleFilter in a CompareTupleFilter
URL: https://github.com/apache/kylin/pull/533#discussion_r273514735
##
File path:
core-metadata/src/main/java/org/apache/kylin/metadata/filter/CompareTupleFilter.java
##
@@ -74,6 +77,9 @@ private CompareTupleFilter(CompareTupleFilter another) {
@Override
public void addChild(TupleFilter child) {
+if (child instanceof CompareTupleFilter) {
Review comment:
We need to refine our IT and add more test cases. To test this case, it's
necessary to add a column with boolean type. However, it's a trivial job and
may need another JIRA to work this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] kyotoYaho commented on a change in pull request #533: kylin-3812 optimize the child CompareTupleFilter in a CompareTupleFilter
kyotoYaho commented on a change in pull request #533: kylin-3812 optimize the
child CompareTupleFilter in a CompareTupleFilter
URL: https://github.com/apache/kylin/pull/533#discussion_r273513936
##
File path:
core-metadata/src/main/java/org/apache/kylin/metadata/filter/CompareTupleFilter.java
##
@@ -283,6 +289,20 @@ public TupleFilter
acceptOptimizeTransformer(FilterOptimizeTransformer transform
return transformer.visit(this);
}
+private TupleFilter optimizeChildCompareTupleFilter(CompareTupleFilter
child) {
+FilterOptimizeTransformer transformer = new
FilterOptimizeTransformer();
+TupleFilter result = child.acceptOptimizeTransformer(transformer);
+if (result == ConstantTupleFilter.TRUE) {
+// use string instead of boolean since it's encoded as string
+result = new ConstantTupleFilter("true");
+} else if (result == ConstantTupleFilter.FALSE) {
+result = new ConstantTupleFilter("false");
+} else {
+this.isNormal = false;
Review comment:
Why do you think AlwaysTrue or AlwaysFalse cannot be pushed down? Current
CompareTupleFilter does not well support the case that both children are
CompareTupleFilter. Only in this case push down will be disabled. This behavior
is not changed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] kyotoYaho commented on a change in pull request #533: kylin-3812 optimize the child CompareTupleFilter in a CompareTupleFilter
kyotoYaho commented on a change in pull request #533: kylin-3812 optimize the
child CompareTupleFilter in a CompareTupleFilter
URL: https://github.com/apache/kylin/pull/533#discussion_r273513936
##
File path:
core-metadata/src/main/java/org/apache/kylin/metadata/filter/CompareTupleFilter.java
##
@@ -283,6 +289,20 @@ public TupleFilter
acceptOptimizeTransformer(FilterOptimizeTransformer transform
return transformer.visit(this);
}
+private TupleFilter optimizeChildCompareTupleFilter(CompareTupleFilter
child) {
+FilterOptimizeTransformer transformer = new
FilterOptimizeTransformer();
+TupleFilter result = child.acceptOptimizeTransformer(transformer);
+if (result == ConstantTupleFilter.TRUE) {
+// use string instead of boolean since it's encoded as string
+result = new ConstantTupleFilter("true");
+} else if (result == ConstantTupleFilter.FALSE) {
+result = new ConstantTupleFilter("false");
+} else {
+this.isNormal = false;
Review comment:
Why do you think AlwaysTrue or AlwaysFalse cannot be pushed down? Current
CompareTupleFilter does not well support the case that any of its children is a
CompareTupleFilter and is not alway true or always false. Only in this case
push down will be disabled. This behavior is not changed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (KYLIN-3812) optimize the child CompareTupleFilter in a CompareTupleFilter
[
https://issues.apache.org/jira/browse/KYLIN-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813482#comment-16813482
]
Zhong Yanghong commented on KYLIN-3812:
---
Hi [~Yifei_Wu94], at least for calcite 1.12.0, the 1=1 constant part is not
optimized.
> optimize the child CompareTupleFilter in a CompareTupleFilter
> -
>
> Key: KYLIN-3812
> URL: https://issues.apache.org/jira/browse/KYLIN-3812
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v3.0.0
>
>
> Currently it's not well supported for CompareTupleFilter to have a child of
> CompareTupleFilter. However, in some cases, it's better to support it.
> {code}
> where (colA = (1=1))
> {code}
> The *(1=1)* can be transformed to "true". And then this filter can be pushed
> down to hbase. Otherwise, the filter *(colA = (1=1))* does not work in hbase.
> And it may return incorrect results for the following SQL:
> {code}
> select colA
>case
>when colB = (1 = 1) then 'B'
>when colC = (1 = 1) then 'C'
>when colD = (1 = 1) then 'D'
>else 'n/a'
>end as phase,
>count(*)
> from T
> where session_date between '2018-08-01' and '2018-08-31'
> group by colA
>case
>when colB = (1 = 1) then 'B'
>when colC = (1 = 1) then 'C'
>when colD = (1 = 1) then 'D'
>else 'n/a'
>end;
> {code}
> In the final result, all of the keys will become 'B'.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3812) optimize the child CompareTupleFilter in a CompareTupleFilter
[
https://issues.apache.org/jira/browse/KYLIN-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813484#comment-16813484
]
Zhong Yanghong commented on KYLIN-3812:
---
It's better to refine Kylin's IT and to include a boolean type column. Then
this case can be covered. Anyone can help on this?
> optimize the child CompareTupleFilter in a CompareTupleFilter
> -
>
> Key: KYLIN-3812
> URL: https://issues.apache.org/jira/browse/KYLIN-3812
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v3.0.0
>
>
> Currently it's not well supported for CompareTupleFilter to have a child of
> CompareTupleFilter. However, in some cases, it's better to support it.
> {code}
> where (colA = (1=1))
> {code}
> The *(1=1)* can be transformed to "true". And then this filter can be pushed
> down to hbase. Otherwise, the filter *(colA = (1=1))* does not work in hbase.
> And it may return incorrect results for the following SQL:
> {code}
> select colA
>case
>when colB = (1 = 1) then 'B'
>when colC = (1 = 1) then 'C'
>when colD = (1 = 1) then 'D'
>else 'n/a'
>end as phase,
>count(*)
> from T
> where session_date between '2018-08-01' and '2018-08-31'
> group by colA
>case
>when colB = (1 = 1) then 'B'
>when colC = (1 = 1) then 'C'
>when colD = (1 = 1) then 'D'
>else 'n/a'
>end;
> {code}
> In the final result, all of the keys will become 'B'.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3943) system-cube.sh has some problems
[ https://issues.apache.org/jira/browse/KYLIN-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Long updated KYLIN-3943: - Attachment: screenshot-1.png > system-cube.sh has some problems > > > Key: KYLIN-3943 > URL: https://issues.apache.org/jira/browse/KYLIN-3943 > Project: Kylin > Issue Type: Bug >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > Attachments: image-2019-04-09-18-50-35-111.png, screenshot-1.png > > > 1, The name of system cube should not hard code. the suffix should get from > config. > !image-2019-04-09-18-50-35-111.png! > 2, Use CubeBuildingCLI instead of rest api to build cube. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3943) system-cube.sh has some problems
[ https://issues.apache.org/jira/browse/KYLIN-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Long updated KYLIN-3943: - Attachment: (was: screenshot-1.png) > system-cube.sh has some problems > > > Key: KYLIN-3943 > URL: https://issues.apache.org/jira/browse/KYLIN-3943 > Project: Kylin > Issue Type: Bug >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > Attachments: image-2019-04-09-18-50-35-111.png > > > 1, The name of system cube should not hard code. the suffix should get from > config. > !image-2019-04-09-18-50-35-111.png! > 2, Use CubeBuildingCLI instead of rest api to build cube. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3943) system-cube.sh has some problems
[ https://issues.apache.org/jira/browse/KYLIN-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Long updated KYLIN-3943: - Attachment: image-2019-04-09-23-05-51-460.png > system-cube.sh has some problems > > > Key: KYLIN-3943 > URL: https://issues.apache.org/jira/browse/KYLIN-3943 > Project: Kylin > Issue Type: Bug >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > Attachments: image-2019-04-09-18-50-35-111.png, > image-2019-04-09-23-05-51-460.png > > > 1, The name of system cube should not hard code. the suffix should get from > config. > !image-2019-04-09-18-50-35-111.png! > 2, Use CubeBuildingCLI instead of rest api to build cube. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3943) system-cube.sh has some problems
[ https://issues.apache.org/jira/browse/KYLIN-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Long updated KYLIN-3943: - Attachment: image-2019-04-09-23-06-09-663.png > system-cube.sh has some problems > > > Key: KYLIN-3943 > URL: https://issues.apache.org/jira/browse/KYLIN-3943 > Project: Kylin > Issue Type: Bug >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > Attachments: image-2019-04-09-18-50-35-111.png > > > 1, The name of system cube should not hard code. the suffix should get from > config. > !image-2019-04-09-18-50-35-111.png! > 2, Use CubeBuildingCLI instead of rest api to build cube. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3943) system-cube.sh has some problems
[ https://issues.apache.org/jira/browse/KYLIN-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Long updated KYLIN-3943: - Attachment: (was: image-2019-04-09-23-06-24-784.png) > system-cube.sh has some problems > > > Key: KYLIN-3943 > URL: https://issues.apache.org/jira/browse/KYLIN-3943 > Project: Kylin > Issue Type: Bug >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > Attachments: image-2019-04-09-18-50-35-111.png > > > 1, The name of system cube should not hard code. the suffix should get from > config. > !image-2019-04-09-18-50-35-111.png! > 2, Use CubeBuildingCLI instead of rest api to build cube. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3943) system-cube.sh has some problems
[ https://issues.apache.org/jira/browse/KYLIN-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Long updated KYLIN-3943: - Attachment: (was: image-2019-04-09-23-05-51-460.png) > system-cube.sh has some problems > > > Key: KYLIN-3943 > URL: https://issues.apache.org/jira/browse/KYLIN-3943 > Project: Kylin > Issue Type: Bug >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > Attachments: image-2019-04-09-18-50-35-111.png > > > 1, The name of system cube should not hard code. the suffix should get from > config. > !image-2019-04-09-18-50-35-111.png! > 2, Use CubeBuildingCLI instead of rest api to build cube. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3943) system-cube.sh has some problems
[ https://issues.apache.org/jira/browse/KYLIN-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Long updated KYLIN-3943: - Attachment: image-2019-04-09-23-06-24-784.png > system-cube.sh has some problems > > > Key: KYLIN-3943 > URL: https://issues.apache.org/jira/browse/KYLIN-3943 > Project: Kylin > Issue Type: Bug >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > Attachments: image-2019-04-09-18-50-35-111.png > > > 1, The name of system cube should not hard code. the suffix should get from > config. > !image-2019-04-09-18-50-35-111.png! > 2, Use CubeBuildingCLI instead of rest api to build cube. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3943) system-cube.sh has some problems
[ https://issues.apache.org/jira/browse/KYLIN-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Long updated KYLIN-3943: - Attachment: (was: image-2019-04-09-23-06-09-663.png) > system-cube.sh has some problems > > > Key: KYLIN-3943 > URL: https://issues.apache.org/jira/browse/KYLIN-3943 > Project: Kylin > Issue Type: Bug >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > Attachments: image-2019-04-09-18-50-35-111.png > > > 1, The name of system cube should not hard code. the suffix should get from > config. > !image-2019-04-09-18-50-35-111.png! > 2, Use CubeBuildingCLI instead of rest api to build cube. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3943) system-cube.sh has some problems
[ https://issues.apache.org/jira/browse/KYLIN-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Long updated KYLIN-3943: - Description: 1, The name of system cube should not hard code. the suffix should get from config. !image-2019-04-09-18-50-35-111.png! 2, server port should not hard code !image-2019-04-09-23-07-19-229.png! was: 1, The name of system cube should not hard code. the suffix should get from config. !image-2019-04-09-18-50-35-111.png! 2, Use CubeBuildingCLI instead of rest api to build cube. > system-cube.sh has some problems > > > Key: KYLIN-3943 > URL: https://issues.apache.org/jira/browse/KYLIN-3943 > Project: Kylin > Issue Type: Bug >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > Attachments: image-2019-04-09-18-50-35-111.png, > image-2019-04-09-23-07-19-229.png > > > 1, The name of system cube should not hard code. the suffix should get from > config. > !image-2019-04-09-18-50-35-111.png! > 2, server port should not hard code > !image-2019-04-09-23-07-19-229.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (KYLIN-3842) kylinProperties.js Unable to get the public configuration of the first line in the front end
[
https://issues.apache.org/jira/browse/KYLIN-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vsevolod Ostapenko reopened KYLIN-3842:
---
_config is one long string that is being searched using simple indexOf (instead
of regex).
The recent changes introduce regression where partial matches will be falsely
picked up.
For example, while searching for property XYZ the following case, incorrect
property assignment will be picked:
{quote}{{# XYZ=foo}}
abcXYZ=bar
XYZ=expected_value{quote}
A trivial fix for the issue with the very first property in the property file
that doesn't start with a comment is to prepend "\n" to _config upon
initialization, if the first character of _config is not "\n".
> kylinProperties.js Unable to get the public configuration of the first line
> in the front end
>
>
> Key: KYLIN-3842
> URL: https://issues.apache.org/jira/browse/KYLIN-3842
> Project: Kylin
> Issue Type: Bug
> Components: Web
>Affects Versions: v2.5.2
>Reporter: Yuzhang QIU
>Assignee: Yuzhang QIU
>Priority: Minor
> Fix For: v2.6.2
>
>
> Hi dear team:
> I'm developing OLAP Platform based on Kylin2.5.2. During my work, I found
> that kylinProperties.js:37(getProperty(name)) can't get the property of the
> first line in the '_config' which initialized through /admin/public_config.
> For example, the public config is
> 'kylin.restclient.connection.default-max-per-route=20\nkylin.restclient.connection.max-total=200\nkylin.engine.default=2\nkylin.storage.default=2\n
> kylin.web.hive-limit=20\nkylin.web.help.length=4\n'. I expected to get 20
> but got '' when I want to get config by key
> 'kylin.restclient.connection.default-max-per-route'. This problem caused by
> 'var keyIndex = _config.indexOf('\n' + name + '=');'(at
> kylinProperties.js:37) return -1 for those names before which don't have an
> \n(at the first line).
> Then, I debug the AdminService.java, KylinConfig.java and found that the
> KylinConfig.java:517(around this line, in method
> exportToString(Collection propertyKeys)) build the public config
> string with a char '\n' after each property, which cause the first property
> don't has '\n' before it.
> Those are what I found, which will cause problem for developers.
> How do you think?
> Best regard
> yuzhang
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3943) system-cube.sh has some problems
[ https://issues.apache.org/jira/browse/KYLIN-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Long updated KYLIN-3943: - Attachment: image-2019-04-09-23-07-19-229.png > system-cube.sh has some problems > > > Key: KYLIN-3943 > URL: https://issues.apache.org/jira/browse/KYLIN-3943 > Project: Kylin > Issue Type: Bug >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > Attachments: image-2019-04-09-18-50-35-111.png, > image-2019-04-09-23-07-19-229.png > > > 1, The name of system cube should not hard code. the suffix should get from > config. > !image-2019-04-09-18-50-35-111.png! > 2, server port should not hard code > !image-2019-04-09-23-07-19-229.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] nichunen commented on issue #575: KYLIN-3843, List Kylin instances with their server mode on web.
nichunen commented on issue #575: KYLIN-3843, List Kylin instances with their server mode on web. URL: https://github.com/apache/kylin/pull/575#issuecomment-481300306 @chenzhx Would you please review the front end code? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen commented on issue #560: KYLIN-3918: Add project name in cube and job pages
nichunen commented on issue #560: KYLIN-3918: Add project name in cube and job pages URL: https://github.com/apache/kylin/pull/560#issuecomment-481300429 @lshmouse Hi, why should project name should be shown on each job even if the job name can be seen on the project drop down list? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3842) kylinProperties.js Unable to get the public configuration of the first line in the front end
[
https://issues.apache.org/jira/browse/KYLIN-3842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813577#comment-16813577
]
Yuzhang QIU commented on KYLIN-3842:
yes you are right , check whether the charAt(index-1) is '\n' will be more
complete
> kylinProperties.js Unable to get the public configuration of the first line
> in the front end
>
>
> Key: KYLIN-3842
> URL: https://issues.apache.org/jira/browse/KYLIN-3842
> Project: Kylin
> Issue Type: Bug
> Components: Web
>Affects Versions: v2.5.2
>Reporter: Yuzhang QIU
>Assignee: Yuzhang QIU
>Priority: Minor
> Fix For: v2.6.2
>
>
> Hi dear team:
> I'm developing OLAP Platform based on Kylin2.5.2. During my work, I found
> that kylinProperties.js:37(getProperty(name)) can't get the property of the
> first line in the '_config' which initialized through /admin/public_config.
> For example, the public config is
> 'kylin.restclient.connection.default-max-per-route=20\nkylin.restclient.connection.max-total=200\nkylin.engine.default=2\nkylin.storage.default=2\n
> kylin.web.hive-limit=20\nkylin.web.help.length=4\n'. I expected to get 20
> but got '' when I want to get config by key
> 'kylin.restclient.connection.default-max-per-route'. This problem caused by
> 'var keyIndex = _config.indexOf('\n' + name + '=');'(at
> kylinProperties.js:37) return -1 for those names before which don't have an
> \n(at the first line).
> Then, I debug the AdminService.java, KylinConfig.java and found that the
> KylinConfig.java:517(around this line, in method
> exportToString(Collection propertyKeys)) build the public config
> string with a char '\n' after each property, which cause the first property
> don't has '\n' before it.
> Those are what I found, which will cause problem for developers.
> How do you think?
> Best regard
> yuzhang
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3944) Streaming Cube UI display empty cube planner info
XiaoXiang Yu created KYLIN-3944: --- Summary: Streaming Cube UI display empty cube planner info Key: KYLIN-3944 URL: https://issues.apache.org/jira/browse/KYLIN-3944 Project: Kylin Issue Type: Bug Components: Website Affects Versions: v2.6.1 Reporter: XiaoXiang Yu Attachments: image-2019-04-10-10-58-17-452.png, image-2019-04-10-10-59-07-532.png In cube design page, I click "Planner" tab, it will show empty table. !image-2019-04-10-10-58-17-452.png! !image-2019-04-10-10-59-07-532.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3944) Streaming Cube UI display empty cube planner info
[ https://issues.apache.org/jira/browse/KYLIN-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaoXiang Yu updated KYLIN-3944: Description: In cube design page, I click "Planner" tab, it will show "Cuboid Distribution" empty table. !image-2019-04-10-10-58-17-452.png! !image-2019-04-10-10-59-07-532.png! was: In cube design page, I click "Planner" tab, it will show empty table. !image-2019-04-10-10-58-17-452.png! !image-2019-04-10-10-59-07-532.png! > Streaming Cube UI display empty cube planner info > - > > Key: KYLIN-3944 > URL: https://issues.apache.org/jira/browse/KYLIN-3944 > Project: Kylin > Issue Type: Bug > Components: Website >Affects Versions: v2.6.1 >Reporter: XiaoXiang Yu >Priority: Major > Attachments: image-2019-04-10-10-58-17-452.png, > image-2019-04-10-10-59-07-532.png > > > > In cube design page, I click "Planner" tab, it will show "Cuboid > Distribution" empty table. > > > !image-2019-04-10-10-58-17-452.png! > > !image-2019-04-10-10-59-07-532.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] chenzhx commented on issue #575: KYLIN-3843, List Kylin instances with their server mode on web.
chenzhx commented on issue #575: KYLIN-3843, List Kylin instances with their server mode on web. URL: https://github.com/apache/kylin/pull/575#issuecomment-481515846 It is fine to me This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (KYLIN-3945) Debug in local get error "Invalid 'log4jConfigLocation' parameter"
Chao Long created KYLIN-3945: Summary: Debug in local get error "Invalid 'log4jConfigLocation' parameter" Key: KYLIN-3945 URL: https://issues.apache.org/jira/browse/KYLIN-3945 Project: Kylin Issue Type: Bug Affects Versions: v2.6.2 Reporter: Chao Long Assignee: Chao Long 严重: Exception sending context initialized event to listener instance of class org.apache.kylin.rest.util.Log4jConfigListener java.lang.IllegalArgumentException: Invalid 'log4jConfigLocation' parameter: class path resource [kylin-server-log4j.properties] cannot be resolved to URL because it does not exist at org.springframework.web.util.Log4jWebConfigurer.initLogging(Log4jWebConfigurer.java:158) at org.springframework.web.util.Log4jConfigListener.contextInitialized(Log4jConfigListener.java:49) at org.apache.kylin.rest.util.Log4jConfigListener.contextInitialized(Log4jConfigListener.java:36) at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5157) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5680) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1707) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1697) at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266) at java.util.concurrent.FutureTask.run(FutureTask.java) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3944) Streaming Cube UI display empty cube planner info
[ https://issues.apache.org/jira/browse/KYLIN-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaoXiang Yu updated KYLIN-3944: Description: In cube design page, I click "Planner" tab, it will show "Cuboid Distribution" empty table. And these empty info won't disappear if I change to another streaming tab. !image-2019-04-10-10-58-17-452.png! !image-2019-04-10-10-59-07-532.png! was: In cube design page, I click "Planner" tab, it will show "Cuboid Distribution" empty table. !image-2019-04-10-10-58-17-452.png! !image-2019-04-10-10-59-07-532.png! > Streaming Cube UI display empty cube planner info > - > > Key: KYLIN-3944 > URL: https://issues.apache.org/jira/browse/KYLIN-3944 > Project: Kylin > Issue Type: Bug > Components: Website >Affects Versions: v2.6.1 >Reporter: XiaoXiang Yu >Priority: Major > Attachments: image-2019-04-10-10-58-17-452.png, > image-2019-04-10-10-59-07-532.png > > > > In cube design page, I click "Planner" tab, it will show "Cuboid > Distribution" empty table. > And these empty info won't disappear if I change to another streaming tab. > > > !image-2019-04-10-10-58-17-452.png! > > !image-2019-04-10-10-59-07-532.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3945) Debug in local get error "Invalid 'log4jConfigLocation' parameter"
[
https://issues.apache.org/jira/browse/KYLIN-3945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Long updated KYLIN-3945:
-
Description:
{code}
严重: Exception sending context initialized event to listener instance of class
org.apache.kylin.rest.util.Log4jConfigListener
java.lang.IllegalArgumentException: Invalid 'log4jConfigLocation' parameter:
class path resource [kylin-server-log4j.properties] cannot be resolved to URL
because it does not exist
at
org.springframework.web.util.Log4jWebConfigurer.initLogging(Log4jWebConfigurer.java:158)
at
org.springframework.web.util.Log4jConfigListener.contextInitialized(Log4jConfigListener.java:49)
at
org.apache.kylin.rest.util.Log4jConfigListener.contextInitialized(Log4jConfigListener.java:36)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5157)
at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5680)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
at
org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1707)
at
org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1697)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
was:
严重: Exception sending context initialized event to listener instance of class
org.apache.kylin.rest.util.Log4jConfigListener
java.lang.IllegalArgumentException: Invalid 'log4jConfigLocation' parameter:
class path resource [kylin-server-log4j.properties] cannot be resolved to URL
because it does not exist
at
org.springframework.web.util.Log4jWebConfigurer.initLogging(Log4jWebConfigurer.java:158)
at
org.springframework.web.util.Log4jConfigListener.contextInitialized(Log4jConfigListener.java:49)
at
org.apache.kylin.rest.util.Log4jConfigListener.contextInitialized(Log4jConfigListener.java:36)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5157)
at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5680)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
at
org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1707)
at
org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1697)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
> Debug in local get error "Invalid 'log4jConfigLocation' parameter"
> --
>
> Key: KYLIN-3945
> URL: https://issues.apache.org/jira/browse/KYLIN-3945
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.6.2
>Reporter: Chao Long
>Assignee: Chao Long
>Priority: Major
>
> {code}
> 严重: Exception sending context initialized event to listener instance of class
> org.apache.kylin.rest.util.Log4jConfigListener
> java.lang.IllegalArgumentException: Invalid 'log4jConfigLocation' parameter:
> class path resource [kylin-server-log4j.properties] cannot be resolved to URL
> because it does not exist
> at
> org.springframework.web.util.Log4jWebConfigurer.initLogging(Log4jWebConfigurer.java:158)
> at
> org.springframework.web.util.Log4jConfigListener.contextInitialized(Log4jConfigListener.java:49)
> at
> org.apache.kylin.rest.util.Log4jConfigListener.contextInitialized(Log4jConfigListener.java:36)
> at
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5157)
> at
> org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5680)
> at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
> at
> org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1707)
> at
> org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1697)
> at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
> at java.util.concurrent.FutureTask.run(FutureTask.java)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was s
[jira] [Commented] (KYLIN-3892) Set cubing job priority
[
https://issues.apache.org/jira/browse/KYLIN-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16814011#comment-16814011
]
Temple Zhou commented on KYLIN-3892:
You can enable this feature by setting the
kylin.job.scheduler.priority-considered=true.
When there are too many jobs running, you can submit the job with
priorityOffset property.
For example:
The number of running jobs beyonds the kylin.job.max-concurrent-jobs(the new
cubing job may be pending), meanwhile you have a cubing job with critical
importance, which is really a dilemma if we can't set the job priority.
But now, all the job will be submitted with priorityOffset 0 (default value)
and you can set the priorityOffset with positive value if the job priority is
higher than normal, on the contrary, set it with negative value.
{code:bash}
curl -X PUT \
http://localhost:7070/kylin/api/cubes/xxx/rebuild \
-H 'Authorization: Basic ***' \
-H 'Content-Type: application/json' \
-H 'cache-control: no-cache' \
-d '{
"buildType": "BUILD",
"startTime": 155269440,
"endTime": 155295360,
"priorityOffset": 10 # The higher the value, the higher the priority
}'
{code}
> Set cubing job priority
> ---
>
> Key: KYLIN-3892
> URL: https://issues.apache.org/jira/browse/KYLIN-3892
> Project: Kylin
> Issue Type: New Feature
> Components: Job Engine
>Affects Versions: v2.4.0, v2.5.0, v2.6.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Minor
> Fix For: v2.6.2
>
>
> The cubing job with high priority will be delayed when there are too many
> tasks running.
> So I want to set the job priority for the important cubing jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (KYLIN-3945) Debug in local get error "Invalid 'log4jConfigLocation' parameter"
[
https://issues.apache.org/jira/browse/KYLIN-3945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Long closed KYLIN-3945.
Resolution: Won't Fix
Fix Version/s: v2.6.1
root cause:
When debug in local, the env property(kylin.env) should be "DEV".
> Debug in local get error "Invalid 'log4jConfigLocation' parameter"
> --
>
> Key: KYLIN-3945
> URL: https://issues.apache.org/jira/browse/KYLIN-3945
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.6.2
>Reporter: Chao Long
>Assignee: Chao Long
>Priority: Major
> Fix For: v2.6.1
>
>
> {code}
> 严重: Exception sending context initialized event to listener instance of class
> org.apache.kylin.rest.util.Log4jConfigListener
> java.lang.IllegalArgumentException: Invalid 'log4jConfigLocation' parameter:
> class path resource [kylin-server-log4j.properties] cannot be resolved to URL
> because it does not exist
> at
> org.springframework.web.util.Log4jWebConfigurer.initLogging(Log4jWebConfigurer.java:158)
> at
> org.springframework.web.util.Log4jConfigListener.contextInitialized(Log4jConfigListener.java:49)
> at
> org.apache.kylin.rest.util.Log4jConfigListener.contextInitialized(Log4jConfigListener.java:36)
> at
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5157)
> at
> org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5680)
> at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
> at
> org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1707)
> at
> org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1697)
> at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
> at java.util.concurrent.FutureTask.run(FutureTask.java)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [kylin] nichunen commented on a change in pull request #582: Read password from file for Beeline
nichunen commented on a change in pull request #582: Read password from file for Beeline URL: https://github.com/apache/kylin/pull/582#discussion_r273795697 ## File path: source-hive/src/main/java/org/apache/kylin/source/hive/BeelineHiveClient.java ## @@ -27,6 +27,7 @@ import java.sql.Statement; import java.util.List; import java.util.Properties; +import java.io.*; Review comment: You can set up your kylin development environment by following http://kylin.apache.org/development/dev_env.html This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (KYLIN-3946) No cube for AVG measure after include count column
Chao Long created KYLIN-3946: Summary: No cube for AVG measure after include count column Key: KYLIN-3946 URL: https://issues.apache.org/jira/browse/KYLIN-3946 Project: Kylin Issue Type: Bug Components: Query Engine Reporter: Chao Long Assignee: Chao Long Fix For: v2.6.2 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [kylin] asfgit commented on issue #590: KYLIN-3912 fix CI
asfgit commented on issue #590: KYLIN-3912 fix CI URL: https://github.com/apache/kylin/pull/590#issuecomment-481546121 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #590: KYLIN-3912 fix CI
asfgit commented on issue #590: KYLIN-3912 fix CI URL: https://github.com/apache/kylin/pull/590#issuecomment-481546127 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] GinaZhai opened a new pull request #590: KYLIN-3912 fix CI
GinaZhai opened a new pull request #590: KYLIN-3912 fix CI URL: https://github.com/apache/kylin/pull/590 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #590: KYLIN-3912 fix CI
asfgit commented on issue #590: KYLIN-3912 fix CI URL: https://github.com/apache/kylin/pull/590#issuecomment-481546125 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3912) Support cube level mapreduce queue config for BeelineHiveClient
[
https://issues.apache.org/jira/browse/KYLIN-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16814079#comment-16814079
]
ASF GitHub Bot commented on KYLIN-3912:
---
GinaZhai commented on pull request #590: KYLIN-3912 fix CI
URL: https://github.com/apache/kylin/pull/590
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support cube level mapreduce queue config for BeelineHiveClient
> ---
>
> Key: KYLIN-3912
> URL: https://issues.apache.org/jira/browse/KYLIN-3912
> Project: Kylin
> Issue Type: Improvement
>Reporter: Liu Shaohui
>Priority: Major
>
> To support multi tenants, we set different mapreduce queue config for
> different projects and cubes, but BeelineHiveClient don't use those configs.
> So the getHiveTableRows api always run on same queue in kylin_hive_conf or
> jdbc url, which cause computing resource competition.
>
> {code:java}
> 2018-11-28 15:37:27,261 ERROR [Scheduler 1950398337 Job
> 08b3ee43-c84d-4039-84c5-a36ecb2cff18-228] execution.AbstractExecutable:383 :
> job:08b3ee43-c84d-4039-84c5-a36ecb2cff18-01 execute finished with exception
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> downstreamError is null.
> Query log:
> http://zjy-hadoop-prc-ct14.bj:28911/log?qid=a05e1629-2072-46dd-9d71-b5722d04b2aa
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:277)
> at
> org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:376)
> at
> org.apache.kylin.source.hive.BeelineHiveClient.getHiveTableRows(BeelineHiveClient.java:108)
> at
> org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiveTableStep.computeRowCount(HiveMRInput.java:304)
> at
> org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiveTableStep.doWork(HiveMRInput.java:354)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:67)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:300)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3946) No cube for AVG measure after include count column
[ https://issues.apache.org/jira/browse/KYLIN-3946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Long updated KYLIN-3946: - Description: Before, avg(col) will use sum(col)/count(1) to calculate. After including count column aggregation KYLIN-3883, avg(col) will use sum(col)/count(col) to calculate. If there is no predefined count(col) measure, query with avg(col) will get exception "NoRealizationFoundException: No realization found for OLAPContext", which will effect the query on old cubes. So we should consider compatibility. > No cube for AVG measure after include count column > -- > > Key: KYLIN-3946 > URL: https://issues.apache.org/jira/browse/KYLIN-3946 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v2.6.2 > > > Before, avg(col) will use sum(col)/count(1) to calculate. > After including count column aggregation KYLIN-3883, avg(col) will use > sum(col)/count(col) to calculate. > If there is no predefined count(col) measure, query with avg(col) will get > exception "NoRealizationFoundException: No realization found for > OLAPContext", which will effect the query on old cubes. So we should consider > compatibility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3812) optimize the child CompareTupleFilter in a CompareTupleFilter
[
https://issues.apache.org/jira/browse/KYLIN-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16814090#comment-16814090
]
Yifei Wu commented on KYLIN-3812:
-
Hi, [~yaho] I found this will optimize in calcite-1.16 which is depended by
the latest Kylin release. So upgrading the Calcite may be a better choice for u.
> optimize the child CompareTupleFilter in a CompareTupleFilter
> -
>
> Key: KYLIN-3812
> URL: https://issues.apache.org/jira/browse/KYLIN-3812
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v3.0.0
>
>
> Currently it's not well supported for CompareTupleFilter to have a child of
> CompareTupleFilter. However, in some cases, it's better to support it.
> {code}
> where (colA = (1=1))
> {code}
> The *(1=1)* can be transformed to "true". And then this filter can be pushed
> down to hbase. Otherwise, the filter *(colA = (1=1))* does not work in hbase.
> And it may return incorrect results for the following SQL:
> {code}
> select colA
>case
>when colB = (1 = 1) then 'B'
>when colC = (1 = 1) then 'C'
>when colD = (1 = 1) then 'D'
>else 'n/a'
>end as phase,
>count(*)
> from T
> where session_date between '2018-08-01' and '2018-08-31'
> group by colA
>case
>when colB = (1 = 1) then 'B'
>when colC = (1 = 1) then 'C'
>when colD = (1 = 1) then 'D'
>else 'n/a'
>end;
> {code}
> In the final result, all of the keys will become 'B'.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [kylin] codecov-io edited a comment on issue #575: KYLIN-3843, List Kylin instances with their server mode on web.
codecov-io edited a comment on issue #575: KYLIN-3843, List Kylin instances with their server mode on web. URL: https://github.com/apache/kylin/pull/575#issuecomment-478833253 # [Codecov](https://codecov.io/gh/apache/kylin/pull/575?src=pr&el=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@49cb815`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `15.27%`. [](https://codecov.io/gh/apache/kylin/pull/575?src=pr&el=tree) ```diff @@Coverage Diff@@ ## master #575 +/- ## = Coverage ? 25.36% Complexity? 5803 = Files ? 1374 Lines ?81205 Branches ?11384 = Hits ?20599 Misses?58592 Partials ? 2014 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/575?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...lin/rest/service/ServiceDiscoveryStateService.java](https://codecov.io/gh/apache/kylin/pull/575/diff?src=pr&el=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3Qvc2VydmljZS9TZXJ2aWNlRGlzY292ZXJ5U3RhdGVTZXJ2aWNlLmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | | | [.../java/org/apache/kylin/common/KylinConfigBase.java](https://codecov.io/gh/apache/kylin/pull/575/diff?src=pr&el=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9LeWxpbkNvbmZpZ0Jhc2UuamF2YQ==) | `13.15% <0%> (ø)` | `41 <0> (?)` | | | [...org/apache/kylin/common/restclient/RestClient.java](https://codecov.io/gh/apache/kylin/pull/575/diff?src=pr&el=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9yZXN0Y2xpZW50L1Jlc3RDbGllbnQuamF2YQ==) | `13.52% <0%> (ø)` | `3 <0> (?)` | | | [...st/controller/ServiceDiscoveryStateController.java](https://codecov.io/gh/apache/kylin/pull/575/diff?src=pr&el=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvY29udHJvbGxlci9TZXJ2aWNlRGlzY292ZXJ5U3RhdGVDb250cm9sbGVyLmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | | | [.../kylin/job/impl/curator/CuratorLeaderSelector.java](https://codecov.io/gh/apache/kylin/pull/575/diff?src=pr&el=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL2N1cmF0b3IvQ3VyYXRvckxlYWRlclNlbGVjdG9yLmphdmE=) | `67.44% <100%> (ø)` | `8 <1> (?)` | | | [...pache/kylin/job/impl/curator/CuratorScheduler.java](https://codecov.io/gh/apache/kylin/pull/575/diff?src=pr&el=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL2N1cmF0b3IvQ3VyYXRvclNjaGVkdWxlci5qYXZh) | `64.1% <100%> (ø)` | `7 <0> (?)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/575?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/575?src=pr&el=footer). Last update [49cb815...443d563](https://codecov.io/gh/apache/kylin/pull/575?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen commented on issue #590: KYLIN-3912 fix CI
nichunen commented on issue #590: KYLIN-3912 fix CI URL: https://github.com/apache/kylin/pull/590#issuecomment-481550075 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (KYLIN-3812) optimize the child CompareTupleFilter in a CompareTupleFilter
[
https://issues.apache.org/jira/browse/KYLIN-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16814090#comment-16814090
]
Yifei Wu edited comment on KYLIN-3812 at 4/10/19 6:25 AM:
--
Hi, [~yaho] I found this will be optimized to a constant true in calcite-1.16
which is depended by the latest Kylin release. So upgrading the Calcite may be
a better choice for u.
was (Author: yifei_wu94):
Hi, [~yaho] I found this will optimize in calcite-1.16 which is depended by
the latest Kylin release. So upgrading the Calcite may be a better choice for u.
> optimize the child CompareTupleFilter in a CompareTupleFilter
> -
>
> Key: KYLIN-3812
> URL: https://issues.apache.org/jira/browse/KYLIN-3812
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v3.0.0
>
>
> Currently it's not well supported for CompareTupleFilter to have a child of
> CompareTupleFilter. However, in some cases, it's better to support it.
> {code}
> where (colA = (1=1))
> {code}
> The *(1=1)* can be transformed to "true". And then this filter can be pushed
> down to hbase. Otherwise, the filter *(colA = (1=1))* does not work in hbase.
> And it may return incorrect results for the following SQL:
> {code}
> select colA
>case
>when colB = (1 = 1) then 'B'
>when colC = (1 = 1) then 'C'
>when colD = (1 = 1) then 'D'
>else 'n/a'
>end as phase,
>count(*)
> from T
> where session_date between '2018-08-01' and '2018-08-31'
> group by colA
>case
>when colB = (1 = 1) then 'B'
>when colC = (1 = 1) then 'C'
>when colD = (1 = 1) then 'D'
>else 'n/a'
>end;
> {code}
> In the final result, all of the keys will become 'B'.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [kylin] codecov-io commented on issue #590: KYLIN-3912 fix CI
codecov-io commented on issue #590: KYLIN-3912 fix CI URL: https://github.com/apache/kylin/pull/590#issuecomment-481556488 # [Codecov](https://codecov.io/gh/apache/kylin/pull/590?src=pr&el=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@49cb815`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/kylin/pull/590?src=pr&el=tree) ```diff @@Coverage Diff@@ ## master #590 +/- ## = Coverage ? 25.27% Complexity? 5805 = Files ? 1374 Lines ?81503 Branches ?11430 = Hits ?20603 Misses?58887 Partials ? 2013 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/590?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...apache/kylin/source/hive/CreateMrHiveDictStep.java](https://codecov.io/gh/apache/kylin/pull/590/diff?src=pr&el=tree#diff-c291cmNlLWhpdmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9oaXZlL0NyZWF0ZU1ySGl2ZURpY3RTdGVwLmphdmE=) | `0% <ø> (ø)` | `0 <0> (?)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/590?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/590?src=pr&el=footer). Last update [49cb815...5e5f2ee](https://codecov.io/gh/apache/kylin/pull/590?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3912) Support cube level mapreduce queue config for BeelineHiveClient
[
https://issues.apache.org/jira/browse/KYLIN-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16814122#comment-16814122
]
ASF subversion and git services commented on KYLIN-3912:
Commit 053b4f61b976ee05af98f41b42514017be373871 in kylin's branch
refs/heads/master from GinaZhai
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=053b4f6 ]
KYLIN-3912 fix CI
> Support cube level mapreduce queue config for BeelineHiveClient
> ---
>
> Key: KYLIN-3912
> URL: https://issues.apache.org/jira/browse/KYLIN-3912
> Project: Kylin
> Issue Type: Improvement
>Reporter: Liu Shaohui
>Priority: Major
>
> To support multi tenants, we set different mapreduce queue config for
> different projects and cubes, but BeelineHiveClient don't use those configs.
> So the getHiveTableRows api always run on same queue in kylin_hive_conf or
> jdbc url, which cause computing resource competition.
>
> {code:java}
> 2018-11-28 15:37:27,261 ERROR [Scheduler 1950398337 Job
> 08b3ee43-c84d-4039-84c5-a36ecb2cff18-228] execution.AbstractExecutable:383 :
> job:08b3ee43-c84d-4039-84c5-a36ecb2cff18-01 execute finished with exception
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> downstreamError is null.
> Query log:
> http://zjy-hadoop-prc-ct14.bj:28911/log?qid=a05e1629-2072-46dd-9d71-b5722d04b2aa
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:277)
> at
> org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:376)
> at
> org.apache.kylin.source.hive.BeelineHiveClient.getHiveTableRows(BeelineHiveClient.java:108)
> at
> org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiveTableStep.computeRowCount(HiveMRInput.java:304)
> at
> org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiveTableStep.doWork(HiveMRInput.java:354)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:67)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:300)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [kylin] nichunen merged pull request #590: KYLIN-3912 fix CI
nichunen merged pull request #590: KYLIN-3912 fix CI URL: https://github.com/apache/kylin/pull/590 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-3912) Support cube level mapreduce queue config for BeelineHiveClient
[
https://issues.apache.org/jira/browse/KYLIN-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16814121#comment-16814121
]
ASF GitHub Bot commented on KYLIN-3912:
---
nichunen commented on pull request #590: KYLIN-3912 fix CI
URL: https://github.com/apache/kylin/pull/590
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Support cube level mapreduce queue config for BeelineHiveClient
> ---
>
> Key: KYLIN-3912
> URL: https://issues.apache.org/jira/browse/KYLIN-3912
> Project: Kylin
> Issue Type: Improvement
>Reporter: Liu Shaohui
>Priority: Major
>
> To support multi tenants, we set different mapreduce queue config for
> different projects and cubes, but BeelineHiveClient don't use those configs.
> So the getHiveTableRows api always run on same queue in kylin_hive_conf or
> jdbc url, which cause computing resource competition.
>
> {code:java}
> 2018-11-28 15:37:27,261 ERROR [Scheduler 1950398337 Job
> 08b3ee43-c84d-4039-84c5-a36ecb2cff18-228] execution.AbstractExecutable:383 :
> job:08b3ee43-c84d-4039-84c5-a36ecb2cff18-01 execute finished with exception
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> downstreamError is null.
> Query log:
> http://zjy-hadoop-prc-ct14.bj:28911/log?qid=a05e1629-2072-46dd-9d71-b5722d04b2aa
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:277)
> at
> org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:376)
> at
> org.apache.kylin.source.hive.BeelineHiveClient.getHiveTableRows(BeelineHiveClient.java:108)
> at
> org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiveTableStep.computeRowCount(HiveMRInput.java:304)
> at
> org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiveTableStep.doWork(HiveMRInput.java:354)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:67)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:300)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
