[jira] [Commented] (KYLIN-5175) kylin4.0如何用PGsql存储元数据
[ https://issues.apache.org/jira/browse/KYLIN-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17519427#comment-17519427 ] Yaqian Zhang commented on KYLIN-5175: - [~bird] Kylin4 does not support PgSQL metadata storage, you can use mysql. > kylin4.0如何用PGsql存储元数据 > - > > Key: KYLIN-5175 > URL: https://issues.apache.org/jira/browse/KYLIN-5175 > Project: Kylin > Issue Type: Improvement > Components: Metadata >Affects Versions: v4.0.0 >Reporter: new bird >Priority: Major > > 配置PGsql存储元数据，报出语法sql语法不一致的问题 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5170) load Data Source 时提示 cannot recognize hive client mode
[ https://issues.apache.org/jira/browse/KYLIN-5170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513832#comment-17513832 ] Yaqian Zhang commented on KYLIN-5170: - Hi [~tokihou]： Kylin 4.01 change the method of load data source, use spark_catalog replace the hive cli. You can refer to this wiki: https://cwiki.apache.org/confluence/display/KYLIN/KIP+10+refactor+hive+and+hadoop+dependency. You can delete the config of "kylin.source.hive.client=cli" from kylin.properties, kylin will use 'spark_catalog' as the default value. > load Data Source 时提示 cannot recognize hive client mode > -- > > Key: KYLIN-5170 > URL: https://issues.apache.org/jira/browse/KYLIN-5170 > Project: Kylin > Issue Type: Bug >Affects Versions: v4.0.1 >Reporter: du qifang >Priority: Major > Attachments: error.png, kylin_properties.png > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5167) when use kylin4.* on spark3.*,can not set spark.sql.adaptive.enabled=true by itself
[ https://issues.apache.org/jira/browse/KYLIN-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17503314#comment-17503314 ] Yaqian Zhang commented on KYLIN-5167: - [~phol] I can't understand the mean of this issue. You can set 'kylin.engine.spark-conf.spark.sql.adaptive.enabled=true' in kylin.properties to config this configuration. > when use kylin4.* on spark3.*,can not set spark.sql.adaptive.enabled=true by > itself > --- > > Key: KYLIN-5167 > URL: https://issues.apache.org/jira/browse/KYLIN-5167 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v4.0.1 >Reporter: Pinghe.zheng >Priority: Critical > Fix For: Future > > > as title -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (KYLIN-5166) Use SQL hint to specify the cube that the user wants to query
[ https://issues.apache.org/jira/browse/KYLIN-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang resolved KYLIN-5166. - Resolution: Fixed > Use SQL hint to specify the cube that the user wants to query > - > > Key: KYLIN-5166 > URL: https://issues.apache.org/jira/browse/KYLIN-5166 > Project: Kylin > Issue Type: Improvement >Affects Versions: v4.0.0-alpha, v4.0.0-beta, v4.0.0, v4.0.1 >Reporter: Yaqian Zhang >Assignee: Yaqian Zhang >Priority: Major > Fix For: v4.0.2 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (KYLIN-5166) Use SQL hint to specify the cube that the user wants to query
[ https://issues.apache.org/jira/browse/KYLIN-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-5166: Fix Version/s: v4.0.2 Affects Version/s: v4.0.1 v4.0.0 v4.0.0-beta v4.0.0-alpha > Use SQL hint to specify the cube that the user wants to query > - > > Key: KYLIN-5166 > URL: https://issues.apache.org/jira/browse/KYLIN-5166 > Project: Kylin > Issue Type: Improvement >Affects Versions: v4.0.0-alpha, v4.0.0-beta, v4.0.0, v4.0.1 >Reporter: Yaqian Zhang >Assignee: Yaqian Zhang >Priority: Major > Fix For: v4.0.2 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (KYLIN-5165) Kylin web UI http 404 error
[ https://issues.apache.org/jira/browse/KYLIN-5165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17502681#comment-17502681 ] Yaqian Zhang edited comment on KYLIN-5165 at 3/8/22, 3:00 AM: -- Hi [~p0124]: I want to know what's your JDK version? was (Author: zhangyaqian): Hi [~p0124]: I want't to know what's your JDK version? > Kylin web UI http 404 error > --- > > Key: KYLIN-5165 > URL: https://issues.apache.org/jira/browse/KYLIN-5165 > Project: Kylin > Issue Type: Bug > Components: Web , Website >Affects Versions: v3.0.2 > Environment: kylin 3.0.2 > hadoop 3.1.3 > hbase 2.0.5 > hive 3.1.2 > zookeeper 3.5.7 >Reporter: Junyi Pang >Priority: Major > Fix For: v3.0.2 > > Attachments: kylin.log > > > 2022-03-05 16:22:59,927 DEBUG [localhost-startStop-1] > security.PasswordPlaceholderConfigurer:174 : Loading properties file from > InputStream resource [resource loaded through InputStream] > 2022-03-05 16:23:01,211 INFO [localhost-startStop-1] > metrics.MetricsManager:142 : Kylin metrics monitor is not enabled > 2022-03-05 16:23:01,645 WARN [localhost-startStop-1] > support.XmlWebApplicationContext:551 : Exception encountered during context > initialization - cancelling refresh attempt: > org.springframework.beans.factory.BeanCreationException: Error creating bean > with name > 'org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter': > Instantiation of bean failed; nested exception is > org.springframework.beans.BeanInstantiationException: Failed to instantiate > [org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter]: > Constructor threw exception; nested exception is > java.lang.ClassCastException: > com.fasterxml.jackson.datatype.jsr310.JavaTimeModule cannot be cast to > com.fasterxml.jackson.databind.Module > 2022-03-05 16:23:01,650 ERROR [localhost-startStop-1] > context.ContextLoader:350 : Context initialization failed > org.springframework.beans.factory.BeanCreationException: Error creating bean > with name > 'org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter': > Instantiation of bean failed; nested exception is > org.springframework.beans.BeanInstantiationException: Failed to instantiate > [org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter]: > Constructor threw exception; nested exception is > java.lang.ClassCastException: > com.fasterxml.jackson.datatype.jsr310.JavaTimeModule cannot be cast to > com.fasterxml.jackson.databind.Module > at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1155) > at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1099) > at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:513) > at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483) > at > org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306) > at > org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) > at > org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302) > at > org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) > at > org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761) > at > org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867) > at > org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543) > at > org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443) > at > org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325) > at > org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107) > at > org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4699) > at > org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5165) > at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) > at > org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:743) > at org.apache.catalina.
[jira] [Commented] (KYLIN-5165) Kylin web UI http 404 error
[ https://issues.apache.org/jira/browse/KYLIN-5165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17502681#comment-17502681 ] Yaqian Zhang commented on KYLIN-5165: - Hi [~p0124]: I want't to know what's your JDK version? > Kylin web UI http 404 error > --- > > Key: KYLIN-5165 > URL: https://issues.apache.org/jira/browse/KYLIN-5165 > Project: Kylin > Issue Type: Bug > Components: Web , Website >Affects Versions: v3.0.2 > Environment: kylin 3.0.2 > hadoop 3.1.3 > hbase 2.0.5 > hive 3.1.2 > zookeeper 3.5.7 >Reporter: Junyi Pang >Priority: Major > Fix For: v3.0.2 > > Attachments: kylin.log > > > 2022-03-05 16:22:59,927 DEBUG [localhost-startStop-1] > security.PasswordPlaceholderConfigurer:174 : Loading properties file from > InputStream resource [resource loaded through InputStream] > 2022-03-05 16:23:01,211 INFO [localhost-startStop-1] > metrics.MetricsManager:142 : Kylin metrics monitor is not enabled > 2022-03-05 16:23:01,645 WARN [localhost-startStop-1] > support.XmlWebApplicationContext:551 : Exception encountered during context > initialization - cancelling refresh attempt: > org.springframework.beans.factory.BeanCreationException: Error creating bean > with name > 'org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter': > Instantiation of bean failed; nested exception is > org.springframework.beans.BeanInstantiationException: Failed to instantiate > [org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter]: > Constructor threw exception; nested exception is > java.lang.ClassCastException: > com.fasterxml.jackson.datatype.jsr310.JavaTimeModule cannot be cast to > com.fasterxml.jackson.databind.Module > 2022-03-05 16:23:01,650 ERROR [localhost-startStop-1] > context.ContextLoader:350 : Context initialization failed > org.springframework.beans.factory.BeanCreationException: Error creating bean > with name > 'org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter': > Instantiation of bean failed; nested exception is > org.springframework.beans.BeanInstantiationException: Failed to instantiate > [org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter]: > Constructor threw exception; nested exception is > java.lang.ClassCastException: > com.fasterxml.jackson.datatype.jsr310.JavaTimeModule cannot be cast to > com.fasterxml.jackson.databind.Module > at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1155) > at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1099) > at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:513) > at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483) > at > org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306) > at > org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) > at > org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302) > at > org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) > at > org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761) > at > org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867) > at > org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543) > at > org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443) > at > org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325) > at > org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107) > at > org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4699) > at > org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5165) > at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) > at > org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:743) > at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:719) > at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:705)
[jira] [Created] (KYLIN-5166) Use SQL hint to specify the cube that the user wants to query
Yaqian Zhang created KYLIN-5166: --- Summary: Use SQL hint to specify the cube that the user wants to query Key: KYLIN-5166 URL: https://issues.apache.org/jira/browse/KYLIN-5166 Project: Kylin Issue Type: Improvement Reporter: Yaqian Zhang Assignee: Yaqian Zhang -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (KYLIN-5162) Fix non-select sql pushdown
Yaqian Zhang created KYLIN-5162: --- Summary: Fix non-select sql pushdown Key: KYLIN-5162 URL: https://issues.apache.org/jira/browse/KYLIN-5162 Project: Kylin Issue Type: Bug Reporter: Yaqian Zhang Assignee: Yaqian Zhang -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5146) After using the Ranger plug-in in Kylin-4.0.1, the page for opening Kylin is 404
[ https://issues.apache.org/jira/browse/KYLIN-5146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17474407#comment-17474407 ] Yaqian Zhang commented on KYLIN-5146: - Hi [~qingquan]: The error in the file log is still ClassNotFound. You can put the missing jar in directory `$KYLIN_HOME/lib/` and continue to try. > After using the Ranger plug-in in Kylin-4.0.1, the page for opening Kylin is > 404 > > > Key: KYLIN-5146 > URL: https://issues.apache.org/jira/browse/KYLIN-5146 > Project: Kylin > Issue Type: Bug >Affects Versions: v4.0.1 > Environment: kylin-4.0.1-bin-spark3 > hadoop 3.1.3 > hive 3.1.2 > spark-3.1.1-bin-hadoop3.2 > ranger 2.0.0 >Reporter: qingquanzhang >Priority: Minor > Attachments: kylin-1.log, kylin.log, kylin.out > > > Everything was fine with Kylin-4.0.1 until the ranger plugin was used > However, Kylin-4.0.0 is fine with the Ranger plugin > > ERROR [localhost-startStop-1] context.ContextLoader:350 : Context > initialization failed > org.springframework.beans.factory.BeanCreationException: Error creating bean > with name 'accessController' defined in URL [jar:file:/opt/kylin-4.0.1-bin-spa > rk3_bak/tomcat/webapps/kylin/WEB-INF/lib/kylin-server-base-4.0.1.jar!/org/apache/kylin/rest/controller/AccessController.class]: > Invocation of init method f > ailed; nested exception is java.lang.NoClassDefFoundError: > com/sun/jersey/api/client/GenericType -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5150) Query visualization does not support numeric fields as dimensions
[
https://issues.apache.org/jira/browse/KYLIN-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17473270#comment-17473270
]
Yaqian Zhang commented on KYLIN-5150:
-
Hi [~qingquan]:
Kylin's visualization for query result is only a primary feature, and it does
not make a deeper adaptation to various types of data.
It is recommended to use external BI tools to connect Kylin for visualization.
> Query visualization does not support numeric fields as dimensions
> -
>
> Key: KYLIN-5150
> URL: https://issues.apache.org/jira/browse/KYLIN-5150
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v4.0.1
>Reporter: qingquanzhang
>Priority: Minor
> Attachments: Visualization of query results.png
>
>
> Here are two tables:
> table 1 ,dept:
> ||column||type||
> |deptno|int|
> |dname | string|
> |loc |int|
> table 2 ,emp:
> ||column||type||
> ||empno || int||
> ||ename||string||
> ||job ||string||
> ||sal ||double||
> |deptno| int|
> sql:
>
> {code:java}
> select job,loc,sum(sal) sum_sal from emp e inner join dept d on d.deptno =
> e.deptno group by job,loc;{code}
>
> When visualizing, kylin will use loc fields as metrics, not dimensions
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (KYLIN-5147) Kubernetes production environment support for Kylin4
[ https://issues.apache.org/jira/browse/KYLIN-5147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17471828#comment-17471828 ] Yaqian Zhang edited comment on KYLIN-5147 at 1/12/22, 3:28 AM: --- Hi [~jinzhong]: We did not provide any guide about deploy kylin4 cluster on Kubernetes, because it was not tested. However, we think that the deploy template of kylin3 on k8s should can be reusable on kylin4. Maybe you can try it on kylin4 according to the guide of kylin3 on k8s: https://kylin.apache.org/cn/docs31/install/kylin_on_kubernetes.html. was (Author: zhangyaqian): Hi [~jinzhong]: We did not provide any guide about deploy kylin4 cluster on Kubernetes, because it was not tested. However, we think that the deploy template of kylin3 on k8s should be reusable on kylin4. Maybe you can try it on kylin4 according to the guide of kylin3 on k8s: https://kylin.apache.org/cn/docs31/install/kylin_on_kubernetes.html. > Kubernetes production environment support for Kylin4 > > > Key: KYLIN-5147 > URL: https://issues.apache.org/jira/browse/KYLIN-5147 > Project: Kylin > Issue Type: Wish >Affects Versions: v4.0.0 >Reporter: Jinzhong Hou >Priority: Major > > Hi, > We want to deploy kylin4 cluster on Kubernetes in our prod. > But the bad news is that we only find a guid document and some resources for > Kylin 3. > Does Kylin 4 has support Kubernetes deploy and is there any guide about this ? > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5151) "Remove","Resubmit",The two buttons are too close together
[ https://issues.apache.org/jira/browse/KYLIN-5151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17472436#comment-17472436 ] Yaqian Zhang commented on KYLIN-5151: - Hi [~qingquan]: I think this is a problem that needs users' attention, not Kylin's issue. > "Remove","Resubmit",The two buttons are too close together > -- > > Key: KYLIN-5151 > URL: https://issues.apache.org/jira/browse/KYLIN-5151 > Project: Kylin > Issue Type: Wish > Components: Query Engine >Affects Versions: all >Reporter: qingquanzhang >Priority: Trivial > Attachments: example.png > > > When I clicked Resubmit, the two buttons were too close to each other, so I > accidentally clicked "Remove", resulting in the deletion of SQL -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5147) Kubernetes production environment support for Kylin4
[ https://issues.apache.org/jira/browse/KYLIN-5147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17471828#comment-17471828 ] Yaqian Zhang commented on KYLIN-5147: - Hi [~jinzhong]: We did not provide any guide about deploy kylin4 cluster on Kubernetes, because it was not tested. However, we think that the deploy template of kylin3 on k8s should be reusable on kylin4. Maybe you can try it on kylin4 according to the guide of kylin3 on k8s: https://kylin.apache.org/cn/docs31/install/kylin_on_kubernetes.html. > Kubernetes production environment support for Kylin4 > > > Key: KYLIN-5147 > URL: https://issues.apache.org/jira/browse/KYLIN-5147 > Project: Kylin > Issue Type: Wish >Affects Versions: v4.0.0 >Reporter: Jinzhong Hou >Priority: Major > > Hi, > We want to deploy kylin4 cluster on Kubernetes in our prod. > But the bad news is that we only find a guid document and some resources for > Kylin 3. > Does Kylin 4 has support Kubernetes deploy and is there any guide about this ? > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5149) isExactlyMatchSet should keep same size with cuboidIds
[ https://issues.apache.org/jira/browse/KYLIN-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17471774#comment-17471774 ] Yaqian Zhang commented on KYLIN-5149: - Hi [~sleep1661]： You mean that `Is Exactly Matched` shows three cuboids hit, but `Cuboid Ids` shows only two cuboids, right? It does seem like a bug. > isExactlyMatchSet should keep same size with cuboidIds > -- > > Key: KYLIN-5149 > URL: https://issues.apache.org/jira/browse/KYLIN-5149 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v4.0.1 >Reporter: hujiahua >Priority: Minor > > In some query case, I find the size of `isExactlyMatchSet`i will bigger than > `icuboidIds`i in query log. > > For example, there is a join scene like this : > ==[QUERY]=== > Query Id: d99bdb35-5d97-b0c0-35a9-db933f3cb64f > SQL: select a.*,b.* from > (select SELLER_ID,sum(PRICE) as PRICE from KYLIN_SALES where PART_DT > > '2012-01-01' and PART_DT < '2012-02-01' GROUP BY SELLER_ID limit 10) a > left join KYLIN_ACCOUNT b on a.SELLER_ID = b.ACCOUNT_ID > User: ADMIN > Success: true > Duration: 19.011 > Project: default > Realization Names: [CUBE[name=my_test2_3], CUBE[name=account_info_t1]] > Cuboid Ids: [6, 31] > Is Exactly Matched: [false, false, false] > Total scan count: 10454 > Total scan files: 2 > Total metadata time: 4ms > Total spark scan time: 0ms > Total scan bytes: 55822 > Total return rows: 0 > Result row count: 10 > Storage cache used: false > Is Query Push-Down: false > Is Prepare: false > Used Spark pool: lightweight_tasks > Trace URL: null > Message: null > Remote Address: 127.0.0.1 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (KYLIN-5148) kylin4 构建 cube 报错，Global dict build error on column ITEM_ID, key distinct count is 5076, and value distinct count is 6771
[ https://issues.apache.org/jira/browse/KYLIN-5148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang reassigned KYLIN-5148: --- Assignee: Yaqian Zhang > kylin4 构建 cube 报错，Global dict build error on column ITEM_ID, key distinct > count is 5076, and value distinct count is 6771 > - > > Key: KYLIN-5148 > URL: https://issues.apache.org/jira/browse/KYLIN-5148 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v4.0.1 > Environment: - kylin-4.0.1-bin-spark3 > - spark-3.1.2-bin-hadoop3.2 > - 使用的 spark local 模式 >Reporter: frankhu855 >Assignee: Yaqian Zhang >Priority: Critical > Fix For: v4.0.1 > > Attachments: image-2022-01-09-17-17-51-453.png, screenshot-1.png > > > !image-2022-01-09-17-17-51-453.png|width=1311,height=488! > 复现步骤： > # 执行 $KYLIN_HOME/bin/sample.sh，导入元数据； > # 克隆 kylin_sales_cube，点击 build 构建； > # 使用的 spark local 模式； -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5148) kylin4 构建 cube 报错，Global dict build error on column ITEM_ID, key distinct count is 5076, and value distinct count is 6771
[ https://issues.apache.org/jira/browse/KYLIN-5148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17471772#comment-17471772 ] Yaqian Zhang commented on KYLIN-5148: - Hi [~frankhu855]: I tested Kylin 4.0.1 in my cdh6 environment with spark-3.1.1-bin-hadoop3.2, and I set spark to local mode and build the sample cube successfully. !screenshot-1.png! Kylin4.0.1 officially supported and tested version is spark 3.1.1, you can replace spark3.1.2 with spark3.1.1 and try again. > kylin4 构建 cube 报错，Global dict build error on column ITEM_ID, key distinct > count is 5076, and value distinct count is 6771 > - > > Key: KYLIN-5148 > URL: https://issues.apache.org/jira/browse/KYLIN-5148 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v4.0.1 > Environment: - kylin-4.0.1-bin-spark3 > - spark-3.1.2-bin-hadoop3.2 > - 使用的 spark local 模式 >Reporter: frankhu855 >Priority: Critical > Fix For: v4.0.1 > > Attachments: image-2022-01-09-17-17-51-453.png, screenshot-1.png > > > !image-2022-01-09-17-17-51-453.png|width=1311,height=488! > 复现步骤： > # 执行 $KYLIN_HOME/bin/sample.sh，导入元数据； > # 克隆 kylin_sales_cube，点击 build 构建； > # 使用的 spark local 模式； -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (KYLIN-5148) kylin4 构建 cube 报错，Global dict build error on column ITEM_ID, key distinct count is 5076, and value distinct count is 6771
[ https://issues.apache.org/jira/browse/KYLIN-5148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-5148: Attachment: screenshot-1.png > kylin4 构建 cube 报错，Global dict build error on column ITEM_ID, key distinct > count is 5076, and value distinct count is 6771 > - > > Key: KYLIN-5148 > URL: https://issues.apache.org/jira/browse/KYLIN-5148 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v4.0.1 > Environment: - kylin-4.0.1-bin-spark3 > - spark-3.1.2-bin-hadoop3.2 > - 使用的 spark local 模式 >Reporter: frankhu855 >Priority: Critical > Fix For: v4.0.1 > > Attachments: image-2022-01-09-17-17-51-453.png, screenshot-1.png > > > !image-2022-01-09-17-17-51-453.png|width=1311,height=488! > 复现步骤： > # 执行 $KYLIN_HOME/bin/sample.sh，导入元数据； > # 克隆 kylin_sales_cube，点击 build 构建； > # 使用的 spark local 模式； -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-5054) Kylin_System project and cubes create time is wrong (1970).
[ https://issues.apache.org/jira/browse/KYLIN-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-5054. --- Resolved in release 3.1.3 (2022-01-06) > Kylin_System project and cubes create time is wrong (1970). > --- > > Key: KYLIN-5054 > URL: https://issues.apache.org/jira/browse/KYLIN-5054 > Project: Kylin > Issue Type: Bug > Components: Others >Affects Versions: v3.1.2 >Reporter: Sonu Kumar Singh >Assignee: Sonu Kumar Singh >Priority: Minor > Fix For: v3.1.3 > > > Kylin_System's project and cubes create time is 1970, that is wrong. It > should be current time when user will setup the Kylin_System project. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-5053) Update the website to link to TravisCI and Github Actions instead of Jenkins
[ https://issues.apache.org/jira/browse/KYLIN-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-5053. --- Resolved in release 3.1.3 (2022-01-06) > Update the website to link to TravisCI and Github Actions instead of Jenkins > > > Key: KYLIN-5053 > URL: https://issues.apache.org/jira/browse/KYLIN-5053 > Project: Kylin > Issue Type: Task > Components: Documentation >Affects Versions: v3.1.2 >Reporter: Martin Tzvetanov Grigorov >Priority: Minor > Fix For: v3.1.3 > > > Currently [https://kylin.apache.org/development/#ci-and-code-analysis] points > to [https://builds.apache.org/job/Kylin-Master-JDK-1.8/] as CI, but this url > leads to error 404 (Not Found). > Since a while Kylin uses > [TravisCI|https://app.travis-ci.com/github/apache/kylin] and [GitHub > Actions|https://github.com/apache/kylin/actions] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-4978) NPE: submit refresh request using restful api when there is no segments
[ https://issues.apache.org/jira/browse/KYLIN-4978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-4978. --- Resolved in release 3.1.3 (2022-01-06) > NPE: submit refresh request using restful api when there is no segments > --- > > Key: KYLIN-4978 > URL: https://issues.apache.org/jira/browse/KYLIN-4978 > Project: Kylin > Issue Type: Bug >Reporter: Kun Liu >Assignee: Kun Liu >Priority: Minor > Fix For: v3.1.3 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-5016) Avoid potential NPE issue in RDBMS Pushdown case
[
https://issues.apache.org/jira/browse/KYLIN-5016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yaqian Zhang closed KYLIN-5016.
---
Resolved in release 3.1.3 (2022-01-06)
> Avoid potential NPE issue in RDBMS Pushdown case
>
>
> Key: KYLIN-5016
> URL: https://issues.apache.org/jira/browse/KYLIN-5016
> Project: Kylin
> Issue Type: Improvement
> Components: RDBMS Source
>Affects Versions: v2.6.5, v3.1.2
>Reporter: rongchuan.jin
>Assignee: rongchuan.jin
>Priority: Minor
> Fix For: v3.1.3
>
>
> When I use pushdown with RDBMS source, I encounter some error when convert
> sql which leads to pushdown failing.
> I find below stacktrace like
> {code:java}
> 2020-12-23 13:14:10,212 ERROR [Query a1bf28bb-de28-433e-ab96-28ce234a1a4a-76]
> conv.SqlConverter : Failed to default convert sql, will use the origin input:
> select 1 from `MOVIES_10M`.`DIM_MOVIES_10M` 2LIMIT 500
> 3java.lang.NullPointerException 4 at
> org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:103) 5 at
> org.apache.calcite.sql.pretty.SqlPrettyWriter.format(SqlPrettyWriter.java:806)
> 6 at
> org.apache.kylin.sdk.datasource.framework.conv.SqlConverter.convertSql(SqlConverter.java:69)
> 7 at
> org.apache.kylin.sdk.datasource.framework.JdbcConnector.convertSql(JdbcConnector.java:91)
> 8 at
> org.apache.kylin.sdk.datasource.PushDownRunnerSDKImpl.executeQuery(PushDownRunnerSDKImpl.java:55)
> 9 at
> org.apache.kylin.query.util.PushDownUtil.tryPushDownQuery(PushDownUtil.java:173)
> 10 at
> org.apache.kylin.query.util.PushDownUtil.tryPushDownSelectQuery(PushDownUtil.java:103)
> 11 at
> org.apache.kylin.rest.service.QueryService.tryPushDownSelectQuery(QueryService.java:773)
> 12 at
> org.apache.kylin.rest.service.QueryService.pushDownQuery(QueryService.java:709)
> 13 at
> org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:700)
> 14 at
> org.apache.kylin.rest.service.QueryService.query(QueryService.java:231) 15 at
> org.apache.kylin.rest.service.QueryService.queryAndUpdateCache(QueryService.java:577)
> 16 at
> org.apache.kylin.rest.service.QueryService.queryWithCache(QueryService.java:512)
> 17 at
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:395){code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (KYLIN-4974) Kylin does not handle fucntion CURRENT_TIME().
[
https://issues.apache.org/jira/browse/KYLIN-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yaqian Zhang closed KYLIN-4974.
---
Resolved in release 3.1.3 (2022-01-06)
> Kylin does not handle fucntion CURRENT_TIME().
> --
>
> Key: KYLIN-4974
> URL: https://issues.apache.org/jira/browse/KYLIN-4974
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v3.1.1
>Reporter: Alexander
>Assignee: Alexander
>Priority: Minor
> Fix For: v3.1.3
>
>
> Select query:
> {code:java}
> ==[QUERY]===
> Query Id: 4e8c6498-9736-604f-ca84-c537e749448b
> SQL: select CURRENT_TIME() from KYLIN_SALES
> {code}
> Response:
> {code:java}
> org.apache.kylin.rest.exception.InternalErrorException: No DataTypeSerializer
> for type timeorg.apache.kylin.rest.exception.InternalErrorException: No
> DataTypeSerializer for type timewhile executing SQL: "select * from (select
> CURRENT_TIME() from KYLIN_SALES) limit 5" at
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:486)
> at
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:401)
> at
> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:93)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (KYLIN-4947) Implement spark engine for cube optimization jobs
[ https://issues.apache.org/jira/browse/KYLIN-4947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-4947. --- Resolved in release 3.1.3 (2022-01-06) > Implement spark engine for cube optimization jobs > - > > Key: KYLIN-4947 > URL: https://issues.apache.org/jira/browse/KYLIN-4947 > Project: Kylin > Issue Type: New Feature > Components: Job Engine >Reporter: JiangYang >Assignee: JiangYang >Priority: Minor > Fix For: v3.1.3 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-4944) Upgrade CentOS version, Hadoop version and Spark version for Kylin Docker image
[ https://issues.apache.org/jira/browse/KYLIN-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-4944. --- Resolved in release 3.1.3 (2022-01-06) > Upgrade CentOS version, Hadoop version and Spark version for Kylin Docker > image > --- > > Key: KYLIN-4944 > URL: https://issues.apache.org/jira/browse/KYLIN-4944 > Project: Kylin > Issue Type: Improvement > Components: Others >Affects Versions: v4.0.0-beta >Reporter: Zhichao Zhang >Assignee: Zhichao Zhang >Priority: Minor > Fix For: v4.0.0, v3.1.3 > > > Currently, the centos version of kylin docker image is 6.9, and the yum of > centos 6.9 is no longer maintained, it needs to upgrade to 7+. > It also can upgrade hadoop to 2.8.5 , spark to 2.4.7. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-4948) Provide an API to allow users to adjust cuboids manually
[ https://issues.apache.org/jira/browse/KYLIN-4948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-4948. --- Resolved in release 3.1.3 (2022-01-06) > Provide an API to allow users to adjust cuboids manually > > > Key: KYLIN-4948 > URL: https://issues.apache.org/jira/browse/KYLIN-4948 > Project: Kylin > Issue Type: New Feature > Components: REST Service >Reporter: JiangYang >Assignee: JiangYang >Priority: Minor > Fix For: v4.0.0, v3.1.3 > > > This API should include the following info: > * Cuboids added > * Cuboids removed -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-4970) Fix spark.executor.extraJavaOptions args
[ https://issues.apache.org/jira/browse/KYLIN-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-4970. --- Resolved in release 3.1.3 (2022-01-06) > Fix spark.executor.extraJavaOptions args > > > Key: KYLIN-4970 > URL: https://issues.apache.org/jira/browse/KYLIN-4970 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Reporter: JiangYang >Assignee: JiangYang >Priority: Minor > Fix For: v3.1.3 > > > in spark doc: spark.executor.extraJavaOption need surrond with "". > !image-2021-04-09-10-18-58-294.png! -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-4971) Add new measure bitmap_map for count distinct measure in UI
[ https://issues.apache.org/jira/browse/KYLIN-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-4971. --- Resolved in release 3.1.3 (2022-01-06) > Add new measure bitmap_map for count distinct measure in UI > > > Key: KYLIN-4971 > URL: https://issues.apache.org/jira/browse/KYLIN-4971 > Project: Kylin > Issue Type: Task >Reporter: JiangYang >Assignee: JiangYang >Priority: Minor > Fix For: v3.1.3 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-5035) Fix Merge Cuboid Statistics EOFException
[ https://issues.apache.org/jira/browse/KYLIN-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-5035. --- Resolved in release 3.1.3 (2022-01-06) > Fix Merge Cuboid Statistics EOFException > > > Key: KYLIN-5035 > URL: https://issues.apache.org/jira/browse/KYLIN-5035 > Project: Kylin > Issue Type: Bug > Components: Job Engine, Spark Engine >Reporter: JiangYang >Assignee: JiangYang >Priority: Major > Fix For: v3.1.3 > > > org.apache.kylin.job.exception.ExecuteException: java.io.EOFException at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:201) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:72) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:180) > at > org.apache.kylin.job.impl.threadpool.DistributedScheduler\$JobRunner.run(DistributedScheduler.java:238) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor\$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) Caused by: java.io.EOFException at > java.io.DataInputStream.readFully(DataInputStream.java:197) at > java.io.DataInputStream.readFully(DataInputStream.java:169) at > org.apache.hadoop.io.SequenceFile\$Reader.init(SequenceFile.java:1918) at > org.apache.hadoop.io.SequenceFile\$Reader.initialize(SequenceFile.java:1883) > at org.apache.hadoop.io.SequenceFile\$Reader.(SequenceFile.java:1832) at > org.apache.kylin.engine.mr.common.CubeStatsReader\$CubeStatsResult.(CubeStatsReader.java:484) > at > org.apache.kylin.engine.mr.common.CubeStatsReader.(CubeStatsReader.java:133) > at > org.apache.kylin.engine.mr.steps.MergeStatisticsWithOldStep.doWork(MergeStatisticsWithOldStep.java:97) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:180) > ... 6 more -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-4982) Add special spark-sql conf in create intermediate flat table
[ https://issues.apache.org/jira/browse/KYLIN-4982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-4982. --- Resolved in release 3.1.3 (2022-01-06) > Add special spark-sql conf in create intermediate flat table > > > Key: KYLIN-4982 > URL: https://issues.apache.org/jira/browse/KYLIN-4982 > Project: Kylin > Issue Type: New Feature > Components: Spark Engine >Reporter: JiangYang >Assignee: JiangYang >Priority: Major > Fix For: v3.1.3 > > > like enable AE... -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-5003) Fail to package Kylin due to legacy front end dependencies
[
https://issues.apache.org/jira/browse/KYLIN-5003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yaqian Zhang closed KYLIN-5003.
---
Resolved in release 3.1.3 (2022-01-06)
> Fail to package Kylin due to legacy front end dependencies
> --
>
> Key: KYLIN-5003
> URL: https://issues.apache.org/jira/browse/KYLIN-5003
> Project: Kylin
> Issue Type: Bug
> Components: Web
>Affects Versions: v3.1.2
>Reporter: nichunen
>Assignee: nichunen
>Priority: Major
> Fix For: v3.1.3
>
>
> I tried to package Kylin with build/script/package.sh, and it failed with
> error
> {quote}bower angular-unstable#1.1.5 ECMDERR Failed
> to execute "git ls-remote --tags --heads
> https://github.com/johannestroeger/bower-angular-unstable.git";, exit code of
> #128 remote: Repository not found. fatal: repository
> 'https://github.com/johannestroeger/bower-angular-unstable.git/' not
> found{quote}
> And the dependency "phantomjs" can not be installed with "npm install
> phantomjs" now.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (KYLIN-5007) queries with limit clause may fail when string dimension is encoded in integer type
[
https://issues.apache.org/jira/browse/KYLIN-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yaqian Zhang closed KYLIN-5007.
---
Resolved in release 3.1.3 (2022-01-06)
> queries with limit clause may fail when string dimension is encoded in
> integer type
> ---
>
> Key: KYLIN-5007
> URL: https://issues.apache.org/jira/browse/KYLIN-5007
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v3.0.2
>Reporter: Congling Xia
>Assignee: Congling Xia
>Priority: Major
> Fix For: v3.1.3
>
> Attachments: image-2021-06-10-10-03-54-775.png
>
>
> Hi, team.
> Recently we encounter a problem that queries may fail if there is a LIMIT in
> the SQL. The SQL looks like:
> {code}
> select gid from some_table group by gid limit 100
> {code}
> The error message is like the following:
> {code:java}
> Not sorted! last: source_v1=null,...,gid=276,... fetched:
> source_v1=null,...,gid=100506,...
> {code}
> After searching the issues list, we find it is similar with KYLIN-2425,
> KYLIN-3089, and KYLIN-4942. We notice that these problems are not completely
> resolved.
> It is an row-key encoding problem, the cube uses integer:4 to encode string
> column _gid_:
> !image-2021-06-10-10-03-54-775.png|width=571,height=141!
> As [~kangkaisen] mensioned in KYLIN-3089, comparator in
> SortMergedPartitionResultIterator is different from the one in
> SortedIteratorMergerWithLimit. SortedIteratorMergerWithLimit compares tuple
> of dimensions in their origin data type "string" rather than the encoded data
> type "integer" in rowkeys. In the exception message above, 276<100506 is
> false because they are compared in "string" type.
> It may be resolved by skipping limit pushdown when column type and encoding
> type may produce different comparing results, but it may lead such queries to
> be slower.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (KYLIN-4983) The stream cube will be paused when user append a batch segment first
[
https://issues.apache.org/jira/browse/KYLIN-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yaqian Zhang closed KYLIN-4983.
---
Resolved in release 3.1.3 (2022-01-06)
> The stream cube will be paused when user append a batch segment first
> -
>
> Key: KYLIN-4983
> URL: https://issues.apache.org/jira/browse/KYLIN-4983
> Project: Kylin
> Issue Type: Bug
> Components: Real-time Streaming
>Reporter: Kun Liu
>Assignee: Kun Liu
>Priority: Blocker
> Fix For: v3.1.3
>
>
> Env:
> stream cube (stream_cube) with lambda, and the window of cube is 1hour
> Before enable the stream_cube, we submit a build restful request with the
> range [2025-01-01,2025-01-02]
>
> Then enable the stream_cube, the receiver node will consume data and create
> segment in real time, but the start time of the new segment is must be less
> than `2025-01-01`.
>
> When the receiver node upload the data of the segment to the HDFS and notify
> the coordinator, and the coordinator will delete the metadata of the stream
> segment like below code, but the segment of data can't be delete in the
> receiver node.
>
> ```
> // If we have a exist historical segment, we should not let new realtime
> segment overwrite it, it is so dangrous,
> // we just delete the entry to ignore the segment which should not exist
> if (segmentRange.getFirst() < minSegmentStart) {
> logger.warn(
> "The cube segment state is not correct because it belongs to historical
> part, cube:{} segment:{}, clear it.",
> cubeName, segmentState.getSegmentName());
> coordinator.getStreamMetadataStore().removeSegmentBuildState(cubeName,
> segmentState.getSegmentName());
> continue;
> }
> ```
> The number of immutable segment will reach to 100.
>
> There are two way to resolve this issue:
>
> # forbid appending segment in stream with lambda for the restful api
> # delete local data and remote hdfs data before remove metadata
> Now i think it is a better way to forbid appending segment in lambda stream
> cube.
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (KYLIN-4964) Receiver consumer thread should be stoped while encounting unrecoverable error
[ https://issues.apache.org/jira/browse/KYLIN-4964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-4964. --- Resolved in release 3.1.3 (2022-01-06) > Receiver consumer thread should be stoped while encounting unrecoverable error > -- > > Key: KYLIN-4964 > URL: https://issues.apache.org/jira/browse/KYLIN-4964 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming >Affects Versions: v3.1.1 >Reporter: Xingxing Di >Assignee: Xingxing Di >Priority: Major > Fix For: v3.1.3 > > > org.apache.kylin.stream.core.storage.columnar.ColumnarMemoryStorePersister#persist > will catch exception and just log it, this behavior will cause critical > problem, like our case: there is no space left on device, `persist` method > failed for many times, we lost several hours of data. > Here is our solution which already tested: > # Throw IllegalStorageException while we cannot do the persist > # Then stop the consumer thread > # Add `consumer_thread_alive` in ConsumerStats for monitoring > # Also fix an another issue which will cause wrong result after restore from > checkpoint > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-4554) Validate "filter condition" on model saving
[ https://issues.apache.org/jira/browse/KYLIN-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-4554. --- Resolved in release 3.1.3 (2022-01-06) > Validate "filter condition" on model saving > --- > > Key: KYLIN-4554 > URL: https://issues.apache.org/jira/browse/KYLIN-4554 > Project: Kylin > Issue Type: Improvement > Components: Metadata >Reporter: Shao Feng Shi >Assignee: Sun BiaoBiao >Priority: Major > Fix For: v4.0.0, v3.1.3 > > Attachments: WechatIMG397.png > > > When define a model, user can enter a filter condition (using to do data > filtering on loading). While the system doesn't validate the filtering > condition's syntax; Later when build the cube, the condition will be send to > Hive, Hive may report an error on the syntax, but at that time, it is > difficult to modify it. > > !WechatIMG397.png! -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-5097) modify hive dict table format from textfile to orc
[ https://issues.apache.org/jira/browse/KYLIN-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-5097. --- Resolved in release 3.1.3 (2022-01-06) > modify hive dict table format from textfile to orc > -- > > Key: KYLIN-5097 > URL: https://issues.apache.org/jira/browse/KYLIN-5097 > Project: Kylin > Issue Type: Bug >Affects Versions: v3.1.1, v3.1.2 >Reporter: fengpod >Assignee: fengpod >Priority: Minor > Fix For: v3.1.3 > > > Textfile format end with ‘\n’ by default， when the data itself contains ‘\n’, > dict table may appear dirty data。 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-4995) Query exception when the query statement contains a single left parenthesis
[
https://issues.apache.org/jira/browse/KYLIN-4995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yaqian Zhang closed KYLIN-4995.
---
Resolved in release 3.1.3 (2022-01-06)
> Query exception when the query statement contains a single left parenthesis
> ---
>
> Key: KYLIN-4995
> URL: https://issues.apache.org/jira/browse/KYLIN-4995
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v3.1.2
>Reporter: Mingming Ge
>Assignee: Mingming Ge
>Priority: Critical
> Fix For: v3.1.3
>
> Attachments: image-2021-05-19-10-15-34-215.png
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Queries like this：
>
> {code:java}
> select '( a + b) * (c+ d ' from t；{code}
> The following exception will occur during execution，
> !image-2021-05-19-10-15-34-215.png|width=592,height=326!
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (KYLIN-4992) Source row count statistics calculated in a wrong way in MergeDictionaryMapper
[ https://issues.apache.org/jira/browse/KYLIN-4992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-4992. --- Resolved in release 3.1.3 (2022-01-06) > Source row count statistics calculated in a wrong way in MergeDictionaryMapper > -- > > Key: KYLIN-4992 > URL: https://issues.apache.org/jira/browse/KYLIN-4992 > Project: Kylin > Issue Type: Bug >Affects Versions: v3.0.0, v2.6.2, v2.6.3, v3.1.0, v3.0.1, v3.0.2, v3.1.1, > v3.1.2 >Reporter: Zhong Yanghong >Assignee: JiangYang >Priority: Critical > Fix For: v3.1.3 > > > With this bug, source row count will be smaller than the correct one and it > will result in smaller cuboid size estimation and smaller region number. > Finally it will impact job and query performance. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (KYLIN-5097) modify hive dict table format from textfile to orc
[ https://issues.apache.org/jira/browse/KYLIN-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang reassigned KYLIN-5097: --- Assignee: fengpod > modify hive dict table format from textfile to orc > -- > > Key: KYLIN-5097 > URL: https://issues.apache.org/jira/browse/KYLIN-5097 > Project: Kylin > Issue Type: Bug >Affects Versions: v3.1.1, v3.1.2 >Reporter: fengpod >Assignee: fengpod >Priority: Minor > Fix For: v3.1.3 > > > Textfile format end with ‘\n’ by default， when the data itself contains ‘\n’, > dict table may appear dirty data。 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (KYLIN-5097) modify hive dict table format from textfile to orc
[ https://issues.apache.org/jira/browse/KYLIN-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang resolved KYLIN-5097. - Resolution: Fixed > modify hive dict table format from textfile to orc > -- > > Key: KYLIN-5097 > URL: https://issues.apache.org/jira/browse/KYLIN-5097 > Project: Kylin > Issue Type: Bug >Affects Versions: v3.1.1, v3.1.2 >Reporter: fengpod >Assignee: fengpod >Priority: Minor > Fix For: v3.1.3 > > > Textfile format end with ‘\n’ by default， when the data itself contains ‘\n’, > dict table may appear dirty data。 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-5112) Use 'return' instead of 'exit' in script prepare_hadoop_dependency.sh
[ https://issues.apache.org/jira/browse/KYLIN-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-5112. --- Resolved in release 4.0.1 (2022-01-06) > Use 'return' instead of 'exit' in script prepare_hadoop_dependency.sh > - > > Key: KYLIN-5112 > URL: https://issues.apache.org/jira/browse/KYLIN-5112 > Project: Kylin > Issue Type: Bug > Components: Tools, Build and Test >Affects Versions: v4.0.1 >Reporter: Yaqian Zhang >Assignee: mukvin >Priority: Minor > Fix For: v4.0.1 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-5111) Record the time spent for each stage of query in kylin4's log
[ https://issues.apache.org/jira/browse/KYLIN-5111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-5111. --- Resolved in release 4.0.1 (2022-01-06) > Record the time spent for each stage of query in kylin4's log > - > > Key: KYLIN-5111 > URL: https://issues.apache.org/jira/browse/KYLIN-5111 > Project: Kylin > Issue Type: Improvement > Components: Metrics >Affects Versions: v4.0.0 >Reporter: Yaqian Zhang >Assignee: Yaqian Zhang >Priority: Minor > Fix For: v4.0.1 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-5086) When query pushdown, only measure result is empty
[ https://issues.apache.org/jira/browse/KYLIN-5086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-5086. --- Resolved in release 4.0.1 (2022-01-06) > When query pushdown, only measure result is empty > - > > Key: KYLIN-5086 > URL: https://issues.apache.org/jira/browse/KYLIN-5086 > Project: Kylin > Issue Type: Bug >Affects Versions: v4.0.0 >Reporter: mukvin >Assignee: mukvin >Priority: Minor > Fix For: v4.0.1 > > Attachments: image-2021-09-09-10-24-30-999.png, > image-2021-09-09-10-25-43-672.png, image-2021-09-09-11-04-25-230.png, > image-2021-09-09-11-06-22-174.png, image-2021-09-09-11-18-23-937.png, > image-2021-09-09-11-19-47-854.png, image-2021-09-09-11-21-07-953.png, > image-2021-09-09-11-23-27-633.png, image-2021-09-09-11-25-27-561.png, > image-2021-09-09-11-26-37-832.png, image-2021-09-09-11-27-34-611.png, > image-2021-09-09-11-28-15-801.png > > > when query pushdown with aggregation func without a alias will get an empty > appearance, but the real result already returned in response. > result as followed: > !image-2021-09-09-10-24-30-999.png! > !image-2021-09-09-10-25-43-672.png! > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-5076) Missing tracking URL in spark standalone mode in kylin4
[ https://issues.apache.org/jira/browse/KYLIN-5076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-5076. --- Resolved in release 4.0.1 (2022-01-06) > Missing tracking URL in spark standalone mode in kylin4 > --- > > Key: KYLIN-5076 > URL: https://issues.apache.org/jira/browse/KYLIN-5076 > Project: Kylin > Issue Type: Improvement >Affects Versions: v4.0.0 >Reporter: Yaqian Zhang >Assignee: Yaqian Zhang >Priority: Minor > Fix For: v4.0.1 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-5090) The error message was not printed in class JobWoker
[
https://issues.apache.org/jira/browse/KYLIN-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yaqian Zhang closed KYLIN-5090.
---
Resolved in release 4.0.1 (2022-01-06)
> The error message was not printed in class JobWoker
> ---
>
> Key: KYLIN-5090
> URL: https://issues.apache.org/jira/browse/KYLIN-5090
> Project: Kylin
> Issue Type: Improvement
>Affects Versions: v4.0.0
>Reporter: Yaqian Zhang
>Assignee: Yaqian Zhang
>Priority: Minor
> Fix For: v4.0.1
>
>
> In the execute method in JobWorker, after the error is thrown, the error
> details are not printed, but called the method for handling the error. When
> the method for handling the error is executed, other errors are generated,
> resulting in the real error information being hidden.
> {code:java}
> private def execute(): Unit = {
> pool.execute(new Runnable {
> override def run(): Unit = {
> try {
> logInfo("Start running job.")
> application.execute(args)
> eventLoop.post(JobSucceeded())
> } catch {
> case exception: NoRetryException =>
> eventLoop.post(UnknownThrowable(exception))
> case throwable: Throwable => eventLoop.post(ResourceLack(throwable))
> }
> }
> })
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (KYLIN-5083) Docker for kylin4 need to check hadoop service before start kylin
[ https://issues.apache.org/jira/browse/KYLIN-5083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-5083. --- Resolved in release 4.0.1 (2022-01-06) > Docker for kylin4 need to check hadoop service before start kylin > - > > Key: KYLIN-5083 > URL: https://issues.apache.org/jira/browse/KYLIN-5083 > Project: Kylin > Issue Type: Bug >Affects Versions: v4.0.0 >Reporter: mukvin >Assignee: mukvin >Priority: Minor > Fix For: v4.0.1 > > Attachments: image-2021-09-06-14-39-35-447.png, > image-2021-09-06-14-43-00-822.png, image-2021-09-08-15-16-06-669.png, > image-2021-09-08-15-18-01-786.png, image-2021-09-08-15-19-52-055.png > > > When start docker of kylin4, someone will found that error in > $KYLIN_HOME/logs/kylin.log as followed: > # kylin webui can't open > # hdfs://spark2_jars not created. In kylin.properties, > kylin.query.spark-conf.spark.yarn.jars is set to > hdfs://localhost:9000/spar2_jars/*. > !image-2021-09-06-14-39-35-447.png|width=411,height=176! > !image-2021-09-06-14-43-00-822.png|width=409,height=107! > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-5071) kylin.engine.build-base-cuboid-enabled=false and kylin.cube.cubeplanner.enabled=true are conflict on build cube step, and will throw NPE.
[ https://issues.apache.org/jira/browse/KYLIN-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-5071. --- Resolved in release 4.0.1 (2022-01-06) > kylin.engine.build-base-cuboid-enabled=false and > kylin.cube.cubeplanner.enabled=true are conflict on build cube step, and will > throw NPE. > - > > Key: KYLIN-5071 > URL: https://issues.apache.org/jira/browse/KYLIN-5071 > Project: Kylin > Issue Type: Bug >Reporter: mukvin >Assignee: Yaqian Zhang >Priority: Minor > Fix For: v4.0.1 > > Attachments: image-2021-09-01-14-22-58-562.png, > image-2021-09-01-14-23-44-046.png, image-2021-09-01-14-24-16-112.png > > > test on kylin docker, then found that the properties > kylin.engine.build-base-cuboid-enabled=false and > kylin.cube.cubeplanner.enabled=true is conflict on build cube step, and will > throw NPE. > WorkRound: > We can set the kylin.engine.build-base-cuboid-enabled=true on the cube level > of properties to build cube successfully. > > Error: > !image-2021-09-01-14-23-44-046.png! > Properties: > !image-2021-09-01-14-22-58-562.png! > > > !image-2021-09-01-14-24-16-112.png! -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-5082) Exactly aggregation for percentile function
[ https://issues.apache.org/jira/browse/KYLIN-5082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-5082. --- Resolved in release 4.0.1 (2022-01-06) > Exactly aggregation for percentile function > --- > > Key: KYLIN-5082 > URL: https://issues.apache.org/jira/browse/KYLIN-5082 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Affects Versions: v4.0.0 >Reporter: Zhiyu Zeng >Assignee: Zhiyu Zeng >Priority: Major > Fix For: v4.0.1 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-5117) Support percentile function after aggregate sub query
[
https://issues.apache.org/jira/browse/KYLIN-5117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yaqian Zhang closed KYLIN-5117.
---
Resolved in release 4.0.1 (2022-01-06)
> Support percentile function after aggregate sub query
> -
>
> Key: KYLIN-5117
> URL: https://issues.apache.org/jira/browse/KYLIN-5117
> Project: Kylin
> Issue Type: New Feature
>Affects Versions: v4.0.0
>Reporter: hujiahua
>Assignee: hujiahua
>Priority: Major
> Fix For: v4.0.1
>
>
> The following query execution will report an error：
> {quote}
> select percentile(t.gmv, 0.5) from ( select buyer_id,sum(price) as gmv from
> kylin_sales group by buyer_id ) t
> java.lang.IllegalArgumentException: Unsupported function name
> PERCENTILE_APPROX while executing SQL: "select percentile(t.gmv, 0.5) from (
> select buyer_id,sum(price) as gmv from kylin_sales group by buyer_id ) t"
> {quote}
> So it's better to support `percentile` function after aggregate sub query.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (KYLIN-5084) Kylin4.0.0 is incompatible with HDP-3.1.5`s Hive-3.1.0
[
https://issues.apache.org/jira/browse/KYLIN-5084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yaqian Zhang closed KYLIN-5084.
---
Resolved in release 4.0.1 (2022-01-06)
> Kylin4.0.0 is incompatible with HDP-3.1.5`s Hive-3.1.0
> --
>
> Key: KYLIN-5084
> URL: https://issues.apache.org/jira/browse/KYLIN-5084
> Project: Kylin
> Issue Type: Improvement
> Components: Metadata
>Affects Versions: v4.0.0
> Environment: HDP-3.1.5+Hadoop-3.1.1+Spark-2.3.0+Hive-3.1.0
>Reporter: lixinzhao
>Assignee: mukvin
>Priority: Major
> Fix For: v4.0.1
>
> Attachments: 捕获.JPG
>
>
> The following error message is displayed during Hive`s table synchronization：
> {code:java}
> org.apache.kylin.rest.exception.InternalErrorException:
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(Lorg/apache/hadoop/hive/conf/HiveConf;)V
> at
> org.apache.kylin.rest.controller.TableController.showHiveDatabases(TableController.java:201)
> ~[kylin-server-base-4.0.0.jar:4.0.0]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_291]
> ...
> Caused by: java.lang.NoSuchMethodError:
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(Lorg/apache/hadoop/hive/conf/HiveConf;)V
>
> at
> org.apache.kylin.source.hive.HiveMetaStoreClientFactory.getHiveMetaStoreClient(HiveMetaStoreClientFactory.java:47)
> ~[kylin-source-hive-4.0.0.jar:4.0.0]
> at
> org.apache.kylin.source.hive.CLIHiveClient.getMetaStoreClient(CLIHiveClient.java:164)
> ~[kylin-source-hive-4.0.0.jar:4.0.0]
> at
> org.apache.kylin.source.hive.CLIHiveClient.getHiveDbNames(CLIHiveClient.java:118)
> ~[kylin-source-hive-4.0.0.jar:4.0.0]
> {code}
> I found the method named getHiveMetaStoreClient of
> org.apache.kylin.source.hive.HiveMetaStoreClientFactory is
> unbefitting,because the HiveMetaStoreClient`s init methos of hive3.1.2 no
> longer accepts parameters of type HiveConf but Configuration.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (KYLIN-5069) Remove find-hive-dependency.sh in Kylin 4
[
https://issues.apache.org/jira/browse/KYLIN-5069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yaqian Zhang closed KYLIN-5069.
---
Resolved in release 4.0.1 (2022-01-06)
> Remove find-hive-dependency.sh in Kylin 4
> -
>
> Key: KYLIN-5069
> URL: https://issues.apache.org/jira/browse/KYLIN-5069
> Project: Kylin
> Issue Type: Improvement
>Affects Versions: v4.0.0
>Reporter: Xiaoxiang Yu
>Assignee: Yaqian Zhang
>Priority: Major
> Fix For: v4.0.1
>
> Attachments: image-2021-08-30-16-10-46-712.png
>
>
> In kylin.sh, Kylin introduces a lot of jars into Kylin instance's class path,
> via find-hive-dependency.sh and "hadoop classpath". So class conflict issues
> happened often, and most of them are related with hive jars. I think it is
> better to remove 'find-hive-dependency.sh' and use SparkSession to fetch
> Hive meta.
>
> Following is part of script kylin.sh :
> {code:java}
> tomcat_classpath=${tomcat_root}/bin/bootstrap.jar:${tomcat_root}/bin/tomcat-juli.jar:${tomcat_root}/lib/*
> export
> KYLIN_TOMCAT_CLASSPATH=${tomcat_classpath}:${KYLIN_HOME}/conf:${KYLIN_HOME}/lib/*:${KYLIN_HOME}/ext/*:${hadoop_dependencies}:${flink_dependency}
> {code}
>
> Some exception looks like this:
> !image-2021-08-30-16-10-46-712.png!
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (KYLIN-5067) CubeBuildJob build unnecessary snapshot
[
https://issues.apache.org/jira/browse/KYLIN-5067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yaqian Zhang closed KYLIN-5067.
---
Resolved in release 4.0.1 (2022-01-06)
> CubeBuildJob build unnecessary snapshot
>
>
> Key: KYLIN-5067
> URL: https://issues.apache.org/jira/browse/KYLIN-5067
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v4.0.0-beta
>Reporter: Xiaoxiang Yu
>Assignee: Xiaoxiang Yu
>Priority: Major
> Fix For: v4.0.1
>
>
> In TPC-H benchmark, the query-13, which contains a 'left outer join', and its
> right table's join key(o_custkey), is not unique. And it will cause the build
> job failed with following exception.
>
> {code:java}
> java.lang.RuntimeException: Error execute
> org.apache.kylin.engine.spark.job.CubeBuildJob
> at
> org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:96)
> at
> org.apache.spark.application.JobWorker$$anon$2.run(JobWorker.scala:55)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.IllegalStateException: Failed to build lookup table
> V_ORDERS snapshot for Dup key found, key= O_CUSTKEY
> at
> org.apache.kylin.engine.spark.builder.CubeSnapshotBuilder$$anonfun$checkDupKey$1.apply(CubeSnapshotBuilder.scala:198)
> at
> org.apache.kylin.engine.spark.builder.CubeSnapshotBuilder$$anonfun$checkDupKey$1.apply(CubeSnapshotBuilder.scala:190)
> at
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
> at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
> at
> org.apache.kylin.engine.spark.builder.CubeSnapshotBuilder.checkDupKey(CubeSnapshotBuilder.scala:189)
> at
> org.apache.kylin.engine.spark.job.ParentSourceChooser.decideFlatTableSource(ParentSourceChooser.scala:83)
> at
> org.apache.kylin.engine.spark.job.ParentSourceChooser$$anonfun$decideSources$1.apply(ParentSourceChooser.scala:71)
> at
> org.apache.kylin.engine.spark.job.ParentSourceChooser$$anonfun$decideSources$1.apply(ParentSourceChooser.scala:66)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> at
> org.apache.kylin.engine.spark.job.ParentSourceChooser.decideSources(ParentSourceChooser.scala:66)
> at
> org.apache.kylin.engine.spark.job.CubeBuildJob.doExecute(CubeBuildJob.java:178)
> at
> org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:304)
> at
> org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:93)
> ... 4 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (KYLIN-4864) Support building and testing Kylin on ARM64 architecture platform
[ https://issues.apache.org/jira/browse/KYLIN-4864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-4864. --- Resolved in release 4.0.1 (2022-01-06) > Support building and testing Kylin on ARM64 architecture platform > - > > Key: KYLIN-4864 > URL: https://issues.apache.org/jira/browse/KYLIN-4864 > Project: Kylin > Issue Type: Improvement >Reporter: liusheng >Priority: Major > Fix For: v3.1.3, v4.0.1 > > > Currently, there are many softwares have support running on ARM64 platform. > We have also done many efforts about making big-data projects support ARM64 > platform. > For an example, Hadoop has published ARM64 platform specific packages: > [https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-3.3.0/hadoop-3.3.0-aarch64.tar.gz] > > and also have ARM specific CI job configured: > [https://ci-hadoop.apache.org/job/Hive-trunk-linux-ARM/] > It would be better to also enable ARM support and setup ARM CI for Kylin > projects -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-5131) java.lang.UnsupportedOperationException: API getTableSnapshots is not supported in Kylin 4.0
[ https://issues.apache.org/jira/browse/KYLIN-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-5131. --- Assignee: Yaqian Zhang Resolved in release 4.0.1 (2022-01-06) > java.lang.UnsupportedOperationException: API getTableSnapshots is not > supported in Kylin 4.0 > > > Key: KYLIN-5131 > URL: https://issues.apache.org/jira/browse/KYLIN-5131 > Project: Kylin > Issue Type: Bug > Components: Others >Affects Versions: v4.0.0 > Environment: CDH 6.3.2; Hadoop 3.0.0; Hive 2.1.1; Spark 2.4.0 >Reporter: Yuming Cao >Assignee: Yaqian Zhang >Priority: Blocker > Fix For: v4.0.1 > > Attachments: kylin-1.out > > > I depolyed the kylin4.0, then loaded a table from tree in Data Source, there > was no changes in the monitor when i [Reload Table] the table， so i viewed > the log. > 16:46:09.653 [http-bio-7070-exec-3] ERROR > org.apache.kylin.rest.controller.BasicController - > java.lang.UnsupportedOperationException: API getTableSnapshots is not > supported in Kylin 4.0 . > at > org.apache.kylin.rest.controller.TableController.getTableSnapshots(TableController.java:227) > ~[kylin-server-base-4.0.0.jar:4.0.0] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5131) java.lang.UnsupportedOperationException: API getTableSnapshots is not supported in Kylin 4.0
[ https://issues.apache.org/jira/browse/KYLIN-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17469696#comment-17469696 ] Yaqian Zhang commented on KYLIN-5131: - Resolved in release 4.0.1 (2022-01-06) > java.lang.UnsupportedOperationException: API getTableSnapshots is not > supported in Kylin 4.0 > > > Key: KYLIN-5131 > URL: https://issues.apache.org/jira/browse/KYLIN-5131 > Project: Kylin > Issue Type: Bug > Components: Others >Affects Versions: v4.0.0 > Environment: CDH 6.3.2; Hadoop 3.0.0; Hive 2.1.1; Spark 2.4.0 >Reporter: Yuming Cao >Priority: Blocker > Fix For: v4.0.1 > > Attachments: kylin-1.out > > > I depolyed the kylin4.0, then loaded a table from tree in Data Source, there > was no changes in the monitor when i [Reload Table] the table， so i viewed > the log. > 16:46:09.653 [http-bio-7070-exec-3] ERROR > org.apache.kylin.rest.controller.BasicController - > java.lang.UnsupportedOperationException: API getTableSnapshots is not > supported in Kylin 4.0 . > at > org.apache.kylin.rest.controller.TableController.getTableSnapshots(TableController.java:227) > ~[kylin-server-base-4.0.0.jar:4.0.0] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (KYLIN-5145) Query Failed:wrong number of arguments while executing SQL
[
https://issues.apache.org/jira/browse/KYLIN-5145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17469160#comment-17469160
]
Yaqian Zhang edited comment on KYLIN-5145 at 1/5/22, 9:55 AM:
--
[~qingquan]
{code:java}
Skip spark which not owned by kylin. SPARK_HOME is
/opt/kylin/spark-3.1.1-bin-hadoop3.2 and KYLIN_HOME is /opt/kylin.
{code}
The reason for this message is that your spark is placed in $kylin_ Home
directory, but it has not been renamed "spark"
, so kylin will not do some jar package replacement for this spark. You can
rename "spark-3.1.1-bin-hadoop3.2" to "spark" and try again.
was (Author: zhangyaqian):
[~qingquan]
{code:java}
Skip spark which not owned by kylin. SPARK_HOME is
/opt/kylin/spark-3.1.1-bin-hadoop3.2 and KYLIN_HOME is /opt/kylin.
{code}
The reason for this message is that your spark is placed in $kylin_ Home
directory, but it has not been renamed {code:java}spark{code}
, so kylin will not do some jar package replacement for this spark. You can
rename "spark-3.1.1-bin-hadoop3.2" to "spark" and try again.
> Query Failed:wrong number of arguments while executing SQL
> --
>
> Key: KYLIN-5145
> URL: https://issues.apache.org/jira/browse/KYLIN-5145
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v4.0.0
> Environment: spark3.1.1
> hadoop3.1.3
> hive:3.1.2
> kylin4.0.0-spark3
>Reporter: qingquanzhang
>Priority: Major
> Attachments: find-spark-dependency.sh, kylin.log
>
>
> An error occurred when I initiated the query:
> java.lang.IllegalArgumentException: wrong number of arguments while executing
> SQL: "select * from (select count(*) from kylin_sales) limit 5"
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (KYLIN-5145) Query Failed:wrong number of arguments while executing SQL
[
https://issues.apache.org/jira/browse/KYLIN-5145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17469160#comment-17469160
]
Yaqian Zhang edited comment on KYLIN-5145 at 1/5/22, 9:54 AM:
--
[~qingquan]
{code:java}
Skip spark which not owned by kylin. SPARK_HOME is
/opt/kylin/spark-3.1.1-bin-hadoop3.2 and KYLIN_HOME is /opt/kylin.
{code}
The reason for this message is that your spark is placed in $kylin_ Home
directory, but it has not been renamed {code:java}spark{code}
, so kylin will not do some jar package replacement for this spark. You can
rename "spark-3.1.1-bin-hadoop3.2" to "spark" and try again.
was (Author: zhangyaqian):
[~qingquan]
{code:java}
Skip spark which not owned by kylin. SPARK_HOME is
/opt/kylin/spark-3.1.1-bin-hadoop3.2 and KYLIN_HOME is /opt/kylin.
{code}
The reason for this message is that your spark is placed in $kylin_ Home
directory, but it has not been renamed
{code:java}
spark
{code}
, so kylin will not do some jar package replacement for this spark. You can
rename "spark-3.1.1-bin-hadoop3.2" to "spark" and try again.
> Query Failed:wrong number of arguments while executing SQL
> --
>
> Key: KYLIN-5145
> URL: https://issues.apache.org/jira/browse/KYLIN-5145
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v4.0.0
> Environment: spark3.1.1
> hadoop3.1.3
> hive:3.1.2
> kylin4.0.0-spark3
>Reporter: qingquanzhang
>Priority: Major
> Attachments: find-spark-dependency.sh, kylin.log
>
>
> An error occurred when I initiated the query:
> java.lang.IllegalArgumentException: wrong number of arguments while executing
> SQL: "select * from (select count(*) from kylin_sales) limit 5"
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (KYLIN-5145) Query Failed:wrong number of arguments while executing SQL
[
https://issues.apache.org/jira/browse/KYLIN-5145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17469160#comment-17469160
]
Yaqian Zhang edited comment on KYLIN-5145 at 1/5/22, 9:54 AM:
--
[~qingquan]
{code:java}
Skip spark which not owned by kylin. SPARK_HOME is
/opt/kylin/spark-3.1.1-bin-hadoop3.2 and KYLIN_HOME is /opt/kylin.
{code}
The reason for this message is that your spark is placed in $kylin_ Home
directory, but it has not been renamed
{code:java}
spark
{code}
, so kylin will not do some jar package replacement for this spark. You can
rename "spark-3.1.1-bin-hadoop3.2" to "spark" and try again.
was (Author: zhangyaqian):
[~qingquan] `Skip spark which not owned by kylin. SPARK_HOME is
/opt/kylin/spark-3.1.1-bin-hadoop3.2 and KYLIN_HOME is /opt/kylin`. The reason
for this message is that your spark is placed in $kylin_ Home directory, but it
has not been renamed `spark`, so kylin will not do some jar package replacement
for this spark. You can rename `spark-3.1.1-bin-hadoop3.2` to `spark` and try
again.
> Query Failed:wrong number of arguments while executing SQL
> --
>
> Key: KYLIN-5145
> URL: https://issues.apache.org/jira/browse/KYLIN-5145
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v4.0.0
> Environment: spark3.1.1
> hadoop3.1.3
> hive:3.1.2
> kylin4.0.0-spark3
>Reporter: qingquanzhang
>Priority: Major
> Attachments: find-spark-dependency.sh, kylin.log
>
>
> An error occurred when I initiated the query:
> java.lang.IllegalArgumentException: wrong number of arguments while executing
> SQL: "select * from (select count(*) from kylin_sales) limit 5"
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (KYLIN-5145) Query Failed:wrong number of arguments while executing SQL
[ https://issues.apache.org/jira/browse/KYLIN-5145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17469160#comment-17469160 ] Yaqian Zhang commented on KYLIN-5145: - [~qingquan] `Skip spark which not owned by kylin. SPARK_HOME is /opt/kylin/spark-3.1.1-bin-hadoop3.2 and KYLIN_HOME is /opt/kylin`. The reason for this message is that your spark is placed in $kylin_ Home directory, but it has not been renamed `spark`, so kylin will not do some jar package replacement for this spark. You can rename `spark-3.1.1-bin-hadoop3.2` to `spark` and try again. > Query Failed:wrong number of arguments while executing SQL > -- > > Key: KYLIN-5145 > URL: https://issues.apache.org/jira/browse/KYLIN-5145 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v4.0.0 > Environment: spark3.1.1 > hadoop3.1.3 > hive:3.1.2 > kylin4.0.0-spark3 >Reporter: qingquanzhang >Priority: Major > Attachments: find-spark-dependency.sh, kylin.log > > > An error occurred when I initiated the query: > java.lang.IllegalArgumentException: wrong number of arguments while executing > SQL: "select * from (select count(*) from kylin_sales) limit 5" -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5145) Query Failed:wrong number of arguments while executing SQL
[ https://issues.apache.org/jira/browse/KYLIN-5145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17468338#comment-17468338 ] Yaqian Zhang commented on KYLIN-5145: - Hi [~qingquan]: Is there more error information in kylin.log? > Query Failed:wrong number of arguments while executing SQL > -- > > Key: KYLIN-5145 > URL: https://issues.apache.org/jira/browse/KYLIN-5145 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v4.0.0 > Environment: spark3.1.1 > hadoop3.1.3 > hive:3.1.2 > kylin4.0.0-spark3 >Reporter: qingquanzhang >Priority: Major > Attachments: kylin.log > > > An error occurred when I initiated the query: > java.lang.IllegalArgumentException: wrong number of arguments while executing > SQL: "select * from (select count(*) from kylin_sales) limit 5" -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (KYLIN-5097) modify hive dict table format from textfile to orc
[ https://issues.apache.org/jira/browse/KYLIN-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-5097: Fix Version/s: v3.1.3 > modify hive dict table format from textfile to orc > -- > > Key: KYLIN-5097 > URL: https://issues.apache.org/jira/browse/KYLIN-5097 > Project: Kylin > Issue Type: Bug >Affects Versions: v3.1.1, v3.1.2, v3.1.3 >Reporter: fengpod >Priority: Minor > Fix For: v3.1.3 > > > Textfile format end with ‘\n’ by default， when the data itself contains ‘\n’, > dict table may appear dirty data。 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (KYLIN-5097) modify hive dict table format from textfile to orc
[ https://issues.apache.org/jira/browse/KYLIN-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-5097: Affects Version/s: (was: v3.1.3) > modify hive dict table format from textfile to orc > -- > > Key: KYLIN-5097 > URL: https://issues.apache.org/jira/browse/KYLIN-5097 > Project: Kylin > Issue Type: Bug >Affects Versions: v3.1.1, v3.1.2 >Reporter: fengpod >Priority: Minor > Fix For: v3.1.3 > > > Textfile format end with ‘\n’ by default， when the data itself contains ‘\n’, > dict table may appear dirty data。 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5144) Apache Calcite Avatica is affected from log4j CVE-2021-44228.
[ https://issues.apache.org/jira/browse/KYLIN-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464954#comment-17464954 ] Yaqian Zhang commented on KYLIN-5144: - Hi [~sonuSINGH], sorry for late. Kylin did package avatica's jar package, but kylin did not introduce the dependency of log4j2 by loading avatica's jar package. > Apache Calcite Avatica is affected from log4j CVE-2021-44228. > - > > Key: KYLIN-5144 > URL: https://issues.apache.org/jira/browse/KYLIN-5144 > Project: Kylin > Issue Type: Improvement > Components: Others >Affects Versions: v3.1.3 >Reporter: Sonu Kumar Singh >Assignee: Sonu Kumar Singh >Priority: Major > Fix For: v3.1.4 > > Attachments: image-2021-12-21-15-08-37-651.png, screenshot-1.png > > > As per Apache Blogs (https://blogs.apache.org/security/entry/cve-2021-44228), > Apache Calcite Avatica is affected from log4j CVE-2021-44228 and there is a > suggestion from the Apache Calcite team to upgrade Apache Calcite Avatica to > 1.20.0 (https://lists.apache.org/thread/3vn3j4fmr2dn9s0x1604pdxz7x4fo8wz) > !image-2021-12-21-15-08-37-651.png|thumbnail! -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5144) Apache Calcite Avatica is affected from log4j CVE-2021-44228.
[ https://issues.apache.org/jira/browse/KYLIN-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464315#comment-17464315 ] Yaqian Zhang commented on KYLIN-5144: - Hi [~sonuSINGH]: I discussed this issue with other developers in the community. Finally, we thought that since we only referenced avatica in kylin's POM, we did not package avatica and its reference components, and did not use avatica's log related functions, kylin's reference to avatica will not introduce the security vulnerability of log4j into kylin. In order to maintain the stability of kylin function as much as possible, we will not consider upgrading avatica for the time being. It's a pity you can't raise this PR, but you can use the upgraded avatica on your local environment first. > Apache Calcite Avatica is affected from log4j CVE-2021-44228. > - > > Key: KYLIN-5144 > URL: https://issues.apache.org/jira/browse/KYLIN-5144 > Project: Kylin > Issue Type: Improvement > Components: Others >Affects Versions: v3.1.3 >Reporter: Sonu Kumar Singh >Assignee: Sonu Kumar Singh >Priority: Major > Fix For: v3.1.4 > > Attachments: image-2021-12-21-15-08-37-651.png > > > As per Apache Blogs (https://blogs.apache.org/security/entry/cve-2021-44228), > Apache Calcite Avatica is affected from log4j CVE-2021-44228 and there is a > suggestion from the Apache Calcite team to upgrade Apache Calcite Avatica to > 1.20.0 (https://lists.apache.org/thread/3vn3j4fmr2dn9s0x1604pdxz7x4fo8wz) > !image-2021-12-21-15-08-37-651.png|thumbnail! -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5144) Apache Calcite Avatica is affected from log4j CVE-2021-44228.
[ https://issues.apache.org/jira/browse/KYLIN-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17463567#comment-17463567 ] Yaqian Zhang commented on KYLIN-5144: - Hi [~sonuSINGH]: Thank you for your report! We will try to upgrade Apache Calcite Avatica to 1.20.0 in Apache Kylin. Since this may cause some compatibility problems, so we need to evaluate and test it for some time before releasing the version. > Apache Calcite Avatica is affected from log4j CVE-2021-44228. > - > > Key: KYLIN-5144 > URL: https://issues.apache.org/jira/browse/KYLIN-5144 > Project: Kylin > Issue Type: Improvement > Components: Others >Affects Versions: v3.1.3 >Reporter: Sonu Kumar Singh >Assignee: Sonu Kumar Singh >Priority: Major > Fix For: v3.1.4 > > Attachments: image-2021-12-21-15-08-37-651.png > > > As per Apache Blogs (https://blogs.apache.org/security/entry/cve-2021-44228), > Apache Calcite Avatica is affected from log4j CVE-2021-44228 and there is a > suggestion from the Apache Calcite team to upgrade Apache Calcite Avatica to > 1.20.0 (https://lists.apache.org/thread/3vn3j4fmr2dn9s0x1604pdxz7x4fo8wz) > !image-2021-12-21-15-08-37-651.png|thumbnail! -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5143) 关于org.apache.kylin.shaded.com.google.common启动报错问题
[
https://issues.apache.org/jira/browse/KYLIN-5143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17462355#comment-17462355
]
Yaqian Zhang commented on KYLIN-5143:
-
What‘s the mean that `hadoop的节点上是部署成功的` and `单独部署`?
What's the difference between them? Are their versions the same?
> 关于org.apache.kylin.shaded.com.google.common启动报错问题
> -
>
> Key: KYLIN-5143
> URL: https://issues.apache.org/jira/browse/KYLIN-5143
> Project: Kylin
> Issue Type: Bug
> Environment: hadoop 2.9.1
> hive 2.3.3
> hbase 1.4.8
> JDK 1.8.0_191
> OS CentOS 7.6.1810
> zookeeper 3.4.12
> kylin 3.1.2
>Reporter: tieguanyin
>Priority: Blocker
> Attachments: kylin问题.md
>
>
> kylin.out信息：
> 1 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option
> MaxPermSize=512M; support was removed in 8.0
> 2 SLF4J: Class path contains multiple SLF4J bindings.
> 3 SLF4J: Found binding in
> [jar:file:/hbase/hbase-1.4.8/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 4 SLF4J: Found binding in
> [jar:file:/hadoop/hadoop-2.9.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 5 SLF4J: Found binding in
> [jar:file:/hive/apache-hive-2.3.3-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 6 SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> 7 SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 8 usage: java org.apache.catalina.startup.Catalina [ -config
> \{pathname} ] [ -nonaming ] \{ -help | start | stop }
> 9 log4j:WARN No appenders could be found for logger
> (net.sf.ehcache.pool.sizeof.filter.AnnotationSizeOfFilter).
> 10 log4j:WARN Please initialize the log4j system properly.
> 11 log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
> for more info.
> 12 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option
> MaxPermSize=512M; support was removed in 8.0
> 13 SLF4J: Class path contains multiple SLF4J bindings.
> 14 SLF4J: Found binding in
> [jar:file:/hbase/hbase-1.4.8/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 15 SLF4J: Found binding in
> [jar:file:/hadoop/hadoop-2.9.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 16 SLF4J: Found binding in
> [jar:file:/hive/apache-hive-2.3.3-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 17 SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> 18 SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 19 usage: java org.apache.catalina.startup.Catalina [ -config
> \{pathname} ] [ -nonaming ] \{ -help | start | stop }
> 20 log4j:WARN No appenders could be found for logger
> (net.sf.ehcache.pool.sizeof.filter.AnnotationSizeOfFilter).
> 21 log4j:WARN Please initialize the log4j system properly.
> 22 log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
> for more info.
> 23 Exception in thread "Thread-11" Exception in thread "Thread-10"
> java.lang.NoClassDefFoundError:
> org/apache/curator/framework/imps/GetDataBuilderImpl
> 24 at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.getData(CuratorFrameworkImpl.java:377)
> 25 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.peekLockInternal(ZookeeperDistributedLock.java:220)
> 26 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.peekLock(ZookeeperDistributedLock.java:208)
> 27 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.unlock(ZookeeperDistributedLock.java:245)
> 28 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.unlockJobEngine(ZookeeperDistributedLock.java:359)
> 29 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperJobLock.unlockJobEngine(ZookeeperJobLock.java:91)
> 30 at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler.shutdown(DefaultScheduler.java:192)
> 31 at
> org.apache.kylin.rest.service.JobService$1.run(JobService.java:146)
> 32 at java.lang.Thread.run(Thread.java:748)
> 33 Caused by: java.lang.ClassNotFoundException:
> org.apache.curator.framework.imps.GetDataBuilderImpl
> 34 at
> org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1951)
> 35 at
> org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1794)
> 36 ... 9 more
> 37 java.lang.NoClassDefFoundError:
> org/apache/kylin/shaded/com/google/common/cache/RemovalCause
> 38 at
[jira] [Commented] (KYLIN-5143) 关于org.apache.kylin.shaded.com.google.common启动报错问题
[
https://issues.apache.org/jira/browse/KYLIN-5143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17462352#comment-17462352
]
Yaqian Zhang commented on KYLIN-5143:
-
It seems that the Hadoop environment you are using is Apache Hadoop. We
recommend using HDP or CDH.
> 关于org.apache.kylin.shaded.com.google.common启动报错问题
> -
>
> Key: KYLIN-5143
> URL: https://issues.apache.org/jira/browse/KYLIN-5143
> Project: Kylin
> Issue Type: Bug
> Environment: hadoop 2.9.1
> hive 2.3.3
> hbase 1.4.8
> JDK 1.8.0_191
> OS CentOS 7.6.1810
> zookeeper 3.4.12
> kylin 3.1.2
>Reporter: tieguanyin
>Priority: Blocker
> Attachments: kylin问题.md
>
>
> kylin.out信息：
> 1 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option
> MaxPermSize=512M; support was removed in 8.0
> 2 SLF4J: Class path contains multiple SLF4J bindings.
> 3 SLF4J: Found binding in
> [jar:file:/hbase/hbase-1.4.8/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 4 SLF4J: Found binding in
> [jar:file:/hadoop/hadoop-2.9.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 5 SLF4J: Found binding in
> [jar:file:/hive/apache-hive-2.3.3-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 6 SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> 7 SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 8 usage: java org.apache.catalina.startup.Catalina [ -config
> \{pathname} ] [ -nonaming ] \{ -help | start | stop }
> 9 log4j:WARN No appenders could be found for logger
> (net.sf.ehcache.pool.sizeof.filter.AnnotationSizeOfFilter).
> 10 log4j:WARN Please initialize the log4j system properly.
> 11 log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
> for more info.
> 12 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option
> MaxPermSize=512M; support was removed in 8.0
> 13 SLF4J: Class path contains multiple SLF4J bindings.
> 14 SLF4J: Found binding in
> [jar:file:/hbase/hbase-1.4.8/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 15 SLF4J: Found binding in
> [jar:file:/hadoop/hadoop-2.9.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 16 SLF4J: Found binding in
> [jar:file:/hive/apache-hive-2.3.3-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 17 SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> 18 SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 19 usage: java org.apache.catalina.startup.Catalina [ -config
> \{pathname} ] [ -nonaming ] \{ -help | start | stop }
> 20 log4j:WARN No appenders could be found for logger
> (net.sf.ehcache.pool.sizeof.filter.AnnotationSizeOfFilter).
> 21 log4j:WARN Please initialize the log4j system properly.
> 22 log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
> for more info.
> 23 Exception in thread "Thread-11" Exception in thread "Thread-10"
> java.lang.NoClassDefFoundError:
> org/apache/curator/framework/imps/GetDataBuilderImpl
> 24 at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.getData(CuratorFrameworkImpl.java:377)
> 25 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.peekLockInternal(ZookeeperDistributedLock.java:220)
> 26 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.peekLock(ZookeeperDistributedLock.java:208)
> 27 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.unlock(ZookeeperDistributedLock.java:245)
> 28 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.unlockJobEngine(ZookeeperDistributedLock.java:359)
> 29 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperJobLock.unlockJobEngine(ZookeeperJobLock.java:91)
> 30 at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler.shutdown(DefaultScheduler.java:192)
> 31 at
> org.apache.kylin.rest.service.JobService$1.run(JobService.java:146)
> 32 at java.lang.Thread.run(Thread.java:748)
> 33 Caused by: java.lang.ClassNotFoundException:
> org.apache.curator.framework.imps.GetDataBuilderImpl
> 34 at
> org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1951)
> 35 at
> org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1794)
> 36 ... 9 more
> 37 java.lang.NoClassDefFoundError:
> org/apache/kylin/shaded/com/google/common/cache/RemovalCause
> 38 at
> org.apache.ky
[jira] [Comment Edited] (KYLIN-5143) 关于org.apache.kylin.shaded.com.google.common启动报错问题
[
https://issues.apache.org/jira/browse/KYLIN-5143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17462351#comment-17462351
]
Yaqian Zhang edited comment on KYLIN-5143 at 12/20/21, 1:53 AM:
Hi [~tieguanyin]:
Can you check whether there is `kylin-shaded-guava-3.1.0.jar` under directory
`${KYLIN_HOME}/tomcat/webapps/kylin/WEB-INF/lib/` ?
was (Author: zhangyaqian):
Hi [~tieguanyin]:
Can you check whether there is `kylin-shaded-guava-3.1.0.jar` under directory
`${KYLIN_HOME}/tomcat/webapps/kylin/WEB-INF/lib/`
> 关于org.apache.kylin.shaded.com.google.common启动报错问题
> -
>
> Key: KYLIN-5143
> URL: https://issues.apache.org/jira/browse/KYLIN-5143
> Project: Kylin
> Issue Type: Bug
> Environment: hadoop 2.9.1
> hive 2.3.3
> hbase 1.4.8
> JDK 1.8.0_191
> OS CentOS 7.6.1810
> zookeeper 3.4.12
> kylin 3.1.2
>Reporter: tieguanyin
>Priority: Blocker
> Attachments: kylin问题.md
>
>
> kylin.out信息：
> 1 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option
> MaxPermSize=512M; support was removed in 8.0
> 2 SLF4J: Class path contains multiple SLF4J bindings.
> 3 SLF4J: Found binding in
> [jar:file:/hbase/hbase-1.4.8/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 4 SLF4J: Found binding in
> [jar:file:/hadoop/hadoop-2.9.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 5 SLF4J: Found binding in
> [jar:file:/hive/apache-hive-2.3.3-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 6 SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> 7 SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 8 usage: java org.apache.catalina.startup.Catalina [ -config
> \{pathname} ] [ -nonaming ] \{ -help | start | stop }
> 9 log4j:WARN No appenders could be found for logger
> (net.sf.ehcache.pool.sizeof.filter.AnnotationSizeOfFilter).
> 10 log4j:WARN Please initialize the log4j system properly.
> 11 log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
> for more info.
> 12 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option
> MaxPermSize=512M; support was removed in 8.0
> 13 SLF4J: Class path contains multiple SLF4J bindings.
> 14 SLF4J: Found binding in
> [jar:file:/hbase/hbase-1.4.8/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 15 SLF4J: Found binding in
> [jar:file:/hadoop/hadoop-2.9.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 16 SLF4J: Found binding in
> [jar:file:/hive/apache-hive-2.3.3-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 17 SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> 18 SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 19 usage: java org.apache.catalina.startup.Catalina [ -config
> \{pathname} ] [ -nonaming ] \{ -help | start | stop }
> 20 log4j:WARN No appenders could be found for logger
> (net.sf.ehcache.pool.sizeof.filter.AnnotationSizeOfFilter).
> 21 log4j:WARN Please initialize the log4j system properly.
> 22 log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
> for more info.
> 23 Exception in thread "Thread-11" Exception in thread "Thread-10"
> java.lang.NoClassDefFoundError:
> org/apache/curator/framework/imps/GetDataBuilderImpl
> 24 at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.getData(CuratorFrameworkImpl.java:377)
> 25 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.peekLockInternal(ZookeeperDistributedLock.java:220)
> 26 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.peekLock(ZookeeperDistributedLock.java:208)
> 27 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.unlock(ZookeeperDistributedLock.java:245)
> 28 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.unlockJobEngine(ZookeeperDistributedLock.java:359)
> 29 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperJobLock.unlockJobEngine(ZookeeperJobLock.java:91)
> 30 at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler.shutdown(DefaultScheduler.java:192)
> 31 at
> org.apache.kylin.rest.service.JobService$1.run(JobService.java:146)
> 32 at java.lang.Thread.run(Thread.java:748)
> 33 Caused by: java.lang.ClassNotFoundException:
> org.apache.curator.framework.imps.GetDataBuilderImpl
> 34 at
> org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1951)
> 35
[jira] [Commented] (KYLIN-5143) 关于org.apache.kylin.shaded.com.google.common启动报错问题
[
https://issues.apache.org/jira/browse/KYLIN-5143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17462351#comment-17462351
]
Yaqian Zhang commented on KYLIN-5143:
-
Hi [~tieguanyin]:
Can you check whether there is `kylin-shaded-guava-3.1.0.jar` under directory
`${KYLIN_HOME}/tomcat/webapps/kylin/WEB-INF/lib/`
> 关于org.apache.kylin.shaded.com.google.common启动报错问题
> -
>
> Key: KYLIN-5143
> URL: https://issues.apache.org/jira/browse/KYLIN-5143
> Project: Kylin
> Issue Type: Bug
> Environment: hadoop 2.9.1
> hive 2.3.3
> hbase 1.4.8
> JDK 1.8.0_191
> OS CentOS 7.6.1810
> zookeeper 3.4.12
> kylin 3.1.2
>Reporter: tieguanyin
>Priority: Blocker
> Attachments: kylin问题.md
>
>
> kylin.out信息：
> 1 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option
> MaxPermSize=512M; support was removed in 8.0
> 2 SLF4J: Class path contains multiple SLF4J bindings.
> 3 SLF4J: Found binding in
> [jar:file:/hbase/hbase-1.4.8/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 4 SLF4J: Found binding in
> [jar:file:/hadoop/hadoop-2.9.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 5 SLF4J: Found binding in
> [jar:file:/hive/apache-hive-2.3.3-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 6 SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> 7 SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 8 usage: java org.apache.catalina.startup.Catalina [ -config
> \{pathname} ] [ -nonaming ] \{ -help | start | stop }
> 9 log4j:WARN No appenders could be found for logger
> (net.sf.ehcache.pool.sizeof.filter.AnnotationSizeOfFilter).
> 10 log4j:WARN Please initialize the log4j system properly.
> 11 log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
> for more info.
> 12 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option
> MaxPermSize=512M; support was removed in 8.0
> 13 SLF4J: Class path contains multiple SLF4J bindings.
> 14 SLF4J: Found binding in
> [jar:file:/hbase/hbase-1.4.8/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 15 SLF4J: Found binding in
> [jar:file:/hadoop/hadoop-2.9.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 16 SLF4J: Found binding in
> [jar:file:/hive/apache-hive-2.3.3-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> 17 SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> 18 SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 19 usage: java org.apache.catalina.startup.Catalina [ -config
> \{pathname} ] [ -nonaming ] \{ -help | start | stop }
> 20 log4j:WARN No appenders could be found for logger
> (net.sf.ehcache.pool.sizeof.filter.AnnotationSizeOfFilter).
> 21 log4j:WARN Please initialize the log4j system properly.
> 22 log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
> for more info.
> 23 Exception in thread "Thread-11" Exception in thread "Thread-10"
> java.lang.NoClassDefFoundError:
> org/apache/curator/framework/imps/GetDataBuilderImpl
> 24 at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.getData(CuratorFrameworkImpl.java:377)
> 25 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.peekLockInternal(ZookeeperDistributedLock.java:220)
> 26 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.peekLock(ZookeeperDistributedLock.java:208)
> 27 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.unlock(ZookeeperDistributedLock.java:245)
> 28 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.unlockJobEngine(ZookeeperDistributedLock.java:359)
> 29 at
> org.apache.kylin.job.lock.zookeeper.ZookeeperJobLock.unlockJobEngine(ZookeeperJobLock.java:91)
> 30 at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler.shutdown(DefaultScheduler.java:192)
> 31 at
> org.apache.kylin.rest.service.JobService$1.run(JobService.java:146)
> 32 at java.lang.Thread.run(Thread.java:748)
> 33 Caused by: java.lang.ClassNotFoundException:
> org.apache.curator.framework.imps.GetDataBuilderImpl
> 34 at
> org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1951)
> 35 at
> org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1794)
> 36 ... 9 more
> 37 java.lang.NoClassDefFoundError:
> org/apache/kylin/shaded/com/google/common/cache/Rem
[jira] [Resolved] (KYLIN-5142) Apache Log4j2report a bug： remote code execution vulnerability was found
[ https://issues.apache.org/jira/browse/KYLIN-5142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang resolved KYLIN-5142. - Fix Version/s: document Assignee: Yaqian Zhang Resolution: Resolved > Apache Log4j2report a bug： remote code execution vulnerability was found > > > Key: KYLIN-5142 > URL: https://issues.apache.org/jira/browse/KYLIN-5142 > Project: Kylin > Issue Type: Bug >Reporter: mzz >Assignee: Yaqian Zhang >Priority: Blocker > Fix For: document > > > Does this bug affect kylin ，and how to solve it？ > [~shaofengshi] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5142) Apache Log4j2report a bug： remote code execution vulnerability was found
[ https://issues.apache.org/jira/browse/KYLIN-5142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17458187#comment-17458187 ] Yaqian Zhang commented on KYLIN-5142: - Hi: You can refer to this email from apache kylin team: https://lists.apache.org/thread/lbgdcnpymjyckxllcyv26yxtfzzwtfwl > Apache Log4j2report a bug： remote code execution vulnerability was found > > > Key: KYLIN-5142 > URL: https://issues.apache.org/jira/browse/KYLIN-5142 > Project: Kylin > Issue Type: Bug >Reporter: mzz >Priority: Blocker > > Does this bug affect kylin ，and how to solve it？ > [~shaofengshi] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (KYLIN-5136) Why there is no loading the Dkylin.spark.dependency jar package when I started kylin.
[ https://issues.apache.org/jira/browse/KYLIN-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang closed KYLIN-5136. --- Resolution: Duplicate > Why there is no loading the Dkylin.spark.dependency jar package when I > started kylin. > - > > Key: KYLIN-5136 > URL: https://issues.apache.org/jira/browse/KYLIN-5136 > Project: Kylin > Issue Type: Bug >Affects Versions: v4.0.0 >Reporter: xbchao >Priority: Major > Attachments: image-2021-11-25-14-04-44-708.png > > > when i started kylin4.0,the command is : // lack of spark jars > /etc/alternatives/jre/bin/java -Xms1024M -Xmx4096M -Xss1024K > -XX:MaxPermSize=512M -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps > -Xloggc:/opt/apps/kylin/logs/kylin.gc.%p -XX:+UseGCLogFileRotation > -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M > -Dlog4j.configuration=file:/opt/apps/kylin/conf/kylin-server-log4j.properties > -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager > -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true > -Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true > -Djava.endorsed.dirs=/opt/apps/kylin/bin/../tomcat/endorsed > -Dcatalina.base=/opt/apps/kylin/bin/../tomcat > -Dcatalina.home=/opt/apps/kylin/bin/../tomcat > -Djava.io.tmpdir=/opt/apps/kylin/bin/../tomcat/temp > -Dkylin.hive.dependency=/usr/lib/hive/conf:/usr/lib/hive/lib/arrow-vector-0.15.1.jar:/usr/lib/hive/lib/commons-logging-1.2.jar:/usr/lib/hive/lib/avro-1.7.7.jar:/usr/lib/hive/lib/hive-exec-2.3.7-amzn-4.jar:/usr/lib/hive/lib/avro-ipc-1.7.7-tests.jar:/usr/lib/hive/lib/hive-jdbc-2.3.7-amzn-4.jar:/usr/lib/hive/lib/hive-jdbc-handler-2.3.7-amzn-4.jar:/usr/lib/hive/lib/avro-ipc-1.7.7.jar:/usr/lib/hive/lib/hive-jdbc-handler.jar:/usr/lib/hive/lib/hive-metastore-2.3.7-amzn-4.jar:/usr/lib/hive/lib/avro-mapred-1.7.7-hadoop2.jar:/usr/lib/hive/lib/hive-serde-2.3.7-amzn-4.jar:/usr/lib/hive/lib/apache-jstl-9.3.27.v20190418.jar:/usr/lib/hive/lib/hive-service-2.3.7-amzn-4.jar:/usr/lib/hive/lib/arrow-format-0.15.1.jar:/usr/lib/hive/lib/hive-service-rpc-2.3.7-amzn-4.jar:/usr/lib/hive/lib/bonecp-0.8.0.RELEASE.jar:/usr/lib/hive/lib/hive-service-rpc.jar:/usr/lib/hive/lib/commons-math-2.2.jar:/usr/lib/hive/lib/libfb303-0.9.3.jar:/usr/lib/hive/lib/commons-math3-3.6.1.jar:/usr/lib/hive/lib/libthrift-0.9.3.jar:/usr/lib/hive/lib/bytebuffer-collections-0.2.5.jar:/usr/lib/hive/lib/log4j-1.2-api-2.6.2.jar:/usr/lib/hive/lib/commons-pool-1.5.4.jar:/usr/lib/hive/lib/log4j-api-2.6.2.jar:/usr/lib/hive/lib/log4j-core-2.6.2.jar:/usr/lib/hive/lib/commons-pool2-2.2.jar:/usr/lib/hive/lib/log4j-jul-2.5.jar:/usr/lib/hive/lib/commons-vfs2-2.0.jar:/usr/lib/hive/lib/log4j-web-2.6.2.jar:/usr/lib/hive/lib/compress-lzf-1.0.3.jar:/usr/lib/hive/lib/HikariCP-2.5.1.jar:/usr/lib/hive/lib/config-magic-0.9.jar:/usr/lib/hive/lib/JavaEWAH-0.3.2.jar:/usr/lib/hive/lib/RoaringBitmap-0.7.45.jar:/usr/lib/hive/lib/curator-client-2.7.1.jar:/usr/lib/hive/lib/ST4-4.0.4.jar:/usr/lib/hive/lib/chill-java-0.9.3.jar:/usr/lib/hive/lib/accumulo-core-1.6.0.jar:/usr/lib/hive/lib/chill_2.11-0.9.3.jar:/usr/lib/hive/lib/accumulo-fate-1.6.0.jar:/usr/lib/hive/lib/classmate-1.0.0.jar:/usr/lib/hive/lib/accumulo-start-1.6.0.jar:/usr/lib/hive/lib/commons-cli-1.2.jar:/usr/lib/hive/lib/accumulo-trace-1.6.0.jar:/usr/lib/hive/lib/commons-codec-1.4.jar:/usr/lib/hive/lib/activation-1.1.1.jar:/usr/lib/hive/lib/commons-collections-3.2.2.jar:/usr/lib/hive/lib/aether-api-0.9.0.M2.jar:/usr/lib/hive/lib/hive-cli.jar:/usr/lib/hive/lib/aether-connector-file-0.9.0.M2.jar:/usr/lib/hive/lib/arrow-memory-0.15.1.jar:/usr/lib/hive/lib/aether-connector-okhttp-0.0.9.jar:/usr/lib/hive/lib/commons-compiler-2.7.6.jar:/usr/lib/hive/lib/aether-impl-0.9.0.M2.jar:/usr/lib/hive/lib/commons-compress-1.9.jar:/usr/lib/hive/lib/aether-spi-0.9.0.M2.jar:/usr/lib/hive/lib/commons-crypto-1.1.0.jar:/usr/lib/hive/lib/aether-util-0.9.0.M2.jar:/usr/lib/hive/lib/commons-dbcp-1.4.jar:/usr/lib/hive/lib/aircompressor-0.8.jar:/usr/lib/hive/lib/curator-framework-2.7.1.jar:/usr/lib/hive/lib/airline-0.7.jar:/usr/lib/hive/lib/curator-recipes-2.7.1.jar:/usr/lib/hive/lib/ant-1.6.5.jar:/usr/lib/hive/lib/datanucleus-core-4.1.17.jar:/usr/lib/hive/lib/ant-1.9.1.jar:/usr/lib/hive/lib/commons-dbcp2-2.0.1.jar:/usr/lib/hive/lib/ant-launcher-1.9.1.jar:/usr/lib/hive/lib/commons-el-1.0.jar:/usr/lib/hive/lib/antlr-runtime-3.5.2.jar:/usr/lib/hive/lib/commons-io-2.4.jar:/usr/lib/hive/lib/antlr4-runtime-4.5.jar:/usr/lib/hive/lib/commons-lang-2.6.jar:/usr/lib/hive/lib/commons-lang3-3.1.jar:/usr/lib/hive/lib/apache-el-8.0.33.jar:/usr/lib/hive/lib/asm-3.1.jar:/usr/lib/hive/lib/disruptor-3.3.0.jar:/usr/lib/hive/lib/asm-6.0.jar:/usr/lib/hive/lib/datanucleus-rdbms-4.1.19.jar:/usr/lib/hive/lib/asm-commons-6.0.jar:/usr/lib/hive/lib/asm-tree-6.0.jar:
[jira] [Commented] (KYLIN-5134) Start kylin4.0,cache.PathChildrenCache:571 : java.lang.IllegalStateException: instance must be started before calling this method
[ https://issues.apache.org/jira/browse/KYLIN-5134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17452129#comment-17452129 ] Yaqian Zhang commented on KYLIN-5134: - Hi [~xbchao]： This error should be generated because the kylin service has not been started successfully or has been stopped. The real error should be printed out before this log. Can you provide the complete log file and environment information? > Start kylin4.0,cache.PathChildrenCache:571 : java.lang.IllegalStateException: > instance must be started before calling this method > - > > Key: KYLIN-5134 > URL: https://issues.apache.org/jira/browse/KYLIN-5134 > Project: Kylin > Issue Type: Bug > Components: Environment >Affects Versions: v4.0.0 >Reporter: xbchao >Priority: Major > > When i start kylin4.0,the kylin.log write: > 2021-11-24 03:09:21,660 ERROR [KylinServerTracker-0] > cache.PathChildrenCache:571 : > java.lang.IllegalStateException: instance must be started before calling this > method > at > org.apache.curator.shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:176) > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.getChildren(CuratorFrameworkImpl.java:391) > at > org.apache.curator.framework.recipes.cache.PathChildrenCache.refresh(PathChildrenCache.java:508) > at > org.apache.curator.framework.recipes.cache.RefreshOperation.invoke(RefreshOperation.java:35) > at > org.apache.curator.framework.recipes.cache.PathChildrenCache$9.run(PathChildrenCache.java:773) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > 2021-11-24 03:09:21,662 INFO [Thread-5] zookeeper.ZooKeeper:693 : Session: > 0x103da3a000f closed > 2021-11-24 03:09:21,662 INFO [localhost-startStop-1-EventThread] > zookeeper.ClientCnxn:522 : EventThread shut down for session: > 0x103da3a000f > 2021-11-24 03:09:21,667 INFO [Thread-5] curator.CuratorLeaderSelector:95 : > ip-kylinlocalhost:7070 is stopped > 2021-11-24 03:09:21,667 ERROR [Curator-LeaderSelector-0] > curator.CuratorLeaderSelector:107 : ip-kylinlocalhost:7070 was interrupted. > java.lang.InterruptedException: sleep interrupted > at java.lang.Thread.sleep(Native Method) > at > org.apache.kylin.job.impl.curator.CuratorLeaderSelector.takeLeadership(CuratorLeaderSelector.java:104) > at > org.apache.curator.framework.recipes.leader.LeaderSelector$WrappedListener.takeLeadership(LeaderSelector.java:559) > at > org.apache.curator.framework.recipes.leader.LeaderSelector.doWork(LeaderSelector.java:421) > at > org.apache.curator.framework.recipes.leader.LeaderSelector.doWorkLoop(LeaderSelector.java:466) > at > org.apache.curator.framework.recipes.leader.LeaderSelector.access$100(LeaderSelector.java:65) > at > org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:246) > at > org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:240) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5132) controller.TableController:199 : HIVE_STATS_JDBC_TIMEOUT java.lang.NoSuchFieldError: HIVE_STATS_JDBC_TIMEOUT
[ https://issues.apache.org/jira/browse/KYLIN-5132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17452127#comment-17452127 ] Yaqian Zhang commented on KYLIN-5132: - HI [~polarisc]： I noticed that you used cdh6. When deploying kylin4 with cdh6, you need to do some preparatory work according to this document: https://cwiki.apache.org/confluence/display/KYLIN/Deploy+Kylin+4+on+CDH+6 > controller.TableController:199 : HIVE_STATS_JDBC_TIMEOUT > java.lang.NoSuchFieldError: HIVE_STATS_JDBC_TIMEOUT > > > Key: KYLIN-5132 > URL: https://issues.apache.org/jira/browse/KYLIN-5132 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Affects Versions: v4.0.1 > Environment: CDH 6.3.2；Hadoop 3.0.0；Hive 2.1.1；Spark 2.4.0 >Reporter: Yuming Cao >Priority: Major > Attachments: QQ截图20211123180315.png, kylin (2)-1.log > > > I tried it twice in total: > For the first time, I added SPARK_HOME in /etc/profile , and then when I > started kylin, it prompted "Skip spark which not owned by kylin. SPARK_HOME > is /opt/cloudera/parcels/CDH/lib/spark and KYLIN_HOME is /opt/kylin. > Please download the correct version of Apache Spark, unzip it, rename it > to'spark' and put it in /opt/kylin directory. > Do not use the spark that comes with your hadoop environment." > So I deleted SPARK_HOME from the environment variable, then used > "$KYLIN_HOME/bin/download-spark.sh" to download > "spark-2.4.7-bin-hadoop2.7.tgz" and automatically unzip it under > $KYLIN_HOME/spark, and started kylin again, and it prompted "Failed to take > action" after I Load Table From Tree, and the log has the following error: > "ERROR [http-bio-7070-exec-6] controller.TableController:199: > HIVE_STATS_JDBC_TIMEOUT > java.lang.NoSuchFieldError: HIVE_STATS_JDBC_TIMEOUT". So what should I do > now, spark under $KYLIN_HOME needs to modify the configuration or how to do > it? > Attach the error part of the log -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5137) 关于jackson-datatype-joda-2.10.0.jar 问题
[ https://issues.apache.org/jira/browse/KYLIN-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17452120#comment-17452120 ] Yaqian Zhang commented on KYLIN-5137: - Hi [~tieguanyin]: The complete error stack will show where the error is caused by the code. Can you provide a complete error stack? > 关于jackson-datatype-joda-2.10.0.jar 问题 > - > > Key: KYLIN-5137 > URL: https://issues.apache.org/jira/browse/KYLIN-5137 > Project: Kylin > Issue Type: Bug > Components: Others >Affects Versions: v3.1.2 > Environment: hadoop 2.9.1 > hive 2.3.3 > hbase 1.4.8 > JDK 1.8.0_191 > OS CentOS 7.6.1810 > zookeeper3.4.12 >Reporter: tieguanyin >Priority: Blocker > > 报错原因：Caused by: java.lang.NoSuchMethodError: > org.joda.time.format.DateTimeFormatter.withZoneUTC()Lorg/joda/time/format/DateTimeFormatter > kylinV3.1.2没有引用jackson-datatype-joda-2.10.0.jar 包，为什么会有相关的方法未找到。源码全局搜索未找到此方法。 > 请问这个是在哪里引用的方法和排查方向:D感谢 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5139) Tomcat 404 error
[ https://issues.apache.org/jira/browse/KYLIN-5139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17451477#comment-17451477 ] Yaqian Zhang commented on KYLIN-5139: - Hi [~keshav11]： Is there anything output in file $KYLIN_HOME/logs/kylin.out ? > Tomcat 404 error > > > Key: KYLIN-5139 > URL: https://issues.apache.org/jira/browse/KYLIN-5139 > Project: Kylin > Issue Type: Bug >Reporter: Keshav >Priority: Major > Attachments: image-2021-11-30-08-54-26-649.png > > > We have tried setting up a new environment with Kylin 3.1.2 with HDP 3.1 but > end up getting Tomcat 404 error. There isn't much in Tomcat logs but access > log with 404 error message. > !image-2021-11-30-08-54-26-649.png! -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5138) kylin停止时报错
[ https://issues.apache.org/jira/browse/KYLIN-5138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17450278#comment-17450278 ] Yaqian Zhang commented on KYLIN-5138: - Hi: Did kylin run normally before running the stop command? It seems that kylin did not load the classes it needed correctly. What version of kylin do you use? > kylin停止时报错 > -- > > Key: KYLIN-5138 > URL: https://issues.apache.org/jira/browse/KYLIN-5138 > Project: Kylin > Issue Type: Bug >Reporter: tieguanyin >Priority: Blocker > > kylin停止时，kylin.out报出这样的信息： > Exception in thread "Thread-10" java.lang.NoClassDefFoundError: > org/apache/kylin/shaded/com/google/common/cache/RemovalCause > 659 at > org.apache.kylin.shaded.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3197) > 660 at > org.apache.kylin.shaded.com.google.common.cache.LocalCache.clear(LocalCache.java:4270) > 661 at > org.apache.kylin.shaded.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909) > 662 at org.apache.kylin.common.util.ZKUtil$2.run(ZKUtil.java:94) > 663 at java.lang.Thread.run(Thread.java:748) > 664 Caused by: java.lang.ClassNotFoundException: > org.apache.kylin.shaded.com.google.common.cache.RemovalCause > 665 at > org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1951) > 666 at > org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1794) > 667 ... 5 more > 668 Exception in thread "Thread-11" java.lang.NoClassDefFoundError: > org/apache/curator/framework/imps/DeleteBuilderImpl > 669 at > org.apache.curator.framework.imps.CuratorFrameworkImpl.delete(CuratorFrameworkImpl.java:361) > 670 at > org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.purgeLockInternal(ZookeeperDistributedLock.java:309) > 671 at > org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.unlockInternal(ZookeeperDistributedLock.java:288) > 672 at > org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.unlock(ZookeeperDistributedLock.java:259) > 673 at > org.apache.kylin.job.lock.zookeeper.ZookeeperDistributedLock.unlockJobEngine(ZookeeperDistributedLock.java:359) > 674 at > org.apache.kylin.job.lock.zookeeper.ZookeeperJobLock.unlockJobEngine(ZookeeperJobLock.java:91) > 675 at > org.apache.kylin.job.impl.threadpool.DefaultScheduler.shutdown(DefaultScheduler.java:188) > 676 at > org.apache.kylin.rest.service.JobService$1.run(JobService.java:149) > 677 at java.lang.Thread.run(Thread.java:748) > 678 Caused by: java.lang.ClassNotFoundException: > org.apache.curator.framework.imps.DeleteBuilderImpl > 679 at > org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1951) > 680 at > org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1794) > 681 ... 9 more > 希望大家看下是什么问题，有什么解决方向和思路，感谢 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (KYLIN-5131) java.lang.UnsupportedOperationException: API getTableSnapshots is not supported in Kylin 4.0
[ https://issues.apache.org/jira/browse/KYLIN-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang resolved KYLIN-5131. - Resolution: Fixed > java.lang.UnsupportedOperationException: API getTableSnapshots is not > supported in Kylin 4.0 > > > Key: KYLIN-5131 > URL: https://issues.apache.org/jira/browse/KYLIN-5131 > Project: Kylin > Issue Type: Bug > Components: Others >Affects Versions: v4.0.0 > Environment: CDH 6.3.2; Hadoop 3.0.0; Hive 2.1.1; Spark 2.4.0 >Reporter: Yuming Cao >Priority: Blocker > Fix For: v4.0.1 > > Attachments: kylin-1.out > > > I depolyed the kylin4.0, then loaded a table from tree in Data Source, there > was no changes in the monitor when i [Reload Table] the table， so i viewed > the log. > 16:46:09.653 [http-bio-7070-exec-3] ERROR > org.apache.kylin.rest.controller.BasicController - > java.lang.UnsupportedOperationException: API getTableSnapshots is not > supported in Kylin 4.0 . > at > org.apache.kylin.rest.controller.TableController.getTableSnapshots(TableController.java:227) > ~[kylin-server-base-4.0.0.jar:4.0.0] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (KYLIN-5131) java.lang.UnsupportedOperationException: API getTableSnapshots is not supported in Kylin 4.0
[ https://issues.apache.org/jira/browse/KYLIN-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-5131: Fix Version/s: v4.0.1 > java.lang.UnsupportedOperationException: API getTableSnapshots is not > supported in Kylin 4.0 > > > Key: KYLIN-5131 > URL: https://issues.apache.org/jira/browse/KYLIN-5131 > Project: Kylin > Issue Type: Bug > Components: Others >Affects Versions: v4.0.0 > Environment: CDH 6.3.2; Hadoop 3.0.0; Hive 2.1.1; Spark 2.4.0 >Reporter: Yuming Cao >Priority: Blocker > Fix For: v4.0.1 > > Attachments: kylin-1.out > > > I depolyed the kylin4.0, then loaded a table from tree in Data Source, there > was no changes in the monitor when i [Reload Table] the table， so i viewed > the log. > 16:46:09.653 [http-bio-7070-exec-3] ERROR > org.apache.kylin.rest.controller.BasicController - > java.lang.UnsupportedOperationException: API getTableSnapshots is not > supported in Kylin 4.0 . > at > org.apache.kylin.rest.controller.TableController.getTableSnapshots(TableController.java:227) > ~[kylin-server-base-4.0.0.jar:4.0.0] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5126) Build kylin 4.0, spark has not been able to submit to the yarn cluster
[
https://issues.apache.org/jira/browse/KYLIN-5126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17444923#comment-17444923
]
Yaqian Zhang commented on KYLIN-5126:
-
Hi [~xbchao]:
In the kylin 4 build job, the first step can only be performed using the spark
local mode. This has nothing to do with your configuration.
In the first step, the error may be due to the jar conflict. However, according
to the log you provided, it seems that kylin did not get the complete log,
resulting in no real error.
You can edit file "$KYLIN_HOME/conf/spark-driver-log4j.properties" and change
"log4j.rootLogger=INFO,hdfs" to "log4j.rootLogger=INFO,logFile". Then restart
kylin and resubmit the failed build job. The full log will then be output to
$KYLIN_HOME/logs/spark/${job-id}.log.
You can provide the complete log to us to analyze this problem.
> Build kylin 4.0, spark has not been able to submit to the yarn cluster
> --
>
> Key: KYLIN-5126
> URL: https://issues.apache.org/jira/browse/KYLIN-5126
> Project: Kylin
> Issue Type: Bug
>Reporter: xbchao
>Priority: Major
>
> When I built kylin 4.0, spark could not be submitted to the yarn cluster. The
> version used was apache-kylin-4.0.0-bin-spark2, which was deployed in the aws
> emr server.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (KYLIN-5123) API getTableSnapshots is not supported in Kylin 4.0
[ https://issues.apache.org/jira/browse/KYLIN-5123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17444920#comment-17444920 ] Yaqian Zhang edited comment on KYLIN-5123 at 11/17/21, 3:34 AM: Hi [~lixz]: This error should not be the real error that caused the cube build to fail. You can view the build log on kylin's monitor page , as shown below: !screenshot-1.png! You can download the complete log and provide it to us to analyze this problem. was (Author: zhangyaqian): Hi [~lixz]: This error should not be the real error that caused the cube build to fail. You can view the build log on kylin's monitor page , as shown below: !screenshot-1.png! > API getTableSnapshots is not supported in Kylin 4.0 > --- > > Key: KYLIN-5123 > URL: https://issues.apache.org/jira/browse/KYLIN-5123 > Project: Kylin > Issue Type: Bug > Components: Others >Affects Versions: v4.0.0 > Environment: HDP-3.1.5，hive-3.1.1，hadoop-3.0.0，spark-2.4.7 >Reporter: lixinzhao >Priority: Critical > Attachments: screenshot-1.png > > > build cube时异常，后台日志报错： > java.lang.UnsupportedOperationException: API getTableSnapshots is not > supported in Kylin 4.0 . > at > org.apache.kylin.rest.controller.TableController.getTableSnapshots(TableController.java:227) > ~[kylin-server-base-4.0.0.jar:4.0.0] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > ~[?:1.8.0_291] > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > ~[?:1.8.0_291] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > ~[?:1.8.0_291] > at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_291] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5123) API getTableSnapshots is not supported in Kylin 4.0
[ https://issues.apache.org/jira/browse/KYLIN-5123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17444920#comment-17444920 ] Yaqian Zhang commented on KYLIN-5123: - Hi [~lixz]: This error should not be the real error that caused the cube build to fail. You can view the build log on kylin's monitor page , as shown below: !screenshot-1.png! > API getTableSnapshots is not supported in Kylin 4.0 > --- > > Key: KYLIN-5123 > URL: https://issues.apache.org/jira/browse/KYLIN-5123 > Project: Kylin > Issue Type: Bug > Components: Others >Affects Versions: v4.0.0 > Environment: HDP-3.1.5，hive-3.1.1，hadoop-3.0.0，spark-2.4.7 >Reporter: lixinzhao >Priority: Critical > Attachments: screenshot-1.png > > > build cube时异常，后台日志报错： > java.lang.UnsupportedOperationException: API getTableSnapshots is not > supported in Kylin 4.0 . > at > org.apache.kylin.rest.controller.TableController.getTableSnapshots(TableController.java:227) > ~[kylin-server-base-4.0.0.jar:4.0.0] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > ~[?:1.8.0_291] > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > ~[?:1.8.0_291] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > ~[?:1.8.0_291] > at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_291] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (KYLIN-5123) API getTableSnapshots is not supported in Kylin 4.0
[ https://issues.apache.org/jira/browse/KYLIN-5123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-5123: Attachment: screenshot-1.png > API getTableSnapshots is not supported in Kylin 4.0 > --- > > Key: KYLIN-5123 > URL: https://issues.apache.org/jira/browse/KYLIN-5123 > Project: Kylin > Issue Type: Bug > Components: Others >Affects Versions: v4.0.0 > Environment: HDP-3.1.5，hive-3.1.1，hadoop-3.0.0，spark-2.4.7 >Reporter: lixinzhao >Priority: Critical > Attachments: screenshot-1.png > > > build cube时异常，后台日志报错： > java.lang.UnsupportedOperationException: API getTableSnapshots is not > supported in Kylin 4.0 . > at > org.apache.kylin.rest.controller.TableController.getTableSnapshots(TableController.java:227) > ~[kylin-server-base-4.0.0.jar:4.0.0] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > ~[?:1.8.0_291] > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > ~[?:1.8.0_291] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > ~[?:1.8.0_291] > at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_291] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5122) yarn资源充足但是却报错:Current available resource in cluster is not sufficient
[ https://issues.apache.org/jira/browse/KYLIN-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17444914#comment-17444914 ] Yaqian Zhang commented on KYLIN-5122: - Hi [~qingquan]： I didn't see the error report you said in your job log, but the error shown in the figure below. !screenshot-1.png! When you use kylin's built-in cube kylin_sales_cube to build, you will encounter this problem. Because kylin_sales_cube has configured 'kylin.engine.build-base-cuboid-enable=false' at the cube level. You can remove this configuration at the cube level and rebuild it. In addition, I noticed that "spark.executor.instances=10" is displayed in your log. You can set this parameter smaller in kylin.properties to avoid insufficient resources, such as "kylin.engine.spark-conf.spark.executor.instances=5". > yarn资源充足但是却报错:Current available resource in cluster is not sufficient > - > > Key: KYLIN-5122 > URL: https://issues.apache.org/jira/browse/KYLIN-5122 > Project: Kylin > Issue Type: Bug >Affects Versions: v4.0.0 >Reporter: qingquanzhang >Priority: Minor > Attachments: image-2021-11-12-18-00-35-884.png, > learn_kylin_866451fe23f04c049c9f53a8ed4aa1be01.log, screenshot-1.png > > > 我尝试调节了一些spark的参数,仍然不能解决问题. > 请教一下,应该调整哪些参数可能会解决这个问题? > !image-2021-11-12-18-00-35-884.png! -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (KYLIN-5122) yarn资源充足但是却报错:Current available resource in cluster is not sufficient
[ https://issues.apache.org/jira/browse/KYLIN-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-5122: Attachment: screenshot-1.png > yarn资源充足但是却报错:Current available resource in cluster is not sufficient > - > > Key: KYLIN-5122 > URL: https://issues.apache.org/jira/browse/KYLIN-5122 > Project: Kylin > Issue Type: Bug >Affects Versions: v4.0.0 >Reporter: qingquanzhang >Priority: Minor > Attachments: image-2021-11-12-18-00-35-884.png, > learn_kylin_866451fe23f04c049c9f53a8ed4aa1be01.log, screenshot-1.png > > > 我尝试调节了一些spark的参数,仍然不能解决问题. > 请教一下,应该调整哪些参数可能会解决这个问题? > !image-2021-11-12-18-00-35-884.png! -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KYLIN-5099) parquet file size is too big in kylin4 with spark3 than kylin3 with mr
[ https://issues.apache.org/jira/browse/KYLIN-5099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442036#comment-17442036 ] Yaqian Zhang commented on KYLIN-5099: - [~tanyao] If you set parameter "kylin.cube.cubeplanner.enabled" to true, kylin4 will pruning cuboids when you build the first segment. And the degree of pruning cuboids is controlled by parameter "kylin.cube.cubeplanner.expansion-threshold". The smaller the value of "kylin.cube.cubeplanner.expansion-threshold" you set, the more cuboids will be pruned. According to my test, when "kylin.cube.cubeplanner.expansion-threshold" be set to 2.5, the number of cuboids after pruning in kylin4 is roughly the same as that in kylin3. > parquet file size is too big in kylin4 with spark3 than kylin3 with mr > -- > > Key: KYLIN-5099 > URL: https://issues.apache.org/jira/browse/KYLIN-5099 > Project: Kylin > Issue Type: Bug >Affects Versions: v4.0.0 >Reporter: tanyao >Priority: Blocker > Attachments: image-2021-10-15-10-43-54-830.png, > image-2021-10-15-10-44-49-178.png, image-2021-10-18-10-00-34-440.png, > image-2021-10-18-10-02-07-998.png, image-2021-10-18-10-06-15-839.png, > screenshot-1.png > > > hi , > i am trying to use spark 3.1.1 as the build engine in kylin4.0, the hive > table has 200W+ rows with orc type, and there are 10 dimensions definded, the > original size is about 50M. > when i use kylin4.0 to build this cube ,the final parquet files size all > together is 1G+，that is to say , a single segment is about 1G+. However , i > use the same hive table data with the same cube model and dimensions , the > hbase segment size is just 100M+. In case of kylin4.0, all spark params are > set automatically by kylin > why this happened? And the building time in kylin4.0 is not faster then > kylin3.1 , even worse! both of them take about 10mins, i can not find the > benefits about kylin4.0 > !image-2021-10-15-10-43-54-830.png! > > !image-2021-10-15-10-44-49-178.png! > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (KYLIN-4502) Add alluxio dependence jar
[ https://issues.apache.org/jira/browse/KYLIN-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-4502: Fix Version/s: v4.0.2 (was: v4.0.1) > Add alluxio dependence jar > -- > > Key: KYLIN-4502 > URL: https://issues.apache.org/jira/browse/KYLIN-4502 > Project: Kylin > Issue Type: New Feature > Components: Spark Engine >Reporter: Harvey Yue >Assignee: Harvey Yue >Priority: Major > Fix For: v4.0.2 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (KYLIN-4964) Receiver consumer thread should be stoped while encounting unrecoverable error
[ https://issues.apache.org/jira/browse/KYLIN-4964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang resolved KYLIN-4964. - Resolution: Fixed > Receiver consumer thread should be stoped while encounting unrecoverable error > -- > > Key: KYLIN-4964 > URL: https://issues.apache.org/jira/browse/KYLIN-4964 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming >Affects Versions: v3.1.1 >Reporter: Xingxing Di >Assignee: Xingxing Di >Priority: Major > Fix For: v3.1.3 > > > org.apache.kylin.stream.core.storage.columnar.ColumnarMemoryStorePersister#persist > will catch exception and just log it, this behavior will cause critical > problem, like our case: there is no space left on device, `persist` method > failed for many times, we lost several hours of data. > Here is our solution which already tested: > # Throw IllegalStorageException while we cannot do the persist > # Then stop the consumer thread > # Add `consumer_thread_alive` in ConsumerStats for monitoring > # Also fix an another issue which will cause wrong result after restore from > checkpoint > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (KYLIN-5054) Kylin_System project and cubes create time is wrong (1970).
[ https://issues.apache.org/jira/browse/KYLIN-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang resolved KYLIN-5054. - Resolution: Fixed > Kylin_System project and cubes create time is wrong (1970). > --- > > Key: KYLIN-5054 > URL: https://issues.apache.org/jira/browse/KYLIN-5054 > Project: Kylin > Issue Type: Bug > Components: Others >Affects Versions: v3.1.2 >Reporter: Sonu Kumar Singh >Assignee: Sonu Kumar Singh >Priority: Minor > Fix For: v3.1.3 > > > Kylin_System's project and cubes create time is 1970, that is wrong. It > should be current time when user will setup the Kylin_System project. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (KYLIN-5032) Not allow change project for a cubeInstance.
[ https://issues.apache.org/jira/browse/KYLIN-5032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang resolved KYLIN-5032. - Resolution: Fixed > Not allow change project for a cubeInstance. > > > Key: KYLIN-5032 > URL: https://issues.apache.org/jira/browse/KYLIN-5032 > Project: Kylin > Issue Type: Improvement >Reporter: JiangYang >Assignee: JiangYang >Priority: Minor > Fix For: v3.1.3 > > > There are two project in Kylin: project1, project2. > Cube1 is in project1 now. > 1.Some one is in project2 (In KYLIN-UI) > 2.edit cube1 with url > [https:///kylin/cubes/edit/cube1|https://kylin.rno.corp.ebay.com/kylin/cubes/edit/PA_AUTO_TRACKING_GROUPS_SD_PROD_C_V2] > 3. then save, now cube1 is in project2. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (KYLIN-5053) Update the website to link to TravisCI and Github Actions instead of Jenkins
[ https://issues.apache.org/jira/browse/KYLIN-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang resolved KYLIN-5053. - Resolution: Fixed > Update the website to link to TravisCI and Github Actions instead of Jenkins > > > Key: KYLIN-5053 > URL: https://issues.apache.org/jira/browse/KYLIN-5053 > Project: Kylin > Issue Type: Task > Components: Documentation >Affects Versions: v3.1.2 >Reporter: Martin Tzvetanov Grigorov >Priority: Minor > Fix For: v3.1.3 > > > Currently [https://kylin.apache.org/development/#ci-and-code-analysis] points > to [https://builds.apache.org/job/Kylin-Master-JDK-1.8/] as CI, but this url > leads to error 404 (Not Found). > Since a while Kylin uses > [TravisCI|https://app.travis-ci.com/github/apache/kylin] and [GitHub > Actions|https://github.com/apache/kylin/actions] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (KYLIN-4978) NPE: submit refresh request using restful api when there is no segments
[ https://issues.apache.org/jira/browse/KYLIN-4978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang resolved KYLIN-4978. - Resolution: Fixed > NPE: submit refresh request using restful api when there is no segments > --- > > Key: KYLIN-4978 > URL: https://issues.apache.org/jira/browse/KYLIN-4978 > Project: Kylin > Issue Type: Bug >Reporter: Kun Liu >Assignee: Kun Liu >Priority: Minor > Fix For: v3.1.3 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (KYLIN-4971) Add new measure bitmap_map for count distinct measure in UI
[ https://issues.apache.org/jira/browse/KYLIN-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang resolved KYLIN-4971. - Resolution: Fixed > Add new measure bitmap_map for count distinct measure in UI > > > Key: KYLIN-4971 > URL: https://issues.apache.org/jira/browse/KYLIN-4971 > Project: Kylin > Issue Type: Task >Reporter: JiangYang >Assignee: JiangYang >Priority: Minor > Fix For: v3.1.3 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (KYLIN-4947) Implement spark engine for cube optimization jobs
[ https://issues.apache.org/jira/browse/KYLIN-4947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang resolved KYLIN-4947. - Resolution: Fixed > Implement spark engine for cube optimization jobs > - > > Key: KYLIN-4947 > URL: https://issues.apache.org/jira/browse/KYLIN-4947 > Project: Kylin > Issue Type: New Feature > Components: Job Engine >Reporter: JiangYang >Assignee: JiangYang >Priority: Minor > Fix For: v3.1.3 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (KYLIN-4970) Fix spark.executor.extraJavaOptions args
[ https://issues.apache.org/jira/browse/KYLIN-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang resolved KYLIN-4970. - Resolution: Fixed > Fix spark.executor.extraJavaOptions args > > > Key: KYLIN-4970 > URL: https://issues.apache.org/jira/browse/KYLIN-4970 > Project: Kylin > Issue Type: Bug > Components: Spark Engine >Reporter: JiangYang >Assignee: JiangYang >Priority: Minor > Fix For: v3.1.3 > > > in spark doc: spark.executor.extraJavaOption need surrond with "". > !image-2021-04-09-10-18-58-294.png! -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (KYLIN-4974) Kylin does not handle fucntion CURRENT_TIME().
[
https://issues.apache.org/jira/browse/KYLIN-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yaqian Zhang resolved KYLIN-4974.
-
Resolution: Fixed
> Kylin does not handle fucntion CURRENT_TIME().
> --
>
> Key: KYLIN-4974
> URL: https://issues.apache.org/jira/browse/KYLIN-4974
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v3.1.1
>Reporter: Alexander
>Assignee: Alexander
>Priority: Minor
> Fix For: v3.1.3
>
>
> Select query:
> {code:java}
> ==[QUERY]===
> Query Id: 4e8c6498-9736-604f-ca84-c537e749448b
> SQL: select CURRENT_TIME() from KYLIN_SALES
> {code}
> Response:
> {code:java}
> org.apache.kylin.rest.exception.InternalErrorException: No DataTypeSerializer
> for type timeorg.apache.kylin.rest.exception.InternalErrorException: No
> DataTypeSerializer for type timewhile executing SQL: "select * from (select
> CURRENT_TIME() from KYLIN_SALES) limit 5" at
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:486)
> at
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:401)
> at
> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:93)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (KYLIN-5066) CubeMigrationCrossClusterCLI - distcp (java.io.FileNotFoundException): Path is not a file
[
https://issues.apache.org/jira/browse/KYLIN-5066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yaqian Zhang updated KYLIN-5066:
Fix Version/s: v3.1.4
(was: v3.1.3)
> CubeMigrationCrossClusterCLI - distcp (java.io.FileNotFoundException): Path
> is not a file
> --
>
> Key: KYLIN-5066
> URL: https://issues.apache.org/jira/browse/KYLIN-5066
> Project: Kylin
> Issue Type: Bug
> Components: Client - CLI
>Affects Versions: v3.1.2
> Environment: HDP 3.1.4
>Reporter: Alexander
>Assignee: Alexander
>Priority: Major
> Fix For: v3.1.4
>
> Attachments:
> 0001-KYLIN-5066-CubeMigrationCrossClusterCLI-distcp-java..patch
>
>
> Try to migrate cube between clusters:
> {code:java}
> $KYLIN_HOME/bin/kylin.sh
> org.apache.kylin.tool.migration.CubeMigrationCrossClusterCLI -kylinUriSrc
> ADMIN:KYLIN@hostname1:7070 -kylinUriDst ADMIN:KYLIN@hostname12:7070 -cube
> CUBE_NAME -execute true -schemaOnly false -overwrite true
> {code}
> Got exception:
> {code:java}
> 2021-08-20 00:07:04,533 ERROR
> [Cube-LOY_TML_FAVCAT_V1_4-data-migration-pool-7]
> migration.CubeMigrationCrossClusterCLI:679 : fail to migrate htable
> KYLIN_7E0QA66PIM due to {} 2021-08-20 00:07:04,533 ERROR
> [Cube-LOY_TML_FAVCAT_V1_4-data-migration-pool-7]
> migration.CubeMigrationCrossClusterCLI:679 : fail to migrate htable
> KYLIN_7E0QA66PIM due to {} java.io.FileNotFoundException: Path is not a file:
> /apps/hbase/data/data/default/KYLIN_7E0QA66PIM at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:90)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:76)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:158)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1931)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:738)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:426)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1025) at
> org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:876) at
> org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:822) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2682)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121)
> at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88)
> at
> org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:864) at
> org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:851) at
> org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:840) at
> org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1004) at
> org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall(DistributedFileSystem.java:320)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall(DistributedFileSystem.java:316)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:328)
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:899) at
> org.apache.hadoop.tools.FileBasedCopyListing.fetchFileList(FileBasedCopyListing.java:78)
> at
> org.apache.hadoop.tools.FileBasedCopyListing.doBuildListing(FileBasedCopyListing.java:69)
> at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:86) at
> org.apache.hadoop.tools.DistCp.createInputFileListing(DistCp.java:368) at
> org.apache.hadoop.tools.DistCp.prepareFileListing(DistCp.java:96) at
> org.apache.hadoop.tools.DistCp.createAndSubmitJob(DistCp.java:205) at
> org.apache.hadoop.tools.DistCp.execute(DistCp.java:182) at
> org.apache.kylin.tool.migration.Cub
[jira] [Resolved] (KYLIN-5035) Fix Merge Cuboid Statistics EOFException
[ https://issues.apache.org/jira/browse/KYLIN-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang resolved KYLIN-5035. - Resolution: Fixed > Fix Merge Cuboid Statistics EOFException > > > Key: KYLIN-5035 > URL: https://issues.apache.org/jira/browse/KYLIN-5035 > Project: Kylin > Issue Type: Bug > Components: Job Engine, Spark Engine >Reporter: JiangYang >Assignee: JiangYang >Priority: Major > Fix For: v3.1.3 > > > org.apache.kylin.job.exception.ExecuteException: java.io.EOFException at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:201) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:72) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:180) > at > org.apache.kylin.job.impl.threadpool.DistributedScheduler\$JobRunner.run(DistributedScheduler.java:238) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor\$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) Caused by: java.io.EOFException at > java.io.DataInputStream.readFully(DataInputStream.java:197) at > java.io.DataInputStream.readFully(DataInputStream.java:169) at > org.apache.hadoop.io.SequenceFile\$Reader.init(SequenceFile.java:1918) at > org.apache.hadoop.io.SequenceFile\$Reader.initialize(SequenceFile.java:1883) > at org.apache.hadoop.io.SequenceFile\$Reader.(SequenceFile.java:1832) at > org.apache.kylin.engine.mr.common.CubeStatsReader\$CubeStatsResult.(CubeStatsReader.java:484) > at > org.apache.kylin.engine.mr.common.CubeStatsReader.(CubeStatsReader.java:133) > at > org.apache.kylin.engine.mr.steps.MergeStatisticsWithOldStep.doWork(MergeStatisticsWithOldStep.java:97) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:180) > ... 6 more -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (KYLIN-4969) Query results may be incorrect if the query filter condition contains derived dimensions
[ https://issues.apache.org/jira/browse/KYLIN-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaqian Zhang updated KYLIN-4969: Fix Version/s: v3.1.4 (was: v3.1.3) > Query results may be incorrect if the query filter condition contains derived > dimensions > > > Key: KYLIN-4969 > URL: https://issues.apache.org/jira/browse/KYLIN-4969 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v3.1.1 >Reporter: zhimin wu >Assignee: zhimin wu >Priority: Major > Fix For: v3.1.4 > > Attachments: image-2021-04-11-16-11-03-058.png, > image-2021-04-11-16-14-10-698.png, image-2021-04-11-16-15-52-352.png, > image-2021-04-11-16-17-45-890.png, image-2021-04-11-16-21-53-243.png > > Original Estimate: 48h > Remaining Estimate: 48h > > There are three tables. > !image-2021-04-11-16-11-03-058.png|width=431,height=280! > Model as follows > !image-2021-04-11-16-14-10-698.png|width=824,height=328! > Cube as follows > !image-2021-04-11-16-15-52-352.png|width=765,height=425! > the query result on kylin > !image-2021-04-11-16-17-45-890.png|width=762,height=450! > resule on hive > !image-2021-04-11-16-21-53-243.png|width=1207,height=913! > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (KYLIN-5007) queries with limit clause may fail when string dimension is encoded in integer type
[
https://issues.apache.org/jira/browse/KYLIN-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yaqian Zhang resolved KYLIN-5007.
-
Resolution: Fixed
> queries with limit clause may fail when string dimension is encoded in
> integer type
> ---
>
> Key: KYLIN-5007
> URL: https://issues.apache.org/jira/browse/KYLIN-5007
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v3.0.2
>Reporter: Congling Xia
>Assignee: Congling Xia
>Priority: Major
> Fix For: v3.1.3
>
> Attachments: image-2021-06-10-10-03-54-775.png
>
>
> Hi, team.
> Recently we encounter a problem that queries may fail if there is a LIMIT in
> the SQL. The SQL looks like:
> {code}
> select gid from some_table group by gid limit 100
> {code}
> The error message is like the following:
> {code:java}
> Not sorted! last: source_v1=null,...,gid=276,... fetched:
> source_v1=null,...,gid=100506,...
> {code}
> After searching the issues list, we find it is similar with KYLIN-2425,
> KYLIN-3089, and KYLIN-4942. We notice that these problems are not completely
> resolved.
> It is an row-key encoding problem, the cube uses integer:4 to encode string
> column _gid_:
> !image-2021-06-10-10-03-54-775.png|width=571,height=141!
> As [~kangkaisen] mensioned in KYLIN-3089, comparator in
> SortMergedPartitionResultIterator is different from the one in
> SortedIteratorMergerWithLimit. SortedIteratorMergerWithLimit compares tuple
> of dimensions in their origin data type "string" rather than the encoded data
> type "integer" in rowkeys. In the exception message above, 276<100506 is
> false because they are compared in "string" type.
> It may be resolved by skipping limit pushdown when column type and encoding
> type may produce different comparing results, but it may lead such queries to
> be slower.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
