[jira] [Updated] (KYLIN-4159) The first step of build cube job will fail and throw "Column 'xx' in where clause is ambiguous" in jdbc datasource.
[ https://issues.apache.org/jira/browse/KYLIN-4159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4159: Fix Version/s: v2.6.4 v3.0.0-beta > The first step of build cube job will fail and throw "Column 'xx' in where > clause is ambiguous" in jdbc datasource. > --- > > Key: KYLIN-4159 > URL: https://issues.apache.org/jira/browse/KYLIN-4159 > Project: Kylin > Issue Type: Bug >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > > Reappear: > In the model of kylin, If we have a column used as a partition column in the > fact table, and there is a column in lookup table has the same name. It can > be reappeared。 > RootCause: > In jdbc datasource , we use Sqoop option "--boundary-query sql" to split > data, the sql like "select max(partition_column), min(partition_column) from > flat_table where flat_table.partition_column> 20120101 and > flat_table.partition.column < 20190101" and there is more than one column > which have the same name of the "partition_column" in the flat_table. the sql > driver can not distinguish the table that partition_column belong to。 > How to resolve: > Alter the sql like: "select max(t.partition_column),min(t.partition_column) > from flat_table where flat_table.partition_column> 20120101 and > flat_table.partition.column < 20190101" > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4159) The first step of build cube job will fail and throw "Column 'xx' in where clause is ambiguous" in jdbc datasource.
[ https://issues.apache.org/jira/browse/KYLIN-4159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4159. - Resolution: Fixed > The first step of build cube job will fail and throw "Column 'xx' in where > clause is ambiguous" in jdbc datasource. > --- > > Key: KYLIN-4159 > URL: https://issues.apache.org/jira/browse/KYLIN-4159 > Project: Kylin > Issue Type: Bug >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > > Reappear: > In the model of kylin, If we have a column used as a partition column in the > fact table, and there is a column in lookup table has the same name. It can > be reappeared。 > RootCause: > In jdbc datasource , we use Sqoop option "--boundary-query sql" to split > data, the sql like "select max(partition_column), min(partition_column) from > flat_table where flat_table.partition_column> 20120101 and > flat_table.partition.column < 20190101" and there is more than one column > which have the same name of the "partition_column" in the flat_table. the sql > driver can not distinguish the table that partition_column belong to。 > How to resolve: > Alter the sql like: "select max(t.partition_column),min(t.partition_column) > from flat_table where flat_table.partition_column> 20120101 and > flat_table.partition.column < 20190101" > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4055) cube quey and ad-hoc query return different meta info
[ https://issues.apache.org/jira/browse/KYLIN-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4055: Fix Version/s: v3.0.0-beta > cube quey and ad-hoc query return different meta info > - > > Key: KYLIN-4055 > URL: https://issues.apache.org/jira/browse/KYLIN-4055 > Project: Kylin > Issue Type: Improvement >Reporter: jinguowei >Assignee: Yaqian Zhang >Priority: Major > Fix For: v3.0.0-beta > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4055) cube quey and ad-hoc query return different meta info
[ https://issues.apache.org/jira/browse/KYLIN-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4055: Fix Version/s: v2.6.4 > cube quey and ad-hoc query return different meta info > - > > Key: KYLIN-4055 > URL: https://issues.apache.org/jira/browse/KYLIN-4055 > Project: Kylin > Issue Type: Improvement >Reporter: jinguowei >Assignee: Yaqian Zhang >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4055) cube quey and ad-hoc query return different meta info
[ https://issues.apache.org/jira/browse/KYLIN-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4055. - Resolution: Fixed > cube quey and ad-hoc query return different meta info > - > > Key: KYLIN-4055 > URL: https://issues.apache.org/jira/browse/KYLIN-4055 > Project: Kylin > Issue Type: Improvement >Reporter: jinguowei >Assignee: Yaqian Zhang >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4157) When using PrepareStatement query, functions within WHERE will cause InternalErrorException
[ https://issues.apache.org/jira/browse/KYLIN-4157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4157: Fix Version/s: v3.0.0-beta > When using PrepareStatement query, functions within WHERE will cause > InternalErrorException > --- > > Key: KYLIN-4157 > URL: https://issues.apache.org/jira/browse/KYLIN-4157 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v2.6.3 >Reporter: Marc Wu >Assignee: Marc Wu >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > Attachments: image-2019-09-04-15-39-52-867.png, > image-2019-09-04-15-39-58-276.png, image-2019-09-04-20-57-46-121.png > > > Hi Kylin Team: > I found a bug when I'm using PreparedStatement query. > Let me use table KYLIN_SALES to explain the scenario. > There is a SQL like: > select LSTG_FORMAT_NAME, sum(PRICE) from KYLIN_SALES where > lower(LSTG_FORMAT_NAME) = 'fp-gtc' group by LSTG_FORMAT_NAME > In some cases, user doesn't know the LSTG_FORMAT_NAME is upper case or lower > case, or they just want to query data ignoring cases. > So assume they use lower(LSTG_FORMAT_NAME) = 'fp-gtc', it's a function > within the filter. > When I execute this SQL on Kylin web console, it can get the right result, > but when I tried to execute it by PreparedStatement query on Postman, it > threw InternalErrorException. !image-2019-09-04-15-39-58-276.png! > > !image-2019-09-04-15-39-52-867.png! > > > h2. Root Cause > I debugged Kylin to find the root cause, it turns out when the filter > contains function, compFilter.getColumn is null, and the > compFilter.getFunction contains lower(LSTG_FORMAT_NAME), so that > compFilter.getColumn().getType().isDateTimeFamily() will throw > NullPointerException, that's the root cause for this 500 error. > > h2. !image-2019-09-04-20-57-46-121.png! Solution > I suggest to check nullable first before using compFilter.getColumn. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4154) Metadata inconsistency between multi Kylin server caused by Broadcaster closing
[ https://issues.apache.org/jira/browse/KYLIN-4154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4154. - Resolution: Fixed > Metadata inconsistency between multi Kylin server caused by Broadcaster > closing > > > Key: KYLIN-4154 > URL: https://issues.apache.org/jira/browse/KYLIN-4154 > Project: Kylin > Issue Type: Bug > Components: Metadata >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v3.0.0 > > > To avoid Broadcaster memory leak, KYLIN-4131 close the metadata sync thread > after receiving "Sync All" event, but there may be some events in the waiting > queue, which haven't been synced yet. So we should sync all events before > closing sync thread. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4154) Metadata inconsistency between multi Kylin server caused by Broadcaster closing
[ https://issues.apache.org/jira/browse/KYLIN-4154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4154: Fix Version/s: (was: v3.0.0) v3.0.0-beta > Metadata inconsistency between multi Kylin server caused by Broadcaster > closing > > > Key: KYLIN-4154 > URL: https://issues.apache.org/jira/browse/KYLIN-4154 > Project: Kylin > Issue Type: Bug > Components: Metadata >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v3.0.0-beta > > > To avoid Broadcaster memory leak, KYLIN-4131 close the metadata sync thread > after receiving "Sync All" event, but there may be some events in the waiting > queue, which haven't been synced yet. So we should sync all events before > closing sync thread. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4158) Query failed for GroupBy an expression of column with limit in SQL
[
https://issues.apache.org/jira/browse/KYLIN-4158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4158:
Fix Version/s: v2.6.4
v3.0.0-beta
> Query failed for GroupBy an expression of column with limit in SQL
> --
>
> Key: KYLIN-4158
> URL: https://issues.apache.org/jira/browse/KYLIN-4158
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v2.6.2
>Reporter: PENG Zhengshuai
>Assignee: PENG Zhengshuai
>Priority: Major
> Fix For: v3.0.0-beta, v2.6.4
>
>
> When executing a sql query, the limit should not be pushdown to storage scan
> level if group by clause is an expression of columns.
> For example:
> {code:java}
> SELECT (CASE WHEN ("TEST_KYLIN_FACT"."LSTG_FORMAT_NAME" IN ('Auction',
> 'FP-GTC')) THEN 'Auction' ELSE "TEST_KYLIN_FACT"."LSTG_FORMAT_NAME" END) AS
> "LSTG_FORMAT_NAME__group_",
> SUM("TEST_KYLIN_FACT"."PRICE") AS "sum_PRICE_ok"
> FROM "TEST_KYLIN_FACT" "TEST_KYLIN_FACT"
> GROUP BY (CASE WHEN ("TEST_KYLIN_FACT"."LSTG_FORMAT_NAME" IN ('Auction',
> 'FP-GTC')) THEN 'Auction' ELSE "TEST_KYLIN_FACT"."LSTG_FORMAT_NAME" END)
> limit 4
> {code}
> In this SQL, group by clause is an expression, the limit should NOT be
> push-down to storage scan level, otherwise, the result is incorrect
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-4158) Query failed for GroupBy an expression of column with limit in SQL
[
https://issues.apache.org/jira/browse/KYLIN-4158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4158.
-
Resolution: Fixed
> Query failed for GroupBy an expression of column with limit in SQL
> --
>
> Key: KYLIN-4158
> URL: https://issues.apache.org/jira/browse/KYLIN-4158
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v2.6.2
>Reporter: PENG Zhengshuai
>Assignee: PENG Zhengshuai
>Priority: Major
> Fix For: v3.0.0-beta, v2.6.4
>
>
> When executing a sql query, the limit should not be pushdown to storage scan
> level if group by clause is an expression of columns.
> For example:
> {code:java}
> SELECT (CASE WHEN ("TEST_KYLIN_FACT"."LSTG_FORMAT_NAME" IN ('Auction',
> 'FP-GTC')) THEN 'Auction' ELSE "TEST_KYLIN_FACT"."LSTG_FORMAT_NAME" END) AS
> "LSTG_FORMAT_NAME__group_",
> SUM("TEST_KYLIN_FACT"."PRICE") AS "sum_PRICE_ok"
> FROM "TEST_KYLIN_FACT" "TEST_KYLIN_FACT"
> GROUP BY (CASE WHEN ("TEST_KYLIN_FACT"."LSTG_FORMAT_NAME" IN ('Auction',
> 'FP-GTC')) THEN 'Auction' ELSE "TEST_KYLIN_FACT"."LSTG_FORMAT_NAME" END)
> limit 4
> {code}
> In this SQL, group by clause is an expression, the limit should NOT be
> push-down to storage scan level, otherwise, the result is incorrect
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-3392) Support NULL value in Sum, Max, Min Aggregation
[ https://issues.apache.org/jira/browse/KYLIN-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-3392. - Resolution: Fixed > Support NULL value in Sum, Max, Min Aggregation > --- > > Key: KYLIN-3392 > URL: https://issues.apache.org/jira/browse/KYLIN-3392 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > > It is treated as 0 when confronted with NULL value in KYLIN's basic aggregate > measure (like sum, max, min). However, to distinguish the NULL value with 0 > is very necessary. > It should be like this > *sum(null, null) = null* > *sum(null, 1) = 1* > *max(null, null) = null* > *max(null, -1) = -1* > *min(null, -1）= -1* > in accordance with Hive and SparkSQL -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-3392) Support NULL value in Sum, Max, Min Aggregation
[ https://issues.apache.org/jira/browse/KYLIN-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3392: Fix Version/s: (was: v3.0.0) v3.0.0-beta > Support NULL value in Sum, Max, Min Aggregation > --- > > Key: KYLIN-3392 > URL: https://issues.apache.org/jira/browse/KYLIN-3392 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > > It is treated as 0 when confronted with NULL value in KYLIN's basic aggregate > measure (like sum, max, min). However, to distinguish the NULL value with 0 > is very necessary. > It should be like this > *sum(null, null) = null* > *sum(null, 1) = 1* > *max(null, null) = null* > *max(null, -1) = -1* > *min(null, -1）= -1* > in accordance with Hive and SparkSQL -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4156) Users need the start_date and end_date of cube for lookup table in where sentence
[
https://issues.apache.org/jira/browse/KYLIN-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4156.
-
Resolution: Fixed
> Users need the start_date and end_date of cube for lookup table in where
> sentence
> --
>
> Key: KYLIN-4156
> URL: https://issues.apache.org/jira/browse/KYLIN-4156
> Project: Kylin
> Issue Type: Improvement
>Reporter: liuzhixin
>Priority: Minor
> Fix For: v3.0.0-beta
>
>
> *Description*
> The users can use ${START_DATE} and ${END_DATE} in where filter which in the
> WebUI.
> And the dynamic variables ${START_DATE} and ${END_DATE} are from the building
> date of Cube.
> *Improvement*
> Sometimes we can improve the filter condition for lookup table with
> ${START_DATE} and ${END_DATE}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4156) Users need the start_date and end_date of cube for lookup table in where sentence
[
https://issues.apache.org/jira/browse/KYLIN-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4156:
Fix Version/s: v3.0.0-beta
> Users need the start_date and end_date of cube for lookup table in where
> sentence
> --
>
> Key: KYLIN-4156
> URL: https://issues.apache.org/jira/browse/KYLIN-4156
> Project: Kylin
> Issue Type: Improvement
>Reporter: liuzhixin
>Priority: Minor
> Fix For: v3.0.0-beta
>
>
> *Description*
> The users can use ${START_DATE} and ${END_DATE} in where filter which in the
> WebUI.
> And the dynamic variables ${START_DATE} and ${END_DATE} are from the building
> date of Cube.
> *Improvement*
> Sometimes we can improve the filter condition for lookup table with
> ${START_DATE} and ${END_DATE}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (KYLIN-4156) Users need the start_date and end_date of cube for lookup table in where sentence
[
https://issues.apache.org/jira/browse/KYLIN-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen reopened KYLIN-4156:
-
> Users need the start_date and end_date of cube for lookup table in where
> sentence
> --
>
> Key: KYLIN-4156
> URL: https://issues.apache.org/jira/browse/KYLIN-4156
> Project: Kylin
> Issue Type: Improvement
>Reporter: liuzhixin
>Priority: Minor
>
> *Description*
> The users can use ${START_DATE} and ${END_DATE} in where filter which in the
> WebUI.
> And the dynamic variables ${START_DATE} and ${END_DATE} are from the building
> date of Cube.
> *Improvement*
> Sometimes we can improve the filter condition for lookup table with
> ${START_DATE} and ${END_DATE}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-4103) Make the user string in granting operation of project is case insensitive
[ https://issues.apache.org/jira/browse/KYLIN-4103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4103. - Resolution: Fixed > Make the user string in granting operation of project is case insensitive > - > > Key: KYLIN-4103 > URL: https://issues.apache.org/jira/browse/KYLIN-4103 > Project: Kylin > Issue Type: Bug >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: v3.0.0-beta, v2.6.4 > > > Currently the user name of login operation is case insensitive. User can > login in kylin with lower case string or upper case string. But it is not > granting operation. > If we use lower case string of user name in project granting operation, there > will be no exception but the user can not set the project. > The reason is that the sid in AccessService/AclService is not case insensitive -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4103) Make the user string in granting operation of project is case insensitive
[ https://issues.apache.org/jira/browse/KYLIN-4103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4103: Fix Version/s: v2.6.4 v3.0.0-beta > Make the user string in granting operation of project is case insensitive > - > > Key: KYLIN-4103 > URL: https://issues.apache.org/jira/browse/KYLIN-4103 > Project: Kylin > Issue Type: Bug >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: v3.0.0-beta, v2.6.4 > > > Currently the user name of login operation is case insensitive. User can > login in kylin with lower case string or upper case string. But it is not > granting operation. > If we use lower case string of user name in project granting operation, there > will be no exception but the user can not set the project. > The reason is that the sid in AccessService/AclService is not case insensitive -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4100) Add overall job number statistics in monitor page
[ https://issues.apache.org/jira/browse/KYLIN-4100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4100: Fix Version/s: v3.0.0-beta > Add overall job number statistics in monitor page > - > > Key: KYLIN-4100 > URL: https://issues.apache.org/jira/browse/KYLIN-4100 > Project: Kylin > Issue Type: Improvement >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: v3.0.0-beta > > Attachments: x.png > > > Currently it's hard to get pending and running job number in mointor page, we > can only continue to click more until the end. > It's better to have an overall job number statistics in monitor page. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4100) Add overall job number statistics in monitor page
[ https://issues.apache.org/jira/browse/KYLIN-4100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4100. - Resolution: Fixed > Add overall job number statistics in monitor page > - > > Key: KYLIN-4100 > URL: https://issues.apache.org/jira/browse/KYLIN-4100 > Project: Kylin > Issue Type: Improvement >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Minor > Fix For: v3.0.0-beta > > Attachments: x.png > > > Currently it's hard to get pending and running job number in mointor page, we > can only continue to click more until the end. > It's better to have an overall job number statistics in monitor page. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4170) Add cube information and query information where hbase deadline happened
[ https://issues.apache.org/jira/browse/KYLIN-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4170: Fix Version/s: (was: v2.6.4) v3.0.0 > Add cube information and query information where hbase deadline happened > > > Key: KYLIN-4170 > URL: https://issues.apache.org/jira/browse/KYLIN-4170 > Project: Kylin > Issue Type: Improvement > Components: Storage - HBase >Affects Versions: v2.6.2 >Reporter: 柳攀 >Priority: Major > Fix For: v3.0.0 > > Original Estimate: 168h > Remaining Estimate: 168h > > When there is a large number of cube query and hbase server is overload, > hbase deadline would happened.But there is no information about cube and > query for us to locate the problem.If we could get cube and query information > in query log, we could solve the query problem quickly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4169) too many logs while DataModelManager init, cause the first RESTful API hang for a long time
[
https://issues.apache.org/jira/browse/KYLIN-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16932232#comment-16932232
]
nichunen commented on KYLIN-4169:
-
Hi [~zhoukangcn], you may edit the metadata of your projects, delete the
un-existed models in projects' metadata.
> too many logs while DataModelManager init, cause the first RESTful API hang
> for a long time
> ---
>
> Key: KYLIN-4169
> URL: https://issues.apache.org/jira/browse/KYLIN-4169
> Project: Kylin
> Issue Type: Bug
>Reporter: ZhouKang
>Priority: Major
>
> Too many logs while DataModelManager init. It is will happend after startup
> or reload metastore.
> In our production enviroment，there are 120,000+ lines.
> {code:java}
> // code placeholder
> grep 'is missing or unloaded yet' kylin.log.8|wc -l
> 121861
> {code}
>
> which cause the first sql query's time up to 40s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4140) Add the time filter for current day jobs and make default values for web configurable
[ https://issues.apache.org/jira/browse/KYLIN-4140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16932347#comment-16932347 ] nichunen commented on KYLIN-4140: - Test pass > Add the time filter for current day jobs and make default values for web > configurable > - > > Key: KYLIN-4140 > URL: https://issues.apache.org/jira/browse/KYLIN-4140 > Project: Kylin > Issue Type: Improvement > Components: Web >Affects Versions: v2.5.2 >Reporter: 柳攀 >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > > We usually prepared our data up to yesterday's zero and build cube for the > latest data before dawn.We monitor the cube build jobs and make sure they > finish on time. As a result, the jobs information before current day is > useless for us, because they have all finished yesterday and hava no > problem.In addition,when there is many information of cubes and jobs, getting > information from hbase will take a lot of time.So we add time filter for > current day jobs and make the default value configurable -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4034) The table should not display in Insight page when the user has no access to the table
[ https://issues.apache.org/jira/browse/KYLIN-4034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4034: Attachment: 屏幕快照 2019-09-18 下午11.09.13.png > The table should not display in Insight page when the user has no access to > the table > - > > Key: KYLIN-4034 > URL: https://issues.apache.org/jira/browse/KYLIN-4034 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.6.2 >Reporter: Na Zhai >Assignee: Yaqian Zhang >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > Attachments: 屏幕快照 2019-09-18 下午11.09.13.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4034) The table should not display in Insight page when the user has no access to the table
[ https://issues.apache.org/jira/browse/KYLIN-4034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16932538#comment-16932538 ] nichunen commented on KYLIN-4034: - !屏幕快照 2019-09-18 下午11.09.13.png! Verified > The table should not display in Insight page when the user has no access to > the table > - > > Key: KYLIN-4034 > URL: https://issues.apache.org/jira/browse/KYLIN-4034 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.6.2 >Reporter: Na Zhai >Assignee: Yaqian Zhang >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > Attachments: 屏幕快照 2019-09-18 下午11.09.13.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4169) too many logs while DataModelManager init, cause the first RESTful API hang for a long time
[
https://issues.apache.org/jira/browse/KYLIN-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16932974#comment-16932974
]
nichunen commented on KYLIN-4169:
-
[~zhoukangcn] You are right, I found the log in my Kylin server too, will you
submit a PR to fix?
> too many logs while DataModelManager init, cause the first RESTful API hang
> for a long time
> ---
>
> Key: KYLIN-4169
> URL: https://issues.apache.org/jira/browse/KYLIN-4169
> Project: Kylin
> Issue Type: Bug
>Reporter: ZhouKang
>Priority: Major
>
> Too many logs while DataModelManager init. It is will happend after startup
> or reload metastore.
> In our production enviroment，there are 120,000+ lines.
> {code:java}
> // code placeholder
> grep 'is missing or unloaded yet' kylin.log.8|wc -l
> 121861
> {code}
>
> which cause the first sql query's time up to 40s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4144) Cannot update user's info because of case-sensitive process
[
https://issues.apache.org/jira/browse/KYLIN-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16932993#comment-16932993
]
nichunen commented on KYLIN-4144:
-
I think there should be some backward compatible work for metadata of user,
user name with lowercase should be converted to uppercase in the metadata.
> Cannot update user's info because of case-sensitive process
> ---
>
> Key: KYLIN-4144
> URL: https://issues.apache.org/jira/browse/KYLIN-4144
> Project: Kylin
> Issue Type: Bug
>Reporter: ZhouKang
>Priority: Major
> Fix For: v2.5.3
>
>
> In https://issues.apache.org/jira/browse/KYLIN-3531 , uppercase of usernames
> will be saved in HBase, which make our case faild.
>
> when users was added useing kylin:v2.4.0 (maybe older), the rawkey in hbase
> is lowercase, when update user's info in Kylin:v2.5.3, there is en error.
> {code:java}
> // code placeholder
> Overwriting conflict /user/, expect old TS xxx, but it is 0, the expected
> new TS: {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (KYLIN-4170) Add cube information and query information when hbase deadline happened
[ https://issues.apache.org/jira/browse/KYLIN-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen reassigned KYLIN-4170: --- Assignee: 柳攀 > Add cube information and query information when hbase deadline happened > --- > > Key: KYLIN-4170 > URL: https://issues.apache.org/jira/browse/KYLIN-4170 > Project: Kylin > Issue Type: Improvement > Components: Storage - HBase >Affects Versions: v2.6.2 >Reporter: 柳攀 >Assignee: 柳攀 >Priority: Major > Fix For: v3.0.0 > > Original Estimate: 168h > Remaining Estimate: 168h > > When there is a large number of cube query and hbase server is overload, > hbase deadline would happened.But there is no information about cube and > query for us to locate the problem.If we could get cube and query information > in query log, we could solve the query problem quickly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (KYLIN-4156) Users need the start_date and end_date of cube for lookup table in where sentence
[
https://issues.apache.org/jira/browse/KYLIN-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen reopened KYLIN-4156:
-
> Users need the start_date and end_date of cube for lookup table in where
> sentence
> --
>
> Key: KYLIN-4156
> URL: https://issues.apache.org/jira/browse/KYLIN-4156
> Project: Kylin
> Issue Type: Improvement
>Reporter: liuzhixin
>Priority: Minor
> Fix For: v3.0.0-beta
>
> Attachments: image-2019-09-20-10-23-50-928.png,
> image-2019-09-20-10-24-16-071.png, image-2019-09-20-10-24-59-221.png,
> image-2019-09-20-10-25-40-510.png, image-2019-09-20-10-26-51-715.png,
> image-2019-09-20-10-27-09-300.png
>
>
> *Description*
> The users can use ${START_DATE} and ${END_DATE} in where filter which in the
> WebUI.
> And the dynamic variables ${START_DATE} and ${END_DATE} are from the building
> date of Cube.
> *Improvement*
> Sometimes we can improve the filter condition for lookup table with
> ${START_DATE} and ${END_DATE}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4156) Users need the start_date and end_date of cube for lookup table in where sentence
[
https://issues.apache.org/jira/browse/KYLIN-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4156:
Fix Version/s: (was: v3.0.0-beta)
v3.0.0
> Users need the start_date and end_date of cube for lookup table in where
> sentence
> --
>
> Key: KYLIN-4156
> URL: https://issues.apache.org/jira/browse/KYLIN-4156
> Project: Kylin
> Issue Type: Improvement
>Reporter: liuzhixin
>Priority: Minor
> Fix For: v3.0.0
>
> Attachments: image-2019-09-20-10-23-50-928.png,
> image-2019-09-20-10-24-16-071.png, image-2019-09-20-10-24-59-221.png,
> image-2019-09-20-10-25-40-510.png, image-2019-09-20-10-26-51-715.png,
> image-2019-09-20-10-27-09-300.png
>
>
> *Description*
> The users can use ${START_DATE} and ${END_DATE} in where filter which in the
> WebUI.
> And the dynamic variables ${START_DATE} and ${END_DATE} are from the building
> date of Cube.
> *Improvement*
> Sometimes we can improve the filter condition for lookup table with
> ${START_DATE} and ${END_DATE}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-3392) Support NULL value in Sum, Max, Min Aggregation
[ https://issues.apache.org/jira/browse/KYLIN-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3392: Fix Version/s: v3.0.0 > Support NULL value in Sum, Max, Min Aggregation > --- > > Key: KYLIN-3392 > URL: https://issues.apache.org/jira/browse/KYLIN-3392 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > Fix For: v3.0.0 > > Attachments: KYLIN-3392-2.png, KYLIN-3392.png, kylin-3.0.0-alpha2.png > > > It is treated as 0 when confronted with NULL value in KYLIN's basic aggregate > measure (like sum, max, min). However, to distinguish the NULL value with 0 > is very necessary. > It should be like this > *sum(null, null) = null* > *sum(null, 1) = 1* > *max(null, null) = null* > *max(null, -1) = -1* > *min(null, -1）= -1* > in accordance with Hive and SparkSQL -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4173) cube list search can not work
[ https://issues.apache.org/jira/browse/KYLIN-4173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4173: Fix Version/s: v3.0.0-beta > cube list search can not work > - > > Key: KYLIN-4173 > URL: https://issues.apache.org/jira/browse/KYLIN-4173 > Project: Kylin > Issue Type: Bug >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > > In the font-end, service/cube.js add a new path "cubeName" in version > d449335d68f01270aa0b6a8093ca12daff4b74bd like > ''cubes/:cubeId/:propName/:propValue/:action/:cubeName''. > this caused the list cube api "cubes?cubeName=xx" change to "cubes/xx" and > match the wrong api. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4173) cube list search can not work
[ https://issues.apache.org/jira/browse/KYLIN-4173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4173. - Resolution: Fixed > cube list search can not work > - > > Key: KYLIN-4173 > URL: https://issues.apache.org/jira/browse/KYLIN-4173 > Project: Kylin > Issue Type: Bug >Reporter: luguosheng >Assignee: luguosheng >Priority: Major > Fix For: v3.0.0-beta > > > In the font-end, service/cube.js add a new path "cubeName" in version > d449335d68f01270aa0b6a8093ca12daff4b74bd like > ''cubes/:cubeId/:propName/:propValue/:action/:cubeName''. > this caused the list cube api "cubes?cubeName=xx" change to "cubes/xx" and > match the wrong api. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4098) Add cube auto merge api
[ https://issues.apache.org/jira/browse/KYLIN-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4098: Fix Version/s: (was: v3.0.0-beta) v3.0.0 > Add cube auto merge api > --- > > Key: KYLIN-4098 > URL: https://issues.apache.org/jira/browse/KYLIN-4098 > Project: Kylin > Issue Type: New Feature >Reporter: Shaohui Liu >Assignee: Shaohui Liu >Priority: Minor > Fix For: v3.0.0 > > Attachments: image-2019-09-19-17-11-49-733.png > > > Currently the auto merging of cube is triggered by the event of new segment > is ready automatically. When the cluster restart, there may be too many > merging job. > It's better to have a rest api to trigger the merging and make it more > controllable. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (KYLIN-4098) Add cube auto merge api
[ https://issues.apache.org/jira/browse/KYLIN-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen reopened KYLIN-4098: - > Add cube auto merge api > --- > > Key: KYLIN-4098 > URL: https://issues.apache.org/jira/browse/KYLIN-4098 > Project: Kylin > Issue Type: New Feature >Reporter: Shaohui Liu >Assignee: Shaohui Liu >Priority: Minor > Fix For: v3.0.0-beta > > Attachments: image-2019-09-19-17-11-49-733.png > > > Currently the auto merging of cube is triggered by the event of new segment > is ready automatically. When the cluster restart, there may be too many > merging job. > It's better to have a rest api to trigger the merging and make it more > controllable. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (KYLIN-3797) Too many or filters may break Kylin server when flatting filter
[ https://issues.apache.org/jira/browse/KYLIN-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen reopened KYLIN-3797: - > Too many or filters may break Kylin server when flatting filter > --- > > Key: KYLIN-3797 > URL: https://issues.apache.org/jira/browse/KYLIN-3797 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Reporter: Gang Ma >Assignee: Gang Ma >Priority: Major > Fix For: v3.0.0-alpha > > > Kylin needs to convert filter into a flat format, like: OR(AND(f1,f2), > AND(f3,f4,f5)…) , so that Kylin can build HBase scan/filter accordingly, but > when there are too many or filters in query, like: AND(OR(f1,f2,...,f1000), > OR(g1,g2,...,g1000), OR(h1,h2,...,h1000)), then the generated flat filter > size will be 1000*1000*1000=1 billion, the flatting process will cause Kylin > server OOM. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-3797) Too many or filters may break Kylin server when flatting filter
[ https://issues.apache.org/jira/browse/KYLIN-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3797: Fix Version/s: v2.6.4 > Too many or filters may break Kylin server when flatting filter > --- > > Key: KYLIN-3797 > URL: https://issues.apache.org/jira/browse/KYLIN-3797 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Reporter: Gang Ma >Assignee: Gang Ma >Priority: Major > Fix For: v3.0.0-alpha, v2.6.4 > > > Kylin needs to convert filter into a flat format, like: OR(AND(f1,f2), > AND(f3,f4,f5)…) , so that Kylin can build HBase scan/filter accordingly, but > when there are too many or filters in query, like: AND(OR(f1,f2,...,f1000), > OR(g1,g2,...,g1000), OR(h1,h2,...,h1000)), then the generated flat filter > size will be 1000*1000*1000=1 billion, the flatting process will cause Kylin > server OOM. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4098) Add cube auto merge api
[ https://issues.apache.org/jira/browse/KYLIN-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16938431#comment-16938431 ] nichunen commented on KYLIN-4098: - The PR will introduce the problem described in https://issues.apache.org/jira/browse/KYLIN-3718, re-open this issue to fix in 3.0ga release > Add cube auto merge api > --- > > Key: KYLIN-4098 > URL: https://issues.apache.org/jira/browse/KYLIN-4098 > Project: Kylin > Issue Type: New Feature >Reporter: Shaohui Liu >Assignee: Shaohui Liu >Priority: Minor > Fix For: v3.0.0 > > Attachments: image-2019-09-19-17-11-49-733.png > > > Currently the auto merging of cube is triggered by the event of new segment > is ready automatically. When the cluster restart, there may be too many > merging job. > It's better to have a rest api to trigger the merging and make it more > controllable. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4010) Auto adjust offset according to query server's timezone for time derived column
[
https://issues.apache.org/jira/browse/KYLIN-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4010.
-
Resolution: Fixed
> Auto adjust offset according to query server's timezone for time derived
> column
> ---
>
> Key: KYLIN-4010
> URL: https://issues.apache.org/jira/browse/KYLIN-4010
> Project: Kylin
> Issue Type: Improvement
> Components: Others
>Affects Versions: v3.0.0-alpha
>Reporter: zengrui
>Assignee: Xiaoxiang Yu
>Priority: Minor
> Fix For: v3.0.0-beta
>
> Attachments: image-2019-07-15-17-15-31-209.png,
> image-2019-07-15-17-17-04-029.png, image-2019-07-15-17-17-39-568.png,
> image-2019-09-22-16-35-23-663.png, image-2019-09-22-16-43-19-248.png,
> image-2019-09-22-20-28-02-402.png, image-2019-09-22-20-35-19-038.png,
> image-2019-09-22-20-53-34-720.png, image-2019-09-24-17-23-42-451.png,
> image-2019-09-24-17-33-29-824.png, image-2019-09-24-17-35-47-589.png,
> image-2019-09-24-17-36-39-611.png
>
>
> h2. Backgroud
> In realtime OLAP, we index real-time event in streaming receiver. We know
> that each event must contains a timestamp column (we often call it event
> time), that value should represent when this event was produced. Because
> event maybe come from different timezone and use local timezone is always
> *error-prone*, so we recommend to use a {color:#DE350B}GMT+0{color}
> timestamp(System.currentTimeMillis()) to avoid such issue.
> I think this is good by design, it is easy to understand and always correct.
> But the *side effect* is that, the end user(business manager behind a BI
> tools) are unhappy because he have to use GMT+0 with date/time related filter
> in SQL and should understand the result should be *shifted* with his local
> timezone. It is not user-firendly and inconvenient for normal user. Because
> user may compare query result from different data source and compare them and
> summarize, use GMT+0 may trouble them.
> h2. Example
> For example, kylin user work in *GMT+8* (maybe in Shanghai) want to know some
> metrics which occured from {color:#DE350B}2019-09-01 12:00:00{color} to
> {color:#DE350B}2019-09-01 14:00:00{color} in his {color:#DE350B}local
> timezone{color}, so he has to {color:#DE350B}rewrite{color} his query (with
> eight hour offset) to following:
> {code:sql}
> select hour_start, count(*)
> from realtime_table
> where hour_start >= "2019-09-01 04:00:00" and hour_start < "2019-09-01
> 06:00:00"
> group by hour_start
> {code}
> And he will get result like :
> ||hour_start ||count||
> |2019-09-01 04:00:00 |139202|
> |2019-09-01 05:00:00 |89398|
> And he must convert to a more meaningful result in his mind, it is realy
> annoying!
> ||hour_start ||count||
> |2019-09-01 12:00:00 |139202|
> |2019-09-01 13:00:00 |89398|
> h2. Desgin
> We should not change the way receiver index event, event time should be
> stored in UTC timestamp. We should auto rewrite sql's event time related
> filter.
> In kylin, filter condition in where clause will be convert to a
> *TupleFilter*, and it looks like *RelNode* in Apache Calicate.
> For where hour_start >= "2019-09-01 12:00:00" and hour_start < "2019-09-01
> 14:00:00", we will send TupleFilter to streaming receiver or region server
> which looks like this:
> {noformat}
> AND
> GreatThanOrEqual
> hout_start
> CAST
> "2019-09-01 12:00:00"
> timestamp
> LessThanOrEqual
> hout_start
> CAST
> "2019-09-01 14:00:00"
> timestamp
> {noformat}
> But for streaming query, we want to change each ConstantTupleFilter and minus
> value for that timestamp. So the TupleFilter which be sent will be following:
> {noformat}
> AND
> GreatThanOrEqual
> hout_start
> CAST
> "2019-09-01 04:00:00"
> timestamp
> LessThanOrEqual
> hout_start
> CAST
> "2019-09-01 06:00:00"
> timestamp
> {noformat}
> Before query result processed by *OLAPEnumerator*, kylin will plus each
> value of time derived column, thus protect row from be filtered by calcite
> generated code.
> So, user will get what he want in his timezone without any burden.
> h2. How to use
> To enable auto shift by time zone, please set
> {color:#DE350B}kylin.stream.auto.just.by.timezone{color} to true.
> You can specific time zone by {color:#DE350B}kylin.web.timezone{color},
> otherwise, time zone will be auto detected.
> Only *time derived column* will be affected.
> h2. Related Issue
> Originally, the event time can only in the format of a long value (UTC
> timestamp). But in some case, the event time is in a format of "-MM-dd
> HH:mm:ss", we use a new class DateTimeParser(introduced in KYLIN-4001) to
> convert such format into a UTC timestamp.
> h3. Old Describe
> In Real-Time Stre
[jira] [Resolved] (KYLIN-4167) Refactor streaming coordinator
[ https://issues.apache.org/jira/browse/KYLIN-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4167. - Resolution: Fixed > Refactor streaming coordinator > -- > > Key: KYLIN-4167 > URL: https://issues.apache.org/jira/browse/KYLIN-4167 > Project: Kylin > Issue Type: Improvement > Components: Real-time Streaming >Reporter: Xiaoxiang Yu >Assignee: Xiaoxiang Yu >Priority: Major > Fix For: v3.0.0-beta > > > h2. Summary > # Currently, *coordinator* has too many responsibility, which violate single > responsibility principle, and it not easy for extension, a good separation of > responsibilities is a recommended way. > # Some cluster level operation has no atomicity guarantee, we should > implement then in idempotent way to achieve final consistency > # Resubmit when job was discarded > # Clarify overall design for realtime OLAP > > h4. StreamingCoordinator > Facade of coordinator, will controll BuildJobSummitter/ReceiverClusterMangaer > and delegate operation to them. > h4. BuildJobSubmitter > The main responsibility of BuildJobSubmitter including: > 1. Try to find candidate segment which ready to submit a build job > 2. Trace the status of candidate segment's build job and promote segment if > it is has met requirements > h4. > h4. ReceiverClusterManager > This class manage operation related to multi streaming receivers. They are > often not atomic and maybe idempotent. > h4. ClusterStateChecker > Basic step of this class: > 1. stop/pause coordinator to avoid underlying concurrency issue > 2. check inconsistent state of all receiver cluster > 3. send summary via mail to kylin admin > 4. if need, call ClusterDoctor to repair inconsistent issue > h4. ClusterDoctor > Repair inconsistent state according to result of ClusterStateChecker > > > h3. Candidate Segment > The candidate segments are those segments what can be saw/perceived by > streaming coordinator, > candidate segment could be divided into following state/queue: > 1. segment which data are uploaded *PARTLY* > 2. segment which data are uploaded completely and *WAITING* to build > 3. segment which in *BUILDING* state, job's state should be one of > (NEW/RUNNING/ERROR/DISCARD) > 4. segment which built *succeed* and wait to be delivered to historical part > (and to be deleted in realtime part) > 5. segment which *in historical part*(HBase Ready Segment) > > By design, segment should transfer to next queue in sequential way(shouldn't > jump the queue), do not break this. > h3. Atomicity > In a multi-step transcation, following acepts should be thought twice: > 1. should *fail fast* or continue when exception thrown. > 2. should API(remote call) be *synchronous* or asynchronous > 3. when transcation failed, could *roll back* always succeed > 4. transcation should be *idempotent* so when it failed, it could be fixed by > retry > > How to ensure whole cluster opreation smoothly without blocking problem. I > divided all multi-step transcation into three kinds: > NotAtomicIdempotent > NotAtomicAndNotIdempotent > NonSideEffect -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (KYLIN-4046) Refine JDBC Source(source.default=8)
[ https://issues.apache.org/jira/browse/KYLIN-4046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen reopened KYLIN-4046: - > Refine JDBC Source(source.default=8) > > > Key: KYLIN-4046 > URL: https://issues.apache.org/jira/browse/KYLIN-4046 > Project: Kylin > Issue Type: Bug > Components: RDBMS Source >Reporter: Xiaoxiang Yu >Assignee: Xiaoxiang Yu >Priority: Critical > Fix For: v3.0.0-alpha2 > > > Currently, the function of ingest from RDBMS(kylin.source.default=8) to Kylin > has some problem such as: > # case-sensitive > # weak dialect compatibility > # mis-use quote character > We need to fix this problem in kylin.source.default=8. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4046) Refine JDBC Source(source.default=8)
[ https://issues.apache.org/jira/browse/KYLIN-4046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4046: Fix Version/s: v2.6.4 > Refine JDBC Source(source.default=8) > > > Key: KYLIN-4046 > URL: https://issues.apache.org/jira/browse/KYLIN-4046 > Project: Kylin > Issue Type: Bug > Components: RDBMS Source >Reporter: Xiaoxiang Yu >Assignee: Xiaoxiang Yu >Priority: Critical > Fix For: v3.0.0-alpha2, v2.6.4 > > > Currently, the function of ingest from RDBMS(kylin.source.default=8) to Kylin > has some problem such as: > # case-sensitive > # weak dialect compatibility > # mis-use quote character > We need to fix this problem in kylin.source.default=8. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4046) Refine JDBC Source(source.default=8)
[ https://issues.apache.org/jira/browse/KYLIN-4046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4046. - Resolution: Fixed > Refine JDBC Source(source.default=8) > > > Key: KYLIN-4046 > URL: https://issues.apache.org/jira/browse/KYLIN-4046 > Project: Kylin > Issue Type: Bug > Components: RDBMS Source >Reporter: Xiaoxiang Yu >Assignee: Xiaoxiang Yu >Priority: Critical > Fix For: v3.0.0-alpha2, v2.6.4 > > > Currently, the function of ingest from RDBMS(kylin.source.default=8) to Kylin > has some problem such as: > # case-sensitive > # weak dialect compatibility > # mis-use quote character > We need to fix this problem in kylin.source.default=8. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4047) Use push-down query when division dynamic column cube query is not supported
[
https://issues.apache.org/jira/browse/KYLIN-4047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4047.
-
Resolution: Fixed
> Use push-down query when division dynamic column cube query is not supported
>
>
> Key: KYLIN-4047
> URL: https://issues.apache.org/jira/browse/KYLIN-4047
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Affects Versions: Future, v2.6.2, v3.0.0-alpha
>Reporter: wangxiaojing
>Priority: Minor
> Fix For: v3.0.0-beta, v2.6.4
>
>
> Hello all ,
> At present, if a table has created cube, the query with division
> dynamic column will throw an exception without pushdown query results, but if
> switching an project, pushdown query will be carried out, which will give
> users inconsistent experience. Can we also use pushdown query when query
> division, multiplication dynamic column throw exceptions ?
> Scene：
> SQL:select round (car_dismatch_cnt*100/answer_order_cnt, 4) from db.table1.
> 1、Executes the SQL on project where has create a cube has db.table1,it will
> throws the right side of the BinaryTuple Expressions columns is not supported
> for / exception and is not queried through pushdown;
> 2、When change a project where not create the table cube , the same query can
> be executed to push down and get the right result.
> 3. Expect that no matter whether a table cube is created or not, when the
> cube query is not success, the result can still be obtained by pushdown
> query, which will give users a consistent experience.
> {code:java}
> // exception
> 2019-06-06 19:04:55,902 ERROR [http-bio-8081-exec-1]
> controller.BasicController:63 :
> org.apache.kylin.rest.exception.InternalErrorException: That the right side
> of the BinaryTupleExpression owns columns is not supported for /
> while executing SQL: "select DT,ORD_AMT/ORD_AMT from db.table1 LIMIT 5"
> at
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:452)
> at
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:373)
> at
> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:87)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
> at
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
> at
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
> at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
> at
> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
> at
> org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$V
[jira] [Updated] (KYLIN-4047) Use push-down query when division dynamic column cube query is not supported
[
https://issues.apache.org/jira/browse/KYLIN-4047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4047:
Fix Version/s: v3.0.0-beta
> Use push-down query when division dynamic column cube query is not supported
>
>
> Key: KYLIN-4047
> URL: https://issues.apache.org/jira/browse/KYLIN-4047
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Affects Versions: Future, v2.6.2, v3.0.0-alpha
>Reporter: wangxiaojing
>Priority: Minor
> Fix For: v3.0.0-beta, v2.6.4
>
>
> Hello all ,
> At present, if a table has created cube, the query with division
> dynamic column will throw an exception without pushdown query results, but if
> switching an project, pushdown query will be carried out, which will give
> users inconsistent experience. Can we also use pushdown query when query
> division, multiplication dynamic column throw exceptions ?
> Scene：
> SQL:select round (car_dismatch_cnt*100/answer_order_cnt, 4) from db.table1.
> 1、Executes the SQL on project where has create a cube has db.table1,it will
> throws the right side of the BinaryTuple Expressions columns is not supported
> for / exception and is not queried through pushdown;
> 2、When change a project where not create the table cube , the same query can
> be executed to push down and get the right result.
> 3. Expect that no matter whether a table cube is created or not, when the
> cube query is not success, the result can still be obtained by pushdown
> query, which will give users a consistent experience.
> {code:java}
> // exception
> 2019-06-06 19:04:55,902 ERROR [http-bio-8081-exec-1]
> controller.BasicController:63 :
> org.apache.kylin.rest.exception.InternalErrorException: That the right side
> of the BinaryTupleExpression owns columns is not supported for /
> while executing SQL: "select DT,ORD_AMT/ORD_AMT from db.table1 LIMIT 5"
> at
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:452)
> at
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:373)
> at
> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:87)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
> at
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
> at
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
> at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
> at
> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
> at
> org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChain
[jira] [Resolved] (KYLIN-4013) Only show the cubes under one model
[ https://issues.apache.org/jira/browse/KYLIN-4013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4013. - Resolution: Fixed > Only show the cubes under one model > --- > > Key: KYLIN-4013 > URL: https://issues.apache.org/jira/browse/KYLIN-4013 > Project: Kylin > Issue Type: Improvement > Components: Web >Affects Versions: v2.5.2 >Reporter: Yuzhang QIU >Assignee: Yuzhang QIU >Priority: Major > Fix For: v3.0.0-alpha2, v2.6.4 > > > Some improvement for UI。 > User may want to see the cubes under specified model. Add an extra action > 'Cubes' in the drop-down list of model to filter cube list -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4047) Use push-down query when division dynamic column cube query is not supported
[
https://issues.apache.org/jira/browse/KYLIN-4047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4047:
Fix Version/s: (was: v3.0.0-beta)
v3.0.0-alpha2
> Use push-down query when division dynamic column cube query is not supported
>
>
> Key: KYLIN-4047
> URL: https://issues.apache.org/jira/browse/KYLIN-4047
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Affects Versions: Future, v2.6.2, v3.0.0-alpha
>Reporter: wangxiaojing
>Priority: Minor
> Fix For: v3.0.0-alpha2, v2.6.4
>
>
> Hello all ,
> At present, if a table has created cube, the query with division
> dynamic column will throw an exception without pushdown query results, but if
> switching an project, pushdown query will be carried out, which will give
> users inconsistent experience. Can we also use pushdown query when query
> division, multiplication dynamic column throw exceptions ?
> Scene：
> SQL:select round (car_dismatch_cnt*100/answer_order_cnt, 4) from db.table1.
> 1、Executes the SQL on project where has create a cube has db.table1,it will
> throws the right side of the BinaryTuple Expressions columns is not supported
> for / exception and is not queried through pushdown;
> 2、When change a project where not create the table cube , the same query can
> be executed to push down and get the right result.
> 3. Expect that no matter whether a table cube is created or not, when the
> cube query is not success, the result can still be obtained by pushdown
> query, which will give users a consistent experience.
> {code:java}
> // exception
> 2019-06-06 19:04:55,902 ERROR [http-bio-8081-exec-1]
> controller.BasicController:63 :
> org.apache.kylin.rest.exception.InternalErrorException: That the right side
> of the BinaryTupleExpression owns columns is not supported for /
> while executing SQL: "select DT,ORD_AMT/ORD_AMT from db.table1 LIMIT 5"
> at
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:452)
> at
> org.apache.kylin.rest.service.QueryService.doQueryWithCache(QueryService.java:373)
> at
> org.apache.kylin.rest.controller.QueryController.query(QueryController.java:87)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
> at
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
> at
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
> at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
> at
> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
> at
> org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
[jira] [Updated] (KYLIN-1856) Kylin shows old error in job step output after resume - specifically in #4 Step Name: Build Dimension Dictionary
[ https://issues.apache.org/jira/browse/KYLIN-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-1856: Fix Version/s: v2.6.4 > Kylin shows old error in job step output after resume - specifically in #4 > Step Name: Build Dimension Dictionary > > > Key: KYLIN-1856 > URL: https://issues.apache.org/jira/browse/KYLIN-1856 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.5.2, v1.5.2.1 >Reporter: Richard Calaba >Assignee: Yaqian Zhang >Priority: Minor > Fix For: v3.0.0-beta, v2.6.4 > > > I have realized that if my job stops with error and I try to recover the > error and resume the job - then the latest step starts again from scratch. > This is fine but in my opinion the log of the Step should clear as well - now > it is showing the error from my previous attempt. > Specifically observed in #4 Step Name: Build Dimension Dictionary - but is > probbaly generic issue. > To correct this: clear the log of the Build Step after the job Step is > resumed. Already when the job step is restarted, not after it is completed. > (if Kylin fails i.e. for out of memory - it silently dies and analyzing the > step log shows wrong error (from previous run) - if it would be empty -> I > would know that most probable cause was that Kylin died) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4057) autoMerge job can not stop
[ https://issues.apache.org/jira/browse/KYLIN-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4057: Fix Version/s: v2.6.4 > autoMerge job can not stop > -- > > Key: KYLIN-4057 > URL: https://issues.apache.org/jira/browse/KYLIN-4057 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.6.2 >Reporter: chenchen >Assignee: wangrupeng >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > Attachments: 5031561517158_.pic_hd.jpg > > > In this version，I used DistributedScheduler。 > In my case, A cube config autoMerge, But the job that automatically merges > segments is wrong ，I can't fix this error, I can only discard and drop the > job, but Kylin's monitoring mechanism creates the same new job. > I want to ask if there is any way to fix this problem without banning cube. > The error is reported as follows. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4093) Slow query pages should be open to all users of the project
[ https://issues.apache.org/jira/browse/KYLIN-4093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4093: Fix Version/s: v3.0.0-beta > Slow query pages should be open to all users of the project > --- > > Key: KYLIN-4093 > URL: https://issues.apache.org/jira/browse/KYLIN-4093 > Project: Kylin > Issue Type: Improvement >Reporter: Shaohui Liu >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > > Currently the show query page only can been seen for kylin admins. It's very > useful for modlers and analysts for this project. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4121) Cleanup hive view intermediate tables after job be finished
[ https://issues.apache.org/jira/browse/KYLIN-4121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4121: Fix Version/s: v2.6.4 > Cleanup hive view intermediate tables after job be finished > --- > > Key: KYLIN-4121 > URL: https://issues.apache.org/jira/browse/KYLIN-4121 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Shao Feng Shi >Assignee: wangrupeng >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > Attachments: image-2019-09-16-19-18-59-663.png, > image-2019-09-16-19-26-31-446.png > > > Reported by community user: > I have a cube with a fact table join a lookup table in hive, and both are > hive view. I submit a job one time per hour. > > Kylin can drop the intermediate fact table , but doesn't drop the > intermediate lookup table . > > I check the source code , and find out that , at '13 step: Hive Cleanup' , > the source code has been commented. > > > It is a legacy issue. Now KYLIN-3515 has fixed that, so the cleanup can be > enabled. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4093) Slow query pages should be open to all users of the project
[ https://issues.apache.org/jira/browse/KYLIN-4093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4093. - Resolution: Fixed > Slow query pages should be open to all users of the project > --- > > Key: KYLIN-4093 > URL: https://issues.apache.org/jira/browse/KYLIN-4093 > Project: Kylin > Issue Type: Improvement >Reporter: Shaohui Liu >Priority: Major > Fix For: v3.0.0-beta, v2.6.4 > > > Currently the show query page only can been seen for kylin admins. It's very > useful for modlers and analysts for this project. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4053) Compare date with timestamp
[
https://issues.apache.org/jira/browse/KYLIN-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4053:
Fix Version/s: v3.0.0-beta
> Compare date with timestamp
> ---
>
> Key: KYLIN-4053
> URL: https://issues.apache.org/jira/browse/KYLIN-4053
> Project: Kylin
> Issue Type: Sub-task
>Affects Versions: Future
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Fix For: v3.0.0-beta
>
>
> The following SQL will fail.
> {code:sql}
> SELECT date '2019-06-16' >= timestamp '2019-06-16 00:00:00'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (KYLIN-4188) Parquet as Cube storage V2
nichunen created KYLIN-4188: --- Summary: Parquet as Cube storage V2 Key: KYLIN-4188 URL: https://issues.apache.org/jira/browse/KYLIN-4188 Project: Kylin Issue Type: New Feature Components: Storage - Parquet Reporter: nichunen Assignee: yiming.xu Fix For: Future Branch "kylin-on-parquet" has implemented and it's ready for public review and evaluation, the issue is https://issues.apache.org/jira/browse/KYLIN-3621. However, as the discussion in https://sematext.com/opensee/m/Kylin/8WImho86LC1RxadI?subj=Evaluate+Kylin+on+Parquet, to make it more stable and with higher performance, some tasks of improvement should be done. The key tasks may be: 1. Calculation of queries on Kylin should be all distributed 2. Remove dictionaries for dimensions 3. Store value with its data type(not binary) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-1242) support building the back end and the front end seperately
[ https://issues.apache.org/jira/browse/KYLIN-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-1242. --- Resolution: Done > support building the back end and the front end seperately > -- > > Key: KYLIN-1242 > URL: https://issues.apache.org/jira/browse/KYLIN-1242 > Project: Kylin > Issue Type: Improvement > Components: Others >Affects Versions: v1.1.1 >Reporter: Wang,Dong >Assignee: Hongbin Ma >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-1484) Migrate KYLIN-1483 to web UI with potentially more intuitive graph visualization
[ https://issues.apache.org/jira/browse/KYLIN-1484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-1484. --- Resolution: Fixed It is done with the feature of Cube Planner > Migrate KYLIN-1483 to web UI with potentially more intuitive graph > visualization > > > Key: KYLIN-1484 > URL: https://issues.apache.org/jira/browse/KYLIN-1484 > Project: Kylin > Issue Type: New Feature > Components: Client - CLI >Affects Versions: v1.5.0 >Reporter: Hongbin Ma >Assignee: Jason Zhong >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-1813) intermediate table in Hive not cleaned up
[ https://issues.apache.org/jira/browse/KYLIN-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-1813. --- Resolution: Cannot Reproduce Tested on 2.6.3, not exist. > intermediate table in Hive not cleaned up > - > > Key: KYLIN-1813 > URL: https://issues.apache.org/jira/browse/KYLIN-1813 > Project: Kylin > Issue Type: Bug > Components: Others >Affects Versions: v1.5.2 > Environment: linux >Reporter: Jie Tao >Assignee: Jason Zhong >Priority: Minor > Attachments: hive.png, jobs.png, json.txt > > > after the cube building with ERROR I discarded the cube, but the cube is not > marked as discarded like other jobs that I discarded during the cube building > was running (see job.png: the job is not black colour marked but has only one > choice "diagnose"). Hence, as I manually clean up intermediate data, I got > info like: > Remove intermediate hive table with job id > 493fd20b-3074-403e-9963-fe4fb7ff7c65 with job status ERROR > 2016-06-17 09:37:12,648 INFO [main StorageCleanupJob:262]: Remove > intermediate hive table with job id 8a377e30-e3ba-4fe2-be12-e7d412afec5ewith > job status ERROR > In this case the intermediate tables were not removed from hive (see > hive.png). In addition, in my Hive other intermediate tables exist that can > not be cleaned up. I feel taht one intermediate table was kept in Hive for > each cube building. Not sure if they are still usefull. The json is also > attached json.txt -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-1573) aggregator expression like sum(a+b) is not working
[ https://issues.apache.org/jira/browse/KYLIN-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-1573. --- Resolution: Fixed Sum(expr) is supported after the fix of KYLIN-3359, now supported after version 2.4.0 > aggregator expression like sum(a+b) is not working > -- > > Key: KYLIN-1573 > URL: https://issues.apache.org/jira/browse/KYLIN-1573 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: 1.x-HBase1.1.3 >Reporter: Babulal >Assignee: liyang >Priority: Minor > Labels: newbie, test > > In Kylin expression like + ,/.* ..ect are not working . > for example > select country, >city, > sum(male_population+Female_population) > > from population > group by country,city. > Error > Caused by: java.lang.NullPointerException > at > org.apache.kylin.cube.CubeCapabilityChecker.tryDimensionAsMeasures(CubeCapabilityChecker.java:170) > at > org.apache.kylin.cube.CubeCapabilityChecker.check(CubeCapabilityChecker.java:73) > at org.apache.kylin.cube.CubeInstance.isCapable(CubeInstance.java:330) > at > org.apache.kylin.query.routing.rules.RemoveUncapableRealizationsRule.apply(RemoveUncapableRealizationsRule.java:36) > at > org.apache.kylin.query.routing.RoutingRule.applyRules(RoutingRule.java:47) > at > org.apache.kylin.query.routing.QueryRouter.selectRealization(QueryRouter.java:63) > at > org.apache.kylin.query.relnode.OLAPToEnumerableConverter.implement(OLAPToEnumerableConverter.java:80) > at > org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:102) > at > org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:92) > at > org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1171) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:297) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:196) > When Query changed as below it is working fine . > select country, >city, > sum(male_population)+sum(Female_population) > > from population > group by country,city. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-3075) Improvement kylin.sh, remove duplicate dependency check, let kylin start faster.
[ https://issues.apache.org/jira/browse/KYLIN-3075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-3075. - Resolution: Fixed Resolved by KYLIN-4028 > Improvement kylin.sh, remove duplicate dependency check, let kylin start > faster. > > > Key: KYLIN-3075 > URL: https://issues.apache.org/jira/browse/KYLIN-3075 > Project: Kylin > Issue Type: Improvement >Reporter: peng.jianhua >Assignee: peng.jianhua >Priority: Minor > Attachments: > 0001-KYLIN-3075-Improvement-kylin.sh-remove-duplicate-dep.patch > > > Improvement kylin.sh, remove duplicate dependency check, let kylin start > faster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3075) Improvement kylin.sh, remove duplicate dependency check, let kylin start faster.
[ https://issues.apache.org/jira/browse/KYLIN-3075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-3075. --- > Improvement kylin.sh, remove duplicate dependency check, let kylin start > faster. > > > Key: KYLIN-3075 > URL: https://issues.apache.org/jira/browse/KYLIN-3075 > Project: Kylin > Issue Type: Improvement >Reporter: peng.jianhua >Assignee: peng.jianhua >Priority: Minor > Attachments: > 0001-KYLIN-3075-Improvement-kylin.sh-remove-duplicate-dep.patch > > > Improvement kylin.sh, remove duplicate dependency check, let kylin start > faster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4180) Prevent abnormal CPU usage by limiting flat filters length
[
https://issues.apache.org/jira/browse/KYLIN-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4180:
Fix Version/s: v2.6.5
v3.0.0-beta
> Prevent abnormal CPU usage by limiting flat filters length
> --
>
> Key: KYLIN-4180
> URL: https://issues.apache.org/jira/browse/KYLIN-4180
> Project: Kylin
> Issue Type: Improvement
> Components: Metadata
>Affects Versions: v2.4.0, v2.5.0, v2.6.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Fix For: v3.0.0-beta, v2.6.5
>
>
> The following SQL will lead to the list returned by
> org.apache.kylin.metadata.filter.TupleFilter#cartesianProduct become very
> large, and the process to producing the List will use lots of CPU.
> {code:sql}
> select * from temp.test_filter
> where ((a >= 0 and a <= 1)
> or(a >= 1 and a <= 2)
> or(a >= 2 and a <= 3)
> ...
> or(a >= 300 and a <= 301)
> ) and ((b >= 0 and b <= 1)
> or(b >= 1 and b <= 2)
> or(b >= 2 and b <= 3)
> ...
> or(b >= 300 and b <= 301)
> )
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (KYLIN-4072) CDH 6.x find-hbase-dependency.sh return with "base-common lib not found"
[ https://issues.apache.org/jira/browse/KYLIN-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen reopened KYLIN-4072: - > CDH 6.x find-hbase-dependency.sh return with "base-common lib not found" > > > Key: KYLIN-4072 > URL: https://issues.apache.org/jira/browse/KYLIN-4072 > Project: Kylin > Issue Type: Bug >Affects Versions: all >Reporter: Xiaoxiang Yu >Priority: Major > Fix For: v3.0.0-alpha2 > > Attachments: image-2019-07-04-17-36-50-112.png, > image-2019-07-04-17-38-40-660.png, image-2019-07-04-17-39-56-973.png > > > In CDH 6.x, I find that execute "sh bin/find-hbase-dependency.sh" can not > find hbase dependency as expected. > > !image-2019-07-04-17-38-40-660.png! > !image-2019-07-04-17-36-50-112.png! > > And I compare the output of "hbase classpath" of different CDH version, I > find the root cause should be the change of the "hbase classpath". That is to > say, newer CDH didn't absolute path of all jars, only parent path. > !image-2019-07-04-17-39-56-973.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4072) CDH 6.x find-hbase-dependency.sh return with "base-common lib not found"
[ https://issues.apache.org/jira/browse/KYLIN-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4072: Fix Version/s: v2.6.5 > CDH 6.x find-hbase-dependency.sh return with "base-common lib not found" > > > Key: KYLIN-4072 > URL: https://issues.apache.org/jira/browse/KYLIN-4072 > Project: Kylin > Issue Type: Bug >Affects Versions: all >Reporter: Xiaoxiang Yu >Priority: Major > Fix For: v3.0.0-alpha2, v2.6.5 > > Attachments: image-2019-07-04-17-36-50-112.png, > image-2019-07-04-17-38-40-660.png, image-2019-07-04-17-39-56-973.png > > > In CDH 6.x, I find that execute "sh bin/find-hbase-dependency.sh" can not > find hbase dependency as expected. > > !image-2019-07-04-17-38-40-660.png! > !image-2019-07-04-17-36-50-112.png! > > And I compare the output of "hbase classpath" of different CDH version, I > find the root cause should be the change of the "hbase classpath". That is to > say, newer CDH didn't absolute path of all jars, only parent path. > !image-2019-07-04-17-39-56-973.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4180) Prevent abnormal CPU usage by limiting flat filters length
[
https://issues.apache.org/jira/browse/KYLIN-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4180:
Description:
strong textThe following SQL will lead to the list returned by
org.apache.kylin.metadata.filter.TupleFilter#cartesianProduct become very
large, and the process to producing the List will use lots of CPU.
{code:sql}
select * from temp.test_filter
where ((a >= 0 and a <= 1)
or(a >= 1 and a <= 2)
or(a >= 2 and a <= 3)
...
or(a >= 300 and a <= 301)
) and ((b >= 0 and b <= 1)
or(b >= 1 and b <= 2)
or(b >= 2 and b <= 3)
...
or(b >= 300 and b <= 301)
)
{code}
was:
The following SQL will lead to the list returned by
org.apache.kylin.metadata.filter.TupleFilter#cartesianProduct become very
large, and the process to producing the List will use lots of CPU.
{code:sql}
select * from temp.test_filter
where ((a >= 0 and a <= 1)
or(a >= 1 and a <= 2)
or(a >= 2 and a <= 3)
...
or(a >= 300 and a <= 301)
) and ((b >= 0 and b <= 1)
or(b >= 1 and b <= 2)
or(b >= 2 and b <= 3)
...
or(b >= 300 and b <= 301)
)
{code}
> Prevent abnormal CPU usage by limiting flat filters length
> --
>
> Key: KYLIN-4180
> URL: https://issues.apache.org/jira/browse/KYLIN-4180
> Project: Kylin
> Issue Type: Improvement
> Components: Metadata
>Affects Versions: v2.4.0, v2.5.0, v2.6.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Fix For: v3.0.0-beta, v2.6.5
>
>
> strong textThe following SQL will lead to the list returned by
> org.apache.kylin.metadata.filter.TupleFilter#cartesianProduct become very
> large, and the process to producing the List will use lots of CPU.
> {code:sql}
> select * from temp.test_filter
> where ((a >= 0 and a <= 1)
> or(a >= 1 and a <= 2)
> or(a >= 2 and a <= 3)
> ...
> or(a >= 300 and a <= 301)
> ) and ((b >= 0 and b <= 1)
> or(b >= 1 and b <= 2)
> or(b >= 2 and b <= 3)
> ...
> or(b >= 300 and b <= 301)
> )
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (KYLIN-4193) More user-friendly page for loading streaming tables
nichunen created KYLIN-4193: --- Summary: More user-friendly page for loading streaming tables Key: KYLIN-4193 URL: https://issues.apache.org/jira/browse/KYLIN-4193 Project: Kylin Issue Type: Improvement Components: Real-time Streaming Affects Versions: v3.0.0-alpha Reporter: nichunen Fix For: v3.0.0 After click "Add Streaming Table V2", the user has to set "TSParser" and "TSPattern", these items may confuse users, they should be made more user-friendly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4165) RT OLAP building job on "Save Cube Dictionaries" step concurrency error
[
https://issues.apache.org/jira/browse/KYLIN-4165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16950002#comment-16950002
]
nichunen commented on KYLIN-4165:
-
+1
> RT OLAP building job on "Save Cube Dictionaries" step concurrency error
> ---
>
> Key: KYLIN-4165
> URL: https://issues.apache.org/jira/browse/KYLIN-4165
> Project: Kylin
> Issue Type: Bug
> Components: Real-time Streaming
>Affects Versions: v3.0.0-alpha
>Reporter: wangxiaojing
>Priority: Major
>
> There is a dictionary version conflict in "Save Cube Dictionaries" step when
> build the realtime fsegment from remote persisted to reday,Which is very
> serious,it will lead to unsuccessful updating of dictionaries by multiple
> jobs concurrently.This may occurs when a cube has many concurrent building
> jobs one the same step ——”Save Cube Dictionaries“ .
> Perhaps a globally distributed lock is needed to avoid one cube concurrency
> running of this step .
> Save Cube Dictionaries log messages:
> {code:java}
> // code placeholder
> org.apache.kylin.common.persistence.WriteConflictException: Overwriting
> conflict
> /dict/DEFAULT.TASK_SNAPSHOT/GROUPVALUE/5387e747-9649-0b17-5a72-ee17f5baea0a.dict,
> expect old TS 1568012509090, but it is 1568012509245at
> org.apache.kylin.storage.hbase.HBaseResourceStore.updateTimestampImpl(HBaseResourceStore.java:372)
> at
> org.apache.kylin.common.persistence.ResourceStore$7.call(ResourceStore.java:465)
> at
> org.apache.kylin.common.persistence.ExponentialBackoffRetry.doWithRetry(ExponentialBackoffRetry.java:52)
> at
> org.apache.kylin.common.persistence.ResourceStore.updateTimestampWithRetry(ResourceStore.java:462)
> at
> org.apache.kylin.common.persistence.ResourceStore.updateTimestampCheckPoint(ResourceStore.java:457)
> at
> org.apache.kylin.common.persistence.ResourceStore.updateTimestamp(ResourceStore.java:452)
> at
> org.apache.kylin.dict.DictionaryManager.updateExistingDictLastModifiedTime(DictionaryManager.java:197)
> at
> org.apache.kylin.dict.DictionaryManager.trySaveNewDict(DictionaryManager.java:157)
> at
> org.apache.kylin.engine.mr.streaming.SaveDictStep.doWork(SaveDictStep.java:122)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
> at
> org.apache.kylin.job.impl.threadpool.DistributedScheduler$JobRunner.run(DistributedScheduler.java:110)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4165) RT OLAP building job on "Save Cube Dictionaries" step concurrency error
[
https://issues.apache.org/jira/browse/KYLIN-4165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4165:
Fix Version/s: v3.0.0
> RT OLAP building job on "Save Cube Dictionaries" step concurrency error
> ---
>
> Key: KYLIN-4165
> URL: https://issues.apache.org/jira/browse/KYLIN-4165
> Project: Kylin
> Issue Type: Bug
> Components: Real-time Streaming
>Affects Versions: v3.0.0-alpha
>Reporter: wangxiaojing
>Priority: Major
> Fix For: v3.0.0
>
>
> There is a dictionary version conflict in "Save Cube Dictionaries" step when
> build the realtime fsegment from remote persisted to reday,Which is very
> serious,it will lead to unsuccessful updating of dictionaries by multiple
> jobs concurrently.This may occurs when a cube has many concurrent building
> jobs one the same step ——”Save Cube Dictionaries“ .
> Perhaps a globally distributed lock is needed to avoid one cube concurrency
> running of this step .
> Save Cube Dictionaries log messages:
> {code:java}
> // code placeholder
> org.apache.kylin.common.persistence.WriteConflictException: Overwriting
> conflict
> /dict/DEFAULT.TASK_SNAPSHOT/GROUPVALUE/5387e747-9649-0b17-5a72-ee17f5baea0a.dict,
> expect old TS 1568012509090, but it is 1568012509245at
> org.apache.kylin.storage.hbase.HBaseResourceStore.updateTimestampImpl(HBaseResourceStore.java:372)
> at
> org.apache.kylin.common.persistence.ResourceStore$7.call(ResourceStore.java:465)
> at
> org.apache.kylin.common.persistence.ExponentialBackoffRetry.doWithRetry(ExponentialBackoffRetry.java:52)
> at
> org.apache.kylin.common.persistence.ResourceStore.updateTimestampWithRetry(ResourceStore.java:462)
> at
> org.apache.kylin.common.persistence.ResourceStore.updateTimestampCheckPoint(ResourceStore.java:457)
> at
> org.apache.kylin.common.persistence.ResourceStore.updateTimestamp(ResourceStore.java:452)
> at
> org.apache.kylin.dict.DictionaryManager.updateExistingDictLastModifiedTime(DictionaryManager.java:197)
> at
> org.apache.kylin.dict.DictionaryManager.trySaveNewDict(DictionaryManager.java:157)
> at
> org.apache.kylin.engine.mr.streaming.SaveDictStep.doWork(SaveDictStep.java:122)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
> at
> org.apache.kylin.job.impl.threadpool.DistributedScheduler$JobRunner.run(DistributedScheduler.java:110)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4053) Compare date with timestamp
[
https://issues.apache.org/jira/browse/KYLIN-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4053:
Fix Version/s: (was: v3.0.0-beta)
v3.0.0
> Compare date with timestamp
> ---
>
> Key: KYLIN-4053
> URL: https://issues.apache.org/jira/browse/KYLIN-4053
> Project: Kylin
> Issue Type: Sub-task
>Affects Versions: Future
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Fix For: v3.0.0
>
>
> The following SQL will fail.
> {code:sql}
> SELECT date '2019-06-16' >= timestamp '2019-06-16 00:00:00'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4053) Compare date with timestamp
[
https://issues.apache.org/jira/browse/KYLIN-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16951772#comment-16951772
]
nichunen commented on KYLIN-4053:
-
Hi [~jiezouSH] Would you please upgrade the calcite package?
> Compare date with timestamp
> ---
>
> Key: KYLIN-4053
> URL: https://issues.apache.org/jira/browse/KYLIN-4053
> Project: Kylin
> Issue Type: Sub-task
>Affects Versions: Future
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Fix For: v3.0.0
>
>
> The following SQL will fail.
> {code:sql}
> SELECT date '2019-06-16' >= timestamp '2019-06-16 00:00:00'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-1671) Date dimension cannot be formatted in tableau
[ https://issues.apache.org/jira/browse/KYLIN-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-1671. - Fix Version/s: v3.0.0-beta Resolution: Fixed > Date dimension cannot be formatted in tableau > - > > Key: KYLIN-1671 > URL: https://issues.apache.org/jira/browse/KYLIN-1671 > Project: Kylin > Issue Type: Bug > Components: Driver - ODBC >Reporter: Dong Li >Assignee: Dong Li >Priority: Minor > Fix For: v3.0.0-beta > > Attachments: Screen Shot 2016-05-10 at 2.21.25 PM.png, datevalues.png > > > In this article: > http://vizpainter.com/dates-in-tableau-make-me-want-to-tear-my-hair-out/ > We know that there should be a DATE VALUES section in right-click context > menu of Date Dimension columns. > But in kylin sample data, this section is empty, like attached snapshot. > Currently there're 2 workarounds: > 1. Leverage tableau's multi-level date columns > 2. Set column as discrete, and set the format as: > right click the column -> Default properties -> Date format -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-1671) Date dimension cannot be formatted in tableau
[ https://issues.apache.org/jira/browse/KYLIN-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-1671: Fix Version/s: (was: v3.0.0-beta) v3.0.0-alpha2 > Date dimension cannot be formatted in tableau > - > > Key: KYLIN-1671 > URL: https://issues.apache.org/jira/browse/KYLIN-1671 > Project: Kylin > Issue Type: Bug > Components: Driver - ODBC >Reporter: Dong Li >Assignee: Dong Li >Priority: Minor > Fix For: v3.0.0-alpha2 > > Attachments: Screen Shot 2016-05-10 at 2.21.25 PM.png, datevalues.png > > > In this article: > http://vizpainter.com/dates-in-tableau-make-me-want-to-tear-my-hair-out/ > We know that there should be a DATE VALUES section in right-click context > menu of Date Dimension columns. > But in kylin sample data, this section is empty, like attached snapshot. > Currently there're 2 workarounds: > 1. Leverage tableau's multi-level date columns > 2. Set column as discrete, and set the format as: > right click the column -> Default properties -> Date format -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-1671) Date dimension cannot be formatted in tableau
[ https://issues.apache.org/jira/browse/KYLIN-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-1671. --- > Date dimension cannot be formatted in tableau > - > > Key: KYLIN-1671 > URL: https://issues.apache.org/jira/browse/KYLIN-1671 > Project: Kylin > Issue Type: Bug > Components: Driver - ODBC >Reporter: Dong Li >Assignee: Dong Li >Priority: Minor > Fix For: v3.0.0-alpha2 > > Attachments: Screen Shot 2016-05-10 at 2.21.25 PM.png, datevalues.png > > > In this article: > http://vizpainter.com/dates-in-tableau-make-me-want-to-tear-my-hair-out/ > We know that there should be a DATE VALUES section in right-click context > menu of Date Dimension columns. > But in kylin sample data, this section is empty, like attached snapshot. > Currently there're 2 workarounds: > 1. Leverage tableau's multi-level date columns > 2. Set column as discrete, and set the format as: > right click the column -> Default properties -> Date format -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-3917) Add max segment merge span to cleanup intermediate data of cube building
[ https://issues.apache.org/jira/browse/KYLIN-3917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-3917: Fix Version/s: (was: Future) v3.0.0 > Add max segment merge span to cleanup intermediate data of cube building > - > > Key: KYLIN-3917 > URL: https://issues.apache.org/jira/browse/KYLIN-3917 > Project: Kylin > Issue Type: Improvement >Reporter: Shaohui Liu >Assignee: Shaohui Liu >Priority: Major > Fix For: v3.0.0 > > > Currently the intermediate data of building cube can not be deleted for it > maybe used for later cubing merging. But it result in double space used in > HDFS. > In actual scenario, we only need month-level segment span in maximum. > So if a span of segment is larger than a month, we think it don't need be > merged and the intermediate data can be deleted. > So we can add a config kylin.cube.max-segment-merge.span, default is -1. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-3917) Add max segment merge span to cleanup intermediate data of cube building
[ https://issues.apache.org/jira/browse/KYLIN-3917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-3917. - Resolution: Fixed > Add max segment merge span to cleanup intermediate data of cube building > - > > Key: KYLIN-3917 > URL: https://issues.apache.org/jira/browse/KYLIN-3917 > Project: Kylin > Issue Type: Improvement >Reporter: Shaohui Liu >Assignee: Shaohui Liu >Priority: Major > Fix For: v3.0.0 > > > Currently the intermediate data of building cube can not be deleted for it > maybe used for later cubing merging. But it result in double space used in > HDFS. > In actual scenario, we only need month-level segment span in maximum. > So if a span of segment is larger than a month, we think it don't need be > merged and the intermediate data can be deleted. > So we can add a config kylin.cube.max-segment-merge.span, default is -1. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4162) After drop the build task on the monitor page, subsequent segments cannot be constructed.
[ https://issues.apache.org/jira/browse/KYLIN-4162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4162: Fix Version/s: (was: v3.0.0-beta) v3.0.0 > After drop the build task on the monitor page, subsequent segments cannot be > constructed. > - > > Key: KYLIN-4162 > URL: https://issues.apache.org/jira/browse/KYLIN-4162 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming >Affects Versions: v3.0.0-alpha2 >Reporter: Yaqian Zhang >Assignee: Yaqian Zhang >Priority: Major > Fix For: v3.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4135) Real time streaming segment build task discard but can't be rebuilt
[ https://issues.apache.org/jira/browse/KYLIN-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4135: Fix Version/s: (was: v3.0.0-beta) v3.0.0 > Real time streaming segment build task discard but can't be rebuilt > > > Key: KYLIN-4135 > URL: https://issues.apache.org/jira/browse/KYLIN-4135 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming >Affects Versions: Future >Reporter: wangxiaojing >Priority: Major > Fix For: v3.0.0 > > > Real time streaming segment build task discard in some case ,but now can't be > rebuilt, and cant't subbmit other new segment tsrange to build if it reach > the cube's max building number( > [https://issues.apache.org/jira/projects/KYLIN/issues/KYLIN-4134?filter=allopenissues]) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4163) CreateFlatHiveTableStep has not yarn app url when hive job running
[
https://issues.apache.org/jira/browse/KYLIN-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4163:
Fix Version/s: v3.0.0
> CreateFlatHiveTableStep has not yarn app url when hive job running
> --
>
> Key: KYLIN-4163
> URL: https://issues.apache.org/jira/browse/KYLIN-4163
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine, Web
>Affects Versions: v3.0.0-alpha
>Reporter: chuxiao
>Priority: Minor
> Fix For: v3.0.0
>
> Attachments: KYLIN-4163.master.001.patch, flathivetablerunning图.jpg
>
>
> CreateFlatHiveTableStep has yarn app url on the monitor web page only when
> job finished, but SparkExecutable has yarn app url when job running.
> this is because of SparkExecutable`s logger has logger listener:
> {code:java}
> final PatternedLogger patternedLogger = new PatternedLogger(logger, new
> PatternedLogger.ILogListener() {
> @Override
> public void onLogEvent(String infoKey, Map info) {
> // only care three properties here
> if (ExecutableConstants.SPARK_JOB_ID.equals(infoKey)
> || ExecutableConstants.YARN_APP_ID.equals(infoKey)
> || ExecutableConstants.YARN_APP_URL.equals(infoKey)) {
> getManager().addJobInfo(getId(), info);
> }
> }
> });{code}
> sometimes creating flat hive table hangs, so user wants to have yarn app url
> when hive job running like attachment.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-4163) CreateFlatHiveTableStep has not yarn app url when hive job running
[
https://issues.apache.org/jira/browse/KYLIN-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4163.
-
Resolution: Fixed
> CreateFlatHiveTableStep has not yarn app url when hive job running
> --
>
> Key: KYLIN-4163
> URL: https://issues.apache.org/jira/browse/KYLIN-4163
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine, Web
>Affects Versions: v3.0.0-alpha
>Reporter: chuxiao
>Priority: Minor
> Fix For: v3.0.0
>
> Attachments: KYLIN-4163.master.001.patch, flathivetablerunning图.jpg
>
>
> CreateFlatHiveTableStep has yarn app url on the monitor web page only when
> job finished, but SparkExecutable has yarn app url when job running.
> this is because of SparkExecutable`s logger has logger listener:
> {code:java}
> final PatternedLogger patternedLogger = new PatternedLogger(logger, new
> PatternedLogger.ILogListener() {
> @Override
> public void onLogEvent(String infoKey, Map info) {
> // only care three properties here
> if (ExecutableConstants.SPARK_JOB_ID.equals(infoKey)
> || ExecutableConstants.YARN_APP_ID.equals(infoKey)
> || ExecutableConstants.YARN_APP_URL.equals(infoKey)) {
> getManager().addJobInfo(getId(), info);
> }
> }
> });{code}
> sometimes creating flat hive table hangs, so user wants to have yarn app url
> when hive job running like attachment.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4169) Too many logs while DataModelManager init, cause the first RESTful API hang for a long time
[
https://issues.apache.org/jira/browse/KYLIN-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-4169:
Fix Version/s: v3.0.0
> Too many logs while DataModelManager init, cause the first RESTful API hang
> for a long time
> ---
>
> Key: KYLIN-4169
> URL: https://issues.apache.org/jira/browse/KYLIN-4169
> Project: Kylin
> Issue Type: Bug
> Components: Metadata
>Reporter: ZhouKang
>Assignee: ZhouKang
>Priority: Major
> Fix For: v3.0.0, v2.6.5
>
>
> Too many logs while DataModelManager init. It is will happend after startup
> or reload metastore.
> In our production enviroment，there are 120,000+ lines.
> {code:java}
> // code placeholder
> grep 'is missing or unloaded yet' kylin.log.8|wc -l
> 121861
> {code}
>
> which cause the first sql query's time up to 40s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-4169) Too many logs while DataModelManager init, cause the first RESTful API hang for a long time
[
https://issues.apache.org/jira/browse/KYLIN-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-4169.
-
Resolution: Fixed
> Too many logs while DataModelManager init, cause the first RESTful API hang
> for a long time
> ---
>
> Key: KYLIN-4169
> URL: https://issues.apache.org/jira/browse/KYLIN-4169
> Project: Kylin
> Issue Type: Bug
> Components: Metadata
>Reporter: ZhouKang
>Assignee: ZhouKang
>Priority: Major
> Fix For: v3.0.0, v2.6.5
>
>
> Too many logs while DataModelManager init. It is will happend after startup
> or reload metastore.
> In our production enviroment，there are 120,000+ lines.
> {code:java}
> // code placeholder
> grep 'is missing or unloaded yet' kylin.log.8|wc -l
> 121861
> {code}
>
> which cause the first sql query's time up to 40s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-3933) Currently replica set related operation need refresh current front-end page
[ https://issues.apache.org/jira/browse/KYLIN-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-3933. - Resolution: Fixed > Currently replica set related operation need refresh current front-end page > --- > > Key: KYLIN-3933 > URL: https://issues.apache.org/jira/browse/KYLIN-3933 > Project: Kylin > Issue Type: Bug > Components: Real-time Streaming, Web >Reporter: Chao Long >Assignee: nichunen >Priority: Minor > Fix For: v3.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4175) Support secondary hbase storage config for hbase cluster migration
[ https://issues.apache.org/jira/browse/KYLIN-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16961756#comment-16961756 ] nichunen commented on KYLIN-4175: - PR merged. Doc needs to be updated too. > Support secondary hbase storage config for hbase cluster migration > -- > > Key: KYLIN-4175 > URL: https://issues.apache.org/jira/browse/KYLIN-4175 > Project: Kylin > Issue Type: Improvement >Reporter: Shaohui Liu >Assignee: Shaohui Liu >Priority: Major > Fix For: v3.0.0 > > > Currently when migrating a kylin cluster from on data center to the other or > the hbase cluster the kylin depends on will be changed from one cluster to > the other, there will be a long down time to migrating the history data from > one cluster to the other or we must rebuild all the history data of cube in > the other cluster. > In xiaomi, we added the support of secondary hbase storage and made the kylin > cluster can query cube data from the old hbase cluster during the migration. > As a result, the migration is very smooth with minimum down time. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4175) Support secondary hbase storage config for hbase cluster migration
[ https://issues.apache.org/jira/browse/KYLIN-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4175: Fix Version/s: v3.0.0 > Support secondary hbase storage config for hbase cluster migration > -- > > Key: KYLIN-4175 > URL: https://issues.apache.org/jira/browse/KYLIN-4175 > Project: Kylin > Issue Type: Improvement >Reporter: Shaohui Liu >Assignee: Shaohui Liu >Priority: Major > Fix For: v3.0.0 > > > Currently when migrating a kylin cluster from on data center to the other or > the hbase cluster the kylin depends on will be changed from one cluster to > the other, there will be a long down time to migrating the history data from > one cluster to the other or we must rebuild all the history data of cube in > the other cluster. > In xiaomi, we added the support of secondary hbase storage and made the kylin > cluster can query cube data from the old hbase cluster during the migration. > As a result, the migration is very smooth with minimum down time. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4175) Support secondary hbase storage config for hbase cluster migration
[ https://issues.apache.org/jira/browse/KYLIN-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4175. - Resolution: Fixed > Support secondary hbase storage config for hbase cluster migration > -- > > Key: KYLIN-4175 > URL: https://issues.apache.org/jira/browse/KYLIN-4175 > Project: Kylin > Issue Type: Improvement >Reporter: Shaohui Liu >Assignee: Shaohui Liu >Priority: Major > Fix For: v3.0.0 > > > Currently when migrating a kylin cluster from on data center to the other or > the hbase cluster the kylin depends on will be changed from one cluster to > the other, there will be a long down time to migrating the history data from > one cluster to the other or we must rebuild all the history data of cube in > the other cluster. > In xiaomi, we added the support of secondary hbase storage and made the kylin > cluster can query cube data from the old hbase cluster during the migration. > As a result, the migration is very smooth with minimum down time. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4212) Add a user profile page
[ https://issues.apache.org/jira/browse/KYLIN-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16961861#comment-16961861 ] nichunen commented on KYLIN-4212: - +1 > Add a user profile page > --- > > Key: KYLIN-4212 > URL: https://issues.apache.org/jira/browse/KYLIN-4212 > Project: Kylin > Issue Type: Improvement >Reporter: Congling Xia >Priority: Major > > Kylin's user/group API has been extended since KYLIN-4122. User can get more > account details and change password by requesting the APIs. But there is no > webpage for a normal user to view such information or do such changes. > In this improvement, the following changes are desired: > # provide a user profile page to show his/her basic information, > # set a button on it to launch the change password dialog for the user -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-1024) UPPER not support in Kylin
[
https://issues.apache.org/jira/browse/KYLIN-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen resolved KYLIN-1024.
-
Resolution: Fixed
> UPPER not support in Kylin
> --
>
> Key: KYLIN-1024
> URL: https://issues.apache.org/jira/browse/KYLIN-1024
> Project: Kylin
> Issue Type: Sub-task
> Components: Query Engine
>Affects Versions: v0.7.1, v0.7.2, v1.0
>Reporter: Huang Hua
>Assignee: luguosheng
>Priority: Minor
> Fix For: v3.0.0
>
>
> UPPER function doesn't work in WHERE clause,
> "org.apache.kylin.rest.exception.InternalErrorException: UPPER while
> executing SQL":
> select "MONDRIAN_REGION"."REGION_PROVINCE" as "c0",
> "MONDRIAN_REGION"."REGION_COUNTRY" as "c1"
> from "MONDRIAN_REGION" as "MONDRIAN_REGION"
> where ("MONDRIAN_REGION"."REGION_COUNTRY" = '??') and
> UPPER("MONDRIAN_REGION"."REGION_PROVINCE") = UPPER('??')
> group by "MONDRIAN_REGION"."REGION_PROVINCE",
> "MONDRIAN_REGION"."REGION_COUNTRY"
> order by CASE WHEN "MONDRIAN_REGION"."REGION_PROVINCE" IS NULL THEN 0 ELSE 1
> END, "MONDRIAN_REGION"."REGION_PROVINCE" ASC
> LIMIT 5
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-1024) UPPER not support in Kylin
[
https://issues.apache.org/jira/browse/KYLIN-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-1024:
Fix Version/s: v3.0.0
> UPPER not support in Kylin
> --
>
> Key: KYLIN-1024
> URL: https://issues.apache.org/jira/browse/KYLIN-1024
> Project: Kylin
> Issue Type: Sub-task
> Components: Query Engine
>Affects Versions: v0.7.1, v0.7.2, v1.0
>Reporter: Huang Hua
>Assignee: luguosheng
>Priority: Minor
> Fix For: v3.0.0
>
>
> UPPER function doesn't work in WHERE clause,
> "org.apache.kylin.rest.exception.InternalErrorException: UPPER while
> executing SQL":
> select "MONDRIAN_REGION"."REGION_PROVINCE" as "c0",
> "MONDRIAN_REGION"."REGION_COUNTRY" as "c1"
> from "MONDRIAN_REGION" as "MONDRIAN_REGION"
> where ("MONDRIAN_REGION"."REGION_COUNTRY" = '??') and
> UPPER("MONDRIAN_REGION"."REGION_PROVINCE") = UPPER('??')
> group by "MONDRIAN_REGION"."REGION_PROVINCE",
> "MONDRIAN_REGION"."REGION_COUNTRY"
> order by CASE WHEN "MONDRIAN_REGION"."REGION_PROVINCE" IS NULL THEN 0 ELSE 1
> END, "MONDRIAN_REGION"."REGION_PROVINCE" ASC
> LIMIT 5
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4194) Throw KylinConfigCannotInitException at STEP "Extract Fact Table Distinct Columns" with spark
[ https://issues.apache.org/jira/browse/KYLIN-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4194: Fix Version/s: v3.0.0 > Throw KylinConfigCannotInitException at STEP "Extract Fact Table Distinct > Columns" with spark > - > > Key: KYLIN-4194 > URL: https://issues.apache.org/jira/browse/KYLIN-4194 > Project: Kylin > Issue Type: Bug >Affects Versions: v3.0.0-beta >Reporter: wangrupeng >Assignee: wangrupeng >Priority: Major > Fix For: v3.0.0 > > Attachments: image-2019-10-16-16-13-00-190.png, kylin_spark.log > > > When I use spark to extract fact table distinct columns values, kylin throw > an exception, > org.apache.kylin.common.KylinConfigCannotInitException: Didn't find > KYLIN_CONF or KYLIN_HOME, please set one of them, > see more details in [^kylin_spark.log] ^^ . I tried cdh5.7 and hdp2.4, both > platforms got this problem. > It seems like an environment variables issue, but when I set KYLIN_CONF or > KYLIN_HOME on each yarn cluster node, it still not work. > As we can see from the log, this exception occurs when TrieDictionaryBuilder > try to build Dictionary objects, but cannot find KylinConfig using > KylinConfig.getInstanceFromEnv() method. > My solution is to add code in spark job by reset > KylinConfig.THREAD_ENV_INSTANCE > from hdfs metadata as the spark job's metadata is stored in hdfs.And finally > it works fine. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4194) Throw KylinConfigCannotInitException at STEP "Extract Fact Table Distinct Columns" with spark
[ https://issues.apache.org/jira/browse/KYLIN-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4194. - Resolution: Fixed > Throw KylinConfigCannotInitException at STEP "Extract Fact Table Distinct > Columns" with spark > - > > Key: KYLIN-4194 > URL: https://issues.apache.org/jira/browse/KYLIN-4194 > Project: Kylin > Issue Type: Bug >Affects Versions: v3.0.0-beta >Reporter: wangrupeng >Assignee: wangrupeng >Priority: Major > Fix For: v3.0.0 > > Attachments: image-2019-10-16-16-13-00-190.png, kylin_spark.log > > > When I use spark to extract fact table distinct columns values, kylin throw > an exception, > org.apache.kylin.common.KylinConfigCannotInitException: Didn't find > KYLIN_CONF or KYLIN_HOME, please set one of them, > see more details in [^kylin_spark.log] ^^ . I tried cdh5.7 and hdp2.4, both > platforms got this problem. > It seems like an environment variables issue, but when I set KYLIN_CONF or > KYLIN_HOME on each yarn cluster node, it still not work. > As we can see from the log, this exception occurs when TrieDictionaryBuilder > try to build Dictionary objects, but cannot find KylinConfig using > KylinConfig.getInstanceFromEnv() method. > My solution is to add code in spark job by reset > KylinConfig.THREAD_ENV_INSTANCE > from hdfs metadata as the spark job's metadata is stored in hdfs.And finally > it works fine. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4194) Throw KylinConfigCannotInitException at STEP "Extract Fact Table Distinct Columns" with spark
[ https://issues.apache.org/jira/browse/KYLIN-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16961943#comment-16961943 ] nichunen commented on KYLIN-4194: - good catch > Throw KylinConfigCannotInitException at STEP "Extract Fact Table Distinct > Columns" with spark > - > > Key: KYLIN-4194 > URL: https://issues.apache.org/jira/browse/KYLIN-4194 > Project: Kylin > Issue Type: Bug >Affects Versions: v3.0.0-beta >Reporter: wangrupeng >Assignee: wangrupeng >Priority: Major > Fix For: v3.0.0 > > Attachments: image-2019-10-16-16-13-00-190.png, kylin_spark.log > > > When I use spark to extract fact table distinct columns values, kylin throw > an exception, > org.apache.kylin.common.KylinConfigCannotInitException: Didn't find > KYLIN_CONF or KYLIN_HOME, please set one of them, > see more details in [^kylin_spark.log] ^^ . I tried cdh5.7 and hdp2.4, both > platforms got this problem. > It seems like an environment variables issue, but when I set KYLIN_CONF or > KYLIN_HOME on each yarn cluster node, it still not work. > As we can see from the log, this exception occurs when TrieDictionaryBuilder > try to build Dictionary objects, but cannot find KylinConfig using > KylinConfig.getInstanceFromEnv() method. > My solution is to add code in spark job by reset > KylinConfig.THREAD_ENV_INSTANCE > from hdfs metadata as the spark job's metadata is stored in hdfs.And finally > it works fine. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-1850) Show Kylin Version on GUI
[ https://issues.apache.org/jira/browse/KYLIN-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-1850: Fix Version/s: v3.0.0 > Show Kylin Version on GUI > - > > Key: KYLIN-1850 > URL: https://issues.apache.org/jira/browse/KYLIN-1850 > Project: Kylin > Issue Type: Improvement > Components: Web >Affects Versions: v1.5.2 >Reporter: qianqiaoneng >Assignee: luguosheng >Priority: Trivial > Fix For: v3.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-1850) Show Kylin Version on GUI
[ https://issues.apache.org/jira/browse/KYLIN-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-1850. - Resolution: Fixed > Show Kylin Version on GUI > - > > Key: KYLIN-1850 > URL: https://issues.apache.org/jira/browse/KYLIN-1850 > Project: Kylin > Issue Type: Improvement > Components: Web >Affects Versions: v1.5.2 >Reporter: qianqiaoneng >Assignee: luguosheng >Priority: Trivial > Fix For: v3.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4198) “bin/system-cube.sh cron” will overwrite user's crontab
[ https://issues.apache.org/jira/browse/KYLIN-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-4198: Fix Version/s: v2.6.5 v3.0.0 > “bin/system-cube.sh cron” will overwrite user's crontab > --- > > Key: KYLIN-4198 > URL: https://issues.apache.org/jira/browse/KYLIN-4198 > Project: Kylin > Issue Type: Improvement > Components: Tools, Build and Test >Reporter: QiangZhang >Priority: Minor > Fix For: v3.0.0, v2.6.5 > > Attachments: image-2019-10-17-18-23-47-266.png > > > When using “sh bin/system-cube.sh cron” add a cron job to build system > cube，current user's crontab will be overwritten，user's previous crontab will > be lost。 > !image-2019-10-17-18-23-47-266.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KYLIN-4198) “bin/system-cube.sh cron” will overwrite user's crontab
[ https://issues.apache.org/jira/browse/KYLIN-4198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-4198. - Resolution: Fixed > “bin/system-cube.sh cron” will overwrite user's crontab > --- > > Key: KYLIN-4198 > URL: https://issues.apache.org/jira/browse/KYLIN-4198 > Project: Kylin > Issue Type: Improvement > Components: Tools, Build and Test >Reporter: QiangZhang >Priority: Minor > Fix For: v3.0.0, v2.6.5 > > Attachments: image-2019-10-17-18-23-47-266.png > > > When using “sh bin/system-cube.sh cron” add a cron job to build system > cube，current user's crontab will be overwritten，user's previous crontab will > be lost。 > !image-2019-10-17-18-23-47-266.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-1857) Show available memory on UI - in System Tab (and other runtime statistics)
[ https://issues.apache.org/jira/browse/KYLIN-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-1857. --- Resolution: Fixed > Show available memory on UI - in System Tab (and other runtime statistics) > -- > > Key: KYLIN-1857 > URL: https://issues.apache.org/jira/browse/KYLIN-1857 > Project: Kylin > Issue Type: Improvement >Affects Versions: v1.5.2, v1.5.2.1 >Reporter: Richard Calaba >Priority: Minor > > I have run into situation that Kylin dies (exception in log says heap out of > memory) if I try to run 3 parallel cubes with high-cardinality dimensions. It > is reproduceable scenario. I have set max snapshot size to 2GB and -Xmx to > 16GB. > If I run the cube build one-by-one -> Kylin doesn't die. > As we have have no idea about memory requirements before we start building > the cube(s) then for now it would be beneficial at least to monitor basic > Kylin VM statistics, i.e.: > -- current memory occupied by snaphots > -- total memory allocation & total free memory > -- how many (and which) temporary (intermediate) objects (in > hive/hbase/filesystem) are created ... -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3338) Expose interface for reusing building flat table
[ https://issues.apache.org/jira/browse/KYLIN-3338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-3338. --- Resolution: Fixed > Expose interface for reusing building flat table > > > Key: KYLIN-3338 > URL: https://issues.apache.org/jira/browse/KYLIN-3338 > Project: Kylin > Issue Type: Improvement > Components: RDBMS Source >Reporter: Shaoxiong Zhan >Assignee: Shaoxiong Zhan >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3511) Kylin apache-kylin-2.4.0-bin-hbase1x by CubeBuildingCLI build error
[ https://issues.apache.org/jira/browse/KYLIN-3511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-3511. --- Resolution: Fixed > Kylin apache-kylin-2.4.0-bin-hbase1x by CubeBuildingCLI build error > --- > > Key: KYLIN-3511 > URL: https://issues.apache.org/jira/browse/KYLIN-3511 > Project: Kylin > Issue Type: Bug > Components: Client - CLI >Affects Versions: v2.4.0 > Environment: ubuntu >Reporter: Phuong Nguyen >Priority: Minor > Fix For: Future > > Attachments: kylin_CubeBuildingCLI_error.txt > > > Hi Admin, > I have apache-kylin-2.4.0-bin-hbase1x when cube build on web is OK but build > by CubeBuildingCLI the same below > sh /zserver/apache-kylin-2.4.0-bin-hbase1x/bin/kylin.sh > org.apache.kylin.tool.job.CubeBuildingCLI --cube cube_csm_mobilecard- > -buildType BUILD --startTime 153480960 --endTime 153489600 > It's error : > ERROR [main] job.CubeBuildingCLI:134 : error start cube building > java.lang.RuntimeException: error execute > org.apache.kylin.tool.job.CubeBuildingCLI > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42) > at org.apache.kylin.tool.job.CubeBuildingCLI.main(CubeBuildingCLI.java:131) > Caused by: java.lang.IllegalStateException: Inconsistent cube desc signature > for CubeDesc [name=cube_csm_mobilecard] > at > org.apache.kylin.tool.job.CubeBuildingCLI.checkCubeDescSignature(CubeBuildingCLI.java:125) > at > org.apache.kylin.tool.job.CubeBuildingCLI.submitJob(CubeBuildingCLI.java:104) > at org.apache.kylin.tool.job.CubeBuildingCLI.run(CubeBuildingCLI.java:99) > at org.apache.kylin.tool.job.CubeBuildingCLI.execute(CubeBuildingCLI.java:91) > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37) > ... 1 more > pls help me and resolve it > thanks, > phuongnh2 > > Hi -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3484) Update Hadoop version to 2.7.7
[ https://issues.apache.org/jira/browse/KYLIN-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen closed KYLIN-3484. --- Resolution: Fixed > Update Hadoop version to 2.7.7 > -- > > Key: KYLIN-3484 > URL: https://issues.apache.org/jira/browse/KYLIN-3484 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Priority: Minor > > We should upgrade the Hadoop 2.7 dependency to 2.7.7, to pick up bug and > security fixes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-3484) Update Hadoop version to 2.7.7
[ https://issues.apache.org/jira/browse/KYLIN-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16962889#comment-16962889 ] nichunen commented on KYLIN-3484: - We will update to hadoop3, close this issue. > Update Hadoop version to 2.7.7 > -- > > Key: KYLIN-3484 > URL: https://issues.apache.org/jira/browse/KYLIN-3484 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Priority: Minor > > We should upgrade the Hadoop 2.7 dependency to 2.7.7, to pick up bug and > security fixes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (KYLIN-3593) Kylin Interpreter on Zeppelin doesn't allow setting project name on paragraph
[
https://issues.apache.org/jira/browse/KYLIN-3593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen closed KYLIN-3593.
---
Resolution: Fixed
> Kylin Interpreter on Zeppelin doesn't allow setting project name on paragraph
> -
>
> Key: KYLIN-3593
> URL: https://issues.apache.org/jira/browse/KYLIN-3593
> Project: Kylin
> Issue Type: Improvement
> Components: Driver - JDBC, Integration
>Affects Versions: v2.4.1
>Reporter: Moises
>Assignee: Na Zhai
>Priority: Minor
>
> Hi guys,
> I have started using Kylin with our Zeppelin Notebook Server, we have
> installed correctly the interpreter and we can run queries to the default
> project specified in the interpreter settings.
>
> The problem is when we try to set the project at the paragraph:
>
> {{%kylin(learn_kylin)}}{{ }}
> {{select part_dt,count ( * ) from kylin_sales group by part_dt}}
>
> Fails with error 500, second paragraph has not specified the project
> (learn_kylin) and executes fine.
>
> {{%kylin}}{{ }}
> {{select part_dt,count( * ) from kylin_sales group by part_dt}}
>
> Our Kylin version is 2.4.0
> Zeppelin Version 0.7.3
>
> Kylin interpreter settings are the default, we followed the tutorial in
> Zeppelin Documentation:
> [https://zeppelin.apache.org/docs/0.7.0/interpreter/kylin.html]
>
>
> Am I writting the paragraph with incorrect syntax??
>
> Thanks in advance
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KYLIN-3741) when the sql result is empty and limit is 0 , should not have "load more" bar
[ https://issues.apache.org/jira/browse/KYLIN-3741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen resolved KYLIN-3741. - Resolution: Fixed > when the sql result is empty and limit is 0 , should not have "load more" bar > - > > Key: KYLIN-3741 > URL: https://issues.apache.org/jira/browse/KYLIN-3741 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v2.3.0 >Reporter: hahayuan >Assignee: hahayuan >Priority: Minor > Fix For: v3.0.0, v2.6.5 > > Attachments: > 0001-KYLIN-3741-when-the-sql-result-is-empty-and-limit-is.patch, > KYLIN-3741.PNG > > > the query result has no result , and we set limit is 0 ,the "MORE" bar should > not exists -- This message was sent by Atlassian Jira (v8.3.4#803005)
