[jira] [Commented] (KYLIN-4440) IllegalStateException thrown when a query contains group by clause with case when of dimensional table columns
[
https://issues.apache.org/jira/browse/KYLIN-4440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17077749#comment-17077749
]
Yifei Wu commented on KYLIN-4440:
-
this issue seems familiar to me, I would like to have a try to fix it .
> IllegalStateException thrown when a query contains group by clause with case
> when of dimensional table columns
> --
>
> Key: KYLIN-4440
> URL: https://issues.apache.org/jira/browse/KYLIN-4440
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v2.2.0, v2.5.1, v2.6.5
>Reporter: Xi Chen
>Priority: Major
> Labels: calcite
>
> When a sub-query contains group by clause with case when of dimensional table
> columns, and the sub-query is joined by another sub-query, the whole query
> fails throwing java.lang.IllegalStateException.
> Example query of kylin_sales_cube:
> {code:java}
> SELECT *
> FROM
> (SELECT part_dt ,
> CASE
> WHEN buyer_id >= 10003000 THEN 'A'
> ELSE kylin_account.account_country
> END AS category ,
> sum(price) AS total_sold ,
> count(DISTINCT seller_id) AS sellers
>FROM kylin_sales
>JOIN kylin_account ON kylin_sales.buyer_id = kylin_account.account_id
>GROUP BY part_dt ,
> CASE
> WHEN buyer_id >= 10003000 THEN 'A'
> ELSE kylin_account.account_country
> END
>ORDER BY part_dt) t1
> JOIN
> (SELECT part_dt ,
> sum(price) AS total_sold ,
> count(DISTINCT seller_id) AS sellers
>FROM kylin_sales
>JOIN kylin_account ON kylin_sales.buyer_id = kylin_account.account_id
>GROUP BY part_dt
>ORDER BY part_dt) t2 ON t1.part_dt = t2.part_dt
> ORDER BY t1.part_dt DESC
> {code}
> Exception message:
> {code:java}
> RowType=7, ColumnRowType=8 while executing SQL: "SELECT * FROM ( SELECT
> part_dt , CASE WHEN buyer_id >= 10003000 THEN 'A' ELSE
> kylin_account.account_country END AS category , sum(price) AS total_sold ,
> count(DISTINCT seller_id) AS sellers FROM kylin_sales LEFT JOIN kylin_account
> ON kylin_sales.buyer_id = kylin_account.account_id GROUP BY part_dt , CASE
> WHEN buyer_id >= 10003000 THEN 'A' ELSE kylin_account.account_country END
> ORDER BY part_dt ) t1 LEFT JOIN ( SELECT part_dt , sum(price) AS total_sold ,
> count(DISTINCT seller_id) AS sellers FROM kylin_sales LEFT JOIN kylin_account
> ON kylin_sales.buyer_id = kylin_account.account_id GROUP BY part_dt ORDER BY
> part_dt ) t2 ON t1.part_dt = t2.part_dt ORDER BY t1.part_dt DESC LIMIT
> 5"{code}
> More message from kylin logs:
> {code:java}
> Caused by: java.lang.IllegalStateException: RowType=7, ColumnRowType=8
> at
> org.apache.kylin.query.relnode.OLAPJoinRel.buildColumnRowType(OLAPJoinRel.java:223)
> at
> org.apache.kylin.query.relnode.OLAPJoinRel.implementOLAP(OLAPJoinRel.java:174)
> at
> org.apache.kylin.query.relnode.OLAPRel$OLAPImplementor.visitChild(OLAPRel.java:84)
> at
> org.apache.kylin.query.relnode.OLAPSortRel.implementOLAP(OLAPSortRel.java:72)
> at
> org.apache.kylin.query.relnode.OLAPRel$OLAPImplementor.visitChild(OLAPRel.java:84)
> at
> org.apache.kylin.query.relnode.OLAPLimitRel.implementOLAP(OLAPLimitRel.java:77)
> at
> org.apache.kylin.query.relnode.OLAPRel$OLAPImplementor.visitChild(OLAPRel.java:84)
> at
> org.apache.kylin.query.relnode.OLAPToEnumerableConverter.implement(OLAPToEnumerableConverter.java:75)
> at
> org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:103)
> at
> org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:92)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1278)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:331)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:230)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:796)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:655)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:618)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:221)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
> at
>
[jira] [Assigned] (KYLIN-4440) IllegalStateException thrown when a query contains group by clause with case when of dimensional table columns
[
https://issues.apache.org/jira/browse/KYLIN-4440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu reassigned KYLIN-4440:
---
Assignee: Yifei Wu
> IllegalStateException thrown when a query contains group by clause with case
> when of dimensional table columns
> --
>
> Key: KYLIN-4440
> URL: https://issues.apache.org/jira/browse/KYLIN-4440
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Affects Versions: v2.2.0, v2.5.1, v2.6.5
>Reporter: Xi Chen
>Assignee: Yifei Wu
>Priority: Major
> Labels: calcite
>
> When a sub-query contains group by clause with case when of dimensional table
> columns, and the sub-query is joined by another sub-query, the whole query
> fails throwing java.lang.IllegalStateException.
> Example query of kylin_sales_cube:
> {code:java}
> SELECT *
> FROM
> (SELECT part_dt ,
> CASE
> WHEN buyer_id >= 10003000 THEN 'A'
> ELSE kylin_account.account_country
> END AS category ,
> sum(price) AS total_sold ,
> count(DISTINCT seller_id) AS sellers
>FROM kylin_sales
>JOIN kylin_account ON kylin_sales.buyer_id = kylin_account.account_id
>GROUP BY part_dt ,
> CASE
> WHEN buyer_id >= 10003000 THEN 'A'
> ELSE kylin_account.account_country
> END
>ORDER BY part_dt) t1
> JOIN
> (SELECT part_dt ,
> sum(price) AS total_sold ,
> count(DISTINCT seller_id) AS sellers
>FROM kylin_sales
>JOIN kylin_account ON kylin_sales.buyer_id = kylin_account.account_id
>GROUP BY part_dt
>ORDER BY part_dt) t2 ON t1.part_dt = t2.part_dt
> ORDER BY t1.part_dt DESC
> {code}
> Exception message:
> {code:java}
> RowType=7, ColumnRowType=8 while executing SQL: "SELECT * FROM ( SELECT
> part_dt , CASE WHEN buyer_id >= 10003000 THEN 'A' ELSE
> kylin_account.account_country END AS category , sum(price) AS total_sold ,
> count(DISTINCT seller_id) AS sellers FROM kylin_sales LEFT JOIN kylin_account
> ON kylin_sales.buyer_id = kylin_account.account_id GROUP BY part_dt , CASE
> WHEN buyer_id >= 10003000 THEN 'A' ELSE kylin_account.account_country END
> ORDER BY part_dt ) t1 LEFT JOIN ( SELECT part_dt , sum(price) AS total_sold ,
> count(DISTINCT seller_id) AS sellers FROM kylin_sales LEFT JOIN kylin_account
> ON kylin_sales.buyer_id = kylin_account.account_id GROUP BY part_dt ORDER BY
> part_dt ) t2 ON t1.part_dt = t2.part_dt ORDER BY t1.part_dt DESC LIMIT
> 5"{code}
> More message from kylin logs:
> {code:java}
> Caused by: java.lang.IllegalStateException: RowType=7, ColumnRowType=8
> at
> org.apache.kylin.query.relnode.OLAPJoinRel.buildColumnRowType(OLAPJoinRel.java:223)
> at
> org.apache.kylin.query.relnode.OLAPJoinRel.implementOLAP(OLAPJoinRel.java:174)
> at
> org.apache.kylin.query.relnode.OLAPRel$OLAPImplementor.visitChild(OLAPRel.java:84)
> at
> org.apache.kylin.query.relnode.OLAPSortRel.implementOLAP(OLAPSortRel.java:72)
> at
> org.apache.kylin.query.relnode.OLAPRel$OLAPImplementor.visitChild(OLAPRel.java:84)
> at
> org.apache.kylin.query.relnode.OLAPLimitRel.implementOLAP(OLAPLimitRel.java:77)
> at
> org.apache.kylin.query.relnode.OLAPRel$OLAPImplementor.visitChild(OLAPRel.java:84)
> at
> org.apache.kylin.query.relnode.OLAPToEnumerableConverter.implement(OLAPToEnumerableConverter.java:75)
> at
> org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:103)
> at
> org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:92)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1278)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:331)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:230)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:796)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:655)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:618)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:221)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
> ... 83 more
> {code}
>
>
[jira] [Commented] (KYLIN-4327) TOPN Comparator may violate its general contract
[
https://issues.apache.org/jira/browse/KYLIN-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17008831#comment-17008831
]
Yifei Wu commented on KYLIN-4327:
-
use the method Double.compare() will avoid this error, so just replace the
above method with Double.compare will be ok
> TOPN Comparator may violate its general contract
> -
>
> Key: KYLIN-4327
> URL: https://issues.apache.org/jira/browse/KYLIN-4327
> Project: Kylin
> Issue Type: Improvement
>Reporter: Yifei Wu
>Assignee: Yifei Wu
>Priority: Minor
>
> in current TopN， it should save top(k) result and keep it in double value,
> just like this:
> ```
> public class Counter implements Serializable{
> ...
> protected T item;
> protected double count;
> ...
> }
> ```
> But its Comparator method use "==" directly to compare the result, it may
> cause the error "*violate its general contract*" when calling it.
> ```
> private static final Comparator ASC_COMPARATOR = new
> Comparator() {
> @Override
> public int compare(Counter o1, Counter o2) {
> return o1.getCount() > o2.getCount() ? 1 : o1.getCount() ==
> o2.getCount() ? 0 : -1;
> }
> };
> ```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4327) TOPN Comparator may violate its general contract
[
https://issues.apache.org/jira/browse/KYLIN-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu updated KYLIN-4327:
Description:
in current TopN， it should save top(k) result and keep it in double value,
just like this:
```
public class Counter implements Serializable{
...
protected T item;
protected double count;
...
}
```
But its Comparator method use "==" directly to compare the result, it may
cause the error "*violate its general contract*" when calling it
```
return o1.getCount() > o2.getCount() ? 1 : o1.getCount() ==
o2.getCount() ? 0 : -1;
```
was:
in current TopN， it should save top(k) result and keep it in double value,
just like this:
```
public class Counter implements Serializable{
...
protected T item;
protected double count;
...
}
```
But its Comparator method use "==" directly to compare the result, it may
cause the error "*violate its general contract*" twhen calling it
```
return o1.getCount() > o2.getCount() ? 1 : o1.getCount() ==
o2.getCount() ? 0 : -1;
```
> TOPN Comparator may violate its general contract
> -
>
> Key: KYLIN-4327
> URL: https://issues.apache.org/jira/browse/KYLIN-4327
> Project: Kylin
> Issue Type: Improvement
>Reporter: Yifei Wu
>Assignee: Yifei Wu
>Priority: Minor
>
> in current TopN， it should save top(k) result and keep it in double value,
> just like this:
> ```
> public class Counter implements Serializable{
> ...
> protected T item;
> protected double count;
> ...
> }
> ```
> But its Comparator method use "==" directly to compare the result, it may
> cause the error "*violate its general contract*" when calling it
> ```
> return o1.getCount() > o2.getCount() ? 1 : o1.getCount() ==
> o2.getCount() ? 0 : -1;
> ```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4327) TOPN Comparator may violate its general contract
[
https://issues.apache.org/jira/browse/KYLIN-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu updated KYLIN-4327:
Description:
in current TopN， it should save top(k) result and keep it in double value,
just like this:
```
public class Counter implements Serializable{
...
protected T item;
protected double count;
...
}
```
But its Comparator method use "==" directly to compare the result, it may
cause the error "*violate its general contract*" when calling it.
```
private static final Comparator ASC_COMPARATOR = new
Comparator() {
@Override
public int compare(Counter o1, Counter o2) {
return o1.getCount() > o2.getCount() ? 1 : o1.getCount() ==
o2.getCount() ? 0 : -1;
}
};
```
was:
in current TopN， it should save top(k) result and keep it in double value,
just like this:
```
public class Counter implements Serializable{
...
protected T item;
protected double count;
...
}
```
But its Comparator method use "==" directly to compare the result, it may
cause the error "*violate its general contract*" when calling it
```
return o1.getCount() > o2.getCount() ? 1 : o1.getCount() ==
o2.getCount() ? 0 : -1;
```
> TOPN Comparator may violate its general contract
> -
>
> Key: KYLIN-4327
> URL: https://issues.apache.org/jira/browse/KYLIN-4327
> Project: Kylin
> Issue Type: Improvement
>Reporter: Yifei Wu
>Assignee: Yifei Wu
>Priority: Minor
>
> in current TopN， it should save top(k) result and keep it in double value,
> just like this:
> ```
> public class Counter implements Serializable{
> ...
> protected T item;
> protected double count;
> ...
> }
> ```
> But its Comparator method use "==" directly to compare the result, it may
> cause the error "*violate its general contract*" when calling it.
> ```
> private static final Comparator ASC_COMPARATOR = new
> Comparator() {
> @Override
> public int compare(Counter o1, Counter o2) {
> return o1.getCount() > o2.getCount() ? 1 : o1.getCount() ==
> o2.getCount() ? 0 : -1;
> }
> };
> ```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4327) TOPN Comparator may violate its general contract
[
https://issues.apache.org/jira/browse/KYLIN-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu updated KYLIN-4327:
Description:
in current TopN， it should save top(k) result and keep it in double value,
just like this:
```
public class Counter implements Serializable{
...
protected T item;
protected double count;
...
}
```
But its Comparator method use "==" directly to compare the result, it may
cause the error "*violate its general contract*" twhen calling it
```
return o1.getCount() > o2.getCount() ? 1 : o1.getCount() ==
o2.getCount() ? 0 : -1;
```
was:
in current TopN， it should save top(k) result and keep it in double value,
just like this:
```
public class Counter implements Serializable{
...
protected T item;
protected double count;
...
}
```
But its Comparator method use "==" directly to compare the result, it may
cause the error when calling it
```
return o1.getCount() > o2.getCount() ? 1 : o1.getCount() ==
o2.getCount() ? 0 : -1;
```
> TOPN Comparator may violate its general contract
> -
>
> Key: KYLIN-4327
> URL: https://issues.apache.org/jira/browse/KYLIN-4327
> Project: Kylin
> Issue Type: Improvement
>Reporter: Yifei Wu
>Assignee: Yifei Wu
>Priority: Minor
>
> in current TopN， it should save top(k) result and keep it in double value,
> just like this:
> ```
> public class Counter implements Serializable{
> ...
> protected T item;
> protected double count;
> ...
> }
> ```
> But its Comparator method use "==" directly to compare the result, it may
> cause the error "*violate its general contract*" twhen calling it
> ```
> return o1.getCount() > o2.getCount() ? 1 : o1.getCount() ==
> o2.getCount() ? 0 : -1;
> ```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KYLIN-4327) TOPN Comparator may violate its general contract
[
https://issues.apache.org/jira/browse/KYLIN-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu updated KYLIN-4327:
Description:
in current TopN， it should save top(k) result and keep it in double value,
just like this:
```
public class Counter implements Serializable{
...
protected T item;
protected double count;
...
}
```
But its Comparator method use "==" directly to compare the result, it may
cause the error when calling it
```
return o1.getCount() > o2.getCount() ? 1 : o1.getCount() ==
o2.getCount() ? 0 : -1;
```
> TOPN Comparator may violate its general contract
> -
>
> Key: KYLIN-4327
> URL: https://issues.apache.org/jira/browse/KYLIN-4327
> Project: Kylin
> Issue Type: Improvement
>Reporter: Yifei Wu
>Assignee: Yifei Wu
>Priority: Minor
>
> in current TopN， it should save top(k) result and keep it in double value,
> just like this:
> ```
> public class Counter implements Serializable{
> ...
> protected T item;
> protected double count;
> ...
> }
> ```
> But its Comparator method use "==" directly to compare the result, it may
> cause the error when calling it
> ```
> return o1.getCount() > o2.getCount() ? 1 : o1.getCount() ==
> o2.getCount() ? 0 : -1;
> ```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-4327) TOPN Comparator may violate its general contract
[ https://issues.apache.org/jira/browse/KYLIN-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17008518#comment-17008518 ] Yifei Wu commented on KYLIN-4327: - PR https://github.com/apache/kylin/pull/1043 > TOPN Comparator may violate its general contract > - > > Key: KYLIN-4327 > URL: https://issues.apache.org/jira/browse/KYLIN-4327 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KYLIN-4327) TOPN Comparator may violate its general contract
[ https://issues.apache.org/jira/browse/KYLIN-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-4327: Summary: TOPN Comparator may violate its general contract (was: TOPN compator may violates its general contract) > TOPN Comparator may violate its general contract > - > > Key: KYLIN-4327 > URL: https://issues.apache.org/jira/browse/KYLIN-4327 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KYLIN-4327) TOPN compator may violates its general contract
Yifei Wu created KYLIN-4327: --- Summary: TOPN compator may violates its general contract Key: KYLIN-4327 URL: https://issues.apache.org/jira/browse/KYLIN-4327 Project: Kylin Issue Type: Improvement Reporter: Yifei Wu Assignee: Yifei Wu -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-4041) CONCAT NULL not working properly
[ https://issues.apache.org/jira/browse/KYLIN-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861825#comment-16861825 ] Yifei Wu commented on KYLIN-4041: - try to fix in PR, plz help to review it: https://github.com/apache/kylin/pull/684 > CONCAT NULL not working properly > > > Key: KYLIN-4041 > URL: https://issues.apache.org/jira/browse/KYLIN-4041 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > to support SQL 'concat' function, Kylin introduces a UDF named ConcatUDF. > but the NULL case cannot deal with correctly, like the below sql: > ``` > select > concat(meta_categ_name, CAST(null as VARCHAR)) as c1, > concat(meta_categ_name, ' Catagory') as c2 > from test_kylin_fact > left JOIN edw.test_cal_dt as test_cal_dt > ON test_kylin_fact.cal_dt = test_cal_dt.cal_dt > left JOIN test_category_groupings > ON test_kylin_fact.leaf_categ_id = test_category_groupings.leaf_categ_id AND > test_kylin_fact.lstg_site_id = test_category_groupings.site_id > left JOIN edw.test_sites as test_sites > ON test_kylin_fact.lstg_site_id = test_sites.site_id > where meta_categ_name = 'Computers' > group by meta_categ_name > order by meta_categ_name > limit 1 > ``` > when CONCAT value with NULL, the result is expected to be still NULL. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-4041) CONCAT NULL not working properly
[ https://issues.apache.org/jira/browse/KYLIN-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-4041: Description: to support SQL 'concat' function, Kylin introduces a UDF named ConcatUDF. but the NULL case cannot deal with correctly, like the below sql: ``` select concat(meta_categ_name, CAST(null as VARCHAR)) as c1, concat(meta_categ_name, ' Catagory') as c2 from test_kylin_fact left JOIN edw.test_cal_dt as test_cal_dt ON test_kylin_fact.cal_dt = test_cal_dt.cal_dt left JOIN test_category_groupings ON test_kylin_fact.leaf_categ_id = test_category_groupings.leaf_categ_id AND test_kylin_fact.lstg_site_id = test_category_groupings.site_id left JOIN edw.test_sites as test_sites ON test_kylin_fact.lstg_site_id = test_sites.site_id where meta_categ_name = 'Computers' group by meta_categ_name order by meta_categ_name limit 1 ``` when CONCAT value with NULL, the result is expected to be still NULL. was: to support SQL 'concat' function, Kylin introduces a UDF named ConcatUDF. but the NULL case cannot deal with correctly, like the below sql: ``` select concat(meta_categ_name, CAST(null as VARCHAR)) as c1, concat(meta_categ_name, ' Catagory') as c2 from test_kylin_fact left JOIN edw.test_cal_dt as test_cal_dt ON test_kylin_fact.cal_dt = test_cal_dt.cal_dt left JOIN test_category_groupings ON test_kylin_fact.leaf_categ_id = test_category_groupings.leaf_categ_id AND test_kylin_fact.lstg_site_id = test_category_groupings.site_id left JOIN edw.test_sites as test_sites ON test_kylin_fact.lstg_site_id = test_sites.site_id where meta_categ_name = 'Computers' group by meta_categ_name order by meta_categ_name limit 1 ``` > CONCAT NULL not working properly > > > Key: KYLIN-4041 > URL: https://issues.apache.org/jira/browse/KYLIN-4041 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > to support SQL 'concat' function, Kylin introduces a UDF named ConcatUDF. > but the NULL case cannot deal with correctly, like the below sql: > ``` > select > concat(meta_categ_name, CAST(null as VARCHAR)) as c1, > concat(meta_categ_name, ' Catagory') as c2 > from test_kylin_fact > left JOIN edw.test_cal_dt as test_cal_dt > ON test_kylin_fact.cal_dt = test_cal_dt.cal_dt > left JOIN test_category_groupings > ON test_kylin_fact.leaf_categ_id = test_category_groupings.leaf_categ_id AND > test_kylin_fact.lstg_site_id = test_category_groupings.site_id > left JOIN edw.test_sites as test_sites > ON test_kylin_fact.lstg_site_id = test_sites.site_id > where meta_categ_name = 'Computers' > group by meta_categ_name > order by meta_categ_name > limit 1 > ``` > when CONCAT value with NULL, the result is expected to be still NULL. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-4041) CONCAT NULL not working properly
Yifei Wu created KYLIN-4041: --- Summary: CONCAT NULL not working properly Key: KYLIN-4041 URL: https://issues.apache.org/jira/browse/KYLIN-4041 Project: Kylin Issue Type: Improvement Reporter: Yifei Wu Assignee: Yifei Wu to support SQL 'concat' function, Kylin introduces a UDF named ConcatUDF. but the NULL case cannot deal with correctly, like the below sql: ``` select concat(meta_categ_name, CAST(null as VARCHAR)) as c1, concat(meta_categ_name, ' Catagory') as c2 from test_kylin_fact left JOIN edw.test_cal_dt as test_cal_dt ON test_kylin_fact.cal_dt = test_cal_dt.cal_dt left JOIN test_category_groupings ON test_kylin_fact.leaf_categ_id = test_category_groupings.leaf_categ_id AND test_kylin_fact.lstg_site_id = test_category_groupings.site_id left JOIN edw.test_sites as test_sites ON test_kylin_fact.lstg_site_id = test_sites.site_id where meta_categ_name = 'Computers' group by meta_categ_name order by meta_categ_name limit 1 ``` -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3812) optimize the child CompareTupleFilter in a CompareTupleFilter
[
https://issues.apache.org/jira/browse/KYLIN-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814090#comment-16814090
]
Yifei Wu edited comment on KYLIN-3812 at 4/10/19 6:25 AM:
--
Hi, [~yaho] I found this will be optimized to a constant true in calcite-1.16
which is depended by the latest Kylin release. So upgrading the Calcite may be
a better choice for u.
was (Author: yifei_wu94):
Hi, [~yaho] I found this will optimize in calcite-1.16 which is depended by
the latest Kylin release. So upgrading the Calcite may be a better choice for u.
> optimize the child CompareTupleFilter in a CompareTupleFilter
> -
>
> Key: KYLIN-3812
> URL: https://issues.apache.org/jira/browse/KYLIN-3812
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v3.0.0
>
>
> Currently it's not well supported for CompareTupleFilter to have a child of
> CompareTupleFilter. However, in some cases, it's better to support it.
> {code}
> where (colA = (1=1))
> {code}
> The *(1=1)* can be transformed to "true". And then this filter can be pushed
> down to hbase. Otherwise, the filter *(colA = (1=1))* does not work in hbase.
> And it may return incorrect results for the following SQL:
> {code}
> select colA
>case
>when colB = (1 = 1) then 'B'
>when colC = (1 = 1) then 'C'
>when colD = (1 = 1) then 'D'
>else 'n/a'
>end as phase,
>count(*)
> from T
> where session_date between '2018-08-01' and '2018-08-31'
> group by colA
>case
>when colB = (1 = 1) then 'B'
>when colC = (1 = 1) then 'C'
>when colD = (1 = 1) then 'D'
>else 'n/a'
>end;
> {code}
> In the final result, all of the keys will become 'B'.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3812) optimize the child CompareTupleFilter in a CompareTupleFilter
[
https://issues.apache.org/jira/browse/KYLIN-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814090#comment-16814090
]
Yifei Wu commented on KYLIN-3812:
-
Hi, [~yaho] I found this will optimize in calcite-1.16 which is depended by
the latest Kylin release. So upgrading the Calcite may be a better choice for u.
> optimize the child CompareTupleFilter in a CompareTupleFilter
> -
>
> Key: KYLIN-3812
> URL: https://issues.apache.org/jira/browse/KYLIN-3812
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v3.0.0
>
>
> Currently it's not well supported for CompareTupleFilter to have a child of
> CompareTupleFilter. However, in some cases, it's better to support it.
> {code}
> where (colA = (1=1))
> {code}
> The *(1=1)* can be transformed to "true". And then this filter can be pushed
> down to hbase. Otherwise, the filter *(colA = (1=1))* does not work in hbase.
> And it may return incorrect results for the following SQL:
> {code}
> select colA
>case
>when colB = (1 = 1) then 'B'
>when colC = (1 = 1) then 'C'
>when colD = (1 = 1) then 'D'
>else 'n/a'
>end as phase,
>count(*)
> from T
> where session_date between '2018-08-01' and '2018-08-31'
> group by colA
>case
>when colB = (1 = 1) then 'B'
>when colC = (1 = 1) then 'C'
>when colD = (1 = 1) then 'D'
>else 'n/a'
>end;
> {code}
> In the final result, all of the keys will become 'B'.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3812) optimize the child CompareTupleFilter in a CompareTupleFilter
[
https://issues.apache.org/jira/browse/KYLIN-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802393#comment-16802393
]
Yifei Wu commented on KYLIN-3812:
-
hello [~yaho], added some comments for ur PR
> optimize the child CompareTupleFilter in a CompareTupleFilter
> -
>
> Key: KYLIN-3812
> URL: https://issues.apache.org/jira/browse/KYLIN-3812
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v3.0.0
>
>
> Currently it's not well supported for CompareTupleFilter to have a child of
> CompareTupleFilter. However, in some cases, it's better to support it.
> {code}
> where (colA = (1=1))
> {code}
> The *(1=1)* can be transformed to "true". And then this filter can be pushed
> down to hbase. Otherwise, the filter *(colA = (1=1))* does not work in hbase.
> And it may return incorrect results for the following SQL:
> {code}
> select colA
>case
>when colB = (1 = 1) then 'B'
>when colC = (1 = 1) then 'C'
>when colD = (1 = 1) then 'D'
>else 'n/a'
>end as phase,
>count(*)
> from T
> where session_date between '2018-08-01' and '2018-08-31'
> group by colA
>case
>when colB = (1 = 1) then 'B'
>when colC = (1 = 1) then 'C'
>when colD = (1 = 1) then 'D'
>else 'n/a'
>end;
> {code}
> In the final result, all of the keys will become 'B'.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3812) optimize the child CompareTupleFilter in a CompareTupleFilter
[
https://issues.apache.org/jira/browse/KYLIN-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802391#comment-16802391
]
Yifei Wu commented on KYLIN-3812:
-
plz add some comment for this root cause, and the confused point is that
calcite won't optimize the constant part?
> optimize the child CompareTupleFilter in a CompareTupleFilter
> -
>
> Key: KYLIN-3812
> URL: https://issues.apache.org/jira/browse/KYLIN-3812
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
>Priority: Major
> Fix For: v3.0.0
>
>
> Currently it's not well supported for CompareTupleFilter to have a child of
> CompareTupleFilter. However, in some cases, it's better to support it.
> {code}
> where (colA = (1=1))
> {code}
> The *(1=1)* can be transformed to "true". And then this filter can be pushed
> down to hbase. Otherwise, the filter *(colA = (1=1))* does not work in hbase.
> And it may return incorrect results for the following SQL:
> {code}
> select colA
>case
>when colB = (1 = 1) then 'B'
>when colC = (1 = 1) then 'C'
>when colD = (1 = 1) then 'D'
>else 'n/a'
>end as phase,
>count(*)
> from T
> where session_date between '2018-08-01' and '2018-08-31'
> group by colA
>case
>when colB = (1 = 1) then 'B'
>when colC = (1 = 1) then 'C'
>when colD = (1 = 1) then 'D'
>else 'n/a'
>end;
> {code}
> In the final result, all of the keys will become 'B'.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3751) Query failed when you set log4j.logger.org.apache.calcite=DEBUG
[ https://issues.apache.org/jira/browse/KYLIN-3751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16731743#comment-16731743 ] Yifei Wu commented on KYLIN-3751: - you can try but not recommended to replace it directly, upgrade Kylin version may be a better choice. In Calcite-core-1.16.0-Kylin-r2, there are several other big changes that should result in some errors for your Kylin version. > Query failed when you set log4j.logger.org.apache.calcite=DEBUG > --- > > Key: KYLIN-3751 > URL: https://issues.apache.org/jira/browse/KYLIN-3751 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v2.5.0 >Reporter: zengrui >Priority: Minor > > Query failed when you set log4j.logger.org.apache.calcite=DEBUG in > kylin-server-log4j.properties.It is a NullPointerException in > VolcanoPlanner(line 891, calcite-core-1.13.0-kylin-r4), root is used before > set. > Then I found that it has been solved in calcite-core-1.16.0-kylin-r2. Can I > update the calcite-core directly? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3704) Upgrade the calcite version to 1.16
[ https://issues.apache.org/jira/browse/KYLIN-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16698786#comment-16698786 ] Yifei Wu commented on KYLIN-3704: - [~Shaofengshi] this is the PR [https://github.com/apache/kylin/pull/361] and has passed the UT and IT in local env, could you help me to review this? > Upgrade the calcite version to 1.16 > - > > Key: KYLIN-3704 > URL: https://issues.apache.org/jira/browse/KYLIN-3704 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > Fix For: v2.5.2 > > > we have updated the customized Calcite version, which rebase the > changes of Apache Calcite 1.16.0. And in this version, we did some changes > to adapt the Apache Calcite, Such as the change the Grouping function > calculate method and adjust the EXTRACT function. > As a result，this issue is also resolved. > https://issues.apache.org/jira/projects/KYLIN/issues/KYLIN-3253?filter=recentlyviewed -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3704) upgrade the calcite version to 1.16
[ https://issues.apache.org/jira/browse/KYLIN-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3704: Description: _emphasized text_ we have updated the customized Calcite version, which rebase the changes of Apache Calcite 1.16.0. And in this version, we did some changes to adapt the Apache Calcite, Such as the change the Grouping function calculate method and adjust the EXTRACT function. (was: _emphasized text_ we have updated the customized Calcite version, which rebase the changes of Apache Calcite 1.16.0. And in this version, we did some changes to adpat change the Groupping function calculate method,) > upgrade the calcite version to 1.16 > - > > Key: KYLIN-3704 > URL: https://issues.apache.org/jira/browse/KYLIN-3704 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > _emphasized text_ we have updated the customized Calcite version, which > rebase the changes of Apache Calcite 1.16.0. And in this version, we did > some changes to adapt the Apache Calcite, Such as the change the Grouping > function calculate method and adjust the EXTRACT function. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3704) upgrade the calcite version to 1.16
[ https://issues.apache.org/jira/browse/KYLIN-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3704: Description: we have updated the customized Calcite version, which rebase the changes of Apache Calcite 1.16.0. And in this version, we did some changes to adapt the Apache Calcite, Such as the change the Grouping function calculate method and adjust the EXTRACT function. As a result，this issue is also resolved. https://issues.apache.org/jira/projects/KYLIN/issues/KYLIN-3253?filter=recentlyviewed was: _emphasized text_ we have updated the customized Calcite version, which rebase the changes of Apache Calcite 1.16.0. And in this version, we did some changes to adapt the Apache Calcite, Such as the change the Grouping function calculate method and adjust the EXTRACT function. As a result，this issue is also resolved. https://issues.apache.org/jira/projects/KYLIN/issues/KYLIN-3253?filter=recentlyviewed > upgrade the calcite version to 1.16 > - > > Key: KYLIN-3704 > URL: https://issues.apache.org/jira/browse/KYLIN-3704 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > we have updated the customized Calcite version, which rebase the > changes of Apache Calcite 1.16.0. And in this version, we did some changes > to adapt the Apache Calcite, Such as the change the Grouping function > calculate method and adjust the EXTRACT function. > As a result，this issue is also resolved. > https://issues.apache.org/jira/projects/KYLIN/issues/KYLIN-3253?filter=recentlyviewed -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3704) upgrade the calcite version to 1.16
[ https://issues.apache.org/jira/browse/KYLIN-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3704: Description: _emphasized text_ we have updated the customized Calcite version, which rebase the changes of Apache Calcite 1.16.0. And in this version, we did some changes to adapt the Apache Calcite, Such as the change the Grouping function calculate method and adjust the EXTRACT function. As a result，this issue is also resolved. https://issues.apache.org/jira/projects/KYLIN/issues/KYLIN-3253?filter=recentlyviewed was:_emphasized text_ we have updated the customized Calcite version, which rebase the changes of Apache Calcite 1.16.0. And in this version, we did some changes to adapt the Apache Calcite, Such as the change the Grouping function calculate method and adjust the EXTRACT function. > upgrade the calcite version to 1.16 > - > > Key: KYLIN-3704 > URL: https://issues.apache.org/jira/browse/KYLIN-3704 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > _emphasized text_ we have updated the customized Calcite version, which > rebase the changes of Apache Calcite 1.16.0. And in this version, we did > some changes to adapt the Apache Calcite, Such as the change the Grouping > function calculate method and adjust the EXTRACT function. > As a result，this issue is also resolved. > https://issues.apache.org/jira/projects/KYLIN/issues/KYLIN-3253?filter=recentlyviewed -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3704) upgrade the calcite version to 1.16
[ https://issues.apache.org/jira/browse/KYLIN-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3704: Description: _emphasized text_ we have updated the customized Calcite version, which rebase the changes of Apache Calcite 1.16.0. And in this version, we did some changes to adpat change the Groupping function calculate method, (was: _emphasized text_ we have updated the customized Calcite version, which rebase the changes of Apache Calcite 1.16.0. releated ) > upgrade the calcite version to 1.16 > - > > Key: KYLIN-3704 > URL: https://issues.apache.org/jira/browse/KYLIN-3704 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > _emphasized text_ we have updated the customized Calcite version, which > rebase the changes of Apache Calcite 1.16.0. And in this version, we did > some changes to adpat change the Groupping function calculate method, -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3704) upgrade the calcite version to 1.16
[ https://issues.apache.org/jira/browse/KYLIN-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3704: Description: _emphasized text_ we have updated the customized Calcite version, which rebase the changes of Apache Calcite 1.16.0. releated was:_emphasized text_ we have updated the customized Calcite version, which rebase the change to Apache Calcite > upgrade the calcite version to 1.16 > - > > Key: KYLIN-3704 > URL: https://issues.apache.org/jira/browse/KYLIN-3704 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > _emphasized text_ we have updated the customized Calcite version, which > rebase the changes of Apache Calcite 1.16.0. > releated -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3704) upgrade the calcite version to 1.16
[ https://issues.apache.org/jira/browse/KYLIN-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3704: Description: _emphasized text_ we have updated the customized Calcite version, which rebase the change to Apache Calcite (was: Current version ) > upgrade the calcite version to 1.16 > - > > Key: KYLIN-3704 > URL: https://issues.apache.org/jira/browse/KYLIN-3704 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > _emphasized text_ we have updated the customized Calcite version, which > rebase the change to Apache Calcite -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3704) upgrade the calcite version to 1.16
[ https://issues.apache.org/jira/browse/KYLIN-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3704: Description: Current version > upgrade the calcite version to 1.16 > - > > Key: KYLIN-3704 > URL: https://issues.apache.org/jira/browse/KYLIN-3704 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > Current version -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3704) upgrade the calcite version to 1.16
Yifei Wu created KYLIN-3704: --- Summary: upgrade the calcite version to 1.16 Key: KYLIN-3704 URL: https://issues.apache.org/jira/browse/KYLIN-3704 Project: Kylin Issue Type: Improvement Reporter: Yifei Wu Assignee: Yifei Wu -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3385) when the cube contain sum(1) measure, tables in "Query" page cannot be load
[
https://issues.apache.org/jira/browse/KYLIN-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566501#comment-16566501
]
Yifei Wu commented on KYLIN-3385:
-
hi, [~Shaofengshi] , I open a new pull request for this, could u help with
reviewing this PR?
[https://github.com/apache/kylin/pull/184]
> when the cube contain sum(1) measure, tables in "Query" page cannot be load
> ---
>
> Key: KYLIN-3385
> URL: https://issues.apache.org/jira/browse/KYLIN-3385
> Project: Kylin
> Issue Type: Bug
>Reporter: Yifei Wu
>Assignee: Yifei Wu
>Priority: Major
> Fix For: v2.5.0
>
> Attachments: 0001-KYLIN-3385-fix-sum-1.patch
>
>
> In the current version, the sum(1), equals with count(*) actually, is
> allowed to add to cube Measures.
> However, if you add, it will cause some errors, such as tables in "Query"
> page cannot be load and the query like this cannot run.
> {code:java}
> select sum(1) from kylin_fact
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (KYLIN-3444) support more reasonable Realization Chooser method in accordance with specific SQL
[ https://issues.apache.org/jira/browse/KYLIN-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu closed KYLIN-3444. --- Resolution: Not A Problem > support more reasonable Realization Chooser method in accordance with > specific SQL > -- > > Key: KYLIN-3444 > URL: https://issues.apache.org/jira/browse/KYLIN-3444 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3444) support more reasonable Realization Chooser method in accordance with specific SQL
[ https://issues.apache.org/jira/browse/KYLIN-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3444: Description: (was: In Kylin，there could be some unexpected results in Choosing Realization. For example, there are 2 models in the same project, called A and B, and each of it owns a Cube and a TableIndex. just like this, project / \ model(A) model(B) / \ / \ CUBE(a) TableIndex(a) CUBE(b) TableIndex(b) Nowadays, the algorithm is designed to choose the matched models, and then get into the model and select the proper and cheapest realizations(cube or TableIndex here). However, it can cause some confusing issue, for example, when running a query with agg functions (like "SELECT SUM(aa) from A GROUP BY bb") and *the CUBE(a) cannot serve this query but CUBE(b) can*, it should be hit the CUBE(b) rather than TableIndex(a) in the case, for there are aggregations, utilizing the cube should be more efficient.) > support more reasonable Realization Chooser method in accordance with > specific SQL > -- > > Key: KYLIN-3444 > URL: https://issues.apache.org/jira/browse/KYLIN-3444 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3444) support more reasonable Realization Chooser method in accordance with specific SQL
[ https://issues.apache.org/jira/browse/KYLIN-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3444: Description: In Kylin，there could be some unexpected results in Choosing Realization. For example, there are 2 models in the same project, called A and B, and each of it owns a Cube and a TableIndex. just like this, project / \ model(A) model(B) / \ / \ CUBE(a) TableIndex(a) CUBE(b) TableIndex(b) Nowadays, the algorithm is designed to choose the matched models, and then get into the model and select the proper and cheapest realizations(cube or TableIndex here). However, it can cause some confusing issue, for example, when running a query with agg functions (like "SELECT SUM(aa) from A GROUP BY bb") and *the CUBE(a) cannot serve this query but CUBE(b) can*, it should be hit the CUBE(b) rather than TableIndex(a) in the case, for there are aggregations, utilizing the cube should be more efficient. was: In Kylin，there could be some unexpected results in Choosing Realization. For example, there are 2 models in the same project, called A and B, and each of it owns a Cube and a TableIndex. just like this, project / \ model(A) model(B) / \ / \ CUBE(a) TableIndex(a) CUBE(b) TableIndex(b) Nowadays, the algorithm is designed to choose the matched models, and then get into the model and select the proper and cheapest realizations(cube or TableIndex here). However, it can cause some confusing issue, for example, when running a query with agg functions (like "SELECT SUM(aa) from A GROUP BY bb") and *the CUBE(a) cannot serve this query but CUBE(b) can*, it should be hit the CUBE(b) rather than TableIndex(a) in the case, for there are aggregations, it utilize the cube > support more reasonable Realization Chooser method in accordance with > specific SQL > -- > > Key: KYLIN-3444 > URL: https://issues.apache.org/jira/browse/KYLIN-3444 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > In Kylin，there could be some unexpected results in Choosing Realization. > For example, there are 2 models in the same project, called A and B, and each > of it owns a Cube and a TableIndex. just like this, > project > / \ > model(A) model(B) > / \ / \ > CUBE(a) TableIndex(a) CUBE(b) TableIndex(b) > Nowadays, the algorithm is designed to choose the matched models, and then > get into the model and select the proper and cheapest realizations(cube or > TableIndex here). However, it can cause some confusing issue, for example, > when running a query with agg functions (like "SELECT SUM(aa) from A GROUP BY > bb") and *the CUBE(a) cannot serve this query but CUBE(b) can*, it should be > hit the CUBE(b) rather than TableIndex(a) in the case, for there are > aggregations, utilizing the cube should be more efficient. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3444) support more reasonable Realization Chooser method in accordance with specific SQL
[ https://issues.apache.org/jira/browse/KYLIN-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3444: Description: In Kylin，there could be some unexpected results in Choosing Realization. For example, there are 2 models in the same project, called A and B, and each of it owns a Cube and a TableIndex. just like this, project / \ model(A) model(B) / \ / \ CUBE(a) TableIndex(a) CUBE(b) TableIndex(b) Nowadays, the algorithm is designed to choose the matched models, and then get into the model and select the proper and cheapest realizations(cube or TableIndex here). However, it can cause some confusing issue, for example, when running a query with agg functions (like "SELECT SUM(aa) from A GROUP BY bb") and *the CUBE(a) cannot serve this query but CUBE(b) can*, it should be hit the CUBE(b) rather than TableIndex(a) in the case, for there are aggregations, it utilize the cube was: In Kylin，there could be some unexpected results in Choosing Realization. For example, there are 2 models in the same project, called A and B, and each of it owns a Cube and a TableIndex. just like this, project / \ model(A) model(B) / \ / \ CUBE(a) TableIndex(a) CUBE(b) TableIndex(b) Nowadays, the algorithm is designed to choose the matched models, and then get into the model and select the proper and cheapest realizations(cube or TableIndex here). However, it can cause some confusing issue, for example, when running a query with agg functions (like "SELECT SUM(aa) from A GROUP BY bb") and *the CUBE(a) cannot serve this query but CUBE(b) can*, it should be hit the CUBE(b) rather than TableIndex(a) in the case, for there are aggregations, it use the Cube > support more reasonable Realization Chooser method in accordance with > specific SQL > -- > > Key: KYLIN-3444 > URL: https://issues.apache.org/jira/browse/KYLIN-3444 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > In Kylin，there could be some unexpected results in Choosing Realization. > For example, there are 2 models in the same project, called A and B, and each > of it owns a Cube and a TableIndex. just like this, > project > / \ > model(A) model(B) > / \ / \ > CUBE(a) TableIndex(a) CUBE(b) TableIndex(b) > Nowadays, the algorithm is designed to choose the matched models, and then > get into the model and select the proper and cheapest realizations(cube or > TableIndex here). However, it can cause some confusing issue, for example, > when running a query with agg functions (like "SELECT SUM(aa) from A GROUP BY > bb") and *the CUBE(a) cannot serve this query but CUBE(b) can*, it should be > hit the CUBE(b) rather than TableIndex(a) in the case, for there are > aggregations, it utilize the cube -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3444) support more reasonable Realization Chooser method in accordance with specific SQL
[ https://issues.apache.org/jira/browse/KYLIN-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3444: Description: In Kylin，there could be some unexpected results in Choosing Realization. For example, there are 2 models in the same project, called A and B, and each of it owns a Cube and a TableIndex. just like this, project / \ model(A) model(B) / \ / \ CUBE(a) TableIndex(a) CUBE(b) TableIndex(b) Nowadays, the algorithm is designed to choose the matched models, and then get into the model and select the proper and cheapest realizations(cube or TableIndex here). However, it can cause some confusing issue, for example, when running a query with agg functions (like "SELECT SUM(aa) from A GROUP BY bb") and *the CUBE(a) cannot serve this query but CUBE(b) can*, it should be hit the CUBE(b) rather than TableIndex(a) in the case, for there are aggregations, it use the Cube was:In Kylin，there could be some unexpected results in Choosing Realization. For example, there are 2 models in the same project, called A and B, and each of it own a Cube and a TableIndex. > support more reasonable Realization Chooser method in accordance with > specific SQL > -- > > Key: KYLIN-3444 > URL: https://issues.apache.org/jira/browse/KYLIN-3444 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > In Kylin，there could be some unexpected results in Choosing Realization. > For example, there are 2 models in the same project, called A and B, and each > of it owns a Cube and a TableIndex. just like this, > project > / \ > model(A) model(B) > / \ / \ > CUBE(a) TableIndex(a) CUBE(b) TableIndex(b) > Nowadays, the algorithm is designed to choose the matched models, and then > get into the model and select the proper and cheapest realizations(cube or > TableIndex here). However, it can cause some confusing issue, for example, > when running a query with agg functions (like "SELECT SUM(aa) from A GROUP BY > bb") and *the CUBE(a) cannot serve this query but CUBE(b) can*, it should be > hit the CUBE(b) rather than TableIndex(a) in the case, for there are > aggregations, it use the Cube -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3444) support more reasonable Realization Chooser method in accordance with specific SQL
[ https://issues.apache.org/jira/browse/KYLIN-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3444: Description: In Kylin，there could be some unexpected results in Choosing Realization. For example, there are 2 models in the same project, called A and B, and each of it own a Cube and a TableIndex. > support more reasonable Realization Chooser method in accordance with > specific SQL > -- > > Key: KYLIN-3444 > URL: https://issues.apache.org/jira/browse/KYLIN-3444 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > In Kylin，there could be some unexpected results in Choosing Realization. For > example, there are 2 models in the same project, called A and B, and each of > it own a Cube and a TableIndex. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3444) support more reasonable Realization Chooser method in accordance with specific SQL
[ https://issues.apache.org/jira/browse/KYLIN-3444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu reassigned KYLIN-3444: --- Assignee: Yifei Wu > support more reasonable Realization Chooser method in accordance with > specific SQL > -- > > Key: KYLIN-3444 > URL: https://issues.apache.org/jira/browse/KYLIN-3444 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3444) support more reasonable Realization Chooser method in accordance with specific SQL
Yifei Wu created KYLIN-3444: --- Summary: support more reasonable Realization Chooser method in accordance with specific SQL Key: KYLIN-3444 URL: https://issues.apache.org/jira/browse/KYLIN-3444 Project: Kylin Issue Type: Improvement Reporter: Yifei Wu -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3392) support NULL value in Sum, Max, Min Aggregation
[ https://issues.apache.org/jira/browse/KYLIN-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494768#comment-16494768 ] Yifei Wu edited comment on KYLIN-3392 at 5/30/18 6:56 AM: -- [~albertoramon] you mean this issue KYLIN_2049 [KYLIN-2049|https://issues.apache.org/jira/browse/KYLIN-2049] was (Author: yifei_wu94): [~albertoramon] you mean this issue KYLIN_2049 > support NULL value in Sum, Max, Min Aggregation > --- > > Key: KYLIN-3392 > URL: https://issues.apache.org/jira/browse/KYLIN-3392 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > It is treated as 0 when confronted with NULL value in KYLIN's basic aggregate > measure (like sum, max, min). However, to distinguish the NULL value with 0 > is very necessary. > It should be like this > *sum(null, null) = null* > *sum(null, 1) = 1* > *max(null, null) = null* > *max(null, -1) = -1* > *min(null, -1）= -1* > in accordance with Hive and SparkSQL -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3392) support NULL value in Sum, Max, Min Aggregation
[ https://issues.apache.org/jira/browse/KYLIN-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494768#comment-16494768 ] Yifei Wu commented on KYLIN-3392: - [~albertoramon] you mean this issue KYLIN_2049 > support NULL value in Sum, Max, Min Aggregation > --- > > Key: KYLIN-3392 > URL: https://issues.apache.org/jira/browse/KYLIN-3392 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > It is treated as 0 when confronted with NULL value in KYLIN's basic aggregate > measure (like sum, max, min). However, to distinguish the NULL value with 0 > is very necessary. > It should be like this > *sum(null, null) = null* > *sum(null, 1) = 1* > *max(null, null) = null* > *max(null, -1) = -1* > *min(null, -1）= -1* > in accordance with Hive and SparkSQL -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3392) support NULL value in Sum, Max, Min Aggregation
[ https://issues.apache.org/jira/browse/KYLIN-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16493495#comment-16493495 ] Yifei Wu commented on KYLIN-3392: - To solve this bug, It is reasonable to add extra code for LONG.null, DOUBLE.null, BigDecimal.null, which will be saved in build cube. > support NULL value in Sum, Max, Min Aggregation > --- > > Key: KYLIN-3392 > URL: https://issues.apache.org/jira/browse/KYLIN-3392 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > It is treated as 0 when confronted with NULL value in KYLIN's basic aggregate > measure (like sum, max, min). However, to distinguish the NULL value with 0 > is very necessary. > It should be like this > *sum(null, null) = null* > *sum(null, 1) = 1* > *max(null, null) = null* > *max(null, -1) = -1* > *min(null, -1）= -1* > in accordance with Hive and SparkSQL -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3392) support NULL value in Sum, Max, Min Aggregation
Yifei Wu created KYLIN-3392: --- Summary: support NULL value in Sum, Max, Min Aggregation Key: KYLIN-3392 URL: https://issues.apache.org/jira/browse/KYLIN-3392 Project: Kylin Issue Type: Bug Reporter: Yifei Wu Assignee: Yifei Wu It is treated as 0 when confronted with NULL value in KYLIN's basic aggregate measure (like sum, max, min). However, to distinguish the NULL value with 0 is very necessary. It should be like this *sum(null, null) = null* *sum(null, 1) = 1* *max(null, null) = null* *max(null, -1) = -1* *min(null, -1）= -1* in accordance with Hive and SparkSQL -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3387) validate duplicate check after TopN remove dumplicate meaures automatically
[ https://issues.apache.org/jira/browse/KYLIN-3387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3387: Summary: validate duplicate check after TopN remove dumplicate meaures automatically (was: validate duplicate check after TopN remove dumplicate meaures automaticaly) > validate duplicate check after TopN remove dumplicate meaures automatically > --- > > Key: KYLIN-3387 > URL: https://issues.apache.org/jira/browse/KYLIN-3387 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3387) validate duplicate check after TopN remove dumplicate meaures automaticaly
Yifei Wu created KYLIN-3387: --- Summary: validate duplicate check after TopN remove dumplicate meaures automaticaly Key: KYLIN-3387 URL: https://issues.apache.org/jira/browse/KYLIN-3387 Project: Kylin Issue Type: Improvement Reporter: Yifei Wu Assignee: Yifei Wu -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3386) the TopN measure validate code refactor to make it more clear
[ https://issues.apache.org/jira/browse/KYLIN-3386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3386: Attachment: 0001-KYLIN-3386-adjust-TopN-measure-validation-code.patch > the TopN measure validate code refactor to make it more clear > - > > Key: KYLIN-3386 > URL: https://issues.apache.org/jira/browse/KYLIN-3386 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Minor > Attachments: 0001-KYLIN-3386-adjust-TopN-measure-validation-code.patch > > > _Nowadays, the FunctionRule used for validating measure info, and it should > call measureType.validate() code rather than add the logical in this 2 > places_ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3386) the TopN measure validate code refactor to make it more clear
[ https://issues.apache.org/jira/browse/KYLIN-3386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490160#comment-16490160 ] Yifei Wu edited comment on KYLIN-3386 at 5/25/18 3:55 AM: -- here is the patch[^0001-KYLIN-3386-adjust-TopN-measure-validation-code.patch] was (Author: yifei_wu94): here is the patch > the TopN measure validate code refactor to make it more clear > - > > Key: KYLIN-3386 > URL: https://issues.apache.org/jira/browse/KYLIN-3386 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Minor > Attachments: 0001-KYLIN-3386-adjust-TopN-measure-validation-code.patch > > > _Nowadays, the FunctionRule used for validating measure info, and it should > call measureType.validate() code rather than add the logical in this 2 > places_ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3386) the TopN measure validate code refactor to make it more clear
[ https://issues.apache.org/jira/browse/KYLIN-3386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3386: Attachment: (was: 0001-KYLIN-3386-adjust-TopN-measure-validation-code.patch) > the TopN measure validate code refactor to make it more clear > - > > Key: KYLIN-3386 > URL: https://issues.apache.org/jira/browse/KYLIN-3386 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Minor > > _Nowadays, the FunctionRule used for validating measure info, and it should > call measureType.validate() code rather than add the logical in this 2 > places_ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3386) the TopN measure validate code refactor to make it more clear
[ https://issues.apache.org/jira/browse/KYLIN-3386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490160#comment-16490160 ] Yifei Wu edited comment on KYLIN-3386 at 5/25/18 3:54 AM: -- here is the patch was (Author: yifei_wu94): here is the patch [^0001-KYLIN-3386-adjust-TopN-measure-validation-code.patch] > the TopN measure validate code refactor to make it more clear > - > > Key: KYLIN-3386 > URL: https://issues.apache.org/jira/browse/KYLIN-3386 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Minor > > _Nowadays, the FunctionRule used for validating measure info, and it should > call measureType.validate() code rather than add the logical in this 2 > places_ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3386) the TopN measure validate code refactor to make it more clear
[ https://issues.apache.org/jira/browse/KYLIN-3386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3386: Attachment: 0001-KYLIN-3386-adjust-TopN-measure-validation-code.patch > the TopN measure validate code refactor to make it more clear > - > > Key: KYLIN-3386 > URL: https://issues.apache.org/jira/browse/KYLIN-3386 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Minor > Attachments: 0001-KYLIN-3386-adjust-TopN-measure-validation-code.patch > > > _Nowadays, the FunctionRule used for validating measure info, and it should > call measureType.validate() code rather than add the logical in this 2 > places_ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3386) the TopN measure validate code refactor to make it more clear
[ https://issues.apache.org/jira/browse/KYLIN-3386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490160#comment-16490160 ] Yifei Wu commented on KYLIN-3386: - here is the patch [^0001-KYLIN-3386-adjust-TopN-measure-validation-code.patch] > the TopN measure validate code refactor to make it more clear > - > > Key: KYLIN-3386 > URL: https://issues.apache.org/jira/browse/KYLIN-3386 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Minor > Attachments: 0001-KYLIN-3386-adjust-TopN-measure-validation-code.patch > > > _Nowadays, the FunctionRule used for validating measure info, and it should > call measureType.validate() code rather than add the logical in this 2 > places_ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3386) the TopN measure validate code refactor to make it more clear
[ https://issues.apache.org/jira/browse/KYLIN-3386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3386: Description: _Nowadays, the FunctionRule used for validating measure info, and it should call measureType.validate() code rather than add the logical in this 2 places_ > the TopN measure validate code refactor to make it more clear > - > > Key: KYLIN-3386 > URL: https://issues.apache.org/jira/browse/KYLIN-3386 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Minor > > _Nowadays, the FunctionRule used for validating measure info, and it should > call measureType.validate() code rather than add the logical in this 2 > places_ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3385) when the cube contain sum(1) measure, tables in "Query" page cannot be load
[
https://issues.apache.org/jira/browse/KYLIN-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu updated KYLIN-3385:
Attachment: 0001-KYLIN-3385-fix-sum-1.patch
> when the cube contain sum(1) measure, tables in "Query" page cannot be load
> ---
>
> Key: KYLIN-3385
> URL: https://issues.apache.org/jira/browse/KYLIN-3385
> Project: Kylin
> Issue Type: Bug
>Reporter: Yifei Wu
>Assignee: Yifei Wu
>Priority: Major
> Attachments: 0001-KYLIN-3385-fix-sum-1.patch
>
>
> In the current version, the sum(1), equals with count(*) actually, is
> allowed to add to cube Measures.
> However, if you add, it will cause some errors, such as tables in "Query"
> page cannot be load and the query like this cannot run.
> {code:java}
> select sum(1) from kylin_fact
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3386) the TopN measure validate code refactor to make it more clear
[ https://issues.apache.org/jira/browse/KYLIN-3386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3386: Priority: Minor (was: Major) > the TopN measure validate code refactor to make it more clear > - > > Key: KYLIN-3386 > URL: https://issues.apache.org/jira/browse/KYLIN-3386 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3386) the TopN measure validate code refactor to make it more clear
[ https://issues.apache.org/jira/browse/KYLIN-3386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu reassigned KYLIN-3386: --- Assignee: Yifei Wu > the TopN measure validate code refactor to make it more clear > - > > Key: KYLIN-3386 > URL: https://issues.apache.org/jira/browse/KYLIN-3386 > Project: Kylin > Issue Type: Improvement >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3386) the TopN measure validate code refactor to make it more clear
Yifei Wu created KYLIN-3386: --- Summary: the TopN measure validate code refactor to make it more clear Key: KYLIN-3386 URL: https://issues.apache.org/jira/browse/KYLIN-3386 Project: Kylin Issue Type: Improvement Reporter: Yifei Wu -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3385) when the cube contain sum(1) measure, tables in "Query" page cannot be load
[
https://issues.apache.org/jira/browse/KYLIN-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490147#comment-16490147
]
Yifei Wu commented on KYLIN-3385:
-
here is the patch:
[^0001-KYLIN-3385-fix-sum-1.patch]
> when the cube contain sum(1) measure, tables in "Query" page cannot be load
> ---
>
> Key: KYLIN-3385
> URL: https://issues.apache.org/jira/browse/KYLIN-3385
> Project: Kylin
> Issue Type: Bug
>Reporter: Yifei Wu
>Assignee: Yifei Wu
>Priority: Major
> Attachments: 0001-KYLIN-3385-fix-sum-1.patch
>
>
> In the current version, the sum(1), equals with count(*) actually, is
> allowed to add to cube Measures.
> However, if you add, it will cause some errors, such as tables in "Query"
> page cannot be load and the query like this cannot run.
> {code:java}
> select sum(1) from kylin_fact
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3385) when the cube contain sum(1) measure, tables in "Query" page cannot be load
[
https://issues.apache.org/jira/browse/KYLIN-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490128#comment-16490128
]
Yifei Wu commented on KYLIN-3385:
-
according to my analysis, the root cause is that
org.apache.kylin.metadata.model.FunctionDesc#getRewriteFieldType don't deal
with the case, so when load tables here will throw an exception.
> when the cube contain sum(1) measure, tables in "Query" page cannot be load
> ---
>
> Key: KYLIN-3385
> URL: https://issues.apache.org/jira/browse/KYLIN-3385
> Project: Kylin
> Issue Type: Bug
>Reporter: Yifei Wu
>Assignee: Yifei Wu
>Priority: Major
>
> In the current version, the sum(1), equals with count(*) actually, is
> allowed to add to cube Measures.
> However, if you add, it will cause some errors, such as tables in "Query"
> page cannot be load and the query like this cannot run.
> {code:java}
> select sum(1) from kylin_fact
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3385) when the cube contain sum(1) measure, tables in "Query" page cannot be load
[
https://issues.apache.org/jira/browse/KYLIN-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu updated KYLIN-3385:
Description:
In the current version, the sum(1), equals with count(*) actually, is allowed
to add to cube Measures.
However, if you add, it will cause some errors, such as tables in "Query" page
cannot be load and the query like this cannot run.
{code:java}
select sum(1) from kylin_fact
{code}
was:
In the current version, the sum(1), equals with count(*) actually, is allowed
to add to cube Measures.
However, if you add, it will cause some errors, such as tables in "Query" page
cannot be load and the query like this also be not right.
{code:java}
select sum(1) from kylin_fact
{code}
> when the cube contain sum(1) measure, tables in "Query" page cannot be load
> ---
>
> Key: KYLIN-3385
> URL: https://issues.apache.org/jira/browse/KYLIN-3385
> Project: Kylin
> Issue Type: Bug
>Reporter: Yifei Wu
>Assignee: Yifei Wu
>Priority: Major
>
> In the current version, the sum(1), equals with count(*) actually, is
> allowed to add to cube Measures.
> However, if you add, it will cause some errors, such as tables in "Query"
> page cannot be load and the query like this cannot run.
> {code:java}
> select sum(1) from kylin_fact
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3385) when the cube contain sum(1) measure, tables in "Query" page cannot be load
[
https://issues.apache.org/jira/browse/KYLIN-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu updated KYLIN-3385:
Description:
In the current version, the sum(1), equals with count(*) actually, is allowed
to add to cube Measures.
However, if you add, it will cause some errors, such as tables in "Query" page
cannot be load and the query like this also be not right.
{code:java}
select sum(1) from kylin_fact
{code}
was:
In the current version, the sum(1), equals with count(*) actually, is allowed
to add to cube Measures.
However, if you add, it will cause some errors, such as the the tables
> when the cube contain sum(1) measure, tables in "Query" page cannot be load
> ---
>
> Key: KYLIN-3385
> URL: https://issues.apache.org/jira/browse/KYLIN-3385
> Project: Kylin
> Issue Type: Bug
>Reporter: Yifei Wu
>Assignee: Yifei Wu
>Priority: Major
>
> In the current version, the sum(1), equals with count(*) actually, is
> allowed to add to cube Measures.
> However, if you add, it will cause some errors, such as tables in "Query"
> page cannot be load and the query like this also be not right.
> {code:java}
> select sum(1) from kylin_fact
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3385) when the cube contain sum(1) measure, the tables in "Query" page cannot be load
[ https://issues.apache.org/jira/browse/KYLIN-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3385: Description: In the current version, the sum(1), equals with count(*) actually, is allowed to add to cube Measures. However, if you add, it will cause some errors, such as the the tables was:In current version, the sum(1), equals with count(*) actually, is allowed to add to cube Measure. > when the cube contain sum(1) measure, the tables in "Query" page cannot be > load > --- > > Key: KYLIN-3385 > URL: https://issues.apache.org/jira/browse/KYLIN-3385 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > In the current version, the sum(1), equals with count(*) actually, is > allowed to add to cube Measures. > However, if you add, it will cause some errors, such as the the tables -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3385) when the cube contain sum(1) measure, tables in "Query" page cannot be load
[ https://issues.apache.org/jira/browse/KYLIN-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3385: Summary: when the cube contain sum(1) measure, tables in "Query" page cannot be load (was: when the cube contain sum(1) measure, the tables in "Query" page cannot be load) > when the cube contain sum(1) measure, tables in "Query" page cannot be load > --- > > Key: KYLIN-3385 > URL: https://issues.apache.org/jira/browse/KYLIN-3385 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > In the current version, the sum(1), equals with count(*) actually, is > allowed to add to cube Measures. > However, if you add, it will cause some errors, such as the the tables -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3385) when the cube contain sum(1) measure, the tables in "Query" page cannot be load
[ https://issues.apache.org/jira/browse/KYLIN-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3385: Description: In current version, the sum(1), equals with count(*) actually, is allowed to add to cube Measure. > when the cube contain sum(1) measure, the tables in "Query" page cannot be > load > --- > > Key: KYLIN-3385 > URL: https://issues.apache.org/jira/browse/KYLIN-3385 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > In current version, the sum(1), equals with count(*) actually, is allowed to > add to cube Measure. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3385) when the cube contain sum(1) measure, the tables in "Query" page cannot be load
[ https://issues.apache.org/jira/browse/KYLIN-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu reassigned KYLIN-3385: --- Assignee: Yifei Wu > when the cube contain sum(1) measure, the tables in "Query" page cannot be > load > --- > > Key: KYLIN-3385 > URL: https://issues.apache.org/jira/browse/KYLIN-3385 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3267) add override MR config at project/cube level only for mem-hungry build steps
[
https://issues.apache.org/jira/browse/KYLIN-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu updated KYLIN-3267:
Description:
Currently, users can override MR config at project/cube level via kylin
properties {{kylin.engine.mr.config-override.}}
However, this applies to all MR steps. It is more reasonable to add
override-configs only for certain memory-hungry steps like in-mem cubing.
was:
Currently, users can override MR config at project/cube level via kylin
properties {{kylin.engine.mr.config-override.}}
However, this applies to all MR steps. It is more reasonable to add
override-configs only for certain memory-hungry steps like in-mem cubing and
"Build columnar page index"
Summary: add override MR config at project/cube level only for
mem-hungry build steps (was: add override MR config at project/cube level
only for mem-hungry build steps like "Build columnar page index")
> add override MR config at project/cube level only for mem-hungry build steps
> -
>
> Key: KYLIN-3267
> URL: https://issues.apache.org/jira/browse/KYLIN-3267
> Project: Kylin
> Issue Type: Bug
>Reporter: Yifei Wu
>Assignee: Yifei Wu
>Priority: Major
>
> Currently, users can override MR config at project/cube level via kylin
> properties {{kylin.engine.mr.config-override.}}
> However, this applies to all MR steps. It is more reasonable to add
> override-configs only for certain memory-hungry steps like in-mem cubing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3267) add override MR config at project/cube level only for mem-hungry build steps like "Build columnar page index"
[
https://issues.apache.org/jira/browse/KYLIN-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387637#comment-16387637
]
Yifei Wu edited comment on KYLIN-3267 at 3/6/18 11:13 AM:
--
My mistake，I will modify the title and description and be more cautious in
later.[~Shaofengshi]
> add override MR config at project/cube level only for mem-hungry build steps
> like "Build columnar page index"
> -
>
> Key: KYLIN-3267
> URL: https://issues.apache.org/jira/browse/KYLIN-3267
> Project: Kylin
> Issue Type: Bug
>Reporter: Yifei Wu
>Assignee: Yifei Wu
>Priority: Major
>
> Currently, users can override MR config at project/cube level via kylin
> properties {{kylin.engine.mr.config-override.}}
> However, this applies to all MR steps. It is more reasonable to add
> override-configs only for certain memory-hungry steps like in-mem cubing and
> "Build columnar page index"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3267) add override MR config at project/cube level only for mem-hungry build steps like "Build columnar page index"
[
https://issues.apache.org/jira/browse/KYLIN-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu updated KYLIN-3267:
Description:
Currently, users can override MR config at project/cube level via kylin
properties {{kylin.engine.mr.config-override.}}
However, this applies to all MR steps. It is more reasonable to add
override-configs only for certain memory-hungry steps like in-mem cubing and
"Build columnar page index"
was:
Currently user can override MR config at project/cube level via kylin
properties {{kylin.engine.mr.config-override.}}
However this applies to all MR steps.
Add override-configs that enable similar project/cube level config, but only
for certain memory-hungry steps like in-mem cubing and "Build columnar page
index"
> add override MR config at project/cube level only for mem-hungry build steps
> like "Build columnar page index"
> -
>
> Key: KYLIN-3267
> URL: https://issues.apache.org/jira/browse/KYLIN-3267
> Project: Kylin
> Issue Type: Bug
>Reporter: Yifei Wu
>Assignee: Yifei Wu
>Priority: Major
>
> Currently, users can override MR config at project/cube level via kylin
> properties {{kylin.engine.mr.config-override.}}
> However, this applies to all MR steps. It is more reasonable to add
> override-configs only for certain memory-hungry steps like in-mem cubing and
> "Build columnar page index"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3267) add override MR config at project/cube level only for mem-hungry build steps like "Build columnar page index"
[
https://issues.apache.org/jira/browse/KYLIN-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu updated KYLIN-3267:
Description:
Currently user can override MR config at project/cube level via kylin
properties {{kylin.engine.mr.config-override.}}
However this applies to all MR steps.
Add override-configs that enable similar project/cube level config, but only
for certain memory-hungry steps like in-mem cubing and "Build columnar page
index"
> add override MR config at project/cube level only for mem-hungry build steps
> like "Build columnar page index"
> -
>
> Key: KYLIN-3267
> URL: https://issues.apache.org/jira/browse/KYLIN-3267
> Project: Kylin
> Issue Type: Bug
>Reporter: Yifei Wu
>Assignee: Yifei Wu
>Priority: Major
>
> Currently user can override MR config at project/cube level via kylin
> properties {{kylin.engine.mr.config-override.}}
> However this applies to all MR steps.
> Add override-configs that enable similar project/cube level config, but only
> for certain memory-hungry steps like in-mem cubing and "Build columnar page
> index"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3267) add override MR config at project/cube level only for mem-hungry build steps like "Build columnar page index"
Yifei Wu created KYLIN-3267: --- Summary: add override MR config at project/cube level only for mem-hungry build steps like "Build columnar page index" Key: KYLIN-3267 URL: https://issues.apache.org/jira/browse/KYLIN-3267 Project: Kylin Issue Type: Bug Reporter: Yifei Wu Assignee: Yifei Wu -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3217) Dimension As Measures should only support Max and Min when the first table in select SQL is fact table
[ https://issues.apache.org/jira/browse/KYLIN-3217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3217: Description: DemensionAsMeaures is a way to get the SQL query's result directly through making use of Calcite, when several special measures cannot be matched with the pre-calculated cube. But it should only support the Max and Min because the data in cubes has been aggregated so as to unaccurate result. (was: DemensionAsMeaures is a way to get the SQL query's result directly through making use of Calcite, when servral special ) > Dimension As Measures should only support Max and Min when the first table in > select SQL is fact table > -- > > Key: KYLIN-3217 > URL: https://issues.apache.org/jira/browse/KYLIN-3217 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Minor > > DemensionAsMeaures is a way to get the SQL query's result directly through > making use of Calcite, when several special measures cannot be matched with > the pre-calculated cube. But it should only support the Max and Min because > the data in cubes has been aggregated so as to unaccurate result. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3217) Dimension As Measures should only support Max and Min when the first table in select SQL is fact table
[ https://issues.apache.org/jira/browse/KYLIN-3217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3217: Description: DemensionAsMeaures is a way to get the SQL query's result directly through making use of Calcite, when several special measures cannot be matched with the pre-calculated cube. But it should only support the Max and Min when selected from the fact table, because the data in cubes has been aggregated so as to inaccurate results. (was: DemensionAsMeaures is a way to get the SQL query's result directly through making use of Calcite, when several special measures cannot be matched with the pre-calculated cube. But it should only support the Max and Min because the data in cubes has been aggregated so as to unaccurate result.) > Dimension As Measures should only support Max and Min when the first table in > select SQL is fact table > -- > > Key: KYLIN-3217 > URL: https://issues.apache.org/jira/browse/KYLIN-3217 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Minor > > DemensionAsMeaures is a way to get the SQL query's result directly through > making use of Calcite, when several special measures cannot be matched with > the pre-calculated cube. But it should only support the Max and Min when > selected from the fact table, because the data in cubes has been aggregated > so as to inaccurate results. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3217) Dimension As Measures should only support Max and Min when the first table in select SQL is fact table
[ https://issues.apache.org/jira/browse/KYLIN-3217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3217: Description: DemensionAsMeaures is a way to get the SQL query's result directly through making use of Calcite, when servral special > Dimension As Measures should only support Max and Min when the first table in > select SQL is fact table > -- > > Key: KYLIN-3217 > URL: https://issues.apache.org/jira/browse/KYLIN-3217 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Minor > > DemensionAsMeaures is a way to get the SQL query's result directly through > making use of Calcite, when servral special -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3217) Dimension As Measures should only support Max and Min when the first table in select SQL is fact table
Yifei Wu created KYLIN-3217: --- Summary: Dimension As Measures should only support Max and Min when the first table in select SQL is fact table Key: KYLIN-3217 URL: https://issues.apache.org/jira/browse/KYLIN-3217 Project: Kylin Issue Type: Bug Reporter: Yifei Wu Assignee: Yifei Wu -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3211) Class IntegerDimEnc shuould give more exception information when the length is exceed the max or less than the min
[
https://issues.apache.org/jira/browse/KYLIN-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu updated KYLIN-3211:
Description:
the message IllegalArgumentException in Class IntegerDimEnc is empty currently,
it may cause confusion when using Kylin or troubleshooting a problem.
{code:java}
// code placeholder
public IntDimEnc(int len) {
if (len <= 0 || len >= CAP.length)
throw new IllegalArgumentException();
this.fixedLen = len;
}
{code}
was:the message IllegalArgumentException
> Class IntegerDimEnc shuould give more exception information when the length
> is exceed the max or less than the min
> --
>
> Key: KYLIN-3211
> URL: https://issues.apache.org/jira/browse/KYLIN-3211
> Project: Kylin
> Issue Type: Bug
>Reporter: Yifei Wu
>Assignee: Yifei Wu
>Priority: Major
>
> the message IllegalArgumentException in Class IntegerDimEnc is empty
> currently, it may cause confusion when using Kylin or troubleshooting a
> problem.
> {code:java}
> // code placeholder
> public IntDimEnc(int len) {
> if (len <= 0 || len >= CAP.length)
> throw new IllegalArgumentException();
> this.fixedLen = len;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3211) Class IntegerDimEnc shuould give more exception information when the length is exceed the max or less than the min
[ https://issues.apache.org/jira/browse/KYLIN-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3211: Summary: Class IntegerDimEnc shuould give more exception information when the length is exceed the max or less than the min (was: Class IntegerDimEnc shuould give more exception information when the length is exceed the max or less than the min e) > Class IntegerDimEnc shuould give more exception information when the length > is exceed the max or less than the min > -- > > Key: KYLIN-3211 > URL: https://issues.apache.org/jira/browse/KYLIN-3211 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > the message IllegalArgumentException -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3211) Class IntegerDimEnc shuould give more exception information when the length is exceed the max or less than the min e
[ https://issues.apache.org/jira/browse/KYLIN-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3211: Summary: Class IntegerDimEnc shuould give more exception information when the length is exceed the max or less than the min e (was: Class IntegerDimEnc shuould give more excetption information when the length is exceed the max or less than the min e) > Class IntegerDimEnc shuould give more exception information when the length > is exceed the max or less than the min e > > > Key: KYLIN-3211 > URL: https://issues.apache.org/jira/browse/KYLIN-3211 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > the message IllegalArgumentException -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3211) Class IntegerDimEnc shuould give more excetption information when the length is exceed the max or less than the min e
[ https://issues.apache.org/jira/browse/KYLIN-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3211: Summary: Class IntegerDimEnc shuould give more excetption information when the length is exceed the max or less than the min e (was: Class IntegerDimEnc shuould give more exception information when the length is exceed the max or less than the min ) > Class IntegerDimEnc shuould give more excetption information when the length > is exceed the max or less than the min e > - > > Key: KYLIN-3211 > URL: https://issues.apache.org/jira/browse/KYLIN-3211 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > the message IllegalArgumentException -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3211) Class IntegerDimEnc shuould give more exception information when the length is exceed the max or less than the min
[ https://issues.apache.org/jira/browse/KYLIN-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3211: Description: the message IllegalArgumentException > Class IntegerDimEnc shuould give more exception information when the length > is exceed the max or less than the min > --- > > Key: KYLIN-3211 > URL: https://issues.apache.org/jira/browse/KYLIN-3211 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Major > > the message IllegalArgumentException -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3211) Class IntegerDimEnc shuould give more exception information when the length is exceed the max or less than the min
Yifei Wu created KYLIN-3211: --- Summary: Class IntegerDimEnc shuould give more exception information when the length is exceed the max or less than the min Key: KYLIN-3211 URL: https://issues.apache.org/jira/browse/KYLIN-3211 Project: Kylin Issue Type: Bug Reporter: Yifei Wu Assignee: Yifei Wu -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3156) Failed to delete meta path in SparkCubingByLayer
[
https://issues.apache.org/jira/browse/KYLIN-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16337056#comment-16337056
]
Yifei Wu commented on KYLIN-3156:
-
[~yimingliu] OK, but it looks like need to take some time to get the root cause
and fix it.
[~liushaohui] it has tested the Spark Cubing when fixing KYLIN-2945, So if
convenient, can you provided your metadata info and more logs about it?
> Failed to delete meta path in SparkCubingByLayer
>
>
> Key: KYLIN-3156
> URL: https://issues.apache.org/jira/browse/KYLIN-3156
> Project: Kylin
> Issue Type: Bug
> Components: Spark Engine
>Reporter: Liu Shaohui
>Assignee: Yifei Wu
>Priority: Major
>
> After KYLIN-2945, the meta url in SparkCubingByLayer will be a string of
> StorageURL not string with path@hdfs format. This will make the
> deleteHDFSMeta method failed in SparkCubingByLayer.
> {quote}
> 2018-01-08,11:51:50,903 ERROR org.apache.spark.deploy.yarn.ApplicationMaster:
> User class threw exception: java.lang.RuntimeException: error execute
> org.apache.kylin.engine.spark.SparkCubingByLayer
> java.lang.RuntimeException: error execute
> org.apache.kylin.engine.spark.SparkCubingByLayer
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42)
> at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:653)
> Caused by: java.lang.IllegalArgumentException: Cannot create FileSystem from
> URI: kylin_tst:kylin_metadata
> at org.apache.kylin.common.util.HadoopUtil.makeURI(HadoopUtil.java:98)
> at
> org.apache.kylin.common.util.HadoopUtil.getFileSystem(HadoopUtil.java:78)
> at
> org.apache.kylin.engine.spark.SparkCubingByLayer.deleteHDFSMeta(SparkCubingByLayer.java:484)
> at
> org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:207)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
> ... 6 more
> Caused by: java.net.URISyntaxException: Illegal character in scheme name at
> index 5: kylin_tst:kylin_metadata
> at java.net.URI$Parser.fail(URI.java:2848)
> at java.net.URI$Parser.checkChars(URI.java:3021)
> at java.net.URI$Parser.parse(URI.java:3048)
> at java.net.URI.(URI.java:588)
> at org.apache.kylin.common.util.HadoopUtil.makeURI(HadoopUtil.java:96)
> ... 10 more
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KYLIN-3156) Failed to delete meta path in SparkCubingByLayer
[
https://issues.apache.org/jira/browse/KYLIN-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu reassigned KYLIN-3156:
---
Assignee: Yifei Wu
> Failed to delete meta path in SparkCubingByLayer
>
>
> Key: KYLIN-3156
> URL: https://issues.apache.org/jira/browse/KYLIN-3156
> Project: Kylin
> Issue Type: Bug
> Components: Spark Engine
>Reporter: Liu Shaohui
>Assignee: Yifei Wu
>Priority: Major
>
> After KYLIN-2945, the meta url in SparkCubingByLayer will be a string of
> StorageURL not string with path@hdfs format. This will make the
> deleteHDFSMeta method failed in SparkCubingByLayer.
> {quote}
> 2018-01-08,11:51:50,903 ERROR org.apache.spark.deploy.yarn.ApplicationMaster:
> User class threw exception: java.lang.RuntimeException: error execute
> org.apache.kylin.engine.spark.SparkCubingByLayer
> java.lang.RuntimeException: error execute
> org.apache.kylin.engine.spark.SparkCubingByLayer
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42)
> at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:653)
> Caused by: java.lang.IllegalArgumentException: Cannot create FileSystem from
> URI: kylin_tst:kylin_metadata
> at org.apache.kylin.common.util.HadoopUtil.makeURI(HadoopUtil.java:98)
> at
> org.apache.kylin.common.util.HadoopUtil.getFileSystem(HadoopUtil.java:78)
> at
> org.apache.kylin.engine.spark.SparkCubingByLayer.deleteHDFSMeta(SparkCubingByLayer.java:484)
> at
> org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:207)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
> ... 6 more
> Caused by: java.net.URISyntaxException: Illegal character in scheme name at
> index 5: kylin_tst:kylin_metadata
> at java.net.URI$Parser.fail(URI.java:2848)
> at java.net.URI$Parser.checkChars(URI.java:3021)
> at java.net.URI$Parser.parse(URI.java:3048)
> at java.net.URI.(URI.java:588)
> at org.apache.kylin.common.util.HadoopUtil.makeURI(HadoopUtil.java:96)
> ... 10 more
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3078) the estimated size of percentile measure is too big
[ https://issues.apache.org/jira/browse/KYLIN-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16275628#comment-16275628 ] Yifei Wu commented on KYLIN-3078: - the key is to clarify the percentile impact on cube size estimate and find a more proper way to estimate the size of percentile measure. For the measure use the T-digest Algorithm to realize it, so it can conclude some regular pattern by the analysis from the T-digest paper and the statistics collected in the local test. > the estimated size of percentile measure is too big > > > Key: KYLIN-3078 > URL: https://issues.apache.org/jira/browse/KYLIN-3078 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Critical > > To set a shard number that will be for controlling the size per shard > properly, we need to estimate cube size through accumulating all dimension > and measure size roughly before building a cube. But the way of calculating > the percentile measure is inaccurate currently and cause too many partitions > for cube storage. Furthermore, it may affect the performance of SQL query. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-3078) the estimated size of percentile measure is too big
[ https://issues.apache.org/jira/browse/KYLIN-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3078: Description: To set a shard number that will be for controlling the size per shard properly, we need to estimate cube size through accumulating all dimension and measure size roughly before building a cube. But the way of calculating the percentile measure is inaccurate currently and cause too many partitions for cube storage. Furthermore, it may affect the performance of SQL query. (was: To set a shard number that will be for controlling the size per shard properly, we need to estimate cube size through accumulating all dimension and measure size roughly before building a cube. But the way of calculating the percentile measure is inaccurate currently and cause too many partitions for cube storage. Furthermore, it may affect the sql query. ) > the estimated size of percentile measure is too big > > > Key: KYLIN-3078 > URL: https://issues.apache.org/jira/browse/KYLIN-3078 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Critical > > To set a shard number that will be for controlling the size per shard > properly, we need to estimate cube size through accumulating all dimension > and measure size roughly before building a cube. But the way of calculating > the percentile measure is inaccurate currently and cause too many partitions > for cube storage. Furthermore, it may affect the performance of SQL query. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-3078) the estimated size of percentile measure is too big
[ https://issues.apache.org/jira/browse/KYLIN-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3078: Description: To set a shard number that will be for controlling the size per shard properly, we need to estimate cube size through accumulating all dimension and measure size roughly before building a cube. But the way of calculating the percentile measure is inaccurate currently and cause too many partitions for cube storage. Furthermore, it may affect the sql query. (was: To set a shard number that will be for controlling the size per shard properly, we need to estimate cube size through accumulating all dimension and measure size roughly before building cube.) > the estimated size of percentile measure is too big > > > Key: KYLIN-3078 > URL: https://issues.apache.org/jira/browse/KYLIN-3078 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Critical > > To set a shard number that will be for controlling the size per shard > properly, we need to estimate cube size through accumulating all dimension > and measure size roughly before building a cube. But the way of calculating > the percentile measure is inaccurate currently and cause too many partitions > for cube storage. Furthermore, it may affect the sql query. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-3078) the estimated size of percentile measure is too big
[ https://issues.apache.org/jira/browse/KYLIN-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3078: Priority: Critical (was: Major) > the estimated size of percentile measure is too big > > > Key: KYLIN-3078 > URL: https://issues.apache.org/jira/browse/KYLIN-3078 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Critical > > To set a shard number that will be for controlling the size per shard > properly, we need to estimate cube size through accumulating all dimension > and measure size roughly before building cube -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-3078) the estimated size of percentile measure is too big
[ https://issues.apache.org/jira/browse/KYLIN-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3078: Description: To set a shard number that will be for controlling the size per shard properly, we need to estimate cube size through accumulating all dimension and measure size roughly before building cube. (was: To set a shard number that will be for controlling the size per shard properly, we need to estimate cube size through accumulating all dimension and measure size roughly before building cube ) > the estimated size of percentile measure is too big > > > Key: KYLIN-3078 > URL: https://issues.apache.org/jira/browse/KYLIN-3078 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu >Priority: Critical > > To set a shard number that will be for controlling the size per shard > properly, we need to estimate cube size through accumulating all dimension > and measure size roughly before building cube. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-3078) the estimated size of percentile measure is too big
[ https://issues.apache.org/jira/browse/KYLIN-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3078: Description: To set a shard number that will be for controlling the size per shard properly, we need to estimate cube size through accumulating all dimension and measure size roughly before building cube (was: before building cube we need to estimate cube size through accumulating all dimension and measure size roughly, ) > the estimated size of percentile measure is too big > > > Key: KYLIN-3078 > URL: https://issues.apache.org/jira/browse/KYLIN-3078 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu > > To set a shard number that will be for controlling the size per shard > properly, we need to estimate cube size through accumulating all dimension > and measure size roughly before building cube -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-3078) the estimated size of percentile measure is too big
[ https://issues.apache.org/jira/browse/KYLIN-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3078: Description: before building cube we need to estimate cube size through accumulating all dimension and measure size roughly, (was: before building cube we need to estimate cube size through caculating all demension and measure size) > the estimated size of percentile measure is too big > > > Key: KYLIN-3078 > URL: https://issues.apache.org/jira/browse/KYLIN-3078 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu > > before building cube we need to estimate cube size through accumulating all > dimension and measure size roughly, -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-3078) the estimated size of percentile measure is too big
[ https://issues.apache.org/jira/browse/KYLIN-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3078: Description: before building cube we need to estimate cube size through caculating all demension and measure size (was: before building cube we need to estimate cube size through caculating a ) > the estimated size of percentile measure is too big > > > Key: KYLIN-3078 > URL: https://issues.apache.org/jira/browse/KYLIN-3078 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu > > before building cube we need to estimate cube size through caculating all > demension and measure size -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-3078) the estimated size of percentile measure is too big
[ https://issues.apache.org/jira/browse/KYLIN-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3078: Description: before building cube we need to estimate cube size through caculating a > the estimated size of percentile measure is too big > > > Key: KYLIN-3078 > URL: https://issues.apache.org/jira/browse/KYLIN-3078 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu > > before building cube we need to estimate cube size through caculating a -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-3078) the estimated size of percentile measure is too big
[ https://issues.apache.org/jira/browse/KYLIN-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3078: Summary: the estimated size of percentile measure is too big (was: the estimated size of percentile measure is over) > the estimated size of percentile measure is too big > > > Key: KYLIN-3078 > URL: https://issues.apache.org/jira/browse/KYLIN-3078 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-3078) the estimated size of percentile measure is over
[ https://issues.apache.org/jira/browse/KYLIN-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu updated KYLIN-3078: Summary: the estimated size of percentile measure is over (was: percentile measure size is estimated ) > the estimated size of percentile measure is over > - > > Key: KYLIN-3078 > URL: https://issues.apache.org/jira/browse/KYLIN-3078 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (KYLIN-3078) the estimated size of percentile measure is over
[ https://issues.apache.org/jira/browse/KYLIN-3078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Wu reassigned KYLIN-3078: --- Assignee: Yifei Wu > the estimated size of percentile measure is over > - > > Key: KYLIN-3078 > URL: https://issues.apache.org/jira/browse/KYLIN-3078 > Project: Kylin > Issue Type: Bug >Reporter: Yifei Wu >Assignee: Yifei Wu > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (KYLIN-3078) percentile measure size is estimated
Yifei Wu created KYLIN-3078: --- Summary: percentile measure size is estimated Key: KYLIN-3078 URL: https://issues.apache.org/jira/browse/KYLIN-3078 Project: Kylin Issue Type: Bug Reporter: Yifei Wu -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-3038) cannot support sum of type-converted column SQL
[
https://issues.apache.org/jira/browse/KYLIN-3038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu updated KYLIN-3038:
Description:
when run the SQL Query, which cubes contains the colomun needed , like
{code}
SELECT SUM({ fn convert("DISTANCE", SQL_DOUBLE) }) from "DEFAULT"."TABLENAME"
group by "DESTCITYNAME" LIMIT 101
{code}
but it cannot be recognized correctly
was:
when run the SQL Query, which cubes contains the colomun , like
{code}
SELECT SUM({ fn convert("DISTANCE", SQL_DOUBLE) }) from "DEFAULT"."TABLENAME"
group by "DESTCITYNAME" LIMIT 101
{code}
but it cannot be recognized correctly
> cannot support sum of type-converted column SQL
> ---
>
> Key: KYLIN-3038
> URL: https://issues.apache.org/jira/browse/KYLIN-3038
> Project: Kylin
> Issue Type: Bug
>Reporter: Yifei Wu
>
> when run the SQL Query, which cubes contains the colomun needed , like
> {code}
> SELECT SUM({ fn convert("DISTANCE", SQL_DOUBLE) }) from "DEFAULT"."TABLENAME"
> group by "DESTCITYNAME" LIMIT 101
> {code}
> but it cannot be recognized correctly
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-3038) cannot support sum of type-converted column SQL
[
https://issues.apache.org/jira/browse/KYLIN-3038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu updated KYLIN-3038:
Description:
when run the SQL Query in kylin like
{code}
SELECT SUM({ fn convert("DISTANCE", SQL_DOUBLE) }) from "DEFAULT"."TABLENAME"
group by "DESTCITYNAME" LIMIT 101
{code}
,
it cannot be recognized correctly
was:
when run the SQL Query like
{code}
SELECT SUM({ fn convert("DISTANCE", SQL_DOUBLE) }) from "DEFAULT"."TABLENAME"
group by "DESTCITYNAME" LIMIT 101
,
it cannot be worked
> cannot support sum of type-converted column SQL
> ---
>
> Key: KYLIN-3038
> URL: https://issues.apache.org/jira/browse/KYLIN-3038
> Project: Kylin
> Issue Type: Bug
>Reporter: Yifei Wu
>
> when run the SQL Query in kylin like
> {code}
> SELECT SUM({ fn convert("DISTANCE", SQL_DOUBLE) }) from "DEFAULT"."TABLENAME"
> group by "DESTCITYNAME" LIMIT 101
> {code}
> ,
> it cannot be recognized correctly
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-3038) cannot support sum of type-converted column SQL
[
https://issues.apache.org/jira/browse/KYLIN-3038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu updated KYLIN-3038:
Description:
when run the SQL Query in kylin like
{code}
SELECT SUM({ fn convert("DISTANCE", SQL_DOUBLE) }) from "DEFAULT"."TABLENAME"
group by "DESTCITYNAME" LIMIT 101
{code}
it cannot be recognized correctly
was:
when run the SQL Query in kylin like
{code}
SELECT SUM({ fn convert("DISTANCE", SQL_DOUBLE) }) from "DEFAULT"."TABLENAME"
group by "DESTCITYNAME" LIMIT 101
{code}
,
it cannot be recognized correctly
> cannot support sum of type-converted column SQL
> ---
>
> Key: KYLIN-3038
> URL: https://issues.apache.org/jira/browse/KYLIN-3038
> Project: Kylin
> Issue Type: Bug
>Reporter: Yifei Wu
>
> when run the SQL Query in kylin like
> {code}
> SELECT SUM({ fn convert("DISTANCE", SQL_DOUBLE) }) from "DEFAULT"."TABLENAME"
> group by "DESTCITYNAME" LIMIT 101
> {code}
> it cannot be recognized correctly
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-3038) cannot support sum of type-converted column SQL
[
https://issues.apache.org/jira/browse/KYLIN-3038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu updated KYLIN-3038:
Description:
when run the SQL Query like
{code}
SELECT SUM({ fn convert("DISTANCE", SQL_DOUBLE) }) from "DEFAULT"."TABLENAME"
group by "DESTCITYNAME" LIMIT 101
,
it cannot be worked
> cannot support sum of type-converted column SQL
> ---
>
> Key: KYLIN-3038
> URL: https://issues.apache.org/jira/browse/KYLIN-3038
> Project: Kylin
> Issue Type: Bug
>Reporter: Yifei Wu
>
> when run the SQL Query like
> {code}
> SELECT SUM({ fn convert("DISTANCE", SQL_DOUBLE) }) from "DEFAULT"."TABLENAME"
> group by "DESTCITYNAME" LIMIT 101
> ,
> it cannot be worked
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (KYLIN-3038) cannot support sum of type-converted column SQL
Yifei Wu created KYLIN-3038: --- Summary: cannot support sum of type-converted column SQL Key: KYLIN-3038 URL: https://issues.apache.org/jira/browse/KYLIN-3038 Project: Kylin Issue Type: Bug Reporter: Yifei Wu -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2945) global dict specific info should not use absolute path when saved it
[
https://issues.apache.org/jira/browse/KYLIN-2945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu updated KYLIN-2945:
Summary: global dict specific info should not use absolute path when saved
it (was: global dict specific info use absolute path when saved it)
> global dict specific info should not use absolute path when saved it
>
>
> Key: KYLIN-2945
> URL: https://issues.apache.org/jira/browse/KYLIN-2945
> Project: Kylin
> Issue Type: Improvement
>Reporter: Yifei Wu
>
> Currently, absolute path is used in the global dictionary for saving the
> location of specific info like the next json.
> {code}
> {
> "uuid" : "b82505d8-5b40-4009-8839-8456500ea6a8",
> "last_modified" : 0,
> "version" : "2.3.0.20500",
> "source_table" : "KYLIN_YIFEI_1011_TESTGLOBALDICTMIGRATION.KYLIN_SALES",
> "source_column" : "PRICE",
> "source_column_index" : 6,
> "data_type" : "decimal(19,4)",
> "input" : {
> "path" :
> "hdfs://sandbox.hortonworks.com:8020/kylin/kylin_default/kylin-d213a963-b5e3-4e75-a1a2-8af93ca83a80/global_dict_cube/fact_distinct_columns/KYLIN_SALES.PRICE",
> "size" : 81150,
> "last_modified_time" : 1508133611074
> },
> "dictionary_class" : "org.apache.kylin.dict.AppendTrieDictionary",
> "cardinality" : 8878
> }^@hdfs://master:8020/kylin/kylin_yifei_1011_testGlobalDictMigration/resources/GlobalDict/dict/KYLIN_YIFEI_1011_TESTGLOBALDICTMIGRATION.KYLIN_SALES/PRICE/
> {code}
> But if you change the path configuration or migrate the data to another
> cluster which node name is different or original host cannot be reached, you
> will meet problems in reusing it, like rebuilding and merge dictionary or
> transfer column from dictionary in building cube, for the dictionary info
> cannot be found.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (KYLIN-2945) global dict specific info use absolute path when saved it
[
https://issues.apache.org/jira/browse/KYLIN-2945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209083#comment-16209083
]
Yifei Wu edited comment on KYLIN-2945 at 10/18/17 11:53 AM:
the solution is changing the absolute path to relative path when creating new
dict and convert the relative path to correct absolute path by getting the
working directory from configuration when reading. At the same time, it deals
with the former absolute path when reading from the FileSystem.
was (Author: yifei_wu94):
the solution is changing the absolute path to relative path when creating new
dict, and dealing with the former absolute path when reading from the
FileSystem at the same time.
> global dict specific info use absolute path when saved it
> -
>
> Key: KYLIN-2945
> URL: https://issues.apache.org/jira/browse/KYLIN-2945
> Project: Kylin
> Issue Type: Improvement
>Reporter: Yifei Wu
>
> Currently, absolute path is used in the global dictionary for saving the
> location of specific info like the next json.
> {code}
> {
> "uuid" : "b82505d8-5b40-4009-8839-8456500ea6a8",
> "last_modified" : 0,
> "version" : "2.3.0.20500",
> "source_table" : "KYLIN_YIFEI_1011_TESTGLOBALDICTMIGRATION.KYLIN_SALES",
> "source_column" : "PRICE",
> "source_column_index" : 6,
> "data_type" : "decimal(19,4)",
> "input" : {
> "path" :
> "hdfs://sandbox.hortonworks.com:8020/kylin/kylin_default/kylin-d213a963-b5e3-4e75-a1a2-8af93ca83a80/global_dict_cube/fact_distinct_columns/KYLIN_SALES.PRICE",
> "size" : 81150,
> "last_modified_time" : 1508133611074
> },
> "dictionary_class" : "org.apache.kylin.dict.AppendTrieDictionary",
> "cardinality" : 8878
> }^@hdfs://master:8020/kylin/kylin_yifei_1011_testGlobalDictMigration/resources/GlobalDict/dict/KYLIN_YIFEI_1011_TESTGLOBALDICTMIGRATION.KYLIN_SALES/PRICE/
> {code}
> But if you change the path configuration or migrate the data to another
> cluster which node name is different or original host cannot be reached, you
> will meet problems in reusing it, like rebuilding and merge dictionary or
> transfer column from dictionary in building cube, for the dictionary info
> cannot be found.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (KYLIN-2945) global dict specific info use absolute path when saved it
[
https://issues.apache.org/jira/browse/KYLIN-2945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209083#comment-16209083
]
Yifei Wu edited comment on KYLIN-2945 at 10/18/17 11:48 AM:
the solution is changing the absolute path to relative path when creating new
dict, and dealing with the former absolute path when reading from the
FileSystem at the same time.
was (Author: yifei_wu94):
will solve it by changing the absolute path to relative path when creating new
dict and dealing with the former absolute path when reading from the FileSystem
> global dict specific info use absolute path when saved it
> -
>
> Key: KYLIN-2945
> URL: https://issues.apache.org/jira/browse/KYLIN-2945
> Project: Kylin
> Issue Type: Improvement
>Reporter: Yifei Wu
>
> Currently, absolute path is used in the global dictionary for saving the
> location of specific info like the next json.
> {code}
> {
> "uuid" : "b82505d8-5b40-4009-8839-8456500ea6a8",
> "last_modified" : 0,
> "version" : "2.3.0.20500",
> "source_table" : "KYLIN_YIFEI_1011_TESTGLOBALDICTMIGRATION.KYLIN_SALES",
> "source_column" : "PRICE",
> "source_column_index" : 6,
> "data_type" : "decimal(19,4)",
> "input" : {
> "path" :
> "hdfs://sandbox.hortonworks.com:8020/kylin/kylin_default/kylin-d213a963-b5e3-4e75-a1a2-8af93ca83a80/global_dict_cube/fact_distinct_columns/KYLIN_SALES.PRICE",
> "size" : 81150,
> "last_modified_time" : 1508133611074
> },
> "dictionary_class" : "org.apache.kylin.dict.AppendTrieDictionary",
> "cardinality" : 8878
> }^@hdfs://master:8020/kylin/kylin_yifei_1011_testGlobalDictMigration/resources/GlobalDict/dict/KYLIN_YIFEI_1011_TESTGLOBALDICTMIGRATION.KYLIN_SALES/PRICE/
> {code}
> But if you change the path configuration or migrate the data to another
> cluster which node name is different or original host cannot be reached, you
> will meet problems in reusing it, like rebuilding and merge dictionary or
> transfer column from dictionary in building cube, for the dictionary info
> cannot be found.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2945) global dict specific info use absolute path when saved it
[
https://issues.apache.org/jira/browse/KYLIN-2945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifei Wu updated KYLIN-2945:
Description:
Currently, absolute path is used in the global dictionary for saving the
location of specific info like the next json.
{code}
{
"uuid" : "b82505d8-5b40-4009-8839-8456500ea6a8",
"last_modified" : 0,
"version" : "2.3.0.20500",
"source_table" : "KYLIN_YIFEI_1011_TESTGLOBALDICTMIGRATION.KYLIN_SALES",
"source_column" : "PRICE",
"source_column_index" : 6,
"data_type" : "decimal(19,4)",
"input" : {
"path" :
"hdfs://sandbox.hortonworks.com:8020/kylin/kylin_default/kylin-d213a963-b5e3-4e75-a1a2-8af93ca83a80/global_dict_cube/fact_distinct_columns/KYLIN_SALES.PRICE",
"size" : 81150,
"last_modified_time" : 1508133611074
},
"dictionary_class" : "org.apache.kylin.dict.AppendTrieDictionary",
"cardinality" : 8878
}^@hdfs://master:8020/kylin/kylin_yifei_1011_testGlobalDictMigration/resources/GlobalDict/dict/KYLIN_YIFEI_1011_TESTGLOBALDICTMIGRATION.KYLIN_SALES/PRICE/
{code}
But if you change the path configuration or migrate the data to another cluster
which node name is different or original host cannot be reached, you will meet
problems in reusing it, like rebuilding and merge dictionary or transfer column
from dictionary in building cube, for the dictionary info cannot be found.
was:
Currently, absolute path is used in the global dictionary for saving the
location of specific info. But if you change the host configuration or migrate
the data to another data. So you may meet problems that the dictionary cannot
be found and reuse it after configuration changed or data migration.
{code}
{
"uuid" : "b82505d8-5b40-4009-8839-8456500ea6a8",
"last_modified" : 0,
"version" : "2.3.0.20500",
"source_table" : "KYLIN_YIFEI_1011_TESTGLOBALDICTMIGRATION.KYLIN_SALES",
"source_column" : "PRICE",
"source_column_index" : 6,
"data_type" : "decimal(19,4)",
"input" : {
"path" :
"hdfs://sandbox.hortonworks.com:8020/kylin/kylin_default/kylin-d213a963-b5e3-4e75-a1a2-8af93ca83a80/global_dict_cube/fact_distinct_columns/KYLIN_SALES.PRICE",
"size" : 81150,
"last_modified_time" : 1508133611074
},
"dictionary_class" : "org.apache.kylin.dict.AppendTrieDictionary",
"cardinality" : 8878
}^@hdfs://master:8020/kylin/kylin_yifei_1011_testGlobalDictMigration/resources/GlobalDict/dict/KYLIN_YIFEI_1011_TESTGLOBALDICTMIGRATION.KYLIN_SALES/PRICE/
{code}
> global dict specific info use absolute path when saved it
> -
>
> Key: KYLIN-2945
> URL: https://issues.apache.org/jira/browse/KYLIN-2945
> Project: Kylin
> Issue Type: Improvement
>Reporter: Yifei Wu
>
> Currently, absolute path is used in the global dictionary for saving the
> location of specific info like the next json.
> {code}
> {
> "uuid" : "b82505d8-5b40-4009-8839-8456500ea6a8",
> "last_modified" : 0,
> "version" : "2.3.0.20500",
> "source_table" : "KYLIN_YIFEI_1011_TESTGLOBALDICTMIGRATION.KYLIN_SALES",
> "source_column" : "PRICE",
> "source_column_index" : 6,
> "data_type" : "decimal(19,4)",
> "input" : {
> "path" :
> "hdfs://sandbox.hortonworks.com:8020/kylin/kylin_default/kylin-d213a963-b5e3-4e75-a1a2-8af93ca83a80/global_dict_cube/fact_distinct_columns/KYLIN_SALES.PRICE",
> "size" : 81150,
> "last_modified_time" : 1508133611074
> },
> "dictionary_class" : "org.apache.kylin.dict.AppendTrieDictionary",
> "cardinality" : 8878
> }^@hdfs://master:8020/kylin/kylin_yifei_1011_testGlobalDictMigration/resources/GlobalDict/dict/KYLIN_YIFEI_1011_TESTGLOBALDICTMIGRATION.KYLIN_SALES/PRICE/
> {code}
> But if you change the path configuration or migrate the data to another
> cluster which node name is different or original host cannot be reached, you
> will meet problems in reusing it, like rebuilding and merge dictionary or
> transfer column from dictionary in building cube, for the dictionary info
> cannot be found.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
