[jira] [Closed] (CALCITE-3511) Failed to serialize SqlNode to json string using jackson
[ https://issues.apache.org/jira/browse/CALCITE-3511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] groobyming closed CALCITE-3511. --- Fix Version/s: 1.21.0 Resolution: Not A Problem > Failed to serialize SqlNode to json string using jackson > > > Key: CALCITE-3511 > URL: https://issues.apache.org/jira/browse/CALCITE-3511 > Project: Calcite > Issue Type: New Feature > Components: core >Affects Versions: 1.21.0 >Reporter: groobyming >Priority: Major > Labels: SqlNode, calcite, calcite-core, jackson, json > Fix For: 1.21.0 > > Attachments: screenshot-1.png > > > Failed to serialize SqlNode to json string using jackson, Exception as > follows： > Caused by: java.lang.UnsupportedOperationException: class > org.apache.calcite.sql.SqlSelectOperator: SELECT > at org.apache.calcite.util.Util.needToImplement(Util.java:967) > at > org.apache.calcite.sql.SqlOperator.getOperandCountRange(SqlOperator.java:201) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:664) > at > com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:689) > ... 32 more -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-3534) Support parse(unparse) alien system non-standard data type
[ https://issues.apache.org/jira/browse/CALCITE-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved CALCITE-3534. - Resolution: Fixed Fixed in [07c1efb|https://github.com/apache/calcite/commit/07c1efbb2b42a0e27f9d01968d357d9c1e17cd51] ! > Support parse(unparse) alien system non-standard data type > -- > > Key: CALCITE-3534 > URL: https://issues.apache.org/jira/browse/CALCITE-3534 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.21.0 >Reporter: Danny Chen >Assignee: Danny Chen >Priority: Major > Labels: pull-request-available > Fix For: 1.22.0 > > Time Spent: 20m > Remaining Estimate: 0h > > For some systems, they have customized SQL type that does not belong to > standard SQL, for example, UNSIGNED is a built-in type in MySQL which is > synonym of INTEGER. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3535) EnumerableJoinRule: remove unnecessary Filter on top of INNER Join
Ruben Q L created CALCITE-3535: -- Summary: EnumerableJoinRule: remove unnecessary Filter on top of INNER Join Key: CALCITE-3535 URL: https://issues.apache.org/jira/browse/CALCITE-3535 Project: Calcite Issue Type: Task Components: core Affects Versions: 1.21.0 Reporter: Ruben Q L Assignee: Ruben Q L Fix For: 1.22.0 With the implementation of CALCITE-2973, now EnumerableHashJoin supports all type of conditions (not just equi joins). However, there is still one TODO in EnumerableHashJoinRule that, in case of an INNER Join, creates a Filter with the non-equi conditions on top of the EnumerableHashJoin, this filter is not really needed, since now EnumerableHashJoinRule can support the full conditions, including the non-equi items. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-3535) EnumerableJoinRule: remove unnecessary Filter on top of INNER Join
[ https://issues.apache.org/jira/browse/CALCITE-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruben Q L updated CALCITE-3535: --- Description: With the implementation of CALCITE-2973, now EnumerableHashJoin supports all type of conditions (not just equi joins). However, there is still one TODO in EnumerableHashJoinRule that, in case of an INNER Join, creates a Filter with the non-equi conditions on top of the EnumerableHashJoin (created only with the equi-conditions), this filter is not really needed, since now EnumerableHashJoinRule can support the full condition, including the equi and non-equi items. was: With the implementation of CALCITE-2973, now EnumerableHashJoin supports all type of conditions (not just equi joins). However, there is still one TODO in EnumerableHashJoinRule that, in case of an INNER Join, creates a Filter with the non-equi conditions on top of the EnumerableHashJoin, this filter is not really needed, since now EnumerableHashJoinRule can support the full conditions, including the non-equi items. > EnumerableJoinRule: remove unnecessary Filter on top of INNER Join > -- > > Key: CALCITE-3535 > URL: https://issues.apache.org/jira/browse/CALCITE-3535 > Project: Calcite > Issue Type: Task > Components: core >Affects Versions: 1.21.0 >Reporter: Ruben Q L >Assignee: Ruben Q L >Priority: Minor > Fix For: 1.22.0 > > > With the implementation of CALCITE-2973, now EnumerableHashJoin supports all > type of conditions (not just equi joins). However, there is still one TODO in > EnumerableHashJoinRule that, in case of an INNER Join, creates a Filter with > the non-equi conditions on top of the EnumerableHashJoin (created only with > the equi-conditions), this filter is not really needed, since now > EnumerableHashJoinRule can support the full condition, including the equi and > non-equi items. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-3535) EnumerableJoinRule: remove unnecessary Filter on top of INNER Join
[ https://issues.apache.org/jira/browse/CALCITE-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruben Q L updated CALCITE-3535: --- Description: With the implementation of CALCITE-2973, now EnumerableHashJoin supports all type of conditions (not just equi joins). However, there is still one TODO in EnumerableHashJoinRule that, in case of an INNER Join, creates a Filter with the non-equi conditions on top of the EnumerableHashJoin (created only with the equi-conditions), this filter is not really needed, since now EnumerableHashJoinRule can support the full condition: equi and non-equi items. was: With the implementation of CALCITE-2973, now EnumerableHashJoin supports all type of conditions (not just equi joins). However, there is still one TODO in EnumerableHashJoinRule that, in case of an INNER Join, creates a Filter with the non-equi conditions on top of the EnumerableHashJoin (created only with the equi-conditions), this filter is not really needed, since now EnumerableHashJoinRule can support the full condition, including the equi and non-equi items. > EnumerableJoinRule: remove unnecessary Filter on top of INNER Join > -- > > Key: CALCITE-3535 > URL: https://issues.apache.org/jira/browse/CALCITE-3535 > Project: Calcite > Issue Type: Task > Components: core >Affects Versions: 1.21.0 >Reporter: Ruben Q L >Assignee: Ruben Q L >Priority: Minor > Fix For: 1.22.0 > > > With the implementation of CALCITE-2973, now EnumerableHashJoin supports all > type of conditions (not just equi joins). However, there is still one TODO in > EnumerableHashJoinRule that, in case of an INNER Join, creates a Filter with > the non-equi conditions on top of the EnumerableHashJoin (created only with > the equi-conditions), this filter is not really needed, since now > EnumerableHashJoinRule can support the full condition: equi and non-equi > items. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3531) AggregateProjectPullUpConstantsRule should not remove deterministic function group key if the function is dynamic
[
https://issues.apache.org/jira/browse/CALCITE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16981814#comment-16981814
]
Rui Wang commented on CALCITE-3531:
---
Thanks Danny for your explanation! Also thanks for links of relevant
discussion, I found those were useful for my self-education.
> AggregateProjectPullUpConstantsRule should not remove deterministic function
> group key if the function is dynamic
> -
>
> Key: CALCITE-3531
> URL: https://issues.apache.org/jira/browse/CALCITE-3531
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.21.0
>Reporter: Danny Chen
>Assignee: Danny Chen
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Now AggregateProjectPullUpConstantsRule simplify the query:
> {code:sql}
> select hiredate
> from sales.emp
> where sal is null and hiredate = current_timestamp
> group by sal, hiredate
> having count(*) > 3
> {code}
> from plan:
> {code:xml}
> LogicalProject(HIREDATE=[$1])
> LogicalFilter(condition=[>($2, 3)])
> LogicalAggregate(group=[{0, 1}], agg#0=[COUNT()])
> LogicalProject(SAL=[$5], HIREDATE=[$4])
> LogicalFilter(condition=[AND(IS NULL($5), =($4, CURRENT_TIMESTAMP))])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> to plan:
> {code:xml}
> LogicalProject(HIREDATE=[$1])
> LogicalFilter(condition=[>($2, 3)])
> LogicalProject(SAL=[$0], HIREDATE=[CURRENT_TIMESTAMP], $f2=[$1])
> LogicalAggregate(group=[{0}], agg#0=[COUNT()])
> LogicalProject(SAL=[$5], HIREDATE=[$4])
> LogicalFilter(condition=[AND(IS NULL($5), =($4,
> CURRENT_TIMESTAMP))])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> which is unsafe, because for stream sql, we need to group data by dateTime,
> also the result is wrong if a batch job runs across days.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (CALCITE-3343) Improve LogicalTableFunctionScan to match table function windowing
[
https://issues.apache.org/jira/browse/CALCITE-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang closed CALCITE-3343.
-
Resolution: Not A Problem
> Improve LogicalTableFunctionScan to match table function windowing
> --
>
> Key: CALCITE-3343
> URL: https://issues.apache.org/jira/browse/CALCITE-3343
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/logical/LogicalTableFunctionScan.java#L40
> {code:java}
> /**
>* Creates a LogicalTableFunctionScan.
>*
>* @param clusterCluster that this relational expression belongs to
>* @param inputs 0 or more relational inputs
>* @param traitSet Trait set
>* @param rexCallFunction invocation expression
>* @param elementTypeElement type of the collection that will implement
>* this table
>* @param rowTypeRow type produced by function
>* @param columnMappings Column mappings associated with this function
>*/
> public LogicalTableFunctionScan(
> RelOptCluster cluster,
> RelTraitSet traitSet,
> List inputs,
> RexNode rexCall,
> Type elementType,
>RelDataType rowType,
> Set columnMappings) {
> super(cluster, traitSet, inputs, rexCall, elementType, rowType,
> columnMappings);
> }
> {code}
> There might be a gap to make LogicalTableFunctionScan work for table function
> windowing:
> 1. we need a RexInputRef parameter (converted from the DESCRIPTOR) to mark
> which column is the watermarked column.
> 2. we need a List of RexNode(RexLiteral) for interval parameters.
> The most straightforward approach while maintaining backward compatible,
> seems is add another constructor with an additional List parameter.
> We could save RexInputRef and intervals (RexLiteral) to this parameter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3343) Improve LogicalTableFunctionScan to match table function windowing
[
https://issues.apache.org/jira/browse/CALCITE-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16981993#comment-16981993
]
Rui Wang commented on CALCITE-3343:
---
It turns out reusing existing rexCall is good enough. Closing this JIRA.
> Improve LogicalTableFunctionScan to match table function windowing
> --
>
> Key: CALCITE-3343
> URL: https://issues.apache.org/jira/browse/CALCITE-3343
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/logical/LogicalTableFunctionScan.java#L40
> {code:java}
> /**
>* Creates a LogicalTableFunctionScan.
>*
>* @param clusterCluster that this relational expression belongs to
>* @param inputs 0 or more relational inputs
>* @param traitSet Trait set
>* @param rexCallFunction invocation expression
>* @param elementTypeElement type of the collection that will implement
>* this table
>* @param rowTypeRow type produced by function
>* @param columnMappings Column mappings associated with this function
>*/
> public LogicalTableFunctionScan(
> RelOptCluster cluster,
> RelTraitSet traitSet,
> List inputs,
> RexNode rexCall,
> Type elementType,
>RelDataType rowType,
> Set columnMappings) {
> super(cluster, traitSet, inputs, rexCall, elementType, rowType,
> columnMappings);
> }
> {code}
> There might be a gap to make LogicalTableFunctionScan work for table function
> windowing:
> 1. we need a RexInputRef parameter (converted from the DESCRIPTOR) to mark
> which column is the watermarked column.
> 2. we need a List of RexNode(RexLiteral) for interval parameters.
> The most straightforward approach while maintaining backward compatible,
> seems is add another constructor with an additional List parameter.
> We could save RexInputRef and intervals (RexLiteral) to this parameter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3531) AggregateProjectPullUpConstantsRule should not remove deterministic function group key if the function is dynamic
[
https://issues.apache.org/jira/browse/CALCITE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16982045#comment-16982045
]
Julian Hyde commented on CALCITE-3531:
--
I think we got this wrong.
First of all, CURRENT_TIMESTAMP should return the same result for the duration
of the query. That is what the SQL standard says. It should do this even for
streaming queries.
Second, if the expression evaluates to the same thing for the duration of the
query, then it is safe to remove it from the GROUP BY clause.
> AggregateProjectPullUpConstantsRule should not remove deterministic function
> group key if the function is dynamic
> -
>
> Key: CALCITE-3531
> URL: https://issues.apache.org/jira/browse/CALCITE-3531
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.21.0
>Reporter: Danny Chen
>Assignee: Danny Chen
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Now AggregateProjectPullUpConstantsRule simplify the query:
> {code:sql}
> select hiredate
> from sales.emp
> where sal is null and hiredate = current_timestamp
> group by sal, hiredate
> having count(*) > 3
> {code}
> from plan:
> {code:xml}
> LogicalProject(HIREDATE=[$1])
> LogicalFilter(condition=[>($2, 3)])
> LogicalAggregate(group=[{0, 1}], agg#0=[COUNT()])
> LogicalProject(SAL=[$5], HIREDATE=[$4])
> LogicalFilter(condition=[AND(IS NULL($5), =($4, CURRENT_TIMESTAMP))])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> to plan:
> {code:xml}
> LogicalProject(HIREDATE=[$1])
> LogicalFilter(condition=[>($2, 3)])
> LogicalProject(SAL=[$0], HIREDATE=[CURRENT_TIMESTAMP], $f2=[$1])
> LogicalAggregate(group=[{0}], agg#0=[COUNT()])
> LogicalProject(SAL=[$5], HIREDATE=[$4])
> LogicalFilter(condition=[AND(IS NULL($5), =($4,
> CURRENT_TIMESTAMP))])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> which is unsafe, because for stream sql, we need to group data by dateTime,
> also the result is wrong if a batch job runs across days.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-3531) AggregateProjectPullUpConstantsRule should not remove deterministic function group key if the function is dynamic
[
https://issues.apache.org/jira/browse/CALCITE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16982045#comment-16982045
]
Julian Hyde edited comment on CALCITE-3531 at 11/26/19 1:45 AM:
I think we got this wrong.
First of all, CURRENT_TIMESTAMP should return the same result for the duration
of the query. That is what the SQL standard says. It should do this even for
streaming queries.
Second, if the expression evaluates to the same thing for the duration of the
query, then it is safe to remove it from the GROUP BY clause.
If you need a "wallclock time" for streaming/continuous queries, invent a new
function, but don't redefine CURRENT_TIMESTAMP.
was (Author: julianhyde):
I think we got this wrong.
First of all, CURRENT_TIMESTAMP should return the same result for the duration
of the query. That is what the SQL standard says. It should do this even for
streaming queries.
Second, if the expression evaluates to the same thing for the duration of the
query, then it is safe to remove it from the GROUP BY clause.
> AggregateProjectPullUpConstantsRule should not remove deterministic function
> group key if the function is dynamic
> -
>
> Key: CALCITE-3531
> URL: https://issues.apache.org/jira/browse/CALCITE-3531
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.21.0
>Reporter: Danny Chen
>Assignee: Danny Chen
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Now AggregateProjectPullUpConstantsRule simplify the query:
> {code:sql}
> select hiredate
> from sales.emp
> where sal is null and hiredate = current_timestamp
> group by sal, hiredate
> having count(*) > 3
> {code}
> from plan:
> {code:xml}
> LogicalProject(HIREDATE=[$1])
> LogicalFilter(condition=[>($2, 3)])
> LogicalAggregate(group=[{0, 1}], agg#0=[COUNT()])
> LogicalProject(SAL=[$5], HIREDATE=[$4])
> LogicalFilter(condition=[AND(IS NULL($5), =($4, CURRENT_TIMESTAMP))])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> to plan:
> {code:xml}
> LogicalProject(HIREDATE=[$1])
> LogicalFilter(condition=[>($2, 3)])
> LogicalProject(SAL=[$0], HIREDATE=[CURRENT_TIMESTAMP], $f2=[$1])
> LogicalAggregate(group=[{0}], agg#0=[COUNT()])
> LogicalProject(SAL=[$5], HIREDATE=[$4])
> LogicalFilter(condition=[AND(IS NULL($5), =($4,
> CURRENT_TIMESTAMP))])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> which is unsafe, because for stream sql, we need to group data by dateTime,
> also the result is wrong if a batch job runs across days.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3496) Hive dialect support with cube and with rollup
[ https://issues.apache.org/jira/browse/CALCITE-3496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16982066#comment-16982066 ] Julian Hyde commented on CALCITE-3496: -- Hive dialect support where? In the JDBC adapter or in the SQL parser? Change the JIRA description and the commit message to fully describe the change, even to someone who does not read the code. > Hive dialect support with cube and with rollup > --- > > Key: CALCITE-3496 > URL: https://issues.apache.org/jira/browse/CALCITE-3496 > Project: Calcite > Issue Type: Wish >Reporter: xzh_dz >Priority: Minor > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > Hive dialect support with cube and with rollup -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3473) Getting unique result for table scan should contain key column(s)
[
https://issues.apache.org/jira/browse/CALCITE-3473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16982076#comment-16982076
]
Julian Hyde commented on CALCITE-3473:
--
I see that the new commit adds {{RelOptTable.getKeys()}}. What should happen if
the table overrides both {{getKeys()}} and {{isKey(ImmutableBitSet)}}? Is the
table responsible for making sure that they are consistent?
> Getting unique result for table scan should contain key column(s)
> -
>
> Key: CALCITE-3473
> URL: https://issues.apache.org/jira/browse/CALCITE-3473
> Project: Calcite
> Issue Type: Improvement
>Reporter: Wang Yanlin
>Assignee: Wang Yanlin
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Currently, *mq.getUniqueKeys()* for *TableScan* returns empty.
> However, for table with key column(s) defined, we can refer uniqueness on key
> column(s), so the result should contain key column(s)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3534) Support parse(unparse) alien system non-standard data type
[ https://issues.apache.org/jira/browse/CALCITE-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16982079#comment-16982079 ] Julian Hyde commented on CALCITE-3534: -- Please revert. You didn't give enough time to review. > Support parse(unparse) alien system non-standard data type > -- > > Key: CALCITE-3534 > URL: https://issues.apache.org/jira/browse/CALCITE-3534 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.21.0 >Reporter: Danny Chen >Assignee: Danny Chen >Priority: Major > Labels: pull-request-available > Fix For: 1.22.0 > > Time Spent: 20m > Remaining Estimate: 0h > > For some systems, they have customized SQL type that does not belong to > standard SQL, for example, UNSIGNED is a built-in type in MySQL which is > synonym of INTEGER. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3536) Wrong semantics in CoalesceImplementor
Jin Xing created CALCITE-3536: - Summary: Wrong semantics in CoalesceImplementor Key: CALCITE-3536 URL: https://issues.apache.org/jira/browse/CALCITE-3536 Project: Calcite Issue Type: Bug Reporter: Jin Xing -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3536) Wrong semantics in CoalesceImplementor
[ https://issues.apache.org/jira/browse/CALCITE-3536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16982088#comment-16982088 ] Julian Hyde commented on CALCITE-3536: -- What are the wrong semantics? Test case? > Wrong semantics in CoalesceImplementor > -- > > Key: CALCITE-3536 > URL: https://issues.apache.org/jira/browse/CALCITE-3536 > Project: Calcite > Issue Type: Bug >Reporter: Jin Xing >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3531) AggregateProjectPullUpConstantsRule should not remove deterministic function group key if the function is dynamic
[
https://issues.apache.org/jira/browse/CALCITE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16982094#comment-16982094
]
Danny Chen commented on CALCITE-3531:
-
Thanks, Julian, i didn't find any evidence that CURRENT_TIMESTAMP should return
the same result for the duration of the query, can you give a reference ? Many
thanks for that.
> AggregateProjectPullUpConstantsRule should not remove deterministic function
> group key if the function is dynamic
> -
>
> Key: CALCITE-3531
> URL: https://issues.apache.org/jira/browse/CALCITE-3531
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.21.0
>Reporter: Danny Chen
>Assignee: Danny Chen
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Now AggregateProjectPullUpConstantsRule simplify the query:
> {code:sql}
> select hiredate
> from sales.emp
> where sal is null and hiredate = current_timestamp
> group by sal, hiredate
> having count(*) > 3
> {code}
> from plan:
> {code:xml}
> LogicalProject(HIREDATE=[$1])
> LogicalFilter(condition=[>($2, 3)])
> LogicalAggregate(group=[{0, 1}], agg#0=[COUNT()])
> LogicalProject(SAL=[$5], HIREDATE=[$4])
> LogicalFilter(condition=[AND(IS NULL($5), =($4, CURRENT_TIMESTAMP))])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> to plan:
> {code:xml}
> LogicalProject(HIREDATE=[$1])
> LogicalFilter(condition=[>($2, 3)])
> LogicalProject(SAL=[$0], HIREDATE=[CURRENT_TIMESTAMP], $f2=[$1])
> LogicalAggregate(group=[{0}], agg#0=[COUNT()])
> LogicalProject(SAL=[$5], HIREDATE=[$4])
> LogicalFilter(condition=[AND(IS NULL($5), =($4,
> CURRENT_TIMESTAMP))])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> which is unsafe, because for stream sql, we need to group data by dateTime,
> also the result is wrong if a batch job runs across days.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3536) Wrong semantics in CoalesceImplementor
[
https://issues.apache.org/jira/browse/CALCITE-3536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jin Xing updated CALCITE-3536:
--
Description:
Currently, COALESCE expression specified in sql string will be rewrite to CASE
by SqlValidator [1], thus CoalesceImplementor is almost no use. But it is
actually used somewhere, for example Sum0Splitter. When running AggregateRemove
and split expression SUM0, a COALESCE is actually created and no chance to be
rewrite to CASE anymore.
In current code, CoalesceImplementor doesn't handle NullAs properly by
semantics.
If we disable the rewrite from COALESCE to CASE in Validator and run below sql
in JDBC Test:
{code:java}
select grocery_sqft from store where coalesce(grocery_sqft, 0) >= 2{code}
We will get below exception:
{code:java}
Caused by: java.lang.RuntimeException: Error while compiling generated Java
code:
public org.apache.calcite.linq4j.Enumerable bind(final
org.apache.calcite.DataContext root) {
final org.apache.calcite.linq4j.Enumerable _inputEnumerable =
org.apache.calcite.schema.Schemas.queryable(root,
root.getRootSchema().getSubSchema("foodmart2"), java.lang.Object[].class,
"store").asEnumerable();
return new org.apache.calcite.linq4j.AbstractEnumerable(){
public org.apache.calcite.linq4j.Enumerator enumerator() {
return new org.apache.calcite.linq4j.Enumerator(){
public final org.apache.calcite.linq4j.Enumerator inputEnumerator =
_inputEnumerable.enumerator();
public void reset() {
inputEnumerator.reset();
}public boolean moveNext() {
while (inputEnumerator.moveNext()) {
final Integer inp16_ = (Integer) ((Object[])
inputEnumerator.current())[16];
if ((inp16_ ? inp16_.intValue() : 0) >= 2) {
return true;
}
}
return false;
}public void close() {
inputEnumerator.close();
}public Object current() {
return (Integer) ((Object[]) inputEnumerator.current())[16];
} };
}};
}
at org.apache.calcite.avatica.Helper.wrap(Helper.java:37)
at org.apache.calcite.avatica.Helper.wrap(Helper.java:37)
at
org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:128)
at
org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:332)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:638)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:502)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:472)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:231)
at
org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550)
at
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
... 29 more
Caused by: org.codehaus.commons.compiler.CompileException: Line 14, Column 28:
Not a boolean expression
at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211)
at org.codehaus.janino.UnitCompiler.compileBoolean2(UnitCompiler.java:3994)
at org.codehaus.janino.UnitCompiler.access$6300(UnitCompiler.java:215){code}
[1]
[https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L1210]
> Wrong semantics in CoalesceImplementor
> --
>
> Key: CALCITE-3536
> URL: https://issues.apache.org/jira/browse/CALCITE-3536
> Project: Calcite
> Issue Type: Bug
>Reporter: Jin Xing
>Priority: Major
>
> Currently, COALESCE expression specified in sql string will be rewrite to
> CASE by SqlValidator [1], thus CoalesceImplementor is almost no use. But it
> is actually used somewhere, for example Sum0Splitter. When running
> AggregateRemove and split expression SUM0, a COALESCE is actually created and
> no chance to be rewrite to CASE anymore.
> In current code, CoalesceImplementor doesn't handle NullAs properly by
> semantics.
> If we disable the rewrite from COALESCE to CASE in Validator and run below
> sql in JDBC Test:
>
> {code:java}
> select grocery_sqft from store where coalesce(grocery_sqft, 0) >= 2{code}
>
> We will get below exception:
>
> {code:java}
> Caused by: java.lang.RuntimeException: Error while compiling generated Java
> code:
> public org.apache.calcite.linq4j.Enumerab
[jira] [Updated] (CALCITE-3536) Wrong semantics in CoalesceImplementor
[
https://issues.apache.org/jira/browse/CALCITE-3536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jin Xing updated CALCITE-3536:
--
Description:
Currently, COALESCE expression specified in sql string will be rewrite to CASE
by SqlValidator [1], thus CoalesceImplementor is almost no use. But it is
actually used somewhere, for example Sum0Splitter. When running AggregateRemove
and split expression SUM0, a COALESCE is actually created and no chance to be
rewrite to CASE anymore.
In current code, CoalesceImplementor doesn't handle NullAs properly by
semantics.
If we disable the rewrite from COALESCE to CASE in Validator and run below sql
in JDBC Test:
{code:java}
select grocery_sqft from store where coalesce(grocery_sqft, 0) >= 2{code}
We will get below exception:
{code:java}
Caused by: java.lang.RuntimeException: Error while compiling generated Java
code:
public org.apache.calcite.linq4j.Enumerable bind(final
org.apache.calcite.DataContext root) {
final org.apache.calcite.linq4j.Enumerable _inputEnumerable =
org.apache.calcite.schema.Schemas.queryable(root,
root.getRootSchema().getSubSchema("foodmart2"), java.lang.Object[].class,
"store").asEnumerable();
return new org.apache.calcite.linq4j.AbstractEnumerable(){
public org.apache.calcite.linq4j.Enumerator enumerator() {
return new org.apache.calcite.linq4j.Enumerator(){
public final org.apache.calcite.linq4j.Enumerator inputEnumerator =
_inputEnumerable.enumerator();
public void reset() {
inputEnumerator.reset();
}public boolean moveNext() {
while (inputEnumerator.moveNext()) {
final Integer inp16_ = (Integer) ((Object[])
inputEnumerator.current())[16];
if ((inp16_ ? inp16_.intValue() : 0) >= 2) {
return true;
}
}
return false;
}public void close() {
inputEnumerator.close();
}public Object current() {
return (Integer) ((Object[]) inputEnumerator.current())[16];
} };
}};
}
at org.apache.calcite.avatica.Helper.wrap(Helper.java:37)
at org.apache.calcite.avatica.Helper.wrap(Helper.java:37)
at
org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:128)
at
org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:332)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:638)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:502)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:472)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:231)
at
org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550)
at
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
... 29 more
Caused by: org.codehaus.commons.compiler.CompileException: Line 14, Column 28:
Not a boolean expression
at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211)
at org.codehaus.janino.UnitCompiler.compileBoolean2(UnitCompiler.java:3994)
at org.codehaus.janino.UnitCompiler.access$6300(UnitCompiler.java:215){code}
[1]
[https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L1210]
was:
Currently, COALESCE expression specified in sql string will be rewrite to CASE
by SqlValidator [1], thus CoalesceImplementor is almost no use. But it is
actually used somewhere, for example Sum0Splitter. When running AggregateRemove
and split expression SUM0, a COALESCE is actually created and no chance to be
rewrite to CASE anymore.
In current code, CoalesceImplementor doesn't handle NullAs properly by
semantics.
If we disable the rewrite from COALESCE to CASE in Validator and run below sql
in JDBC Test:
{code:java}
select grocery_sqft from store where coalesce(grocery_sqft, 0) >= 2{code}
We will get below exception:
{code:java}
Caused by: java.lang.RuntimeException: Error while compiling generated Java
code:
public org.apache.calcite.linq4j.Enumerable bind(final
org.apache.calcite.DataContext root) {
final org.apache.calcite.linq4j.Enumerable _inputEnumerable =
org.apache.calcite.schema.Schemas.queryable(root,
root.getRootSchema().getSubSchema("foodmart2"), java.lang.Object[].class,
"store").asEnumerable();
return new org.apache.calcite.linq4j.AbstractEnumerable(){
public org.apache.cal
[jira] [Reopened] (CALCITE-3531) AggregateProjectPullUpConstantsRule should not remove deterministic function group key if the function is dynamic
[
https://issues.apache.org/jira/browse/CALCITE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen reopened CALCITE-3531:
-
> AggregateProjectPullUpConstantsRule should not remove deterministic function
> group key if the function is dynamic
> -
>
> Key: CALCITE-3531
> URL: https://issues.apache.org/jira/browse/CALCITE-3531
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.21.0
>Reporter: Danny Chen
>Assignee: Danny Chen
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Now AggregateProjectPullUpConstantsRule simplify the query:
> {code:sql}
> select hiredate
> from sales.emp
> where sal is null and hiredate = current_timestamp
> group by sal, hiredate
> having count(*) > 3
> {code}
> from plan:
> {code:xml}
> LogicalProject(HIREDATE=[$1])
> LogicalFilter(condition=[>($2, 3)])
> LogicalAggregate(group=[{0, 1}], agg#0=[COUNT()])
> LogicalProject(SAL=[$5], HIREDATE=[$4])
> LogicalFilter(condition=[AND(IS NULL($5), =($4, CURRENT_TIMESTAMP))])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> to plan:
> {code:xml}
> LogicalProject(HIREDATE=[$1])
> LogicalFilter(condition=[>($2, 3)])
> LogicalProject(SAL=[$0], HIREDATE=[CURRENT_TIMESTAMP], $f2=[$1])
> LogicalAggregate(group=[{0}], agg#0=[COUNT()])
> LogicalProject(SAL=[$5], HIREDATE=[$4])
> LogicalFilter(condition=[AND(IS NULL($5), =($4,
> CURRENT_TIMESTAMP))])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> which is unsafe, because for stream sql, we need to group data by dateTime,
> also the result is wrong if a batch job runs across days.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3531) AggregateProjectPullUpConstantsRule should not remove deterministic function group key if the function is dynamic
[
https://issues.apache.org/jira/browse/CALCITE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16982099#comment-16982099
]
Danny Chen commented on CALCITE-3531:
-
I found this one in SQL standard 2011:
bq. Let S be an that is not generally contained in a
. All
bq. s that are contained in s that
are generally contained, without
bq. an intervening whose subject routines do not include
an SQL function, either in S
bq. without an intervening or in an contained in the
bq. of a trigger activated as a consequence of executing S,
are effectively evaluated simultaneously.
bq. The time of evaluation of a during the execution
of S and its activated
bq. triggers is implementation-dependent.
But i'm not sure what it really means.
> AggregateProjectPullUpConstantsRule should not remove deterministic function
> group key if the function is dynamic
> -
>
> Key: CALCITE-3531
> URL: https://issues.apache.org/jira/browse/CALCITE-3531
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.21.0
>Reporter: Danny Chen
>Assignee: Danny Chen
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Now AggregateProjectPullUpConstantsRule simplify the query:
> {code:sql}
> select hiredate
> from sales.emp
> where sal is null and hiredate = current_timestamp
> group by sal, hiredate
> having count(*) > 3
> {code}
> from plan:
> {code:xml}
> LogicalProject(HIREDATE=[$1])
> LogicalFilter(condition=[>($2, 3)])
> LogicalAggregate(group=[{0, 1}], agg#0=[COUNT()])
> LogicalProject(SAL=[$5], HIREDATE=[$4])
> LogicalFilter(condition=[AND(IS NULL($5), =($4, CURRENT_TIMESTAMP))])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> to plan:
> {code:xml}
> LogicalProject(HIREDATE=[$1])
> LogicalFilter(condition=[>($2, 3)])
> LogicalProject(SAL=[$0], HIREDATE=[CURRENT_TIMESTAMP], $f2=[$1])
> LogicalAggregate(group=[{0}], agg#0=[COUNT()])
> LogicalProject(SAL=[$5], HIREDATE=[$4])
> LogicalFilter(condition=[AND(IS NULL($5), =($4,
> CURRENT_TIMESTAMP))])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> which is unsafe, because for stream sql, we need to group data by dateTime,
> also the result is wrong if a batch job runs across days.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-3531) AggregateProjectPullUpConstantsRule should not remove deterministic function group key if the function is dynamic
[
https://issues.apache.org/jira/browse/CALCITE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16982099#comment-16982099
]
Danny Chen edited comment on CALCITE-3531 at 11/26/19 3:16 AM:
---
I found this one in SQL standard 2011:
bq. Let S be an that is not generally contained in a
. All s that are contained in s that are generally contained, without an intervening whose subject routines do not include an SQL function, either in S
without an intervening or in an contained in the of a trigger activated as a
consequence of executing S, are effectively evaluated simultaneously. The time
of evaluation of a during the execution of S and its
activated triggers is implementation-dependent.
But i'm not sure what it really means.
was (Author: danny0405):
I found this one in SQL standard 2011:
bq. Let S be an that is not generally contained in a
. All
bq. s that are contained in s that
are generally contained, without
bq. an intervening whose subject routines do not include
an SQL function, either in S
bq. without an intervening or in an contained in the
bq. of a trigger activated as a consequence of executing S,
are effectively evaluated simultaneously.
bq. The time of evaluation of a during the execution
of S and its activated
bq. triggers is implementation-dependent.
But i'm not sure what it really means.
> AggregateProjectPullUpConstantsRule should not remove deterministic function
> group key if the function is dynamic
> -
>
> Key: CALCITE-3531
> URL: https://issues.apache.org/jira/browse/CALCITE-3531
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.21.0
>Reporter: Danny Chen
>Assignee: Danny Chen
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Now AggregateProjectPullUpConstantsRule simplify the query:
> {code:sql}
> select hiredate
> from sales.emp
> where sal is null and hiredate = current_timestamp
> group by sal, hiredate
> having count(*) > 3
> {code}
> from plan:
> {code:xml}
> LogicalProject(HIREDATE=[$1])
> LogicalFilter(condition=[>($2, 3)])
> LogicalAggregate(group=[{0, 1}], agg#0=[COUNT()])
> LogicalProject(SAL=[$5], HIREDATE=[$4])
> LogicalFilter(condition=[AND(IS NULL($5), =($4, CURRENT_TIMESTAMP))])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> to plan:
> {code:xml}
> LogicalProject(HIREDATE=[$1])
> LogicalFilter(condition=[>($2, 3)])
> LogicalProject(SAL=[$0], HIREDATE=[CURRENT_TIMESTAMP], $f2=[$1])
> LogicalAggregate(group=[{0}], agg#0=[COUNT()])
> LogicalProject(SAL=[$5], HIREDATE=[$4])
> LogicalFilter(condition=[AND(IS NULL($5), =($4,
> CURRENT_TIMESTAMP))])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> which is unsafe, because for stream sql, we need to group data by dateTime,
> also the result is wrong if a batch job runs across days.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3536) Wrong semantics in CoalesceImplementor
[
https://issues.apache.org/jira/browse/CALCITE-3536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-3536:
Labels: pull-request-available (was: )
> Wrong semantics in CoalesceImplementor
> --
>
> Key: CALCITE-3536
> URL: https://issues.apache.org/jira/browse/CALCITE-3536
> Project: Calcite
> Issue Type: Bug
>Reporter: Jin Xing
>Priority: Major
> Labels: pull-request-available
>
> Currently, COALESCE expression specified in sql string will be rewrite to
> CASE by SqlValidator [1], thus CoalesceImplementor is almost no use. But it
> is actually used somewhere, for example Sum0Splitter. When running
> AggregateRemove and split expression SUM0, a COALESCE is actually created and
> no chance to be rewrite to CASE anymore.
> In current code, CoalesceImplementor doesn't handle NullAs properly by
> semantics.
> If we disable the rewrite from COALESCE to CASE in Validator and run below
> sql in JDBC Test:
> {code:java}
> select grocery_sqft from store where coalesce(grocery_sqft, 0) >= 2{code}
> We will get below exception:
> {code:java}
> Caused by: java.lang.RuntimeException: Error while compiling generated Java
> code:
> public org.apache.calcite.linq4j.Enumerable bind(final
> org.apache.calcite.DataContext root) {
> final org.apache.calcite.linq4j.Enumerable _inputEnumerable =
> org.apache.calcite.schema.Schemas.queryable(root,
> root.getRootSchema().getSubSchema("foodmart2"), java.lang.Object[].class,
> "store").asEnumerable();
> return new org.apache.calcite.linq4j.AbstractEnumerable(){
> public org.apache.calcite.linq4j.Enumerator enumerator() {
> return new org.apache.calcite.linq4j.Enumerator(){
> public final org.apache.calcite.linq4j.Enumerator inputEnumerator
> = _inputEnumerable.enumerator();
> public void reset() {
> inputEnumerator.reset();
> }public boolean moveNext() {
> while (inputEnumerator.moveNext()) {
> final Integer inp16_ = (Integer) ((Object[])
> inputEnumerator.current())[16];
> if ((inp16_ ? inp16_.intValue() : 0) >= 2) {
> return true;
> }
> }
> return false;
> }public void close() {
> inputEnumerator.close();
> }public Object current() {
> return (Integer) ((Object[]) inputEnumerator.current())[16];
> } };
> }};
> }
> at org.apache.calcite.avatica.Helper.wrap(Helper.java:37)
> at org.apache.calcite.avatica.Helper.wrap(Helper.java:37)
> at
> org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:128)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:332)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:638)
>
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:502)
>
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:472)
>
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:231)
>
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550)
>
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
>
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> ... 29 more
> Caused by: org.codehaus.commons.compiler.CompileException: Line 14, Column
> 28: Not a boolean expression
> at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211)
> at org.codehaus.janino.UnitCompiler.compileBoolean2(UnitCompiler.java:3994)
> at org.codehaus.janino.UnitCompiler.access$6300(UnitCompiler.java:215){code}
> [1]
> [https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L1210]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3536) Wrong semantics in CoalesceImplementor
[
https://issues.apache.org/jira/browse/CALCITE-3536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16982104#comment-16982104
]
Jin Xing commented on CALCITE-3536:
---
Yes, I added in the description and proposed a PR to illustrate my fix ~
> Wrong semantics in CoalesceImplementor
> --
>
> Key: CALCITE-3536
> URL: https://issues.apache.org/jira/browse/CALCITE-3536
> Project: Calcite
> Issue Type: Bug
>Reporter: Jin Xing
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, COALESCE expression specified in sql string will be rewrite to
> CASE by SqlValidator [1], thus CoalesceImplementor is almost no use. But it
> is actually used somewhere, for example Sum0Splitter. When running
> AggregateRemove and split expression SUM0, a COALESCE is actually created and
> no chance to be rewrite to CASE anymore.
> In current code, CoalesceImplementor doesn't handle NullAs properly by
> semantics.
> If we disable the rewrite from COALESCE to CASE in Validator and run below
> sql in JDBC Test:
> {code:java}
> select grocery_sqft from store where coalesce(grocery_sqft, 0) >= 2{code}
> We will get below exception:
> {code:java}
> Caused by: java.lang.RuntimeException: Error while compiling generated Java
> code:
> public org.apache.calcite.linq4j.Enumerable bind(final
> org.apache.calcite.DataContext root) {
> final org.apache.calcite.linq4j.Enumerable _inputEnumerable =
> org.apache.calcite.schema.Schemas.queryable(root,
> root.getRootSchema().getSubSchema("foodmart2"), java.lang.Object[].class,
> "store").asEnumerable();
> return new org.apache.calcite.linq4j.AbstractEnumerable(){
> public org.apache.calcite.linq4j.Enumerator enumerator() {
> return new org.apache.calcite.linq4j.Enumerator(){
> public final org.apache.calcite.linq4j.Enumerator inputEnumerator
> = _inputEnumerable.enumerator();
> public void reset() {
> inputEnumerator.reset();
> }public boolean moveNext() {
> while (inputEnumerator.moveNext()) {
> final Integer inp16_ = (Integer) ((Object[])
> inputEnumerator.current())[16];
> if ((inp16_ ? inp16_.intValue() : 0) >= 2) {
> return true;
> }
> }
> return false;
> }public void close() {
> inputEnumerator.close();
> }public Object current() {
> return (Integer) ((Object[]) inputEnumerator.current())[16];
> } };
> }};
> }
> at org.apache.calcite.avatica.Helper.wrap(Helper.java:37)
> at org.apache.calcite.avatica.Helper.wrap(Helper.java:37)
> at
> org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:128)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:332)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:638)
>
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:502)
>
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:472)
>
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:231)
>
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550)
>
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
>
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> ... 29 more
> Caused by: org.codehaus.commons.compiler.CompileException: Line 14, Column
> 28: Not a boolean expression
> at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211)
> at org.codehaus.janino.UnitCompiler.compileBoolean2(UnitCompiler.java:3994)
> at org.codehaus.janino.UnitCompiler.access$6300(UnitCompiler.java:215){code}
> [1]
> [https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L1210]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3536) Wrong semantics in CoalesceImplementor
[
https://issues.apache.org/jira/browse/CALCITE-3536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jin Xing updated CALCITE-3536:
--
Description:
Currently, COALESCE expression specified in sql string will be rewrite to CASE
by SqlValidator [1], thus CoalesceImplementor is almost no use. But it is
actually used somewhere, for example Sum0Splitter. When running AggregateRemove
and split expression SUM0, a COALESCE is actually created and no chance to be
rewrite to CASE anymore.
In current code, CoalesceImplementor doesn't handle NullAs properly by
semantics.
If we disable the rewrite from COALESCE to CASE in Validator and run below sql
in JDBC Test:
{code:java}
select grocery_sqft from store where coalesce(grocery_sqft, 0) >= 2{code}
We will get below exception:
{code:java}
Caused by: java.lang.RuntimeException: Error while compiling generated Java
code:
public org.apache.calcite.linq4j.Enumerable bind(final
org.apache.calcite.DataContext root) {
final org.apache.calcite.linq4j.Enumerable _inputEnumerable =
org.apache.calcite.schema.Schemas.queryable(root,
root.getRootSchema().getSubSchema("foodmart2"), java.lang.Object[].class,
"store").asEnumerable();
return new org.apache.calcite.linq4j.AbstractEnumerable(){
public org.apache.calcite.linq4j.Enumerator enumerator() {
return new org.apache.calcite.linq4j.Enumerator(){
public final org.apache.calcite.linq4j.Enumerator inputEnumerator =
_inputEnumerable.enumerator();
public void reset() {
inputEnumerator.reset();
}public boolean moveNext() {
while (inputEnumerator.moveNext()) {
final Integer inp16_ = (Integer) ((Object[])
inputEnumerator.current())[16];
if ((inp16_ ? inp16_.intValue() : 0) >= 2) {
return true;
}
}
return false;
}public void close() {
inputEnumerator.close();
}public Object current() {
return (Integer) ((Object[]) inputEnumerator.current())[16];
} };
}};
}
at org.apache.calcite.avatica.Helper.wrap(Helper.java:37)
at org.apache.calcite.avatica.Helper.wrap(Helper.java:37)
at
org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:128)
at
org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:332)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:638)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:502)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:472)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:231)
at
org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550)
at
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
... 29 more
Caused by: org.codehaus.commons.compiler.CompileException: Line 14, Column 28:
Not a boolean expression
at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211)
at org.codehaus.janino.UnitCompiler.compileBoolean2(UnitCompiler.java:3994)
at org.codehaus.janino.UnitCompiler.access$6300(UnitCompiler.java:215){code}
This issue was found when I try to resolve CALCITE-3124 and CALCITE-3505
[1]
[https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L1210]
was:
Currently, COALESCE expression specified in sql string will be rewrite to CASE
by SqlValidator [1], thus CoalesceImplementor is almost no use. But it is
actually used somewhere, for example Sum0Splitter. When running AggregateRemove
and split expression SUM0, a COALESCE is actually created and no chance to be
rewrite to CASE anymore.
In current code, CoalesceImplementor doesn't handle NullAs properly by
semantics.
If we disable the rewrite from COALESCE to CASE in Validator and run below sql
in JDBC Test:
{code:java}
select grocery_sqft from store where coalesce(grocery_sqft, 0) >= 2{code}
We will get below exception:
{code:java}
Caused by: java.lang.RuntimeException: Error while compiling generated Java
code:
public org.apache.calcite.linq4j.Enumerable bind(final
org.apache.calcite.DataContext root) {
final org.apache.calcite.linq4j.Enumerable _inputEnumerable =
org.apache.calcite.schema.Schemas.queryable(root,
root.getRootSchema().getSubSchema("foodmart2"), java.lang.Object[].class,
"store").asEnumerable();
return new org.apach
[jira] [Commented] (CALCITE-3521) CalciteSystemProperty can't load config file
[
https://issues.apache.org/jira/browse/CALCITE-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16982191#comment-16982191
]
Jin Xing commented on CALCITE-3521:
---
PR by [~hyuan]
[https://github.com/apache/calcite/pull/1604]
> CalciteSystemProperty can't load config file
>
>
> Key: CALCITE-3521
> URL: https://issues.apache.org/jira/browse/CALCITE-3521
> Project: Calcite
> Issue Type: Bug
>Reporter: Haisheng Yuan
>Priority: Major
>
> I am using calcite as a standalone library, want to specify default charset
> and collation through global config file {{saffron.properties}}.
> But it failed to load property file. I think it should use
> {{Thread.currentThread().getContextClassLoader()}} to get class loader.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3466) JDBC adapter dropped group by statement in subquery
[
https://issues.apache.org/jira/browse/CALCITE-3466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-3466:
Labels: pull-request-available (was: )
> JDBC adapter dropped group by statement in subquery
> ---
>
> Key: CALCITE-3466
> URL: https://issues.apache.org/jira/browse/CALCITE-3466
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.21.0, 1.22.0
>Reporter: Wang Weidong
>Assignee: Wang Weidong
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Attachments: screenshot-1.png
>
>
> While convert RelNode to SqlNode, the "group by" statement in subquery was
> dropped unexpectedly. For example,
> original sql is:
> {code:sql}
> select a, avg(c1) from
> (
> select a,sum(d),b as c1 from t group by b,a
> ) as t
> group by a
> {code}
> RelNode converted by SqlNode is
> {code:sql}
> LogicalAggregate(group=[{0}], EXPR$1=[AVG($1)])
> LogicalProject(A=[$1], C1=[$0])
> LogicalAggregate(group=[{0, 1}], EXPR$1=[SUM($2)])
> LogicalProject(B=[$1], A=[$0], D=[$3])
> EnumerableTableScan(table=[[T_TESTPROJECTMERGE]])
> {code}
> And then convert this RelNode to SqlNode, the sql of SqlNode is like this:
> {code:sql}
> SELECT `A`, AVG(`B`)
> FROM `T`
> GROUP BY `A`
> {code}
> We can find that "group by" in subquery was dropped which may cause SqlNode
> converted by RelNode being not equivalent to RelNode.
> After debugging, I've found code
> at
> https://github.com/apache/calcite/blob/aa98c1575762718b238af8ddd44a390607de7602/core/src/main/java/org/apache/calcite/rel/rel2sql/RelToSqlConverter.java#L333
>
> calls org.apache.calcite.rel.rel2sql.SqlImplementor.Builder#setGroupBy
> without judge whether "group by" already exists.
> Finally, I think in this case, the main select and sub-select shouldn't be
> merged.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3521) CalciteSystemProperty can't load config file
[
https://issues.apache.org/jira/browse/CALCITE-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-3521:
Labels: pull-request-available (was: )
> CalciteSystemProperty can't load config file
>
>
> Key: CALCITE-3521
> URL: https://issues.apache.org/jira/browse/CALCITE-3521
> Project: Calcite
> Issue Type: Bug
>Reporter: Haisheng Yuan
>Priority: Major
> Labels: pull-request-available
>
> I am using calcite as a standalone library, want to specify default charset
> and collation through global config file {{saffron.properties}}.
> But it failed to load property file. I think it should use
> {{Thread.currentThread().getContextClassLoader()}} to get class loader.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
