[jira] [Commented] (CALCITE-4741) AbstractRelNode#getId can overflow into a negative value causing CompileException in certain Enumerable implement methods
[
https://issues.apache.org/jira/browse/CALCITE-4741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402078#comment-17402078
]
Ruben Q L commented on CALCITE-4741:
Thanks [~hyuan] for pointing it out! I will link that ticket to the current one.
> AbstractRelNode#getId can overflow into a negative value causing
> CompileException in certain Enumerable implement methods
> -
>
> Key: CALCITE-4741
> URL: https://issues.apache.org/jira/browse/CALCITE-4741
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.27.0
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.28.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{AbstractRelNode#getId}} can overflow into a negative value.
> This id is used by the {{implement}} method of several Enumerable operators
> ({{EnumerableBatchNestedLoopJoin}} and {{EnumerableMergeUnion}}) to guarantee
> a unique variable name in the dynamic code, e.g.:
> {code:java}
> final ParameterExpression corrArgList =
> Expressions.parameter(Modifier.FINAL,
> List.class, "corrList" + this.getId());
> {code}
> However, if the id is negative, this variable name will have a "-", causing a
> {{CompileException}}:
> {noformat}
> ...
> final org.apache.calcite.linq4j.Enumerable _inputEnumerable1 =
> org.apache.calcite.linq4j.EnumerableDefaults.correlateBatchJoin(org.apache.calcite.linq4j.JoinType.LEFT,
> left0, new org.apache.calcite.linq4j.function.Function1() {
> public org.apache.calcite.linq4j.AbstractEnumerable apply(final
> java.util.List corrList-1794407755) {
> ...
> Caused by: java.util.concurrent.ExecutionException:
> org.codehaus.commons.compiler.CompileException: Line 134, Column 103: One of
> ', )' expected instead of '-'
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-2736) ReduceExpressionsRule never reduces dynamic expressions but this should be configurable
[ https://issues.apache.org/jira/browse/CALCITE-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402309#comment-17402309 ] Yingyu commented on CALCITE-2736: - [~vladimirsitnikov]Thanks for your help in review and your comment. I agree with you that in general we should avoid negative naming. I've updated the naming using `treatDynamicCallsAsConstants` as suggested. Regarding the use case the direct usage would be to reduce query_user as part of planning despite it being a dynamic function. > ReduceExpressionsRule never reduces dynamic expressions but this should be > configurable > --- > > Key: CALCITE-2736 > URL: https://issues.apache.org/jira/browse/CALCITE-2736 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > There are situations where it is helpful to reduce dynamic SqlCalls. Right > now, the ReduceExpressionsRule always avoids doing this. We should enhance > the rule so this can be configurable depending on where in planning the rule > is used. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-2736) ReduceExpressionsRule never reduces dynamic expressions but this should be configurable
[
https://issues.apache.org/jira/browse/CALCITE-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402457#comment-17402457
]
Julian Hyde commented on CALCITE-2736:
--
The PR looks almost ready. A few comments:
* Expand the javadoc for the test methods. Something like "Tests that a
dynamic function (USER) is treated as a constant, and reduced, if \{link
ReduceExpressionsRule.Conig#treatDynamicCallsAsConstant} is false. Test case
for issue XXX."
* Javadoc for treatDynamicCallsAsConstant should say what the default is, and
give a brief example of the effect of the default.
* In the test, I don't think you need a new tester. You just need a new
executor, right? {{testReduceConstantsRequiresExecutor}} manages to set an
executor. Maybe you could call {{sql.withTransform}}
* The two test methods have quite a lot of common code. Can you figure out a
way to refactor them so that the common code is common and the differences are
apparent.
* Thanks for grouping the tests with other constant-reduction tests.
> ReduceExpressionsRule never reduces dynamic expressions but this should be
> configurable
> ---
>
> Key: CALCITE-2736
> URL: https://issues.apache.org/jira/browse/CALCITE-2736
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Jacques Nadeau
>Assignee: Jacques Nadeau
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> There are situations where it is helpful to reduce dynamic SqlCalls. Right
> now, the ReduceExpressionsRule always avoids doing this. We should enhance
> the rule so this can be configurable depending on where in planning the rule
> is used.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4740) JDBC adapter generates incorrect HAVING clause in BigQuery dialect
[
https://issues.apache.org/jira/browse/CALCITE-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated CALCITE-4740:
-
Labels: pull-request-available (was: )
> JDBC adapter generates incorrect HAVING clause in BigQuery dialect
> --
>
> Key: CALCITE-4740
> URL: https://issues.apache.org/jira/browse/CALCITE-4740
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.27.0
> Environment: jdk8
> macos
>Reporter: yanjing.wang
>Priority: Minor
> Labels: pull-request-available
>
> I'm converting
> {code:java}
> String query = select
> "product_id" , sum("shelf_width")
> from "foodmart"."product"
> where "product_id" > 0
> group by "product_id" having sum("shelf_width") > 1{code}
> with big query dialect, but results
> {code:java}
> SELECT product_id, SUM(shelf_width)
> FROM foodmart.product
> WHERE product_id > 0
> GROUP BY product_id
> HAVING `EXPR$1` > 1
> {code}
> Maybe we need replace having clause aliases when stripping trivial aliases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4740) JDBC adapter generates incorrect HAVING clause in BigQuery dialect
[
https://issues.apache.org/jira/browse/CALCITE-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402463#comment-17402463
]
Julian Hyde commented on CALCITE-4740:
--
There are a few places where "expr$" occurs:
{noformat}
$ git ls-files |xargs fgrep -i 'expr$"'
core/src/main/java/org/apache/calcite/rel/rel2sql/RelToSqlConverter.java:
if (fieldName.toLowerCase(Locale.ROOT).startsWith("expr$")) {
core/src/main/java/org/apache/calcite/rel/rel2sql/SqlImplementor.java:
.startsWith("expr$")) {
core/src/main/java/org/apache/calcite/sql/SqlUtil.java:return "EXPR$" +
ordinal;
core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorUtil.java:
(original, attempt, size) -> Util.first(original, "EXPR$") + attempt;
core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java:
String aliasBase = (alias == null) ? "EXPR$" : alias;
core/src/main/java/org/apache/calcite/tools/RelBuilder.java:name !=
null ? name : "expr$" + i);{noformat}
You could rationalize a few of them into a method in SqlUtil or
SqlValidatorUtil.
> JDBC adapter generates incorrect HAVING clause in BigQuery dialect
> --
>
> Key: CALCITE-4740
> URL: https://issues.apache.org/jira/browse/CALCITE-4740
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.27.0
> Environment: jdk8
> macos
>Reporter: yanjing.wang
>Priority: Minor
> Labels: pull-request-available
>
> I'm converting
> {code:java}
> String query = select
> "product_id" , sum("shelf_width")
> from "foodmart"."product"
> where "product_id" > 0
> group by "product_id" having sum("shelf_width") > 1{code}
> with big query dialect, but results
> {code:java}
> SELECT product_id, SUM(shelf_width)
> FROM foodmart.product
> WHERE product_id > 0
> GROUP BY product_id
> HAVING `EXPR$1` > 1
> {code}
> Maybe we need replace having clause aliases when stripping trivial aliases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-2823) Prevent optimization of non-deterministic functions
[
https://issues.apache.org/jira/browse/CALCITE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402464#comment-17402464
]
Julian Hyde commented on CALCITE-2823:
--
Here's how I see the hierarchy:
* for non-deterministic functions such as {{RAND}}, the result from two calls
may be different, even if they are called with the same arguments, in the same
row, in the same statement;
* for functions such as YYY, the result is the same if you make the same
sequence of calls, with the same arguments, to a given instance of the function;
* for dynamic functions such as {{CURRENT_TIMESTAMP}}, the result from two
calls may be different if called with different arguments, or in different
statements; but within a statement, is always the same if called with the same
arguments, even if they are called in different rows;
* for functions such as Oracle's {{NEXTVAL}} and MySQL's {{RAND()}}, the
result from two calls may be different if called with different arguments, or
in different statements, or in different rows of the same statement; but within
the same row of the same statement, is always the same if called with the same
arguments;
* for pure functions such as {{SUBSTRING}}, the result depends only on the
arguments.
Not sure what and should be.
> Prevent optimization of non-deterministic functions
> ---
>
> Key: CALCITE-2823
> URL: https://issues.apache.org/jira/browse/CALCITE-2823
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Siddharth Teotia
>Priority: Major
>
> RexSimplify and RexProgram optimize expressions like A() = A() to true, but
> if A() is a non-deterministic expression, this optimization is invalid.
> Change RexSimplify and RexProgram to check if expression is deterministic or
> not, in order to decide about the optimization.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-2823) Prevent optimization of non-deterministic functions
[
https://issues.apache.org/jira/browse/CALCITE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402464#comment-17402464
]
Julian Hyde edited comment on CALCITE-2823 at 8/21/21, 1:00 AM:
Here's how I see the hierarchy:
* for non-deterministic functions such as {{RAND}}, the result from two calls
may be different, even if they are called with the same arguments, in the same
row, in the same statement;
* for functions such as YYY, the result is the same if you make the same
sequence of calls, with the same arguments, to a given instance of the function;
* for dynamic functions such as {{CURRENT_TIMESTAMP}}, the result from two
calls may be different if called with different arguments, or in different
statements; but within a statement, is always the same if called with the same
arguments, even if they are called in different rows;
* for functions such as Oracle's {{NEXTVAL}} and MySQL's {{RAND()}}, the
result from two calls may be different if called with different arguments, or
in different statements, or in different rows of the same statement; but within
the same row of the same statement, is always the same if called with the same
arguments;
* for pure functions such as {{SUBSTRING}}, the result depends only on the
arguments.
Not sure what and should be.
Maybe there should be an enum that contains all of these classifications.
was (Author: julianhyde):
Here's how I see the hierarchy:
* for non-deterministic functions such as {{RAND}}, the result from two calls
may be different, even if they are called with the same arguments, in the same
row, in the same statement;
* for functions such as YYY, the result is the same if you make the same
sequence of calls, with the same arguments, to a given instance of the function;
* for dynamic functions such as {{CURRENT_TIMESTAMP}}, the result from two
calls may be different if called with different arguments, or in different
statements; but within a statement, is always the same if called with the same
arguments, even if they are called in different rows;
* for functions such as Oracle's {{NEXTVAL}} and MySQL's {{RAND()}}, the
result from two calls may be different if called with different arguments, or
in different statements, or in different rows of the same statement; but within
the same row of the same statement, is always the same if called with the same
arguments;
* for pure functions such as {{SUBSTRING}}, the result depends only on the
arguments.
Not sure what and should be.
> Prevent optimization of non-deterministic functions
> ---
>
> Key: CALCITE-2823
> URL: https://issues.apache.org/jira/browse/CALCITE-2823
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Siddharth Teotia
>Priority: Major
>
> RexSimplify and RexProgram optimize expressions like A() = A() to true, but
> if A() is a non-deterministic expression, this optimization is invalid.
> Change RexSimplify and RexProgram to check if expression is deterministic or
> not, in order to decide about the optimization.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4748) duplicate columnName in grouping sets get the wrong result
duan xiong created CALCITE-4748:
---
Summary: duplicate columnName in grouping sets get the wrong result
Key: CALCITE-4748
URL: https://issues.apache.org/jira/browse/CALCITE-4748
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.27.0
Reporter: duan xiong
Assignee: duan xiong
When run SQL1:
{code:java}
select prod_id from foodmart.sales_fact_1997 as s group by grouping
sets(prod_id,prod_id){code}
and SQL2:
{code:java}
select prod_id from foodmart.sales_fact_1997 as s group by grouping
sets(prod_id){code}
get the SAME result. This is wrong. SQL1 return result's number = SQL2 return
result*2 number
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
