[jira] [Commented] (CALCITE-4741) AbstractRelNode#getId can overflow into a negative value causing CompileException in certain Enumerable implement methods
[ https://issues.apache.org/jira/browse/CALCITE-4741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402078#comment-17402078 ] Ruben Q L commented on CALCITE-4741: Thanks [~hyuan] for pointing it out! I will link that ticket to the current one. > AbstractRelNode#getId can overflow into a negative value causing > CompileException in certain Enumerable implement methods > - > > Key: CALCITE-4741 > URL: https://issues.apache.org/jira/browse/CALCITE-4741 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.27.0 >Reporter: Ruben Q L >Assignee: Ruben Q L >Priority: Minor > Labels: pull-request-available > Fix For: 1.28.0 > > Time Spent: 10m > Remaining Estimate: 0h > > {{AbstractRelNode#getId}} can overflow into a negative value. > This id is used by the {{implement}} method of several Enumerable operators > ({{EnumerableBatchNestedLoopJoin}} and {{EnumerableMergeUnion}}) to guarantee > a unique variable name in the dynamic code, e.g.: > {code:java} > final ParameterExpression corrArgList = > Expressions.parameter(Modifier.FINAL, > List.class, "corrList" + this.getId()); > {code} > However, if the id is negative, this variable name will have a "-", causing a > {{CompileException}}: > {noformat} > ... > final org.apache.calcite.linq4j.Enumerable _inputEnumerable1 = > org.apache.calcite.linq4j.EnumerableDefaults.correlateBatchJoin(org.apache.calcite.linq4j.JoinType.LEFT, > left0, new org.apache.calcite.linq4j.function.Function1() { > public org.apache.calcite.linq4j.AbstractEnumerable apply(final > java.util.List corrList-1794407755) { > ... > Caused by: java.util.concurrent.ExecutionException: > org.codehaus.commons.compiler.CompileException: Line 134, Column 103: One of > ', )' expected instead of '-' > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-2736) ReduceExpressionsRule never reduces dynamic expressions but this should be configurable
[ https://issues.apache.org/jira/browse/CALCITE-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402309#comment-17402309 ] Yingyu commented on CALCITE-2736: - [~vladimirsitnikov]Thanks for your help in review and your comment. I agree with you that in general we should avoid negative naming. I've updated the naming using `treatDynamicCallsAsConstants` as suggested. Regarding the use case the direct usage would be to reduce query_user as part of planning despite it being a dynamic function. > ReduceExpressionsRule never reduces dynamic expressions but this should be > configurable > --- > > Key: CALCITE-2736 > URL: https://issues.apache.org/jira/browse/CALCITE-2736 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > There are situations where it is helpful to reduce dynamic SqlCalls. Right > now, the ReduceExpressionsRule always avoids doing this. We should enhance > the rule so this can be configurable depending on where in planning the rule > is used. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-2736) ReduceExpressionsRule never reduces dynamic expressions but this should be configurable
[ https://issues.apache.org/jira/browse/CALCITE-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402457#comment-17402457 ] Julian Hyde commented on CALCITE-2736: -- The PR looks almost ready. A few comments: * Expand the javadoc for the test methods. Something like "Tests that a dynamic function (USER) is treated as a constant, and reduced, if \{link ReduceExpressionsRule.Conig#treatDynamicCallsAsConstant} is false. Test case for issue XXX." * Javadoc for treatDynamicCallsAsConstant should say what the default is, and give a brief example of the effect of the default. * In the test, I don't think you need a new tester. You just need a new executor, right? {{testReduceConstantsRequiresExecutor}} manages to set an executor. Maybe you could call {{sql.withTransform}} * The two test methods have quite a lot of common code. Can you figure out a way to refactor them so that the common code is common and the differences are apparent. * Thanks for grouping the tests with other constant-reduction tests. > ReduceExpressionsRule never reduces dynamic expressions but this should be > configurable > --- > > Key: CALCITE-2736 > URL: https://issues.apache.org/jira/browse/CALCITE-2736 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > There are situations where it is helpful to reduce dynamic SqlCalls. Right > now, the ReduceExpressionsRule always avoids doing this. We should enhance > the rule so this can be configurable depending on where in planning the rule > is used. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4740) JDBC adapter generates incorrect HAVING clause in BigQuery dialect
[ https://issues.apache.org/jira/browse/CALCITE-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde updated CALCITE-4740: - Labels: pull-request-available (was: ) > JDBC adapter generates incorrect HAVING clause in BigQuery dialect > -- > > Key: CALCITE-4740 > URL: https://issues.apache.org/jira/browse/CALCITE-4740 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.27.0 > Environment: jdk8 > macos >Reporter: yanjing.wang >Priority: Minor > Labels: pull-request-available > > I'm converting > {code:java} > String query = select > "product_id" , sum("shelf_width") > from "foodmart"."product" > where "product_id" > 0 > group by "product_id" having sum("shelf_width") > 1{code} > with big query dialect, but results > {code:java} > SELECT product_id, SUM(shelf_width) > FROM foodmart.product > WHERE product_id > 0 > GROUP BY product_id > HAVING `EXPR$1` > 1 > {code} > Maybe we need replace having clause aliases when stripping trivial aliases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4740) JDBC adapter generates incorrect HAVING clause in BigQuery dialect
[ https://issues.apache.org/jira/browse/CALCITE-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402463#comment-17402463 ] Julian Hyde commented on CALCITE-4740: -- There are a few places where "expr$" occurs: {noformat} $ git ls-files |xargs fgrep -i 'expr$"' core/src/main/java/org/apache/calcite/rel/rel2sql/RelToSqlConverter.java: if (fieldName.toLowerCase(Locale.ROOT).startsWith("expr$")) { core/src/main/java/org/apache/calcite/rel/rel2sql/SqlImplementor.java: .startsWith("expr$")) { core/src/main/java/org/apache/calcite/sql/SqlUtil.java:return "EXPR$" + ordinal; core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorUtil.java: (original, attempt, size) -> Util.first(original, "EXPR$") + attempt; core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java: String aliasBase = (alias == null) ? "EXPR$" : alias; core/src/main/java/org/apache/calcite/tools/RelBuilder.java:name != null ? name : "expr$" + i);{noformat} You could rationalize a few of them into a method in SqlUtil or SqlValidatorUtil. > JDBC adapter generates incorrect HAVING clause in BigQuery dialect > -- > > Key: CALCITE-4740 > URL: https://issues.apache.org/jira/browse/CALCITE-4740 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.27.0 > Environment: jdk8 > macos >Reporter: yanjing.wang >Priority: Minor > Labels: pull-request-available > > I'm converting > {code:java} > String query = select > "product_id" , sum("shelf_width") > from "foodmart"."product" > where "product_id" > 0 > group by "product_id" having sum("shelf_width") > 1{code} > with big query dialect, but results > {code:java} > SELECT product_id, SUM(shelf_width) > FROM foodmart.product > WHERE product_id > 0 > GROUP BY product_id > HAVING `EXPR$1` > 1 > {code} > Maybe we need replace having clause aliases when stripping trivial aliases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-2823) Prevent optimization of non-deterministic functions
[ https://issues.apache.org/jira/browse/CALCITE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402464#comment-17402464 ] Julian Hyde commented on CALCITE-2823: -- Here's how I see the hierarchy: * for non-deterministic functions such as {{RAND}}, the result from two calls may be different, even if they are called with the same arguments, in the same row, in the same statement; * for functions such as YYY, the result is the same if you make the same sequence of calls, with the same arguments, to a given instance of the function; * for dynamic functions such as {{CURRENT_TIMESTAMP}}, the result from two calls may be different if called with different arguments, or in different statements; but within a statement, is always the same if called with the same arguments, even if they are called in different rows; * for functions such as Oracle's {{NEXTVAL}} and MySQL's {{RAND()}}, the result from two calls may be different if called with different arguments, or in different statements, or in different rows of the same statement; but within the same row of the same statement, is always the same if called with the same arguments; * for pure functions such as {{SUBSTRING}}, the result depends only on the arguments. Not sure what and should be. > Prevent optimization of non-deterministic functions > --- > > Key: CALCITE-2823 > URL: https://issues.apache.org/jira/browse/CALCITE-2823 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Siddharth Teotia >Priority: Major > > RexSimplify and RexProgram optimize expressions like A() = A() to true, but > if A() is a non-deterministic expression, this optimization is invalid. > Change RexSimplify and RexProgram to check if expression is deterministic or > not, in order to decide about the optimization. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-2823) Prevent optimization of non-deterministic functions
[ https://issues.apache.org/jira/browse/CALCITE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17402464#comment-17402464 ] Julian Hyde edited comment on CALCITE-2823 at 8/21/21, 1:00 AM: Here's how I see the hierarchy: * for non-deterministic functions such as {{RAND}}, the result from two calls may be different, even if they are called with the same arguments, in the same row, in the same statement; * for functions such as YYY, the result is the same if you make the same sequence of calls, with the same arguments, to a given instance of the function; * for dynamic functions such as {{CURRENT_TIMESTAMP}}, the result from two calls may be different if called with different arguments, or in different statements; but within a statement, is always the same if called with the same arguments, even if they are called in different rows; * for functions such as Oracle's {{NEXTVAL}} and MySQL's {{RAND()}}, the result from two calls may be different if called with different arguments, or in different statements, or in different rows of the same statement; but within the same row of the same statement, is always the same if called with the same arguments; * for pure functions such as {{SUBSTRING}}, the result depends only on the arguments. Not sure what and should be. Maybe there should be an enum that contains all of these classifications. was (Author: julianhyde): Here's how I see the hierarchy: * for non-deterministic functions such as {{RAND}}, the result from two calls may be different, even if they are called with the same arguments, in the same row, in the same statement; * for functions such as YYY, the result is the same if you make the same sequence of calls, with the same arguments, to a given instance of the function; * for dynamic functions such as {{CURRENT_TIMESTAMP}}, the result from two calls may be different if called with different arguments, or in different statements; but within a statement, is always the same if called with the same arguments, even if they are called in different rows; * for functions such as Oracle's {{NEXTVAL}} and MySQL's {{RAND()}}, the result from two calls may be different if called with different arguments, or in different statements, or in different rows of the same statement; but within the same row of the same statement, is always the same if called with the same arguments; * for pure functions such as {{SUBSTRING}}, the result depends only on the arguments. Not sure what and should be. > Prevent optimization of non-deterministic functions > --- > > Key: CALCITE-2823 > URL: https://issues.apache.org/jira/browse/CALCITE-2823 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Siddharth Teotia >Priority: Major > > RexSimplify and RexProgram optimize expressions like A() = A() to true, but > if A() is a non-deterministic expression, this optimization is invalid. > Change RexSimplify and RexProgram to check if expression is deterministic or > not, in order to decide about the optimization. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4748) duplicate columnName in grouping sets get the wrong result
duan xiong created CALCITE-4748: --- Summary: duplicate columnName in grouping sets get the wrong result Key: CALCITE-4748 URL: https://issues.apache.org/jira/browse/CALCITE-4748 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.27.0 Reporter: duan xiong Assignee: duan xiong When run SQL1: {code:java} select prod_id from foodmart.sales_fact_1997 as s group by grouping sets(prod_id,prod_id){code} and SQL2: {code:java} select prod_id from foodmart.sales_fact_1997 as s group by grouping sets(prod_id){code} get the SAME result. This is wrong. SQL1 return result's number = SQL2 return result*2 number -- This message was sent by Atlassian Jira (v8.3.4#803005)