[jira] [Commented] (CALCITE-5881) Support to get foreign keys metadata in RelMetadataQuery
[
https://issues.apache.org/jira/browse/CALCITE-5881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750925#comment-17750925
]
Chunwei Lei commented on CALCITE-5881:
--
Sounds reasonable.
> Support to get foreign keys metadata in RelMetadataQuery
>
>
> Key: CALCITE-5881
> URL: https://issues.apache.org/jira/browse/CALCITE-5881
> Project: Calcite
> Issue Type: New Feature
>Reporter: JingDas
>Assignee: JingDas
>Priority: Major
> Labels: pull-request-available
>
> We can get constraints by RelOptTable#getReferentialConstraints method, but
> maybe can't get appropriate constraints at top relNode.
> For example:
> SQL:
> {code:java}
> SELECT DEPT.name, emp_agg.deptno, emp_agg.ename, DEPT.deptno
> FROM DEPT
> RIGHT JOIN
> (SELECT COUNT(sal), deptno, ename FROM EMP GROUP BY deptno, ename) emp_agg
> ON DEPT.deptno = emp_agg.deptno
> WHERE emp_agg.ename = 'job'{code}
> The relNode is:
> {code:java}
> LogicalProject(NAME=[$1], DEPTNO=[$3], ENAME=[$4], DEPTNO0=[$0])
> LogicalFilter(condition=[=($4, 'job')])
> LogicalJoin(condition=[=($0, $3)], joinType=[right])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalProject(EXPR$0=[$2], DEPTNO=[$0], ENAME=[$1])
> LogicalAggregate(group=[{0, 1}], EXPR$0=[COUNT($2)])
> LogicalProject(DEPTNO=[$7], ENAME=[$1], SAL=[$5])
> LogicalTableScan(table=[[CATALOG, SALES, EMPNULLABLES]]){code}
> where the foreign key is the DEPTNO column of CATALOG.SALES.EMP table,
> reference the DEPTNO unique column of CATALOG.SALES.DEPT table.
> When we want to get foreign keys metadata on `LogicalJoin` or top
> `LogicalProject`, There is no such method currently, it seems that we should
> trace the field column origin to get the foreign key and corresponding unique
> key.
>
> The final result of this feature is something likely as following:
> When we want to get foreign keys metadata on `LogicalJoin`, the `LogicalJoin`
> rowType is
> {code:java}
> RecordType(INTEGER DEPTNO, VARCHAR(10) NAME, BIGINT EXPR$0, INTEGER DEPTNO0,
> VARCHAR(20) ENAME).{code}
> We expect the foreign keys metadata:
> {code:java}
> foreignColumns bitset is {3}
> uniqueColumns bitset is {0}{code}
> When we want to get foreign keys metadata on top `LogicalProject`, the
> `LogicalProject` rowType is
> {code:java}
> RecordType(VARCHAR(10) NAME, INTEGER DEPTNO, VARCHAR(20) ENAME, INTEGER
> DEPTNO0).{code}
> We expect the foreigns key metadata:
> {code:java}
> foreignColumns bitset is {1}
> uniqueColumns bitset is {3}{code}
> All the foreign or unique columns is 0 based index.
> Foreign keys metadata is very useful in many optimize scenes. Such as it can
> be used in join eliminate when join type is inner join and some other star
> schema query optimize.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5861) Optimization rules do not constant-fold expressions in window bounds
[
https://issues.apache.org/jira/browse/CALCITE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744849#comment-17744849
]
Chunwei Lei edited comment on CALCITE-5861 at 7/20/23 2:52 AM:
---
I tried the query mentioned in PostgreSQL and it works:
{code:sql}
postgres=# select COUNT(*) over ( ORDER BY empno ROWS BETWEEN 5 + 5 PRECEDING
AND 1 PRECEDING) AS w_avg from emp;
w_avg
---
(0 rows)
{code}
So I agree with your proposal.
Besides, the following sql cannot run in PostgreSQL:
{code:sql}
postgres=# select COUNT(*) over ( ORDER BY empno ROWS BETWEEN 5 + empno
PRECEDING AND 1 PRECEDING) AS w_avg from emp;
ERROR: argument of ROWS must not contain variables
LINE 1: ...t COUNT(*) over ( ORDER BY empno ROWS BETWEEN 5 + empno PREC...
{code}
but it does not throw an exception in calcite. We should fix it too. WDYT?
was (Author: chunwei lei):
I tried the query mentioned in PostgreSQL and it works:
{code:sql}
postgres=# select COUNT(*) over ( ORDER BY empno ROWS BETWEEN 5 + 5 PRECEDING
AND 1 PRECEDING) AS w_avg from emp;
w_avg
---
(0 rows)
{code}
So I agree with your proposal.
Besides, the following sql cannot run in PostgreSQL:
{code:sql}
postgres=# select COUNT(*) over ( ORDER BY empno ROWS BETWEEN 5 + empno
PRECEDING AND 1 PRECEDING) AS w_avg from emp;
ERROR: argument of ROWS must not contain variables
LINE 1: ...t COUNT(*) over ( ORDER BY empno ROWS BETWEEN 5 + empno PREC...
{code}
but it does not throw an exception in calcite.
> Optimization rules do not constant-fold expressions in window bounds
>
>
> Key: CALCITE-5861
> URL: https://issues.apache.org/jira/browse/CALCITE-5861
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.34.0
>Reporter: Mihai Budiu
>Priority: Minor
> Labels: pull-request-available
>
> This bug is a bit more complicated to describe, and there is a possibility
> that I am doing something wrong.
> Consider the following test that can be added to RelOptRulesTest:
> {code:java}
> @Test void testExpressionPreceding() {
> HepProgramBuilder preBuilder = new HepProgramBuilder();
> preBuilder.addRuleInstance(CoreRules.WINDOW_REDUCE_EXPRESSIONS);
>
> preBuilder.addRuleInstance(CoreRules.PROJECT_TO_LOGICAL_PROJECT_AND_WINDOW);
> HepProgramBuilder builder = new HepProgramBuilder();
> builder.addRuleInstance(CoreRules.PROJECT_REDUCE_EXPRESSIONS);
> HepPlanner hepPlanner = new HepPlanner(builder.build());
> final String sql =
> "select COUNT(*) over (\n"
> + "ORDER BY empno\n"
> + "ROWS BETWEEN 5 + 5 PRECEDING AND 1 PRECEDING) AS w_avg\n"
> + " from emp\n";
> sql(sql)
> .withPre(preBuilder.build())
> .withPlanner(hepPlanner)
> .check();
> }
> {code}
> The plan before looks like this:
> {code:java}
> LogicalProject($0=[$2])
> LogicalWindow(window#0=[window(order by [0] rows between $1 PRECEDING and
> $2 PRECEDING aggs [COUNT()])])
> LogicalProject(EMPNO=[$0], $1=[+(5, 5)])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> The problem is that the expression "5+5", which appears in the window bounds,
> has not been reduced to a constant by the rule WINDOW_REDUCE_EXPRESSIONS.
> Moreover, the next optimization rule PROJECT_TO_LOGICAL_PROJECT_AND_WINDOW
> has pushed this expression into the LogicalProject. So it appears locally
> that the LogicalWindow no longer has a constant bound, which is required by
> the SQL language spec (it is constant, but that is no longer apparent in the
> query). (At least our code generator is upset by this state of affairs.)
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5861) Optimization rules do not constant-fold expressions in window bounds
[
https://issues.apache.org/jira/browse/CALCITE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744849#comment-17744849
]
Chunwei Lei commented on CALCITE-5861:
--
I tried the query mentioned in PostgreSQL and it works:
{code:sql}
postgres=# select COUNT(*) over ( ORDER BY empno ROWS BETWEEN 5 + 5 PRECEDING
AND 1 PRECEDING) AS w_avg from emp;
w_avg
---
(0 rows)
{code}
So I agree with your proposal.
Besides, the following sql cannot run in PostgreSQL:
{code:sql}
postgres=# select COUNT(*) over ( ORDER BY empno ROWS BETWEEN 5 + empno
PRECEDING AND 1 PRECEDING) AS w_avg from emp;
ERROR: argument of ROWS must not contain variables
LINE 1: ...t COUNT(*) over ( ORDER BY empno ROWS BETWEEN 5 + empno PREC...
{code}
but it does not throw an exception in calcite.
> Optimization rules do not constant-fold expressions in window bounds
>
>
> Key: CALCITE-5861
> URL: https://issues.apache.org/jira/browse/CALCITE-5861
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.34.0
>Reporter: Mihai Budiu
>Priority: Minor
> Labels: pull-request-available
>
> This bug is a bit more complicated to describe, and there is a possibility
> that I am doing something wrong.
> Consider the following test that can be added to RelOptRulesTest:
> {code:java}
> @Test void testExpressionPreceding() {
> HepProgramBuilder preBuilder = new HepProgramBuilder();
> preBuilder.addRuleInstance(CoreRules.WINDOW_REDUCE_EXPRESSIONS);
>
> preBuilder.addRuleInstance(CoreRules.PROJECT_TO_LOGICAL_PROJECT_AND_WINDOW);
> HepProgramBuilder builder = new HepProgramBuilder();
> builder.addRuleInstance(CoreRules.PROJECT_REDUCE_EXPRESSIONS);
> HepPlanner hepPlanner = new HepPlanner(builder.build());
> final String sql =
> "select COUNT(*) over (\n"
> + "ORDER BY empno\n"
> + "ROWS BETWEEN 5 + 5 PRECEDING AND 1 PRECEDING) AS w_avg\n"
> + " from emp\n";
> sql(sql)
> .withPre(preBuilder.build())
> .withPlanner(hepPlanner)
> .check();
> }
> {code}
> The plan before looks like this:
> {code:java}
> LogicalProject($0=[$2])
> LogicalWindow(window#0=[window(order by [0] rows between $1 PRECEDING and
> $2 PRECEDING aggs [COUNT()])])
> LogicalProject(EMPNO=[$0], $1=[+(5, 5)])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> The problem is that the expression "5+5", which appears in the window bounds,
> has not been reduced to a constant by the rule WINDOW_REDUCE_EXPRESSIONS.
> Moreover, the next optimization rule PROJECT_TO_LOGICAL_PROJECT_AND_WINDOW
> has pushed this expression into the LogicalProject. So it appears locally
> that the LogicalWindow no longer has a constant bound, which is required by
> the SQL language spec (it is constant, but that is no longer apparent in the
> query). (At least our code generator is upset by this state of affairs.)
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5837) After RexUtil.pullFactors method,sometimes the condition order of RexNode is not the same as before
[
https://issues.apache.org/jira/browse/CALCITE-5837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17742626#comment-17742626
]
Chunwei Lei commented on CALCITE-5837:
--
+1.
> After RexUtil.pullFactors method,sometimes the condition order of RexNode is
> not the same as before
> ---
>
> Key: CALCITE-5837
> URL: https://issues.apache.org/jira/browse/CALCITE-5837
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.34.0
>Reporter: LakeShen
>Priority: Minor
> Fix For: 1.36.0
>
>
> In my rule, I used the RexUtil.pullFactors to simplify the Filter's condition
> and I tested the rule with TPC-H q19.sql.
> When I print the plan tree with RelOptUtil.toString,sometimes the plan is :
> {code:java}
> LogicalAggregate(group=[{}], revenue=[SUM($0)])
> LogicalProject($f0=[*($5, -(1, $6))])
> LogicalFilter(condition=[AND(=($16, $1), SEARCH($14, Sarg['AIR':CHAR(7),
> 'AIR REG']:CHAR(7)), =($13, 'DELIVER IN PERSON'), OR(AND(=($19, 'Brand#12'),
> SEARCH($22, Sarg['SM BOX':CHAR(7), 'SM CASE', 'SM PACK', 'SM
> PKG':CHAR(7)]:CHAR(7)), >=($4, 1), <=($4, +(1, 10)), SEARCH($21,
> Sarg[[1..5]])), AND(=($19, 'Brand#23'), SEARCH($22, Sarg['MED BAG':CHAR(8),
> 'MED BOX':CHAR(8), 'MED PACK', 'MED PKG':CHAR(8)]:CHAR(8)), >=($4, 10),
> <=($4, +(10, 10)), SEARCH($21, Sarg[[1..10]])), AND(=($19, 'Brand#34'),
> SEARCH($22, Sarg['LG BOX':CHAR(7), 'LG CASE', 'LG PACK', 'LG
> PKG':CHAR(7)]:CHAR(7)), >=($4, 20), <=($4, +(20, 10)), SEARCH($21,
> Sarg[[1..15]]])
> LogicalJoin(condition=[true], joinType=[inner])
> LogicalTableScan(table=[[tpch, LINEITEM]])
> LogicalTableScan(table=[[tpch, PART]])
> {code}
> and sometimes the plan is :
> {code:java}
> LogicalAggregate(group=[{}], revenue=[SUM($0)])
> LogicalProject($f0=[*($5, -(1, $6))])
> LogicalFilter(condition=[AND(=($16, $1), =($13, 'DELIVER IN PERSON'),
> SEARCH($14, Sarg['AIR':CHAR(7), 'AIR REG']:CHAR(7)), OR(AND(=($19,
> 'Brand#12'), SEARCH($22, Sarg['SM BOX':CHAR(7), 'SM CASE', 'SM PACK', 'SM
> PKG':CHAR(7)]:CHAR(7)), >=($4, 1), <=($4, +(1, 10)), SEARCH($21,
> Sarg[[1..5]])), AND(=($19, 'Brand#23'), SEARCH($22, Sarg['MED BAG':CHAR(8),
> 'MED BOX':CHAR(8), 'MED PACK', 'MED PKG':CHAR(8)]:CHAR(8)), >=($4, 10),
> <=($4, +(10, 10)), SEARCH($21, Sarg[[1..10]])), AND(=($19, 'Brand#34'),
> SEARCH($22, Sarg['LG BOX':CHAR(7), 'LG CASE', 'LG PACK', 'LG
> PKG':CHAR(7)]:CHAR(7)), >=($4, 20), <=($4, +(20, 10)), SEARCH($21,
> Sarg[[1..15]]])
> LogicalJoin(condition=[true], joinType=[inner])
> LogicalTableScan(table=[[tpch, LINEITEM]])
> LogicalTableScan(table=[[tpch, PART]])
> {code}
> The above two SQL Plan semantics are the same, the only difference is that
> the Filter conditions are in different order.
> Although this has no effect on the SQL execution results, it is difficult for
> me to monitor my plan because of the variability of the plan.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5842) LogicalProject deepHashCode creates same value with different RowType
[
https://issues.apache.org/jira/browse/CALCITE-5842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17742622#comment-17742622
]
Chunwei Lei commented on CALCITE-5842:
--
bq. The 2 logicalProjects from the 2 LogicalTableScans have same hashCode value
based on the deepHashCode method in LogicalProject.
AFAIU, it means the exps and the input of 2 projects are same. We can infer
that the RowType of 2 Projects is same too. So even we consider the RowType, it
does not make a difference. Correct me if I am wrong.
> LogicalProject deepHashCode creates same value with different RowType
> --
>
> Key: CALCITE-5842
> URL: https://issues.apache.org/jira/browse/CALCITE-5842
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.32.0
>Reporter: Yu Tian
>Priority: Major
>
> The LogicalProject class has deepEquals0 and deepHashCode0 methods, in the
> deepEquals0 method, it consider getRowType() as one equal standard, however,
> in the deepHashCode0, it is missing the getRowType() to generated the hash
> value. Do we do this on purpose or it is a bug?
> [https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/core/Project.java#L348,L368]
>
>
> The reason we ask is that we are trying 2 use cases from our side.
> The first one is two LogicalTableScan with similar configurations, which are
> connected to 2 separate LogicalFiler, then we LogicalJoin these 2 together.
> One issue we noticed is that, in HepPlanner, it has logics as below
>
> package org.apache.calcite.plan.hep.HepPlanner
> {code:java}
> // try to find equivalent rel only if DAG is allowed
> if (!noDag) {
> // Now, check if an equivalent vertex already exists in graph.
> HepRelVertex equivVertex = mapDigestToVertex.get(rel.getRelDigest());
> if (equivVertex != null) {
> // Use existing vertex.
> return equivVertex;
> }
> } {code}
>
> The 2 logicalProjects from the 2 LogicalTableScans have same hashCode value
> based on the deepHashCode method in LogicalProject, because it didn’t
> consider the getRowType() value, the planner is replacing LogicalTableScan2
> with LogicalTableScan1, in fact, we should treat them as separate items to
> process.
>
> Another use case we have, we have 2 diagrams, each diagram with
> LogicalTableScan, LogicalFiler, LogicalTableModify, LogicalTableScan have
> similar setup with different rowType information. This time, HepPlanner is
> passing, since it has separate HepPlanner stage, so above issue is not
> happening. However, when it reach the VolcanoPlanner, the logics
>
> package org.apache.calcite.plan.volcano.VolcanoPlanner
> {code:java}
> // If it is equivalent to an existing expression, return the set that
> // the equivalent expression belongs to.
> RelDigest digest = rel.getRelDigest();
> RelNode equivExp = mapDigestToRel.get(digest); {code}
>
> The map replace the LogicalTableScan1 with LogicalTableScan2 in the
> LogicalProject stage since they have same hashCode, and the map is reusing
> earlier processed RelNode, which caused the issues.
>
> Here are the proposals we have,
>
> * Narrow Scope change: LogicalProject is the most frequently used project
> type, we only change it.
> ** Modify the LogicalProject method deepHashCode method to use
> {code:java}
> @Override public int deepHashCode() {
> return Objects.hash(traitSet, input.deepHashCode(), exps, hints,
> getRowType());
> }{code}
> Consider the getRowType() value in the hash generation will resolve the
> issue, since the rowType contains the field names and data types information.
>
> * Whole Scope change: Change the deepHashCode method in Project class.
> ** Similar change as above, however, the scope of this change is wide
> compared to the first one.
>
> Is it something we can consider to improve in the following release of Apache
> Calcite?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5639) RexSimplify should remove IS NOT NULL check when LIKE comparison is present
[
https://issues.apache.org/jira/browse/CALCITE-5639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17711168#comment-17711168
]
Chunwei Lei commented on CALCITE-5639:
--
I can't agree more. It would be great if we can find a more general solution.
> RexSimplify should remove IS NOT NULL check when LIKE comparison is present
> ---
>
> Key: CALCITE-5639
> URL: https://issues.apache.org/jira/browse/CALCITE-5639
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.34.0
>Reporter: Leonid Chistov
>Assignee: Leonid Chistov
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Consider query like
> {code:java}
> select r_reason_desc from reason
> where r_reason_desc is not null and r_reason_desc like '%child%'
> {code}
> where "r_reason_desc" is a nullable field.
> When RexSimplify::simplifyFilterPredicates is called on that conjunction of
> expressions, expression is not simplified, meaning that redundant "is not
> null" check is not removed.
> In the same time, if query like
> {code:java}
> select r_reason_desc from reason
> where r_reason_desc is not null and r_reason_desc >= 'A'{code}
> is passed to optimizer, redundant "is not null" check is eliminated.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5267) Remove useless variable 'newCasts' in AggregateCaseToFilterRule
[
https://issues.apache.org/jira/browse/CALCITE-5267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17600690#comment-17600690
]
Chunwei Lei edited comment on CALCITE-5267 at 9/6/22 9:27 AM:
--
I checked the history. The rule is introduced from Druid[1], in which newCasts
is used. It may be better to correct it instead of removing it.
[1]
https://github.com/apache/druid/pull/4360/files#diff-809e3a73d1485dad83629598ea5bcbab8f5ce7724e9572975b052176c44fbdf7
was (Author: chunwei lei):
I checked the history. The rule is introduced from Druid[1], in which {{
newCasts }} is used. It may be better to correct it instead of removing it.
[1]
https://github.com/apache/druid/pull/4360/files#diff-809e3a73d1485dad83629598ea5bcbab8f5ce7724e9572975b052176c44fbdf7
> Remove useless variable 'newCasts' in AggregateCaseToFilterRule
> ---
>
> Key: CALCITE-5267
> URL: https://issues.apache.org/jira/browse/CALCITE-5267
> Project: Calcite
> Issue Type: Bug
>Reporter: Wenzhuang Zhu
>Assignee: Wenzhuang Zhu
>Priority: Minor
> Labels: pull-request-available
> Original Estimate: 24h
> Time Spent: 10m
> Remaining Estimate: 23h 50m
>
> AggregateCaseToFilterRule::onMatch contains a variable 'newCasts' that has
> never been used.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5267) Remove useless variable 'newCasts' in AggregateCaseToFilterRule
[
https://issues.apache.org/jira/browse/CALCITE-5267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17600690#comment-17600690
]
Chunwei Lei edited comment on CALCITE-5267 at 9/6/22 9:26 AM:
--
I checked the history. The rule is introduced from Druid[1], in which {{
newCasts }} is used. It may be better to correct it instead of removing it.
[1]
https://github.com/apache/druid/pull/4360/files#diff-809e3a73d1485dad83629598ea5bcbab8f5ce7724e9572975b052176c44fbdf7
was (Author: chunwei lei):
I checked the history. The rule is introduced from Druid[1], in which newCasts
is used. It may be better to correct it instead of remove it.
[1]https://github.com/apache/druid/pull/4360/files#diff-809e3a73d1485dad83629598ea5bcbab8f5ce7724e9572975b052176c44fbdf7
> Remove useless variable 'newCasts' in AggregateCaseToFilterRule
> ---
>
> Key: CALCITE-5267
> URL: https://issues.apache.org/jira/browse/CALCITE-5267
> Project: Calcite
> Issue Type: Bug
>Reporter: Wenzhuang Zhu
>Assignee: Wenzhuang Zhu
>Priority: Minor
> Labels: pull-request-available
> Original Estimate: 24h
> Time Spent: 10m
> Remaining Estimate: 23h 50m
>
> AggregateCaseToFilterRule::onMatch contains a variable 'newCasts' that has
> never been used.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5267) Remove useless variable 'newCasts' in AggregateCaseToFilterRule
[ https://issues.apache.org/jira/browse/CALCITE-5267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17600690#comment-17600690 ] Chunwei Lei edited comment on CALCITE-5267 at 9/6/22 9:26 AM: -- I checked the history. The rule is introduced from Druid[1], in which newCasts is used. It may be better to correct it instead of remove it. [1]https://github.com/apache/druid/pull/4360/files#diff-809e3a73d1485dad83629598ea5bcbab8f5ce7724e9572975b052176c44fbdf7 was (Author: chunwei lei): Makes sense. > Remove useless variable 'newCasts' in AggregateCaseToFilterRule > --- > > Key: CALCITE-5267 > URL: https://issues.apache.org/jira/browse/CALCITE-5267 > Project: Calcite > Issue Type: Bug >Reporter: Wenzhuang Zhu >Assignee: Wenzhuang Zhu >Priority: Minor > Labels: pull-request-available > Original Estimate: 24h > Time Spent: 10m > Remaining Estimate: 23h 50m > > AggregateCaseToFilterRule::onMatch contains a variable 'newCasts' that has > never been used. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-5267) Remove useless variable 'newCasts' in AggregateCaseToFilterRule
[ https://issues.apache.org/jira/browse/CALCITE-5267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17600690#comment-17600690 ] Chunwei Lei commented on CALCITE-5267: -- Makes sense. > Remove useless variable 'newCasts' in AggregateCaseToFilterRule > --- > > Key: CALCITE-5267 > URL: https://issues.apache.org/jira/browse/CALCITE-5267 > Project: Calcite > Issue Type: Bug >Reporter: Wenzhuang Zhu >Assignee: Wenzhuang Zhu >Priority: Minor > Labels: pull-request-available > Original Estimate: 24h > Time Spent: 10m > Remaining Estimate: 23h 50m > > AggregateCaseToFilterRule::onMatch contains a variable 'newCasts' that has > never been used. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-5231) Unable to create Oracle JDBC connection in DBeaver
[
https://issues.apache.org/jira/browse/CALCITE-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17577257#comment-17577257
]
Chunwei Lei commented on CALCITE-5231:
--
Hi, [~kartik.kud...@exate.com]. It's better to send an email to the dev mailing
list if you need help.
> Unable to create Oracle JDBC connection in DBeaver
> --
>
> Key: CALCITE-5231
> URL: https://issues.apache.org/jira/browse/CALCITE-5231
> Project: Calcite
> Issue Type: Bug
> Components: core, jdbc-adapter
>Affects Versions: 1.22.0
>Reporter: kartik
>Priority: Major
> Attachments: Calcite JDBC-DBeaver-connection-Issue.png,
> JDBC-calcite-connection-Error-Message.png
>
>
> I am trying to connect my Oracle database using calcite in DBever.
> Driver class used : org.apache.calcite.jdbc.Driver
> URL - jdbc:oracle:thin:XXX:1521:
>
> But when I use same same URL and driver with main java program . it works
> fine .
> Connection con= DriverManager.getConnection("jdbc:calcite:schemaType=JDBC;
> schema.jdbcUser=USERNAME; " +
> "schema.jdbcPassword=XX;
> schema.jdbcUrl=jdbc:oracle:thin:@X:1521:ORCL;
> schema.conformance=ORACLE_10; schema.fun=oracle" );
>
>
> I have cloned calcite project and running main program on that . Please let
> me know if I have done any mistake .
>
> After Oracle , I need to work for mySql using calcite .
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (CALCITE-4223) Introducing column statistics to RelOptTable
[ https://issues.apache.org/jira/browse/CALCITE-4223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunwei Lei resolved CALCITE-4223. -- Fix Version/s: 1.32.0 Resolution: Fixed Fixed in https://github.com/apache/calcite/commit/7e0057e8de93930f1b2952a1cbcee8ad7a6bfb4b. Thank you for your view, [~asolimando]! > Introducing column statistics to RelOptTable > > > Key: CALCITE-4223 > URL: https://issues.apache.org/jira/browse/CALCITE-4223 > Project: Calcite > Issue Type: Improvement >Reporter: Chunwei Lei >Assignee: Chunwei Lei >Priority: Major > Labels: pull-request-available > Fix For: 1.32.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > Many systems depend on column statistics to compute more accurate stats, such > as NDV, average column size, and so on. It would be nice if Calcite can > provide such an interface. > Column statistics might include NDV, average/max column length, number of > nulls, number of trues, number of falses and so on. > What do you think? > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join
[
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17571209#comment-17571209
]
Chunwei Lei commented on CALCITE-3890:
--
Thank you for your attention, [~zabetak]. IMHO, the most important point about
how to implement this rule is how to avoid applying the rule infinitely, which
is the big difference between the new rule I added and the one used in Hive.
To achieve this goal, Hive uses some extra data structures to save the
generated predicates, while the new rule I added just uses MetadataQuery and
RexSimplify to see whether the ISNOTNULL predicate is redundant or not.
> Derive IS NOT NULL filter for the inputs of inner join
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may
> not be null. For instance,
>
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null/b.id is not null and push down them into the
> child node of the join. Then it becomes
> {code:java}
> select * from (select* from a where id is null) t1 join (select * from b
> where id is not null) on t1.id = t2.id;
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-4223) Introducing column statistics to RelOptTable
[ https://issues.apache.org/jira/browse/CALCITE-4223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17567773#comment-17567773 ] Chunwei Lei commented on CALCITE-4223: -- I would like to merge the PR in the next 48hours if no objections appear. > Introducing column statistics to RelOptTable > > > Key: CALCITE-4223 > URL: https://issues.apache.org/jira/browse/CALCITE-4223 > Project: Calcite > Issue Type: Improvement >Reporter: Chunwei Lei >Assignee: Chunwei Lei >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > Many systems depend on column statistics to compute more accurate stats, such > as NDV, average column size, and so on. It would be nice if Calcite can > provide such an interface. > Column statistics might include NDV, average/max column length, number of > nulls, number of trues, number of falses and so on. > What do you think? > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-4223) Introducing column statistics to RelOptTable
[ https://issues.apache.org/jira/browse/CALCITE-4223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17567773#comment-17567773 ] Chunwei Lei edited comment on CALCITE-4223 at 7/18/22 2:08 AM: --- I would like to merge the PR in the next 48 hours if no objections appear. was (Author: chunwei lei): I would like to merge the PR in the next 48hours if no objections appear. > Introducing column statistics to RelOptTable > > > Key: CALCITE-4223 > URL: https://issues.apache.org/jira/browse/CALCITE-4223 > Project: Calcite > Issue Type: Improvement >Reporter: Chunwei Lei >Assignee: Chunwei Lei >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > Many systems depend on column statistics to compute more accurate stats, such > as NDV, average column size, and so on. It would be nice if Calcite can > provide such an interface. > Column statistics might include NDV, average/max column length, number of > nulls, number of trues, number of falses and so on. > What do you think? > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-4223) Introducing column statistics to RelOptTable
[ https://issues.apache.org/jira/browse/CALCITE-4223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17562492#comment-17562492 ] Chunwei Lei commented on CALCITE-4223: -- Could you please review the PR, [~julianhyde]? > Introducing column statistics to RelOptTable > > > Key: CALCITE-4223 > URL: https://issues.apache.org/jira/browse/CALCITE-4223 > Project: Calcite > Issue Type: Improvement >Reporter: Chunwei Lei >Assignee: Chunwei Lei >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > Many systems depend on column statistics to compute more accurate stats, such > as NDV, average column size, and so on. It would be nice if Calcite can > provide such an interface. > Column statistics might include NDV, average/max column length, number of > nulls, number of trues, number of falses and so on. > What do you think? > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-4223) Introducing column statistics to RelOptTable
[ https://issues.apache.org/jira/browse/CALCITE-4223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17562018#comment-17562018 ] Chunwei Lei commented on CALCITE-4223: -- Thank you for your reply, [~asolimando]. I would like to continue this work. Hope it can help. > Introducing column statistics to RelOptTable > > > Key: CALCITE-4223 > URL: https://issues.apache.org/jira/browse/CALCITE-4223 > Project: Calcite > Issue Type: Improvement >Reporter: Chunwei Lei >Assignee: Chunwei Lei >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Many systems depend on column statistics to compute more accurate stats, such > as NDV, average column size, and so on. It would be nice if Calcite can > provide such an interface. > Column statistics might include NDV, average/max column length, number of > nulls, number of trues, number of falses and so on. > What do you think? > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-4223) Introducing column statistics to RelOptTable
Title: Message Title Chunwei Lei commented on CALCITE-4223 Re: Introducing column statistics to RelOptTable It has been a long time since the last discussion. Recently I have time to move it forward. After I reviewed all discussions above, I opened a new PR as Julian suggested: https://github.com/apache/calcite/pull/2845. I am not sure whether it is the best way to introduce the column stats. So I would like to see what others think about it, especially those involved in the downstream projects. Add Comment This message was sent by Atlassian Jira (v8.20.10#820010-sha1:ace47f9)
[jira] [Resolved] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join
[
https://issues.apache.org/jira/browse/CALCITE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei resolved CALCITE-5195.
--
Resolution: Fixed
Fixed in
https://github.com/apache/calcite/commit/300f5200fe50870da418366613df1cd09921b1b6.
> ArrayIndexOutOfBoundsException when inferring more equal conditions from join
> condition for semi join
> -
>
> Key: CALCITE-5195
> URL: https://issues.apache.org/jira/browse/CALCITE-5195
> Project: Calcite
> Issue Type: Bug
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> The following test can reproduce the exception.
> {code:java}
> @Test void testJoinConditionPushdown4() {
> final Function relFn = b -> {
> RelNode left = b.scan("EMP")
> .project(
> b.field("DEPTNO"),
> b.field("ENAME"))
> .build();
> RelNode right = b.scan("DEPT")
> .project(
> b.field("DEPTNO"),
> b.field("DNAME"))
> .build();
> b.push(left).push(right);
> RexInputRef ref1 = b.field(2, 0, "DEPTNO");
> RexInputRef ref2 = b.field(2, 1, "DEPTNO");
> RexInputRef ref3 = b.field(2, 1, "DNAME");
> RexCall cond1 = (RexCall) b.equals(ref1, ref2);
> RexCall cond2 = (RexCall) b.equals(ref1, ref3);
> RexNode cond = b.and(cond1, cond2);
> return b.semiJoin(cond)
> .project(b.field(0))
> .build();
> };
> relFn(relFn)
> .withRule(
> CoreRules.JOIN_PUSH_EXPRESSIONS,
> CoreRules.JOIN_CONDITION_PUSH,
> CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
> CoreRules.JOIN_REDUCE_EXPRESSIONS,
> CoreRules.FILTER_REDUCE_EXPRESSIONS)
> .check();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join
[
https://issues.apache.org/jira/browse/CALCITE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17558951#comment-17558951
]
Chunwei Lei commented on CALCITE-5195:
--
Go ahead, [~libenchao]! You can open another JIRA for it.
> ArrayIndexOutOfBoundsException when inferring more equal conditions from join
> condition for semi join
> -
>
> Key: CALCITE-5195
> URL: https://issues.apache.org/jira/browse/CALCITE-5195
> Project: Calcite
> Issue Type: Bug
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> The following test can reproduce the exception.
> {code:java}
> @Test void testJoinConditionPushdown4() {
> final Function relFn = b -> {
> RelNode left = b.scan("EMP")
> .project(
> b.field("DEPTNO"),
> b.field("ENAME"))
> .build();
> RelNode right = b.scan("DEPT")
> .project(
> b.field("DEPTNO"),
> b.field("DNAME"))
> .build();
> b.push(left).push(right);
> RexInputRef ref1 = b.field(2, 0, "DEPTNO");
> RexInputRef ref2 = b.field(2, 1, "DEPTNO");
> RexInputRef ref3 = b.field(2, 1, "DNAME");
> RexCall cond1 = (RexCall) b.equals(ref1, ref2);
> RexCall cond2 = (RexCall) b.equals(ref1, ref3);
> RexNode cond = b.and(cond1, cond2);
> return b.semiJoin(cond)
> .project(b.field(0))
> .build();
> };
> relFn(relFn)
> .withRule(
> CoreRules.JOIN_PUSH_EXPRESSIONS,
> CoreRules.JOIN_CONDITION_PUSH,
> CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
> CoreRules.JOIN_REDUCE_EXPRESSIONS,
> CoreRules.FILTER_REDUCE_EXPRESSIONS)
> .check();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join
[
https://issues.apache.org/jira/browse/CALCITE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17558283#comment-17558283
]
Chunwei Lei edited comment on CALCITE-5195 at 6/27/22 2:25 AM:
---
Thanks for your reply,[~libenchao]. I tried the following sql:
{code:sql}
select *
from emp
where exists(select * from dept where emp.deptno = dept.deptno and emp.empno =
dept.deptno)
{code}
. But the produced plan isn't what I want:
{code:java}
LogicalProject(DEPTNO=[$7])
LogicalFilter(condition=[IS NOT NULL($11)])
LogicalJoin(condition=[AND(=($0, $9), =($7, $10))], joinType=[left])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
LogicalAggregate(group=[{0, 1}], agg#0=[MIN($2)])
LogicalProject(DEPTNO=[$0], DEPTNO2=[$0], $f0=[true])
LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
{code}
It seems difficult to use a sql to produce a semi join with join conditions
like {{A.x = B.y and A.x = B.z}}.
was (Author: chunwei lei):
Thanks for your reply,[~libenchao]. I tried the following sql:
{code:sql}
select *
from emp
where exists(select * from dept where emp.deptno = dept.deptno and emp.empno =
dept.deptno)
{code}
. But the produced plan isn't what I want:
{code:java}
LogicalProject(DEPTNO=[$7])
LogicalFilter(condition=[IS NOT NULL($11)])
LogicalJoin(condition=[AND(=($0, $9), =($7, $10))], joinType=[left])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
LogicalAggregate(group=[{0, 1}], agg#0=[MIN($2)])
LogicalProject(DEPTNO=[$0], DEPTNO2=[$0], $f0=[true])
LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
{code}
It seems difficult to use a sql to produce a semi join with join conditions
like {{A.x = B.y and A.x = B.z}.
> ArrayIndexOutOfBoundsException when inferring more equal conditions from join
> condition for semi join
> -
>
> Key: CALCITE-5195
> URL: https://issues.apache.org/jira/browse/CALCITE-5195
> Project: Calcite
> Issue Type: Bug
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> The following test can reproduce the exception.
> {code:java}
> @Test void testJoinConditionPushdown4() {
> final Function relFn = b -> {
> RelNode left = b.scan("EMP")
> .project(
> b.field("DEPTNO"),
> b.field("ENAME"))
> .build();
> RelNode right = b.scan("DEPT")
> .project(
> b.field("DEPTNO"),
> b.field("DNAME"))
> .build();
> b.push(left).push(right);
> RexInputRef ref1 = b.field(2, 0, "DEPTNO");
> RexInputRef ref2 = b.field(2, 1, "DEPTNO");
> RexInputRef ref3 = b.field(2, 1, "DNAME");
> RexCall cond1 = (RexCall) b.equals(ref1, ref2);
> RexCall cond2 = (RexCall) b.equals(ref1, ref3);
> RexNode cond = b.and(cond1, cond2);
> return b.semiJoin(cond)
> .project(b.field(0))
> .build();
> };
> relFn(relFn)
> .withRule(
> CoreRules.JOIN_PUSH_EXPRESSIONS,
> CoreRules.JOIN_CONDITION_PUSH,
> CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
> CoreRules.JOIN_REDUCE_EXPRESSIONS,
> CoreRules.FILTER_REDUCE_EXPRESSIONS)
> .check();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join
[
https://issues.apache.org/jira/browse/CALCITE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17558283#comment-17558283
]
Chunwei Lei commented on CALCITE-5195:
--
Thanks for your reply,[~libenchao]. I tried the following sql:
{code:sql}
select *
from emp
where exists(select * from dept where emp.deptno = dept.deptno and emp.empno =
dept.deptno)
{code}
. But the produced plan isn't what I want:
{code:java}
LogicalProject(DEPTNO=[$7])
LogicalFilter(condition=[IS NOT NULL($11)])
LogicalJoin(condition=[AND(=($0, $9), =($7, $10))], joinType=[left])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
LogicalAggregate(group=[{0, 1}], agg#0=[MIN($2)])
LogicalProject(DEPTNO=[$0], DEPTNO2=[$0], $f0=[true])
LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
{code}
It seems difficult to use a sql to produce a semi join with join conditions
like {{A.x = B.y and A.x = B.z}.
> ArrayIndexOutOfBoundsException when inferring more equal conditions from join
> condition for semi join
> -
>
> Key: CALCITE-5195
> URL: https://issues.apache.org/jira/browse/CALCITE-5195
> Project: Calcite
> Issue Type: Bug
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> The following test can reproduce the exception.
> {code:java}
> @Test void testJoinConditionPushdown4() {
> final Function relFn = b -> {
> RelNode left = b.scan("EMP")
> .project(
> b.field("DEPTNO"),
> b.field("ENAME"))
> .build();
> RelNode right = b.scan("DEPT")
> .project(
> b.field("DEPTNO"),
> b.field("DNAME"))
> .build();
> b.push(left).push(right);
> RexInputRef ref1 = b.field(2, 0, "DEPTNO");
> RexInputRef ref2 = b.field(2, 1, "DEPTNO");
> RexInputRef ref3 = b.field(2, 1, "DNAME");
> RexCall cond1 = (RexCall) b.equals(ref1, ref2);
> RexCall cond2 = (RexCall) b.equals(ref1, ref3);
> RexNode cond = b.and(cond1, cond2);
> return b.semiJoin(cond)
> .project(b.field(0))
> .build();
> };
> relFn(relFn)
> .withRule(
> CoreRules.JOIN_PUSH_EXPRESSIONS,
> CoreRules.JOIN_CONDITION_PUSH,
> CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
> CoreRules.JOIN_REDUCE_EXPRESSIONS,
> CoreRules.FILTER_REDUCE_EXPRESSIONS)
> .check();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join
[
https://issues.apache.org/jira/browse/CALCITE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17557766#comment-17557766
]
Chunwei Lei edited comment on CALCITE-5195 at 6/23/22 2:08 AM:
---
The following sql also can reproduce the bug.
{code:sql}
select *
from emp e semi join dept d
on e.deptno = d.deptno and e.empno = d.deptno
{code}
But it threw an exception due to the keyword {{semi}} can't be parsed
correctly. Therefore, I had to use RelBuilder to create such sql to reproduce
the bug. Is there a way to support such sql?
was (Author: chunwei lei):
The following sql also can reproduce the bug.
{code:sql}
select *
from emp e semi join dept d
on e.deptno = d.deptno and e.empno = d.deptno
{code}
But it threw an exception due to the keyword {{semi}} can't be parsed
correctly. Therefore, I had to use RelBuilder to create such sql to reproduce
the bug.
> ArrayIndexOutOfBoundsException when inferring more equal conditions from join
> condition for semi join
> -
>
> Key: CALCITE-5195
> URL: https://issues.apache.org/jira/browse/CALCITE-5195
> Project: Calcite
> Issue Type: Bug
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> The following test can reproduce the exception.
> {code:java}
> @Test void testJoinConditionPushdown4() {
> final Function relFn = b -> {
> RelNode left = b.scan("EMP")
> .project(
> b.field("DEPTNO"),
> b.field("ENAME"))
> .build();
> RelNode right = b.scan("DEPT")
> .project(
> b.field("DEPTNO"),
> b.field("DNAME"))
> .build();
> b.push(left).push(right);
> RexInputRef ref1 = b.field(2, 0, "DEPTNO");
> RexInputRef ref2 = b.field(2, 1, "DEPTNO");
> RexInputRef ref3 = b.field(2, 1, "DNAME");
> RexCall cond1 = (RexCall) b.equals(ref1, ref2);
> RexCall cond2 = (RexCall) b.equals(ref1, ref3);
> RexNode cond = b.and(cond1, cond2);
> return b.semiJoin(cond)
> .project(b.field(0))
> .build();
> };
> relFn(relFn)
> .withRule(
> CoreRules.JOIN_PUSH_EXPRESSIONS,
> CoreRules.JOIN_CONDITION_PUSH,
> CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
> CoreRules.JOIN_REDUCE_EXPRESSIONS,
> CoreRules.FILTER_REDUCE_EXPRESSIONS)
> .check();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join
[
https://issues.apache.org/jira/browse/CALCITE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17557766#comment-17557766
]
Chunwei Lei commented on CALCITE-5195:
--
The following sql also can reproduce the bug.
{code:sql}
select *
from emp e semi join dept d
on e.deptno = d.deptno and e.empno = d.deptno
{code}
But it threw an exception due to the keyword {{semi}} can't be parsed
correctly. Therefore, I had to use RelBuilder to create such sql to reproduce
the bug.
> ArrayIndexOutOfBoundsException when inferring more equal conditions from join
> condition for semi join
> -
>
> Key: CALCITE-5195
> URL: https://issues.apache.org/jira/browse/CALCITE-5195
> Project: Calcite
> Issue Type: Bug
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> The following test can reproduce the exception.
> {code:java}
> @Test void testJoinConditionPushdown4() {
> final Function relFn = b -> {
> RelNode left = b.scan("EMP")
> .project(
> b.field("DEPTNO"),
> b.field("ENAME"))
> .build();
> RelNode right = b.scan("DEPT")
> .project(
> b.field("DEPTNO"),
> b.field("DNAME"))
> .build();
> b.push(left).push(right);
> RexInputRef ref1 = b.field(2, 0, "DEPTNO");
> RexInputRef ref2 = b.field(2, 1, "DEPTNO");
> RexInputRef ref3 = b.field(2, 1, "DNAME");
> RexCall cond1 = (RexCall) b.equals(ref1, ref2);
> RexCall cond2 = (RexCall) b.equals(ref1, ref3);
> RexNode cond = b.and(cond1, cond2);
> return b.semiJoin(cond)
> .project(b.field(0))
> .build();
> };
> relFn(relFn)
> .withRule(
> CoreRules.JOIN_PUSH_EXPRESSIONS,
> CoreRules.JOIN_CONDITION_PUSH,
> CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
> CoreRules.JOIN_REDUCE_EXPRESSIONS,
> CoreRules.FILTER_REDUCE_EXPRESSIONS)
> .check();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join
[
https://issues.apache.org/jira/browse/CALCITE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17557222#comment-17557222
]
Chunwei Lei commented on CALCITE-5195:
--
PR: https://github.com/apache/calcite/pull/2837. Could you please review it,
[~libenchao]?
> ArrayIndexOutOfBoundsException when inferring more equal conditions from join
> condition for semi join
> -
>
> Key: CALCITE-5195
> URL: https://issues.apache.org/jira/browse/CALCITE-5195
> Project: Calcite
> Issue Type: Bug
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Fix For: 1.31.0
>
>
> The following test can reproduce the exception.
> {code:java}
> @Test void testJoinConditionPushdown4() {
> final Function relFn = b -> {
> RelNode left = b.scan("EMP")
> .project(
> b.field("DEPTNO"),
> b.field("ENAME"))
> .build();
> RelNode right = b.scan("DEPT")
> .project(
> b.field("DEPTNO"),
> b.field("DNAME"))
> .build();
> b.push(left).push(right);
> RexInputRef ref1 = b.field(2, 0, "DEPTNO");
> RexInputRef ref2 = b.field(2, 1, "DEPTNO");
> RexInputRef ref3 = b.field(2, 1, "DNAME");
> RexCall cond1 = (RexCall) b.equals(ref1, ref2);
> RexCall cond2 = (RexCall) b.equals(ref1, ref3);
> RexNode cond = b.and(cond1, cond2);
> return b.semiJoin(cond)
> .project(b.field(0))
> .build();
> };
> relFn(relFn)
> .withRule(
> CoreRules.JOIN_PUSH_EXPRESSIONS,
> CoreRules.JOIN_CONDITION_PUSH,
> CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
> CoreRules.JOIN_REDUCE_EXPRESSIONS,
> CoreRules.FILTER_REDUCE_EXPRESSIONS)
> .check();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join
[
https://issues.apache.org/jira/browse/CALCITE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17557218#comment-17557218
]
Chunwei Lei commented on CALCITE-5195:
--
I plan to open a PR today so that it can be included in 1.31.
> ArrayIndexOutOfBoundsException when inferring more equal conditions from join
> condition for semi join
> -
>
> Key: CALCITE-5195
> URL: https://issues.apache.org/jira/browse/CALCITE-5195
> Project: Calcite
> Issue Type: Bug
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Fix For: 1.31.0
>
>
> The following test can reproduce the exception.
> {code:java}
> @Test void testJoinConditionPushdown4() {
> final Function relFn = b -> {
> RelNode left = b.scan("EMP")
> .project(
> b.field("DEPTNO"),
> b.field("ENAME"))
> .build();
> RelNode right = b.scan("DEPT")
> .project(
> b.field("DEPTNO"),
> b.field("DNAME"))
> .build();
> b.push(left).push(right);
> RexInputRef ref1 = b.field(2, 0, "DEPTNO");
> RexInputRef ref2 = b.field(2, 1, "DEPTNO");
> RexInputRef ref3 = b.field(2, 1, "DNAME");
> RexCall cond1 = (RexCall) b.equals(ref1, ref2);
> RexCall cond2 = (RexCall) b.equals(ref1, ref3);
> RexNode cond = b.and(cond1, cond2);
> return b.semiJoin(cond)
> .project(b.field(0))
> .build();
> };
> relFn(relFn)
> .withRule(
> CoreRules.JOIN_PUSH_EXPRESSIONS,
> CoreRules.JOIN_CONDITION_PUSH,
> CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
> CoreRules.JOIN_REDUCE_EXPRESSIONS,
> CoreRules.FILTER_REDUCE_EXPRESSIONS)
> .check();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join
[
https://issues.apache.org/jira/browse/CALCITE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei updated CALCITE-5195:
-
Fix Version/s: 1.31.0
> ArrayIndexOutOfBoundsException when inferring more equal conditions from join
> condition for semi join
> -
>
> Key: CALCITE-5195
> URL: https://issues.apache.org/jira/browse/CALCITE-5195
> Project: Calcite
> Issue Type: Bug
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Fix For: 1.31.0
>
>
> The following test can reproduce the exception.
> {code:java}
> @Test void testJoinConditionPushdown4() {
> final Function relFn = b -> {
> RelNode left = b.scan("EMP")
> .project(
> b.field("DEPTNO"),
> b.field("ENAME"))
> .build();
> RelNode right = b.scan("DEPT")
> .project(
> b.field("DEPTNO"),
> b.field("DNAME"))
> .build();
> b.push(left).push(right);
> RexInputRef ref1 = b.field(2, 0, "DEPTNO");
> RexInputRef ref2 = b.field(2, 1, "DEPTNO");
> RexInputRef ref3 = b.field(2, 1, "DNAME");
> RexCall cond1 = (RexCall) b.equals(ref1, ref2);
> RexCall cond2 = (RexCall) b.equals(ref1, ref3);
> RexNode cond = b.and(cond1, cond2);
> return b.semiJoin(cond)
> .project(b.field(0))
> .build();
> };
> relFn(relFn)
> .withRule(
> CoreRules.JOIN_PUSH_EXPRESSIONS,
> CoreRules.JOIN_CONDITION_PUSH,
> CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
> CoreRules.JOIN_REDUCE_EXPRESSIONS,
> CoreRules.FILTER_REDUCE_EXPRESSIONS)
> .check();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join
Chunwei Lei created CALCITE-5195:
Summary: ArrayIndexOutOfBoundsException when inferring more equal
conditions from join condition for semi join
Key: CALCITE-5195
URL: https://issues.apache.org/jira/browse/CALCITE-5195
Project: Calcite
Issue Type: Bug
Reporter: Chunwei Lei
Assignee: Chunwei Lei
The following test can reproduce the exception.
{code:java}
@Test void testJoinConditionPushdown4() {
final Function relFn = b -> {
RelNode left = b.scan("EMP")
.project(
b.field("DEPTNO"),
b.field("ENAME"))
.build();
RelNode right = b.scan("DEPT")
.project(
b.field("DEPTNO"),
b.field("DNAME"))
.build();
b.push(left).push(right);
RexInputRef ref1 = b.field(2, 0, "DEPTNO");
RexInputRef ref2 = b.field(2, 1, "DEPTNO");
RexInputRef ref3 = b.field(2, 1, "DNAME");
RexCall cond1 = (RexCall) b.equals(ref1, ref2);
RexCall cond2 = (RexCall) b.equals(ref1, ref3);
RexNode cond = b.and(cond1, cond2);
return b.semiJoin(cond)
.project(b.field(0))
.build();
};
relFn(relFn)
.withRule(
CoreRules.JOIN_PUSH_EXPRESSIONS,
CoreRules.JOIN_CONDITION_PUSH,
CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
CoreRules.JOIN_REDUCE_EXPRESSIONS,
CoreRules.FILTER_REDUCE_EXPRESSIONS)
.check();
}
{code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (CALCITE-5191) Allow sort by alias in BigQuery
[
https://issues.apache.org/jira/browse/CALCITE-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556160#comment-17556160
]
Chunwei Lei edited comment on CALCITE-5191 at 6/20/22 2:23 AM:
---
Fixed in
https://github.com/apache/calcite/commit/fa65a2ec460abcd574e7f28672478a0510da65a1.
Thank you for your PR, [~wenruimeng]!
was (Author: chunwei lei):
Fixed in
https://github.com/apache/calcite/commit/bbdef5389908121ee67ba3a7381266ab586f36bb.
Thank you for your PR, [~wenruimeng]!
> Allow sort by alias in BigQuery
> ---
>
> Key: CALCITE-5191
> URL: https://issues.apache.org/jira/browse/CALCITE-5191
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Wenrui Meng
>Assignee: Wenrui Meng
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> select 1 as a order by a {code}
> the above statement is valid in BigQuery, but failed in current validation
> flow due to allow alias not enabled for the BigQuery conformance.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Resolved] (CALCITE-5191) Allow sort by alias in BigQuery
[
https://issues.apache.org/jira/browse/CALCITE-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei resolved CALCITE-5191.
--
Resolution: Fixed
Fixed in
https://github.com/apache/calcite/commit/bbdef5389908121ee67ba3a7381266ab586f36bb.
Thank you for your PR, [~wenruimeng]!
> Allow sort by alias in BigQuery
> ---
>
> Key: CALCITE-5191
> URL: https://issues.apache.org/jira/browse/CALCITE-5191
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Wenrui Meng
>Assignee: Wenrui Meng
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code:java}
> select 1 as a order by a {code}
> the above statement is valid in BigQuery, but failed in current validation
> flow due to allow alias not enabled for the BigQuery conformance.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (CALCITE-5193) Push filter whose conditions include join keys and are composed by OR into inputs of full join
[
https://issues.apache.org/jira/browse/CALCITE-5193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554917#comment-17554917
]
Chunwei Lei edited comment on CALCITE-5193 at 6/16/22 6:53 AM:
---
[~libenchao] yes, you're right. I changed the description.
was (Author: chunwei lei):
[~libenchao]yes, you're right. I changed the description.
> Push filter whose conditions include join keys and are composed by OR into
> inputs of full join
> --
>
> Key: CALCITE-5193
> URL: https://issues.apache.org/jira/browse/CALCITE-5193
> Project: Calcite
> Issue Type: Improvement
>Reporter: Chunwei Lei
>Priority: Major
>
> For example,
> {code:sql}
> select * from a full join b on a.id=b.id where a.id=1 or b.id=2;
> {code}
> can be transformed to
> {code:sql}
> select * from
> (select * from a where id=1 or id=2) a
> full join
> (select * from b where id=1 or id=2) b
> on a.id=b.id;
> {code}
> If {{a}} and {{b}} are both partitioned tables and id is the partition key,
> we can do partition pruning with this transformation, which is a big
> improvement.
> This improvement is inspired by the query
> {code:java}
> select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt,
> b.pt)='20220601';
> {code}
> which costs a lot due to it scans all partitions in table {{a}} and {{b}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-5193) Push filter whose conditions include join keys and are composed by OR into inputs of full join
[
https://issues.apache.org/jira/browse/CALCITE-5193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei updated CALCITE-5193:
-
Description:
For example,
{code:sql}
select * from a full join b on a.id=b.id where a.id=1 or b.id=2;
{code}
can be transformed to
{code:sql}
select * from
(select * from a where id=1 or id=2) a
full join
(select * from b where id=1 or id=2) b
on a.id=b.id;
{code}
If {{a}} and {{b}} are both partitioned tables and id is the partition key, we
can do partition pruning with this transformation, which is a big improvement.
This improvement is inspired by the query
{code:java}
select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt,
b.pt)='20220601';
{code}
which costs a lot due to it scans all partitions in table {{a}} and {{b}}.
was:
For example,
{code:sql}
select * from a full join b on a.id=b.id where a.id=1 or b.id=2;
{code}
can be transformed to
{code:java}
select * from
(select * from a where id=1 or id=2) a
full join
(select * from b where id=1 or id=2) b
on a.id=b.id;
{code}
If {{a}} and {{b}} are both partitioned tables and id is the partition key, we
can do partition pruning with this transformation, which is a big improvement.
This improvement is inspired by the query
{code:java}
select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt,
b.pt)='20220601';
{code}
which costs a lot due to it scans all partitions in table {{a}} and {{b}}.
> Push filter whose conditions include join keys and are composed by OR into
> inputs of full join
> --
>
> Key: CALCITE-5193
> URL: https://issues.apache.org/jira/browse/CALCITE-5193
> Project: Calcite
> Issue Type: Improvement
>Reporter: Chunwei Lei
>Priority: Major
>
> For example,
> {code:sql}
> select * from a full join b on a.id=b.id where a.id=1 or b.id=2;
> {code}
> can be transformed to
> {code:sql}
> select * from
> (select * from a where id=1 or id=2) a
> full join
> (select * from b where id=1 or id=2) b
> on a.id=b.id;
> {code}
> If {{a}} and {{b}} are both partitioned tables and id is the partition key,
> we can do partition pruning with this transformation, which is a big
> improvement.
> This improvement is inspired by the query
> {code:java}
> select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt,
> b.pt)='20220601';
> {code}
> which costs a lot due to it scans all partitions in table {{a}} and {{b}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5193) Push filter whose conditions include join keys and are composed by OR into inputs of full join
[
https://issues.apache.org/jira/browse/CALCITE-5193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554917#comment-17554917
]
Chunwei Lei commented on CALCITE-5193:
--
[~libenchao]yes, you're right. I changed the description.
> Push filter whose conditions include join keys and are composed by OR into
> inputs of full join
> --
>
> Key: CALCITE-5193
> URL: https://issues.apache.org/jira/browse/CALCITE-5193
> Project: Calcite
> Issue Type: Improvement
>Reporter: Chunwei Lei
>Priority: Major
>
> For example,
> {code:java}
> select * from a full join b on a.id=b.id where a.id=1 or b.id=2
> {code}
> can be transformed to
> {code:java}
> select * from
> (select * from a where id=1) a
> full join
> (select * from b where id=2) b
> on a.id=b.id;
> {code}
> If {{a}} and {{b}} are both partitioned tables and id is the partition key,
> we can do partition pruning with this transformation, which is a big
> improvement.
> This improvement is inspired by the query
> {code:java}
> select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt,
> b.pt)='20220601';
> {code}
> which costs a lot due to it scans all partitions in table {{a}} and {{b}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-5193) Push filter whose conditions include join keys and are composed by OR into inputs of full join
[
https://issues.apache.org/jira/browse/CALCITE-5193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei updated CALCITE-5193:
-
Description:
For example,
{code:sql}
select * from a full join b on a.id=b.id where a.id=1 or b.id=2;
{code}
can be transformed to
{code:java}
select * from
(select * from a where id=1 or id=2) a
full join
(select * from b where id=1 or id=2) b
on a.id=b.id;
{code}
If {{a}} and {{b}} are both partitioned tables and id is the partition key, we
can do partition pruning with this transformation, which is a big improvement.
This improvement is inspired by the query
{code:java}
select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt,
b.pt)='20220601';
{code}
which costs a lot due to it scans all partitions in table {{a}} and {{b}}.
was:
For example,
{code:java}
select * from a full join b on a.id=b.id where a.id=1 or b.id=2
{code}
can be transformed to
{code:java}
select * from
(select * from a where id=1) a
full join
(select * from b where id=2) b
on a.id=b.id;
{code}
If {{a}} and {{b}} are both partitioned tables and id is the partition key, we
can do partition pruning with this transformation, which is a big improvement.
This improvement is inspired by the query
{code:java}
select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt,
b.pt)='20220601';
{code}
which costs a lot due to it scans all partitions in table {{a}} and {{b}}.
> Push filter whose conditions include join keys and are composed by OR into
> inputs of full join
> --
>
> Key: CALCITE-5193
> URL: https://issues.apache.org/jira/browse/CALCITE-5193
> Project: Calcite
> Issue Type: Improvement
>Reporter: Chunwei Lei
>Priority: Major
>
> For example,
> {code:sql}
> select * from a full join b on a.id=b.id where a.id=1 or b.id=2;
> {code}
> can be transformed to
> {code:java}
> select * from
> (select * from a where id=1 or id=2) a
> full join
> (select * from b where id=1 or id=2) b
> on a.id=b.id;
> {code}
> If {{a}} and {{b}} are both partitioned tables and id is the partition key,
> we can do partition pruning with this transformation, which is a big
> improvement.
> This improvement is inspired by the query
> {code:java}
> select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt,
> b.pt)='20220601';
> {code}
> which costs a lot due to it scans all partitions in table {{a}} and {{b}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-5193) Push filter whose conditions include join keys and are composed by OR into inputs of full join
[
https://issues.apache.org/jira/browse/CALCITE-5193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei updated CALCITE-5193:
-
Description:
For example,
{code:java}
select * from a full join b on a.id=b.id where a.id=1 or b.id=2
{code}
can be transformed to
{code:java}
select * from
(select * from a where id=1) a
full join
(select * from b where id=2) b
on a.id=b.id;
{code}
If {{a}} and {{b}} are both partitioned tables and id is the partition key, we
can do partition pruning with this transformation, which is a big improvement.
This improvement is inspired by the query
{code:java}
select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt,
b.pt)='20220601';
{code}
which costs a lot due to it scans all partitions in table {{a}} and {{b}}.
was:
For example,
{code:java}
select * from a full join b on a.id=b.id where a.id=1 or b.id=2
{code}
can be transformed to
{code:java}
select * from
(select * from a where id=1) a
full join
(select * from b where id=2) b
on a.id=b.id;
{code}
If {{a}} and {{b}} are both partitioned tables and id is the partition key, we
can do partition pruning with this transformation, which is a big improvement.
This improvement is inspired by query
{code:java}
select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt,
b.pt)='20220601';
{code}
which costs a lot due to it scans all partitions in table a and b.
> Push filter whose conditions include join keys and are composed by OR into
> inputs of full join
> --
>
> Key: CALCITE-5193
> URL: https://issues.apache.org/jira/browse/CALCITE-5193
> Project: Calcite
> Issue Type: Improvement
>Reporter: Chunwei Lei
>Priority: Major
>
> For example,
> {code:java}
> select * from a full join b on a.id=b.id where a.id=1 or b.id=2
> {code}
> can be transformed to
> {code:java}
> select * from
> (select * from a where id=1) a
> full join
> (select * from b where id=2) b
> on a.id=b.id;
> {code}
> If {{a}} and {{b}} are both partitioned tables and id is the partition key,
> we can do partition pruning with this transformation, which is a big
> improvement.
> This improvement is inspired by the query
> {code:java}
> select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt,
> b.pt)='20220601';
> {code}
> which costs a lot due to it scans all partitions in table {{a}} and {{b}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (CALCITE-5193) Push filter whose conditions include join keys and are composed by OR into inputs of full join
Chunwei Lei created CALCITE-5193:
Summary: Push filter whose conditions include join keys and are
composed by OR into inputs of full join
Key: CALCITE-5193
URL: https://issues.apache.org/jira/browse/CALCITE-5193
Project: Calcite
Issue Type: Improvement
Reporter: Chunwei Lei
For example,
{code:java}
select * from a full join b on a.id=b.id where a.id=1 or b.id=2
{code}
can be transformed to
{code:java}
select * from
(select * from a where id=1) a
full join
(select * from b where id=2) b
on a.id=b.id;
{code}
If {{a}} and {{b}} are both partitioned tables and id is the partition key, we
can do partition pruning with this transformation, which is a big improvement.
This improvement is inspired by query
{code:java}
select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt,
b.pt)='20220601';
{code}
which costs a lot due to it scans all partitions in table a and b.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5191) Allow sort by alias in BigQuery
[
https://issues.apache.org/jira/browse/CALCITE-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554341#comment-17554341
]
Chunwei Lei commented on CALCITE-5191:
--
The doc[1] has said that BigQuery supports alias in the ORDER BY clause.
[1]https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax#order_by_clause
> Allow sort by alias in BigQuery
> ---
>
> Key: CALCITE-5191
> URL: https://issues.apache.org/jira/browse/CALCITE-5191
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Wenrui Meng
>Assignee: Wenrui Meng
>Priority: Major
> Fix For: 1.31.0
>
>
> {code:java}
> select 1 as a order by a {code}
> the above statement is valid in BigQuery, but failed in current validation
> flow due to allow alias not enabled for the BigQuery conformance.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (CALCITE-5191) Allow sort by alias in BigQuery
[
https://issues.apache.org/jira/browse/CALCITE-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554341#comment-17554341
]
Chunwei Lei edited comment on CALCITE-5191 at 6/15/22 1:51 AM:
---
The doc[1] has said that BigQuery supports alias in the ORDER BY clause.
[1]
https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax#order_by_clause
was (Author: chunwei lei):
The doc[1] has said that BigQuery supports alias in the ORDER BY clause.
[1]https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax#order_by_clause
> Allow sort by alias in BigQuery
> ---
>
> Key: CALCITE-5191
> URL: https://issues.apache.org/jira/browse/CALCITE-5191
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Wenrui Meng
>Assignee: Wenrui Meng
>Priority: Major
> Fix For: 1.31.0
>
>
> {code:java}
> select 1 as a order by a {code}
> the above statement is valid in BigQuery, but failed in current validation
> flow due to allow alias not enabled for the BigQuery conformance.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (CALCITE-5169) 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
[
https://issues.apache.org/jira/browse/CALCITE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554339#comment-17554339
]
Chunwei Lei edited comment on CALCITE-5169 at 6/15/22 1:44 AM:
---
Fixed in
[https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b].
Thank you for your PR, [~libenchao]!
was (Author: chunwei lei):
Fixed in
[https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b]
. Thank you for your PR, [~libenchao]!
> 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
>
>
> Key: CALCITE-5169
> URL: https://issues.apache.org/jira/browse/CALCITE-5169
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> As described in the title, {{xx < 1 OR xx > 1}} should be simplified to {{xx
> <> 1}}.
> See discussion in ML:
> https://lists.apache.org/thread/hsdgx0jb3214v6k9p7slbkk3xsbl0b7f
> The original problem is that {{ename <> '' and ename <> '3'}} could be
> simplified to {{Sarg}} while {{(ename < '' or ename > '') and (ename < '3' or
> ename > '3')}} cannot.
> Since {{<}} and {{<>}} have different {{SqlOperandTypeChecker.Consistency}},
> the type inference result is different for these two cases. We suspect that
> the type inference result leads to the difference of simplification.
> However, the real reason is that {{xx < 1 OR xx > 1}} cannot be simplified to
> {{xx <> 1}} in simplification, which prevents {{(ename < '' or ename > '')
> and (ename < '3' or ename > '3')}} being simplified to {{ename <> '' and
> ename <> '3'}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (CALCITE-5169) 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
[
https://issues.apache.org/jira/browse/CALCITE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554339#comment-17554339
]
Chunwei Lei edited comment on CALCITE-5169 at 6/15/22 1:44 AM:
---
Fixed in
[https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b]
. Thank you for your PR, [~libenchao]!
was (Author: chunwei lei):
Fixed in
https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b.Thank
you for your PR, [~libenchao]!
> 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
>
>
> Key: CALCITE-5169
> URL: https://issues.apache.org/jira/browse/CALCITE-5169
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> As described in the title, {{xx < 1 OR xx > 1}} should be simplified to {{xx
> <> 1}}.
> See discussion in ML:
> https://lists.apache.org/thread/hsdgx0jb3214v6k9p7slbkk3xsbl0b7f
> The original problem is that {{ename <> '' and ename <> '3'}} could be
> simplified to {{Sarg}} while {{(ename < '' or ename > '') and (ename < '3' or
> ename > '3')}} cannot.
> Since {{<}} and {{<>}} have different {{SqlOperandTypeChecker.Consistency}},
> the type inference result is different for these two cases. We suspect that
> the type inference result leads to the difference of simplification.
> However, the real reason is that {{xx < 1 OR xx > 1}} cannot be simplified to
> {{xx <> 1}} in simplification, which prevents {{(ename < '' or ename > '')
> and (ename < '3' or ename > '3')}} being simplified to {{ename <> '' and
> ename <> '3'}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (CALCITE-5169) 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
[
https://issues.apache.org/jira/browse/CALCITE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554339#comment-17554339
]
Chunwei Lei edited comment on CALCITE-5169 at 6/15/22 1:43 AM:
---
Fixed in
https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b.Thank
you for your PR, [~libenchao]!
was (Author: chunwei lei):
Fixed in
[https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b|https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b.].
Thank you for your PR, [~libenchao]!
> 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
>
>
> Key: CALCITE-5169
> URL: https://issues.apache.org/jira/browse/CALCITE-5169
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> As described in the title, {{xx < 1 OR xx > 1}} should be simplified to {{xx
> <> 1}}.
> See discussion in ML:
> https://lists.apache.org/thread/hsdgx0jb3214v6k9p7slbkk3xsbl0b7f
> The original problem is that {{ename <> '' and ename <> '3'}} could be
> simplified to {{Sarg}} while {{(ename < '' or ename > '') and (ename < '3' or
> ename > '3')}} cannot.
> Since {{<}} and {{<>}} have different {{SqlOperandTypeChecker.Consistency}},
> the type inference result is different for these two cases. We suspect that
> the type inference result leads to the difference of simplification.
> However, the real reason is that {{xx < 1 OR xx > 1}} cannot be simplified to
> {{xx <> 1}} in simplification, which prevents {{(ename < '' or ename > '')
> and (ename < '3' or ename > '3')}} being simplified to {{ename <> '' and
> ename <> '3'}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Resolved] (CALCITE-5169) 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
[
https://issues.apache.org/jira/browse/CALCITE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei resolved CALCITE-5169.
--
Resolution: Fixed
Fixed in
[https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b|https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b.].
Thank you for your PR, [~libenchao]!
> 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
>
>
> Key: CALCITE-5169
> URL: https://issues.apache.org/jira/browse/CALCITE-5169
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> As described in the title, {{xx < 1 OR xx > 1}} should be simplified to {{xx
> <> 1}}.
> See discussion in ML:
> https://lists.apache.org/thread/hsdgx0jb3214v6k9p7slbkk3xsbl0b7f
> The original problem is that {{ename <> '' and ename <> '3'}} could be
> simplified to {{Sarg}} while {{(ename < '' or ename > '') and (ename < '3' or
> ename > '3')}} cannot.
> Since {{<}} and {{<>}} have different {{SqlOperandTypeChecker.Consistency}},
> the type inference result is different for these two cases. We suspect that
> the type inference result leads to the difference of simplification.
> However, the real reason is that {{xx < 1 OR xx > 1}} cannot be simplified to
> {{xx <> 1}} in simplification, which prevents {{(ename < '' or ename > '')
> and (ename < '3' or ename > '3')}} being simplified to {{ename <> '' and
> ename <> '3'}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (CALCITE-5169) 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
[
https://issues.apache.org/jira/browse/CALCITE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554339#comment-17554339
]
Chunwei Lei edited comment on CALCITE-5169 at 6/15/22 1:42 AM:
---
Fixed in
[https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b|https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b.].
Thank you for your PR, [~libenchao]!
was (Author: chunwei lei):
Fixed in
[https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b|https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b.].
Thank you for your PR, [~libenchao]!
> 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
>
>
> Key: CALCITE-5169
> URL: https://issues.apache.org/jira/browse/CALCITE-5169
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> As described in the title, {{xx < 1 OR xx > 1}} should be simplified to {{xx
> <> 1}}.
> See discussion in ML:
> https://lists.apache.org/thread/hsdgx0jb3214v6k9p7slbkk3xsbl0b7f
> The original problem is that {{ename <> '' and ename <> '3'}} could be
> simplified to {{Sarg}} while {{(ename < '' or ename > '') and (ename < '3' or
> ename > '3')}} cannot.
> Since {{<}} and {{<>}} have different {{SqlOperandTypeChecker.Consistency}},
> the type inference result is different for these two cases. We suspect that
> the type inference result leads to the difference of simplification.
> However, the real reason is that {{xx < 1 OR xx > 1}} cannot be simplified to
> {{xx <> 1}} in simplification, which prevents {{(ename < '' or ename > '')
> and (ename < '3' or ename > '3')}} being simplified to {{ename <> '' and
> ename <> '3'}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5191) Allow sort by alias in BigQuery
[
https://issues.apache.org/jira/browse/CALCITE-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554337#comment-17554337
]
Chunwei Lei commented on CALCITE-5191:
--
Good catch. The PR looks good to me.
> Allow sort by alias in BigQuery
> ---
>
> Key: CALCITE-5191
> URL: https://issues.apache.org/jira/browse/CALCITE-5191
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Wenrui Meng
>Assignee: Wenrui Meng
>Priority: Major
> Fix For: 1.31.0
>
>
> {code:java}
> select 1 as a order by a {code}
> the above statement is valid in BigQuery, but failed in current validation
> flow due to allow alias not enabled for the BigQuery conformance.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5169) 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
[
https://issues.apache.org/jira/browse/CALCITE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17553366#comment-17553366
]
Chunwei Lei commented on CALCITE-5169:
--
The PR looks good. I will merge the PR shortly.
> 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
>
>
> Key: CALCITE-5169
> URL: https://issues.apache.org/jira/browse/CALCITE-5169
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> As described in the title, {{xx < 1 OR xx > 1}} should be simplified to {{xx
> <> 1}}.
> See discussion in ML:
> https://lists.apache.org/thread/hsdgx0jb3214v6k9p7slbkk3xsbl0b7f
> The original problem is that {{ename <> '' and ename <> '3'}} could be
> simplified to {{Sarg}} while {{(ename < '' or ename > '') and (ename < '3' or
> ename > '3')}} cannot.
> Since {{<}} and {{<>}} have different {{SqlOperandTypeChecker.Consistency}},
> the type inference result is different for these two cases. We suspect that
> the type inference result leads to the difference of simplification.
> However, the real reason is that {{xx < 1 OR xx > 1}} cannot be simplified to
> {{xx <> 1}} in simplification, which prevents {{(ename < '' or ename > '')
> and (ename < '3' or ename > '3')}} being simplified to {{ename <> '' and
> ename <> '3'}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (CALCITE-5169) 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
[
https://issues.apache.org/jira/browse/CALCITE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17551352#comment-17551352
]
Chunwei Lei edited comment on CALCITE-5169 at 6/8/22 2:18 AM:
--
[~julianhyde], could you please double confirm the PR is the right way to fix
the bug?
was (Author: chunwei lei):
Could you please double confirm the PR is the right way to fix the bug,
[~julianhyde]?
> 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
>
>
> Key: CALCITE-5169
> URL: https://issues.apache.org/jira/browse/CALCITE-5169
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> As described in the title, {{xx < 1 OR xx > 1}} should be simplified to {{xx
> <> 1}}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5169) 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
[
https://issues.apache.org/jira/browse/CALCITE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17551352#comment-17551352
]
Chunwei Lei commented on CALCITE-5169:
--
Could you please double confirm the PR is the right way to fix the bug,
[~julianhyde]?
> 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
>
>
> Key: CALCITE-5169
> URL: https://issues.apache.org/jira/browse/CALCITE-5169
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> As described in the title, {{xx < 1 OR xx > 1}} should be simplified to {{xx
> <> 1}}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5179) In RelToSqlConverter, AssertionError for values with more than two items when SqlDialect#supportsAliasedValues is false
[ https://issues.apache.org/jira/browse/CALCITE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17551350#comment-17551350 ] Chunwei Lei commented on CALCITE-5179: -- I have put the label "LGTM-will-merge-soon". I will merge it shortly if no objections appear. > In RelToSqlConverter, AssertionError for values with more than two items > when SqlDialect#supportsAliasedValues is false > > > Key: CALCITE-5179 > URL: https://issues.apache.org/jira/browse/CALCITE-5179 > Project: Calcite > Issue Type: Bug >Reporter: Steven Talbot >Assignee: Jiajun Xie >Priority: Major > Labels: pull-request-available > Fix For: 1.31.0 > > Time Spent: 20m > Remaining Estimate: 0h > > The code at > [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/rel/rel2sql/RelToSqlConverter.java#L802-L803] > creates a UNION ALL with an arbitrary number of operands, but [the UNION_ALL > operator is a > SqlSetOperator|https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/fun/SqlStdOperatorTable.java#L108-L109] > which is a "SqlBinaryOperator" (Binary syntax). So that generates an > assertion error when it unparses more than two values at > [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/SqlUtil.java#L426] > > I don't _think_ I've done anything nonstandard, other than use a dialect with > that flag set to false. To reproduce, create a values RelNode with more than > two items, convert it with RelToSqlConverter with a dialect using > supportsAliasedValues() = false (like Redshift), and attempt to unparse the > resulting SqlNode. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (CALCITE-5150) Parser should parse subquery with order by inside array constructor
[
https://issues.apache.org/jira/browse/CALCITE-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei resolved CALCITE-5150.
--
Resolution: Fixed
Fixed in
[https://github.com/apache/calcite/commit/4a4114419b0341fa4887b4838195ef7855c00e22|https://github.com/apache/calcite/commit/4a4114419b0341fa4887b4838195ef7855c00e22.].
Thank you for your PR, [~dmsysolyatin]!
> Parser should parse subquery with order by inside array constructor
> ---
>
> Key: CALCITE-5150
> URL: https://issues.apache.org/jira/browse/CALCITE-5150
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Dmitry Sysolyatin
>Assignee: Dmitry Sysolyatin
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Parser can't parse query that looks like:
> {code}
> SELECT array(select x from (VALUES(1)) x ORDER BY x)
> {code}
> {code}
> Caused by: org.apache.calcite.sql.parser.babel.ParseException: Encountered
> "ORDER" at line 1, column 42.
> Was expecting one of:
> ")" ...
> "NATURAL" ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (CALCITE-5179) AssertionError in values with more than two items when SqlDialect#supportsAliasedValues is false
[ https://issues.apache.org/jira/browse/CALCITE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17550261#comment-17550261 ] Chunwei Lei edited comment on CALCITE-5179 at 6/6/22 2:11 AM: -- IMOP, it would be great if the title can show that the error only appears when converting rel node to sql. was (Author: chunwei lei): IMOP, it would be great if the title can specify the error that only appears when converting rel node to sql. > AssertionError in values with more than two items when > SqlDialect#supportsAliasedValues is false > > > Key: CALCITE-5179 > URL: https://issues.apache.org/jira/browse/CALCITE-5179 > Project: Calcite > Issue Type: Bug >Reporter: Steven Talbot >Assignee: Jiajun Xie >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The code at > [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/rel/rel2sql/RelToSqlConverter.java#L802-L803] > creates a UNION ALL with an arbitrary number of operands, but [the UNION_ALL > operator is a > SqlSetOperator|https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/fun/SqlStdOperatorTable.java#L108-L109] > which is a "SqlBinaryOperator" (Binary syntax). So that generates an > assertion error when it unparses more than two values at > [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/SqlUtil.java#L426] > > I don't _think_ I've done anything nonstandard, other than use a dialect with > that flag set to false. To reproduce, create a values RelNode with more than > two items, convert it with RelToSqlConverter with a dialect using > supportsAliasedValues() = false (like Redshift), and attempt to unparse the > resulting SqlNode. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Comment Edited] (CALCITE-5179) AssertionError in values with more than two items when SqlDialect#supportsAliasedValues is false
[ https://issues.apache.org/jira/browse/CALCITE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17550261#comment-17550261 ] Chunwei Lei edited comment on CALCITE-5179 at 6/6/22 2:09 AM: -- IMOP, it would be great if the title can specify the error that only appears when converting rel node to sql. was (Author: chunwei lei): IMOP, it would be great if the title can specify the error appears when converting rel node to sql. > AssertionError in values with more than two items when > SqlDialect#supportsAliasedValues is false > > > Key: CALCITE-5179 > URL: https://issues.apache.org/jira/browse/CALCITE-5179 > Project: Calcite > Issue Type: Bug >Reporter: Steven Talbot >Assignee: Jiajun Xie >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The code at > [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/rel/rel2sql/RelToSqlConverter.java#L802-L803] > creates a UNION ALL with an arbitrary number of operands, but [the UNION_ALL > operator is a > SqlSetOperator|https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/fun/SqlStdOperatorTable.java#L108-L109] > which is a "SqlBinaryOperator" (Binary syntax). So that generates an > assertion error when it unparses more than two values at > [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/SqlUtil.java#L426] > > I don't _think_ I've done anything nonstandard, other than use a dialect with > that flag set to false. To reproduce, create a values RelNode with more than > two items, convert it with RelToSqlConverter with a dialect using > supportsAliasedValues() = false (like Redshift), and attempt to unparse the > resulting SqlNode. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Comment Edited] (CALCITE-5179) AssertionError in values with more than two items when SqlDialect#supportsAliasedValues is false
[ https://issues.apache.org/jira/browse/CALCITE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17550261#comment-17550261 ] Chunwei Lei edited comment on CALCITE-5179 at 6/6/22 2:07 AM: -- IMOP, it would be great if the title can specify the error appears when converting rel node to sql. was (Author: chunwei lei): IMOP, it would be great if the error appears when converting rel node to sql. > AssertionError in values with more than two items when > SqlDialect#supportsAliasedValues is false > > > Key: CALCITE-5179 > URL: https://issues.apache.org/jira/browse/CALCITE-5179 > Project: Calcite > Issue Type: Bug >Reporter: Steven Talbot >Assignee: Jiajun Xie >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The code at > [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/rel/rel2sql/RelToSqlConverter.java#L802-L803] > creates a UNION ALL with an arbitrary number of operands, but [the UNION_ALL > operator is a > SqlSetOperator|https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/fun/SqlStdOperatorTable.java#L108-L109] > which is a "SqlBinaryOperator" (Binary syntax). So that generates an > assertion error when it unparses more than two values at > [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/SqlUtil.java#L426] > > I don't _think_ I've done anything nonstandard, other than use a dialect with > that flag set to false. To reproduce, create a values RelNode with more than > two items, convert it with RelToSqlConverter with a dialect using > supportsAliasedValues() = false (like Redshift), and attempt to unparse the > resulting SqlNode. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (CALCITE-5179) AssertionError in values with more than two items when SqlDialect#supportsAliasedValues is false
[ https://issues.apache.org/jira/browse/CALCITE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17550261#comment-17550261 ] Chunwei Lei commented on CALCITE-5179: -- IMOP, it would be great if the error appears when converting rel node to sql. > AssertionError in values with more than two items when > SqlDialect#supportsAliasedValues is false > > > Key: CALCITE-5179 > URL: https://issues.apache.org/jira/browse/CALCITE-5179 > Project: Calcite > Issue Type: Bug >Reporter: Steven Talbot >Assignee: Jiajun Xie >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The code at > [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/rel/rel2sql/RelToSqlConverter.java#L802-L803] > creates a UNION ALL with an arbitrary number of operands, but [the UNION_ALL > operator is a > SqlSetOperator|https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/fun/SqlStdOperatorTable.java#L108-L109] > which is a "SqlBinaryOperator" (Binary syntax). So that generates an > assertion error when it unparses more than two values at > [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/SqlUtil.java#L426] > > I don't _think_ I've done anything nonstandard, other than use a dialect with > that flag set to false. To reproduce, create a values RelNode with more than > two items, convert it with RelToSqlConverter with a dialect using > supportsAliasedValues() = false (like Redshift), and attempt to unparse the > resulting SqlNode. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (CALCITE-5170) Assertion error on range distribution creation
[
https://issues.apache.org/jira/browse/CALCITE-5170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei resolved CALCITE-5170.
--
Resolution: Fixed
Fixed in
[https://github.com/apache/calcite/commit/a5af80642787e3f1d24671d3a6339568d0c6b201|https://github.com/apache/calcite/commit/a5af80642787e3f1d24671d3a6339568d0c6b201.].
Thank you for your PR, [~rkondakov]!
> Assertion error on range distribution creation
> --
>
> Key: CALCITE-5170
> URL: https://issues.apache.org/jira/browse/CALCITE-5170
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.30.0
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> An assertion error occurs on range distribution creation. This test fails:
> {code:java}
> @Test void testRangeRelDistributionKeys() {
> RelDistributions.range(Arrays.asList(0, 1));
> }
> {code}
> This happens because there is an incorrect assertion in
> {{RelDistributionImpl}} constructor:
> {code:java}
> assert type == Type.HASH_DISTRIBUTED
> || type == Type.RANDOM_DISTRIBUTED
> || keys.isEmpty();
> {code}
> It should be {{RANGE_DISTRIBUTED}} instead of {{RANDOM_DISTRIBUTED}}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5157) ClassCastException in checkRollUp with DOT operator
[
https://issues.apache.org/jira/browse/CALCITE-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17544091#comment-17544091
]
Chunwei Lei commented on CALCITE-5157:
--
I am not sure whether the proposed fix is the best way. Let's see others'
points.
> ClassCastException in checkRollUp with DOT operator
> ---
>
> Key: CALCITE-5157
> URL: https://issues.apache.org/jira/browse/CALCITE-5157
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.30.0
>Reporter: Viliam Durina
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When a query contains nested field access and is using parentheses to
> disambiguate the identifier, the {{SqlValidatorImpl.checkRollup()}} method
> throws a {{ClassCastException}}, assuming that the input of the DOT operator
> is a {{SqlCall}}. I think this assumption is wrong, the DOT operator
> typically has {{SqlIdentifier}} as an input, probably also other classes.
> Here's the stack trace:
> {noformat}
> java.lang.ClassCastException: class org.apache.calcite.sql.SqlIdentifier
> cannot be cast to class org.apache.calcite.sql.SqlCall
> (org.apache.calcite.sql.SqlIdentifier and org.apache.calcite.sql.SqlCall are
> in unnamed module of loader 'app')
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3730)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3749)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUpInSelectList(SqlValidatorImpl.java:3673)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3661)
> at
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)
> at
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1100)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1071)
> at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:247)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1046)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:752)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:587)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:257)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:648)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:514)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:484)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:234)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:623)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:677)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> ... 67 more
> {noformat}
> The problem can be reproduced by modifying the
> {{ReflectiveSchemaTest.testSelectWithFieldAccessOnFirstLevelRecordType()}}
> test and putting {{au."birthPlace"}} into parentheses:
> {code:sql}
> select (au."birthPlace")."city" as city from ...
> {code}
> Putting identifiers into parentheses is common to disambiguate field access
> from identifier qualification. For example, if the {{au}} prefix is removed
> from the query in the {{testSelectWithFieldAccessOnFirstLevelRecordType}}
> test, it will fail with {{{}Table 'birthPlace' not found{}}}. But if we put
> {{\"birthPlace\"}} into parentheses, then it is correctly recognized as a
> column of the {{authors}} table.
> I'm not sure about the correct fix to this issue which would not break the
> roll-up functionality, perhaps its author [~zhumayun] can help reviewing the
> PR or suggesting another fix. I'll soon create a PR with a proposed fix.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5150) Parser should parse subquery with order by inside array constructor
[
https://issues.apache.org/jira/browse/CALCITE-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17544092#comment-17544092
]
Chunwei Lei commented on CALCITE-5150:
--
Get it! The PR looks good to me.
> Parser should parse subquery with order by inside array constructor
> ---
>
> Key: CALCITE-5150
> URL: https://issues.apache.org/jira/browse/CALCITE-5150
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Dmitry Sysolyatin
>Assignee: Dmitry Sysolyatin
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Parser can't parse query that looks like:
> {code}
> SELECT array(select x from (VALUES(1)) x ORDER BY x)
> {code}
> {code}
> Caused by: org.apache.calcite.sql.parser.babel.ParseException: Encountered
> "ORDER" at line 1, column 42.
> Was expecting one of:
> ")" ...
> "NATURAL" ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5170) Assertion error on range distribution creation
[
https://issues.apache.org/jira/browse/CALCITE-5170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17543741#comment-17543741
]
Chunwei Lei commented on CALCITE-5170:
--
Good catch! The PR looks good.
> Assertion error on range distribution creation
> --
>
> Key: CALCITE-5170
> URL: https://issues.apache.org/jira/browse/CALCITE-5170
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.30.0
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
>
> An assertion error occurs on range distribution creation. This test fails:
> {code:java}
> @Test void testRangeRelDistributionKeys() {
> RelDistributions.range(Arrays.asList(0, 1));
> }
> {code}
> This happens because there is an incorrect assertion in
> {{RelDistributionImpl}} constructor:
> {code:java}
> assert type == Type.HASH_DISTRIBUTED
> || type == Type.RANDOM_DISTRIBUTED
> || keys.isEmpty();
> {code}
> It should be {{RANGE_DISTRIBUTED}} instead of {{RANDOM_DISTRIBUTED}}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Resolved] (CALCITE-5162) RelMdUniqueKeys can return more precise unique keys for Aggregate
[
https://issues.apache.org/jira/browse/CALCITE-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei resolved CALCITE-5162.
--
Fix Version/s: 1.31.0
Resolution: Fixed
Fixed in
https://github.com/apache/calcite/commit/f278efb0411ba29c7bd167f5d02e566bf542acb3.
> RelMdUniqueKeys can return more precise unique keys for Aggregate
> -
>
> Key: CALCITE-5162
> URL: https://issues.apache.org/jira/browse/CALCITE-5162
> Project: Calcite
> Issue Type: Improvement
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, RelMdUniqueKeys always returns group by keys as the unique key.
> However, it can return more precise unique keys by looking through group by
> keys. For instance:
> {code:java}
> select deptno, deptname, count(*) from dept group by deptno, deptname;{code}
> RelMdUniqueKeys can return {{deptno}} as the unique key instead of (deptno,
> deptname) if {{deptno}} is unique.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Resolved] (CALCITE-5073) JoinConditionPushRule cannot infer 'LHS.C1 = LHS.C2' from 'LHS.C1 = RHS.C1 AND LHS.C2 = RHS.C1'
[
https://issues.apache.org/jira/browse/CALCITE-5073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei resolved CALCITE-5073.
--
Resolution: Fixed
Fixed in
[https://github.com/apache/calcite/commit/4969b9690efe999c522daf1151c4a00a33be0eb0|https://github.com/apache/calcite/commit/4969b9690efe999c522daf1151c4a00a33be0eb0.].
Thank you for your PR, [~libenchao]!
> JoinConditionPushRule cannot infer 'LHS.C1 = LHS.C2' from 'LHS.C1 = RHS.C1
> AND LHS.C2 = RHS.C1'
> ---
>
> Key: CALCITE-5073
> URL: https://issues.apache.org/jira/browse/CALCITE-5073
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> For the below SQL:
> {code:java}
> SELECT * FROM T1, T2, T3
> WHERE T1.id = T3.id AND T2.id = T3.id{code}
> Currently, JoinConditionPushRule cannot infer the condition 'T1.id = T2.id'
> for the first join relation.
> There is another rule {{JoinPushThroughJoinRule}}, it can handle the above
> case in a different way (by reordering joins).
> This can be a more general optimization, which like the title says, we can
> infer more conditions from join condition (or above filter condition) to the
> Join's operands.
> It's reported in ML:
> https://lists.apache.org/thread/6fs09cgyz5rzrty90632ywoo477q0gdk
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Resolved] (CALCITE-5163) MysqlSqlDialect unparse LISTAGG aggregate function error
[
https://issues.apache.org/jira/browse/CALCITE-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei resolved CALCITE-5163.
--
Fix Version/s: 1.31.0
Resolution: Fixed
Fixed in
[https://github.com/apache/calcite/commit/52c9b7836bbed90038f28a933c0cb56d2d69cd28|https://github.com/apache/calcite/commit/52c9b7836bbed90038f28a933c0cb56d2d69cd28.].
Thank you for your PR, [~wojustme]!
> MysqlSqlDialect unparse LISTAGG aggregate function error
>
>
> Key: CALCITE-5163
> URL: https://issues.apache.org/jira/browse/CALCITE-5163
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Xurenhe
>Assignee: Xurenhe
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> CALCITE-4349 supports GROUP_CONCAT aggregate function. [The
> code|https://github.com/apache/calcite/blob/7c1e2746b94a50f0bb08571287acb5327fdc1b16/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L5746]
> translated *GROUP_CONCAT* to *LISTAGG* during sql_to_rel.
> But, *MysqlSqlDialect* keep *LISTAGG* after uparsing query's relnode, the
> result sql couldn't be executed in MySQL engine.
>
> I expect the following test case to be successful, but it's failed.
> {code:java}
> // test in org.apache.calcite.rel.rel2sql.RelToSqlConverterTest
> @Test void testMySqlGroupConcat() {
> final String query = "select\n"
> + "listagg(distinct \"product_name\", ',') within group(order by
> \"cases_per_pallet\")"
> + "from \"product\"\n"
> + "group by \"product_id\"\n";
> final String expected = "SELECT GROUP_CONCAT(DISTINCT `product_name` ORDER
> BY "
> + "`cases_per_pallet` IS NULL, `cases_per_pallet` SEPARATOR ',')\n"
> + "FROM `foodmart`.`product`\n"
> + "GROUP BY `product_id`";
> sql(query).withMysql().ok(expected);
> }{code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Resolved] (CALCITE-5166) Method accept(RelShuttle) is not overridden in LogicalCalc and LogicalTableModify
[
https://issues.apache.org/jira/browse/CALCITE-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei resolved CALCITE-5166.
--
Resolution: Fixed
Fixed in
[https://github.com/apache/calcite/commit/9bdfd9a178f493b235d8785afd94fd0c998e8cce|https://github.com/apache/calcite/commit/9bdfd9a178f493b235d8785afd94fd0c998e8cce.].
Thank you for your PR, [~rkondakov]!
> Method accept(RelShuttle) is not overridden in LogicalCalc and
> LogicalTableModify
> -
>
> Key: CALCITE-5166
> URL: https://issues.apache.org/jira/browse/CALCITE-5166
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.30.0
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
> Labels: pull-request-available, pull_request_available
> Fix For: 1.31.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Method {{RelNode#accept(RelShuttle)}} is not overridden for {{LogicalCalc}}
> and {{{}LogicalTableModify{}}}. This leads to the bug when logic implemented
> in {{RelShuttle#visit(LogicalCalc)}} is never applied because {{visit()}}
> method is never called due to incorrect dispatching.
> This test will fail without {{accept(RelShuttle)}} method is overridden in
> Calc:
> {code:java}
> @Test void testRelShuttleForLogicalCalc() {
> final String sql = "select ename from emp";
> final RelNode rel = sql(sql).toRel();
> final HepProgramBuilder programBuilder = HepProgram.builder();
> programBuilder.addRuleInstance(CoreRules.PROJECT_TO_CALC);
> final HepPlanner planner = new HepPlanner(programBuilder.build());
> planner.setRoot(rel);
> final RelNode calc = planner.findBestExp();
> final List rels = new ArrayList<>();
> final RelShuttleImpl visitor = new RelShuttleImpl() {
> @Override public RelNode visit(LogicalCalc calc) {
> RelNode visitedRel = super.visit(calc);
> rels.add(visitedRel);
> return visitedRel;
> }
> };
> calc.accept(visitor);
> assertThat(rels.size(), is(1));
> assertThat(rels.get(0), isA(LogicalCalc.class));
> } {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5163) MysqlSqlDialect unparse LISTAGG aggregate function error
[
https://issues.apache.org/jira/browse/CALCITE-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541238#comment-17541238
]
Chunwei Lei commented on CALCITE-5163:
--
Looks basically good. I left some minor comments.
> MysqlSqlDialect unparse LISTAGG aggregate function error
>
>
> Key: CALCITE-5163
> URL: https://issues.apache.org/jira/browse/CALCITE-5163
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Xurenhe
>Assignee: Xurenhe
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> CALCITE-4349 supports GROUP_CONCAT aggregate function. [The
> code|https://github.com/apache/calcite/blob/7c1e2746b94a50f0bb08571287acb5327fdc1b16/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L5746]
> translated *GROUP_CONCAT* to *LISTAGG* during sql_to_rel.
> But, *MysqlSqlDialect* keep *LISTAGG* after uparsing query's relnode, the
> result sql couldn't be executed in MySQL engine.
>
> I expect the following test case to be successful, but it's failed.
> {code:java}
> // test in org.apache.calcite.rel.rel2sql.RelToSqlConverterTest
> @Test void testMySqlGroupConcat() {
> final String query = "select\n"
> + "listagg(distinct \"product_name\", ',') within group(order by
> \"cases_per_pallet\")"
> + "from \"product\"\n"
> + "group by \"product_id\"\n";
> final String expected = "SELECT GROUP_CONCAT(DISTINCT `product_name` ORDER
> BY "
> + "`cases_per_pallet` IS NULL, `cases_per_pallet` SEPARATOR ',')\n"
> + "FROM `foodmart`.`product`\n"
> + "GROUP BY `product_id`";
> sql(query).withMysql().ok(expected);
> }{code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5166) Method accept(RelShuttle) is not overridden in LogicalCalc and LogicalTableModify
[
https://issues.apache.org/jira/browse/CALCITE-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17540725#comment-17540725
]
Chunwei Lei commented on CALCITE-5166:
--
Makes sense. There're some operators that have the same issue, such as
{{{}TableSpool{}}}, {{{}TableFunctionScan{}}}. Could you please check whether
they need to override the method too?
> Method accept(RelShuttle) is not overridden in LogicalCalc and
> LogicalTableModify
> -
>
> Key: CALCITE-5166
> URL: https://issues.apache.org/jira/browse/CALCITE-5166
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.30.0
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
> Labels: pull_request_available
> Fix For: 1.31.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Method {{RelNode#accept(RelShuttle)}} is not overridden for {{LogicalCalc}}
> and {{{}LogicalTableModify{}}}. This leads to the bug when logic implemented
> in {{RelShuttle#visit(LogicalCalc)}} is never applied because {{visit()}}
> method is never called due to incorrect dispatching.
> This test will fail without {{accept(RelShuttle)}} method is overridden in
> Calc:
> {code:java}
> @Test void testRelShuttleForLogicalCalc() {
> final String sql = "select ename from emp";
> final RelNode rel = sql(sql).toRel();
> final HepProgramBuilder programBuilder = HepProgram.builder();
> programBuilder.addRuleInstance(CoreRules.PROJECT_TO_CALC);
> final HepPlanner planner = new HepPlanner(programBuilder.build());
> planner.setRoot(rel);
> final RelNode calc = planner.findBestExp();
> final List rels = new ArrayList<>();
> final RelShuttleImpl visitor = new RelShuttleImpl() {
> @Override public RelNode visit(LogicalCalc calc) {
> RelNode visitedRel = super.visit(calc);
> rels.add(visitedRel);
> return visitedRel;
> }
> };
> calc.accept(visitor);
> assertThat(rels.size(), is(1));
> assertThat(rels.get(0), isA(LogicalCalc.class));
> } {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Resolved] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join
[
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei resolved CALCITE-3890.
--
Resolution: Fixed
Fixed in
https://github.com/apache/calcite/commit/acf82f7784823c30fb7a64e905c3acacd0ed4f2b.
> Derive IS NOT NULL filter for the inputs of inner join
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may
> not be null. For instance,
>
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null/b.id is not null and push down them into the
> child node of the join. Then it becomes
> {code:java}
> select * from (select* from a where id is null) t1 join (select * from b
> where id is not null) on t1.id = t2.id;
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (CALCITE-5162) RelMdUniqueKeys can return more precise unique keys for Aggregate
Chunwei Lei created CALCITE-5162:
Summary: RelMdUniqueKeys can return more precise unique keys for
Aggregate
Key: CALCITE-5162
URL: https://issues.apache.org/jira/browse/CALCITE-5162
Project: Calcite
Issue Type: Improvement
Reporter: Chunwei Lei
Assignee: Chunwei Lei
Currently, RelMdUniqueKeys always returns group by keys as the unique key.
However, it can return more precise unique keys by looking through group by
keys. For instance:
{code:java}
select deptno, deptname, count(*) from dept group by deptno, deptname;{code}
RelMdUniqueKeys can return {{deptno}} as the unique key instead of (deptno,
deptname) if {{deptno}} is unique.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5073) JoinConditionPushRule cannot infer 'LHS.C1 = LHS.C2' from 'LHS.C1 = RHS.C1 AND LHS.C2 = RHS.C1'
[
https://issues.apache.org/jira/browse/CALCITE-5073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539190#comment-17539190
]
Chunwei Lei commented on CALCITE-5073:
--
I am +1 for this positive change. I will find some time to review the PR this
week.
> JoinConditionPushRule cannot infer 'LHS.C1 = LHS.C2' from 'LHS.C1 = RHS.C1
> AND LHS.C2 = RHS.C1'
> ---
>
> Key: CALCITE-5073
> URL: https://issues.apache.org/jira/browse/CALCITE-5073
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> For the below SQL:
> {code:java}
> SELECT * FROM T1, T2, T3
> WHERE T1.id = T3.id AND T2.id = T3.id{code}
> Currently, JoinConditionPushRule cannot infer the condition 'T1.id = T2.id'
> for the first join relation.
> There is another rule {{JoinPushThroughJoinRule}}, it can handle the above
> case in a different way (by reordering joins).
> This can be a more general optimization, which like the title says, we can
> infer more conditions from join condition (or above filter condition) to the
> Join's operands.
> It's reported in ML:
> https://lists.apache.org/thread/6fs09cgyz5rzrty90632ywoo477q0gdk
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (CALCITE-5157) ClassCastException in checkRollUp with DOT operator
[
https://issues.apache.org/jira/browse/CALCITE-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539174#comment-17539174
]
Chunwei Lei edited comment on CALCITE-5157 at 5/19/22 2:51 AM:
---
I did some investigation on this. It seems `authors`.`birthPlace`.`city' is
recognized as a {{SqlCall}} rather than {{{}SqlIdentifier{}}}, which leads to
the error. Therefore, I wonder if the right way is to fix the wrong
recognization.
was (Author: chunwei lei):
I did some investigation on this. It seems {{`authors`.`birthPlace`.`city'
}}{{is recognized as a SqlCall}} rather than {{{}SqlIdentifier{}}}, which leads
to the error. Therefore, I wonder if the right way is to fix the wrong
recognization.
> ClassCastException in checkRollUp with DOT operator
> ---
>
> Key: CALCITE-5157
> URL: https://issues.apache.org/jira/browse/CALCITE-5157
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.30.0
>Reporter: Viliam Durina
>Priority: Major
>
> When a query contains nested field access and is using parentheses to
> disambiguate the identifier, the {{SqlValidatorImpl.checkRollup()}} method
> throws a {{{}ClassCastException{}}}, assuming that the input of the DOT
> operator is a {{{}SqlCall{}}}. I think this assumption is wrong, the DOT
> operator typically has {{SqlIdentifier}} as an input, probably also other
> classes.
> Here's the stack trace:
> {{java.lang.ClassCastException: class org.apache.calcite.sql.SqlIdentifier
> cannot be cast to class org.apache.calcite.sql.SqlCall
> (org.apache.calcite.sql.SqlIdentifier and org.apache.calcite.sql.SqlCall are
> in unnamed module of loader 'app')}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3730)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3749)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUpInSelectList(SqlValidatorImpl.java:3673)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3661)}}
> {{ at
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)}}
> {{ at
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1100)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1071)}}
> {{ at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:247)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1046)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:752)}}
> {{ at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:587)}}
> {{ at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:257)}}
> {{ at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)}}
> {{ at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:648)}}
> {{ at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:514)}}
> {{ at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:484)}}
> {{ at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:234)}}
> {{ at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:623)}}
> {{ at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:677)}}
> {{ at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)}}
> {{ ... 67 more}}
> The problem can be reproduced by modifying the
> {{ReflectiveSchemaTest.testSelectWithFieldAccessOnFirstLevelRecordType()}}
> test and putting {{au."birthPlace"}} into parentheses:
> {{select (au."birthPlace")."city" as city from ... }}
> Putting identifiers into parentheses is common to disambiguate field access
> from identifier qualification. For exmaple, if the {{au}} prefix is removed
> from the query in the {{testSelectWithFieldAccessOnFirstLevelRecordType}}
> test, it will fail with {{{}Table 'birthPlace' not found{}}}. But if we put
> {{\"birthPlace\"}} into parentheses, then it is correctly recognized as a
> column of the {{authors}} table.
> I'm not sure about the correct fix to this issue which would not break the
> roll-up functionality, perhaps its author [~zhumayun] can help reviewing the
> PR or suggesting another fix. I'll soon create a PR with a proposed fix.
--
This message was sent by Atlassian Jira
(v8
[jira] [Comment Edited] (CALCITE-5157) ClassCastException in checkRollUp with DOT operator
[
https://issues.apache.org/jira/browse/CALCITE-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539174#comment-17539174
]
Chunwei Lei edited comment on CALCITE-5157 at 5/19/22 2:49 AM:
---
I did some investigation on this. It seems {{`authors`.`birthPlace`.`city'
}}{{is recognized as a SqlCall}} rather than {{{}SqlIdentifier{}}}, which leads
to the error. Therefore, I wonder if the right way is to fix the wrong
recognization.
was (Author: chunwei lei):
I did some investigation on this. It seems {{`authors`.`birthPlace`.`city` }}is
recognized as a {{SqlCall rather than {{{}SqlIdentifier{}}}, which leads to
the error. Therefore, I wonder if the right way is to fix the wrong
recognization.
> ClassCastException in checkRollUp with DOT operator
> ---
>
> Key: CALCITE-5157
> URL: https://issues.apache.org/jira/browse/CALCITE-5157
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.30.0
>Reporter: Viliam Durina
>Priority: Major
>
> When a query contains nested field access and is using parentheses to
> disambiguate the identifier, the {{SqlValidatorImpl.checkRollup()}} method
> throws a {{{}ClassCastException{}}}, assuming that the input of the DOT
> operator is a {{{}SqlCall{}}}. I think this assumption is wrong, the DOT
> operator typically has {{SqlIdentifier}} as an input, probably also other
> classes.
> Here's the stack trace:
> {{java.lang.ClassCastException: class org.apache.calcite.sql.SqlIdentifier
> cannot be cast to class org.apache.calcite.sql.SqlCall
> (org.apache.calcite.sql.SqlIdentifier and org.apache.calcite.sql.SqlCall are
> in unnamed module of loader 'app')}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3730)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3749)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUpInSelectList(SqlValidatorImpl.java:3673)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3661)}}
> {{ at
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)}}
> {{ at
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1100)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1071)}}
> {{ at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:247)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1046)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:752)}}
> {{ at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:587)}}
> {{ at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:257)}}
> {{ at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)}}
> {{ at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:648)}}
> {{ at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:514)}}
> {{ at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:484)}}
> {{ at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:234)}}
> {{ at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:623)}}
> {{ at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:677)}}
> {{ at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)}}
> {{ ... 67 more}}
> The problem can be reproduced by modifying the
> {{ReflectiveSchemaTest.testSelectWithFieldAccessOnFirstLevelRecordType()}}
> test and putting {{au."birthPlace"}} into parentheses:
> {{select (au."birthPlace")."city" as city from ... }}
> Putting identifiers into parentheses is common to disambiguate field access
> from identifier qualification. For exmaple, if the {{au}} prefix is removed
> from the query in the {{testSelectWithFieldAccessOnFirstLevelRecordType}}
> test, it will fail with {{{}Table 'birthPlace' not found{}}}. But if we put
> {{\"birthPlace\"}} into parentheses, then it is correctly recognized as a
> column of the {{authors}} table.
> I'm not sure about the correct fix to this issue which would not break the
> roll-up functionality, perhaps its author [~zhumayun] can help reviewing the
> PR or suggesting another fix. I'll soon create a PR with a proposed fix.
--
This message was sent by Atlassian Ji
[jira] [Comment Edited] (CALCITE-5157) ClassCastException in checkRollUp with DOT operator
[
https://issues.apache.org/jira/browse/CALCITE-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539174#comment-17539174
]
Chunwei Lei edited comment on CALCITE-5157 at 5/19/22 2:49 AM:
---
I did some investigation on this. It seems {{`authors`.`birthPlace`.`city` }}is
recognized as a {{SqlCall rather than {{{}SqlIdentifier{}}}, which leads to
the error. Therefore, I wonder if the right way is to fix the wrong
recognization.
was (Author: chunwei lei):
I did some investigation on this. It seems {{`authors`.`birthPlace`.`city` }}is
recognized as a {{SqlCall}} rather than {{{}SqlIdentifier{}}}, which leads to
the error. Therefore, I wonder if the right way is to fix the wrong
recognization.
> ClassCastException in checkRollUp with DOT operator
> ---
>
> Key: CALCITE-5157
> URL: https://issues.apache.org/jira/browse/CALCITE-5157
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.30.0
>Reporter: Viliam Durina
>Priority: Major
>
> When a query contains nested field access and is using parentheses to
> disambiguate the identifier, the {{SqlValidatorImpl.checkRollup()}} method
> throws a {{{}ClassCastException{}}}, assuming that the input of the DOT
> operator is a {{{}SqlCall{}}}. I think this assumption is wrong, the DOT
> operator typically has {{SqlIdentifier}} as an input, probably also other
> classes.
> Here's the stack trace:
> {{java.lang.ClassCastException: class org.apache.calcite.sql.SqlIdentifier
> cannot be cast to class org.apache.calcite.sql.SqlCall
> (org.apache.calcite.sql.SqlIdentifier and org.apache.calcite.sql.SqlCall are
> in unnamed module of loader 'app')}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3730)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3749)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUpInSelectList(SqlValidatorImpl.java:3673)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3661)}}
> {{ at
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)}}
> {{ at
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1100)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1071)}}
> {{ at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:247)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1046)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:752)}}
> {{ at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:587)}}
> {{ at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:257)}}
> {{ at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)}}
> {{ at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:648)}}
> {{ at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:514)}}
> {{ at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:484)}}
> {{ at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:234)}}
> {{ at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:623)}}
> {{ at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:677)}}
> {{ at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)}}
> {{ ... 67 more}}
> The problem can be reproduced by modifying the
> {{ReflectiveSchemaTest.testSelectWithFieldAccessOnFirstLevelRecordType()}}
> test and putting {{au."birthPlace"}} into parentheses:
> {{select (au."birthPlace")."city" as city from ... }}
> Putting identifiers into parentheses is common to disambiguate field access
> from identifier qualification. For exmaple, if the {{au}} prefix is removed
> from the query in the {{testSelectWithFieldAccessOnFirstLevelRecordType}}
> test, it will fail with {{{}Table 'birthPlace' not found{}}}. But if we put
> {{\"birthPlace\"}} into parentheses, then it is correctly recognized as a
> column of the {{authors}} table.
> I'm not sure about the correct fix to this issue which would not break the
> roll-up functionality, perhaps its author [~zhumayun] can help reviewing the
> PR or suggesting another fix. I'll soon create a PR with a proposed fix.
--
This message was sent by Atlassian Ji
[jira] [Commented] (CALCITE-5157) ClassCastException in checkRollUp with DOT operator
[
https://issues.apache.org/jira/browse/CALCITE-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539174#comment-17539174
]
Chunwei Lei commented on CALCITE-5157:
--
I did some investigation on this. It seems {{`authors`.`birthPlace`.`city` }}is
recognized as a {{SqlCall}} rather than {{{}SqlIdentifier{}}}, which leads to
the error. Therefore, I wonder if the right way is to fix the wrong
recognization.
> ClassCastException in checkRollUp with DOT operator
> ---
>
> Key: CALCITE-5157
> URL: https://issues.apache.org/jira/browse/CALCITE-5157
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.30.0
>Reporter: Viliam Durina
>Priority: Major
>
> When a query contains nested field access and is using parentheses to
> disambiguate the identifier, the {{SqlValidatorImpl.checkRollup()}} method
> throws a {{{}ClassCastException{}}}, assuming that the input of the DOT
> operator is a {{{}SqlCall{}}}. I think this assumption is wrong, the DOT
> operator typically has {{SqlIdentifier}} as an input, probably also other
> classes.
> Here's the stack trace:
> {{java.lang.ClassCastException: class org.apache.calcite.sql.SqlIdentifier
> cannot be cast to class org.apache.calcite.sql.SqlCall
> (org.apache.calcite.sql.SqlIdentifier and org.apache.calcite.sql.SqlCall are
> in unnamed module of loader 'app')}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3730)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3749)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUpInSelectList(SqlValidatorImpl.java:3673)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3661)}}
> {{ at
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)}}
> {{ at
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1100)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1071)}}
> {{ at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:247)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1046)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:752)}}
> {{ at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:587)}}
> {{ at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:257)}}
> {{ at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)}}
> {{ at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:648)}}
> {{ at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:514)}}
> {{ at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:484)}}
> {{ at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:234)}}
> {{ at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:623)}}
> {{ at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:677)}}
> {{ at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)}}
> {{ ... 67 more}}
> The problem can be reproduced by modifying the
> {{ReflectiveSchemaTest.testSelectWithFieldAccessOnFirstLevelRecordType()}}
> test and putting {{au."birthPlace"}} into parentheses:
> {{select (au."birthPlace")."city" as city from ... }}
> Putting identifiers into parentheses is common to disambiguate field access
> from identifier qualification. For exmaple, if the {{au}} prefix is removed
> from the query in the {{testSelectWithFieldAccessOnFirstLevelRecordType}}
> test, it will fail with {{{}Table 'birthPlace' not found{}}}. But if we put
> {{\"birthPlace\"}} into parentheses, then it is correctly recognized as a
> column of the {{authors}} table.
> I'm not sure about the correct fix to this issue which would not break the
> roll-up functionality, perhaps its author [~zhumayun] can help reviewing the
> PR or suggesting another fix. I'll soon create a PR with a proposed fix.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (CALCITE-5157) ClassCastException in checkRollUp with DOT operator
[
https://issues.apache.org/jira/browse/CALCITE-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539168#comment-17539168
]
Chunwei Lei edited comment on CALCITE-5157 at 5/19/22 2:34 AM:
---
FYI: I tried the following SQL in PostgreSQL and it works well.
{code:java}
postgres=# SELECT ("item")."name" FROM "on_hand" ;
name
fuzzy dice
(1 row){code}
So I think it's a bug.
was (Author: chunwei lei):
FYI: I tried the following SQL in PostgreSQL and it works well.
{code:java}
postgres=# SELECT ("item")."name" FROM "on_hand" ;
name
fuzzy dice
(1 row){code}
> ClassCastException in checkRollUp with DOT operator
> ---
>
> Key: CALCITE-5157
> URL: https://issues.apache.org/jira/browse/CALCITE-5157
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.30.0
>Reporter: Viliam Durina
>Priority: Major
>
> When a query contains nested field access and is using parentheses to
> disambiguate the identifier, the {{SqlValidatorImpl.checkRollup()}} method
> throws a {{{}ClassCastException{}}}, assuming that the input of the DOT
> operator is a {{{}SqlCall{}}}. I think this assumption is wrong, the DOT
> operator typically has {{SqlIdentifier}} as an input, probably also other
> classes.
> Here's the stack trace:
> {{java.lang.ClassCastException: class org.apache.calcite.sql.SqlIdentifier
> cannot be cast to class org.apache.calcite.sql.SqlCall
> (org.apache.calcite.sql.SqlIdentifier and org.apache.calcite.sql.SqlCall are
> in unnamed module of loader 'app')}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3730)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3749)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUpInSelectList(SqlValidatorImpl.java:3673)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3661)}}
> {{ at
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)}}
> {{ at
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1100)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1071)}}
> {{ at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:247)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1046)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:752)}}
> {{ at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:587)}}
> {{ at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:257)}}
> {{ at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)}}
> {{ at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:648)}}
> {{ at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:514)}}
> {{ at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:484)}}
> {{ at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:234)}}
> {{ at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:623)}}
> {{ at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:677)}}
> {{ at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)}}
> {{ ... 67 more}}
> The problem can be reproduced by modifying the
> {{ReflectiveSchemaTest.testSelectWithFieldAccessOnFirstLevelRecordType()}}
> test and putting {{au."birthPlace"}} into parentheses:
> {{select (au."birthPlace")."city" as city from ... }}
> Putting identifiers into parentheses is common to disambiguate field access
> from identifier qualification. For exmaple, if the {{au}} prefix is removed
> from the query in the {{testSelectWithFieldAccessOnFirstLevelRecordType}}
> test, it will fail with {{{}Table 'birthPlace' not found{}}}. But if we put
> {{\"birthPlace\"}} into parentheses, then it is correctly recognized as a
> column of the {{authors}} table.
> I'm not sure about the correct fix to this issue which would not break the
> roll-up functionality, perhaps its author [~zhumayun] can help reviewing the
> PR or suggesting another fix. I'll soon create a PR with a proposed fix.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5157) ClassCastException in checkRollUp with DOT operator
[
https://issues.apache.org/jira/browse/CALCITE-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539168#comment-17539168
]
Chunwei Lei commented on CALCITE-5157:
--
FYI: I tried the following SQL in PostgreSQL and it works well.
{code:java}
postgres=# SELECT ("item")."name" FROM "on_hand" ;
name
fuzzy dice
(1 row){code}
> ClassCastException in checkRollUp with DOT operator
> ---
>
> Key: CALCITE-5157
> URL: https://issues.apache.org/jira/browse/CALCITE-5157
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.30.0
>Reporter: Viliam Durina
>Priority: Major
>
> When a query contains nested field access and is using parentheses to
> disambiguate the identifier, the {{SqlValidatorImpl.checkRollup()}} method
> throws a {{{}ClassCastException{}}}, assuming that the input of the DOT
> operator is a {{{}SqlCall{}}}. I think this assumption is wrong, the DOT
> operator typically has {{SqlIdentifier}} as an input, probably also other
> classes.
> Here's the stack trace:
> {{java.lang.ClassCastException: class org.apache.calcite.sql.SqlIdentifier
> cannot be cast to class org.apache.calcite.sql.SqlCall
> (org.apache.calcite.sql.SqlIdentifier and org.apache.calcite.sql.SqlCall are
> in unnamed module of loader 'app')}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3730)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3749)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUpInSelectList(SqlValidatorImpl.java:3673)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3661)}}
> {{ at
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)}}
> {{ at
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1100)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1071)}}
> {{ at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:247)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1046)}}
> {{ at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:752)}}
> {{ at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:587)}}
> {{ at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:257)}}
> {{ at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)}}
> {{ at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:648)}}
> {{ at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:514)}}
> {{ at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:484)}}
> {{ at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:234)}}
> {{ at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:623)}}
> {{ at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:677)}}
> {{ at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)}}
> {{ ... 67 more}}
> The problem can be reproduced by modifying the
> {{ReflectiveSchemaTest.testSelectWithFieldAccessOnFirstLevelRecordType()}}
> test and putting {{au."birthPlace"}} into parentheses:
> {{select (au."birthPlace")."city" as city from ... }}
> Putting identifiers into parentheses is common to disambiguate field access
> from identifier qualification. For exmaple, if the {{au}} prefix is removed
> from the query in the {{testSelectWithFieldAccessOnFirstLevelRecordType}}
> test, it will fail with {{{}Table 'birthPlace' not found{}}}. But if we put
> {{\"birthPlace\"}} into parentheses, then it is correctly recognized as a
> column of the {{authors}} table.
> I'm not sure about the correct fix to this issue which would not break the
> roll-up functionality, perhaps its author [~zhumayun] can help reviewing the
> PR or suggesting another fix. I'll soon create a PR with a proposed fix.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join
[
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei updated CALCITE-3890:
-
Fix Version/s: 1.31.0
> Derive IS NOT NULL filter for the inputs of inner join
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may
> not be null. For instance,
>
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null/b.id is not null and push down them into the
> child node of the join. Then it becomes
> {code:java}
> select * from (select* from a where id is null) t1 join (select * from b
> where id is not null) on t1.id = t2.id;
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Resolved] (CALCITE-5149) Refine RelMdColumnUniqueness for Aggregate by considering intersect keys between target keys and group keys
[ https://issues.apache.org/jira/browse/CALCITE-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunwei Lei resolved CALCITE-5149. -- Fix Version/s: 1.31.0 Resolution: Fixed Fixed in https://github.com/apache/calcite/commit/c5d45cb094407658f5f933240bc4492b6239e45e. > Refine RelMdColumnUniqueness for Aggregate by considering intersect keys > between target keys and group keys > > > Key: CALCITE-5149 > URL: https://issues.apache.org/jira/browse/CALCITE-5149 > Project: Calcite > Issue Type: Improvement >Reporter: Chunwei Lei >Assignee: Chunwei Lei >Priority: Major > Labels: pull-request-available > Fix For: 1.31.0 > > Time Spent: 20m > Remaining Estimate: 0h > > For instance, > columns: [0, 1] > group keys: [0, 2] > If [0] is unique, then [0, 1] is unique too. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Comment Edited] (CALCITE-5073) JoinConditionPushRule cannot infer 'LHS.C1 = LHS.C2' from 'LHS.C1 = RHS.C1 AND LHS.C2 = RHS.C1'
[
https://issues.apache.org/jira/browse/CALCITE-5073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537896#comment-17537896
]
Chunwei Lei edited comment on CALCITE-5073 at 5/17/22 3:37 AM:
---
FYI: Flink has a rule named {{JoinConditionEqualityTransferRule}} [1] to do the
same thing.
[1]
https://github.com/apache/flink/blob/master/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/rules/logical/JoinConditionEqualityTransferRule.scala
was (Author: chunwei lei):
FYI: Flink has a rule named {{JoinConditionEqualityTransferRule}} [1] to do the
same thing.
[1]https://github.com/apache/flink/blob/master/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/rules/logical/JoinConditionEqualityTransferRule.scala
> JoinConditionPushRule cannot infer 'LHS.C1 = LHS.C2' from 'LHS.C1 = RHS.C1
> AND LHS.C2 = RHS.C1'
> ---
>
> Key: CALCITE-5073
> URL: https://issues.apache.org/jira/browse/CALCITE-5073
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> For the below SQL:
> {code:java}
> SELECT * FROM T1, T2, T3
> WHERE T1.id = T3.id AND T2.id = T3.id{code}
> Currently, JoinConditionPushRule cannot infer the condition 'T1.id = T2.id'
> for the first join relation.
> There is another rule {{JoinPushThroughJoinRule}}, it can handle the above
> case in a different way (by reordering joins).
> This can be a more general optimization, which like the title says, we can
> infer more conditions from join condition (or above filter condition) to the
> Join's operands.
> It's reported in ML:
> https://lists.apache.org/thread/6fs09cgyz5rzrty90632ywoo477q0gdk
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5073) JoinConditionPushRule cannot infer 'LHS.C1 = LHS.C2' from 'LHS.C1 = RHS.C1 AND LHS.C2 = RHS.C1'
[
https://issues.apache.org/jira/browse/CALCITE-5073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537896#comment-17537896
]
Chunwei Lei commented on CALCITE-5073:
--
FYI: Flink has a rule named {{JoinConditionEqualityTransferRule}} [1] to do the
same thing.
[1]https://github.com/apache/flink/blob/master/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/rules/logical/JoinConditionEqualityTransferRule.scala
> JoinConditionPushRule cannot infer 'LHS.C1 = LHS.C2' from 'LHS.C1 = RHS.C1
> AND LHS.C2 = RHS.C1'
> ---
>
> Key: CALCITE-5073
> URL: https://issues.apache.org/jira/browse/CALCITE-5073
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> For the below SQL:
> {code:java}
> SELECT * FROM T1, T2, T3
> WHERE T1.id = T3.id AND T2.id = T3.id{code}
> Currently, JoinConditionPushRule cannot infer the condition 'T1.id = T2.id'
> for the first join relation.
> There is another rule {{JoinPushThroughJoinRule}}, it can handle the above
> case in a different way (by reordering joins).
> This can be a more general optimization, which like the title says, we can
> infer more conditions from join condition (or above filter condition) to the
> Join's operands.
> It's reported in ML:
> https://lists.apache.org/thread/6fs09cgyz5rzrty90632ywoo477q0gdk
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Resolved] (CALCITE-5137) EnumerableUncollect throws NPE if input has ((List) null)
[
https://issues.apache.org/jira/browse/CALCITE-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei resolved CALCITE-5137.
--
Fix Version/s: 1.31.0
Resolution: Fixed
Fixed in
[https://github.com/apache/calcite/commit/664e24921c521b9b8df03e4744d878a69dfeb10e.]
Thank you for your PR, [~dmsysolyatin]!
> EnumerableUncollect throws NPE if input has ((List) null)
> -
>
> Key: CALCITE-5137
> URL: https://issues.apache.org/jira/browse/CALCITE-5137
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Dmitry Sysolyatin
>Assignee: Dmitry Sysolyatin
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> EnumerableUncollect throws NPE when an input has element = ((List) null)
> Example:
> {code}
> SELECT * FROM UNNEST(CAST(null AS INTEGER ARRAY))
> {code}
> In a real situation, it can appear if to use left join. For example:
> {code}
> SELECT ARRAY(SELECT * FROM UNNEST(t.x)) FROM (VALUES(1)) LEFT JOIN (SELECT
> ARRAY[1] as x, 2 as y) t ON t.y = 1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Resolved] (CALCITE-5142) Upgrade Calcite to Avatica 1.21.0
[ https://issues.apache.org/jira/browse/CALCITE-5142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunwei Lei resolved CALCITE-5142. -- Resolution: Fixed Fixed in [https://github.com/apache/calcite/commit/4ce91575dbe553c9dd09396f9c76b9fda8bba1bb.] Thank you for the PR, [~libenchao] ! > Upgrade Calcite to Avatica 1.21.0 > - > > Key: CALCITE-5142 > URL: https://issues.apache.org/jira/browse/CALCITE-5142 > Project: Calcite > Issue Type: Improvement >Affects Versions: 1.30.0 >Reporter: Benchao Li >Assignee: Benchao Li >Priority: Major > Labels: pull-request-available > Fix For: 1.31.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Avatica 1.21.0 was just > [released|https://lists.apache.org/thread/nspxj2jq0owt982woyxnty6bdzstbyxd], > Calcite should use the latest Avatica version. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (CALCITE-5137) EnumerableUncollect throws NPE if input has ((List) null)
[
https://issues.apache.org/jira/browse/CALCITE-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537305#comment-17537305
]
Chunwei Lei commented on CALCITE-5137:
--
The PR looks good to me. I will merge the PR if no other comments in the next
24 hours.
> EnumerableUncollect throws NPE if input has ((List) null)
> -
>
> Key: CALCITE-5137
> URL: https://issues.apache.org/jira/browse/CALCITE-5137
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Dmitry Sysolyatin
>Assignee: Dmitry Sysolyatin
>Priority: Major
> Labels: pull-request-available
>
> EnumerableUncollect throws NPE when an input has element = ((List) null)
> Example:
> {code}
> SELECT * FROM UNNEST(CAST(null AS INTEGER ARRAY))
> {code}
> In a real situation, it can appear if to use left join. For example:
> {code}
> SELECT ARRAY(SELECT * FROM UNNEST(t.x)) FROM (VALUES(1)) LEFT JOIN (SELECT
> ARRAY[1] as x, 2 as y) t ON t.y = 1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5150) Parser should parse subquery with order by inside array constructor
[
https://issues.apache.org/jira/browse/CALCITE-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537303#comment-17537303
]
Chunwei Lei commented on CALCITE-5150:
--
Thank you for reporting this, [~dmsysolyatin] . Honestly, I have not seen this
syntax before. Are there other systems that support this syntax?
> Parser should parse subquery with order by inside array constructor
> ---
>
> Key: CALCITE-5150
> URL: https://issues.apache.org/jira/browse/CALCITE-5150
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Dmitry Sysolyatin
>Assignee: Dmitry Sysolyatin
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Parser can't parse query that looks like:
> {code}
> SELECT array(select x from (VALUES(1)) x ORDER BY x)
> {code}
> {code}
> Caused by: org.apache.calcite.sql.parser.babel.ParseException: Encountered
> "ORDER" at line 1, column 42.
> Was expecting one of:
> ")" ...
> "NATURAL" ...
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-5149) Refine RelMdColumnUniqueness for Aggregate by considering intersect keys between target keys and group keys
[ https://issues.apache.org/jira/browse/CALCITE-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunwei Lei updated CALCITE-5149: - Summary: Refine RelMdColumnUniqueness for Aggregate by considering intersect keys between target keys and group keys (was: Refine RelMdColumnUniqueness for Aggregate by considering intersect keys) > Refine RelMdColumnUniqueness for Aggregate by considering intersect keys > between target keys and group keys > > > Key: CALCITE-5149 > URL: https://issues.apache.org/jira/browse/CALCITE-5149 > Project: Calcite > Issue Type: Improvement >Reporter: Chunwei Lei >Assignee: Chunwei Lei >Priority: Major > > For instance, > columns: [0, 1] > group keys: [0, 2] > If [0] is unique, then [0, 1] is unique too. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (CALCITE-5081) Group keys of Aggregate are wrongly changed during decorrelation
[
https://issues.apache.org/jira/browse/CALCITE-5081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei resolved CALCITE-5081.
--
Fix Version/s: 1.31.0
Resolution: Fixed
Fixed in
[https://github.com/apache/calcite/commit/1bce280a2957326dc5c249cfd079edfd2c54adf4.]
Thank you for your PR, [~libenchao] !
> Group keys of Aggregate are wrongly changed during decorrelation
>
>
> Key: CALCITE-5081
> URL: https://issues.apache.org/jira/browse/CALCITE-5081
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Shiven Dvrk
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.31.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Bug in RelDecorrelator.decorrelate(Aggregate, boolean) is changing the group
> keys. We have following tables
> table1:
> {noformat}
> "ID", SqlTypeName.VARCHAR,
> "K509", SqlTypeName.BIGINT,
> "K505", SqlTypeName.BIGINT
> {noformat}
> table2:
> {noformat}
> "K14507", SqlTypeName.VARCHAR,
> "K14506", SqlTypeName.BIGINT,
> "K14509", SqlTypeName.BIGINT
> {noformat}
> A source relnode representing the following SQL:
> {code} SELECT
> `K505`,
> SUM(`test_total`) AS ` $ f1`
> FROM
> (
> SELECT
> ` $ cor0`.`K509`,
> ` $ cor0`.`K505`,
> ` $ cor0`.` $ f0` AS `test_total`
> FROM
> `table1` AS ` $ cor0`,
> LATERAL (
> SELECT
> SUM(`K14506`) AS ` $ f0`
> FROM
> `table2`
> WHERE
> `K14507` = ` $ cor0`.`ID`) AS `t1`
> )
> AS `t2`
> WHERE
> `t2`.`K505` = 1000
> GROUP BY
> `K505`
> {code}
>
> After decorelation, the decorrelated relNode represents the following
> incorrect SQL:
> {code} SELECT
> `t`.`K509`,
> SUM(`t1`.` $ f1`) AS ` $ f1`
> FROM
> (
> SELECT
> *
> FROM
> `table1`
> WHERE
> `K505` = 1000
> )
> AS `t`
> LEFT JOIN
> (
> SELECT
> `K14507`,
> SUM(`K14506`) AS ` $ f1`
> FROM
> `table2`
> GROUP BY
> `K14507`
> )
> AS `t1`
> ON `t`.`ID` = `t1`.`K14507`
> GROUP BY
> `t`.`K509`
> {code}
> it changed the group key.
>
> it looks like the logic in RelDecorrelator.decorrelate(Aggregate, boolean)
> is always picking the 0th index group key.
> We built the relNode using the following logic(used sqls above to explain
> the problem)
> {code}
> val builder = RelBuilder.create(createConfig())
> val v = Holder.of[RexCorrelVariable](null)
> val relNode = builder
> .scan("table1")
> .variable(v)
> .scan("table2")
> .filter(builder.equals(builder.field(0), builder.field(v.get(), "ID")))
> .project(builder.field("K14506"))
> .aggregate(builder.groupKey(), builder.sum(builder.field(0)))
> .project(builder.alias(builder.field(0), "test_total"))
> .correlate(JoinRelType.LEFT, v.get().id, builder.field(2, 0, "ID"))
> .project(builder.field(1), builder.field(2), builder.field(3))
> .filter(builder.equals(builder.field(1), builder.literal(1000)))
> .aggregate(builder.groupKey("K505"), builder.sum(builder.field(2)))
> .build()
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-5149) Refine RelMdColumnUniqueness for Aggregate by considering intersect keys
[ https://issues.apache.org/jira/browse/CALCITE-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunwei Lei updated CALCITE-5149: - Description: For instance, columns: [0, 1] group keys: [0, 2] If [0] is unique, then [0, 1] is unique too. was: For instance, columns: [0, 1] group by [0, 2] If [0] is unique, then [0, 1] is unique too. > Refine RelMdColumnUniqueness for Aggregate by considering intersect keys > > > Key: CALCITE-5149 > URL: https://issues.apache.org/jira/browse/CALCITE-5149 > Project: Calcite > Issue Type: Improvement >Reporter: Chunwei Lei >Assignee: Chunwei Lei >Priority: Major > > For instance, > columns: [0, 1] > group keys: [0, 2] > If [0] is unique, then [0, 1] is unique too. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (CALCITE-5149) Refine RelMdColumnUniqueness for Aggregate by considering intersect keys
[
https://issues.apache.org/jira/browse/CALCITE-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei updated CALCITE-5149:
-
Description:
For instance,
columns: [0, 1]
group by [0, 2]
If [0] is unique, then [0, 1] is unique too.
was:
Currently, RelMdColumnUniqueness always returns null when meeting the Window
operator. We can improve it by considering its partition keys. For instance,
{code:java}
select empno, rank () over (partition by empno order by empno) as rn from
emp{code}
If {{empno}} is the primary key of the table, {{empno}} is unique obviously.
> Refine RelMdColumnUniqueness for Aggregate by considering intersect keys
>
>
> Key: CALCITE-5149
> URL: https://issues.apache.org/jira/browse/CALCITE-5149
> Project: Calcite
> Issue Type: Improvement
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>
> For instance,
> columns: [0, 1]
> group by [0, 2]
> If [0] is unique, then [0, 1] is unique too.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-5149) Refine RelMdColumnUniqueness for Aggregate by considering intersect keys
[
https://issues.apache.org/jira/browse/CALCITE-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei updated CALCITE-5149:
-
Summary: Refine RelMdColumnUniqueness for Aggregate by considering
intersect keys (was: Refine RelMdColumnUniqueness for Window by considering
partition keys)
> Refine RelMdColumnUniqueness for Aggregate by considering intersect keys
>
>
> Key: CALCITE-5149
> URL: https://issues.apache.org/jira/browse/CALCITE-5149
> Project: Calcite
> Issue Type: Improvement
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>
> Currently, RelMdColumnUniqueness always returns null when meeting the Window
> operator. We can improve it by considering its partition keys. For instance,
> {code:java}
> select empno, rank () over (partition by empno order by empno) as rn from
> emp{code}
> If {{empno}} is the primary key of the table, {{empno}} is unique obviously.
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (CALCITE-5149) Refine RelMdColumnUniqueness for Window by considering partition keys
Chunwei Lei created CALCITE-5149:
Summary: Refine RelMdColumnUniqueness for Window by considering
partition keys
Key: CALCITE-5149
URL: https://issues.apache.org/jira/browse/CALCITE-5149
Project: Calcite
Issue Type: Improvement
Reporter: Chunwei Lei
Assignee: Chunwei Lei
Currently, RelMdColumnUniqueness always returns null when meeting the Window
operator. We can improve it by considering its partition keys. For instance,
{code:java}
select empno, rank () over (partition by empno order by empno) as rn from
emp{code}
If {{empno}} is the primary key of the table, {{empno}} is unique obviously.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Resolved] (CALCITE-5125) Extend "||" operator to work with arrays
[ https://issues.apache.org/jira/browse/CALCITE-5125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunwei Lei resolved CALCITE-5125. -- Fix Version/s: 1.31.0 Resolution: Fixed Fixed in [https://github.com/apache/calcite/commit/b7dbee54df60720ece301ba39cd928844e1a42ee.] Thank you for your PR, [~dmsysolyatin] ! > Extend "||" operator to work with arrays > > > Key: CALCITE-5125 > URL: https://issues.apache.org/jira/browse/CALCITE-5125 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Dmitry Sysolyatin >Assignee: Dmitry Sysolyatin >Priority: Major > Labels: pull-request-available > Fix For: 1.31.0 > > Time Spent: 50m > Remaining Estimate: 0h > > "||" operator can be used only for string concatenation but it would be good > to use it also for array concatenation as PostgreSQL -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join
[
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17535815#comment-17535815
]
Chunwei Lei commented on CALCITE-3890:
--
Exactly. I think {{Strong}} can handle it well.
> Derive IS NOT NULL filter for the inputs of inner join
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may
> not be null. For instance,
>
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null/b.id is not null and push down them into the
> child node of the join. Then it becomes
> {code:java}
> select * from (select* from a where id is null) t1 join (select * from b
> where id is not null) on t1.id = t2.id;
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join
[
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534652#comment-17534652
]
Chunwei Lei commented on CALCITE-3890:
--
I am trying to use {{Strong}} to analyze the join condition, as Julian said.
> Derive IS NOT NULL filter for the inputs of inner join
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may
> not be null. For instance,
>
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null/b.id is not null and push down them into the
> child node of the join. Then it becomes
> {code:java}
> select * from (select* from a where id is null) t1 join (select * from b
> where id is not null) on t1.id = t2.id;
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5125) Extend "||" operator to work with arrays
[ https://issues.apache.org/jira/browse/CALCITE-5125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534640#comment-17534640 ] Chunwei Lei commented on CALCITE-5125: -- The PR looks good to me. I will merge the PR if there're no more comments in the next 24 hours. > Extend "||" operator to work with arrays > > > Key: CALCITE-5125 > URL: https://issues.apache.org/jira/browse/CALCITE-5125 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Dmitry Sysolyatin >Assignee: Dmitry Sysolyatin >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > "||" operator can be used only for string concatenation but it would be good > to use it also for array concatenation as PostgreSQL -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (CALCITE-5131) Remove redundant type cast
[ https://issues.apache.org/jira/browse/CALCITE-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunwei Lei resolved CALCITE-5131. -- Fix Version/s: 1.31.0 Resolution: Fixed Fixed in [https://github.com/apache/calcite/commit/f58a6aa8d04accbca3d1237eddfd82dbd3553cf8.] Thank you for your review, [~asolimando] ! > Remove redundant type cast > -- > > Key: CALCITE-5131 > URL: https://issues.apache.org/jira/browse/CALCITE-5131 > Project: Calcite > Issue Type: Improvement >Reporter: Chunwei Lei >Assignee: Chunwei Lei >Priority: Major > Labels: pull-request-available > Fix For: 1.31.0 > > Attachments: image-2022-05-05-11-03-26-735.png, index.html, > script.js, styles.css > > Time Spent: 20m > Remaining Estimate: 0h > > There're a number of redundant type casts. I suggest removing them. > !image-2022-05-05-11-03-26-735.png! > > What do you think? BTW, it seems the CheckStyle plugin doesn't provide such a > check. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join
[
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534637#comment-17534637
]
Chunwei Lei commented on CALCITE-3890:
--
{quote}The description does not say that the filter is a relational expression,
nor does it say that the filter is before the join rather than after.
{quote}
The description has been updated.
> Derive IS NOT NULL filter for the inputs of inner join
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may
> not be null. For instance,
>
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null/b.id is not null and push down them into the
> child node of the join. Then it becomes
> {code:java}
> select * from (select* from a where id is null) t1 join (select * from b
> where id is not null) on t1.id = t2.id;
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join
[
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei updated CALCITE-3890:
-
Description:
We can infer IS NOT NULL predicate from join which implies some columns may not
be null. For instance,
{code:java}
select * from a join b on a.id = b.id;
{code}
we can infer a.id is not null/b.id is not null and push down them into the
child node of the join. Then it becomes
{code:java}
select * from (select* from a where id is null) t1 join (select * from b where
id is not null) on t1.id = t2.id;
{code}
was:
We can infer IS NOT NULL predicate from join which implies some columns may not
be null. For instance,
{code:java}
select * from a join b on a.id = b.id;
{code}
we can infer a.id is not null and b.id is not null and pushdown them into the
child node of the join. Then it becomes
{code:java}
select * from (select* from a where id is null) t1 join (select * from b where
id is not null) on t1.id = t2.id;
{code}
> Derive IS NOT NULL filter for the inputs of inner join
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may
> not be null. For instance,
>
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null/b.id is not null and push down them into the
> child node of the join. Then it becomes
> {code:java}
> select * from (select* from a where id is null) t1 join (select * from b
> where id is not null) on t1.id = t2.id;
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join
[
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei updated CALCITE-3890:
-
Description:
We can infer IS NOT NULL predicate from join which implies some columns may not
be null. For instance,
{code:java}
select * from a join b on a.id = b.id;
{code}
we can infer a.id is not null and b.id is not null and pushdown them into the
child node of the join. Then it becomes
{code:java}
select * from (select* from a where id is null) t1 join (select * from b where
id is not null) on t1.id = t2.id;
{code}
was:
We can infer IS NOT NULL predicate from join which implies some columns may not
be null. For instance,
{code:java}
select * from a join b on a.id = b.id;
{code}
we can infer a.id is not null and b.id is not null.
> Derive IS NOT NULL filter for the inputs of inner join
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may
> not be null. For instance,
>
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null and b.id is not null and pushdown them into the
> child node of the join. Then it becomes
> {code:java}
> select * from (select* from a where id is null) t1 join (select * from b
> where id is not null) on t1.id = t2.id;
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join
[
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534632#comment-17534632
]
Chunwei Lei commented on CALCITE-3890:
--
{quote}Emp e LEFT JOIN Dept d USING (deptno)
becomes
(SELECT * FROM Emp WHERE deptno IS NOT NULL) LEFT JOIN Dept d USING (deptno)
{quote}
AFAIK, this transformation is wrong. For left/right join, the
non-NULL-generating sides may have null values and they can not be filtered in
advance.
> Derive IS NOT NULL filter for the inputs of inner join
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may
> not be null. For instance,
>
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null and b.id is not null.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join
[
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei updated CALCITE-3890:
-
Summary: Derive IS NOT NULL filter for the inputs of inner join (was:
Derive IS NOT NULL filter from inner join)
> Derive IS NOT NULL filter for the inputs of inner join
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may
> not be null. For instance,
>
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null and b.id is not null.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (CALCITE-3890) Derive IS NOT NULL filter from inner join
[
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei updated CALCITE-3890:
-
Summary: Derive IS NOT NULL filter from inner join (was: Infer IS NOT NULL
predicate from join)
> Derive IS NOT NULL filter from inner join
> -
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>
> We can infer IS NOT NULL predicate from join which implies some columns may
> not be null. For instance,
>
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null and b.id is not null.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-3890) Infer IS NOT NULL predicate from join
[
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534097#comment-17534097
]
Chunwei Lei commented on CALCITE-3890:
--
I opened a PR for this feature: [https://github.com/apache/calcite/pull/2800.]
It would be great if someone can review it.
> Infer IS NOT NULL predicate from join
> -
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Chunwei Lei
>Assignee: Vineet Garg
>Priority: Major
>
> We can infer IS NOT NULL predicate from join which implies some columns may
> not be null. For instance,
>
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null and b.id is not null.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Assigned] (CALCITE-3890) Infer IS NOT NULL predicate from join
[
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei reassigned CALCITE-3890:
Assignee: Chunwei Lei (was: Vineet Garg)
> Infer IS NOT NULL predicate from join
> -
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>
> We can infer IS NOT NULL predicate from join which implies some columns may
> not be null. For instance,
>
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null and b.id is not null.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (CALCITE-5131) Remove redundant type cast
[ https://issues.apache.org/jira/browse/CALCITE-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534078#comment-17534078 ] Chunwei Lei commented on CALCITE-5131: -- Thank you for your trust. I will merge the PR if no other comments in the next 24 hours. > Remove redundant type cast > -- > > Key: CALCITE-5131 > URL: https://issues.apache.org/jira/browse/CALCITE-5131 > Project: Calcite > Issue Type: Improvement >Reporter: Chunwei Lei >Assignee: Chunwei Lei >Priority: Major > Labels: pull-request-available > Attachments: image-2022-05-05-11-03-26-735.png, index.html, > script.js, styles.css > > Time Spent: 10m > Remaining Estimate: 0h > > There're a number of redundant type casts. I suggest removing them. > !image-2022-05-05-11-03-26-735.png! > > What do you think? BTW, it seems the CheckStyle plugin doesn't provide such a > check. -- This message was sent by Atlassian Jira (v8.20.7#820007)
