[jira] [Created] (CALCITE-3928) Canonicalization doesn't do field trimming before materialized matching
Jin Xing created CALCITE-3928: - Summary: Canonicalization doesn't do field trimming before materialized matching Key: CALCITE-3928 URL: https://issues.apache.org/jira/browse/CALCITE-3928 Project: Calcite Issue Type: Bug Reporter: Jin Xing -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3925) Support materialized view matching for LogicalCorrelate in SubsitutionVisitor
Jin Xing created CALCITE-3925: - Summary: Support materialized view matching for LogicalCorrelate in SubsitutionVisitor Key: CALCITE-3925 URL: https://issues.apache.org/jira/browse/CALCITE-3925 Project: Calcite Issue Type: Bug Reporter: Jin Xing -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3760) Rewriting function without considering determinism can break query semantics
Jin Xing created CALCITE-3760:
-
Summary: Rewriting function without considering determinism can
break query semantics
Key: CALCITE-3760
URL: https://issues.apache.org/jira/browse/CALCITE-3760
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jin Xing
Assignee: Jin Xing
Calcite rewrite some *SqlFunctions* during validation. But whether the function
is deterministic is not considered. For a non-deterministic operator, the
rewriting can break semantics. Additionally there's no interface for user to
specify the determinism for a UDF/UDAF.
Say I have non-deterministic UDF & UDAF and run sql like below
{code:java}
select coalesce(udf(col0), 100) from foo;
select nullif(udaf(col0), 1024) from foo;{code}
They will be rewritten as
{code:java}
select case when udf(col0) is not null then udf(col0) else 100 end
from foo;
select case when udaf(col0)=1024 then null udaf(col0)
from foo{code}
As we can see that non-deterministic UDF & UDAF are called multiple times after
written. Thus the condition in WHEN clause might NOT be held all the time.
We need to provide an interface for user to specify the determinism in UDF/UDAF
and consider whether a SqlNode is deterministic when rewriting.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3758) FilterTableScanRule generate wrong mapping for filter condition when underlying is BindableTableScan
Jin Xing created CALCITE-3758: - Summary: FilterTableScanRule generate wrong mapping for filter condition when underlying is BindableTableScan Key: CALCITE-3758 URL: https://issues.apache.org/jira/browse/CALCITE-3758 Project: Calcite Issue Type: Improvement Components: core Reporter: Jin Xing Assignee: Jin Xing When FilterTableScanRule push filter condition into ProjectableFilterableTable, the filter condition should be adjusted to be based on the internal table of BindableTableScan. This line[1] is rectified as *Mappings.source* [1][https://github.com/apache/calcite/blob/571731b80a58eb095ebac7123285c375e7afff90/core/src/main/java/org/apache/calcite/rel/rules/FilterTableScanRule.java#L124] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3729) Filters failed to be pushed down when it's identical to join condition.
Jin Xing created CALCITE-3729:
-
Summary: Filters failed to be pushed down when it's identical to
join condition.
Key: CALCITE-3729
URL: https://issues.apache.org/jira/browse/CALCITE-3729
Project: Calcite
Issue Type: Improvement
Reporter: Jin Xing
Assignee: Jin Xing
FilterJoinRule optimize below sql as
{code:java}
select * from sales.dept d inner join sales.emp e
on d.deptno = e.deptno and d.deptno > e.mgr
where d.deptno > e.mgr
LogicalProject(DEPTNO=[$0], NAME=[$1], EMPNO=[$2], ENAME=[$3], JOB=[$4],
MGR=[$5], HIREDATE=[$6], SAL=[$7], COMM=[$8], DEPTNO0=[$9], SLACKER=[$10])
LogicalFilter(condition=[>($0, $5)])
LogicalJoin(condition=[AND(=($0, $9), >($0, $5))], joinType=[inner])
LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
{code}
The outer Filter failed to be pushed down into Join and removed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3727) Deduce nullability of RelDataTypeField for Join
Jin Xing created CALCITE-3727:
-
Summary: Deduce nullability of RelDataTypeField for Join
Key: CALCITE-3727
URL: https://issues.apache.org/jira/browse/CALCITE-3727
Project: Calcite
Issue Type: Improvement
Reporter: Jin Xing
If I have two tables and run a sql like below
{code:java}
table: emps
empno int
deptno int
table:depts
deptno int
select emps.deptno deptno, ... from
emps inner join depts
on emps.deptno = depts.deptno{code}
Its' a very common case that we build star model like above.
Currently row type of the RelNode plan have "isNullable = true" of "deptno"
column, i.e. Calcite doesn't look into join condition "emps.deptno =
depts.deptno" and deduce that the result deptno should have "isNullable=false"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3667) EnumerableMergeJoin should not use take-while enumerator
Jin Xing created CALCITE-3667: - Summary: EnumerableMergeJoin should not use take-while enumerator Key: CALCITE-3667 URL: https://issues.apache.org/jira/browse/CALCITE-3667 Project: Calcite Issue Type: Improvement Reporter: Jin Xing Assignee: Jin Xing -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3658) TableModify of Update contains correlated variable by mistake.
Jin Xing created CALCITE-3658: - Summary: TableModify of Update contains correlated variable by mistake. Key: CALCITE-3658 URL: https://issues.apache.org/jira/browse/CALCITE-3658 Project: Calcite Issue Type: Improvement Reporter: Jin Xing -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3549) Lex config for view expanding is not supported
Jin Xing created CALCITE-3549:
-
Summary: Lex config for view expanding is not supported
Key: CALCITE-3549
URL: https://issues.apache.org/jira/browse/CALCITE-3549
Project: Calcite
Issue Type: Improvement
Reporter: Jin Xing
UT as below:
{code:java}
// JdbcTest.java
@Test public void testDEV() {
modelWithView("select * from EMPLOYEES where deptno = 10", null)
.with(Lex.JAVA)
.query("select * from adhoc.V order by name desc")
.returns("");
}
{code}
Failed with
{code:java}
org.apache.calcite.sql.validate.SqlValidatorException: Column 'DEPTNO' not
found in any table; did you mean 'deptno'?
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3536) Wrong semantics in CoalesceImplementor
Jin Xing created CALCITE-3536: - Summary: Wrong semantics in CoalesceImplementor Key: CALCITE-3536 URL: https://issues.apache.org/jira/browse/CALCITE-3536 Project: Calcite Issue Type: Bug Reporter: Jin Xing -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3505) Infinite matching of FilterProjectTransposeRule causes stackoverflow
Jin Xing created CALCITE-3505: - Summary: Infinite matching of FilterProjectTransposeRule causes stackoverflow Key: CALCITE-3505 URL: https://issues.apache.org/jira/browse/CALCITE-3505 Project: Calcite Issue Type: Bug Components: core Reporter: Jin Xing -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3495) RelDecorrelator generate plan with different semantics when handle Aggregate
Jin Xing created CALCITE-3495: - Summary: RelDecorrelator generate plan with different semantics when handle Aggregate Key: CALCITE-3495 URL: https://issues.apache.org/jira/browse/CALCITE-3495 Project: Calcite Issue Type: Bug Reporter: Jin Xing -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3478) Reconstructure of materialized view tests.
Jin Xing created CALCITE-3478: - Summary: Reconstructure of materialized view tests. Key: CALCITE-3478 URL: https://issues.apache.org/jira/browse/CALCITE-3478 Project: Calcite Issue Type: Improvement Components: core Reporter: Jin Xing Assignee: Jin Xing Reconstructure of materialized view tests. Motivation of Reconstructure Currently there are two strategies for materialized view matching: strategy-1. Substitution based (SubstitutionVisitor.java) [1] strategy-2. Plan structural information based (AbstractMaterializedViewRule.java) [2] The two strategies are controlled by a single connection config of "materializationsEnabled". Calcite will apply strategy-1 firstly and then strategy-2. The two strategies are tested in a single integration test called MaterializationTest.java, As a result we cannot run tests separately for a single strategy, which leads to: 1. When some new matching patterns are supported by strategy-1, we might need to update the old result plan, which was previously matched and generated by stragegy-2 e.g. [3], and corresponding testing pattern for stragegy-2 will be lost. 2. Some test failures are even hidden, e.g. MaterializationTest#testJoinMaterialization2 is not supported by stragegy-2, but strategy-1 lets the test passed. 3. Hard to test internals for SubstutionVisitor.java, e.g. [4] has to struggle and create a unit test Of course we can add more system config or connection config just for testing and circle around some of the dilemas I mentioned above. But it will make the code messy. Materialized view matching str ategies are so important and worth a through unit test and to be kept clean. Additionally, this JIRA targets to clean the code of MaterializationTest.java. As more and more fixes get applied, this Java file tends to be messy: 1. Helping methods and testing methods are mixed without good order. 2. Lots of methods called checkMaterialize. We need to sort it out if there's need to add more params, e.g. [5] 3. Some tests are not concise enough, e.g. testJoinMaterialization9 Approach of Reconstructure 1. Create unit test MaterializedViewSubstitutionVisitorTest to test strategy-1 2. Create unit test MaterializedViewRelOptRulesTest to test strategy-2 3. Move tests from MaterializationTest to unit tests correspondingly, and keep MaterializationTest for integration tests. [1] https://calcite.apache.org/docs/materialized_views.html#substitution-via-rules-transformation [2] https://calcite.apache.org/docs/materialized_views.html#rewriting-using-plan-structural-information [3] https://github.com/apache/calcite/pull/1451/files#diff-d7e9e44fcb5fb1b98198415a3f78f167R1831 [4] https://github.com/apache/calcite/pull/1555 [5] https://github.com/apache/calcite/pull/1504 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3448) AggregateOnProjectToAggregateUnifyRule ignores Project incorrectly when there's missing grouping or mapping breaks ordering
jin xing created CALCITE-3448: - Summary: AggregateOnProjectToAggregateUnifyRule ignores Project incorrectly when there's missing grouping or mapping breaks ordering Key: CALCITE-3448 URL: https://issues.apache.org/jira/browse/CALCITE-3448 Project: Calcite Issue Type: Improvement Components: core Reporter: jin xing Assignee: jin xing -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3439) Support Intersect and Minus in RelMdPredicates
jin xing created CALCITE-3439: - Summary: Support Intersect and Minus in RelMdPredicates Key: CALCITE-3439 URL: https://issues.apache.org/jira/browse/CALCITE-3439 Project: Calcite Issue Type: Improvement Reporter: jin xing Assignee: jin xing Currently only Union is supported in RelMdPredicates. We might also support Intersect and Minus; This issue was found when resolving CALCITE-3428 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3431) SemiJoinRule doesn't work when right-hand side is natural distinct on join keys
jin xing created CALCITE-3431: - Summary: SemiJoinRule doesn't work when right-hand side is natural distinct on join keys Key: CALCITE-3431 URL: https://issues.apache.org/jira/browse/CALCITE-3431 Project: Calcite Issue Type: Improvement Components: core Reporter: jin xing Assignee: jin xing -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3428) Refine RelMdColumnUniqueness for Filter by considering constant columns
jin xing created CALCITE-3428: - Summary: Refine RelMdColumnUniqueness for Filter by considering constant columns Key: CALCITE-3428 URL: https://issues.apache.org/jira/browse/CALCITE-3428 Project: Calcite Issue Type: Improvement Reporter: jin xing Assignee: jin xing -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3409) Add an interface in MaterializedViewSubstitutionVisitor to allow registering UnifyRule
jin xing created CALCITE-3409: - Summary: Add an interface in MaterializedViewSubstitutionVisitor to allow registering UnifyRule Key: CALCITE-3409 URL: https://issues.apache.org/jira/browse/CALCITE-3409 Project: Calcite Issue Type: Improvement Reporter: jin xing Assignee: jin xing -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3405) Pruning columns for ProjectableFilterable when project is not simple mapping
jin xing created CALCITE-3405: - Summary: Pruning columns for ProjectableFilterable when project is not simple mapping Key: CALCITE-3405 URL: https://issues.apache.org/jira/browse/CALCITE-3405 Project: Calcite Issue Type: Bug Reporter: jin xing Assignee: jin xing -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3399) RelFieldTrimmer trim fields for UNION, INTERSECT, INTERSECT ALL, EXCEPT, EXCEPT ALL
jin xing created CALCITE-3399: - Summary: RelFieldTrimmer trim fields for UNION, INTERSECT, INTERSECT ALL, EXCEPT, EXCEPT ALL Key: CALCITE-3399 URL: https://issues.apache.org/jira/browse/CALCITE-3399 Project: Calcite Issue Type: Bug Components: core Reporter: jin xing Assignee: jin xing RelFieldTrimmer#trimFields provides functionality to trim fields for UNION, UNION ALL, INTERSECT, INTERSECT ALL, EXCEPT, EXCEPT ALL; But UNION ALL, INTERSECT, INTERSECT ALL, EXCEPT, EXCEPT ALL works by checking duplication. Column pruning on inputs of SetOp might lead to different semantics. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3396) Materialization matching succeeds when query and view are both of UNION but have different 'all' property
jin xing created CALCITE-3396: - Summary: Materialization matching succeeds when query and view are both of UNION but have different 'all' property Key: CALCITE-3396 URL: https://issues.apache.org/jira/browse/CALCITE-3396 Project: Calcite Issue Type: Bug Components: core Reporter: jin xing Assignee: jin xing -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3373) Decorrelate boolean context IN or existential subquery directly into SEMI/ANTI join
jin xing created CALCITE-3373: - Summary: Decorrelate boolean context IN or existential subquery directly into SEMI/ANTI join Key: CALCITE-3373 URL: https://issues.apache.org/jira/browse/CALCITE-3373 Project: Calcite Issue Type: Improvement Components: core Reporter: jin xing The current approach to generate semi-join from IN/EXISTS is as below: # Decorrelation # Transforming by SemiJoinRule to get a semi-join A more straightforward approach is to generate the semi-join right in Decorrelation. What's more, we should also have an approach to generate anti-join right in Decorrelation.(Currently there's no analogous rule to SemiJoinRule for anti-join) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3367) Add AntiJoinRule to convert project-filter-join-aggregate into anti-join
jin xing created CALCITE-3367: - Summary: Add AntiJoinRule to convert project-filter-join-aggregate into anti-join Key: CALCITE-3367 URL: https://issues.apache.org/jira/browse/CALCITE-3367 Project: Calcite Issue Type: Improvement Components: core Reporter: jin xing -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3366) RelDecorrelator supports Union
jin xing created CALCITE-3366: - Summary: RelDecorrelator supports Union Key: CALCITE-3366 URL: https://issues.apache.org/jira/browse/CALCITE-3366 Project: Calcite Issue Type: Improvement Reporter: jin xing -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3363) JoinUnionTransposeRule.RIGHT_UNION should not match SEMI/ANTI Join
jin xing created CALCITE-3363: - Summary: JoinUnionTransposeRule.RIGHT_UNION should not match SEMI/ANTI Join Key: CALCITE-3363 URL: https://issues.apache.org/jira/browse/CALCITE-3363 Project: Calcite Issue Type: Bug Components: core Reporter: jin xing JoinUnionTransposeRule works by pull up union from below to top of join. Thus it should not match semi/anti join by semantics. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3357) Trivial null checking in RelSet#addAbstractConverters
jin xing created CALCITE-3357: - Summary: Trivial null checking in RelSet#addAbstractConverters Key: CALCITE-3357 URL: https://issues.apache.org/jira/browse/CALCITE-3357 Project: Calcite Issue Type: Bug Components: core Reporter: jin xing In current code of *RelSet#addAbstractConverters*, null checking of *curRelTrait* happens after clause of assert *curRelTrait.getTraitDef() == traitDef*; It makes more sense to adjust the order; In my understanding, this issue was not found previously for two reasons: # AbstractConverter is not enabled by default ([https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/adapter/enumerable/EnumerableConvention.java#L65]) # It rarely happens that two algebra expression operators have identical semantics but different *RelTraitDef* -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3334) Refinement for Substitution-Based MV Matching
jin xing created CALCITE-3334: - Summary: Refinement for Substitution-Based MV Matching Key: CALCITE-3334 URL: https://issues.apache.org/jira/browse/CALCITE-3334 Project: Calcite Issue Type: Improvement Components: core Reporter: jin xing -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3292) SqlToRelConverter#substituteSubQuery fails with NullPointerException when converting SqlUpdate.
jin xing created CALCITE-3292: - Summary: SqlToRelConverter#substituteSubQuery fails with NullPointerException when converting SqlUpdate. Key: CALCITE-3292 URL: https://issues.apache.org/jira/browse/CALCITE-3292 Project: Calcite Issue Type: Bug Components: core Reporter: jin xing Assignee: jin xing -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3276) Add MV rules to match Join on compensating Project(s)
jin xing created CALCITE-3276: - Summary: Add MV rules to match Join on compensating Project(s) Key: CALCITE-3276 URL: https://issues.apache.org/jira/browse/CALCITE-3276 Project: Calcite Issue Type: Sub-task Components: core Reporter: jin xing -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3274) Add FilterOnProjectToFilterUnifyRule for materialization matching
jin xing created CALCITE-3274: - Summary: Add FilterOnProjectToFilterUnifyRule for materialization matching Key: CALCITE-3274 URL: https://issues.apache.org/jira/browse/CALCITE-3274 Project: Calcite Issue Type: Sub-task Components: core Reporter: jin xing -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3273) Materialization matching rules that consider compensating MutableProject
jin xing created CALCITE-3273: - Summary: Materialization matching rules that consider compensating MutableProject Key: CALCITE-3273 URL: https://issues.apache.org/jira/browse/CALCITE-3273 Project: Calcite Issue Type: Improvement Components: core Reporter: jin xing -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3262) Refine doc of SubstitutionVisitor.java
jin xing created CALCITE-3262:
-
Summary: Refine doc of SubstitutionVisitor.java
Key: CALCITE-3262
URL: https://issues.apache.org/jira/browse/CALCITE-3262
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: jin xing
Assignee: jin xing
Current doc of {{SubstitutionVisitor.java}} says the supported core relational
operators are {{@link org.apache.calcite.rel.logical.LogicalTableScan}}, and so
on.
But with {{convertTableAccess=true}}
(https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L5636),
it's a {{EnumerableTableScan}} below {{MutableScan}}, which is inconsistent
with the doc.
And what's more, {{MutableRels}} and {{SubstitutionVisitor}} supporting scope
doesn't limit to be org.apache.calcite.rel.logical.LogicalXXX.
So I think it might make sense to update/refine the doc to say that the
supported core relational operators are
{code:java}
* {@link org.apache.calcite.rel.core.TableScan},
* {@link org.apache.calcite.rel.core.Filter},
* {@link org.apache.calcite.rel.core.Project},
* {@link org.apache.calcite.rel.core.Join},
* {@link org.apache.calcite.rel.core.Union},
* {@link org.apache.calcite.rel.core.Aggregate}.
{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (CALCITE-3256) Add ProjectOnProjectToProjectUnifyRule for materialization matching.
jin xing created CALCITE-3256:
-
Summary: Add ProjectOnProjectToProjectUnifyRule for
materialization matching.
Key: CALCITE-3256
URL: https://issues.apache.org/jira/browse/CALCITE-3256
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: jin xing
Assignee: jin xing
In current implementation of rules in SubstitutionVisitor.java &
MaterializedViewSubstitutionVisitor.java, it's quite common to add a
compensating Project on top of child node of target(MV-rel) during matching.
But afterwards the next round matching should be able to handle such a
compensated Project and match upward along the plan tree. Otherwise we fail the
matching. After all, the goal of matching is to transform the query and let a
complete 'target' show up in the transformed query plan.
I found cases where the compensated Project cannot be properly handled.
{code:java}
MV:
select deptno, sum(salary) + 2, sum(commission)
from emps
group by deptno
Query:
select deptno, sum(salary) + 2
from emps
group by deptno
{code}
After matching of the Aggregates, a compensating Project is added, but
afterwards matching fails to handle it.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (CALCITE-3249) Substitution#getRexShuttle does not consider RexLiteral
jin xing created CALCITE-3249: - Summary: Substitution#getRexShuttle does not consider RexLiteral Key: CALCITE-3249 URL: https://issues.apache.org/jira/browse/CALCITE-3249 Project: Calcite Issue Type: Improvement Components: core Reporter: jin xing Assignee: jin xing -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (CALCITE-3239) Calc#accept(RexShuttle shuttle) does not update rowType
jin xing created CALCITE-3239:
-
Summary: Calc#accept(RexShuttle shuttle) does not update rowType
Key: CALCITE-3239
URL: https://issues.apache.org/jira/browse/CALCITE-3239
Project: Calcite
Issue Type: Bug
Components: core
Reporter: jin xing
In current code, {{Calc#accept(RexShuttle shuttle)}} keep the original
{{rowType}}, even when the expression list is modified by the shuttle.
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/core/Calc.java#L175
It might be wrong and a new {{rowType}} should be derived.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (CALCITE-3226) RelBuilder doesn't keep the alias when scan from an expanded view
jin xing created CALCITE-3226:
-
Summary: RelBuilder doesn't keep the alias when scan from an
expanded view
Key: CALCITE-3226
URL: https://issues.apache.org/jira/browse/CALCITE-3226
Project: Calcite
Issue Type: Bug
Components: core
Reporter: jin xing
In current implementation of {{RelBuilder::scan}}
([https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/tools/RelBuilder.java#L1048]),
the alias can be derived and recorded into {{Frame}} only when the {{RelNode}}
is a {{TableScan}}
([https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/tools/RelBuilder.java#L2754]).
But when {{RelBuilder::scan}} from an expanded view, the node is not a
{{TableScan}} and the alias is not kept. Below test failed - we cannot
reference a field by alias "MYVIEW".
{code:java}
@Test public void testExpandViewShouldKeepAlias() throws SQLException {
try (Connection connection = DriverManager.getConnection("jdbc:calcite:")) {
final Frameworks.ConfigBuilder configBuilder =
expandingConfig(connection);
final RelOptTable.ViewExpander viewExpander =
(RelOptTable.ViewExpander)
Frameworks.getPlanner(configBuilder.build());
final RelFactories.TableScanFactory tableScanFactory =
RelFactories.expandingScanFactory(viewExpander,
RelFactories.DEFAULT_TABLE_SCAN_FACTORY);
configBuilder.context(Contexts.of(tableScanFactory));
final RelBuilder builder = RelBuilder.create(configBuilder.build());
RelNode node =
builder.scan("MYVIEW")
.project(
builder.field(1, "MYVIEW", "EMPNO"),
builder.field(1, "MYVIEW", "ENAME"))
.build();
String expected =
"LogicalProject(EMPNO=[$0], ENAME=[$1])\n"
+ " LogicalFilter(condition=[=(1, 1)])\n"
+ "LogicalTableScan(table=[[scott, EMP]])\n";
assertThat(node, hasTree(expected));
}
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (CALCITE-3223) Non-RexInputRef may fails the matching of FilterToProjectUnifyRule during 'invert' by mistake.
jin xing created CALCITE-3223:
-
Summary: Non-RexInputRef may fails the matching of
FilterToProjectUnifyRule during 'invert' by mistake.
Key: CALCITE-3223
URL: https://issues.apache.org/jira/browse/CALCITE-3223
Project: Calcite
Issue Type: Bug
Components: core
Reporter: jin xing
In current code of
{{FilterToProjectUnifyRule::invert}}(https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/plan/SubstitutionVisitor.java#L1124),
the implementation
1. Fails the matching when there is Non-RexInputRef in the projects
2. Didn't check if all {{exprList}} has already ben set correctly.
As a result below tests fails.
{code:java}
@Test public void testFilterToProject0() {
String union =
"select * from \"emps\" where \"empid\" > 300\n"
+ "union all select * from \"emps\" where \"empid\" < 200";
String mv = "select *, \"empid\" * 2 from (" + union + ")";
String query = "select * from (" + union + ") where (\"empid\" * 2) > 3";
checkMaterialize(mv, query);
}
@Test public void testFilterToProject1() {
String agg =
"select \"deptno\", count(*) as \"c\", sum(\"salary\") as \"s\"\n"
+ "from \"emps\" group by \"deptno\"";
String mv = "select \"c\", \"s\", \"s\" from (" + agg + ")";
String query = "select * from (" + agg + ") where (\"s\" * 0.8) > 1";
checkNoMaterialize(mv, query, HR_FKUK_MODEL);
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (CALCITE-3214) Add UnionToUnionRule for materialization matching
jin xing created CALCITE-3214:
-
Summary: Add UnionToUnionRule for materialization matching
Key: CALCITE-3214
URL: https://issues.apache.org/jira/browse/CALCITE-3214
Project: Calcite
Issue Type: Bug
Components: core
Reporter: jin xing
Below materialization matching fails now
{code:java}
@Test public void testDEV() {
String sql0 = "select * from \"emps\" where \"empid\" < 300";
String sql1 = "select * from \"emps\" where \"empid\" > 200";
checkMaterialize(sql0 + " union all " + sql1, sql1 + " union all " + sql0);
}
{code}
This issue proposes to add a rule for union matching
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (CALCITE-3211) MutableRel returned from MutableRels::toMutables may lose reference to parent
jin xing created CALCITE-3211: - Summary: MutableRel returned from MutableRels::toMutables may lose reference to parent Key: CALCITE-3211 URL: https://issues.apache.org/jira/browse/CALCITE-3211 Project: Calcite Issue Type: Bug Reporter: jin xing -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (CALCITE-3209) Store inputs of MutableMultiRel by ArrayList rather than ImmutableList
jin xing created CALCITE-3209:
-
Summary: Store inputs of MutableMultiRel by ArrayList rather than
ImmutableList
Key: CALCITE-3209
URL: https://issues.apache.org/jira/browse/CALCITE-3209
Project: Calcite
Issue Type: Bug
Components: core
Reporter: jin xing
In current implementation of {{MutableMultiRel}}, {{inputs}} is implemented by
a {{ImmutableList}}, thus impossible to be changed by
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/mutable/MutableMultiRel.java#L42
{code:java}
@Override public void setInput(int ordinalInParent, MutableRel input)
{code}
Seems a mistake.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (CALCITE-3206) Better error message when param type incompatibility
jin xing created CALCITE-3206: - Summary: Better error message when param type incompatibility Key: CALCITE-3206 URL: https://issues.apache.org/jira/browse/CALCITE-3206 Project: Calcite Issue Type: Improvement Components: core Reporter: jin xing -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (CALCITE-3203) When matching materializations, match Project with child of Aggregate
jin xing created CALCITE-3203: - Summary: When matching materializations, match Project with child of Aggregate Key: CALCITE-3203 URL: https://issues.apache.org/jira/browse/CALCITE-3203 Project: Calcite Issue Type: Improvement Components: core Reporter: jin xing -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (CALCITE-3197) Convert data of Timestamp/Time/Date as original form when enumerating from ArrayTable
jin xing created CALCITE-3197:
-
Summary: Convert data of Timestamp/Time/Date as original form when
enumerating from ArrayTable
Key: CALCITE-3197
URL: https://issues.apache.org/jira/browse/CALCITE-3197
Project: Calcite
Issue Type: Bug
Components: core
Reporter: jin xing
In current implementation ColumnLoader, data of
{{Rep.JAVA_SQL_TIMESTAMP/Rep.JAVA_SQL_TIME/Rep.JAVA_SQL_DATE}} are converted as
numeric during loading.
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/adapter/clone/ColumnLoader.java#L234
But current code seems forgot to revert the data back to original form when
enumerating.
As a result, below test is failing now
{code:java}
// MaterializationTest.java
@Test public void testTimestampType() {
String sql = "select \"eventid\", \"ts\"\n"
+ "from \"events\"\n"
+ "where \"eventid\" > 5";
checkMaterialize(sql, sql);
}{code}
For type of {{Rep.JAVA_SQL_TIMESTAMP/Rep.JAVA_SQL_TIME/Rep.JAVA_SQL_DATE}},
cursor acesses {{TimestampAccessor/TimeAccessor/DateAccessor}}, which expect
column value as {{Timestamp/Time/Date}}.
It make sense to 'unwrap' the data as original form when enumerating from
{{ArrayTable}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (CALCITE-3182) Trim unused fields for plan of materialized-view before matching.
jin xing created CALCITE-3182:
-
Summary: Trim unused fields for plan of materialized-view before
matching.
Key: CALCITE-3182
URL: https://issues.apache.org/jira/browse/CALCITE-3182
Project: Calcite
Issue Type: Improvement
Reporter: jin xing
In current code, before matching query with materialized-view, unused fields of
query is trimed but materialized-view is not. Thus below simple SQL fails to be
matched though query and materialized-
view are exactly the same
```
@Test public void testMaterializationAfterTrimingOfUnusedFields() {
String sql =
"select \"y\".\"deptno\", \"y\".\"name\", \"x\".\"sum_salary\"\n" +
"from\n" +
" (select \"deptno\", sum(\"salary\") \"sum_salary\" from \"emps\" group by
\"deptno\") \"x\"\n" +
" join\n" +
" \"depts\" \"y\"\n" +
" on \"x\".\"deptno\"=\"y\".\"deptno\"\n";
checkMaterialize(sql, sql);
}
```
Checking \{{CalciteMaterializer}}
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/prepare/CalciteMaterializer.java#L83
, I think the code intends to do the trimming, but didn't call the method.
Since query is trimed for unused fields in anyway
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/prepare/CalcitePrepareImpl.java#L995
, thus I think there's no n
ecessity to keep the materialized-view un-trimmed
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-3167) Remove redundant overriding methods of equals in EnumerableTableScan.java
jin xing created CALCITE-3167: - Summary: Remove redundant overriding methods of equals in EnumerableTableScan.java Key: CALCITE-3167 URL: https://issues.apache.org/jira/browse/CALCITE-3167 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.19.0 Reporter: jin xing In current code of EnumerableTableScan.java, methods of equals are overrided for matching of EnumerableTableScans. But after optimizing with the same HEP planner, EnumerableTableScans from two plans but with the same digest will the share the same Java object. https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/plan/RelOptMaterializations.java#L192 I think it's ok to remove the redundant overriding methods in EnumerableTableScan.java -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2997) Avoid pushing down join condition in SqlToRelConverter
jin xing created CALCITE-2997:
-
Summary: Avoid pushing down join condition in SqlToRelConverter
Key: CALCITE-2997
URL: https://issues.apache.org/jira/browse/CALCITE-2997
Project: Calcite
Issue Type: Bug
Reporter: jin xing
In current code, *SqlToRelConverter:createJoin* is calling
*RelOptUtil.pushDownJoinConditions* for optimization. And we can find below
conversion from *SqlNode* to *RelNode*:
{code:java}
SqlNode:
select * from A join B on A.x = B.x * 2
RelNode (Logical-Plan):
Join (condition:col0=col1)
|-Project(x as col0)
| |-Scan
|-Project(x * 2 as col1)
|-Scan{code}
As we can see the logical plan(*RelNode*) posted above is not the pure
reflection of the original SQL String(*SqlNode*). The optimization is mixed
into the phase on which AST is converted to Logical-Plan. Actually optimizing
rule of JoinPushExpressionsRule is doing exactly the same kind of thing. Shall
we just keep the optimization inside Optimized-Logical-Plan ? I mean shall we
avoid calling *RelOptUtil.pushDownJoinConditions* in
*SqlToRelConverter:createJoin*
I raised this issue because that we are doing something based on the
Logical-Plan. And it makes us really confused that the Logical-Plan doesn't
corresponds to SqlNode.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
