[jira] [Commented] (CALCITE-4999) `least restrictive` type inference does not work when one operand is scalar and another is row with one field
[
https://issues.apache.org/jira/browse/CALCITE-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492473#comment-17492473
]
Ruben Q L commented on CALCITE-4999:
If I an not mistaken, the current Calcite implementation seems more powerful
(or more flexible) that e.g. the BigQuery one, because the ARRAY subquery
allows multiple columns.
For instance a single column ARRAY:
{code:sql}
SELECT r.r_name, ARRAY(
SELECT n.n_name FROM nation n WHERE n.n_regionkey = r.r_regionkey
)
FROM region r ORDER BY r.r_name
{code}
Returns a record type with one field for the ARRAY subquery (which seemed fine
so far, but it could potentially be a scalar: the point of this discussion).
But it also supports the case of a multi-column ARRAY:
{code:sql}
SELECT r.r_name, ARRAY(
SELECT n.n_nationkey, n.n_name FROM nation n WHERE n.n_regionkey =
r.r_regionkey
)
FROM region r ORDER BY r.r_name
{code}
It returns a record type with two fields for the ARRAY subquery (which is
perfectly fine).
If we changed the first case to return a scalar, wouldn't it be a misalignment
compared to the second? (not sure, just raising the question)
> `least restrictive` type inference does not work when one operand is scalar
> and another is row with one field
> -
>
> Key: CALCITE-4999
> URL: https://issues.apache.org/jira/browse/CALCITE-4999
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Dmitry Sysolyatin
>Assignee: Dmitry Sysolyatin
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> I faced with a issue that the following query:
> {code:java}
> SELECT ARRAY_CONCAT(ARRAY['1', '2'], array(select 'toast.' || x from
> unnest(ARRAY['1','2']) x)) {code}
> didn't work, because of:
> {code:java}
> java.lang.IllegalArgumentException: Cannot infer return type for
> ARRAY_CONCAT; operand types: [CHAR(1) ARRAY, RecordType(CHAR(7) EXPR$0)
> ARRAY]{code}
> `least restrictive` type inference does not work when one operand is scalar
> and another is row with one field
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (CALCITE-5011) CassandraAdapterDataTypesTest fails with initialization error
Alessandro Solimando created CALCITE-5011:
-
Summary: CassandraAdapterDataTypesTest fails with initialization
error
Key: CALCITE-5011
URL: https://issues.apache.org/jira/browse/CALCITE-5011
Project: Calcite
Issue Type: Bug
Components: tests
Affects Versions: 1.29.0
Reporter: Alessandro Solimando
Assignee: Alessandro Solimando
{noformat}
> Task :cassandra:test FAILED
FAILURE 0.0sec, org.apache.calcite.test.CassandraAdapterTest >
initializationError
com.datastax.oss.driver.api.core.servererrors.WriteTimeoutException:
Cassandra timeout during SIMPLE write query at consistency LOCAL_ONE (1 replica
were required but only 0 acknowledged the write)
at
com.datastax.oss.driver.api.core.servererrors.WriteTimeoutException.copy(WriteTimeoutException.java:96)
at
com.datastax.oss.driver.internal.core.util.concurrent.CompletableFutures.getUninterruptibly(CompletableFutures.java:149)
at
com.datastax.oss.driver.internal.core.cql.CqlRequestSyncProcessor.process(CqlRequestSyncProcessor.java:53)
at
com.datastax.oss.driver.internal.core.cql.CqlRequestSyncProcessor.process(CqlRequestSyncProcessor.java:30)
at
com.datastax.oss.driver.internal.core.session.DefaultSession.execute(DefaultSession.java:230)
at
com.datastax.oss.driver.api.core.cql.SyncCqlSession.execute(SyncCqlSession.java:54)
at
com.datastax.oss.driver.api.core.cql.SyncCqlSession.execute(SyncCqlSession.java:78)
at
org.cassandraunit.utils.CqlOperations.lambda$execute$0(CqlOperations.java:18)
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
at
java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
at org.cassandraunit.CQLDataLoader.load(CQLDataLoader.java:34)
at
org.apache.calcite.test.CassandraAdapterTest.load(CassandraAdapterTest.java:52)
FAILURE 38.2sec,1 completed, 1 failed, 0 skipped,
org.apache.calcite.test.CassandraAdapterTest
{noformat}
The root cause is probably a session leak, as suggested by the warning:
{noformat}
CassandraAdapterDataTypesTest > testSimpleTypesRowType() STANDARD_OUT
2022-02-14 04:36:56,298 [ForkJoinPool-1-worker-1] WARN - You have too many
session instances: 15 active, expected less than 4 (see
'advanced.session-leak.threshold' in the configuration)
> Task :core:forbiddenApisMain
{noformat}
For the full error see:
https://ci.appveyor.com/project/ApacheSoftwareFoundation/calcite/builds/42562506/job/4ppus57wtv36689d
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (CALCITE-2552) JDBC adapter generates Non-Dialect Specific Order By Field Identifier
[
https://issues.apache.org/jira/browse/CALCITE-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yanjing.wang updated CALCITE-2552:
--
Fix Version/s: 1.30.0
> JDBC adapter generates Non-Dialect Specific Order By Field Identifier
> -
>
> Key: CALCITE-2552
> URL: https://issues.apache.org/jira/browse/CALCITE-2552
> Project: Calcite
> Issue Type: Bug
> Components: jdbc-adapter
>Reporter: KrishnaKant Agrawal
>Assignee: yanjing.wang
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.30.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The Order By Keys when resolved during RelToSql Conversion is hardcoded to
> fallback to Original Expression instead of an alias(if present)
>
> This should be based on the Dialect set in RelToSqlConverter.
>
> If the Order By Key is a Projection which was Aliased, the Output query
> contains the original column name instead of the Alias, which is not allowed
> in HIVE as valid Order By keys are to be selected from Select List
> Identifiers only.
>
> Generated SQL(Failing in hive!),
> {code:java}
> Select first_name as n1 from emp order by first_name;{code}
> Expected SQL,
> {code:java}
> Select first_name as n1 from emp order by n1;{code}
> The Code Excerpt Responsible for this particular use case:-
> (SqlImplementor.java:952)
> {code:java}
> public SqlNode field(int ordinal) {
> final SqlNode selectItem = selectList.get(ordinal);
> switch (selectItem.getKind()) {
> case AS:
> return ((SqlCall) selectItem).operand(0);
> }
> return selectItem;
> }{code}
>
> I create the TableScan, Projection(with the alias) & Sort in the mentioned
> order.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (CALCITE-4912) Confusing javadoc of RexSimplify.simplify
[
https://issues.apache.org/jira/browse/CALCITE-4912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492577#comment-17492577
]
xiong duan commented on CALCITE-4912:
-
There are some typos in RexProgramTest too. I will fix it together in this PR.
> Confusing javadoc of RexSimplify.simplify
> -
>
> Key: CALCITE-4912
> URL: https://issues.apache.org/jira/browse/CALCITE-4912
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.28.0
>Reporter: Alexander Trushev
>Priority: Minor
>
> Javadoc of RexSimplify.simplify says that
> {code}
> simplify(x = 1 AND y = 2 AND NOT x = 1) returns y = 2
> {code}
> but it is wrong for any policy:
> x <- 1
> y <- 2
> x = 1 AND y = 2 AND NOT x = 1 == false
> y = 2 == true
> I guess it should have been like
> {code}
> simplify(y = 2 AND (x = 1 OR NOT x = 1 OR x IS NULL)) returns y = 2
> {code}
> which is true for any policy:
> {code}
> Origin: AND(=($1, CAST(2):INTEGER), OR(=($0, CAST(1):INTEGER),
> NOT(=($0, CAST(1):INTEGER)), IS NULL($0)))
> Simplify(Unknown): =($1, 2)
> Simplify(False): =($1, 2)
> Simplify(True): =($1, 2)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (CALCITE-5011) CassandraAdapterDataTypesTest fails with initialization error
[
https://issues.apache.org/jira/browse/CALCITE-5011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-5011:
Labels: pull-request-available test-stability (was: test-stability)
> CassandraAdapterDataTypesTest fails with initialization error
> -
>
> Key: CALCITE-5011
> URL: https://issues.apache.org/jira/browse/CALCITE-5011
> Project: Calcite
> Issue Type: Bug
> Components: tests
>Affects Versions: 1.29.0
>Reporter: Alessandro Solimando
>Assignee: Alessandro Solimando
>Priority: Major
> Labels: pull-request-available, test-stability
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {noformat}
> > Task :cassandra:test FAILED
> FAILURE 0.0sec, org.apache.calcite.test.CassandraAdapterTest >
> initializationError
> com.datastax.oss.driver.api.core.servererrors.WriteTimeoutException:
> Cassandra timeout during SIMPLE write query at consistency LOCAL_ONE (1
> replica were required but only 0 acknowledged the write)
> at
> com.datastax.oss.driver.api.core.servererrors.WriteTimeoutException.copy(WriteTimeoutException.java:96)
> at
> com.datastax.oss.driver.internal.core.util.concurrent.CompletableFutures.getUninterruptibly(CompletableFutures.java:149)
> at
> com.datastax.oss.driver.internal.core.cql.CqlRequestSyncProcessor.process(CqlRequestSyncProcessor.java:53)
> at
> com.datastax.oss.driver.internal.core.cql.CqlRequestSyncProcessor.process(CqlRequestSyncProcessor.java:30)
> at
> com.datastax.oss.driver.internal.core.session.DefaultSession.execute(DefaultSession.java:230)
> at
> com.datastax.oss.driver.api.core.cql.SyncCqlSession.execute(SyncCqlSession.java:54)
> at
> com.datastax.oss.driver.api.core.cql.SyncCqlSession.execute(SyncCqlSession.java:78)
> at
> org.cassandraunit.utils.CqlOperations.lambda$execute$0(CqlOperations.java:18)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at
> java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
> at org.cassandraunit.CQLDataLoader.load(CQLDataLoader.java:34)
> at
> org.apache.calcite.test.CassandraAdapterTest.load(CassandraAdapterTest.java:52)
> FAILURE 38.2sec,1 completed, 1 failed, 0 skipped,
> org.apache.calcite.test.CassandraAdapterTest
> {noformat}
> The root cause is probably a session leak, as suggested by the warning:
> {noformat}
> CassandraAdapterDataTypesTest > testSimpleTypesRowType() STANDARD_OUT
> 2022-02-14 04:36:56,298 [ForkJoinPool-1-worker-1] WARN - You have too
> many session instances: 15 active, expected less than 4 (see
> 'advanced.session-leak.threshold' in the configuration)
> > Task :core:forbiddenApisMain
> {noformat}
> For the full error see:
> https://ci.appveyor.com/project/ApacheSoftwareFoundation/calcite/builds/42562506/job/4ppus57wtv36689d
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (CALCITE-5011) CassandraAdapterDataTypesTest fails with initialization error
[
https://issues.apache.org/jira/browse/CALCITE-5011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492655#comment-17492655
]
Alessandro Solimando commented on CALCITE-5011:
---
CassandraSchema is creating a new session in "create" method, try to re-use
sessions if possible (same host, port, keyspace, username and password).
According to the [driver's
Javadoc|https://www.javadoc.io/doc/com.datastax.cassandra/cassandra-driver-core/3.11.0/com/datastax/driver/core/Session.html],
a session is a heavy-object which is meant to be shared.
> CassandraAdapterDataTypesTest fails with initialization error
> -
>
> Key: CALCITE-5011
> URL: https://issues.apache.org/jira/browse/CALCITE-5011
> Project: Calcite
> Issue Type: Bug
> Components: tests
>Affects Versions: 1.29.0
>Reporter: Alessandro Solimando
>Assignee: Alessandro Solimando
>Priority: Major
> Labels: pull-request-available, test-stability
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {noformat}
> > Task :cassandra:test FAILED
> FAILURE 0.0sec, org.apache.calcite.test.CassandraAdapterTest >
> initializationError
> com.datastax.oss.driver.api.core.servererrors.WriteTimeoutException:
> Cassandra timeout during SIMPLE write query at consistency LOCAL_ONE (1
> replica were required but only 0 acknowledged the write)
> at
> com.datastax.oss.driver.api.core.servererrors.WriteTimeoutException.copy(WriteTimeoutException.java:96)
> at
> com.datastax.oss.driver.internal.core.util.concurrent.CompletableFutures.getUninterruptibly(CompletableFutures.java:149)
> at
> com.datastax.oss.driver.internal.core.cql.CqlRequestSyncProcessor.process(CqlRequestSyncProcessor.java:53)
> at
> com.datastax.oss.driver.internal.core.cql.CqlRequestSyncProcessor.process(CqlRequestSyncProcessor.java:30)
> at
> com.datastax.oss.driver.internal.core.session.DefaultSession.execute(DefaultSession.java:230)
> at
> com.datastax.oss.driver.api.core.cql.SyncCqlSession.execute(SyncCqlSession.java:54)
> at
> com.datastax.oss.driver.api.core.cql.SyncCqlSession.execute(SyncCqlSession.java:78)
> at
> org.cassandraunit.utils.CqlOperations.lambda$execute$0(CqlOperations.java:18)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at
> java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
> at org.cassandraunit.CQLDataLoader.load(CQLDataLoader.java:34)
> at
> org.apache.calcite.test.CassandraAdapterTest.load(CassandraAdapterTest.java:52)
> FAILURE 38.2sec,1 completed, 1 failed, 0 skipped,
> org.apache.calcite.test.CassandraAdapterTest
> {noformat}
> The root cause is probably a session leak, as suggested by the warning:
> {noformat}
> CassandraAdapterDataTypesTest > testSimpleTypesRowType() STANDARD_OUT
> 2022-02-14 04:36:56,298 [ForkJoinPool-1-worker-1] WARN - You have too
> many session instances: 15 active, expected less than 4 (see
> 'advanced.session-leak.threshold' in the configuration)
> > Task :core:forbiddenApisMain
> {noformat}
> For the full error see:
> https://ci.appveyor.com/project/ApacheSoftwareFoundation/calcite/builds/42562506/job/4ppus57wtv36689d
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (CALCITE-4539) Support pluggable metadata handlers and caching
[ https://issues.apache.org/jira/browse/CALCITE-4539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492685#comment-17492685 ] Chang chen commented on CALCITE-4539: - hi [~jamesstarr] What is the status of this issue? > Support pluggable metadata handlers and caching > --- > > Key: CALCITE-4539 > URL: https://issues.apache.org/jira/browse/CALCITE-4539 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: James Starr >Priority: Major > Labels: pull-request-available > Time Spent: 4h 20m > Remaining Estimate: 0h > > Calcite janino backed metadata does not support all the functionality that > the java reflection backed metadata system, such as customizing the cache to > use soft refs or have a max size. Furthermore, the janino backed metadata > provider has many opinion that other are not universally shared, such as lazy > loading, requiring all rels to registered to a global singleton, use of > thread locals. There is no easy way to currently override these opinions. > Calcites metadata system should consist of 3 APIs: metadata consumer, > metadata implementors, metadata behavior. The metadata consumer need to able > to fetch the appropriate metadata and invalidate the cache. Currently this > is all done through RelMetadataQuery with some very leaky abstractions. > Metadata implementors create functions that extract the metadata for > particular node types. These functions are registered via > ReflectiveRelMetadataProvider and ChainedRelMetadataProvider. Attempting to > configure the internal behavior does not currently have an api. Currently > portions of it a scattered through out various class and are frequently not > extensible. These could all be concentrated in a single class > MetadataHandlerProvider, to allow for configuration of eager/lazy loading > behavior, caching policy(thread safe, max size, soft references), or a clear > api for some to go their own way for handler generation. > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4995) AssertionError caused by RelFieldTrimmer on SEMI/ANTI join
[
https://issues.apache.org/jira/browse/CALCITE-4995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492710#comment-17492710
]
Ruben Q L commented on CALCITE-4995:
Fixed via
https://github.com/apache/calcite/commit/04029ff5a1918c373dced4c62750fa7093b311ab
> AssertionError caused by RelFieldTrimmer on SEMI/ANTI join
> --
>
> Key: CALCITE-4995
> URL: https://issues.apache.org/jira/browse/CALCITE-4995
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.29.0
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.30.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> It seems {{RelFieldTrimmer}} can cause an {{AssertionError}} (or later an
> {{ArrayIndexOutOfBoundsException}} if assertions are disabled) on certain
> plans involving SEMI/ANTI join (i.e. joins that do NOT project the RHS
> fields).
> The root cause seems to be the "early return" in
> {{RelFieldTrimmer#trimFields(Join join, ImmutableBitSet fieldsUsed,
> Set extraFields)}} when nothing has been trimmed inside
> join's inputs (so the join itself can be return as it is):
> {code:java}
> if (changeCount == 0
> && mapping.isIdentity()) {
> return result(join, Mappings.createIdentity(fieldCount));
> }
> {code}
> The problem is that this {{fieldCount}} is an addition of LHS + RHS fields (+
> system fields); but in case of a SEMI/ANTI the mappings to be returned must
> not consider RHS fields (since they are not projected by these join types).
> The problem only happens here (when the trimmer does not trim the join).
> Notice that, a few lines below, in the "other return scenario" of the method
> (when something has been trimmed), there is a special treatment of the
> mapping for ANTI/SEMI, so things will work fine in this case:
> {code:java}
> switch (join.getJoinType()) {
> case SEMI:
> case ANTI:
> // For SemiJoins and AntiJoins only map fields from the left-side
> if (join.getJoinType() == JoinRelType.SEMI) {
> relBuilder.semiJoin(newConditionExpr);
> } else {
> relBuilder.antiJoin(newConditionExpr);
> }
> Mapping inputMapping = inputMappings.get(0);
> mapping = Mappings.create(MappingType.INVERSE_SURJECTION,
> join.getRowType().getFieldCount(),
> newSystemFieldCount + inputMapping.getTargetCount());
> for (int i = 0; i < newSystemFieldCount; ++i) {
> mapping.set(i, i);
> }
> offset = systemFieldCount;
> newOffset = newSystemFieldCount;
> for (IntPair pair : inputMapping) {
> mapping.set(pair.source + offset, pair.target + newOffset);
> }
> break;
> default:
> relBuilder.join(join.getJoinType(), newConditionExpr);
> }
> relBuilder.hints(join.getHints());
> return result(relBuilder.build(), mapping);
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Resolved] (CALCITE-4995) AssertionError caused by RelFieldTrimmer on SEMI/ANTI join
[
https://issues.apache.org/jira/browse/CALCITE-4995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L resolved CALCITE-4995.
Resolution: Fixed
> AssertionError caused by RelFieldTrimmer on SEMI/ANTI join
> --
>
> Key: CALCITE-4995
> URL: https://issues.apache.org/jira/browse/CALCITE-4995
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.29.0
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.30.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> It seems {{RelFieldTrimmer}} can cause an {{AssertionError}} (or later an
> {{ArrayIndexOutOfBoundsException}} if assertions are disabled) on certain
> plans involving SEMI/ANTI join (i.e. joins that do NOT project the RHS
> fields).
> The root cause seems to be the "early return" in
> {{RelFieldTrimmer#trimFields(Join join, ImmutableBitSet fieldsUsed,
> Set extraFields)}} when nothing has been trimmed inside
> join's inputs (so the join itself can be return as it is):
> {code:java}
> if (changeCount == 0
> && mapping.isIdentity()) {
> return result(join, Mappings.createIdentity(fieldCount));
> }
> {code}
> The problem is that this {{fieldCount}} is an addition of LHS + RHS fields (+
> system fields); but in case of a SEMI/ANTI the mappings to be returned must
> not consider RHS fields (since they are not projected by these join types).
> The problem only happens here (when the trimmer does not trim the join).
> Notice that, a few lines below, in the "other return scenario" of the method
> (when something has been trimmed), there is a special treatment of the
> mapping for ANTI/SEMI, so things will work fine in this case:
> {code:java}
> switch (join.getJoinType()) {
> case SEMI:
> case ANTI:
> // For SemiJoins and AntiJoins only map fields from the left-side
> if (join.getJoinType() == JoinRelType.SEMI) {
> relBuilder.semiJoin(newConditionExpr);
> } else {
> relBuilder.antiJoin(newConditionExpr);
> }
> Mapping inputMapping = inputMappings.get(0);
> mapping = Mappings.create(MappingType.INVERSE_SURJECTION,
> join.getRowType().getFieldCount(),
> newSystemFieldCount + inputMapping.getTargetCount());
> for (int i = 0; i < newSystemFieldCount; ++i) {
> mapping.set(i, i);
> }
> offset = systemFieldCount;
> newOffset = newSystemFieldCount;
> for (IntPair pair : inputMapping) {
> mapping.set(pair.source + offset, pair.target + newOffset);
> }
> break;
> default:
> relBuilder.join(join.getJoinType(), newConditionExpr);
> }
> relBuilder.hints(join.getHints());
> return result(relBuilder.build(), mapping);
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (CALCITE-4539) Support pluggable metadata handlers and caching
[ https://issues.apache.org/jira/browse/CALCITE-4539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492884#comment-17492884 ] James Starr commented on CALCITE-4539: -- [~baibaichen] All except the pluggable cache is done. > Support pluggable metadata handlers and caching > --- > > Key: CALCITE-4539 > URL: https://issues.apache.org/jira/browse/CALCITE-4539 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: James Starr >Priority: Major > Labels: pull-request-available > Time Spent: 4h 20m > Remaining Estimate: 0h > > Calcite janino backed metadata does not support all the functionality that > the java reflection backed metadata system, such as customizing the cache to > use soft refs or have a max size. Furthermore, the janino backed metadata > provider has many opinion that other are not universally shared, such as lazy > loading, requiring all rels to registered to a global singleton, use of > thread locals. There is no easy way to currently override these opinions. > Calcites metadata system should consist of 3 APIs: metadata consumer, > metadata implementors, metadata behavior. The metadata consumer need to able > to fetch the appropriate metadata and invalidate the cache. Currently this > is all done through RelMetadataQuery with some very leaky abstractions. > Metadata implementors create functions that extract the metadata for > particular node types. These functions are registered via > ReflectiveRelMetadataProvider and ChainedRelMetadataProvider. Attempting to > configure the internal behavior does not currently have an api. Currently > portions of it a scattered through out various class and are frequently not > extensible. These could all be concentrated in a single class > MetadataHandlerProvider, to allow for configuration of eager/lazy loading > behavior, caching policy(thread safe, max size, soft references), or a clear > api for some to go their own way for handler generation. > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (CALCITE-4999) `least restrictive` type inference does not work when one operand is scalar and another is row with one field
[
https://issues.apache.org/jira/browse/CALCITE-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492925#comment-17492925
]
Julian Hyde commented on CALCITE-4999:
--
[~rubenql], I had that concern too. But if the SQL standard says that ARRAY
sub-query should have 1 column and the result type is an ARRAY of that column's
type, then we should comply with that standard. We can go beyond the standard,
and support queries with 2 or more columns, and return an ARRAY of ROW objects,
or whatever, but when we are within the parameters of the standard, we should
comply with the standard.
Hence we need to check what the standard says.
> `least restrictive` type inference does not work when one operand is scalar
> and another is row with one field
> -
>
> Key: CALCITE-4999
> URL: https://issues.apache.org/jira/browse/CALCITE-4999
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Dmitry Sysolyatin
>Assignee: Dmitry Sysolyatin
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> I faced with a issue that the following query:
> {code:java}
> SELECT ARRAY_CONCAT(ARRAY['1', '2'], array(select 'toast.' || x from
> unnest(ARRAY['1','2']) x)) {code}
> didn't work, because of:
> {code:java}
> java.lang.IllegalArgumentException: Cannot infer return type for
> ARRAY_CONCAT; operand types: [CHAR(1) ARRAY, RecordType(CHAR(7) EXPR$0)
> ARRAY]{code}
> `least restrictive` type inference does not work when one operand is scalar
> and another is row with one field
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (CALCITE-4323) View with ORDER BY throws AssertionError during view expansion
[
https://issues.apache.org/jira/browse/CALCITE-4323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492946#comment-17492946
]
Julian Hyde commented on CALCITE-4323:
--
Reviewing and testing now, and will merge shortly.
Any reason you included 'select.getFetch() != null' but not '
select.getOffset() != null'?
> View with ORDER BY throws AssertionError during view expansion
> --
>
> Key: CALCITE-4323
> URL: https://issues.apache.org/jira/browse/CALCITE-4323
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Jiatao Tao
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2020-10-10-23-58-39-822.png,
> image-2020-10-10-23-59-22-297.png
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> If you have a view that has an ORDER BY clause, and try to execute a query on
> that view, SqlToRelConverter throws AssertionError during view expansion. For
> example,
> {noformat}
> create view v as select * from "EMPLOYEES" order by "deptno";
> select * from V;
> java.lang.AssertionError
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.requiredCollation(SqlToRelConverter.java:634)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.requiredCollation(SqlToRelConverter.java:629)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:578)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.expandView(CalcitePrepareImpl.java:1072)
> at
> org.apache.calcite.plan.ViewExpanders$1.expandView(ViewExpanders.java:52)
> at
> org.apache.calcite.schema.impl.ViewTable.expandView(ViewTable.java:127)
> at org.apache.calcite.schema.impl.ViewTable.toRel(ViewTable.java:120)
> at
> org.apache.calcite.prepare.RelOptTableImpl.toRel(RelOptTableImpl.java:285)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:3605)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2522)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2160)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2109)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2066)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:662)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:643)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3458)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:569)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:242)
> {noformat}
> I suspected that it would be a problem for when the view does not project the
> sort column(s), but it seems to be a problem even if the columns are
> projected.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (CALCITE-4323) View with ORDER BY throws AssertionError during view expansion
[
https://issues.apache.org/jira/browse/CALCITE-4323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated CALCITE-4323:
-
Fix Version/s: 1.30.0
> View with ORDER BY throws AssertionError during view expansion
> --
>
> Key: CALCITE-4323
> URL: https://issues.apache.org/jira/browse/CALCITE-4323
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Jiatao Tao
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.30.0
>
> Attachments: image-2020-10-10-23-58-39-822.png,
> image-2020-10-10-23-59-22-297.png
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> If you have a view that has an ORDER BY clause, and try to execute a query on
> that view, SqlToRelConverter throws AssertionError during view expansion. For
> example,
> {noformat}
> create view v as select * from "EMPLOYEES" order by "deptno";
> select * from V;
> java.lang.AssertionError
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.requiredCollation(SqlToRelConverter.java:634)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.requiredCollation(SqlToRelConverter.java:629)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:578)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.expandView(CalcitePrepareImpl.java:1072)
> at
> org.apache.calcite.plan.ViewExpanders$1.expandView(ViewExpanders.java:52)
> at
> org.apache.calcite.schema.impl.ViewTable.expandView(ViewTable.java:127)
> at org.apache.calcite.schema.impl.ViewTable.toRel(ViewTable.java:120)
> at
> org.apache.calcite.prepare.RelOptTableImpl.toRel(RelOptTableImpl.java:285)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:3605)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2522)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2160)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2109)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2066)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:662)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:643)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3458)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:569)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:242)
> {noformat}
> I suspected that it would be a problem for when the view does not project the
> sort column(s), but it seems to be a problem even if the columns are
> projected.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (CALCITE-4323) View with ORDER BY throws AssertionError during view expansion
[
https://issues.apache.org/jira/browse/CALCITE-4323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492965#comment-17492965
]
Jiajun Xie commented on CALCITE-4323:
-
[~julianhyde] Sorry, I thought there must be a limit if there was an offset, so
I didn't use it. Can you provide a simple case? I'll add it to the unit test.
> View with ORDER BY throws AssertionError during view expansion
> --
>
> Key: CALCITE-4323
> URL: https://issues.apache.org/jira/browse/CALCITE-4323
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Jiatao Tao
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.30.0
>
> Attachments: image-2020-10-10-23-58-39-822.png,
> image-2020-10-10-23-59-22-297.png
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> If you have a view that has an ORDER BY clause, and try to execute a query on
> that view, SqlToRelConverter throws AssertionError during view expansion. For
> example,
> {noformat}
> create view v as select * from "EMPLOYEES" order by "deptno";
> select * from V;
> java.lang.AssertionError
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.requiredCollation(SqlToRelConverter.java:634)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.requiredCollation(SqlToRelConverter.java:629)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:578)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.expandView(CalcitePrepareImpl.java:1072)
> at
> org.apache.calcite.plan.ViewExpanders$1.expandView(ViewExpanders.java:52)
> at
> org.apache.calcite.schema.impl.ViewTable.expandView(ViewTable.java:127)
> at org.apache.calcite.schema.impl.ViewTable.toRel(ViewTable.java:120)
> at
> org.apache.calcite.prepare.RelOptTableImpl.toRel(RelOptTableImpl.java:285)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:3605)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2522)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2160)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2109)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2066)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:662)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:643)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3458)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:569)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:242)
> {noformat}
> I suspected that it would be a problem for when the view does not project the
> sort column(s), but it seems to be a problem even if the columns are
> projected.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (CALCITE-4539) Support pluggable metadata handlers and caching
[ https://issues.apache.org/jira/browse/CALCITE-4539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492969#comment-17492969 ] Chang chen commented on CALCITE-4539: - [~jamesstarr] thanks for update. What's the issue blocking the pluggable cache? > Support pluggable metadata handlers and caching > --- > > Key: CALCITE-4539 > URL: https://issues.apache.org/jira/browse/CALCITE-4539 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: James Starr >Priority: Major > Labels: pull-request-available > Time Spent: 4h 20m > Remaining Estimate: 0h > > Calcite janino backed metadata does not support all the functionality that > the java reflection backed metadata system, such as customizing the cache to > use soft refs or have a max size. Furthermore, the janino backed metadata > provider has many opinion that other are not universally shared, such as lazy > loading, requiring all rels to registered to a global singleton, use of > thread locals. There is no easy way to currently override these opinions. > Calcites metadata system should consist of 3 APIs: metadata consumer, > metadata implementors, metadata behavior. The metadata consumer need to able > to fetch the appropriate metadata and invalidate the cache. Currently this > is all done through RelMetadataQuery with some very leaky abstractions. > Metadata implementors create functions that extract the metadata for > particular node types. These functions are registered via > ReflectiveRelMetadataProvider and ChainedRelMetadataProvider. Attempting to > configure the internal behavior does not currently have an api. Currently > portions of it a scattered through out various class and are frequently not > extensible. These could all be concentrated in a single class > MetadataHandlerProvider, to allow for configuration of eager/lazy loading > behavior, caching policy(thread safe, max size, soft references), or a clear > api for some to go their own way for handler generation. > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (CALCITE-4323) View with ORDER BY throws AssertionError during view expansion
[
https://issues.apache.org/jira/browse/CALCITE-4323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492965#comment-17492965
]
Jiajun Xie edited comment on CALCITE-4323 at 2/16/22, 2:37 AM:
---
[~julianhyde] Sorry, I thought there must be a limit if there was an offset, so
I didn't use it.I find a simple case
{code:java}
select * from \"adhoc\".V offset 10{code}
I'll add it to the unit test.
was (Author: jiajunbernoulli):
[~julianhyde] Sorry, I thought there must be a limit if there was an offset, so
I didn't use it. Can you provide a simple case? I'll add it to the unit test.
> View with ORDER BY throws AssertionError during view expansion
> --
>
> Key: CALCITE-4323
> URL: https://issues.apache.org/jira/browse/CALCITE-4323
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Jiatao Tao
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.30.0
>
> Attachments: image-2020-10-10-23-58-39-822.png,
> image-2020-10-10-23-59-22-297.png
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> If you have a view that has an ORDER BY clause, and try to execute a query on
> that view, SqlToRelConverter throws AssertionError during view expansion. For
> example,
> {noformat}
> create view v as select * from "EMPLOYEES" order by "deptno";
> select * from V;
> java.lang.AssertionError
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.requiredCollation(SqlToRelConverter.java:634)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.requiredCollation(SqlToRelConverter.java:629)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:578)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.expandView(CalcitePrepareImpl.java:1072)
> at
> org.apache.calcite.plan.ViewExpanders$1.expandView(ViewExpanders.java:52)
> at
> org.apache.calcite.schema.impl.ViewTable.expandView(ViewTable.java:127)
> at org.apache.calcite.schema.impl.ViewTable.toRel(ViewTable.java:120)
> at
> org.apache.calcite.prepare.RelOptTableImpl.toRel(RelOptTableImpl.java:285)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:3605)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2522)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2160)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2109)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2066)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:662)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:643)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3458)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:569)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:242)
> {noformat}
> I suspected that it would be a problem for when the view does not project the
> sort column(s), but it seems to be a problem even if the columns are
> projected.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (CALCITE-5012) SubstitutionVisitor need a new rule to support remain top calc in mv.
Xurenhe created CALCITE-5012:
Summary: SubstitutionVisitor need a new rule to support remain top
calc in mv.
Key: CALCITE-5012
URL: https://issues.apache.org/jira/browse/CALCITE-5012
Project: Calcite
Issue Type: Bug
Reporter: Xurenhe
SubstitutionVisitor is a bottom-up to rewrite query's plan by mv.
Rewriting query successfully, when mv's plan has become a part of the query
plan.
The current rules are as follows:
{code:java}
AggregateOnCalcToAggregateUnifyRule
AggregateToAggregateUnifyRule
CalcToCalcUnifyRule
IntersectOnCalcsToIntersectUnifyRule
IntersectToIntersectUnifyRule
JoinOnCalcsToJoinUnifyRule
JoinOnLeftCalcToJoinUnifyRule
JoinOnRightCalcToJoinUnifyRule
ScanToCalcUnifyRule
TrivialRule
UnionOnCalcsToUnionUnifyRule
UnionToUnionUnifyRule {code}
Their aims:
* Predicate compensation by pulling-up
* Project compensation by pulling-up
* Aggregate roll-up
But, Rewriting query fails, when top rel of mv is calc.
And this rel just works for rearrangement of field's order or unused value
filling.
Let me give some examples:
Case1
{code:java}
--query
SELECT deptno,
sum(salary),
sum(k)
FROM
(SELECT deptno,
salary,
100 AS k
FROM emps)
GROUP BY deptno;
--mv
SELECT deptno,
sum(salary),
sum(commission) + 1,
sum(k)
FROM
(SELECT deptno,
salary,
commission,
100 AS k
FROM emps)
GROUP BY deptno;
--query's plan
LogicalAggregate(group=[{0}], EXPR$1=[SUM($1)], EXPR$2=[SUM($2)])
LogicalCalc(expr#0..4=[{inputs}], expr#5=[100], deptno=[$t1], salary=[$t3],
k=[$t5])
LogicalTableScan(table=[[hr, emps]])
--mv's plan
LogicalCalc(expr#0..3=[{inputs}], expr#4=[1], expr#5=[+($t2, $t4)],
proj#0..1=[{exprs}], EXPR$2=[$t5], EXPR$3=[$t3])
LogicalAggregate(group=[{0}], EXPR$1=[SUM($1)], agg#1=[SUM($2)],
EXPR$3=[SUM($3)])
LogicalCalc(expr#0..4=[{inputs}], expr#5=[100], deptno=[$t1], salary=[$t3],
commission=[$t4], k=[$t5])
LogicalTableScan(table=[[hr, emps]]) {code}
Case2
{code:java}
--query
SELECT deptno,
salary
FROM emps
UNION
SELECT deptno,
salary
FROM emps;
--mv
SELECT deptno,
salary,
'hello' AS k
FROM
(SELECT deptno,
salary
FROM emps
UNION SELECT deptno,
salary
FROM emps);
--query's plan
LogicalUnion(all=[false])
LogicalCalc(expr#0..4=[{inputs}], deptno=[$t1], salary=[$t3])
LogicalTableScan(table=[[hr, emps]])
LogicalCalc(expr#0..4=[{inputs}], deptno=[$t1], salary=[$t3])
LogicalTableScan(table=[[hr, emps]])
--mv's plan
LogicalCalc(expr#0..1=[{inputs}], expr#2=['hello'], proj#0..2=[{exprs}])
LogicalUnion(all=[false])
LogicalCalc(expr#0..4=[{inputs}], deptno=[$t1], salary=[$t3])
LogicalTableScan(table=[[hr, emps]])
LogicalCalc(expr#0..4=[{inputs}], deptno=[$t1], salary=[$t3])
LogicalTableScan(table=[[hr, emps]]) {code}
We could define a new rule of HolderToCalcUnifyRule to solve these problems
Related issues:
* CALCITE-5000: https://issues.apache.org/jira/browse/CALCITE-5000
* CALCITE-4376: https://issues.apache.org/jira/browse/CALCITE-4376
Holder is top wrapped rel for query, link to
`org.apache.calcite.rel.mutable.Holder`
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (CALCITE-5012) SubstitutionVisitor need a new rule to support remain top calc in mv.
[
https://issues.apache.org/jira/browse/CALCITE-5012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xurenhe updated CALCITE-5012:
-
Description:
SubstitutionVisitor is a bottom-up to rewrite query's plan by mv.
Rewriting query successfully, when mv's plan has become a part of the query
plan.
The current rules are as follows:
{code:java}
AggregateOnCalcToAggregateUnifyRule
AggregateToAggregateUnifyRule
CalcToCalcUnifyRule
IntersectOnCalcsToIntersectUnifyRule
IntersectToIntersectUnifyRule
JoinOnCalcsToJoinUnifyRule
JoinOnLeftCalcToJoinUnifyRule
JoinOnRightCalcToJoinUnifyRule
ScanToCalcUnifyRule
TrivialRule
UnionOnCalcsToUnionUnifyRule
UnionToUnionUnifyRule {code}
Their aims:
* Predicate compensation by pulling-up
* Project compensation by pulling-up
* Aggregate roll-up
But, Rewriting query fails, when top rel of mv is calc.
And this rel just works for rearrangement of field's order or unused value
filling.
Let me give some examples:
*Case1:*
{code:java}
--query
SELECT deptno,
sum(salary),
sum(k)
FROM
(SELECT deptno,
salary,
100 AS k
FROM emps)
GROUP BY deptno;
--mv
SELECT deptno,
sum(salary),
sum(commission) + 1,
sum(k)
FROM
(SELECT deptno,
salary,
commission,
100 AS k
FROM emps)
GROUP BY deptno;
--query's plan
LogicalAggregate(group=[{0}], EXPR$1=[SUM($1)], EXPR$2=[SUM($2)])
LogicalCalc(expr#0..4=[{inputs}], expr#5=[100], deptno=[$t1], salary=[$t3],
k=[$t5])
LogicalTableScan(table=[[hr, emps]])
--mv's plan
LogicalCalc(expr#0..3=[{inputs}], expr#4=[1], expr#5=[+($t2, $t4)],
proj#0..1=[{exprs}], EXPR$2=[$t5], EXPR$3=[$t3])
LogicalAggregate(group=[{0}], EXPR$1=[SUM($1)], agg#1=[SUM($2)],
EXPR$3=[SUM($3)])
LogicalCalc(expr#0..4=[{inputs}], expr#5=[100], deptno=[$t1], salary=[$t3],
commission=[$t4], k=[$t5])
LogicalTableScan(table=[[hr, emps]]) {code}
*Case2:*
{code:java}
--query
SELECT deptno,
salary
FROM emps
UNION
SELECT deptno,
salary
FROM emps;
--mv
SELECT deptno,
salary,
'hello' AS k
FROM
(SELECT deptno,
salary
FROM emps
UNION SELECT deptno,
salary
FROM emps);
--query's plan
LogicalUnion(all=[false])
LogicalCalc(expr#0..4=[{inputs}], deptno=[$t1], salary=[$t3])
LogicalTableScan(table=[[hr, emps]])
LogicalCalc(expr#0..4=[{inputs}], deptno=[$t1], salary=[$t3])
LogicalTableScan(table=[[hr, emps]])
--mv's plan
LogicalCalc(expr#0..1=[{inputs}], expr#2=['hello'], proj#0..2=[{exprs}])
LogicalUnion(all=[false])
LogicalCalc(expr#0..4=[{inputs}], deptno=[$t1], salary=[$t3])
LogicalTableScan(table=[[hr, emps]])
LogicalCalc(expr#0..4=[{inputs}], deptno=[$t1], salary=[$t3])
LogicalTableScan(table=[[hr, emps]]) {code}
We could define a new rule of HolderToCalcUnifyRule to solve these problems
Related issues:
* CALCITE-5000: https://issues.apache.org/jira/browse/CALCITE-5000
* CALCITE-4376: https://issues.apache.org/jira/browse/CALCITE-4376
Holder is top wrapped rel for query, link to
`org.apache.calcite.rel.mutable.Holder`
was:
SubstitutionVisitor is a bottom-up to rewrite query's plan by mv.
Rewriting query successfully, when mv's plan has become a part of the query
plan.
The current rules are as follows:
{code:java}
AggregateOnCalcToAggregateUnifyRule
AggregateToAggregateUnifyRule
CalcToCalcUnifyRule
IntersectOnCalcsToIntersectUnifyRule
IntersectToIntersectUnifyRule
JoinOnCalcsToJoinUnifyRule
JoinOnLeftCalcToJoinUnifyRule
JoinOnRightCalcToJoinUnifyRule
ScanToCalcUnifyRule
TrivialRule
UnionOnCalcsToUnionUnifyRule
UnionToUnionUnifyRule {code}
Their aims:
* Predicate compensation by pulling-up
* Project compensation by pulling-up
* Aggregate roll-up
But, Rewriting query fails, when top rel of mv is calc.
And this rel just works for rearrangement of field's order or unused value
filling.
Let me give some examples:
Case1
{code:java}
--query
SELECT deptno,
sum(salary),
sum(k)
FROM
(SELECT deptno,
salary,
100 AS k
FROM emps)
GROUP BY deptno;
--mv
SELECT deptno,
sum(salary),
sum(commission) + 1,
sum(k)
FROM
(SELECT deptno,
salary,
commission,
100 AS k
FROM emps)
GROUP BY deptno;
--query's plan
LogicalAggregate(group=[{0}], EXPR$1=[SUM($1)], EXPR$2=[SUM($2)])
LogicalCalc(expr#0..4=[{inputs}], expr#5=[100], deptno=[$t1], salary=[$t3],
k=[$t5])
LogicalTableScan(table=[[hr, emps]])
--mv's plan
LogicalCalc(expr#0..3=[{inputs}], expr#4=[1], expr#5=[+($t2, $t4)],
proj#0..1=[{exprs}], EXPR$2=[$t5], EXPR$3=[$t3])
LogicalAggregate(group=[{0}], EXPR$1=[SUM($1)], agg#1=[SUM($2)],
EXPR$3=[SUM($3)])
LogicalCalc(expr#0..4=[{inputs}], expr#5=[100], deptno=[$t1], salary=[$t3],
commission=[$t4], k=[$t5])
LogicalTableScan(table=[[hr, emps]]) {code}
Case2
{code:java}
--query
SELECT deptno,
salary
FRO
