[jira] [Commented] (CALCITE-3975) ProjectFilterTransposeRule should succeed for project that happens to reference all input columns
[
https://issues.apache.org/jira/browse/CALCITE-3975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17117317#comment-17117317
]
Julian Hyde commented on CALCITE-3975:
--
It's not the 'dynamic star' that [~zabetak] suspects. It is the fact that we
all columns are referenced.
The permutation is essential - rules have to produce something equivalent to
their input, and have no way to say that they permuted the columns except to
generate the inverse permutation.
However I think we can detect the permutation case, i.e. narrow the criteria
that would prevent this rule from firing.
> ProjectFilterTransposeRule should succeed for project that happens to
> reference all input columns
> -
>
> Key: CALCITE-3975
> URL: https://issues.apache.org/jira/browse/CALCITE-3975
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.22.0
>Reporter: Steven Talbot
>Assignee: Julian Hyde
>Priority: Major
>
> ... that is, I think
> If I make the trivial fix of just "only skip trivial projects", something
> like
> {noformat}
> && origProj.getProjects().stream().allMatch((proj) -> proj instanceof
> RexInputRef) {noformat}
> at
> [https://github.com/apache/calcite/blob/571731b80a58eb095ebac7123285c375e7afff90/core/src/main/java/org/apache/calcite/rel/rules/PushProjector.java#L354
>
> |https://github.com/apache/calcite/blob/571731b80a58eb095ebac7123285c375e7afff90/core/src/main/java/org/apache/calcite/rel/rules/PushProjector.java#L354]HepPlanner
> goes into infinite recursion with the rule.
> But here's the test case:
>
> {code:java}
> @Test public void testPushProjectPastFilter3() {
> final String sql = "select empno + deptno, ename, job, mgr, hiredate, sal,
> comm, slacker from emp where sal = 10 * comm\n"
> + "and upper(ename) = 'FOO'";
> sql(sql).withRule(ProjectFilterTransposeRule.INSTANCE).check();
> }
> {code}
>
>
>
> {noformat}
>
>
>
>
>
>
>
>
>
>
> {noformat}
>
> There's no reason the rule shouldn't succeed here, right? Or am I missing
> something?
> The reason this rule goes into an infinite recursion with hepplanner is
> because it sticks a project on top after transpose to handle common
> expressions extracted from the filter and the project. Ideally, it could have
> a mode where it could avoid doing that and do a true "transpose" if there was
> no need for it. For example, I don't think there is a a need for a reproject
> on top in this test case: you can just transpose and everything works as it
> should. This would be another way to avoid infinite recursion.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3975) ProjectFilterTransposeRule should succeed for project that happens to reference all input columns
[
https://issues.apache.org/jira/browse/CALCITE-3975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated CALCITE-3975:
-
Affects Version/s: 1.22.0
> ProjectFilterTransposeRule should succeed for project that happens to
> reference all input columns
> -
>
> Key: CALCITE-3975
> URL: https://issues.apache.org/jira/browse/CALCITE-3975
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.22.0
>Reporter: Steven Talbot
>Assignee: Julian Hyde
>Priority: Major
>
> ... that is, I think
> If I make the trivial fix of just "only skip trivial projects", something
> like
> {noformat}
> && origProj.getProjects().stream().allMatch((proj) -> proj instanceof
> RexInputRef) {noformat}
> at
> [https://github.com/apache/calcite/blob/571731b80a58eb095ebac7123285c375e7afff90/core/src/main/java/org/apache/calcite/rel/rules/PushProjector.java#L354
>
> |https://github.com/apache/calcite/blob/571731b80a58eb095ebac7123285c375e7afff90/core/src/main/java/org/apache/calcite/rel/rules/PushProjector.java#L354]HepPlanner
> goes into infinite recursion with the rule.
> But here's the test case:
>
> {code:java}
> @Test public void testPushProjectPastFilter3() {
> final String sql = "select empno + deptno, ename, job, mgr, hiredate, sal,
> comm, slacker from emp where sal = 10 * comm\n"
> + "and upper(ename) = 'FOO'";
> sql(sql).withRule(ProjectFilterTransposeRule.INSTANCE).check();
> }
> {code}
>
>
>
> {noformat}
>
>
>
>
>
>
>
>
>
>
> {noformat}
>
> There's no reason the rule shouldn't succeed here, right? Or am I missing
> something?
> The reason this rule goes into an infinite recursion with hepplanner is
> because it sticks a project on top after transpose to handle common
> expressions extracted from the filter and the project. Ideally, it could have
> a mode where it could avoid doing that and do a true "transpose" if there was
> no need for it. For example, I don't think there is a a need for a reproject
> on top in this test case: you can just transpose and everything works as it
> should. This would be another way to avoid infinite recursion.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (CALCITE-3975) ProjectFilterTransposeRule should succeed for project that happens to reference all input columns
[
https://issues.apache.org/jira/browse/CALCITE-3975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde reassigned CALCITE-3975:
Assignee: Julian Hyde
> ProjectFilterTransposeRule should succeed for project that happens to
> reference all input columns
> -
>
> Key: CALCITE-3975
> URL: https://issues.apache.org/jira/browse/CALCITE-3975
> Project: Calcite
> Issue Type: Bug
>Reporter: Steven Talbot
>Assignee: Julian Hyde
>Priority: Major
>
> ... that is, I think
> If I make the trivial fix of just "only skip trivial projects", something
> like
> {noformat}
> && origProj.getProjects().stream().allMatch((proj) -> proj instanceof
> RexInputRef) {noformat}
> at
> [https://github.com/apache/calcite/blob/571731b80a58eb095ebac7123285c375e7afff90/core/src/main/java/org/apache/calcite/rel/rules/PushProjector.java#L354
>
> |https://github.com/apache/calcite/blob/571731b80a58eb095ebac7123285c375e7afff90/core/src/main/java/org/apache/calcite/rel/rules/PushProjector.java#L354]HepPlanner
> goes into infinite recursion with the rule.
> But here's the test case:
>
> {code:java}
> @Test public void testPushProjectPastFilter3() {
> final String sql = "select empno + deptno, ename, job, mgr, hiredate, sal,
> comm, slacker from emp where sal = 10 * comm\n"
> + "and upper(ename) = 'FOO'";
> sql(sql).withRule(ProjectFilterTransposeRule.INSTANCE).check();
> }
> {code}
>
>
>
> {noformat}
>
>
>
>
>
>
>
>
>
>
> {noformat}
>
> There's no reason the rule shouldn't succeed here, right? Or am I missing
> something?
> The reason this rule goes into an infinite recursion with hepplanner is
> because it sticks a project on top after transpose to handle common
> expressions extracted from the filter and the project. Ideally, it could have
> a mode where it could avoid doing that and do a true "transpose" if there was
> no need for it. For example, I don't think there is a a need for a reproject
> on top in this test case: you can just transpose and everything works as it
> should. This would be another way to avoid infinite recursion.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3946) Add parser support for MULTISET/SET and VOLATILE modifiers in CREATE TABLE statements
[
https://issues.apache.org/jira/browse/CALCITE-3946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated CALCITE-3946:
-
Fix Version/s: 1.24.0
> Add parser support for MULTISET/SET and VOLATILE modifiers in CREATE TABLE
> statements
> -
>
> Key: CALCITE-3946
> URL: https://issues.apache.org/jira/browse/CALCITE-3946
> Project: Calcite
> Issue Type: Improvement
> Components: babel
>Affects Versions: 1.22.0
>Reporter: Drew Schmitt
>Assignee: Julian Hyde
>Priority: Major
> Fix For: 1.24.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Add support to Calcite's Babel parser for {{MULTISET}}/{{SET}} and
> {{VOLATILE}} modifiers in {{CREATE TABLE}} statements.
> The syntax for these statements is:
> {code:sql}
> CREATE TABLE [SET|MULTISET] [VOLATILE]
>[IF NOT EXISTS] ( , ...);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (CALCITE-3946) Add parser support for MULTISET/SET and VOLATILE modifiers in CREATE TABLE statements
[
https://issues.apache.org/jira/browse/CALCITE-3946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde reassigned CALCITE-3946:
Assignee: Julian Hyde
> Add parser support for MULTISET/SET and VOLATILE modifiers in CREATE TABLE
> statements
> -
>
> Key: CALCITE-3946
> URL: https://issues.apache.org/jira/browse/CALCITE-3946
> Project: Calcite
> Issue Type: Improvement
> Components: babel
>Affects Versions: 1.22.0
>Reporter: Drew Schmitt
>Assignee: Julian Hyde
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Add support to Calcite's Babel parser for {{MULTISET}}/{{SET}} and
> {{VOLATILE}} modifiers in {{CREATE TABLE}} statements.
> The syntax for these statements is:
> {code:sql}
> CREATE TABLE [SET|MULTISET] [VOLATILE]
>[IF NOT EXISTS] ( , ...);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4026) When the cassandra primary key is of type bigint,he query will report an error
taojiayun created CALCITE-4026: -- Summary: When the cassandra primary key is of type bigint,he query will report an error Key: CALCITE-4026 URL: https://issues.apache.org/jira/browse/CALCITE-4026 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.22.0 Reporter: taojiayun Fix For: next select "id","apps_info" from "dws_in_flow" where "id" = 1. This is the query，id is the primary key and is of type bigint, but the execution is wrong . java.sql.SQLException: Error while executing SQL "select "id" from "dws_in_flow" where "id" = 1": Invalid STRING constant (1) for "id" of type bigint at org.apache.calcite.avatica.Helper.createException(Helper.java:56) at org.apache.calcite.avatica.Helper.createException(Helper.java:41) at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163) at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227) at io.terminus.dice.fdp.service.utils.CalciteUtilTest.testCassandra(CalciteUtilTest.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) Caused by: com.datastax.driver.core.exceptions.InvalidQueryException: Invalid STRING constant (1) for "id" of type bigint at com.datastax.driver.core.exceptions.InvalidQueryException.copy(InvalidQueryException.java:50) at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37) at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245) at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68) at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:43) at org.apache.calcite.adapter.cassandra.CassandraTable$2.enumerator(CassandraTable.java:195) at org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33) at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:90) at org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:184) at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:64) at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:43) at org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:667) at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:566) at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675) at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156) ... 24 more Caused by: com.datastax.driver.core.exceptions.InvalidQueryException: Invalid STRING constant (1) for "id" of type bigint at com.datastax.driver.core.Responses$Error.asException(Responses.java:147) at com.datastax.driver.core.DefaultResultSetFuture.onSet(DefaultResultSetFuture.java:179) at
[jira] [Commented] (CALCITE-4025) DelegatingScope#fullyQualify may exit too soon when resolving table alias
[
https://issues.apache.org/jira/browse/CALCITE-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17117160#comment-17117160
]
Laurent Goujon commented on CALCITE-4025:
-
Thanks for helping me on the the test case, I'll try to implement your
suggestion.
> DelegatingScope#fullyQualify may exit too soon when resolving table alias
> -
>
> Key: CALCITE-4025
> URL: https://issues.apache.org/jira/browse/CALCITE-4025
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Laurent Goujon
>Priority: Major
>
> When fully qualifing an id with {{DelegatingScope#fullyQualify()}} method,
> there's a look going over all the possible sub prefixes and trying to resolve
> them, until one succeed. But if not, the same lookup is done using a liberal
> name matcher. If that one resolves, then the method exit early with an
> exception message instead of going over the whole loop or trying for a fully
> qualified table.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (CALCITE-4025) DelegatingScope#fullyQualify may exit too soon when resolving table alias
[
https://issues.apache.org/jira/browse/CALCITE-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laurent Goujon reassigned CALCITE-4025:
---
Assignee: Laurent Goujon
> DelegatingScope#fullyQualify may exit too soon when resolving table alias
> -
>
> Key: CALCITE-4025
> URL: https://issues.apache.org/jira/browse/CALCITE-4025
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Laurent Goujon
>Assignee: Laurent Goujon
>Priority: Major
>
> When fully qualifing an id with {{DelegatingScope#fullyQualify()}} method,
> there's a look going over all the possible sub prefixes and trying to resolve
> them, until one succeed. But if not, the same lookup is done using a liberal
> name matcher. If that one resolves, then the method exit early with an
> exception message instead of going over the whole loop or trying for a fully
> qualified table.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4025) DelegatingScope#fullyQualify may exit too soon when resolving table alias
[
https://issues.apache.org/jira/browse/CALCITE-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17117157#comment-17117157
]
Julian Hyde commented on CALCITE-4025:
--
I'd like to see a test case. Looking up qualified names is complicated, because
if you have "FROM x.y.z" as a table then later you are allowed to use the
fully-qualified table name as if it were a table alias, e.g. "SELECT x.y.z.c"
and "SELECT z.c" are both valid.
> DelegatingScope#fullyQualify may exit too soon when resolving table alias
> -
>
> Key: CALCITE-4025
> URL: https://issues.apache.org/jira/browse/CALCITE-4025
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Laurent Goujon
>Priority: Major
>
> When fully qualifing an id with {{DelegatingScope#fullyQualify()}} method,
> there's a look going over all the possible sub prefixes and trying to resolve
> them, until one succeed. But if not, the same lookup is done using a liberal
> name matcher. If that one resolves, then the method exit early with an
> exception message instead of going over the whole loop or trying for a fully
> qualified table.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4025) DelegatingScope#fullyQualify may exit too soon when resolving table alias
[
https://issues.apache.org/jira/browse/CALCITE-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17117149#comment-17117149
]
Laurent Goujon commented on CALCITE-4025:
-
I mentioned it as a possible issue on the Calcite mailing list but didn't get
any feedback on it:
https://mail-archives.apache.org/mod_mbox/calcite-dev/202005.mbox/%3CCAGQXjMwWr%2Bot_-rjTzo6dG1are3DM%3DWp3vQYNsB640JTSGgLbQ%40mail.gmail.com%3E
> DelegatingScope#fullyQualify may exit too soon when resolving table alias
> -
>
> Key: CALCITE-4025
> URL: https://issues.apache.org/jira/browse/CALCITE-4025
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Laurent Goujon
>Priority: Major
>
> When fully qualifing an id with {{DelegatingScope#fullyQualify()}} method,
> there's a look going over all the possible sub prefixes and trying to resolve
> them, until one succeed. But if not, the same lookup is done using a liberal
> name matcher. If that one resolves, then the method exit early with an
> exception message instead of going over the whole loop or trying for a fully
> qualified table.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4025) DelegatingScope#fullyQualify may exit too soon when resolving table alias
[
https://issues.apache.org/jira/browse/CALCITE-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laurent Goujon updated CALCITE-4025:
Issue Type: Bug (was: Improvement)
> DelegatingScope#fullyQualify may exit too soon when resolving table alias
> -
>
> Key: CALCITE-4025
> URL: https://issues.apache.org/jira/browse/CALCITE-4025
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Laurent Goujon
>Priority: Major
>
> When fully qualifing an id with {{DelegatingScope#fullyQualify()}} method,
> there's a look going over all the possible sub prefixes and trying to resolve
> them, until one succeed. But if not, the same lookup is done using a liberal
> name matcher. If that one resolves, then the method exit early with an
> exception message instead of going over the whole loop or trying for a fully
> qualified table.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4025) DelegatingScope#fullyQualify may exit too soon when resolving table alias
Laurent Goujon created CALCITE-4025:
---
Summary: DelegatingScope#fullyQualify may exit too soon when
resolving table alias
Key: CALCITE-4025
URL: https://issues.apache.org/jira/browse/CALCITE-4025
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: Laurent Goujon
When fully qualifing an id with {{DelegatingScope#fullyQualify()}} method,
there's a look going over all the possible sub prefixes and trying to resolve
them, until one succeed. But if not, the same lookup is done using a liberal
name matcher. If that one resolves, then the method exit early with an
exception message instead of going over the whole loop or trying for a fully
qualified table.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-4014) calcite convert(column using utf8mb4) throws exception
[ https://issues.apache.org/jira/browse/CALCITE-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde resolved CALCITE-4014. -- Resolution: Duplicate Duplicate of CALCITE-111. > calcite convert(column using utf8mb4) throws exception > -- > > Key: CALCITE-4014 > URL: https://issues.apache.org/jira/browse/CALCITE-4014 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.21.0 >Reporter: groobyming >Priority: Major > Attachments: image-2020-05-20-17-00-28-611.png > > > Hello everyone, I found that when performing the following sql > verification, I will be prompted that the UTF8MB4 field cannot be found in > the table, but after reading the source code calcite is built-in to support > this convert function, is it not fully implemented yet? > > SQL is as follows: > select convert(name using utf8mb4) from test b where name = '4' > > Error Message: > !image-2020-05-20-17-00-28-611.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4014) calcite convert(column using utf8mb4) throws exception
[
https://issues.apache.org/jira/browse/CALCITE-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17117023#comment-17117023
]
Julian Hyde commented on CALCITE-4014:
--
Yes, we support {{CONVERT}} in the parser (see
{{SqlParserTest.testConvertAndTranslate}}) but not in the validator (where this
error is happening - validating an identifier as if it is an expression, not a
character set) or downstream (note that {{SqlOperatorBaseTest.testConvertFunc}}
is empty).
Contributions welcome!
> calcite convert(column using utf8mb4) throws exception
> --
>
> Key: CALCITE-4014
> URL: https://issues.apache.org/jira/browse/CALCITE-4014
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.21.0
>Reporter: groobyming
>Priority: Major
> Attachments: image-2020-05-20-17-00-28-611.png
>
>
> Hello everyone, I found that when performing the following sql
> verification, I will be prompted that the UTF8MB4 field cannot be found in
> the table, but after reading the source code calcite is built-in to support
> this convert function, is it not fully implemented yet?
>
> SQL is as follows:
> select convert(name using utf8mb4) from test b where name = '4'
>
> Error Message:
> !image-2020-05-20-17-00-28-611.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-4014) calcite convert(column using utf8mb4) throws exception
[
https://issues.apache.org/jira/browse/CALCITE-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17112734#comment-17112734
]
Julian Hyde edited comment on CALCITE-4014 at 5/26/20, 8:16 PM:
Thanks, [~groobyming] for reporting this ~
I saw the syntax we support from the parser:
{noformat}
{ s = span(); }
e = Expression(ExprContext.ACCEPT_SUB_QUERY) {
args = startList(e);
}
name = SimpleIdentifier() {
args.add(name);
}
{
return SqlStdOperatorTable.CONVERT.createCall(s.end(this), args);
}
{noformat}
That means we did support CONVERT( ... USING ...) clause. But you are right, we
do not translate the charSet correctly, there needs more effort in the convert
table [1], we need to translate the charSet as a constant instead of a column
name.
[1]
https://github.com/apache/calcite/blob/05376d6bff11ccef690addb4f9c40ab16dd20bec/core/src/main/java/org/apache/calcite/rex/RexSqlStandardConvertletTable.java#L103
was (Author: danny0405):
Thanks, [~groobyming] for reporting this ~
I saw the syntax we support from the parser:
```sql
{ s = span(); }
e = Expression(ExprContext.ACCEPT_SUB_QUERY) {
args = startList(e);
}
name = SimpleIdentifier() {
args.add(name);
}
{
return SqlStdOperatorTable.CONVERT.createCall(s.end(this), args);
}
```
That means we did support CONVERT( ... USING ...) clause. But you are right, we
do not translate the charSet correctly, there needs more effort in the convert
table [1], we need to translate the charSet as a constant instead of a column
name.
[1]
https://github.com/apache/calcite/blob/05376d6bff11ccef690addb4f9c40ab16dd20bec/core/src/main/java/org/apache/calcite/rex/RexSqlStandardConvertletTable.java#L103
> calcite convert(column using utf8mb4) throws exception
> --
>
> Key: CALCITE-4014
> URL: https://issues.apache.org/jira/browse/CALCITE-4014
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.21.0
>Reporter: groobyming
>Priority: Major
> Attachments: image-2020-05-20-17-00-28-611.png
>
>
> Hello everyone, I found that when performing the following sql
> verification, I will be prompted that the UTF8MB4 field cannot be found in
> the table, but after reading the source code calcite is built-in to support
> this convert function, is it not fully implemented yet?
>
> SQL is as follows:
> select convert(name using utf8mb4) from test b where name = '4'
>
> Error Message:
> !image-2020-05-20-17-00-28-611.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-4009) Revert traitset mapping that was added to ProjectJoinTransposeRule
[ https://issues.apache.org/jira/browse/CALCITE-4009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan resolved CALCITE-4009. Resolution: Fixed Fixed in https://github.com/apache/calcite/commit/2b1254bdec18f9e869f3287fb8ab471903e97829. > Revert traitset mapping that was added to ProjectJoinTransposeRule > -- > > Key: CALCITE-4009 > URL: https://issues.apache.org/jira/browse/CALCITE-4009 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Haisheng Yuan >Assignee: Haisheng Yuan >Priority: Major > Fix For: 1.24.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Revert traitset mapping that was added to ProjectJoinTransposeRule by > CALCITE-3353. Now it becomes obsolete, we should fail fast if that happens. > Otherwise, all the downstream projects that uses this rule will be wasted > time to deal with traitsets they don't need. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3989) Release Calcite 1.23.0
[ https://issues.apache.org/jira/browse/CALCITE-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116964#comment-17116964 ] Haisheng Yuan commented on CALCITE-3989: See mailing list discussion: https://lists.apache.org/thread.html/r1b0cf72ff0235feb2b8c734d7132bff13a8bea83184006f01904da9a%40%3Cdev.calcite.apache.org%3E > Release Calcite 1.23.0 > -- > > Key: CALCITE-3989 > URL: https://issues.apache.org/jira/browse/CALCITE-3989 > Project: Calcite > Issue Type: Task >Reporter: Haisheng Yuan >Assignee: Haisheng Yuan >Priority: Major > Fix For: 1.23.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3989) Release Calcite 1.23.0
[ https://issues.apache.org/jira/browse/CALCITE-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116963#comment-17116963 ] Haisheng Yuan commented on CALCITE-3989: [~mamo], I have created a ticket for this: https://issues.apache.org/jira/browse/INFRA-20326 > Release Calcite 1.23.0 > -- > > Key: CALCITE-3989 > URL: https://issues.apache.org/jira/browse/CALCITE-3989 > Project: Calcite > Issue Type: Task >Reporter: Haisheng Yuan >Assignee: Haisheng Yuan >Priority: Major > Fix For: 1.23.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3972) Allow RelBuilder to create RelNode with convention and use it for trait convert
[
https://issues.apache.org/jira/browse/CALCITE-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116960#comment-17116960
]
Haisheng Yuan commented on CALCITE-3972:
Agree with [~rkondakov]. My take is more radical, {{LogicalSort}} should not
exist at all. Enforcers should only exist in physical world, not logical world.
> Allow RelBuilder to create RelNode with convention and use it for trait
> convert
> ---
>
> Key: CALCITE-3972
> URL: https://issues.apache.org/jira/browse/CALCITE-3972
> Project: Calcite
> Issue Type: Bug
>Reporter: Xiening Dai
>Assignee: Xiening Dai
>Priority: Major
> Time Spent: 2h
> Remaining Estimate: 0h
>
> 1. Provide Convention.transformRelBuilder() to transform an existing
> RelBuilder into one with specific convention.
> 2. RelBuilder provides withRelFactories() method to allow caller swap the
> underlying RelFactories and create a new builder.
> 3. Use the new interface in RelCollationTraitDef for converting into
> RelCollation traits
> We can avoid ~1/3 of total rule firings in a N way join case with this change.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3989) Release Calcite 1.23.0
[ https://issues.apache.org/jira/browse/CALCITE-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116959#comment-17116959 ] Magnus Mogren commented on CALCITE-3989: [~hyuan] i cannot find the 1.23.0 release in Maven Central > Release Calcite 1.23.0 > -- > > Key: CALCITE-3989 > URL: https://issues.apache.org/jira/browse/CALCITE-3989 > Project: Calcite > Issue Type: Task >Reporter: Haisheng Yuan >Assignee: Haisheng Yuan >Priority: Major > Fix For: 1.23.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-3972) Allow RelBuilder to create RelNode with convention and use it for trait convert
[ https://issues.apache.org/jira/browse/CALCITE-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116954#comment-17116954 ] Roman Kondakov edited comment on CALCITE-3972 at 5/26/20, 6:24 PM: --- I agree with [~hyuan]. I also think we should not register logical converters (LogicalSort without limit and offset or LogicalExchange) in the planner's MEMO at all. As it implemented in my pet cascades-style optimizer. It helps to avoid not-needed multiple RelSets merging and rules firing. All physical properties should be enforced by physical converters only at the final step of Group (RelSet) optimization. was (Author: rkondakov): I agree with [~hyuan]. I also think we should not register logical converters (LogicalSort or LogicalExchange) in the planner's MEMO at all. As it implemented in my pet cascades-style optimizer. It helps to avoid not-needed multiple RelSets merging and rules firing. All physical properties should be enforced by physical converters only at the final step of Group (RelSet) optimization. > Allow RelBuilder to create RelNode with convention and use it for trait > convert > --- > > Key: CALCITE-3972 > URL: https://issues.apache.org/jira/browse/CALCITE-3972 > Project: Calcite > Issue Type: Bug >Reporter: Xiening Dai >Assignee: Xiening Dai >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > 1. Provide Convention.transformRelBuilder() to transform an existing > RelBuilder into one with specific convention. > 2. RelBuilder provides withRelFactories() method to allow caller swap the > underlying RelFactories and create a new builder. > 3. Use the new interface in RelCollationTraitDef for converting into > RelCollation traits > We can avoid ~1/3 of total rule firings in a N way join case with this change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3972) Allow RelBuilder to create RelNode with convention and use it for trait convert
[ https://issues.apache.org/jira/browse/CALCITE-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116954#comment-17116954 ] Roman Kondakov commented on CALCITE-3972: - I agree with [~hyuan]. I also think we should not register logical converters (LogicalSort or LogicalExchange) in the planner's MEMO at all. As it implemented in my pet cascades-style optimizer. It helps to avoid not-needed multiple RelSets merging and rules firing. All physical properties should be enforced by physical converters only at the final step of Group (RelSet) optimization. > Allow RelBuilder to create RelNode with convention and use it for trait > convert > --- > > Key: CALCITE-3972 > URL: https://issues.apache.org/jira/browse/CALCITE-3972 > Project: Calcite > Issue Type: Bug >Reporter: Xiening Dai >Assignee: Xiening Dai >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > 1. Provide Convention.transformRelBuilder() to transform an existing > RelBuilder into one with specific convention. > 2. RelBuilder provides withRelFactories() method to allow caller swap the > underlying RelFactories and create a new builder. > 3. Use the new interface in RelCollationTraitDef for converting into > RelCollation traits > We can avoid ~1/3 of total rule firings in a N way join case with this change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4023) Remove or deprecate ProjectSortTransposeRule
[
https://issues.apache.org/jira/browse/CALCITE-4023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116914#comment-17116914
]
Haisheng Yuan commented on CALCITE-4023:
The rule is useless.
> Remove or deprecate ProjectSortTransposeRule
>
>
> Key: CALCITE-4023
> URL: https://issues.apache.org/jira/browse/CALCITE-4023
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Haisheng Yuan
>Priority: Major
>
> It never worked. The check condition {{if (sort.getClass() != Sort.class)}}
> is always true.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3972) Allow RelBuilder to create RelNode with convention and use it for trait convert
[
https://issues.apache.org/jira/browse/CALCITE-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116911#comment-17116911
]
Haisheng Yuan commented on CALCITE-3972:
The fact that Sort can participate rule matching is the culprit.
Sort changes the relation's physical property, but doesn't change the logical
property.
Use
[testSortJoinTranspose2|https://github.com/apache/calcite/commit/0715f5b55f363a58e3dd8c20caac0024e19be413#diff-de15ea9da479ca31d38de70365967392R4070]
as example,
{code:java}
Before:
LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8], DEPTNO0=[$9], NAME=[$10])
LogicalSort(sort0=[$10], dir0=[ASC])
LogicalJoin(condition=[=($7, $9)], joinType=[right])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
LogicalProject(DEPTNO=[$0], NAME=[$1])
LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
After:
LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8], DEPTNO0=[$9], NAME=[$10])
LogicalSort(sort0=[$10], dir0=[ASC])
LogicalJoin(condition=[=($7, $9)], joinType=[right])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
LogicalSort(sort0=[$1], dir0=[ASC])
LogicalProject(DEPTNO=[$0], NAME=[$1])
LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
{code}
If we combine the sort with any operator in the original plan, the logical
properties are all the same. After the rule execution, LogicalJoin has a new
right input, even the sort in the right input can be the same RelSet as
LogicalProject (unfortunately it isn't), the new join right input changed (the
logical join is requesting a collation on right input), the join's digest
changed, it will be viewed as a whole new join, then will apply all the logical
transformations that it can apply.
Although the rule above only applies on outer join, the same problem happens on
SortProjectTransposeRule.
Now come back to the problem in JDBCTest.testJoinManyWays(),
JoinPushThroughJoinRule's one rule operand is RelNode.class, which means any
new node in the join's input RelSet will trigger the rule. But in this rule, we
don't care about what exact relnode it is, we just want the whole group as a
placeholder. Any new logical sort, physical sort, and abstract converter will
all trigger the matches of JoinPushThroughJoinRule. This is extremely
unnecessary.
If we change
{code:java}
operand(RelNode.class, any())),
{code}
to
{code:java}
operandJ(RelNode.class, null, n -> !n.isEnforcer(), any())),
{code}
It will achieve the same effect as generating EnumerableSort directly, but
still generating LogicalSort in RelCollationTraitDef, without affecting rules
like, SortProjectTranspose, SortJoinTranspose, SortJoinCopy.
The total rule apply count of JoinPushThroughJoinRule cut from 9000 to 900,
reduced by 90%. This will again reduce the ProjectMergeRule a lot, because
every join reorder generate at least a new LogicalProject in Calcite.
Now the rule count is:
{code:java}
Rules
Attempts Time (us)
ProjectMergeRule:force_mode
14,064 2,680,177
EnumerableProjectRule(in:NONE,out:ENUMERABLE)
974 271,608
JoinPushThroughJoinRule:left
449 209,768
JoinPushThroughJoinRule:right
449 2,949
AggregatePullUpConstantsRule
291 17,947
AggregateProjectMergeRule
277 83,288
ProjectFilterTransposeRule
207 30,300
EnumerableJoinRule(in:NONE,out:ENUMERABLE)
108 70,179
EnumerableMergeJoinRule(in:NONE,out:ENUMERABLE)
108 46,111
JoinPushExpressionsRule
108 10,807
{code}
> Allow RelBuilder to create RelNode with convention and use it for trait
> convert
> ---
>
> Key: CALCITE-3972
> URL: https://issues.apache.org/jira/browse/CALCITE-3972
> Project: Calcite
> Issue Type: Bug
>Reporter: Xiening Dai
>Assignee: Xiening Dai
>Priority: Major
> Time Spent: 2h
> Remaining Estimate: 0h
>
> 1. Provide Convention.transformRelBuilder() to transform an existing
> RelBuilder into one with specific convention.
> 2. RelBuilder provides withRelFactories() method to allow caller swap the
> underlying RelFactories
[jira] [Commented] (CALCITE-4023) Remove or deprecate ProjectSortTransposeRule
[
https://issues.apache.org/jira/browse/CALCITE-4023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116882#comment-17116882
]
Xiening Dai commented on CALCITE-4023:
--
Is this a bug to fix or the rule is just useless?
> Remove or deprecate ProjectSortTransposeRule
>
>
> Key: CALCITE-4023
> URL: https://issues.apache.org/jira/browse/CALCITE-4023
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Haisheng Yuan
>Priority: Major
>
> It never worked. The check condition {{if (sort.getClass() != Sort.class)}}
> is always true.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4001) Should support dialect when unparsing SqlDataTypeSpec
[ https://issues.apache.org/jira/browse/CALCITE-4001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116815#comment-17116815 ] Xurenhe commented on CALCITE-4001: -- Sorry for so late reply. That's exactly what I want. THX a lot~ [~julianhyde] > Should support dialect when unparsing SqlDataTypeSpec > - > > Key: CALCITE-4001 > URL: https://issues.apache.org/jira/browse/CALCITE-4001 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Xurenhe >Priority: Minor > > SqlBasicTypeNameSpec cannot cover all data types of other sql's engine, when > doing SqlNode To SqlString, > such as cast(id as bigint), but some sql-engine only support cast(id as long). > I think there should be based on dialect, just like > org.apache.calcite.sql.SqlDialect#unparseOffsetFetch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (CALCITE-4001) Should support dialect when unparsing SqlDataTypeSpec
[ https://issues.apache.org/jira/browse/CALCITE-4001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xurenhe closed CALCITE-4001. Resolution: Not A Problem > Should support dialect when unparsing SqlDataTypeSpec > - > > Key: CALCITE-4001 > URL: https://issues.apache.org/jira/browse/CALCITE-4001 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Xurenhe >Priority: Minor > > SqlBasicTypeNameSpec cannot cover all data types of other sql's engine, when > doing SqlNode To SqlString, > such as cast(id as bigint), but some sql-engine only support cast(id as long). > I think there should be based on dialect, just like > org.apache.calcite.sql.SqlDialect#unparseOffsetFetch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4024) In top-down optimizer, forbid Sort (non-limit) to participate any rule matches
Haisheng Yuan created CALCITE-4024: -- Summary: In top-down optimizer, forbid Sort (non-limit) to participate any rule matches Key: CALCITE-4024 URL: https://issues.apache.org/jira/browse/CALCITE-4024 Project: Calcite Issue Type: Improvement Components: core Reporter: Haisheng Yuan In top-down optimizer, forbid enforcer operator, e.g. Sort (non-limit) to participate any rule matches. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4023) Remove or deprecate ProjectSortTransposeRule
Haisheng Yuan created CALCITE-4023:
--
Summary: Remove or deprecate ProjectSortTransposeRule
Key: CALCITE-4023
URL: https://issues.apache.org/jira/browse/CALCITE-4023
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: Haisheng Yuan
It never worked. The check condition {{if (sort.getClass() != Sort.class)}} is
always true.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3951) Support different string comparison based on SqlCollation
[ https://issues.apache.org/jira/browse/CALCITE-3951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruben Q L updated CALCITE-3951: --- Fix Version/s: 1.24.0 > Support different string comparison based on SqlCollation > - > > Key: CALCITE-3951 > URL: https://issues.apache.org/jira/browse/CALCITE-3951 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Ruben Q L >Assignee: Ruben Q L >Priority: Major > Labels: pull-request-available > Fix For: 1.24.0 > > Time Spent: 3h > Remaining Estimate: 0h > > Currently SqlCollation defines concepts like Coercibility, Charset, Locale, > etc. However, we cannot specify on a certain collation that e.g. a string > field should use case insensitive comparison. The goal of this ticket is to > evolve SqlCollation to support that, and adapt the corresponding classes to > use that (optional) "non-standard" comparison. -- This message was sent by Atlassian Jira (v8.3.4#803005)
