[jira] [Commented] (CALCITE-5553) Calcite RelStructuredTypeFlattener produces bad plan for single field structs
[
https://issues.apache.org/jira/browse/CALCITE-5553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697172#comment-17697172
]
Andrew Pilloud commented on CALCITE-5553:
-
I think I have a workable fix now, which involves checking to ensure
`getType().isStruct()` is consistent for the fields before and after the
flatterer runs.
> Calcite RelStructuredTypeFlattener produces bad plan for single field structs
> -
>
> Key: CALCITE-5553
> URL: https://issues.apache.org/jira/browse/CALCITE-5553
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0, 1.28.0, 1.33.0
>Reporter: Andrew Pilloud
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> A Beam user brought us a query that produces an invalid plan after
> upgrading to a newer version of Beam (moving them from Calcite 1.20 to
> 1.28). I was able to write a test case which demonstrates this issue
> with a trivial filter on a table containing a nested struct with a
> single field. The issue appears to be coming out of
> RelStructuredTypeFlattener.rewrite and I bisected it to a single
> commit,
> [e44beba28|https://github.com/apache/calcite/commit/e44beba286ea9049c5fd00c3a3b0e4a4f1c03356].
> Removing the noFlatteningForInput function added in this commit fixes
> the issue. That method doesn't appear to be looking at types at all,
> only mismatches in field count, I expect so any single field struct
> would hit this. Any suggestions on how we can work around this in
> Beam?
> The test query and plans are as follows, the test case is in [PR
> 3092|https://github.com/apache/calcite/pull/3092].
> {code}
> select dn.skill from sales.dept_single dn WHERE dn.skill.type = ''
> {code}
> Expected plan:
> {noformat}
> LogicalProject(SKILL=[ROW($0)])
> LogicalFilter(condition=[=($0, '')])
> LogicalProject(TYPE=[$0.TYPE])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT_SINGLE]])
> {noformat}
> Actual Plan:
> {noformat}
> LogicalProject(SKILL=[ROW($0)])
> LogicalFilter(condition=[=($0.TYPE, '')])
> LogicalProject(TYPE=[$0.TYPE])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT_SINGLE]])
> {noformat}
> Plan before flatten:
> {noformat}
> LogicalProject(SKILL=[$0])
> LogicalFilter(condition=[=($0.TYPE, '')])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT_SINGLE]])
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5553) Calcite RelStructuredTypeFlattener produces bad plan for single field structs
Andrew Pilloud created CALCITE-5553: --- Summary: Calcite RelStructuredTypeFlattener produces bad plan for single field structs Key: CALCITE-5553 URL: https://issues.apache.org/jira/browse/CALCITE-5553 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.33.0, 1.28.0, 1.23.0 Reporter: Andrew Pilloud A Beam user brought us a query that produces an invalid plan after upgrading to a newer version of Beam (moving them from Calcite 1.20 to 1.28). I was able to write a test case which demonstrates this issue with a trivial filter on a table containing a nested struct with a single field. The issue appears to be coming out of RelStructuredTypeFlattener.rewrite and I bisected it to a single commit: https://github.com/apache/calcite/commit/e44beba286ea9049c5fd00c3a3b0e4a4f1c03356 Removing the noFlatteningForInput function added in this commit fixes the issue. That method doesn't appear to be looking at types at all, only mismatches in field count, I expect so any single field struct would hit this. Any suggestions on how we can work around this in Beam? The test query and plans are as follows, the test case is here https://github.com/apache/calcite/pull/3092 select dn.skill from sales.dept_single dn WHERE dn.skill.type = '' Expected plan: LogicalProject(SKILL=[ROW($0)]) LogicalFilter(condition=[=($0, '')]) LogicalProject(TYPE=[$0.TYPE]) LogicalTableScan(table=[[CATALOG, SALES, DEPT_SINGLE]]) Actual Plan: LogicalProject(SKILL=[ROW($0)]) LogicalFilter(condition=[=($0.TYPE, '')]) LogicalProject(TYPE=[$0.TYPE]) LogicalTableScan(table=[[CATALOG, SALES, DEPT_SINGLE]]) Plan before flatten: LogicalProject(SKILL=[$0]) LogicalFilter(condition=[=($0.TYPE, '')]) LogicalTableScan(table=[[CATALOG, SALES, DEPT_SINGLE]]) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-4759) JoinPushThroughJoinRule does not match two joins
Andrew Pilloud created CALCITE-4759: --- Summary: JoinPushThroughJoinRule does not match two joins Key: CALCITE-4759 URL: https://issues.apache.org/jira/browse/CALCITE-4759 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.25.0 Reporter: Andrew Pilloud JoinPushThroughJoinRule doesn't ever get to the onMatch() condition, even when presented with a plan with two back to back joins. I found this as a test failure while upgrading Beam from Calcite 1.20 to 1.25. Looks like this was introduced by https://github.com/apache/calcite/pull/2024, I was able to workaround the issue by patching in the match rule prior to this change: https://github.com/apache/beam/pull/14729/commits/22994a0c093c91f3af58fc4a7824e8971a449907 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4408) Implement Oracle SUBSTR function
[
https://issues.apache.org/jira/browse/CALCITE-4408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17238393#comment-17238393
]
Andrew Pilloud commented on CALCITE-4408:
-
I believe CALCITE-2572 was implemented to the standard, however I don't believe
Oracle matches the standard.
> Implement Oracle SUBSTR function
>
>
> Key: CALCITE-4408
> URL: https://issues.apache.org/jira/browse/CALCITE-4408
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: James Starr
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Oracle SUBSTR function throws an exception when matching operand exception.
>
> {code:java}
> public abstract class SqlOperatorBaseTest {
> ...
> @Test void testSubString() {
> SqlTester t = tester(SqlLibrary.ORACLE);
> t.setFor(SqlLibraryOperators.SUBSTR);
> t.checkString("substr(CAST('abc' AS varchar(3)), 1, 3)", "ab",
> "VARCHAR(3) NOT NULL");
> t.checkString("substr(CAST('abc' AS varchar(3)), 1)", "abc", "VARCHAR(3)
> NOT NULL");
> }
> ...
> {code}
> throws
> {noformat}
> class org.apache.calcite.sql.SqlFunction: SUBSTR
> java.lang.UnsupportedOperationException: class
> org.apache.calcite.sql.SqlFunction: SUBSTR
> at org.apache.calcite.util.Util.needToImplement(Util.java:1080)
> at
> org.apache.calcite.sql.SqlOperator.getOperandCountRange(SqlOperator.java:203)
> at
> org.apache.calcite.sql.SqlUtil.lambda$filterRoutinesByParameterCount$3(SqlUtil.java:610)
> at com.google.common.collect.Iterators$5.computeNext(Iterators.java:637)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
> at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
> at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
> at com.google.common.collect.Iterators.addAll(Iterators.java:355)
> at com.google.common.collect.Lists.newArrayList(Lists.java:143)
> at
> org.apache.calcite.sql.SqlUtil.lookupSubjectRoutines(SqlUtil.java:541)
> at org.apache.calcite.sql.SqlUtil.lookupRoutine(SqlUtil.java:470)
> at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:248)
> at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:226)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5933)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5920)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:138)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1775)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1760)
> at org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:275)
> at org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:467)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5629)
> at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:115)
> at org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:274)
> at org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:467)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5629)
> at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:115)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1024)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:732)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.parseAndValidate(AbstractSqlTester.java:177)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.getResultType(AbstractSqlTester.java:165)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.getColumnType(AbstractSqlTester.java:157)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.check(AbstractSqlTester.java:479)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.check(AbstractSqlTester.java:464)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.checkString(AbstractSqlTester.java:449)
> at
>
[jira] [Commented] (CALCITE-4051) SESSION TVF Optimizations
[ https://issues.apache.org/jira/browse/CALCITE-4051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17128725#comment-17128725 ] Andrew Pilloud commented on CALCITE-4051: - Did you mean to file this on Beam? > SESSION TVF Optimizations > - > > Key: CALCITE-4051 > URL: https://issues.apache.org/jira/browse/CALCITE-4051 > Project: Calcite > Issue Type: Improvement >Reporter: Rui Wang >Priority: Major > > Current TVF Session has the following limitation: > Use a "dummy" key to go through a GBK to compute window_start and window_end. > This introduces an extra shuffle so it costs more. > One optimizations we can do is always merge TableFunctionScanRel with > AggregationRel, thus we can use only one GBK to compute both SESSION > window_start and window_end, and aggregations. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3714) BitString.createFromBytes reverses order of the bits in each byte
Andrew Pilloud created CALCITE-3714: --- Summary: BitString.createFromBytes reverses order of the bits in each byte Key: CALCITE-3714 URL: https://issues.apache.org/jira/browse/CALCITE-3714 Project: Calcite Issue Type: Bug Components: core Reporter: Andrew Pilloud BitString.createFromBytes reverses the order of the bits in each byte. (Passing in 'a' I expect 0111 but get 1110.) BitString.createFromHexString works as expected, and appears to be the one everyone is using. The "fix" might be to just delete the broken method. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3472) SESSION_END returns same value as SESSION_START
[ https://issues.apache.org/jira/browse/CALCITE-3472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16966865#comment-16966865 ] Andrew Pilloud commented on CALCITE-3472: - [~amaliujia] wrote a nice description of this problem in BEAM-5799. I don't have permissions to resolve this issue, but it is a duplicate of CALCITE-2645 > SESSION_END returns same value as SESSION_START > --- > > Key: CALCITE-3472 > URL: https://issues.apache.org/jira/browse/CALCITE-3472 > Project: Calcite > Issue Type: Improvement > Components: stream >Reporter: Pablo Estrada >Priority: Major > > We've found that the session_end function seems to be returning the same > values as session_start. Furthermore, it seems that they may have the same > implementation, though I'm not fully familiar with the code: > > [https://github.com/apache/calcite/blob/d7946a94adfd2e788f5d324910944dd65dab11ee/core/src/main/java/org/apache/calcite/sql2rel/AuxiliaryConverter.java#L50-L68] > > An example with Beam SQL: > > 0: BeamSQL> SELECT > SESSION_START(scores_stream.event_time, INTERVAL '1' SECOND), > SESSION_END(scores_stream.event_time, INTERVAL '1' SECOND), > scores_stream.team, > SUM(scores_stream.score), > COUNT(*) > FROM > scores_stream > GROUP BY scores_stream.team, > SESSION(scores_stream.event_time, INTERVAL '1' SECOND) LIMIT 3;SELECT > . . . . . > SESSION_START(scores_stream.event_time, INTERVAL '1' SECOND), > . . . . . > SESSION_END(scores_stream.event_time, INTERVAL '1' SECOND), > . . . . . > scores_stream.team, > . . . . . > SUM(scores_stream.score), > . . . . . > COUNT(*) > . . . . . > FROM > . . . . . > scores_stream > . . . . . > GROUP BY scores_stream.team, > . . . . . > > T 3; SESSION(scores_stream.event_time, INTERVAL '1' SECOND) LIMI > +++--++-+ > | EXPR$0 | EXPR$1 | team | EXPR$3 | EXPR$4 | > +++--++-+ > | 2019-11-04 04:11:38 | 2019-11-04 04:11:38 | blue | 420 | 7 | > | 2019-11-04 04:11:38 | 2019-11-04 04:11:38 | red | 960 | 18 | > | 2019-11-04 04:11:42 | 2019-11-04 04:11:42 | blue | 452 | 11 | > +++--++-+ > 3 rows selected (9.197 seconds) > 0: BeamSQL> -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (CALCITE-3040) Avatica drops exception messages starting in 1.14.0
[
https://issues.apache.org/jira/browse/CALCITE-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Pilloud closed CALCITE-3040.
---
> Avatica drops exception messages starting in 1.14.0
> ---
>
> Key: CALCITE-3040
> URL: https://issues.apache.org/jira/browse/CALCITE-3040
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.14.0
> Environment: Beam 2.13.0-SNAPSHOT
> Avatica 1.14.0
> [https://github.com/apache/beam/pull/8437]
>Reporter: Andrew Pilloud
>Priority: Major
> Fix For: avatica-1.15.0
>
>
> With Avatica 1.13.0 we get detailed exceptions:
> {code:java}
> Welcome to Beam SQL 2.13.0-SNAPSHOT (based on sqlline version 1.4.0)
> 0: BeamSQL> create external table testing (a int) type text location
> '/tmp/test';
> No rows affected (0.246 seconds)
> 0: BeamSQL> set runner = bogus;
> No rows affected (0.009 seconds)
> 0: BeamSQL> select * from testing;
> Error: Error while executing SQL "select * from testing": Unknown 'runner'
> specified 'bogus', supported pipeline runners [DirectRunner] (state=,code=0)
> 0: BeamSQL>
> {code}
> In 1.14.0 the details of the failure are missing:
> {code:java}
> Welcome to Beam SQL 2.13.0-SNAPSHOT (based on sqlline version 1.4.0)
> 0: BeamSQL> create external table testing (a int) type text location
> '/tmp/test';
> No rows affected (0.244 seconds)
> 0: BeamSQL> set runner = bogus;
> No rows affected (0.009 seconds)
> 0: BeamSQL> select * from testing;
> Error: Error while executing SQL "select * from testing" (state=,code=0)
> 0: BeamSQL>
> {code}
> This appears to be caused by CALCITE-2845
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3040) Avatica drops exception messages starting in 1.14.0
[
https://issues.apache.org/jira/browse/CALCITE-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16836502#comment-16836502
]
Andrew Pilloud commented on CALCITE-3040:
-
Verified with all the clients.
> Avatica drops exception messages starting in 1.14.0
> ---
>
> Key: CALCITE-3040
> URL: https://issues.apache.org/jira/browse/CALCITE-3040
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.14.0
> Environment: Beam 2.13.0-SNAPSHOT
> Avatica 1.14.0
> [https://github.com/apache/beam/pull/8437]
>Reporter: Andrew Pilloud
>Priority: Major
> Fix For: avatica-1.15.0
>
>
> With Avatica 1.13.0 we get detailed exceptions:
> {code:java}
> Welcome to Beam SQL 2.13.0-SNAPSHOT (based on sqlline version 1.4.0)
> 0: BeamSQL> create external table testing (a int) type text location
> '/tmp/test';
> No rows affected (0.246 seconds)
> 0: BeamSQL> set runner = bogus;
> No rows affected (0.009 seconds)
> 0: BeamSQL> select * from testing;
> Error: Error while executing SQL "select * from testing": Unknown 'runner'
> specified 'bogus', supported pipeline runners [DirectRunner] (state=,code=0)
> 0: BeamSQL>
> {code}
> In 1.14.0 the details of the failure are missing:
> {code:java}
> Welcome to Beam SQL 2.13.0-SNAPSHOT (based on sqlline version 1.4.0)
> 0: BeamSQL> create external table testing (a int) type text location
> '/tmp/test';
> No rows affected (0.244 seconds)
> 0: BeamSQL> set runner = bogus;
> No rows affected (0.009 seconds)
> 0: BeamSQL> select * from testing;
> Error: Error while executing SQL "select * from testing" (state=,code=0)
> 0: BeamSQL>
> {code}
> This appears to be caused by CALCITE-2845
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3040) Avatica drops exception messages starting in 1.14.0
[
https://issues.apache.org/jira/browse/CALCITE-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16831091#comment-16831091
]
Andrew Pilloud commented on CALCITE-3040:
-
Thanks for turning this around so quickly! Sorry I didn't test the 1.14.0
release before it went out, I'll try to keep on top of that in the future.
> Avatica drops exception messages starting in 1.14.0
> ---
>
> Key: CALCITE-3040
> URL: https://issues.apache.org/jira/browse/CALCITE-3040
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.14.0
> Environment: Beam 2.13.0-SNAPSHOT
> Avatica 1.14.0
> [https://github.com/apache/beam/pull/8437]
>Reporter: Andrew Pilloud
>Priority: Major
> Fix For: avatica-1.15.0
>
>
> With Avatica 1.13.0 we get detailed exceptions:
> {code:java}
> Welcome to Beam SQL 2.13.0-SNAPSHOT (based on sqlline version 1.4.0)
> 0: BeamSQL> create external table testing (a int) type text location
> '/tmp/test';
> No rows affected (0.246 seconds)
> 0: BeamSQL> set runner = bogus;
> No rows affected (0.009 seconds)
> 0: BeamSQL> select * from testing;
> Error: Error while executing SQL "select * from testing": Unknown 'runner'
> specified 'bogus', supported pipeline runners [DirectRunner] (state=,code=0)
> 0: BeamSQL>
> {code}
> In 1.14.0 the details of the failure are missing:
> {code:java}
> Welcome to Beam SQL 2.13.0-SNAPSHOT (based on sqlline version 1.4.0)
> 0: BeamSQL> create external table testing (a int) type text location
> '/tmp/test';
> No rows affected (0.244 seconds)
> 0: BeamSQL> set runner = bogus;
> No rows affected (0.009 seconds)
> 0: BeamSQL> select * from testing;
> Error: Error while executing SQL "select * from testing" (state=,code=0)
> 0: BeamSQL>
> {code}
> This appears to be caused by CALCITE-2845
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3040) Avatica drops exception messages starting in 1.14.0
[
https://issues.apache.org/jira/browse/CALCITE-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16831089#comment-16831089
]
Andrew Pilloud commented on CALCITE-3040:
-
Our tests are replicating the behavior of common JDBC clients. This is not
something that can be fixed by changing a single test or client.
> Avatica drops exception messages starting in 1.14.0
> ---
>
> Key: CALCITE-3040
> URL: https://issues.apache.org/jira/browse/CALCITE-3040
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.14.0
> Environment: Beam 2.13.0-SNAPSHOT
> Avatica 1.14.0
> [https://github.com/apache/beam/pull/8437]
>Reporter: Andrew Pilloud
>Priority: Major
> Fix For: avatica-1.15.0
>
>
> With Avatica 1.13.0 we get detailed exceptions:
> {code:java}
> Welcome to Beam SQL 2.13.0-SNAPSHOT (based on sqlline version 1.4.0)
> 0: BeamSQL> create external table testing (a int) type text location
> '/tmp/test';
> No rows affected (0.246 seconds)
> 0: BeamSQL> set runner = bogus;
> No rows affected (0.009 seconds)
> 0: BeamSQL> select * from testing;
> Error: Error while executing SQL "select * from testing": Unknown 'runner'
> specified 'bogus', supported pipeline runners [DirectRunner] (state=,code=0)
> 0: BeamSQL>
> {code}
> In 1.14.0 the details of the failure are missing:
> {code:java}
> Welcome to Beam SQL 2.13.0-SNAPSHOT (based on sqlline version 1.4.0)
> 0: BeamSQL> create external table testing (a int) type text location
> '/tmp/test';
> No rows affected (0.244 seconds)
> 0: BeamSQL> set runner = bogus;
> No rows affected (0.009 seconds)
> 0: BeamSQL> select * from testing;
> Error: Error while executing SQL "select * from testing" (state=,code=0)
> 0: BeamSQL>
> {code}
> This appears to be caused by CALCITE-2845
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3040) Avatica drops exception messages starting in 1.14.0
[
https://issues.apache.org/jira/browse/CALCITE-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830674#comment-16830674
]
Andrew Pilloud commented on CALCITE-3040:
-
SQuirreL SQL before:
{code:java}
Error: Error while executing SQL "select * from testing": Unknown 'runner'
specified 'bogus', supported pipeline runners [DirectRunner]
SQLState: null
ErrorCode: 0
{code}
After:
{code:java}
Error: Error while executing SQL "SELECT * from testing"
SQLState: null
ErrorCode: 0
{code}
> Avatica drops exception messages starting in 1.14.0
> ---
>
> Key: CALCITE-3040
> URL: https://issues.apache.org/jira/browse/CALCITE-3040
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.14.0
> Environment: Beam 2.13.0-SNAPSHOT
> Avatica 1.14.0
> [https://github.com/apache/beam/pull/8437]
>Reporter: Andrew Pilloud
>Priority: Major
>
> With Avatica 1.13.0 we get detailed exceptions:
> {code:java}
> Welcome to Beam SQL 2.13.0-SNAPSHOT (based on sqlline version 1.4.0)
> 0: BeamSQL> create external table testing (a int) type text location
> '/tmp/test';
> No rows affected (0.246 seconds)
> 0: BeamSQL> set runner = bogus;
> No rows affected (0.009 seconds)
> 0: BeamSQL> select * from testing;
> Error: Error while executing SQL "select * from testing": Unknown 'runner'
> specified 'bogus', supported pipeline runners [DirectRunner] (state=,code=0)
> 0: BeamSQL>
> {code}
> In 1.14.0 the details of the failure are missing:
> {code:java}
> Welcome to Beam SQL 2.13.0-SNAPSHOT (based on sqlline version 1.4.0)
> 0: BeamSQL> create external table testing (a int) type text location
> '/tmp/test';
> No rows affected (0.244 seconds)
> 0: BeamSQL> set runner = bogus;
> No rows affected (0.009 seconds)
> 0: BeamSQL> select * from testing;
> Error: Error while executing SQL "select * from testing" (state=,code=0)
> 0: BeamSQL>
> {code}
> This appears to be caused by CALCITE-2845
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3040) Avatica drops exception messages starting in 1.14.0
[
https://issues.apache.org/jira/browse/CALCITE-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830670#comment-16830670
]
Andrew Pilloud commented on CALCITE-3040:
-
This is not a SQLLine specific issue. It occurs with many common JDBC UIs. For
example, SQL Workbench/J Before:
{code:java}
An error occurred when executing the SQL command:
select * from testing
Error while executing SQL "select * from testing": Unknown 'runner' specified
'bogus', supported pipeline runners [DirectRunner]
1 statement failed.
Execution time: 0.67s
{code}
After:
{code:java}
An error occurred when executing the SQL command:
select * from testing
Error while executing SQL "select * from testing"
1 statement failed.
Execution time: 0.71s
{code}
> Avatica drops exception messages starting in 1.14.0
> ---
>
> Key: CALCITE-3040
> URL: https://issues.apache.org/jira/browse/CALCITE-3040
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.14.0
> Environment: Beam 2.13.0-SNAPSHOT
> Avatica 1.14.0
> [https://github.com/apache/beam/pull/8437]
>Reporter: Andrew Pilloud
>Priority: Major
>
> With Avatica 1.13.0 we get detailed exceptions:
> {code:java}
> Welcome to Beam SQL 2.13.0-SNAPSHOT (based on sqlline version 1.4.0)
> 0: BeamSQL> create external table testing (a int) type text location
> '/tmp/test';
> No rows affected (0.246 seconds)
> 0: BeamSQL> set runner = bogus;
> No rows affected (0.009 seconds)
> 0: BeamSQL> select * from testing;
> Error: Error while executing SQL "select * from testing": Unknown 'runner'
> specified 'bogus', supported pipeline runners [DirectRunner] (state=,code=0)
> 0: BeamSQL>
> {code}
> In 1.14.0 the details of the failure are missing:
> {code:java}
> Welcome to Beam SQL 2.13.0-SNAPSHOT (based on sqlline version 1.4.0)
> 0: BeamSQL> create external table testing (a int) type text location
> '/tmp/test';
> No rows affected (0.244 seconds)
> 0: BeamSQL> set runner = bogus;
> No rows affected (0.009 seconds)
> 0: BeamSQL> select * from testing;
> Error: Error while executing SQL "select * from testing" (state=,code=0)
> 0: BeamSQL>
> {code}
> This appears to be caused by CALCITE-2845
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-3040) Avatica drops exception messages starting in 1.14.0
[
https://issues.apache.org/jira/browse/CALCITE-3040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Pilloud updated CALCITE-3040:
Environment:
Beam 2.13.0-SNAPSHOT
Avatica 1.14.0
[https://github.com/apache/beam/pull/8437]
was:
SQLline 1.4.0
Beam 2.13.0-SNAPSHOT
Avatica 1.14.0
[https://github.com/apache/beam/pull/8437]
> Avatica drops exception messages starting in 1.14.0
> ---
>
> Key: CALCITE-3040
> URL: https://issues.apache.org/jira/browse/CALCITE-3040
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.14.0
> Environment: Beam 2.13.0-SNAPSHOT
> Avatica 1.14.0
> [https://github.com/apache/beam/pull/8437]
>Reporter: Andrew Pilloud
>Priority: Major
>
> With Avatica 1.13.0 we get detailed exceptions:
> {code:java}
> Welcome to Beam SQL 2.13.0-SNAPSHOT (based on sqlline version 1.4.0)
> 0: BeamSQL> create external table testing (a int) type text location
> '/tmp/test';
> No rows affected (0.246 seconds)
> 0: BeamSQL> set runner = bogus;
> No rows affected (0.009 seconds)
> 0: BeamSQL> select * from testing;
> Error: Error while executing SQL "select * from testing": Unknown 'runner'
> specified 'bogus', supported pipeline runners [DirectRunner] (state=,code=0)
> 0: BeamSQL>
> {code}
> In 1.14.0 the details of the failure are missing:
> {code:java}
> Welcome to Beam SQL 2.13.0-SNAPSHOT (based on sqlline version 1.4.0)
> 0: BeamSQL> create external table testing (a int) type text location
> '/tmp/test';
> No rows affected (0.244 seconds)
> 0: BeamSQL> set runner = bogus;
> No rows affected (0.009 seconds)
> 0: BeamSQL> select * from testing;
> Error: Error while executing SQL "select * from testing" (state=,code=0)
> 0: BeamSQL>
> {code}
> This appears to be caused by CALCITE-2845
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-3040) Avatica drops exception messages starting in 1.14.0
Andrew Pilloud created CALCITE-3040:
---
Summary: Avatica drops exception messages starting in 1.14.0
Key: CALCITE-3040
URL: https://issues.apache.org/jira/browse/CALCITE-3040
Project: Calcite
Issue Type: Bug
Components: avatica
Affects Versions: avatica-1.14.0
Environment: SQLline 1.4.0
Beam 2.13.0-SNAPSHOT
Avatica 1.14.0
[https://github.com/apache/beam/pull/8437]
Reporter: Andrew Pilloud
With Avatica 1.13.0 we get detailed exceptions:
{code:java}
Welcome to Beam SQL 2.13.0-SNAPSHOT (based on sqlline version 1.4.0)
0: BeamSQL> create external table testing (a int) type text location
'/tmp/test';
No rows affected (0.246 seconds)
0: BeamSQL> set runner = bogus;
No rows affected (0.009 seconds)
0: BeamSQL> select * from testing;
Error: Error while executing SQL "select * from testing": Unknown 'runner'
specified 'bogus', supported pipeline runners [DirectRunner] (state=,code=0)
0: BeamSQL>
{code}
In 1.14.0 the details of the failure are missing:
{code:java}
Welcome to Beam SQL 2.13.0-SNAPSHOT (based on sqlline version 1.4.0)
0: BeamSQL> create external table testing (a int) type text location
'/tmp/test';
No rows affected (0.244 seconds)
0: BeamSQL> set runner = bogus;
No rows affected (0.009 seconds)
0: BeamSQL> select * from testing;
Error: Error while executing SQL "select * from testing" (state=,code=0)
0: BeamSQL>
{code}
This appears to be caused by CALCITE-2845
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2783) Null checks incorrectly optimized by RexSimplfy.simplifyBooleanCase.
Andrew Pilloud created CALCITE-2783:
---
Summary: Null checks incorrectly optimized by
RexSimplfy.simplifyBooleanCase.
Key: CALCITE-2783
URL: https://issues.apache.org/jira/browse/CALCITE-2783
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.18.0
Environment: Using Apache Beam
1ad4affb450809842670df0434e12d953af31e166. (See BEAM-6401).
Does not reproduce on Calcite 064974d543874e47a9da8ea0595824dde398bee1.
Does reproduce on Calcite b470a0cd4572c9f6c4c0e9b51926b97c5af58d3f, 1.18, and
master@62b47aeeb7eeb59beaf5b8f3b54a5c58ba4ca76d.
(This is introduced by CALCITE-1413)
Reporter: Andrew Pilloud
Assignee: Julian Hyde
This first appears in 1.18, and is introduced by CALCITE-1413.
I have a table "table" with column "f0" of nullable booleans. Table contains
the following values:
{code:java}
true
false
null
{code}
Working query:
{code:java}
SELECT coalesce(f0, true) FROM table;
true
false
true
{code}
Broken query:
{code:java}
SELECT coalesce(f0, true) = true FROM table;
Caused by: java.lang.NullPointerException
at
org.apache.calcite.runtime.SqlFunctions.cannotConvert(SqlFunctions.java:1457)
at org.apache.calcite.runtime.SqlFunctions.toBoolean(SqlFunctions.java:1480)
at SC.eval0(Unknown Source)
{code}
Broken generated code:
{code:java}
{
final Boolean inp0_ = ((Row) c.element()).getBoolean(0);
c.output(Row.withSchema(outputSchema).addValue((SqlFunctions.isTrue(inp0_) ||
inp0_ == null ? Boolean.TRUE : inp0_ == null ? (Boolean) null : Boolean.FALSE)
== null ? (Boolean) null : Boolean.valueOf(SqlFunctions.toBoolean(inp0_) ||
inp0_ == null)).build());
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (CALCITE-2710) Some SqlFunctions use LOCAL_TZ constant instead of using timeZone()
[ https://issues.apache.org/jira/browse/CALCITE-2710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Pilloud closed CALCITE-2710. --- Resolution: Invalid > Some SqlFunctions use LOCAL_TZ constant instead of using timeZone() > --- > > Key: CALCITE-2710 > URL: https://issues.apache.org/jira/browse/CALCITE-2710 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Andrew Pilloud >Assignee: Julian Hyde >Priority: Major > > There are several methods in SqlFunctions which use a LOCAL_TZ constant to > determine the file. This constant is the default system time zone, but the > local time zone can be modified on the connection. This results in incorrect > behavior if the system time zone and connection time zone differ, for example > adding an offset and removing a different offset. These functions should be > calling the timeZone method which fetches the local timezone from the > DataContext. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2710) Some SqlFunctions use LOCAL_TZ constant instead of using timeZone()
[ https://issues.apache.org/jira/browse/CALCITE-2710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16701121#comment-16701121 ] Andrew Pilloud commented on CALCITE-2710: - Yes, I see that now. (Passing UDFs time in local timezone values seems a little strange, but if it converts back it should work.) So what it looks like I'm seeing is that ReflectiveCallNotNullImplementor converts from internal values to these types but doesn't convert back to the internal types. I have a custom 'castOutput' function that converts Calcite types back to Beam rows, if I send the java.sql types through the appropriate conversion back the issues I am seeing goes away. So the bug is on my end, not Calcite. > Some SqlFunctions use LOCAL_TZ constant instead of using timeZone() > --- > > Key: CALCITE-2710 > URL: https://issues.apache.org/jira/browse/CALCITE-2710 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Andrew Pilloud >Assignee: Julian Hyde >Priority: Major > > There are several methods in SqlFunctions which use a LOCAL_TZ constant to > determine the file. This constant is the default system time zone, but the > local time zone can be modified on the connection. This results in incorrect > behavior if the system time zone and connection time zone differ, for example > adding an offset and removing a different offset. These functions should be > calling the timeZone method which fetches the local timezone from the > DataContext. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2710) Some SqlFunctions use LOCAL_TZ constant instead of using timeZone()
[ https://issues.apache.org/jira/browse/CALCITE-2710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16699679#comment-16699679 ] Andrew Pilloud commented on CALCITE-2710: - I think this is just a plumbing issue, I plan to work on it shortly. > Some SqlFunctions use LOCAL_TZ constant instead of using timeZone() > --- > > Key: CALCITE-2710 > URL: https://issues.apache.org/jira/browse/CALCITE-2710 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Andrew Pilloud >Assignee: Julian Hyde >Priority: Major > > There are several methods in SqlFunctions which use a LOCAL_TZ constant to > determine the file. This constant is the default system time zone, but the > local time zone can be modified on the connection. This results in incorrect > behavior if the system time zone and connection time zone differ, for example > adding an offset and removing a different offset. These functions should be > calling the timeZone method which fetches the local timezone from the > DataContext. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2710) Some SqlFunctions use LOCAL_TZ constant instead of using timeZone()
Andrew Pilloud created CALCITE-2710: --- Summary: Some SqlFunctions use LOCAL_TZ constant instead of using timeZone() Key: CALCITE-2710 URL: https://issues.apache.org/jira/browse/CALCITE-2710 Project: Calcite Issue Type: Bug Components: core Reporter: Andrew Pilloud Assignee: Julian Hyde There are several methods in SqlFunctions which use a LOCAL_TZ constant to determine the file. This constant is the default system time zone, but the local time zone can be modified on the connection. This results in incorrect behavior if the system time zone and connection time zone differ, for example adding an offset and removing a different offset. These functions should be calling the timeZone method which fetches the local timezone from the DataContext. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2612) Cassandra tests fork and crash maven in Java 1.8
Andrew Pilloud created CALCITE-2612:
---
Summary: Cassandra tests fork and crash maven in Java 1.8
Key: CALCITE-2612
URL: https://issues.apache.org/jira/browse/CALCITE-2612
Project: Calcite
Issue Type: Bug
Components: cassandra
Affects Versions: 1.17.0
Environment: OpenJDK 1.8
Reporter: Andrew Pilloud
Cassandra tests appear to be forking and crashing maven as of
[https://github.com/apache/calcite/commit/bd0e140026c409aa932cebf0e1fe223ab3a5319a]
{code:java}
[INFO] ---
[INFO] T E S T S
[INFO] ---
2018-10-04 13:55:12 [pool-2-thread-1] ERROR o.a.cassandra.service.StartupChecks
- cassandra.jmx.local.port missing from cassandra-env.sh, unable to start local
JMX service.
[WARNING] Corrupted STDOUT by directly writing to native stream in forked JVM
1. See FAQ web page and the dump file
/home/apilloud/calcite/cassandra/target/surefire-reports/2018-10-04T13-49-53_586-jvmRun1.dumpstream
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] Reactor Summary:
[INFO]
[INFO] Calcite SUCCESS [ 2.369 s]
[INFO] Calcite Linq4j . SUCCESS [ 23.280 s]
[INFO] Calcite Core ... SUCCESS [04:56 min]
[INFO] Calcite Babel .. SUCCESS [ 5.472 s]
[INFO] Calcite Cassandra .. FAILURE [ 6.769 s]
[INFO] Calcite Druid .. SKIPPED
[INFO] Calcite Elasticsearch .. SKIPPED
[INFO] Calcite Examples ... SKIPPED
[INFO] Calcite Example CSV SKIPPED
[INFO] Calcite Example Function ... SKIPPED
[INFO] Calcite File ... SKIPPED
[INFO] Calcite Geode .. SKIPPED
[INFO] Calcite MongoDB SKIPPED
[INFO] Calcite Pig SKIPPED
[INFO] Calcite Piglet . SKIPPED
[INFO] Calcite Plus ... SKIPPED
[INFO] Calcite Server . SKIPPED
[INFO] Calcite Spark .. SKIPPED
[INFO] Calcite Splunk . SKIPPED
[INFO] Calcite Ubenchmark . SKIPPED
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 05:34 min
[INFO] Finished at: 2018-10-04T13:55:17-07:00
[INFO] Final Memory: 72M/974M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.21.0:test (default-test) on
project calcite-cassandra: There are test failures.
[ERROR]
[ERROR] Please refer to
/home/apilloud/calcite/cassandra/target/surefire-reports for the individual
test results.
[ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump,
[date].dumpstream and [date]-jvmRun[N].dumpstream.
[ERROR] The forked VM terminated without properly saying goodbye. VM crash or
System.exit called?
[ERROR] Command was /bin/sh -c cd /home/apilloud/calcite/cassandra && java
-Xmx1536m -jar
/home/apilloud/calcite/cassandra/target/surefire/surefirebooter2335040866535881101.jar
/home/apilloud/calcite/cassandra/target/surefire
2018-10-04T13-49-53_586-jvmRun1 surefire5656092402505825682tmp
surefire_38865239179641475436tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 134
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: The
forked VM terminated without properly saying goodbye. VM crash or System.exit
called?
[ERROR] Command was /bin/sh -c cd /home/apilloud/calcite/cassandra && java
-Xmx1536m -jar
/home/apilloud/calcite/cassandra/target/surefire/surefirebooter2335040866535881101.jar
/home/apilloud/calcite/cassandra/target/surefire
2018-10-04T13-49-53_586-jvmRun1 surefire5656092402505825682tmp
surefire_38865239179641475436tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 134
[ERROR] at
org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:671)
[ERROR] at
org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:533)
[ERROR] at
org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:278)
[ERROR] at
[jira] [Commented] (CALCITE-2529) linq4j should promote integer to floating point when generating function calls
[ https://issues.apache.org/jira/browse/CALCITE-2529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638771#comment-16638771 ] Andrew Pilloud commented on CALCITE-2529: - Thanks for taking a look, I've cleaned up the code as you suggested and it is much simpler now. In my quick scan I did not find any open issues for this specific case, but CALCITE-1467 covers another case not handled here. I've already looked into that one and the fix is much more involved. I will spend a little more time today looking for related Jira issues and scanning for redundant functions. > linq4j should promote integer to floating point when generating function calls > -- > > Key: CALCITE-2529 > URL: https://issues.apache.org/jira/browse/CALCITE-2529 > Project: Calcite > Issue Type: Improvement >Reporter: Andrew Pilloud >Assignee: Julian Hyde >Priority: Major > > For example, I get "RuntimeException: while resolving method 'atan2[double, > int]'" when trying to execute a query with "ATAN2(c_double, 2)" (where > c_double is a column containing the double type). atan2[double, double] > should resolve as a valid implementation in this case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2529) linq4j should promote integer to floating point when generating function calls
[ https://issues.apache.org/jira/browse/CALCITE-2529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637478#comment-16637478 ] Andrew Pilloud commented on CALCITE-2529: - Hi Julian, are you the best person to review this or is this something [~vladimirsitnikov] should look at? > linq4j should promote integer to floating point when generating function calls > -- > > Key: CALCITE-2529 > URL: https://issues.apache.org/jira/browse/CALCITE-2529 > Project: Calcite > Issue Type: Improvement >Reporter: Andrew Pilloud >Assignee: Julian Hyde >Priority: Major > > For example, I get "RuntimeException: while resolving method 'atan2[double, > int]'" when trying to execute a query with "ATAN2(c_double, 2)" (where > c_double is a column containing the double type). atan2[double, double] > should resolve as a valid implementation in this case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2571) TRIM does not match the behavior of most SQL implementations
[ https://issues.apache.org/jira/browse/CALCITE-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624205#comment-16624205 ] Andrew Pilloud commented on CALCITE-2571: - I couldn't figure out how to get at SqlConformance after the parser, but I agree there are probably better ways to do this. I tried passing the conformance flag from the parser but the only paths I could see involved adding another operator or plumbing it through as an operand. > TRIM does not match the behavior of most SQL implementations > > > Key: CALCITE-2571 > URL: https://issues.apache.org/jira/browse/CALCITE-2571 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Andrew Pilloud >Assignee: Julian Hyde >Priority: Major > > Most SQL implementations trim all characters if more then one is provided to > the trim function. Calcite's SqlFunctions implementation silently drops the > extra characters, resulting in data corruption. > Here is a sampling of implementations that trim all characters: > MySQL - > https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_trim > Sql Server - > https://docs.microsoft.com/en-us/sql/t-sql/functions/trim-transact-sql?view=sql-server-2017 > Flink - > https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sql.html > Spark - https://spark.apache.org/docs/2.3.0/api/sql/index.html#trim > Postgres - https://www.postgresql.org/docs/9.1/static/functions-string.html > SqLite - https://www.sqlite.org/lang_corefunc.html#trim > Also see: CALCITE-2530 BEAM-4704 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2571) TRIM does not match the behavior of most SQL implementations
[ https://issues.apache.org/jira/browse/CALCITE-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622806#comment-16622806 ] Andrew Pilloud commented on CALCITE-2571: - I rewrote the patch to only enable extended trim in relevant conformance levels (mysql, sql server) and throw an exception otherwise. I've added tests for both cases. > TRIM does not match the behavior of most SQL implementations > > > Key: CALCITE-2571 > URL: https://issues.apache.org/jira/browse/CALCITE-2571 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Andrew Pilloud >Assignee: Julian Hyde >Priority: Major > > Most SQL implementations trim all characters if more then one is provided to > the trim function. Calcite's SqlFunctions implementation silently drops the > extra characters, resulting in data corruption. > Here is a sampling of implementations that trim all characters: > MySQL - > https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_trim > Sql Server - > https://docs.microsoft.com/en-us/sql/t-sql/functions/trim-transact-sql?view=sql-server-2017 > Flink - > https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sql.html > Spark - https://spark.apache.org/docs/2.3.0/api/sql/index.html#trim > Postgres - https://www.postgresql.org/docs/9.1/static/functions-string.html > SqLite - https://www.sqlite.org/lang_corefunc.html#trim > Also see: CALCITE-2530 BEAM-4704 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2572) Calcite substring fails with a start position less than 1.
[ https://issues.apache.org/jira/browse/CALCITE-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621287#comment-16621287 ] Andrew Pilloud commented on CALCITE-2572: - I have updated the code to more closely match the text from the standard and added new test cases. > Calcite substring fails with a start position less than 1. > -- > > Key: CALCITE-2572 > URL: https://issues.apache.org/jira/browse/CALCITE-2572 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Andrew Pilloud >Assignee: Julian Hyde >Priority: Major > > Calcite substring throws a IndexOutOfBoundsException with a position less > than 1. Per the SQL standard, the position should be treated as the larger of > the provided value and 1, however many implementations treat negative values > as starting from the end of the string. > Extended standard: > https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions162.htm > https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_substring > https://spark.apache.org/docs/2.3.0/api/sql/index.html#substring > https://www.sqlite.org/lang_corefunc.html#substr > Follow the standard: > https://docs.microsoft.com/en-us/sql/t-sql/functions/substring-transact-sql?view=sql-server-2017 > https://www.postgresql.org/docs/9.1/static/functions-string.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2572) Calcite substring fails with a start position less than 1.
[ https://issues.apache.org/jira/browse/CALCITE-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621014#comment-16621014 ] Andrew Pilloud commented on CALCITE-2572: - Proposed fix: https://github.com/apache/calcite/pull/842 > Calcite substring fails with a start position less than 1. > -- > > Key: CALCITE-2572 > URL: https://issues.apache.org/jira/browse/CALCITE-2572 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Andrew Pilloud >Assignee: Julian Hyde >Priority: Major > > Calcite substring throws a IndexOutOfBoundsException with a position less > than 1. Per the SQL standard, the position should be treated as the larger of > the provided value and 1, however many implementations treat negative values > as starting from the end of the string. > Extended standard: > https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions162.htm > https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_substring > https://spark.apache.org/docs/2.3.0/api/sql/index.html#substring > https://www.sqlite.org/lang_corefunc.html#substr > Follow the standard: > https://docs.microsoft.com/en-us/sql/t-sql/functions/substring-transact-sql?view=sql-server-2017 > https://www.postgresql.org/docs/9.1/static/functions-string.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2572) Calcite substring fails with a start position less than 1.
Andrew Pilloud created CALCITE-2572: --- Summary: Calcite substring fails with a start position less than 1. Key: CALCITE-2572 URL: https://issues.apache.org/jira/browse/CALCITE-2572 Project: Calcite Issue Type: Bug Components: core Reporter: Andrew Pilloud Assignee: Julian Hyde Calcite substring throws a IndexOutOfBoundsException with a position less than 1. Per the SQL standard, the position should be treated as the larger of the provided value and 1, however many implementations treat negative values as starting from the end of the string. Extended standard: https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions162.htm https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_substring https://spark.apache.org/docs/2.3.0/api/sql/index.html#substring https://www.sqlite.org/lang_corefunc.html#substr Follow the standard: https://docs.microsoft.com/en-us/sql/t-sql/functions/substring-transact-sql?view=sql-server-2017 https://www.postgresql.org/docs/9.1/static/functions-string.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2571) TRIM does not match the behavior of most SQL implementations
[ https://issues.apache.org/jira/browse/CALCITE-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620983#comment-16620983 ] Andrew Pilloud commented on CALCITE-2571: - Proposed fix: https://github.com/apache/calcite/pull/840 > TRIM does not match the behavior of most SQL implementations > > > Key: CALCITE-2571 > URL: https://issues.apache.org/jira/browse/CALCITE-2571 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Andrew Pilloud >Assignee: Julian Hyde >Priority: Major > > Most SQL implementations trim all characters if more then one is provided to > the trim function. Calcite's SqlFunctions implementation silently drops the > extra characters, resulting in data corruption. > Here is a sampling of implementations that trim all characters: > MySQL - > https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_trim > Sql Server - > https://docs.microsoft.com/en-us/sql/t-sql/functions/trim-transact-sql?view=sql-server-2017 > Flink - > https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sql.html > Spark - https://spark.apache.org/docs/2.3.0/api/sql/index.html#trim > Postgres - https://www.postgresql.org/docs/9.1/static/functions-string.html > SqLite - https://www.sqlite.org/lang_corefunc.html#trim > Also see: CALCITE-2530 BEAM-4704 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2571) TRIM does not match the behavior of most SQL implementations
Andrew Pilloud created CALCITE-2571: --- Summary: TRIM does not match the behavior of most SQL implementations Key: CALCITE-2571 URL: https://issues.apache.org/jira/browse/CALCITE-2571 Project: Calcite Issue Type: Bug Components: core Reporter: Andrew Pilloud Assignee: Julian Hyde Most SQL implementations trim all characters if more then one is provided to the trim function. Calcite's SqlFunctions implementation silently drops the extra characters, resulting in data corruption. Here is a sampling of implementations that trim all characters: MySQL - https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_trim Sql Server - https://docs.microsoft.com/en-us/sql/t-sql/functions/trim-transact-sql?view=sql-server-2017 Flink - https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sql.html Spark - https://spark.apache.org/docs/2.3.0/api/sql/index.html#trim Postgres - https://www.postgresql.org/docs/9.1/static/functions-string.html SqLite - https://www.sqlite.org/lang_corefunc.html#trim Also see: CALCITE-2530 BEAM-4704 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2529) linq4j should promote integer to floating point when generating function calls
[ https://issues.apache.org/jira/browse/CALCITE-2529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603633#comment-16603633 ] Andrew Pilloud commented on CALCITE-2529: - Proposed fix is here: https://github.com/apache/calcite/pull/818 > linq4j should promote integer to floating point when generating function calls > -- > > Key: CALCITE-2529 > URL: https://issues.apache.org/jira/browse/CALCITE-2529 > Project: Calcite > Issue Type: Improvement >Reporter: Andrew Pilloud >Assignee: Julian Hyde >Priority: Major > > For example, I get "RuntimeException: while resolving method 'atan2[double, > int]'" when trying to execute a query with "ATAN2(c_double, 2)" (where > c_double is a column containing the double type). atan2[double, double] > should resolve as a valid implementation in this case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2529) linq4j should promote integer to floating point when generating function calls
Andrew Pilloud created CALCITE-2529: --- Summary: linq4j should promote integer to floating point when generating function calls Key: CALCITE-2529 URL: https://issues.apache.org/jira/browse/CALCITE-2529 Project: Calcite Issue Type: Improvement Reporter: Andrew Pilloud Assignee: Julian Hyde For example, I get "RuntimeException: while resolving method 'atan2[double, int]'" when trying to execute a query with "ATAN2(c_double, 2)" (where c_double is a column containing the double type). atan2[double, double] should resolve as a valid implementation in this case. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2394) Avatica applies calendar offset to timestamps when they should remain unchanged
[
https://issues.apache.org/jira/browse/CALCITE-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539259#comment-16539259
]
Andrew Pilloud commented on CALCITE-2394:
-
Looks like there is a config option in ConnectionProperty for timeZone and Beam
should be setting that to UTC, which will result in Avatica applying the local
timezone offset of 0.
> Avatica applies calendar offset to timestamps when they should remain
> unchanged
> ---
>
> Key: CALCITE-2394
> URL: https://issues.apache.org/jira/browse/CALCITE-2394
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Major
>
> This code converts a millis-since-epoch value to a timestamp in three
> different accessors:
> {code}
> class AbstractCursor {
> ...
> static Timestamp longToTimestamp(long v, Calendar calendar) {
> if (calendar != null) {
> v -= calendar.getTimeZone().getOffset(v);
> }
> return new Timestamp(v);
> }
> }
> {code}
> But {{new Timestamp(millis)}} always accepts millis-since-epoch in GMT.
> The use in {{DateFromNumberAccessor}} is probably OK: it fabricates
> millis-since-epoch from a date, so applying the offset is appropriate to hit
> midnight in that locale.
> But both {{TimeFromNumberAccessor}} and {{TimestampFromNumberAccessor}}
> should leave the millis absolute.
> This manifests as timestamp actual values being shifted by the current locale
> (in addition to later display adjustments).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2353) Allow user to override SqlSetOption in alterStatementParserMethods
[ https://issues.apache.org/jira/browse/CALCITE-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502770#comment-16502770 ] Andrew Pilloud commented on CALCITE-2353: - That is what I intended. I opened a PR that does this; https://github.com/apache/calcite/pull/722 > Allow user to override SqlSetOption in alterStatementParserMethods > -- > > Key: CALCITE-2353 > URL: https://issues.apache.org/jira/browse/CALCITE-2353 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Andrew Pilloud >Assignee: Julian Hyde >Priority: Major > > If alterStatementParserMethods are added before SqlSetOption then the user > can replace SqlSetOption. This change matches how statementParserMethods are > added at the top level. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2353) Allow user to override SqlSetOption in alterStatementParserMethods
Andrew Pilloud created CALCITE-2353: --- Summary: Allow user to override SqlSetOption in alterStatementParserMethods Key: CALCITE-2353 URL: https://issues.apache.org/jira/browse/CALCITE-2353 Project: Calcite Issue Type: Bug Components: core Reporter: Andrew Pilloud Assignee: Julian Hyde If alterStatementParserMethods are added before SqlSetOption then the user can replace SqlSetOption. This change matches how statementParserMethods are added at the top level. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (CALCITE-2248) Take advantage of Calcite DDL
[ https://issues.apache.org/jira/browse/CALCITE-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Pilloud closed CALCITE-2248. --- Resolution: Invalid > Take advantage of Calcite DDL > - > > Key: CALCITE-2248 > URL: https://issues.apache.org/jira/browse/CALCITE-2248 > Project: Calcite > Issue Type: New Feature >Reporter: Andrew Pilloud >Assignee: Julian Hyde >Priority: Major > > In Calcite 1.15 abstract support for DDL moved into the calcite core. We > should take advantage of that. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-2248) Take advantage of Calcite DDL
Andrew Pilloud created CALCITE-2248: --- Summary: Take advantage of Calcite DDL Key: CALCITE-2248 URL: https://issues.apache.org/jira/browse/CALCITE-2248 Project: Calcite Issue Type: New Feature Reporter: Andrew Pilloud Assignee: Julian Hyde In Calcite 1.15 abstract support for DDL moved into the calcite core. We should take advantage of that. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2248) Take advantage of Calcite DDL
[ https://issues.apache.org/jira/browse/CALCITE-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Pilloud updated CALCITE-2248: Oops, wrong project. > Take advantage of Calcite DDL > - > > Key: CALCITE-2248 > URL: https://issues.apache.org/jira/browse/CALCITE-2248 > Project: Calcite > Issue Type: New Feature >Reporter: Andrew Pilloud >Assignee: Julian Hyde >Priority: Major > > In Calcite 1.15 abstract support for DDL moved into the calcite core. We > should take advantage of that. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (CALCITE-2177) TUMBLE_START does not respect AS when SELECT and GROUP BY match
[
https://issues.apache.org/jira/browse/CALCITE-2177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Pilloud closed CALCITE-2177.
---
Resolution: Fixed
> TUMBLE_START does not respect AS when SELECT and GROUP BY match
> ---
>
> Key: CALCITE-2177
> URL: https://issues.apache.org/jira/browse/CALCITE-2177
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.13.0
>Reporter: Andrew Pilloud
>Assignee: Julian Hyde
>Priority: Major
> Fix For: 1.15.0
>
>
> When the order of SELECT and GROUP BY arguments match, the output of
> TUMBLE_START and HOP_START can not be named with AS. Found while trying to
> implement Nexmark benchmark queries in Apache Beam.
> This query fails with "An exception occured while executing the Java class.
> null: InvocationTargetException: Type 'RecordType(TIMESTAMP(0) $f0)' has no
> field 'starttime'"
> {code:sql}
> SELECT B.starttime FROM
>(SELECT
> TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
> FROM Bid B1
> GROUP BY
> TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
> {code}
> This more useful query also fails with "An exception occured while executing
> the Java class. null: InvocationTargetException: Type 'RecordType(BIGINT
> auction, BIGINT price, TIMESTAMP(0) $f2)' has no field 'starttime'"
> {code:sql}
> SELECT B.starttime FROM
>(SELECT B1.auction, B1.price,
> TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
> FROM Bid B1
> GROUP BY B1.auction, B1.price,
> TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
> {code}
> However swap the order of the select arguments and it works as expected:
> {code:sql}
> SELECT B.starttime FROM
>(SELECT B1.price, B1.auction,
> TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
> FROM Bid B1
> GROUP BY B1.auction, B1.price,
> TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
> {code}
>
> More detailed stack from second query:
> {code:java}
> Caused by: java.lang.AssertionError: Type 'RecordType(TIMESTAMP(0) $f0,
> BIGINT auction, BIGINT price)' has no field 'starttime'
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.rex.RexBuilder.makeFieldAccess(RexBuilder.java:175)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:3533)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.access$1800(SqlToRelConverter.java:210)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4545)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3904)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql.SqlIdentifier.accept(SqlIdentifier.java:344)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4438)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectList(SqlToRelConverter.java:3744)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:662)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:619)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3054)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:555)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.prepare.PlannerImpl.rel(PlannerImpl.java:232)
> at
> org.apache.beam.sdk.extensions.sql.impl.planner.BeamQueryPlanner.convertToRelNode(BeamQueryPlanner.java:164)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2177) TUMBLE_START does not respect AS when SELECT and GROUP BY match
[
https://issues.apache.org/jira/browse/CALCITE-2177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Pilloud updated CALCITE-2177:
Fix Version/s: 1.15.0
> TUMBLE_START does not respect AS when SELECT and GROUP BY match
> ---
>
> Key: CALCITE-2177
> URL: https://issues.apache.org/jira/browse/CALCITE-2177
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.13.0
>Reporter: Andrew Pilloud
>Assignee: Julian Hyde
>Priority: Major
> Fix For: 1.15.0
>
>
> When the order of SELECT and GROUP BY arguments match, the output of
> TUMBLE_START and HOP_START can not be named with AS. Found while trying to
> implement Nexmark benchmark queries in Apache Beam.
> This query fails with "An exception occured while executing the Java class.
> null: InvocationTargetException: Type 'RecordType(TIMESTAMP(0) $f0)' has no
> field 'starttime'"
> {code:sql}
> SELECT B.starttime FROM
>(SELECT
> TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
> FROM Bid B1
> GROUP BY
> TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
> {code}
> This more useful query also fails with "An exception occured while executing
> the Java class. null: InvocationTargetException: Type 'RecordType(BIGINT
> auction, BIGINT price, TIMESTAMP(0) $f2)' has no field 'starttime'"
> {code:sql}
> SELECT B.starttime FROM
>(SELECT B1.auction, B1.price,
> TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
> FROM Bid B1
> GROUP BY B1.auction, B1.price,
> TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
> {code}
> However swap the order of the select arguments and it works as expected:
> {code:sql}
> SELECT B.starttime FROM
>(SELECT B1.price, B1.auction,
> TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
> FROM Bid B1
> GROUP BY B1.auction, B1.price,
> TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
> {code}
>
> More detailed stack from second query:
> {code:java}
> Caused by: java.lang.AssertionError: Type 'RecordType(TIMESTAMP(0) $f0,
> BIGINT auction, BIGINT price)' has no field 'starttime'
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.rex.RexBuilder.makeFieldAccess(RexBuilder.java:175)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:3533)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.access$1800(SqlToRelConverter.java:210)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4545)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3904)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql.SqlIdentifier.accept(SqlIdentifier.java:344)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4438)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectList(SqlToRelConverter.java:3744)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:662)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:619)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3054)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:555)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.prepare.PlannerImpl.rel(PlannerImpl.java:232)
> at
> org.apache.beam.sdk.extensions.sql.impl.planner.BeamQueryPlanner.convertToRelNode(BeamQueryPlanner.java:164)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2177) TUMBLE_START does not respect AS when SELECT and GROUP BY match
[
https://issues.apache.org/jira/browse/CALCITE-2177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366380#comment-16366380
]
Andrew Pilloud commented on CALCITE-2177:
-
This is fixed in 1.15.0, should have updated first!
> TUMBLE_START does not respect AS when SELECT and GROUP BY match
> ---
>
> Key: CALCITE-2177
> URL: https://issues.apache.org/jira/browse/CALCITE-2177
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.13.0
>Reporter: Andrew Pilloud
>Assignee: Julian Hyde
>Priority: Major
> Fix For: 1.15.0
>
>
> When the order of SELECT and GROUP BY arguments match, the output of
> TUMBLE_START and HOP_START can not be named with AS. Found while trying to
> implement Nexmark benchmark queries in Apache Beam.
> This query fails with "An exception occured while executing the Java class.
> null: InvocationTargetException: Type 'RecordType(TIMESTAMP(0) $f0)' has no
> field 'starttime'"
> {code:sql}
> SELECT B.starttime FROM
>(SELECT
> TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
> FROM Bid B1
> GROUP BY
> TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
> {code}
> This more useful query also fails with "An exception occured while executing
> the Java class. null: InvocationTargetException: Type 'RecordType(BIGINT
> auction, BIGINT price, TIMESTAMP(0) $f2)' has no field 'starttime'"
> {code:sql}
> SELECT B.starttime FROM
>(SELECT B1.auction, B1.price,
> TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
> FROM Bid B1
> GROUP BY B1.auction, B1.price,
> TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
> {code}
> However swap the order of the select arguments and it works as expected:
> {code:sql}
> SELECT B.starttime FROM
>(SELECT B1.price, B1.auction,
> TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
> FROM Bid B1
> GROUP BY B1.auction, B1.price,
> TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
> {code}
>
> More detailed stack from second query:
> {code:java}
> Caused by: java.lang.AssertionError: Type 'RecordType(TIMESTAMP(0) $f0,
> BIGINT auction, BIGINT price)' has no field 'starttime'
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.rex.RexBuilder.makeFieldAccess(RexBuilder.java:175)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:3533)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.access$1800(SqlToRelConverter.java:210)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4545)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3904)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql.SqlIdentifier.accept(SqlIdentifier.java:344)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4438)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectList(SqlToRelConverter.java:3744)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:662)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:619)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3054)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:555)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.prepare.PlannerImpl.rel(PlannerImpl.java:232)
> at
> org.apache.beam.sdk.extensions.sql.impl.planner.BeamQueryPlanner.convertToRelNode(BeamQueryPlanner.java:164)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2177) TUMBLE_START does not respect AS when SELECT and GROUP BY match
[
https://issues.apache.org/jira/browse/CALCITE-2177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Pilloud updated CALCITE-2177:
Affects Version/s: (was: 1.15.0)
> TUMBLE_START does not respect AS when SELECT and GROUP BY match
> ---
>
> Key: CALCITE-2177
> URL: https://issues.apache.org/jira/browse/CALCITE-2177
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.13.0
>Reporter: Andrew Pilloud
>Assignee: Julian Hyde
>Priority: Major
>
> When the order of SELECT and GROUP BY arguments match, the output of
> TUMBLE_START and HOP_START can not be named with AS. Found while trying to
> implement Nexmark benchmark queries in Apache Beam.
> This query fails with "An exception occured while executing the Java class.
> null: InvocationTargetException: Type 'RecordType(TIMESTAMP(0) $f0)' has no
> field 'starttime'"
> {code:sql}
> SELECT B.starttime FROM
>(SELECT
> TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
> FROM Bid B1
> GROUP BY
> TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
> {code}
> This more useful query also fails with "An exception occured while executing
> the Java class. null: InvocationTargetException: Type 'RecordType(BIGINT
> auction, BIGINT price, TIMESTAMP(0) $f2)' has no field 'starttime'"
> {code:sql}
> SELECT B.starttime FROM
>(SELECT B1.auction, B1.price,
> TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
> FROM Bid B1
> GROUP BY B1.auction, B1.price,
> TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
> {code}
> However swap the order of the select arguments and it works as expected:
> {code:sql}
> SELECT B.starttime FROM
>(SELECT B1.price, B1.auction,
> TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
> FROM Bid B1
> GROUP BY B1.auction, B1.price,
> TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
> {code}
>
> More detailed stack from second query:
> {code:java}
> Caused by: java.lang.AssertionError: Type 'RecordType(TIMESTAMP(0) $f0,
> BIGINT auction, BIGINT price)' has no field 'starttime'
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.rex.RexBuilder.makeFieldAccess(RexBuilder.java:175)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:3533)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.access$1800(SqlToRelConverter.java:210)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4545)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3904)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql.SqlIdentifier.accept(SqlIdentifier.java:344)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4438)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectList(SqlToRelConverter.java:3744)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:662)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:619)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3054)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:555)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.prepare.PlannerImpl.rel(PlannerImpl.java:232)
> at
> org.apache.beam.sdk.extensions.sql.impl.planner.BeamQueryPlanner.convertToRelNode(BeamQueryPlanner.java:164)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2177) TUMBLE_START does not respect AS when SELECT and GROUP BY match
[
https://issues.apache.org/jira/browse/CALCITE-2177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Pilloud updated CALCITE-2177:
Affects Version/s: 1.15.0
> TUMBLE_START does not respect AS when SELECT and GROUP BY match
> ---
>
> Key: CALCITE-2177
> URL: https://issues.apache.org/jira/browse/CALCITE-2177
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.13.0, 1.15.0
>Reporter: Andrew Pilloud
>Assignee: Julian Hyde
>Priority: Major
>
> When the order of SELECT and GROUP BY arguments match, the output of
> TUMBLE_START and HOP_START can not be named with AS. Found while trying to
> implement Nexmark benchmark queries in Apache Beam.
> This query fails with "An exception occured while executing the Java class.
> null: InvocationTargetException: Type 'RecordType(TIMESTAMP(0) $f0)' has no
> field 'starttime'"
> {code:sql}
> SELECT B.starttime FROM
>(SELECT
> TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
> FROM Bid B1
> GROUP BY
> TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
> {code}
> This more useful query also fails with "An exception occured while executing
> the Java class. null: InvocationTargetException: Type 'RecordType(BIGINT
> auction, BIGINT price, TIMESTAMP(0) $f2)' has no field 'starttime'"
> {code:sql}
> SELECT B.starttime FROM
>(SELECT B1.auction, B1.price,
> TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
> FROM Bid B1
> GROUP BY B1.auction, B1.price,
> TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
> {code}
> However swap the order of the select arguments and it works as expected:
> {code:sql}
> SELECT B.starttime FROM
>(SELECT B1.price, B1.auction,
> TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
> FROM Bid B1
> GROUP BY B1.auction, B1.price,
> TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
> {code}
>
> More detailed stack from second query:
> {code:java}
> Caused by: java.lang.AssertionError: Type 'RecordType(TIMESTAMP(0) $f0,
> BIGINT auction, BIGINT price)' has no field 'starttime'
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.rex.RexBuilder.makeFieldAccess(RexBuilder.java:175)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:3533)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.access$1800(SqlToRelConverter.java:210)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4545)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3904)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql.SqlIdentifier.accept(SqlIdentifier.java:344)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4438)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectList(SqlToRelConverter.java:3744)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:662)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:619)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3054)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:555)
> at
> org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.prepare.PlannerImpl.rel(PlannerImpl.java:232)
> at
> org.apache.beam.sdk.extensions.sql.impl.planner.BeamQueryPlanner.convertToRelNode(BeamQueryPlanner.java:164)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2177) TUMBLE_START does not respect AS when SELECT and GROUP BY match
Andrew Pilloud created CALCITE-2177:
---
Summary: TUMBLE_START does not respect AS when SELECT and GROUP BY
match
Key: CALCITE-2177
URL: https://issues.apache.org/jira/browse/CALCITE-2177
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.13.0
Reporter: Andrew Pilloud
Assignee: Julian Hyde
When the order of SELECT and GROUP BY arguments match, the output of
TUMBLE_START and HOP_START can not be named with AS. Found while trying to
implement Nexmark benchmark queries in Apache Beam.
This query fails with "An exception occured while executing the Java class.
null: InvocationTargetException: Type 'RecordType(TIMESTAMP(0) $f0)' has no
field 'starttime'"
{code:sql}
SELECT B.starttime FROM
(SELECT
TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
FROM Bid B1
GROUP BY
TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
{code}
This more useful query also fails with "An exception occured while executing
the Java class. null: InvocationTargetException: Type 'RecordType(BIGINT
auction, BIGINT price, TIMESTAMP(0) $f2)' has no field 'starttime'"
{code:sql}
SELECT B.starttime FROM
(SELECT B1.auction, B1.price,
TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
FROM Bid B1
GROUP BY B1.auction, B1.price,
TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
{code}
However swap the order of the select arguments and it works as expected:
{code:sql}
SELECT B.starttime FROM
(SELECT B1.price, B1.auction,
TUMBLE_START(B1.dateTime, INTERVAL '1' SECOND) AS starttime
FROM Bid B1
GROUP BY B1.auction, B1.price,
TUMBLE(B1.dateTime, INTERVAL '1' SECOND)) B
{code}
More detailed stack from second query:
{code:java}
Caused by: java.lang.AssertionError: Type 'RecordType(TIMESTAMP(0) $f0, BIGINT
auction, BIGINT price)' has no field 'starttime'
at
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.rex.RexBuilder.makeFieldAccess(RexBuilder.java:175)
at
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:3533)
at
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.access$1800(SqlToRelConverter.java:210)
at
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4545)
at
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3904)
at
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql.SqlIdentifier.accept(SqlIdentifier.java:344)
at
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4438)
at
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectList(SqlToRelConverter.java:3744)
at
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:662)
at
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:619)
at
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3054)
at
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:555)
at
org.apache.beam.sdks.java.extensions.sql.repackaged.org.apache.calcite.prepare.PlannerImpl.rel(PlannerImpl.java:232)
at
org.apache.beam.sdk.extensions.sql.impl.planner.BeamQueryPlanner.convertToRelNode(BeamQueryPlanner.java:164)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
