[jira] [Resolved] (CALCITE-5703) Reduce amount of generated runtime code
[
https://issues.apache.org/jira/browse/CALCITE-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu resolved CALCITE-5703.
---
Resolution: Fixed
> Reduce amount of generated runtime code
> ---
>
> Key: CALCITE-5703
> URL: https://issues.apache.org/jira/browse/CALCITE-5703
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.34.0
>Reporter: Evgeny Stanilovsky
>Assignee: Evgeny Stanilovsky
>Priority: Major
> Labels: patch-available, pull-request-available
> Fix For: 1.35.0
>
>
> In some cases runtime generates code like :
> {noformat}
> return case_when_value == null ? (String) null : some_oparation();
> or
> return input_value == null ? (Long) null : Long.valueOf(...;
> {noformat}
> this redundant casting probably not harmful, but there is another side -
> maximum method size, this size jdk[1], janino [2] throws : *Code grows beyond
> 64 KB* . This PR reduces code generated by calcite runtime thus more huge
> expressions can be executed.
> [1]
> https://github.com/openjdk/jdk/blob/d22bcc813eea719b817d3d541a843594675c0ca9/src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassFile.java#L101
> [2]
> https://github.com/janino-compiler/janino/blob/e69022f5aaabd36edc08a2074360d62514493a19/janino/src/main/java/org/codehaus/janino/CodeContext.java#L699
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5703) Reduce amount of generated runtime code
[
https://issues.apache.org/jira/browse/CALCITE-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17727489#comment-17727489
]
Feng Zhu commented on CALCITE-5703:
---
Fixed in
[https://github.com/apache/calcite/commit/3817b0e42c07a6b185f3c1b921f648ff28e8a3b7]
Thanks for your work [~zstan]
> Reduce amount of generated runtime code
> ---
>
> Key: CALCITE-5703
> URL: https://issues.apache.org/jira/browse/CALCITE-5703
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.34.0
>Reporter: Evgeny Stanilovsky
>Assignee: Evgeny Stanilovsky
>Priority: Major
> Labels: patch-available, pull-request-available
> Fix For: 1.35.0
>
>
> In some cases runtime generates code like :
> {noformat}
> return case_when_value == null ? (String) null : some_oparation();
> or
> return input_value == null ? (Long) null : Long.valueOf(...;
> {noformat}
> this redundant casting probably not harmful, but there is another side -
> maximum method size, this size jdk[1], janino [2] throws : *Code grows beyond
> 64 KB* . This PR reduces code generated by calcite runtime thus more huge
> expressions can be executed.
> [1]
> https://github.com/openjdk/jdk/blob/d22bcc813eea719b817d3d541a843594675c0ca9/src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassFile.java#L101
> [2]
> https://github.com/janino-compiler/janino/blob/e69022f5aaabd36edc08a2074360d62514493a19/janino/src/main/java/org/codehaus/janino/CodeContext.java#L699
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (CALCITE-3627) Incorrect null semantic for ROW function
[
https://issues.apache.org/jira/browse/CALCITE-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu resolved CALCITE-3627.
---
Resolution: Fixed
Fixed in
[https://github.com/apache/calcite/commit/8d21c3f2f0b75d788e70bbeea9746695f2fde552.|https://github.com/apache/calcite/commit/8d21c3f2f0b75d788e70bbeea9746695f2fde552,]
> Incorrect null semantic for ROW function
>
>
> Key: CALCITE-3627
> URL: https://issues.apache.org/jira/browse/CALCITE-3627
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h
> Remaining Estimate: 0h
>
> In CALCITE-2464, an in-depth discussion on nullability of structured type has
> been conducted.
> According to SQL standards, a record's value (with structure type) is
> considered null if and only if all of its fields are null.
> Based on that, the *ROW* function's return type can be nullable.
> Moreover, current implementor for ROW function's _*NullPolicy*_
> (_NullPolicy.ANY_) contradicts its runtime behavior.
> {code:java}
> Query:
> select ROW(1, cast(null as int))
> Result:
> {1, null}{code}
> However, according to the NullPolicy, the result should be null.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (CALCITE-4777) Casting from DECIMAL to BOOLEAN throws an exception
[
https://issues.apache.org/jira/browse/CALCITE-4777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422616#comment-17422616
]
Feng Zhu commented on CALCITE-4777:
---
[~xuyangzhong] Can we create an umbrella ticket to put these separated ones
together?
> Casting from DECIMAL to BOOLEAN throws an exception
> ---
>
> Key: CALCITE-4777
> URL: https://issues.apache.org/jira/browse/CALCITE-4777
> Project: Calcite
> Issue Type: Bug
>Reporter: xuyangzhong
>Assignee: xuyangzhong
>Priority: Major
> Labels: pull-request-available
> Attachments: calcite.png
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> My sql is the following:
> {code:java}
> // code placeholder
> select * from test where cast (0.10915913549909961 as boolean){code}
>
> I want to simplify the cast. An exception is thrown:
>
> {code:java}
> // code placeholder
> Exception in thread "main" java.lang.RuntimeException: while resolving method
> 'booleanValue' in class class java.math.BigDecimal at
> org.apache.calcite.linq4j.tree.Expressions.call(Expressions.java:424) at
> org.apache.calcite.linq4j.tree.Expressions.call(Expressions.java:435) at
> org.apache.calcite.linq4j.tree.Expressions.unbox(Expressions.java:1453) at
> org.apache.calcite.adapter.enumerable.EnumUtils.convert(EnumUtils.java:398)
> at
> org.apache.calcite.adapter.enumerable.EnumUtils.convert(EnumUtils.java:326)
> at
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translateCast(RexToLixTranslator.java:543)
> at
> org.apache.calcite.adapter.enumerable.RexImpTable$CastImplementor.implementSafe(RexImpTable.java:2450)
> at
> org.apache.calcite.adapter.enumerable.RexImpTable$AbstractRexCallImplementor.genValueStatement(RexImpTable.java:2894)
> at
> org.apache.calcite.adapter.enumerable.RexImpTable$AbstractRexCallImplementor.implement(RexImpTable.java:2859)
> at
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitCall(RexToLixTranslator.java:1089)
> at
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitCall(RexToLixTranslator.java:90)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:174) at
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitLocalRef(RexToLixTranslator.java:975)
> at
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.visitLocalRef(RexToLixTranslator.java:90)
> at org.apache.calcite.rex.RexLocalRef.accept(RexLocalRef.java:75) at
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translate(RexToLixTranslator.java:237)
> at
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translate(RexToLixTranslator.java:231)
> at
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translateList(RexToLixTranslator.java:823)
> at
> org.apache.calcite.adapter.enumerable.RexToLixTranslator.translateProjects(RexToLixTranslator.java:198)
> at org.apache.calcite.rex.RexExecutorImpl.compile(RexExecutorImpl.java:90)
> at org.apache.calcite.rex.RexExecutorImpl.compile(RexExecutorImpl.java:66) at
> org.apache.calcite.rex.RexExecutorImpl.reduce(RexExecutorImpl.java:128) at
> org.apache.calcite.rex.RexSimplify.simplifyCast(RexSimplify.java:2101) at
> org.apache.calcite.rex.RexSimplify.simplify(RexSimplify.java:326) at
> org.apache.calcite.rex.RexSimplify.simplifyUnknownAs(RexSimplify.java:287) at
> org.apache.flink.table.examples.java.tests.CalciteTest.main(CalciteTest.java:47)
> Caused by: java.lang.NoSuchMethodException:
> java.math.BigDecimal.booleanValue() at
> java.lang.Class.getMethod(Class.java:1786) at
> org.apache.calcite.linq4j.tree.Expressions.call(Expressions.java:421) ... 25
> more
> {code}
> In order to avoid that I used the wrong rule or it caused by my bad coding, i
> write the test case following:
>
> {code:java}
> // code placeholder
> JavaTypeFactory typeFactory = new
> JavaTypeFactoryImpl(RelDataTypeSystem.DEFAULT);
> RexBuilder rexBuilder = new RexBuilder(typeFactory);
> final RexSimplify simplify = new RexSimplify(rexBuilder,
> RelOptPredicateList.EMPTY, RexUtil.EXECUTOR);
> RelDataType type = new BasicSqlType(typeFactory.getTypeSystem(),
> SqlTypeName.BOOLEAN);
> RelDataType bb = new
> BasicSqlType(typeFactory.getTypeSystem(),SqlTypeName.DECIMAL,18,17);
> SqlOperator op = new SqlCastFunction();
> RexLiteral lt =
> rexBuilder.makeExactLiteral(BigDecimal.valueOf(0.10915913549909961),bb);
> List list = new ArrayList<>();
> list.add(lt);
> RexNode rexNode = rexBuilder.makeCall(type,op,list);
> simplify.simplifyUnknownAs(rexNode, RexUnknownAs.UNKNOWN);
> {code}
> and it throws the same exception.
>
> Actually, the cast simplify operation will enter the function _translateCast_
> in _RexToLixTranslator_.It misses the "case BOOLEAN" and uses the convert in
> EnumUtils. However, because the Decima
[jira] [Commented] (CALCITE-4812) RelBuilder.join should error multiple correlate ids
[ https://issues.apache.org/jira/browse/CALCITE-4812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422591#comment-17422591 ] Feng Zhu commented on CALCITE-4812: --- Could you provide a test case? > RelBuilder.join should error multiple correlate ids > --- > > Key: CALCITE-4812 > URL: https://issues.apache.org/jira/browse/CALCITE-4812 > Project: Calcite > Issue Type: Improvement >Reporter: James Starr >Assignee: James Starr >Priority: Major > > Relbuilder.join does not correctly handle multiple correlate ids. An error > should be thrown when multiple ids are passed in. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4810) Failed to cast a double field (which isn't a literal) to boolean in the test case.
[
https://issues.apache.org/jira/browse/CALCITE-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422545#comment-17422545
]
Feng Zhu commented on CALCITE-4810:
---
Any standards can be inferred to help us review the cast behaviors between two
types?
Maybe we need to check more pairs, rather than only the (_double_, _boolean_)
case.
> Failed to cast a double field (which isn't a literal) to boolean in the test
> case.
> --
>
> Key: CALCITE-4810
> URL: https://issues.apache.org/jira/browse/CALCITE-4810
> Project: Calcite
> Issue Type: Bug
>Reporter: xuyangzhong
>Priority: Minor
>
> The cause of this matter is that when I add a test case in
> SqlOperatorBaseTest:
> {code:java}
> // code placeholder
> @Test void myTest(){
> tester.checkBoolean("cast(0.1e0 as boolean)",false);
> }
> {code}
> The tester will turn the single sql to these two SQLs which will be all
> tested:
> {code:java}
> // SQL 1
> values (cast(0.1e0 as boolean))
> // SQL 2
> select cast(p0 as boolean) from (values (0.1e0)) as t(p0)
> {code}
> The SQL1 passed by the RexSimplify. But The SQL2 throws a exception following:
> {code:java}
> // code placeholder
> java.sql.SQLException: Error while executing SQL "select cast(p0 as boolean)
> from (values (0.1e0)) as t(p0)": Error while compiling generated Java code:
> public org.apache.calcite.linq4j.Enumerable bind(final
> org.apache.calcite.DataContext root) {
> final org.apache.calcite.linq4j.Enumerable _inputEnumerable =
> org.apache.calcite.linq4j.Linq4j.asEnumerable(new Double[] {
> 0.1D});
> return new org.apache.calcite.linq4j.AbstractEnumerable(){
> public org.apache.calcite.linq4j.Enumerator enumerator() {
> return new org.apache.calcite.linq4j.Enumerator(){
> public final org.apache.calcite.linq4j.Enumerator
> inputEnumerator = _inputEnumerable.enumerator();
> public void reset() {
> inputEnumerator.reset();
> }
>
> public boolean moveNext() {
> return inputEnumerator.moveNext();
> }
>
> public void close() {
> inputEnumerator.close();
> }
>
> public Object current() {
> return (boolean)
> org.apache.calcite.runtime.SqlFunctions.toDouble(inputEnumerator.current());
> }
>
> };
> }
>
> };
> }
>
>
> public Class getElementType() {
> return boolean.class;
> }
>
>
>
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
> at
> org.apache.calcite.sql.test.SqlOperatorBaseTest$TesterImpl.check(SqlOperatorBaseTest.java:10371)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.check(AbstractSqlTester.java:464)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.checkBoolean(AbstractSqlTester.java:433)
> at
> org.apache.calcite.sql.test.SqlOperatorBaseTest.myTest(SqlOperatorBaseTest.java:1619)
> at
> org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675)
> at
> org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:125)
> at
> org.junit.jupiter.engine.extension.TimeoutInvocation.proceed(TimeoutInvocation.java:46)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:139)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:131)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:81)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:104)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:62)
> at
> org.junit.jupiter.engine.execution.InvocationInt
[jira] [Commented] (CALCITE-4805) The IN sub-query can't be converted to OR when value list contains Null.
[
https://issues.apache.org/jira/browse/CALCITE-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17420435#comment-17420435
]
Feng Zhu commented on CALCITE-4805:
---
Did you ever checked https://issues.apache.org/jira/browse/CALCITE-373 and the
behaviors in other databases?
> The IN sub-query can't be converted to OR when value list contains Null.
>
>
> Key: CALCITE-4805
> URL: https://issues.apache.org/jira/browse/CALCITE-4805
> Project: Calcite
> Issue Type: Bug
>Reporter: duan xiong
>Assignee: duan xiong
>Priority: Major
>
> for example:
> {code:java}
> select * from emp where deptno in (null,2){code}
> Now the RelNode is:
> {code:java}
> LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
> SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
> LogicalJoin(condition=[=($7, $9)], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalAggregate(group=[{0}])
> LogicalValues(tuples=[[{ null }, { 2 }]]){code}
> The RelNode should be:
> {code:java}
> LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
> SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
> LogicalFilter(condition=[OR(=($7, null), =($7, 2))])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]]){code}
> So I think when the sub-query value list size is under the
> threshold(IN_SUB_QUERY_THRESHOLD), they can be converted to OR(no matter have
> the NullLiteral or not).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-4532) Rex simplification fails with an exception on nested CASTs with ANY
[
https://issues.apache.org/jira/browse/CALCITE-4532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu resolved CALCITE-4532.
---
Resolution: Fixed
Fixed in
[https://github.com/apache/calcite/commit/f368cbe26a7e0fff2089dc84bf82e726069cdf54]
, thanks for your work [~gierlachg]!
> Rex simplification fails with an exception on nested CASTs with ANY
> ---
>
> Key: CALCITE-4532
> URL: https://issues.apache.org/jira/browse/CALCITE-4532
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.24.0
>Reporter: Grzegorz Gierlach
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> {code:java}
> @Test public void testNestedBigInt() {
> CalciteAssert.AssertThat assertThat = CalciteAssert.that();
> String query = "SELECT CAST(CAST(42 AS BIGINT) AS ANY) FROM
> (VALUES(1))";
> assertThat.query(query).returns("EXPR$0=42\n");
> }
> {code}
> fails with:
> {code:java}
> Error while executing SQL "SELECT CAST(CAST(42 AS BIGINT) AS ANY)
> FROM (VALUES(1))": While compiling [CAST(42:BIGINT):ANY NOT
> NULL]Error while executing SQL "SELECT CAST(CAST(42 AS BIGINT) AS
> ANY) FROM (VALUES(1))": While compiling [CAST(42:BIGINT):ANY NOT
> NULL]java.sql.SQLException: Error while executing SQL "SELECT
> CAST(CAST(42 AS BIGINT) AS ANY) FROM (VALUES(1))": While compiling
> [CAST(42:BIGINT):ANY NOT NULL] at
> org.apache.calcite.avatica.Helper.createException(Helper.java:56) at
> org.apache.calcite.avatica.Helper.createException(Helper.java:41) at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
> at org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:534)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.lambda$returns$1(CalciteAssert.java:1535)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.withConnection(CalciteAssert.java:1474)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1533)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1523)
> at
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1486)
> at org.apache.calcite.test.JdbcTest.testNestedBigInt(JdbcTest.java:7707) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:564) at
> org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675)
> at
> org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:125)
> at
> org.junit.jupiter.engine.extension.TimeoutInvocation.proceed(TimeoutInvocation.java:46)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:139)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:131)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:81)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:104)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:62)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:43)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:35)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
> at
> org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:202)
> at
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.inv
[jira] [Closed] (CALCITE-3775) Implicit lookup methods in SimpleCalciteSchema ignore case sensitivity parameter
[ https://issues.apache.org/jira/browse/CALCITE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Zhu closed CALCITE-3775. - Resolution: Fixed > Implicit lookup methods in SimpleCalciteSchema ignore case sensitivity > parameter > > > Key: CALCITE-3775 > URL: https://issues.apache.org/jira/browse/CALCITE-3775 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.21.0 >Reporter: Stamatis Zampetakis >Priority: Minor > Labels: pull-request-available > Time Spent: 11h > Remaining Estimate: 0h > > SimpleCalciteSchema contains methods for finding tables, sub-schemas, and > types. The API allows to specify if the lookup will be case sensitive or > insensitive but the respective parameter is ignored in the implementation of > the method. > It is not a big problem since most of the time CachingCalciteSchema is used, > which does not have this problem. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3775) Implicit lookup methods in SimpleCalciteSchema ignore case sensitivity parameter
[ https://issues.apache.org/jira/browse/CALCITE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17376977#comment-17376977 ] Feng Zhu commented on CALCITE-3775: --- Fixed in [https://github.com/apache/calcite/commit/138c2bc4c5cf6a957b7ba66c8e166288e28a7a81|https://github.com/apache/calcite/commit/138c2bc4c5cf6a957b7ba66c8e166288e28a7a81.] [~alon.eldar], thanks for your work! > Implicit lookup methods in SimpleCalciteSchema ignore case sensitivity > parameter > > > Key: CALCITE-3775 > URL: https://issues.apache.org/jira/browse/CALCITE-3775 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.21.0 >Reporter: Stamatis Zampetakis >Priority: Minor > Labels: pull-request-available > Time Spent: 11h > Remaining Estimate: 0h > > SimpleCalciteSchema contains methods for finding tables, sub-schemas, and > types. The API allows to specify if the lookup will be case sensitive or > insensitive but the respective parameter is ignored in the implementation of > the method. > It is not a big problem since most of the time CachingCalciteSchema is used, > which does not have this problem. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3775) Implicit lookup methods in SimpleCalciteSchema ignore case sensitivity parameter
[ https://issues.apache.org/jira/browse/CALCITE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17373278#comment-17373278 ] Feng Zhu commented on CALCITE-3775: --- I also prefer option 1, which has already applied in our product environment. Sorry for the late reply, as I am very busy in recent months, really thanks [~alon.eldar] and [~zabetak] for pushing it forward. [~alon.eldar], could you take over it and open another PR? > Implicit lookup methods in SimpleCalciteSchema ignore case sensitivity > parameter > > > Key: CALCITE-3775 > URL: https://issues.apache.org/jira/browse/CALCITE-3775 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.21.0 >Reporter: Stamatis Zampetakis >Priority: Minor > Labels: pull-request-available > Time Spent: 7h 50m > Remaining Estimate: 0h > > SimpleCalciteSchema contains methods for finding tables, sub-schemas, and > types. The API allows to specify if the lookup will be case sensitive or > insensitive but the respective parameter is ignored in the implementation of > the method. > It is not a big problem since most of the time CachingCalciteSchema is used, > which does not have this problem. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-2854) Codegen compile error when implementing unary minus function with data type BigDecimal
[
https://issues.apache.org/jira/browse/CALCITE-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu resolved CALCITE-2854.
---
Resolution: Fixed
Fixed in
[https://github.com/apache/calcite/commit/b306668d796a19ade127dce7b392f970cffd4b39],
thanks for your work [~yuqi]!
> Codegen compile error when implementing unary minus function with data type
> BigDecimal
> --
>
> Key: CALCITE-2854
> URL: https://issues.apache.org/jira/browse/CALCITE-2854
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: Lai Zhou
>Priority: Major
> Labels: pull-request-available
> Time Spent: 8h 10m
> Remaining Estimate: 0h
>
> case:
> {code:java}
> @Test public void test() throws Exception {
> final String cr = "create table t (a DECIMAL)";
> final String se = "select -a as aa from t";
> Connection c = connect();
> Statement s = c.createStatement();
> s.execute(cr);
> s.executeQuery(se);
> }
> {code}
> error msg:
> {code:java}
> Caused by: org.codehaus.commons.compiler.CompileException: Line 21, Column
> 71: Object of type "java.math.BigDecimal" cannot be converted to a numeric
> type
> {code}
> {code:java}
> public Object current() {
> final java.math.BigDecimal current = inputEnumerator.current() == null ?
> (java.math.BigDecimal) null :
> org.apache.calcite.runtime.SqlFunctions.toBigDecimal(inputEnumerator.current());
> return current == null ? (java.math.BigDecimal) null : - current;
> }
> {code}
> `- current` is not right ,we'd better constructor a new BigDecimal object to
> to replace this code snippet，like
> {code:java}
> return current == null ? (java.math.BigDecimal) null :
> SqlFunctions.negativeDecimal(current);
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-2854) Codegen compile error when implementing unary minus function with data type BigDecimal
[
https://issues.apache.org/jira/browse/CALCITE-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu updated CALCITE-2854:
--
Summary: Codegen compile error when implementing unary minus function with
data type BigDecimal (was: Codegen compile error for decimal nagate)
> Codegen compile error when implementing unary minus function with data type
> BigDecimal
> --
>
> Key: CALCITE-2854
> URL: https://issues.apache.org/jira/browse/CALCITE-2854
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: Lai Zhou
>Priority: Major
> Labels: pull-request-available
> Time Spent: 8h 10m
> Remaining Estimate: 0h
>
> case:
> {code:java}
> @Test public void test() throws Exception {
> final String cr = "create table t (a DECIMAL)";
> final String se = "select -a as aa from t";
> Connection c = connect();
> Statement s = c.createStatement();
> s.execute(cr);
> s.executeQuery(se);
> }
> {code}
> error msg:
> {code:java}
> Caused by: org.codehaus.commons.compiler.CompileException: Line 21, Column
> 71: Object of type "java.math.BigDecimal" cannot be converted to a numeric
> type
> {code}
> {code:java}
> public Object current() {
> final java.math.BigDecimal current = inputEnumerator.current() == null ?
> (java.math.BigDecimal) null :
> org.apache.calcite.runtime.SqlFunctions.toBigDecimal(inputEnumerator.current());
> return current == null ? (java.math.BigDecimal) null : - current;
> }
> {code}
> `- current` is not right ,we'd better constructor a new BigDecimal object to
> to replace this code snippet，like
> {code:java}
> return current == null ? (java.math.BigDecimal) null :
> SqlFunctions.negativeDecimal(current);
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4133) Shouldn't trim fields when it's under Union
[
https://issues.apache.org/jira/browse/CALCITE-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17161212#comment-17161212
]
Feng Zhu commented on CALCITE-4133:
---
It seems to be duplicate with CALCITE-3399.
> Shouldn't trim fields when it's under Union
> ---
>
> Key: CALCITE-4133
> URL: https://issues.apache.org/jira/browse/CALCITE-4133
> Project: Calcite
> Issue Type: Bug
>Reporter: Jiatao Tao
>Assignee: Jiatao Tao
>Priority: Major
>
> Consider case like this:
>
> {code:java}
> String sql =
> "select \n"
> + " gender, \n"
> + " count(*) \n"
> + "from \n"
> + "(\n"
> + " (select gender, empno from EMPS) \n"
> + " union \n"
> + " (select gender, empno from EMPS)\n"
> + ") group by gender";
> {code}
> And the plan is:
>
> {code:java}
> EnumerableAggregate(group=[{0}], EXPR$1=[COUNT()])
> EnumerableUnion(all=[false])
> EnumerableProject(GENDER=[$3])
> EnumerableTableScan(table=[[SALES, EMPS]])
> EnumerableProject(GENDER=[$3])
> EnumerableTableScan(table=[[SALES, EMPS]])
> {code}
>
> The col empno has been trimmed, this changes the semantics cuz union will
> distinct values.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-2854) Codegen compile error for decimal nagate
[
https://issues.apache.org/jira/browse/CALCITE-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu updated CALCITE-2854:
--
Summary: Codegen compile error for decimal nagate (was: Code generation
error for UNARY_MINUS operator call with DECIMAL operands)
> Codegen compile error for decimal nagate
>
>
> Key: CALCITE-2854
> URL: https://issues.apache.org/jira/browse/CALCITE-2854
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: Lai Zhou
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h
> Remaining Estimate: 0h
>
> case:
> {code:java}
> @Test public void test() throws Exception {
> final String cr = "create table t (a DECIMAL)";
> final String se = "select -a as aa from t";
> Connection c = connect();
> Statement s = c.createStatement();
> s.execute(cr);
> s.executeQuery(se);
> }
> {code}
> error msg:
> {code:java}
> Caused by: org.codehaus.commons.compiler.CompileException: Line 21, Column
> 71: Object of type "java.math.BigDecimal" cannot be converted to a numeric
> type
> {code}
> {code:java}
> public Object current() {
> final java.math.BigDecimal current = inputEnumerator.current() == null ?
> (java.math.BigDecimal) null :
> org.apache.calcite.runtime.SqlFunctions.toBigDecimal(inputEnumerator.current());
> return current == null ? (java.math.BigDecimal) null : - current;
> }
> {code}
> `- current` is not right ,we'd better constructor a new BigDecimal object to
> to replace this code snippet，like
> {code:java}
> return current == null ? (java.math.BigDecimal) null :
> SqlFunctions.negativeDecimal(current);
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-4000) Support OFFSET parameter in TUMBLE/HOP table functions
[
https://issues.apache.org/jira/browse/CALCITE-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu resolved CALCITE-4000.
---
Resolution: Fixed
Fixed in
[https://github.com/apache/calcite/commit/9accb166b50417763a34a1d5664ab23ae12716fd],
thanks for your work [~amaliujia]!
> Support OFFSET parameter in TUMBLE/HOP table functions
> ---
>
> Key: CALCITE-4000
> URL: https://issues.apache.org/jira/browse/CALCITE-4000
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.24.0
>
> Time Spent: 6.5h
> Remaining Estimate: 0h
>
> TUMBLE/HOP have an optional "OFFSET" parameter to indicate how many time off
> from 0 for windowing. The OFFSET parameter is of interval type.
> {code:java}
> TUMBLE(table, size [, offset])
> HOP(table, size, shift, [, offset])
> {code}
> One example:
> Say here is a TUMBLE call with the following output
> TUMBLE(table, INTERNVAL '5' MINUTE)
> ||rowtime||window_start||window_end||
> |2020-06-20 12:23:40|2020-06-20 12:20:00|2020-06-20 12:25:00|
> |2020-06-20 12:21:40|2020-06-20 12:20:00|2020-06-20 12:25:00|
> if using the optional offset parameter
> TUMBLE(table, INTERNVAL '5' MINUTE, INTERNVAL '2' MINUTE)
> ||rowtime||window_start||window_end||
> |2020-06-20 12:23:40|2020-06-20 12:22:00|2020-06-20 12:27:00|
> |2020-06-20 12:21:40|2020-06-20 12:17:00|2020-06-20 12:22:00|
> OFFSET is applied per window basis, so the example above will also apply for
> HOP.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-2861) Exception occurs when do arithmetic between decimal and other numeric types
[
https://issues.apache.org/jira/browse/CALCITE-2861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu resolved CALCITE-2861.
---
Resolution: Duplicate
> Exception occurs when do arithmetic between decimal and other numeric types
> ---
>
> Key: CALCITE-2861
> URL: https://issues.apache.org/jira/browse/CALCITE-2861
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: yuqi
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> 1. add decimal type in ReflectiveSchemaTest#EveryType
>
> {code:java}
> public static class EveryType {
>...
> public final String string;
> public final BigDecimal decimal;
> public EveryType(
> ...
> String string,
> BigDecimal decimal) {
>...
> this.string = string;
> this.decimal = decimal;
> }
> {code}
> 2. Reconstruct ReflectiveSchemaTest#CatchallSchema#everyTypes with the new
> constructor, for example you can't assign the decimal field 1.2.
> 3. Run ReflectiveSchemaTest#testOp()
> 4. Exceptions are as followings
>
> {code:java}
> java.lang.AssertionError
> at
> org.apache.calcite.sql.type.ReturnTypes.lambda$static$6(ReturnTypes.java:510)
> at
> org.apache.calcite.sql.type.SqlTypeTransformCascade.inferReturnType(SqlTypeTransformCascade.java:54)
> at
> org.apache.calcite.sql.type.SqlReturnTypeInferenceChain.inferReturnType(SqlReturnTypeInferenceChain.java:56)
> at
> org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:470)
> at
> org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:437)
> at org.apache.calcite.sql.SqlOperator.deriveType(SqlOperator.java:515)
> at
> org.apache.calcite.sql.SqlBinaryOperator.deriveType(SqlBinaryOperator.java:144)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5547)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5534)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:138)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1639)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1624)
> at
> org.apache.calcite.sql.SqlAsOperator.deriveType(SqlAsOperator.java:133)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5547)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5534)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:138)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1639)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1624)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:461)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4052)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3315)
> at
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
> at
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:977)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:953)
> at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:928)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:632)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:556)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:265)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:772)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:636)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:606)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:229)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareA
[jira] [Commented] (CALCITE-2861) Exception occurs when do arithmetic between decimal and other numeric types
[
https://issues.apache.org/jira/browse/CALCITE-2861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157111#comment-17157111
]
Feng Zhu commented on CALCITE-2861:
---
This issue has already been addressed in CALCITE-3245.
> Exception occurs when do arithmetic between decimal and other numeric types
> ---
>
> Key: CALCITE-2861
> URL: https://issues.apache.org/jira/browse/CALCITE-2861
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: yuqi
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> 1. add decimal type in ReflectiveSchemaTest#EveryType
>
> {code:java}
> public static class EveryType {
>...
> public final String string;
> public final BigDecimal decimal;
> public EveryType(
> ...
> String string,
> BigDecimal decimal) {
>...
> this.string = string;
> this.decimal = decimal;
> }
> {code}
> 2. Reconstruct ReflectiveSchemaTest#CatchallSchema#everyTypes with the new
> constructor, for example you can't assign the decimal field 1.2.
> 3. Run ReflectiveSchemaTest#testOp()
> 4. Exceptions are as followings
>
> {code:java}
> java.lang.AssertionError
> at
> org.apache.calcite.sql.type.ReturnTypes.lambda$static$6(ReturnTypes.java:510)
> at
> org.apache.calcite.sql.type.SqlTypeTransformCascade.inferReturnType(SqlTypeTransformCascade.java:54)
> at
> org.apache.calcite.sql.type.SqlReturnTypeInferenceChain.inferReturnType(SqlReturnTypeInferenceChain.java:56)
> at
> org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:470)
> at
> org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:437)
> at org.apache.calcite.sql.SqlOperator.deriveType(SqlOperator.java:515)
> at
> org.apache.calcite.sql.SqlBinaryOperator.deriveType(SqlBinaryOperator.java:144)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5547)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5534)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:138)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1639)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1624)
> at
> org.apache.calcite.sql.SqlAsOperator.deriveType(SqlAsOperator.java:133)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5547)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5534)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:138)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1639)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1624)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:461)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4052)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3315)
> at
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
> at
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:977)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:953)
> at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:928)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:632)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:556)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:265)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:772)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:636)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:606)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:229)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteM
[jira] [Resolved] (CALCITE-4112) Planner should decorrelate when CalciteConnectionConfig is null
[ https://issues.apache.org/jira/browse/CALCITE-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Zhu resolved CALCITE-4112. --- Resolution: Fixed > Planner should decorrelate when CalciteConnectionConfig is null > --- > > Key: CALCITE-4112 > URL: https://issues.apache.org/jira/browse/CALCITE-4112 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jiatao Tao >Assignee: Jiatao Tao >Priority: Minor > Labels: pull-request-available > Fix For: 1.24.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4112) Planner should decorrelate when CalciteConnectionConfig is null
[ https://issues.apache.org/jira/browse/CALCITE-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Zhu updated CALCITE-4112: -- Fix Version/s: 1.24.0 > Planner should decorrelate when CalciteConnectionConfig is null > --- > > Key: CALCITE-4112 > URL: https://issues.apache.org/jira/browse/CALCITE-4112 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jiatao Tao >Assignee: Jiatao Tao >Priority: Minor > Labels: pull-request-available > Fix For: 1.24.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4112) Planner should decorrelate when CalciteConnectionConfig is null
[ https://issues.apache.org/jira/browse/CALCITE-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157096#comment-17157096 ] Feng Zhu commented on CALCITE-4112: --- Fixed in [https://github.com/apache/calcite/pull/2059/commits/1bf890c2e0beea6587f9a5363517bc7879f4a44c], thanks for you PR [~Aron.tao]! > Planner should decorrelate when CalciteConnectionConfig is null > --- > > Key: CALCITE-4112 > URL: https://issues.apache.org/jira/browse/CALCITE-4112 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jiatao Tao >Assignee: Jiatao Tao >Priority: Minor > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4116) Remove unused code for tracking RexNode's nullable state in codegen
[ https://issues.apache.org/jira/browse/CALCITE-4116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157095#comment-17157095 ] Feng Zhu commented on CALCITE-4116: --- Fixed in https://github.com/apache/calcite/pull/2060/commits/eee31874edde84e1ebfd05f0cc1632ebde16a4c7. > Remove unused code for tracking RexNode's nullable state in codegen > --- > > Key: CALCITE-4116 > URL: https://issues.apache.org/jira/browse/CALCITE-4116 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.23.0 >Reporter: Feng Zhu >Assignee: Feng Zhu >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > The PR in CALCITE-3224 takes RexNode's nullable state as immutable during > codegen. > Since it has been merged, we can safely remove unused code that changes and > tracks RexNode's nullable. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4116) Remove unused code for tracking RexNode's nullable state in codegen
[ https://issues.apache.org/jira/browse/CALCITE-4116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Zhu updated CALCITE-4116: -- Fix Version/s: 1.24.0 > Remove unused code for tracking RexNode's nullable state in codegen > --- > > Key: CALCITE-4116 > URL: https://issues.apache.org/jira/browse/CALCITE-4116 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.23.0 >Reporter: Feng Zhu >Assignee: Feng Zhu >Priority: Major > Labels: pull-request-available > Fix For: 1.24.0 > > Time Spent: 1h > Remaining Estimate: 0h > > The PR in CALCITE-3224 takes RexNode's nullable state as immutable during > codegen. > Since it has been merged, we can safely remove unused code that changes and > tracks RexNode's nullable. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-4116) Remove unused code for tracking RexNode's nullable state in codegen
[ https://issues.apache.org/jira/browse/CALCITE-4116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Zhu resolved CALCITE-4116. --- Resolution: Fixed > Remove unused code for tracking RexNode's nullable state in codegen > --- > > Key: CALCITE-4116 > URL: https://issues.apache.org/jira/browse/CALCITE-4116 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.23.0 >Reporter: Feng Zhu >Assignee: Feng Zhu >Priority: Major > Labels: pull-request-available > Fix For: 1.24.0 > > Time Spent: 1h > Remaining Estimate: 0h > > The PR in CALCITE-3224 takes RexNode's nullable state as immutable during > codegen. > Since it has been merged, we can safely remove unused code that changes and > tracks RexNode's nullable. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4108) ClassCastException: java.lang.Float cannot be cast to java.lang.Double
[
https://issues.apache.org/jira/browse/CALCITE-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17154481#comment-17154481
]
Feng Zhu commented on CALCITE-4108:
---
[~rr111], I have tested both branch-1.23 and master branch. They both work
well.
Could you provide more details?
Are you running the query based on calcite branch-1.23.0?
Is *tp* table in one JDBC datasource?
Is *o* is in another JDBC datasource or a datasource with other type?
Which is the underlying database?
> ClassCastException: java.lang.Float cannot be cast to java.lang.Double
> --
>
> Key: CALCITE-4108
> URL: https://issues.apache.org/jira/browse/CALCITE-4108
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
> Environment: JDK 1.8
>Reporter: Roman Churganov
>Assignee: Feng Zhu
>Priority: Critical
>
> running a query like :
> {{select tp.o, MAX( tp.n )}}
> {{ from tp join o on tp.o = o.k}}
> {{ group by tp.o}}
> type of tp.n in Schema is FLOAT, value of tp.n returned from table is of type
> java.lang.Float, Calcite makes grouping internally and get error
>
> {{java.lang.ClassCastException: java.lang.Float cannot be cast to
> java.lang.Doublejava.lang.ClassCastException: java.lang.Float cannot be cast
> to java.lang.Double at Baz$1.apply(Unknown Source) at Baz$1.apply(Unknown
> Source) at Baz$1.apply(Unknown Source) at
> org.apache.calcite.adapter.enumerable.SequencedAdderAggregateLambdaFactory$AccumulatorAdderSeq.apply(SequencedAdderAggregateLambdaFactory.java:81)
> at
> org.apache.calcite.linq4j.EnumerableDefaults.groupBy_(EnumerableDefaults.java:806)
> at
> org.apache.calcite.linq4j.EnumerableDefaults.groupBy(EnumerableDefaults.java:739)
> at
> org.apache.calcite.linq4j.DefaultEnumerable.groupBy(DefaultEnumerable.java:299)
> at Baz.bind(Unknown Source)}}
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4108) ClassCastException: java.lang.Float cannot be cast to java.lang.Double
[
https://issues.apache.org/jira/browse/CALCITE-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17154477#comment-17154477
]
Feng Zhu commented on CALCITE-4108:
---
I still cannot reproduce it.
Test case1: both *tp* and *o* in same pg datasource
Test case2: *tp* in one pg datasource, *o* in another pg datasource
{code:java}
public static class Record2_0 implements java.io.Serializable {
public double f0;
public boolean f1;
public Record2_0() {}
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof Record2_0)) {
return false;
}
return this.f0 == ((Record2_0) o).f0 && this.f1 == ((Record2_0) o).f1;
} public int hashCode() {
int h = 0;
h = org.apache.calcite.runtime.Utilities.hash(h, this.f0);
h = org.apache.calcite.runtime.Utilities.hash(h, this.f1);
return h;
} public int compareTo(Record2_0 that) {
int c;
c = org.apache.calcite.runtime.Utilities.compare(this.f0, that.f0);
if (c != 0) {
return c;
}
c = org.apache.calcite.runtime.Utilities.compare(this.f1, that.f1);
if (c != 0) {
return c;
}
return 0;
} public String toString() {
return "{f0=" + this.f0 + ", f1=" + this.f1 + "}";
}}public org.apache.calcite.linq4j.Enumerable bind(final
org.apache.calcite.DataContext root) {
final org.apache.calcite.runtime.ResultSetEnumerable enumerable =
org.apache.calcite.runtime.ResultSetEnumerable.of((javax.sql.DataSource)
root.getRootSchema().getSubSchema("pg").unwrap(javax.sql.DataSource.class),
"SELECT *\r\nFROM \"public\".\"tp\"", new
org.apache.calcite.linq4j.function.Function1() {
public org.apache.calcite.linq4j.function.Function0 apply(final
java.sql.ResultSet resultSet) {
return new org.apache.calcite.linq4j.function.Function0() {
public Object apply() {
try {
final Object[] values = new Object[2];
values[0] = resultSet.getObject(1);
values[1] = resultSet.getDouble(2);
if (resultSet.wasNull()) {
values[1] = null;
}
return values;
} catch (java.sql.SQLException e) {
throw new RuntimeException(
e);
}
}
}
;
}
public Object apply(final Object resultSet) {
return apply(
(java.sql.ResultSet) resultSet);
}
}
);
enumerable.setTimeout(root);
final org.apache.calcite.runtime.ResultSetEnumerable enumerable0 =
org.apache.calcite.runtime.ResultSetEnumerable.of((javax.sql.DataSource)
root.getRootSchema().getSubSchema("pg2").unwrap(javax.sql.DataSource.class),
"SELECT *\r\nFROM \"public\".\"oo\"", new
org.apache.calcite.linq4j.function.Function1() {
public org.apache.calcite.linq4j.function.Function0 apply(final
java.sql.ResultSet resultSet) {
return new org.apache.calcite.linq4j.function.Function0() {
public Object apply() {
try {
final Object value;
value = resultSet.getObject(1);
return value;
} catch (java.sql.SQLException e) {
throw new RuntimeException(
e);
}
}
}
;
}
public Object apply(final Object resultSet) {
return apply(
(java.sql.ResultSet) resultSet);
}
}
);
enumerable0.setTimeout(root);
java.util.List accumulatorAdders = new java.util.LinkedList();
accumulatorAdders.add(new org.apache.calcite.linq4j.function.Function2() {
public Record2_0 apply(Record2_0 acc, Object[] in) {
final Double inp1_ = (Double) in[1];
if (inp1_ != null) {
acc.f1 = true;
acc.f0 = org.apache.calcite.runtime.SqlFunctions.greater(acc.f0,
inp1_.doubleValue());
}
return acc;
}
public Record2_0 apply(Object acc, Object in) {
return apply(
(Record2_0) acc,
(Object[]) in);
}
}
);
org.apache.calcite.adapter.enumerable.AggregateLambdaFactory lambdaFactory =
new org.apache.calcite.adapter.enumerable.BasicAggregateLambdaFactory(
new org.apache.calcite.linq4j.function.Function0() {
public Object apply() {
double a0s0;
boolean a0s1;
a0s1 = false;
a0s0 = Double.longBitsToDouble(-4503599627370497L);
Record2_0 record0;
record0 = new Record2_0();
record0.f0 = a0s0;
record0.f1 = a0s1;
return record0;
}
}
,
accumulatorAdders);
return
org.apache.calcite.linq4j.EnumerableDefaults.mergeJoin(enumerable.orderBy(new
org.apache.calcite.linq4j.function.Function1() {
public String apply(Object[] v) {
return v[0] == null ? (String) null : v[0].toString();
}
public Object apply(Object v) {
return apply(
(Object[]) v);
}
}
, org.apache.calcite.linq4j.function.Funct
[jira] [Created] (CALCITE-4116) Remove unused code for tracking RexNode's nullable state in codegen
Feng Zhu created CALCITE-4116: - Summary: Remove unused code for tracking RexNode's nullable state in codegen Key: CALCITE-4116 URL: https://issues.apache.org/jira/browse/CALCITE-4116 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.23.0 Reporter: Feng Zhu Assignee: Feng Zhu The PR in CALCITE-3224 takes RexNode's nullable state as immutable during codegen. Since it has been merged, we can safely remove unused code that changes and tracks RexNode's nullable. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4108) ClassCastException: java.lang.Float cannot be cast to java.lang.Double
[
https://issues.apache.org/jira/browse/CALCITE-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17154372#comment-17154372
]
Feng Zhu commented on CALCITE-4108:
---
[~rr111], thanks for your feedback, I will make another try.
> ClassCastException: java.lang.Float cannot be cast to java.lang.Double
> --
>
> Key: CALCITE-4108
> URL: https://issues.apache.org/jira/browse/CALCITE-4108
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
> Environment: JDK 1.8
>Reporter: Roman Churganov
>Assignee: Feng Zhu
>Priority: Critical
>
> running a query like :
> {{select tp.o, MAX( tp.n )}}
> {{ from tp join o on tp.o = o.k}}
> {{ group by tp.o}}
> type of tp.n in Schema is FLOAT, value of tp.n returned from table is of type
> java.lang.Float, Calcite makes grouping internally and get error
>
> {{java.lang.ClassCastException: java.lang.Float cannot be cast to
> java.lang.Doublejava.lang.ClassCastException: java.lang.Float cannot be cast
> to java.lang.Double at Baz$1.apply(Unknown Source) at Baz$1.apply(Unknown
> Source) at Baz$1.apply(Unknown Source) at
> org.apache.calcite.adapter.enumerable.SequencedAdderAggregateLambdaFactory$AccumulatorAdderSeq.apply(SequencedAdderAggregateLambdaFactory.java:81)
> at
> org.apache.calcite.linq4j.EnumerableDefaults.groupBy_(EnumerableDefaults.java:806)
> at
> org.apache.calcite.linq4j.EnumerableDefaults.groupBy(EnumerableDefaults.java:739)
> at
> org.apache.calcite.linq4j.DefaultEnumerable.groupBy(DefaultEnumerable.java:299)
> at Baz.bind(Unknown Source)}}
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4108) ClassCastException: java.lang.Float cannot be cast to java.lang.Double
[
https://issues.apache.org/jira/browse/CALCITE-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17154150#comment-17154150
]
Feng Zhu commented on CALCITE-4108:
---
How about *tp.o*'s type? I failed to reproduce it in my local environment.
Have you tried the latest master branch?
> ClassCastException: java.lang.Float cannot be cast to java.lang.Double
> --
>
> Key: CALCITE-4108
> URL: https://issues.apache.org/jira/browse/CALCITE-4108
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
> Environment: JDK 1.8
>Reporter: Roman Churganov
>Priority: Critical
>
> running a query like :
> {{select tp.o, MAX( tp.n )}}
> {{ from tp join o on tp.o = o.k}}
> {{ group by tp.o}}
> type of tp.n in Schema is FLOAT, value of tp.n returned from table is of type
> java.lang.Float, Calcite makes grouping internally and get error
>
> {{java.lang.ClassCastException: java.lang.Float cannot be cast to
> java.lang.Doublejava.lang.ClassCastException: java.lang.Float cannot be cast
> to java.lang.Double at Baz$1.apply(Unknown Source) at Baz$1.apply(Unknown
> Source) at Baz$1.apply(Unknown Source) at
> org.apache.calcite.adapter.enumerable.SequencedAdderAggregateLambdaFactory$AccumulatorAdderSeq.apply(SequencedAdderAggregateLambdaFactory.java:81)
> at
> org.apache.calcite.linq4j.EnumerableDefaults.groupBy_(EnumerableDefaults.java:806)
> at
> org.apache.calcite.linq4j.EnumerableDefaults.groupBy(EnumerableDefaults.java:739)
> at
> org.apache.calcite.linq4j.DefaultEnumerable.groupBy(DefaultEnumerable.java:299)
> at Baz.bind(Unknown Source)}}
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (CALCITE-4108) ClassCastException: java.lang.Float cannot be cast to java.lang.Double
[
https://issues.apache.org/jira/browse/CALCITE-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu reassigned CALCITE-4108:
-
Assignee: Feng Zhu
> ClassCastException: java.lang.Float cannot be cast to java.lang.Double
> --
>
> Key: CALCITE-4108
> URL: https://issues.apache.org/jira/browse/CALCITE-4108
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
> Environment: JDK 1.8
>Reporter: Roman Churganov
>Assignee: Feng Zhu
>Priority: Critical
>
> running a query like :
> {{select tp.o, MAX( tp.n )}}
> {{ from tp join o on tp.o = o.k}}
> {{ group by tp.o}}
> type of tp.n in Schema is FLOAT, value of tp.n returned from table is of type
> java.lang.Float, Calcite makes grouping internally and get error
>
> {{java.lang.ClassCastException: java.lang.Float cannot be cast to
> java.lang.Doublejava.lang.ClassCastException: java.lang.Float cannot be cast
> to java.lang.Double at Baz$1.apply(Unknown Source) at Baz$1.apply(Unknown
> Source) at Baz$1.apply(Unknown Source) at
> org.apache.calcite.adapter.enumerable.SequencedAdderAggregateLambdaFactory$AccumulatorAdderSeq.apply(SequencedAdderAggregateLambdaFactory.java:81)
> at
> org.apache.calcite.linq4j.EnumerableDefaults.groupBy_(EnumerableDefaults.java:806)
> at
> org.apache.calcite.linq4j.EnumerableDefaults.groupBy(EnumerableDefaults.java:739)
> at
> org.apache.calcite.linq4j.DefaultEnumerable.groupBy(DefaultEnumerable.java:299)
> at Baz.bind(Unknown Source)}}
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4108) ClassCastException: java.lang.Float cannot be cast to java.lang.Double
[
https://issues.apache.org/jira/browse/CALCITE-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153185#comment-17153185
]
Feng Zhu commented on CALCITE-4108:
---
Hi, [~rr111], could you provide more schema information? i.e, data type of
*tp.o* and *o.k*?
I will try to reproduce the issue.
> ClassCastException: java.lang.Float cannot be cast to java.lang.Double
> --
>
> Key: CALCITE-4108
> URL: https://issues.apache.org/jira/browse/CALCITE-4108
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
> Environment: JDK 1.8
>Reporter: Roman Churganov
>Priority: Critical
>
> running a query like :
> {{select tp.o, MAX( tp.n )}}
> {{ from tp join o on tp.o = o.k}}
> {{ group by tp.o}}
> type of tp.n in Schema is FLOAT, value of tp.n returned from table is of type
> java.lang.Float, Calcite makes grouping internally and get error
>
> {{java.lang.ClassCastException: java.lang.Float cannot be cast to
> java.lang.Doublejava.lang.ClassCastException: java.lang.Float cannot be cast
> to java.lang.Double at Baz$1.apply(Unknown Source) at Baz$1.apply(Unknown
> Source) at Baz$1.apply(Unknown Source) at
> org.apache.calcite.adapter.enumerable.SequencedAdderAggregateLambdaFactory$AccumulatorAdderSeq.apply(SequencedAdderAggregateLambdaFactory.java:81)
> at
> org.apache.calcite.linq4j.EnumerableDefaults.groupBy_(EnumerableDefaults.java:806)
> at
> org.apache.calcite.linq4j.EnumerableDefaults.groupBy(EnumerableDefaults.java:739)
> at
> org.apache.calcite.linq4j.DefaultEnumerable.groupBy(DefaultEnumerable.java:299)
> at Baz.bind(Unknown Source)}}
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4045) ReduceExpressionsRule$JoinReduceExpressionsRule(java.lang.IndexOutOfBoundsException: Index: 0, Size: 0)
[
https://issues.apache.org/jira/browse/CALCITE-4045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17126644#comment-17126644
]
Feng Zhu commented on CALCITE-4045:
---
Hi, [~jamie12221] thanks for reporting this issue. Could you provide the sql or
a complete test case?
> ReduceExpressionsRule$JoinReduceExpressionsRule(java.lang.IndexOutOfBoundsException:
> Index: 0, Size: 0)
>
>
> Key: CALCITE-4045
> URL: https://issues.apache.org/jira/browse/CALCITE-4045
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
>Reporter: jamie12221
>Priority: Trivial
>
> {code:java}
> [DEBUG] EXPLAIN select (select c.id from db1.company as c where c.id = t.id
> limit 0 ,1) from db1.travelrecord as t where t.id = 1 LIMIT 500 ,1
> 15:32:08 T=noBindingExecutor2 [org.apache.calcite.sql.parser toTree at
> 641]-[DEBUG] Reduced `c`.`id` = `t`.`id`
> 15:32:08 T=noBindingExecutor2 [org.apache.calcite.sql.parser toTree at
> 641]-[DEBUG] Reduced `t`.`id` = 1
> 15:32:08 T=noBindingExecutor2 [org.apache.calcite.sql2rel convertQuery at
> 581]-[DEBUG] Plan after converting SqlNode to RelNode
> LogicalSort(offset=[500], fetch=[1])
> LogicalProject(EXPR$0=[$6])
> LogicalCorrelate(correlation=[$cor0], joinType=[left],
> requiredColumns=[{0}])
> LogicalFilter(condition=[=($0, 1)])
> LogicalTableScan(table=[[db1, travelrecord]])
> LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)])
> LogicalSort(offset=[0], fetch=[1])
> LogicalProject(id=[$0])
> LogicalFilter(condition=[=(CAST($0):BIGINT, $cor0.id)])
> LogicalTableScan(table=[[db1, company]])15:32:08
> T=noBindingExecutor2
> [org.apache.calcite.plan.AbstractRelOptPlanner.rule_execution_summary
> dumpRuleAttemptsInfo at 295]-[DEBUG] Rule Attempts Info for HepPlanner
> 15:32:08 T=noBindingExecutor2
> [org.apache.calcite.plan.AbstractRelOptPlanner.rule_execution_summary
> dumpRuleAttemptsInfo at 296]-[DEBUG]
> Rules
> Attempts Time (us)15:32:08 T=noBindingExecutor2
> [org.apache.calcite.plan.RelOptPlanner notifyChosen at 386]-[DEBUG] For final
> plan, using
> rel#266:LogicalSort.NONE.[](input=HepRelVertex#265,offset=500,fetch=1)
> 15:32:08 T=noBindingExecutor2 [org.apache.calcite.plan.RelOptPlanner
> notifyChosen at 386]-[DEBUG] For final plan, using
> rel#264:LogicalProject.NONE.[](input=HepRelVertex#263,exprs=[$6])
> 15:32:08 T=noBindingExecutor2 [org.apache.calcite.plan.RelOptPlanner
> notifyChosen at 386]-[DEBUG] For final plan, using
> rel#262:LogicalCorrelate.NONE.[](left=HepRelVertex#252,right=HepRelVertex#261,correlation=$cor0,joinType=left,requiredColumns={0})
> 15:32:08 T=noBindingExecutor2 [org.apache.calcite.plan.RelOptPlanner
> notifyChosen at 386]-[DEBUG] For final plan, using
> rel#251:LogicalFilter.NONE.[](input=HepRelVertex#250,condition==(1, $0))
> 15:32:08 T=noBindingExecutor2 [org.apache.calcite.plan.RelOptPlanner
> notifyChosen at 386]-[DEBUG] For final plan, using
> rel#238:LogicalTableScan.NONE.[](table=[db1, travelrecord])
> 15:32:08 T=noBindingExecutor2 [org.apache.calcite.plan.RelOptPlanner
> notifyChosen at 386]-[DEBUG] For final plan, using
> rel#260:LogicalAggregate.NONE.[](input=HepRelVertex#259,group={},agg#0=SINGLE_VALUE($0))
> 15:32:08 T=noBindingExecutor2 [org.apache.calcite.plan.RelOptPlanner
> notifyChosen at 386]-[DEBUG] For final plan, using
> rel#258:LogicalSort.NONE.[](input=HepRelVertex#257,offset=0,fetch=1)
> 15:32:08 T=noBindingExecutor2 [org.apache.calcite.plan.RelOptPlanner
> notifyChosen at 386]-[DEBUG] For final plan, using
> rel#256:LogicalProject.NONE.[](input=HepRelVertex#255,inputs=0)
> 15:32:08 T=noBindingExecutor2 [org.apache.calcite.plan.RelOptPlanner
> notifyChosen at 386]-[DEBUG] For final plan, using
> rel#254:LogicalFilter.NONE.[](input=HepRelVertex#253,condition==($cor0.id,
> CAST($0):BIGINT))
> 15:32:08 T=noBindingExecutor2 [org.apache.calcite.plan.RelOptPlanner
> notifyChosen at 386]-[DEBUG] For final plan, using
> rel#240:LogicalTableScan.NONE.[](table=[db1, company])
> 15:32:08 T=noBindingExecutor2 [org.apache.calcite.sql2rel decorrelateQuery at
> 210]-[DEBUG] Plan after removing Correlator
> LogicalSort(offset=[500], fetch=[1])
> LogicalProject(EXPR$0=[$6])
> LogicalCorrelate(correlation=[$cor0], joinType=[left],
> requiredColumns=[{0}])
> LogicalFilter(condition=[=($0, 1)])
> LogicalTableScan(table=[[db1, travelrecord]])
> LogicalAggregate(group=[{}], agg#0=[SINGLE_VALUE($0)])
> LogicalSort(offset=[0], fetch=[1])
> LogicalProject(id=[$0])
> LogicalFilter(condi
[jira] [Updated] (CALCITE-4047) at org.apache.calcite.rel.rules.PushProjector.getAdjustments(java.lang.ArrayIndexOutOfBoundsException: 0)
[
https://issues.apache.org/jira/browse/CALCITE-4047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu updated CALCITE-4047:
--
Description:
{code:java}
28:LogicalUnion(all=[true])
21:LogicalProject
20:LogicalTableScan(table=[[db1]])
23:LogicalProject
22:LogicalTableScan(table=[[db2]])
25:LogicalProject
24:LogicalTableScan(table=[[db3]])
27:LogicalProject
26:LogicalTableScan(table=[[db3]])
{code}
{code:java}
16:32:01 T=noBindingExecutor0 [org.apache.calcite.plan.RelOptPlanner fireRule
at 324]-[DEBUG] call#3: Apply rule [ProjectSetOpTransposeRule] to
[rel#44:LogicalProject.NONE.[](input=HepRelVertex#43,exprs=[1]),
rel#42:LogicalUnion.NONE.[](input#0=HepRelVertex#32,input#1=HepRelVertex#35,input#2=HepRelVertex#38,input#3=HepRelVertex#41,all=true)]16:32:01
T=noBindingExecutor0 [io.mycat.proxy.session.Session setLastMessage at
122]-[ERROR] java.lang.ArrayIndexOutOfBoundsException: 0 at
org.apache.calcite.rel.rules.PushProjector.getAdjustments(PushProjector.java:572)
at
org.apache.calcite.rel.rules.ProjectSetOpTransposeRule.onMatch(ProjectSetOpTransposeRule.java:92)
at
org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:338)
at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:540) at
org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:405) at
org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:270)
at
org.apache.calcite.plan.hep.HepInstruction$RuleCollection.execute(HepInstruction.java:74)
at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:201)
at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:188)
{code}
was:
{code:java}
28:LogicalUnion(all=[true])
21:LogicalProject
20:LogicalTableScan(table=[[db1]])
23:LogicalProject
22:LogicalTableScan(table=[[db2]])
25:LogicalProject
24:LogicalTableScan(table=[[db3]])
27:LogicalProject
26:LogicalTableScan(table=[[db3]])
{code}
{code:java}
16:32:01 T=noBindingExecutor0 [org.apache.calcite.plan.RelOptPlanner fireRule
at 324]-[DEBUG] call#3: Apply rule [ProjectSetOpTransposeRule] to
[rel#44:LogicalProject.NONE.[](input=HepRelVertex#43,exprs=[1]),
rel#42:LogicalUnion.NONE.[](input#0=HepRelVertex#32,input#1=HepRelVertex#35,input#2=HepRelVertex#38,input#3=HepRelVertex#41,all=true)]16:32:01
T=noBindingExecutor0 [io.mycat.proxy.session.Session setLastMessage at
122]-[ERROR] java.lang.ArrayIndexOutOfBoundsException: 0 at
org.apache.calcite.rel.rules.PushProjector.getAdjustments(PushProjector.java:572)
at
org.apache.calcite.rel.rules.ProjectSetOpTransposeRule.onMatch(ProjectSetOpTransposeRule.java:92)
at
org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:338)
at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:540) at
org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:405) at
org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:270)
at
org.apache.calcite.plan.hep.HepInstruction$RuleCollection.execute(HepInstruction.java:74)
at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:201)
at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:188)
{code}
[travelrecord|http://dict.youdao.com/search?q=travelrecord&keyfrom=chrome.extension]
[详细|http://www.youdao.com/search?q=travelrecord&ue=utf8&keyfrom=chrome.extension]X
没有英汉互译结果
[请尝试网页搜索|http://www.youdao.com/search?q=travelrecord&ue=utf8&keyfrom=chrome.extension]
> at
> org.apache.calcite.rel.rules.PushProjector.getAdjustments(java.lang.ArrayIndexOutOfBoundsException:
> 0)
> -
>
> Key: CALCITE-4047
> URL: https://issues.apache.org/jira/browse/CALCITE-4047
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
>Reporter: jamie12221
>Priority: Trivial
>
>
>
> {code:java}
> 28:LogicalUnion(all=[true])
> 21:LogicalProject
> 20:LogicalTableScan(table=[[db1]])
> 23:LogicalProject
> 22:LogicalTableScan(table=[[db2]])
> 25:LogicalProject
> 24:LogicalTableScan(table=[[db3]])
> 27:LogicalProject
> 26:LogicalTableScan(table=[[db3]])
> {code}
>
>
> {code:java}
> 16:32:01 T=noBindingExecutor0 [org.apache.calcite.plan.RelOptPlanner fireRule
> at 324]-[DEBUG] call#3: Apply rule [ProjectSetOpTransposeRule] to
> [rel#44:LogicalProject.NONE.[](input=HepRelVertex#43,exprs=[1]),
> rel#42:LogicalUnion.NONE.[](input#0=HepRelVertex#32,input#1=HepRelVertex#35,input#2=HepRelVertex#38,input#3=HepRelVertex#41,all=true)]16:32:01
> T=noBindingExecutor0 [io.mycat.proxy.session.Session setLastMessage at
> 122]-[ERROR] java.lang.ArrayIndexOutO
[jira] [Commented] (CALCITE-4040) Nonstandard SqlAggFunction rollup NPE Exception
[
https://issues.apache.org/jira/browse/CALCITE-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125641#comment-17125641
]
Feng Zhu commented on CALCITE-4040:
---
[~xzh_dz], thanks for clarification. Current Jira title&commit message is not
very intuitive, could you improve it?
> Nonstandard SqlAggFunction rollup NPE Exception
> ---
>
> Key: CALCITE-4040
> URL: https://issues.apache.org/jira/browse/CALCITE-4040
> Project: Calcite
> Issue Type: Wish
>Reporter: xzh_dz
>Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When i try to rollup some SqlAggFunctions in my project,I find something
> wrong.
> A case can be reproduced as below:
> MaterializationTest:
> {code:java}
> @Test public void testSqlAggFunctionRollup() {
> checkNoMaterialize(
> "select \"empid\", stddev_pop(\"deptno\") from \"emps\" group by
> \"empid\", \"deptno\"",
> "select \"empid\", stddev_pop(\"deptno\") from \"emps\" group by
> \"empid\"",
> HR_FKUK_MODEL);
> }
> {code}
> If sql is not materialized, it should not throw an exception.
> Exception:
> {code:java}
> java.sql.SQLException: Error while executing SQL "explain plan for select
> "empid", stddev_pop("deptno") from "emps" group by "empid"": null
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:528)
> Caused by: java.lang.NullPointerException
> at java.util.Objects.requireNonNull(Objects.java:203)
> at
> org.apache.calcite.rel.core.AggregateCall.(AggregateCall.java:98)
> at
> org.apache.calcite.rel.core.AggregateCall.create(AggregateCall.java:198)
> at
> org.apache.calcite.plan.SubstitutionVisitor.unifyAggregates(SubstitutionVisitor.java:1854)
> at
> org.apache.calcite.plan.SubstitutionVisitor$AggregateToAggregateUnifyRule.apply(SubstitutionVisitor.java:1545)
> at
> org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:544)
> at
> org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:478)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4040) Nonstandard SqlAggFunction rollup NPE Exception
[
https://issues.apache.org/jira/browse/CALCITE-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125489#comment-17125489
]
Feng Zhu commented on CALCITE-4040:
---
Is this Exception only thrown in non-standard aggregation functions?
> Nonstandard SqlAggFunction rollup NPE Exception
> ---
>
> Key: CALCITE-4040
> URL: https://issues.apache.org/jira/browse/CALCITE-4040
> Project: Calcite
> Issue Type: Wish
>Reporter: xzh_dz
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When i try to rollup some SqlAggFunctions in my project,I find something
> wrong.
> A case can be reproduced as below:
> MaterializationTest:
> {code:java}
> @Test public void testSqlAggFunctionRollup() {
> checkNoMaterialize(
> "select \"empid\", stddev_pop(\"deptno\") from \"emps\" group by
> \"empid\", \"deptno\"",
> "select \"empid\", stddev_pop(\"deptno\") from \"emps\" group by
> \"empid\"",
> HR_FKUK_MODEL);
> }
> {code}
> If sql is not materialized, it should not throw an exception.
> Exception:
> {code:java}
> java.sql.SQLException: Error while executing SQL "explain plan for select
> "empid", stddev_pop("deptno") from "emps" group by "empid"": null
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
> at
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:528)
> Caused by: java.lang.NullPointerException
> at java.util.Objects.requireNonNull(Objects.java:203)
> at
> org.apache.calcite.rel.core.AggregateCall.(AggregateCall.java:98)
> at
> org.apache.calcite.rel.core.AggregateCall.create(AggregateCall.java:198)
> at
> org.apache.calcite.plan.SubstitutionVisitor.unifyAggregates(SubstitutionVisitor.java:1854)
> at
> org.apache.calcite.plan.SubstitutionVisitor$AggregateToAggregateUnifyRule.apply(SubstitutionVisitor.java:1545)
> at
> org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:544)
> at
> org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:478)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4035) Adapter for REST interface with query or path parameters
[
https://issues.apache.org/jira/browse/CALCITE-4035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120709#comment-17120709
]
Feng Zhu commented on CALCITE-4035:
---
Great! We also have some similar plans on REST APIs, thrift interface and etc.
Do you start this work?
> Adapter for REST interface with query or path parameters
>
>
> Key: CALCITE-4035
> URL: https://issues.apache.org/jira/browse/CALCITE-4035
> Project: Calcite
> Issue Type: New Feature
>Reporter: David Hait
>Priority: Major
>
> A typical microservice with a REST interface will be queryable by an HTTP GET
> with query parameters or path parameters, and return JSON. It would be
> useful to have an adapter that allows access to this sort of interface. One
> use case would be to allow a join between a database table and a table which
> represents the interface. SQL "where" items could be mapped to specific
> query parameters or path parameters on the web API.
> So, for example,
> {{ SELECT * from service.MyEndpoint where id=123;}}
> could get translated into
> {{ GET http:/service/MyEndpoint/123}}
> or alternatively
> {{ GET [http://service/MyEndpoint?id=123]}}
> The mapping between SQL terms and URL path would be specified in a model
> file.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4010) Revise codegen for EnumerableMergeJoin to accept any sort order
[ https://issues.apache.org/jira/browse/CALCITE-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110785#comment-17110785 ] Feng Zhu commented on CALCITE-4010: --- I'd like to take it. > Revise codegen for EnumerableMergeJoin to accept any sort order > --- > > Key: CALCITE-4010 > URL: https://issues.apache.org/jira/browse/CALCITE-4010 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Haisheng Yuan >Priority: Major > > Given MergeJoin on foo.a=bar.a and foo.b=bar.b, > The codegen for EnumerableMergeJoin always sort tuples by ascending order, > nulls last, but after 1.23.0 calcite can generate MergeJoin on collation of > (b,a), or even (a,b,c), (b,a,c), with any asc/desc, null direction, which are > all legit. So the codegen should sort tuples exactly according to the > requirement of collation. > See > https://github.com/apache/calcite/blob/master/linq4j/src/main/java/org/apache/calcite/linq4j/EnumerableDefaults.java#L1966 > and > https://github.com/apache/calcite/pull/1833#discussion_r384445110 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (CALCITE-4010) Revise codegen for EnumerableMergeJoin to accept any sort order
[ https://issues.apache.org/jira/browse/CALCITE-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Zhu reassigned CALCITE-4010: - Assignee: Feng Zhu > Revise codegen for EnumerableMergeJoin to accept any sort order > --- > > Key: CALCITE-4010 > URL: https://issues.apache.org/jira/browse/CALCITE-4010 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Haisheng Yuan >Assignee: Feng Zhu >Priority: Major > > Given MergeJoin on foo.a=bar.a and foo.b=bar.b, > The codegen for EnumerableMergeJoin always sort tuples by ascending order, > nulls last, but after 1.23.0 calcite can generate MergeJoin on collation of > (b,a), or even (a,b,c), (b,a,c), with any asc/desc, null direction, which are > all legit. So the codegen should sort tuples exactly according to the > requirement of collation. > See > https://github.com/apache/calcite/blob/master/linq4j/src/main/java/org/apache/calcite/linq4j/EnumerableDefaults.java#L1966 > and > https://github.com/apache/calcite/pull/1833#discussion_r384445110 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3998) Bad datatype for sum(n), it should be BIGINT but it is sometimes INTEGER
[
https://issues.apache.org/jira/browse/CALCITE-3998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17106848#comment-17106848
]
Feng Zhu commented on CALCITE-3998:
---
{quote}I also noted that sometimes the type of sum(N) where N is an INTEGER
column sometimes it is now reported by Calcite as INTEGER and sometimes as a
BIGINT. In 1.22 every time is reported as BIGINT.{quote}
I guess this non-deterministic behavior may be related with planner and rules.
_RelDataTypeSystem.deriveSumType_ is deterministic and in calcite-1.22.0, it
always returns the argument type.
> Bad datatype for sum(n), it should be BIGINT but it is sometimes INTEGER
>
>
> Key: CALCITE-3998
> URL: https://issues.apache.org/jira/browse/CALCITE-3998
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.23.0
>Reporter: Enrico Olivelli
>Priority: Blocker
>
> I also noted that sometimes the type of sum(N) where N is an INTEGER column
> sometimes it is now reported by Calcite as INTEGER and sometimes as a BIGINT.
> In 1.22 every time is reported as BIGINT.
> So we have another test failing.
> SELECT sum(n1), count(*) as cc, k1
> FROM tblspace1.tsql
> GROUP by k1
> ORDER BY sum(n1)
> Here sum(n1) is reported now a INTEGER, previously it was a BIGINT. I would
> prefer to see it as a BIGINT in order to prevent overflows
> Here are the plans:
> {noformat}
> INFO: Query: SELECT sum(n1), count(*) as cc, k1 FROM tblspace1.tsql GROUP by
> k1 ORDER BY sum(n1) -- Logical Plan
> LogicalSort(sort0=[$0], dir0=[ASC]): rowcount = 2.0, cumulative cost =
> {10.52595367432 rows, 37.0 cpu, 0.0 io}, id = 1038
> LogicalProject(EXPR$0=[$1], CC=[$2], K1=[$0]): rowcount = 2.0, cumulative
> cost = {8.52595367432 rows, 13.0 cpu, 0.0 io}, id = 1037
> LogicalAggregate(group=[{0}], EXPR$0=[SUM($1)], CC=[COUNT()]): rowcount =
> 2.0, cumulative cost = {6.52595367432 rows, 7.0 cpu, 0.0 io}, id = 1035
> LogicalProject(K1=[$0], n1=[$1]): rowcount = 2.0, cumulative cost =
> {4.0 rows, 7.0 cpu, 0.0 io}, id = 1034
> LogicalTableScan(table=[[tblspace1, tsql]]): rowcount = 2.0,
> cumulative cost = {2.0 rows, 3.0 cpu, 0.0 io}, id = 1032
> May 12, 2020 11:07:37 AM herddb.sql.CalcitePlanner runPlanner
> INFO: Query: SELECT sum(n1), count(*) as cc, k1 FROM tblspace1.tsql GROUP by
> k1 ORDER BY sum(n1) -- Best Plan
> EnumerableSort(sort0=[$0], dir0=[ASC]): rowcount = 2.0, cumulative cost =
> {5.0 rows, 31.0 cpu, 0.0 io}, id = 1245
> EnumerableProject(EXPR$0=[$1], CC=[1:BIGINT], K1=[$0]): rowcount = 2.0,
> cumulative cost = {3.0 rows, 7.0 cpu, 0.0 io}, id = 1244
> EnumerableInterpreter: rowcount = 2.0, cumulative cost = {1.0 rows, 1.0
> cpu, 0.0 io}, id = 1243
> BindableTableScan(table=[[tblspace1, tsql]], projects=[[0, 1]]):
> rowcount = 2.0, cumulative cost = {0.016 rows, 0.024 cpu, 0.0 io}, id = 1055
> {noformat}
> Within the same test case with the same tables the result of this query is
> not changed
> SELECT sum(n1) as ss, min(n1) as mi, max(n1) as ma FROM tblspace1.tsql
> INFO: Query: SELECT sum(n1) as ss, min(n1) as mi, max(n1) as ma FROM
> tblspace1.tsql -- Logical Plan
> {noformat}
> LogicalAggregate(group=[{}], SS=[SUM($0)], MI=[MIN($0)], MA=[MAX($0)]):
> rowcount = 1.0, cumulative cost = {5.387500047683716 rows, 5.0 cpu, 0.0 io},
> id = 1253
> LogicalProject(n1=[$1]): rowcount = 2.0, cumulative cost = {4.0 rows, 5.0
> cpu, 0.0 io}, id = 1252
> LogicalTableScan(table=[[tblspace1, tsql]]): rowcount = 2.0, cumulative
> cost = {2.0 rows, 3.0 cpu, 0.0 io}, id = 1250
> May 12, 2020 11:08:48 AM herddb.sql.CalcitePlanner runPlanner
> INFO: Query: SELECT sum(n1) as ss, min(n1) as mi, max(n1) as ma FROM
> tblspace1.tsql -- Best Plan
> EnumerableAggregate(group=[{}], SS=[SUM($0)], MI=[MIN($0)], MA=[MAX($0)]):
> rowcount = 1.0, cumulative cost = {2.387500047683716 rows, 1.0 cpu, 0.0 io},
> id = 1295
> EnumerableInterpreter: rowcount = 2.0, cumulative cost = {1.0 rows, 1.0
> cpu, 0.0 io}, id = 1294
> BindableTableScan(table=[[tblspace1, tsql]], projects=[[1]]): rowcount =
> 2.0, cumulative cost = {0.012 rows, 0.018002 cpu, 0.0 io}, id =
> 1265
> {noformat}
> This is the test on HerdDB
> https://github.com/diennea/herddb/blob/vote-calcite-123/herddb-core/src/test/java/herddb/sql/SimplerPlannerTest.java#L237
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3780) SESSION Table-valued Function
[
https://issues.apache.org/jira/browse/CALCITE-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu resolved CALCITE-3780.
---
Fix Version/s: 1.23.0
Resolution: Fixed
Fixed in
[https://github.com/apache/calcite/commit/890eb61ef486e2192110cefe4cac5aa6f150],
thanks for your work [~amaliujia]!
> SESSION Table-valued Function
> -
>
> Key: CALCITE-3780
> URL: https://issues.apache.org/jira/browse/CALCITE-3780
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
> Fix For: 1.23.0
>
>
> We can create SESSION table-valued function to replace GROUP BY SESSION for
> inactive gap session functionality:
> {code:sql}
> SELECT *
> FROM TABLE SESSION (
> data => TABLE Bid ,
> timecol => DESCRIPTOR ( bidtime ) ,
> keycol => DESCRIPTOR(key),
> inactive_gap => INTERVAL '10' MINUTES )
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-3737) HOP Table-valued Function
[
https://issues.apache.org/jira/browse/CALCITE-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17103283#comment-17103283
]
Feng Zhu edited comment on CALCITE-3737 at 5/9/20, 12:26 PM:
-
Fixed via
[https://github.com/apache/calcite/commit/40e588de5f999034e5030b12cdbc90f4073808fe],
thanks for your PR [~amaliujia]!
was (Author: donnyzone):
Fixed via
[https://github.com/apache/calcite/commit/890eb61ef486e2192110cefe4cac5aa6f150],
thanks for your PR [~amaliujia]!
> HOP Table-valued Function
> -
>
> Key: CALCITE-3737
> URL: https://issues.apache.org/jira/browse/CALCITE-3737
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 20h
> Remaining Estimate: 0h
>
> Hopping windows place intervals of a fixed size evenly spaced across event
> time. Most importantly, in the most common use a given event time timestamp
> will generally fall into more than one window.
> The table-valued function Hop may produce zero, one, or multiple rows
> corresponding to each row of input. Hop takes four required parameters and
> one optional parameter. All parameters are analogous to those for Tumble
> except for hopsize, which specifies the duration between the starting points
> (and endpoints) of the hopping windows, allowing for overlapping windows
> (hopsize < dur, common) or gaps in the data (hopsize > dur, rarely useful).
> {code:java}
> Hop (data , timecol , dur, hopsize)
> {code}
> The return value of Hop is a relation that includes all columns of data as
> well as additional event time columns wstart and wend. Here is an example
> (from https://s.apache.org/streaming-beam-sql ):
> {code:sql}
> SELECT *
> FROM Hop (
> data=> TABLE Bids ,
> timecol => DESCRIPTOR ( bidtime ) ,
> dur => INTERVAL '10' MINUTES ,
> hopsize => INTERVAL '5' MINUTES );
> --
> | wstart | wend | bidtime | price | item |
> --
> | 8:00 | 8:10 | 8:07| $2| A|
> | 8:05 | 8:15 | 8:07| $2| A|
> | 8:05 | 8:15 | 8:11| $3| B|
> | 8:10 | 8:20 | 8:11| $3| B|
> | 8:00 | 8:10 | 8:05| $4| C|
> | 8:05 | 8:15 | 8:05| $4| C|
> | 8:00 | 8:10 | 8:09| $5| D|
> | 8:05 | 8:15 | 8:09| $5| D|
> | 8:05 | 8:15 | 8:13| $1| E|
> | 8:10 | 8:20 | 8:13| $1| E|
> | 8:10 | 8:20 | 8:17| $6| F|
> | 8:15 | 8:25 | 8:17| $6| F|
> --
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3737) HOP Table-valued Function
[
https://issues.apache.org/jira/browse/CALCITE-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu resolved CALCITE-3737.
---
Resolution: Fixed
Fixed via
[https://github.com/apache/calcite/commit/890eb61ef486e2192110cefe4cac5aa6f150],
thanks for your PR [~amaliujia]!
> HOP Table-valued Function
> -
>
> Key: CALCITE-3737
> URL: https://issues.apache.org/jira/browse/CALCITE-3737
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 20h
> Remaining Estimate: 0h
>
> Hopping windows place intervals of a fixed size evenly spaced across event
> time. Most importantly, in the most common use a given event time timestamp
> will generally fall into more than one window.
> The table-valued function Hop may produce zero, one, or multiple rows
> corresponding to each row of input. Hop takes four required parameters and
> one optional parameter. All parameters are analogous to those for Tumble
> except for hopsize, which specifies the duration between the starting points
> (and endpoints) of the hopping windows, allowing for overlapping windows
> (hopsize < dur, common) or gaps in the data (hopsize > dur, rarely useful).
> {code:java}
> Hop (data , timecol , dur, hopsize)
> {code}
> The return value of Hop is a relation that includes all columns of data as
> well as additional event time columns wstart and wend. Here is an example
> (from https://s.apache.org/streaming-beam-sql ):
> {code:sql}
> SELECT *
> FROM Hop (
> data=> TABLE Bids ,
> timecol => DESCRIPTOR ( bidtime ) ,
> dur => INTERVAL '10' MINUTES ,
> hopsize => INTERVAL '5' MINUTES );
> --
> | wstart | wend | bidtime | price | item |
> --
> | 8:00 | 8:10 | 8:07| $2| A|
> | 8:05 | 8:15 | 8:07| $2| A|
> | 8:05 | 8:15 | 8:11| $3| B|
> | 8:10 | 8:20 | 8:11| $3| B|
> | 8:00 | 8:10 | 8:05| $4| C|
> | 8:05 | 8:15 | 8:05| $4| C|
> | 8:00 | 8:10 | 8:09| $5| D|
> | 8:05 | 8:15 | 8:09| $5| D|
> | 8:05 | 8:15 | 8:13| $1| E|
> | 8:10 | 8:20 | 8:13| $1| E|
> | 8:10 | 8:20 | 8:17| $6| F|
> | 8:15 | 8:25 | 8:17| $6| F|
> --
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3866) "numeric field overflow" when running the generated SQL in PostgreSQL
[
https://issues.apache.org/jira/browse/CALCITE-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu resolved CALCITE-3866.
---
Resolution: Fixed
Fixed via
[https://github.com/apache/calcite/commit/e081c5b4227a57defe47246d8ff3e6f7cce838e4],
thanks for your PR [~winipanda]!
> "numeric field overflow" when running the generated SQL in PostgreSQL
> -
>
> Key: CALCITE-3866
> URL: https://issues.apache.org/jira/browse/CALCITE-3866
> Project: Calcite
> Issue Type: Bug
>Reporter: TANG Wen-hui
>Assignee: TANG Wen-hui
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When we try to generate a sql after applying
> AggregateJoinTransposeRule.EXTENDED, the result sql can not run on
> PostgreSQL, and throws the following exception:
> {code:java}
> PSQLException: ERROR: numeric field overflow Detail: A field with precision
> 7, scale 2 must round to an absolute value less than 10^5.
> {code}
> I found that the main reason is that :
> the return types of sum may have wrong precision when the type of
> its operand is the decimal with precision, for example:
> {code:java}
> @Test public void testSum() {
> String query =
> "select sum(e1.\"store_sales\"), sum(e2.\"store_sales\") from
> \"sales_fact_dec_1998\" as "
> + "e1 , \"sales_fact_dec_1998\" as e2 where e1.\"product_id\" =
> e2.\"product_id\"";
> String expect = "";
> HepProgramBuilder builder = new HepProgramBuilder();
> builder.addRuleClass(FilterJoinRule.class);
> builder.addRuleClass(AggregateProjectMergeRule.class);
> builder.addRuleClass(AggregateJoinTransposeRule.class);
> HepPlanner hepPlanner = new HepPlanner(builder.build());
> RuleSet rules = RuleSets.ofList(FilterJoinRule.FILTER_ON_JOIN,
> FilterJoinRule.JOIN,
> AggregateProjectMergeRule.INSTANCE,
> AggregateJoinTransposeRule.EXTENDED);
> sql(query).withPostgresql().optimize(rules, hepPlanner).ok(expect);
> }
> {code}
> the result generated sql of the query is :
> {code:java}
> SELECT SUM(CAST(\"t\".\"EXPR$0\" * \"t0\".\"$f1\" AS DECIMAL(10, 4))),
> SUM(CAST(\"t\".\"$f2\" * \"t0\".\"EXPR$1\" AS DECIMAL(10, 4)))
> FROM (SELECT \"product_id\", SUM(\"store_sales\") AS \"EXPR$0\", COUNT(*) AS
> \"$f2\"
> FROM \"foodmart\".\"sales_fact_dec_1998\"
> GROUP BY \"product_id\") AS \"t\"
> INNER JOIN (SELECT \"product_id\", COUNT(*) AS \"$f1\", SUM(\"store_sales\")
> AS \"EXPR$1\"
> FROM \"foodmart\".\"sales_fact_dec_1998\"
> GROUP BY \"product_id\") AS \"t0\" ON \"t\".\"product_id\" =
> \"t0\".\"product_id\"
> {code}
> AggregateJoinTransposeRule.EXTENDED generates a Aggregate to sum up the
> sub-totals:
> {code:java}
> // Aggregate above to sum up the sub-totals
> final List newAggCalls = new ArrayList<>();
> final int groupCount = aggregate.getGroupCount();
> final int newLeftWidth = sides.get(0).newInput.getRowType().getFieldCount();
> final List projects =
> new ArrayList<>(
> rexBuilder.identityProjects(relBuilder.peek().getRowType()));
> for (Ord aggCall : Ord.zip(aggregate.getAggCallList())) {
> final SqlAggFunction aggregation = aggCall.e.getAggregation();
> final SqlSplittableAggFunction splitter =
> Objects.requireNonNull(
> aggregation.unwrap(SqlSplittableAggFunction.class));
> final Integer leftSubTotal = sides.get(0).split.get(aggCall.i);
> final Integer rightSubTotal = sides.get(1).split.get(aggCall.i);
> newAggCalls.add(
> splitter.topSplit(rexBuilder, registry(projects),
> groupCount, relBuilder.peek().getRowType(), aggCall.e,
> leftSubTotal == null ? -1 : leftSubTotal,
> rightSubTotal == null ? -1 : rightSubTotal + newLeftWidth));
> }
> public AggregateCall topSplit(RexBuilder rexBuilder,
> Registry extra, int offset, RelDataType inputRowType,
> AggregateCall aggregateCall, int leftSubTotal, int rightSubTotal) {
> final List merges = new ArrayList<>();
> final List fieldList = inputRowType.getFieldList();
> if (leftSubTotal >= 0) {
> final RelDataType type = fieldList.get(leftSubTotal).getType();
> merges.add(rexBuilder.makeInputRef(type, leftSubTotal));
> }
> if (rightSubTotal >= 0) {
> final RelDataType type = fieldList.get(rightSubTotal).getType();
> merges.add(rexBuilder.makeInputRef(type, rightSubTotal));
> }
> RexNode node;
> switch (merges.size()) {
> case 1:
> node = merges.get(0);
> break;
> case 2:
> node = rexBuilder.makeCall(SqlStdOperatorTable.MULTIPLY, merges);
> node = rexBuilder.makeAbstractCast(aggregateCall.type, node);
> break;
> default:
> throw new AssertionError("unexpected count " + merges);
> }
> int ordinal = extra.register(node);
> return Ag
[jira] [Updated] (CALCITE-3866) "numeric field overflow" when running the generated SQL in PostgreSQL
[
https://issues.apache.org/jira/browse/CALCITE-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu updated CALCITE-3866:
--
Summary: "numeric field overflow" when running the generated SQL in
PostgreSQL (was: ReturnTypes.AGG_SUM may cause "numeric field overflow" on
PostgreSQL when generate the sql after using the rule
AggregateJoinTransposeRule.EXTENDED.)
> "numeric field overflow" when running the generated SQL in PostgreSQL
> -
>
> Key: CALCITE-3866
> URL: https://issues.apache.org/jira/browse/CALCITE-3866
> Project: Calcite
> Issue Type: Bug
>Reporter: TANG Wen-hui
>Assignee: TANG Wen-hui
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When we try to generate a sql after applying
> AggregateJoinTransposeRule.EXTENDED, the result sql can not run on
> PostgreSQL, and throws the following exception:
> {code:java}
> PSQLException: ERROR: numeric field overflow Detail: A field with precision
> 7, scale 2 must round to an absolute value less than 10^5.
> {code}
> I found that the main reason is that :
> the return types of sum may have wrong precision when the type of
> its operand is the decimal with precision, for example:
> {code:java}
> @Test public void testSum() {
> String query =
> "select sum(e1.\"store_sales\"), sum(e2.\"store_sales\") from
> \"sales_fact_dec_1998\" as "
> + "e1 , \"sales_fact_dec_1998\" as e2 where e1.\"product_id\" =
> e2.\"product_id\"";
> String expect = "";
> HepProgramBuilder builder = new HepProgramBuilder();
> builder.addRuleClass(FilterJoinRule.class);
> builder.addRuleClass(AggregateProjectMergeRule.class);
> builder.addRuleClass(AggregateJoinTransposeRule.class);
> HepPlanner hepPlanner = new HepPlanner(builder.build());
> RuleSet rules = RuleSets.ofList(FilterJoinRule.FILTER_ON_JOIN,
> FilterJoinRule.JOIN,
> AggregateProjectMergeRule.INSTANCE,
> AggregateJoinTransposeRule.EXTENDED);
> sql(query).withPostgresql().optimize(rules, hepPlanner).ok(expect);
> }
> {code}
> the result generated sql of the query is :
> {code:java}
> SELECT SUM(CAST(\"t\".\"EXPR$0\" * \"t0\".\"$f1\" AS DECIMAL(10, 4))),
> SUM(CAST(\"t\".\"$f2\" * \"t0\".\"EXPR$1\" AS DECIMAL(10, 4)))
> FROM (SELECT \"product_id\", SUM(\"store_sales\") AS \"EXPR$0\", COUNT(*) AS
> \"$f2\"
> FROM \"foodmart\".\"sales_fact_dec_1998\"
> GROUP BY \"product_id\") AS \"t\"
> INNER JOIN (SELECT \"product_id\", COUNT(*) AS \"$f1\", SUM(\"store_sales\")
> AS \"EXPR$1\"
> FROM \"foodmart\".\"sales_fact_dec_1998\"
> GROUP BY \"product_id\") AS \"t0\" ON \"t\".\"product_id\" =
> \"t0\".\"product_id\"
> {code}
> AggregateJoinTransposeRule.EXTENDED generates a Aggregate to sum up the
> sub-totals:
> {code:java}
> // Aggregate above to sum up the sub-totals
> final List newAggCalls = new ArrayList<>();
> final int groupCount = aggregate.getGroupCount();
> final int newLeftWidth = sides.get(0).newInput.getRowType().getFieldCount();
> final List projects =
> new ArrayList<>(
> rexBuilder.identityProjects(relBuilder.peek().getRowType()));
> for (Ord aggCall : Ord.zip(aggregate.getAggCallList())) {
> final SqlAggFunction aggregation = aggCall.e.getAggregation();
> final SqlSplittableAggFunction splitter =
> Objects.requireNonNull(
> aggregation.unwrap(SqlSplittableAggFunction.class));
> final Integer leftSubTotal = sides.get(0).split.get(aggCall.i);
> final Integer rightSubTotal = sides.get(1).split.get(aggCall.i);
> newAggCalls.add(
> splitter.topSplit(rexBuilder, registry(projects),
> groupCount, relBuilder.peek().getRowType(), aggCall.e,
> leftSubTotal == null ? -1 : leftSubTotal,
> rightSubTotal == null ? -1 : rightSubTotal + newLeftWidth));
> }
> public AggregateCall topSplit(RexBuilder rexBuilder,
> Registry extra, int offset, RelDataType inputRowType,
> AggregateCall aggregateCall, int leftSubTotal, int rightSubTotal) {
> final List merges = new ArrayList<>();
> final List fieldList = inputRowType.getFieldList();
> if (leftSubTotal >= 0) {
> final RelDataType type = fieldList.get(leftSubTotal).getType();
> merges.add(rexBuilder.makeInputRef(type, leftSubTotal));
> }
> if (rightSubTotal >= 0) {
> final RelDataType type = fieldList.get(rightSubTotal).getType();
> merges.add(rexBuilder.makeInputRef(type, rightSubTotal));
> }
> RexNode node;
> switch (merges.size()) {
> case 1:
> node = merges.get(0);
> break;
> case 2:
> node = rexBuilder.makeCall(SqlStdOperatorTable.MULTIPLY, merges);
> node = rexBuilder.makeAbstractCast(aggregateCall.type, node);
> break;
> default:
> throw new AssertionError("
[jira] [Resolved] (CALCITE-3447) Fix equivalents in method SubstitutionVisitor#go
[ https://issues.apache.org/jira/browse/CALCITE-3447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Zhu resolved CALCITE-3447. --- Fix Version/s: 1.23.0 Resolution: Fixed > Fix equivalents in method SubstitutionVisitor#go > > > Key: CALCITE-3447 > URL: https://issues.apache.org/jira/browse/CALCITE-3447 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: daimin >Priority: Minor > Labels: pull-request-available > Fix For: 1.23.0 > > Time Spent: 6h 50m > Remaining Estimate: 0h > > Code segment here depends on `hashcode` and `equals` methods of class > `MutableRel`: > [https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/plan/SubstitutionVisitor.java#L492-L502] > > However the implementations of class `MutableScan` delegates to class > `TableScan`, which directly relies on implementations of class `Object`. This > leads to a situation that two `MutableScan` on the exactly same table will > not be considered as equivalent. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3447) Fix equivalents in method SubstitutionVisitor#go
[ https://issues.apache.org/jira/browse/CALCITE-3447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17087667#comment-17087667 ] Feng Zhu commented on CALCITE-3447: --- Fixed in [https://github.com/apache/calcite/commit/bf0ca495a4690f9f66374e9e67af8760c09719c7], thanks for your work, [~daimin], [~jinxing6...@126.com]! > Fix equivalents in method SubstitutionVisitor#go > > > Key: CALCITE-3447 > URL: https://issues.apache.org/jira/browse/CALCITE-3447 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: daimin >Priority: Minor > Labels: pull-request-available > Time Spent: 6h 50m > Remaining Estimate: 0h > > Code segment here depends on `hashcode` and `equals` methods of class > `MutableRel`: > [https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/plan/SubstitutionVisitor.java#L492-L502] > > However the implementations of class `MutableScan` delegates to class > `TableScan`, which directly relies on implementations of class `Object`. This > leads to a situation that two `MutableScan` on the exactly same table will > not be considered as equivalent. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-3079) Successive dependent windows cannot be implemented in same expression level
[
https://issues.apache.org/jira/browse/CALCITE-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu resolved CALCITE-3079.
---
Resolution: Duplicate
> Successive dependent windows cannot be implemented in same expression level
> ---
>
> Key: CALCITE-3079
> URL: https://issues.apache.org/jira/browse/CALCITE-3079
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Recently, we encountered an IndexOutOfBoundsException when running a
> complicated query containing successive dependent windows.The issue can be
> reproduced by the following simple query on table *t1(a, b, c)*.
> {code:java}
> Q1:
> select sum(s) over (partition by aa) as ss " +
> from (
> select a as aa, sum(b) over (partition by a, c) as s
> from t1
> ) t2";{code}
> The exception is:
> {code:java}
> Exception in thread "main" java.sql.SQLException: Error while executing SQL
> "select sum(s) over (partition by aa) as ss from (select a as aa, sum(b) over
> (partition by a, c) as s from t1) t2": index (0) must be less than size (0)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
> at org.apache.calcite.JDBCDemo.main(JDBCDemo.java:70)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
> Caused by: java.lang.IndexOutOfBoundsException: index (0) must be less than
> size (0)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293)
> at
> com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:67)
> at
> org.apache.calcite.adapter.enumerable.EnumUtils$2.get(EnumUtils.java:115)
> at
> org.apache.calcite.adapter.enumerable.EnumUtils$2.get(EnumUtils.java:110)
> at
> org.apache.calcite.adapter.enumerable.EnumerableWindow.lambda$implement$0(EnumerableWindow.java:442)
> at
> org.apache.calcite.adapter.enumerable.EnumerableWindow$3.rexArguments(EnumerableWindow.java:854)
> .
> {code}
>
> However, the modified query below can be executed in a right way.
> {code:java}
> Q2:
> select sum(s) over (partition by aa) as ss " +
> from (
> select a as aa, sum(b) over (partition by a, c) + 0 as s
> from t1
> ) t2{code}
> This issue is caused by
> *_ProjectToWindowRule_*({color:#ff}CalcRelSplitter{color}). When
> splitting window expressions in Project node, the rule ignores to check
> whether a window and its input window are in the same level.Due to such
> beheavior, two successive window expressions are implemented in same level
> and the RelNode after transformation is:
> {code:java}
> LogicalProject($0=[$4])
> LogicalWindow(window#0=[window(partition {0, 2} order by [] range between
> UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($1)])],
> window#1=[window(partition {0} order by [] range between UNBOUNDED PRECEDING
> and UNBOUNDED FOLLOWING aggs [SUM($3)])])
> EnumerableTableScan(subset=[rel#7:Subset#0.ENUMERABLE.[]], table=[[ttt,
> test]]){code}
> As for *Q2*, two window expressions are not "successive", an _*Add(+)*_
> operation results to implementing them in different levels. The RelNode after
> transformation is:
> {code:java}
> LogicalProject($0=[$2])
> LogicalWindow(window#0=[window(partition {0} order by [] range between
> UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($1)])])
> LogicalProject(a=[$0], $1=[+($3, 0)])
> LogicalWindow(window#0=[window(partition {0, 2} order by [] range
> between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($1)])])
> EnumerableTableScan(subset=[rel#7:Subset#0.ENUMERABLE.[]],
> table=[[ttt, test]]){code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (CALCITE-3079) Successive dependent windows cannot be implemented in same expression level
[
https://issues.apache.org/jira/browse/CALCITE-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu reopened CALCITE-3079:
---
> Successive dependent windows cannot be implemented in same expression level
> ---
>
> Key: CALCITE-3079
> URL: https://issues.apache.org/jira/browse/CALCITE-3079
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Recently, we encountered an IndexOutOfBoundsException when running a
> complicated query containing successive dependent windows.The issue can be
> reproduced by the following simple query on table *t1(a, b, c)*.
> {code:java}
> Q1:
> select sum(s) over (partition by aa) as ss " +
> from (
> select a as aa, sum(b) over (partition by a, c) as s
> from t1
> ) t2";{code}
> The exception is:
> {code:java}
> Exception in thread "main" java.sql.SQLException: Error while executing SQL
> "select sum(s) over (partition by aa) as ss from (select a as aa, sum(b) over
> (partition by a, c) as s from t1) t2": index (0) must be less than size (0)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
> at org.apache.calcite.JDBCDemo.main(JDBCDemo.java:70)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
> Caused by: java.lang.IndexOutOfBoundsException: index (0) must be less than
> size (0)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293)
> at
> com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:67)
> at
> org.apache.calcite.adapter.enumerable.EnumUtils$2.get(EnumUtils.java:115)
> at
> org.apache.calcite.adapter.enumerable.EnumUtils$2.get(EnumUtils.java:110)
> at
> org.apache.calcite.adapter.enumerable.EnumerableWindow.lambda$implement$0(EnumerableWindow.java:442)
> at
> org.apache.calcite.adapter.enumerable.EnumerableWindow$3.rexArguments(EnumerableWindow.java:854)
> .
> {code}
>
> However, the modified query below can be executed in a right way.
> {code:java}
> Q2:
> select sum(s) over (partition by aa) as ss " +
> from (
> select a as aa, sum(b) over (partition by a, c) + 0 as s
> from t1
> ) t2{code}
> This issue is caused by
> *_ProjectToWindowRule_*({color:#ff}CalcRelSplitter{color}). When
> splitting window expressions in Project node, the rule ignores to check
> whether a window and its input window are in the same level.Due to such
> beheavior, two successive window expressions are implemented in same level
> and the RelNode after transformation is:
> {code:java}
> LogicalProject($0=[$4])
> LogicalWindow(window#0=[window(partition {0, 2} order by [] range between
> UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($1)])],
> window#1=[window(partition {0} order by [] range between UNBOUNDED PRECEDING
> and UNBOUNDED FOLLOWING aggs [SUM($3)])])
> EnumerableTableScan(subset=[rel#7:Subset#0.ENUMERABLE.[]], table=[[ttt,
> test]]){code}
> As for *Q2*, two window expressions are not "successive", an _*Add(+)*_
> operation results to implementing them in different levels. The RelNode after
> transformation is:
> {code:java}
> LogicalProject($0=[$2])
> LogicalWindow(window#0=[window(partition {0} order by [] range between
> UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($1)])])
> LogicalProject(a=[$0], $1=[+($3, 0)])
> LogicalWindow(window#0=[window(partition {0, 2} order by [] range
> between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($1)])])
> EnumerableTableScan(subset=[rel#7:Subset#0.ENUMERABLE.[]],
> table=[[ttt, test]]){code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3079) Successive dependent windows cannot be implemented in same expression level
[
https://issues.apache.org/jira/browse/CALCITE-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu resolved CALCITE-3079.
---
Resolution: Duplicate
> Successive dependent windows cannot be implemented in same expression level
> ---
>
> Key: CALCITE-3079
> URL: https://issues.apache.org/jira/browse/CALCITE-3079
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Recently, we encountered an IndexOutOfBoundsException when running a
> complicated query containing successive dependent windows.The issue can be
> reproduced by the following simple query on table *t1(a, b, c)*.
> {code:java}
> Q1:
> select sum(s) over (partition by aa) as ss " +
> from (
> select a as aa, sum(b) over (partition by a, c) as s
> from t1
> ) t2";{code}
> The exception is:
> {code:java}
> Exception in thread "main" java.sql.SQLException: Error while executing SQL
> "select sum(s) over (partition by aa) as ss from (select a as aa, sum(b) over
> (partition by a, c) as s from t1) t2": index (0) must be less than size (0)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
> at org.apache.calcite.JDBCDemo.main(JDBCDemo.java:70)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
> Caused by: java.lang.IndexOutOfBoundsException: index (0) must be less than
> size (0)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293)
> at
> com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:67)
> at
> org.apache.calcite.adapter.enumerable.EnumUtils$2.get(EnumUtils.java:115)
> at
> org.apache.calcite.adapter.enumerable.EnumUtils$2.get(EnumUtils.java:110)
> at
> org.apache.calcite.adapter.enumerable.EnumerableWindow.lambda$implement$0(EnumerableWindow.java:442)
> at
> org.apache.calcite.adapter.enumerable.EnumerableWindow$3.rexArguments(EnumerableWindow.java:854)
> .
> {code}
>
> However, the modified query below can be executed in a right way.
> {code:java}
> Q2:
> select sum(s) over (partition by aa) as ss " +
> from (
> select a as aa, sum(b) over (partition by a, c) + 0 as s
> from t1
> ) t2{code}
> This issue is caused by
> *_ProjectToWindowRule_*({color:#ff}CalcRelSplitter{color}). When
> splitting window expressions in Project node, the rule ignores to check
> whether a window and its input window are in the same level.Due to such
> beheavior, two successive window expressions are implemented in same level
> and the RelNode after transformation is:
> {code:java}
> LogicalProject($0=[$4])
> LogicalWindow(window#0=[window(partition {0, 2} order by [] range between
> UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($1)])],
> window#1=[window(partition {0} order by [] range between UNBOUNDED PRECEDING
> and UNBOUNDED FOLLOWING aggs [SUM($3)])])
> EnumerableTableScan(subset=[rel#7:Subset#0.ENUMERABLE.[]], table=[[ttt,
> test]]){code}
> As for *Q2*, two window expressions are not "successive", an _*Add(+)*_
> operation results to implementing them in different levels. The RelNode after
> transformation is:
> {code:java}
> LogicalProject($0=[$2])
> LogicalWindow(window#0=[window(partition {0} order by [] range between
> UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($1)])])
> LogicalProject(a=[$0], $1=[+($3, 0)])
> LogicalWindow(window#0=[window(partition {0, 2} order by [] range
> between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($1)])])
> EnumerableTableScan(subset=[rel#7:Subset#0.ENUMERABLE.[]],
> table=[[ttt, test]]){code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3079) Successive dependent windows cannot be implemented in same expression level
[
https://issues.apache.org/jira/browse/CALCITE-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084725#comment-17084725
]
Feng Zhu commented on CALCITE-3079:
---
[~danny0405] Thanks for reminder. It turns out Julian fixed this issue in the
end.
"The problem in this case was not with ProjectMergeRule or with
RelBuilder.project, but I fixed those also."
> Successive dependent windows cannot be implemented in same expression level
> ---
>
> Key: CALCITE-3079
> URL: https://issues.apache.org/jira/browse/CALCITE-3079
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Recently, we encountered an IndexOutOfBoundsException when running a
> complicated query containing successive dependent windows.The issue can be
> reproduced by the following simple query on table *t1(a, b, c)*.
> {code:java}
> Q1:
> select sum(s) over (partition by aa) as ss " +
> from (
> select a as aa, sum(b) over (partition by a, c) as s
> from t1
> ) t2";{code}
> The exception is:
> {code:java}
> Exception in thread "main" java.sql.SQLException: Error while executing SQL
> "select sum(s) over (partition by aa) as ss from (select a as aa, sum(b) over
> (partition by a, c) as s from t1) t2": index (0) must be less than size (0)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
> at org.apache.calcite.JDBCDemo.main(JDBCDemo.java:70)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
> Caused by: java.lang.IndexOutOfBoundsException: index (0) must be less than
> size (0)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293)
> at
> com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:67)
> at
> org.apache.calcite.adapter.enumerable.EnumUtils$2.get(EnumUtils.java:115)
> at
> org.apache.calcite.adapter.enumerable.EnumUtils$2.get(EnumUtils.java:110)
> at
> org.apache.calcite.adapter.enumerable.EnumerableWindow.lambda$implement$0(EnumerableWindow.java:442)
> at
> org.apache.calcite.adapter.enumerable.EnumerableWindow$3.rexArguments(EnumerableWindow.java:854)
> .
> {code}
>
> However, the modified query below can be executed in a right way.
> {code:java}
> Q2:
> select sum(s) over (partition by aa) as ss " +
> from (
> select a as aa, sum(b) over (partition by a, c) + 0 as s
> from t1
> ) t2{code}
> This issue is caused by
> *_ProjectToWindowRule_*({color:#ff}CalcRelSplitter{color}). When
> splitting window expressions in Project node, the rule ignores to check
> whether a window and its input window are in the same level.Due to such
> beheavior, two successive window expressions are implemented in same level
> and the RelNode after transformation is:
> {code:java}
> LogicalProject($0=[$4])
> LogicalWindow(window#0=[window(partition {0, 2} order by [] range between
> UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($1)])],
> window#1=[window(partition {0} order by [] range between UNBOUNDED PRECEDING
> and UNBOUNDED FOLLOWING aggs [SUM($3)])])
> EnumerableTableScan(subset=[rel#7:Subset#0.ENUMERABLE.[]], table=[[ttt,
> test]]){code}
> As for *Q2*, two window expressions are not "successive", an _*Add(+)*_
> operation results to implementing them in different levels. The RelNode after
> transformation is:
> {code:java}
> LogicalProject($0=[$2])
> LogicalWindow(window#0=[window(partition {0} order by [] range between
> UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($1)])])
> LogicalProject(a=[$0], $1=[+($3, 0)])
> LogicalWindow(window#0=[window(partition {0, 2} order by [] range
> between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($1)])])
> EnumerableTableScan(subset=[rel#7:Subset#0.ENUMERABLE.[]],
> table=[[ttt, test]]){code}
>
[jira] [Reopened] (CALCITE-3079) Successive dependent windows cannot be implemented in same expression level
[
https://issues.apache.org/jira/browse/CALCITE-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu reopened CALCITE-3079:
---
> Successive dependent windows cannot be implemented in same expression level
> ---
>
> Key: CALCITE-3079
> URL: https://issues.apache.org/jira/browse/CALCITE-3079
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Recently, we encountered an IndexOutOfBoundsException when running a
> complicated query containing successive dependent windows.The issue can be
> reproduced by the following simple query on table *t1(a, b, c)*.
> {code:java}
> Q1:
> select sum(s) over (partition by aa) as ss " +
> from (
> select a as aa, sum(b) over (partition by a, c) as s
> from t1
> ) t2";{code}
> The exception is:
> {code:java}
> Exception in thread "main" java.sql.SQLException: Error while executing SQL
> "select sum(s) over (partition by aa) as ss from (select a as aa, sum(b) over
> (partition by a, c) as s from t1) t2": index (0) must be less than size (0)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
> at org.apache.calcite.JDBCDemo.main(JDBCDemo.java:70)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
> Caused by: java.lang.IndexOutOfBoundsException: index (0) must be less than
> size (0)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293)
> at
> com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:67)
> at
> org.apache.calcite.adapter.enumerable.EnumUtils$2.get(EnumUtils.java:115)
> at
> org.apache.calcite.adapter.enumerable.EnumUtils$2.get(EnumUtils.java:110)
> at
> org.apache.calcite.adapter.enumerable.EnumerableWindow.lambda$implement$0(EnumerableWindow.java:442)
> at
> org.apache.calcite.adapter.enumerable.EnumerableWindow$3.rexArguments(EnumerableWindow.java:854)
> .
> {code}
>
> However, the modified query below can be executed in a right way.
> {code:java}
> Q2:
> select sum(s) over (partition by aa) as ss " +
> from (
> select a as aa, sum(b) over (partition by a, c) + 0 as s
> from t1
> ) t2{code}
> This issue is caused by
> *_ProjectToWindowRule_*({color:#ff}CalcRelSplitter{color}). When
> splitting window expressions in Project node, the rule ignores to check
> whether a window and its input window are in the same level.Due to such
> beheavior, two successive window expressions are implemented in same level
> and the RelNode after transformation is:
> {code:java}
> LogicalProject($0=[$4])
> LogicalWindow(window#0=[window(partition {0, 2} order by [] range between
> UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($1)])],
> window#1=[window(partition {0} order by [] range between UNBOUNDED PRECEDING
> and UNBOUNDED FOLLOWING aggs [SUM($3)])])
> EnumerableTableScan(subset=[rel#7:Subset#0.ENUMERABLE.[]], table=[[ttt,
> test]]){code}
> As for *Q2*, two window expressions are not "successive", an _*Add(+)*_
> operation results to implementing them in different levels. The RelNode after
> transformation is:
> {code:java}
> LogicalProject($0=[$2])
> LogicalWindow(window#0=[window(partition {0} order by [] range between
> UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($1)])])
> LogicalProject(a=[$0], $1=[+($3, 0)])
> LogicalWindow(window#0=[window(partition {0, 2} order by [] range
> between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($1)])])
> EnumerableTableScan(subset=[rel#7:Subset#0.ENUMERABLE.[]],
> table=[[ttt, test]]){code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3079) Successive dependent windows cannot be implemented in same expression level
[
https://issues.apache.org/jira/browse/CALCITE-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084668#comment-17084668
]
Feng Zhu commented on CALCITE-3079:
---
[~danny0405] CALCITE-3876 only addresses the problem in RelToSqlConverter.
For this Jira, we need to fix RelBuilder and ProjectMergeRule.
I reopen it and will update the PR in next week.
> Successive dependent windows cannot be implemented in same expression level
> ---
>
> Key: CALCITE-3079
> URL: https://issues.apache.org/jira/browse/CALCITE-3079
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Recently, we encountered an IndexOutOfBoundsException when running a
> complicated query containing successive dependent windows.The issue can be
> reproduced by the following simple query on table *t1(a, b, c)*.
> {code:java}
> Q1:
> select sum(s) over (partition by aa) as ss " +
> from (
> select a as aa, sum(b) over (partition by a, c) as s
> from t1
> ) t2";{code}
> The exception is:
> {code:java}
> Exception in thread "main" java.sql.SQLException: Error while executing SQL
> "select sum(s) over (partition by aa) as ss from (select a as aa, sum(b) over
> (partition by a, c) as s from t1) t2": index (0) must be less than size (0)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
> at org.apache.calcite.JDBCDemo.main(JDBCDemo.java:70)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
> Caused by: java.lang.IndexOutOfBoundsException: index (0) must be less than
> size (0)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293)
> at
> com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:67)
> at
> org.apache.calcite.adapter.enumerable.EnumUtils$2.get(EnumUtils.java:115)
> at
> org.apache.calcite.adapter.enumerable.EnumUtils$2.get(EnumUtils.java:110)
> at
> org.apache.calcite.adapter.enumerable.EnumerableWindow.lambda$implement$0(EnumerableWindow.java:442)
> at
> org.apache.calcite.adapter.enumerable.EnumerableWindow$3.rexArguments(EnumerableWindow.java:854)
> .
> {code}
>
> However, the modified query below can be executed in a right way.
> {code:java}
> Q2:
> select sum(s) over (partition by aa) as ss " +
> from (
> select a as aa, sum(b) over (partition by a, c) + 0 as s
> from t1
> ) t2{code}
> This issue is caused by
> *_ProjectToWindowRule_*({color:#ff}CalcRelSplitter{color}). When
> splitting window expressions in Project node, the rule ignores to check
> whether a window and its input window are in the same level.Due to such
> beheavior, two successive window expressions are implemented in same level
> and the RelNode after transformation is:
> {code:java}
> LogicalProject($0=[$4])
> LogicalWindow(window#0=[window(partition {0, 2} order by [] range between
> UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($1)])],
> window#1=[window(partition {0} order by [] range between UNBOUNDED PRECEDING
> and UNBOUNDED FOLLOWING aggs [SUM($3)])])
> EnumerableTableScan(subset=[rel#7:Subset#0.ENUMERABLE.[]], table=[[ttt,
> test]]){code}
> As for *Q2*, two window expressions are not "successive", an _*Add(+)*_
> operation results to implementing them in different levels. The RelNode after
> transformation is:
> {code:java}
> LogicalProject($0=[$2])
> LogicalWindow(window#0=[window(partition {0} order by [] range between
> UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($1)])])
> LogicalProject(a=[$0], $1=[+($3, 0)])
> LogicalWindow(window#0=[window(partition {0, 2} order by [] range
> between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($1)])])
> EnumerableTableScan(subset=[rel#7:Subset#0.ENUMERABLE.[]],
> table=[[ttt, test]]){code}
>
--
[jira] [Resolved] (CALCITE-3895) When the group sets of Aggregate is not null, union of its members should contain group key
[
https://issues.apache.org/jira/browse/CALCITE-3895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu resolved CALCITE-3895.
---
Resolution: Not A Problem
> When the group sets of Aggregate is not null, union of its members should
> contain group key
> ---
>
> Key: CALCITE-3895
> URL: https://issues.apache.org/jira/browse/CALCITE-3895
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.22.0
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Major
> Fix For: 1.23.0
>
>
> In general, the following query will fail in validation phase with “deptno
> is not being grouped”.
> {code:java}
> select deptno, count(*) as c
> from emp
> group by grouping sets (())
> {code}
> However, there is no constraint in Aggregate. For example, we can still
> create the RelNode by RelBuilder.
> {code:java}
> @Test void testAggregate6() {
> // Equivalent SQL (illegal):
> // SELECT deptno, count(*) AS C
> // FROM emp
> // GROUP BY grouping sets (())
> final RelBuilder builder = RelBuilder.create(config().build());
> RelNode aggregate = builder.scan("EMP")
> .aggregate(
> builder.groupKey(ImmutableBitSet.of(0),
> (Iterable) ImmutableList.of(
> ImmutableBitSet.of())),
> builder.countStar("C"))
> .build();
> });
> }
> {code}
> It is interesting that the rutime can process illegal RelNode and return
> result as expected.
> This issue is exposed in CALCITE-3893. However, we can also find the same
> case in _*RelMetadataTest#checkAverageRowSize*_. So I opened an individual
> Jira to discuss it.
> I propose to add the constraint below: assert
> {code:java}
> ImmutableBitSet.union(groupSets).contains(groupSet)
> : "the union of group sets should contain group key";
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3893) SQL with GROUP_ID may generate wrong plan
[
https://issues.apache.org/jira/browse/CALCITE-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu resolved CALCITE-3893.
---
Resolution: Not A Problem
> SQL with GROUP_ID may generate wrong plan
> -
>
> Key: CALCITE-3893
> URL: https://issues.apache.org/jira/browse/CALCITE-3893
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.22.0
>Reporter: Shuo Cheng
>Assignee: Feng Zhu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Consider the following SQL:
> {code:java}
> select
> deptno, group_id() as g, count(*) as c
> from emp
> group by grouping sets (deptno, (), ())
> {code}
> the plan after SqlToRel is:
> {code:java}
> LogicalUnion(all=[true])
> LogicalProject(DEPTNO=[$0], G=[0:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}, {}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalProject(DEPTNO=[$0], G=[1:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> I'm afraid there's some semantic problems here. As `groups` of the second
> Aggregate is empty, then what is `$0` meaning in the Project above it. Maybe
> that we want is:
> {code:java}
> LogicalUnion(all=[true])
> LogicalProject(DEPTNO=[$0], G=[0:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}, {}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalProject(DEPTNO=[$0], G=[1:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}]], C=[COUNT()])
> LogicalProject(DEPTNO=[null])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]]){code}
> I noticed this is introduced by CALCITE-1824, cc [~donnyzone],
> [~vladimirsitnikov].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3930) When converting Aggregate to SQL, add Having clause if a group key is not in group sets
Feng Zhu created CALCITE-3930:
-
Summary: When converting Aggregate to SQL, add Having clause if a
group key is not in group sets
Key: CALCITE-3930
URL: https://issues.apache.org/jira/browse/CALCITE-3930
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Feng Zhu
According to the discussion in CALCITE-3893 and CALCITE-3895, RelNode below is
allowed in Calcite.
{code:java}
final RelBuilder builder = RelBuilder.create(config().build());
RelNode aggregate = builder.scan("EMP")
.aggregate(
builder.groupKey(ImmutableBitSet.of(0),
(Iterable) ImmutableList.of(
ImmutableBitSet.of())),
builder.countStar("C"))
.build();
});
LogicalAggregate(group=[{0}], groups=[[{}]], C=[COUNT()])
{code}
But the SQL is not correct ('deptno is not being grouped').
{code:java}
SELECT deptno, count(*) AS C
FROM emp
GROUP BY grouping sets (())
{code}
We need to address such cases.
{code:java}
SELECT deptno, count(*) AS C
FROM emp
GROUP BY grouping sets (deptno,())
Having GROUPING_ID(deptno) = 1
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3737) HOP Table-valued Function
[
https://issues.apache.org/jira/browse/CALCITE-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu updated CALCITE-3737:
--
Fix Version/s: 1.23.0
> HOP Table-valued Function
> -
>
> Key: CALCITE-3737
> URL: https://issues.apache.org/jira/browse/CALCITE-3737
> Project: Calcite
> Issue Type: Sub-task
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> Hopping windows place intervals of a fixed size evenly spaced across event
> time. Most importantly, in the most common use a given event time timestamp
> will generally fall into more than one window.
> The table-valued function Hop may produce zero, one, or multiple rows
> corresponding to each row of input. Hop takes four required parameters and
> one optional parameter. All parameters are analogous to those for Tumble
> except for hopsize, which specifies the duration between the starting points
> (and endpoints) of the hopping windows, allowing for overlapping windows
> (hopsize < dur, common) or gaps in the data (hopsize > dur, rarely useful).
> {code:java}
> Hop (data , timecol , dur, hopsize)
> {code}
> The return value of Hop is a relation that includes all columns of data as
> well as additional event time columns wstart and wend. Here is an example
> (from https://s.apache.org/streaming-beam-sql ):
> {code:sql}
> SELECT *
> FROM Hop (
> data=> TABLE Bids ,
> timecol => DESCRIPTOR ( bidtime ) ,
> dur => INTERVAL '10' MINUTES ,
> hopsize => INTERVAL '5' MINUTES );
> --
> | wstart | wend | bidtime | price | item |
> --
> | 8:00 | 8:10 | 8:07| $2| A|
> | 8:05 | 8:15 | 8:07| $2| A|
> | 8:05 | 8:15 | 8:11| $3| B|
> | 8:10 | 8:20 | 8:11| $3| B|
> | 8:00 | 8:10 | 8:05| $4| C|
> | 8:05 | 8:15 | 8:05| $4| C|
> | 8:00 | 8:10 | 8:09| $5| D|
> | 8:05 | 8:15 | 8:09| $5| D|
> | 8:05 | 8:15 | 8:13| $1| E|
> | 8:10 | 8:20 | 8:13| $1| E|
> | 8:10 | 8:20 | 8:17| $6| F|
> | 8:15 | 8:25 | 8:17| $6| F|
> --
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3733) In JDBC adapter, when generating SQL for MYSQL, generate TIMESTAMP type as DATETIME for CAST
[ https://issues.apache.org/jira/browse/CALCITE-3733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Zhu resolved CALCITE-3733. --- Fix Version/s: 1.23.0 Resolution: Fixed > In JDBC adapter, when generating SQL for MYSQL, generate TIMESTAMP type as > DATETIME for CAST > > > Key: CALCITE-3733 > URL: https://issues.apache.org/jira/browse/CALCITE-3733 > Project: Calcite > Issue Type: Bug > Components: jdbc-adapter >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Fix For: 1.23.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Queries containing cast timestamp literal is rewritten into cast as > TIMESTAMP. This syntax is not supported with MySql. Since TIMESTAMP is > equivalent to DATETIME it should be rewritten into CAST as DATETIME. > I will provide a test case later. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3733) In JDBC adapter, when generating SQL for MYSQL, generate TIMESTAMP type as DATETIME for CAST
[ https://issues.apache.org/jira/browse/CALCITE-3733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082792#comment-17082792 ] Feng Zhu commented on CALCITE-3733: --- Fixed in [https://github.com/apache/calcite/commit/9262d8d250eebfe126b07863ba89b1d4e6ec6219], thanks for your PR [~vgarg] ! > In JDBC adapter, when generating SQL for MYSQL, generate TIMESTAMP type as > DATETIME for CAST > > > Key: CALCITE-3733 > URL: https://issues.apache.org/jira/browse/CALCITE-3733 > Project: Calcite > Issue Type: Bug > Components: jdbc-adapter >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > Queries containing cast timestamp literal is rewritten into cast as > TIMESTAMP. This syntax is not supported with MySql. Since TIMESTAMP is > equivalent to DATETIME it should be rewritten into CAST as DATETIME. > I will provide a test case later. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3893) SQL with GROUP_ID may generate wrong plan
[
https://issues.apache.org/jira/browse/CALCITE-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17081981#comment-17081981
]
Feng Zhu commented on CALCITE-3893:
---
Ok, thanks for your feedback. I'd like to make a final investigation on some
other DB products.
If there is no strong evidence for the constraints. I will close this Jira and
CALCITE-3895, and open another one for Rel-To-SQL transformation.
> SQL with GROUP_ID may generate wrong plan
> -
>
> Key: CALCITE-3893
> URL: https://issues.apache.org/jira/browse/CALCITE-3893
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.22.0
>Reporter: Shuo Cheng
>Assignee: Feng Zhu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Consider the following SQL:
> {code:java}
> select
> deptno, group_id() as g, count(*) as c
> from emp
> group by grouping sets (deptno, (), ())
> {code}
> the plan after SqlToRel is:
> {code:java}
> LogicalUnion(all=[true])
> LogicalProject(DEPTNO=[$0], G=[0:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}, {}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalProject(DEPTNO=[$0], G=[1:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> I'm afraid there's some semantic problems here. As `groups` of the second
> Aggregate is empty, then what is `$0` meaning in the Project above it. Maybe
> that we want is:
> {code:java}
> LogicalUnion(all=[true])
> LogicalProject(DEPTNO=[$0], G=[0:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}, {}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalProject(DEPTNO=[$0], G=[1:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}]], C=[COUNT()])
> LogicalProject(DEPTNO=[null])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]]){code}
> I noticed this is introduced by CALCITE-1824, cc [~donnyzone],
> [~vladimirsitnikov].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3893) SQL with GROUP_ID may generate wrong plan
[
https://issues.apache.org/jira/browse/CALCITE-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17080233#comment-17080233
]
Feng Zhu commented on CALCITE-3893:
---
{quote}If someone can back up the "a bit hard to implement", I will listen. One
data point against that argument is that when [~vlsi] and [~donnyzone]
implemented CALCITE-1824, {{EnumerableAggregate.implement}} did the right thing
first time.
But I think it is more likely that people think the plan just looks a bit weird.
{quote}
Yes, current implementor ({{EnumerableAggregate.implement)}} can handle the
plan correctly (as expected).
But if we allow this pattern, we need to fix the logic of RelToSqlConverter.
{quote}If you don't like it, or your implementor can't handle it, fell free to
write a rule to transform it to
{quote}
The runtime behevior of P1 and P2 are different.
{noformat}
P1:
LogicalAggregate(group=[{0}], groups=[[{}]], C=[COUNT()])
SQL: select null, count(*) as c from emp
P2
LogicalProject(null AS DEPTNO, $1)
LogicalAggregate(group=[{0}], groups=[[{}]], C=[COUNT()])
SQL: select null, c from (select deptno, count(*) as c from emp)
{noformat}
> SQL with GROUP_ID may generate wrong plan
> -
>
> Key: CALCITE-3893
> URL: https://issues.apache.org/jira/browse/CALCITE-3893
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.22.0
>Reporter: Shuo Cheng
>Assignee: Feng Zhu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Consider the following SQL:
> {code:java}
> select
> deptno, group_id() as g, count(*) as c
> from emp
> group by grouping sets (deptno, (), ())
> {code}
> the plan after SqlToRel is:
> {code:java}
> LogicalUnion(all=[true])
> LogicalProject(DEPTNO=[$0], G=[0:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}, {}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalProject(DEPTNO=[$0], G=[1:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> I'm afraid there's some semantic problems here. As `groups` of the second
> Aggregate is empty, then what is `$0` meaning in the Project above it. Maybe
> that we want is:
> {code:java}
> LogicalUnion(all=[true])
> LogicalProject(DEPTNO=[$0], G=[0:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}, {}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalProject(DEPTNO=[$0], G=[1:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}]], C=[COUNT()])
> LogicalProject(DEPTNO=[null])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]]){code}
> I noticed this is introduced by CALCITE-1824, cc [~donnyzone],
> [~vladimirsitnikov].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-3899) cast( string as date ) throw exception in dynamic parameter
[
https://issues.apache.org/jira/browse/CALCITE-3899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077762#comment-17077762
]
Feng Zhu edited comment on CALCITE-3899 at 4/8/20, 2:51 AM:
It is a legal PrepareStatement.
But it may be difficult to solve the problem in an elegant way until we
internally support date/time/timestamp types in Calcite.
was (Author: donnyzone):
It is a legal PrepareStatement.
It may be difficult to solve the problem until we internally support
date/time/timestamp types in Calcite.
> cast( string as date ) throw exception in dynamic parameter
> ---
>
> Key: CALCITE-3899
> URL: https://issues.apache.org/jira/browse/CALCITE-3899
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.16.0
>Reporter: Zhixiong Chen
>Priority: Major
>
> sql like::
> SELECT CAST( ? AS DATE)
> FROM
> Table GROUP BY CAST( ? AS DATE)
> the dynamic parameter is string like '2020-04-07'
>
> But it throw exception like java.lang.String cannot be cast to
> java.lang.Integer
> then I find in calcite plan it will cast string as integer and throw exception
> public Object current()
> { return (Integer) root.get("?0"); }
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3899) cast( string as date ) throw exception in dynamic parameter
[
https://issues.apache.org/jira/browse/CALCITE-3899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077762#comment-17077762
]
Feng Zhu commented on CALCITE-3899:
---
It is a legal PrepareStatement.
It may be difficult to solve the problem until we internally support
date/time/timestamp types in Calcite.
> cast( string as date ) throw exception in dynamic parameter
> ---
>
> Key: CALCITE-3899
> URL: https://issues.apache.org/jira/browse/CALCITE-3899
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.16.0
>Reporter: Zhixiong Chen
>Priority: Major
>
> sql like::
> SELECT CAST( ? AS DATE)
> FROM
> Table GROUP BY CAST( ? AS DATE)
> the dynamic parameter is string like '2020-04-07'
>
> But it throw exception like java.lang.String cannot be cast to
> java.lang.Integer
> then I find in calcite plan it will cast string as integer and throw exception
> public Object current()
> { return (Integer) root.get("?0"); }
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3895) When the group sets of Aggregate is not null, union of its members should contain group key
[
https://issues.apache.org/jira/browse/CALCITE-3895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077748#comment-17077748
]
Feng Zhu commented on CALCITE-3895:
---
{{groupSet}} == {{union(groupSets) is not a new requirement. It is inferred
from the following two constraints.}}
(1) All members of groupSets must be sub-sets of groupSet (already existed)
(2) {{groupSet}} is the subset of {{union(groupSets) (in this Jira)}}
> When the group sets of Aggregate is not null, union of its members should
> contain group key
> ---
>
> Key: CALCITE-3895
> URL: https://issues.apache.org/jira/browse/CALCITE-3895
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.22.0
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Major
> Fix For: 1.23.0
>
>
> In general, the following query will fail in validation phase with “deptno
> is not being grouped”.
> {code:java}
> select deptno, count(*) as c
> from emp
> group by grouping sets (())
> {code}
> However, there is no constraint in Aggregate. For example, we can still
> create the RelNode by RelBuilder.
> {code:java}
> @Test void testAggregate6() {
> // Equivalent SQL (illegal):
> // SELECT deptno, count(*) AS C
> // FROM emp
> // GROUP BY grouping sets (())
> final RelBuilder builder = RelBuilder.create(config().build());
> RelNode aggregate = builder.scan("EMP")
> .aggregate(
> builder.groupKey(ImmutableBitSet.of(0),
> (Iterable) ImmutableList.of(
> ImmutableBitSet.of())),
> builder.countStar("C"))
> .build();
> });
> }
> {code}
> It is interesting that the rutime can process illegal RelNode and return
> result as expected.
> This issue is exposed in CALCITE-3893. However, we can also find the same
> case in _*RelMetadataTest#checkAverageRowSize*_. So I opened an individual
> Jira to discuss it.
> I propose to add the constraint below: assert
> {code:java}
> ImmutableBitSet.union(groupSets).contains(groupSet)
> : "the union of group sets should contain group key";
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-3895) When the group sets of Aggregate is not null, union of its members should contain group key
[
https://issues.apache.org/jira/browse/CALCITE-3895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076824#comment-17076824
]
Feng Zhu edited comment on CALCITE-3895 at 4/7/20, 2:03 AM:
{quote}And equivalently, every element of {{groupSets}} should contain
{{groupSet}}.
{quote}
The java doc in Aggregate says: "All members of groupSets must be sub-sets of
groupSet."
And there is already a constraint check (with assert).
{code:java}
for (ImmutableBitSet set : groupSets) {
assert groupSet.contains(set);
}
{code}
{quote}A separate question. Should we require that {{union(groupSets)}} equals
{{groupSet}}?
{quote}
Good idea! I think we can combine these two constraints into: _union(groupSets)
equals groupSet_
Denote _groupSet_ as _g_, _groupSets_ as {_g1, g2, ... gn_}
_gi ⊆ g_ && _g ⊆ g1∪g2∪...∪gn_
<=> *g == g1∪g2∪...∪gn*
was (Author: donnyzone):
{quote}And equivalently, every element of {{groupSets}} should contain
{{groupSet}}.
{quote}
The java doc in Aggregate says: "All members of groupSets must be sub-sets of
groupSet."
And there is already a constraint check (with assert).
{code:java}
for (ImmutableBitSet set : groupSets) {
assert groupSet.contains(set);
}
{code}
{quote}A separate question. Should we require that {{union(groupSets)}} equals
{{groupSet}}?
{quote}
Good idea! I think we can combine these two constraints into: _union(groupSets)
equals groupSet_
Denote _groupSet_ as _g_, _groupSets_ as {_g1, g2, ... gn_}
_gi ⊆ g_ && _g ⊆ g1∪g2∪...∪gn_
=> *g == g1∪g2∪...∪gn*
> When the group sets of Aggregate is not null, union of its members should
> contain group key
> ---
>
> Key: CALCITE-3895
> URL: https://issues.apache.org/jira/browse/CALCITE-3895
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.22.0
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Major
> Fix For: 1.23.0
>
>
> In general, the following query will fail in validation phase with “deptno
> is not being grouped”.
> {code:java}
> select deptno, count(*) as c
> from emp
> group by grouping sets (())
> {code}
> However, there is no constraint in Aggregate. For example, we can still
> create the RelNode by RelBuilder.
> {code:java}
> @Test void testAggregate6() {
> // Equivalent SQL (illegal):
> // SELECT deptno, count(*) AS C
> // FROM emp
> // GROUP BY grouping sets (())
> final RelBuilder builder = RelBuilder.create(config().build());
> RelNode aggregate = builder.scan("EMP")
> .aggregate(
> builder.groupKey(ImmutableBitSet.of(0),
> (Iterable) ImmutableList.of(
> ImmutableBitSet.of())),
> builder.countStar("C"))
> .build();
> });
> }
> {code}
> It is interesting that the rutime can process illegal RelNode and return
> result as expected.
> This issue is exposed in CALCITE-3893. However, we can also find the same
> case in _*RelMetadataTest#checkAverageRowSize*_. So I opened an individual
> Jira to discuss it.
> I propose to add the constraint below: assert
> {code:java}
> ImmutableBitSet.union(groupSets).contains(groupSet)
> : "the union of group sets should contain group key";
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3895) When the group sets of Aggregate is not null, union of its members should contain group key
[
https://issues.apache.org/jira/browse/CALCITE-3895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076824#comment-17076824
]
Feng Zhu commented on CALCITE-3895:
---
{quote}And equivalently, every element of {{groupSets}} should contain
{{groupSet}}.
{quote}
The java doc in Aggregate says: "All members of groupSets must be sub-sets of
groupSet."
And there is already a constraint check (with assert).
{code:java}
for (ImmutableBitSet set : groupSets) {
assert groupSet.contains(set);
}
{code}
{quote}A separate question. Should we require that {{union(groupSets)}} equals
{{groupSet}}?
{quote}
Good idea! I think we can combine these two constraints into: _union(groupSets)
equals groupSet_
Denote _groupSet_ as _g_, _groupSets_ as {_g1, g2, ... gn_}
_gi ⊆ g_ && _g ⊆ g1∪g2∪...∪gn_
=> *g == g1∪g2∪...∪gn*
> When the group sets of Aggregate is not null, union of its members should
> contain group key
> ---
>
> Key: CALCITE-3895
> URL: https://issues.apache.org/jira/browse/CALCITE-3895
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.22.0
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Major
> Fix For: 1.23.0
>
>
> In general, the following query will fail in validation phase with “deptno
> is not being grouped”.
> {code:java}
> select deptno, count(*) as c
> from emp
> group by grouping sets (())
> {code}
> However, there is no constraint in Aggregate. For example, we can still
> create the RelNode by RelBuilder.
> {code:java}
> @Test void testAggregate6() {
> // Equivalent SQL (illegal):
> // SELECT deptno, count(*) AS C
> // FROM emp
> // GROUP BY grouping sets (())
> final RelBuilder builder = RelBuilder.create(config().build());
> RelNode aggregate = builder.scan("EMP")
> .aggregate(
> builder.groupKey(ImmutableBitSet.of(0),
> (Iterable) ImmutableList.of(
> ImmutableBitSet.of())),
> builder.countStar("C"))
> .build();
> });
> }
> {code}
> It is interesting that the rutime can process illegal RelNode and return
> result as expected.
> This issue is exposed in CALCITE-3893. However, we can also find the same
> case in _*RelMetadataTest#checkAverageRowSize*_. So I opened an individual
> Jira to discuss it.
> I propose to add the constraint below: assert
> {code:java}
> ImmutableBitSet.union(groupSets).contains(groupSet)
> : "the union of group sets should contain group key";
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3895) When the group sets of Aggregate is not null, union of its members should contain group key
Feng Zhu created CALCITE-3895:
-
Summary: When the group sets of Aggregate is not null, union of
its members should contain group key
Key: CALCITE-3895
URL: https://issues.apache.org/jira/browse/CALCITE-3895
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.22.0
Reporter: Feng Zhu
Assignee: Feng Zhu
Fix For: 1.23.0
In general, the following query will fail in validation phase with “deptno is
not being grouped”.
{code:java}
select deptno, count(*) as c
from emp
group by grouping sets (())
{code}
However, there is no constraint in Aggregate. For example, we can still create
the RelNode by RelBuilder.
{code:java}
@Test void testAggregate6() {
// Equivalent SQL (illegal):
// SELECT deptno, count(*) AS C
// FROM emp
// GROUP BY grouping sets (())
final RelBuilder builder = RelBuilder.create(config().build());
RelNode aggregate = builder.scan("EMP")
.aggregate(
builder.groupKey(ImmutableBitSet.of(0),
(Iterable) ImmutableList.of(
ImmutableBitSet.of())),
builder.countStar("C"))
.build();
});
}
{code}
It is interesting that the rutime can process illegal RelNode and return result
as expected.
This issue is exposed in CALCITE-3893. However, we can also find the same case
in _*RelMetadataTest#checkAverageRowSize*_. So I opened an individual Jira to
discuss it.
I propose to add the constraint below: assert
{code:java}
ImmutableBitSet.union(groupSets).contains(groupSet)
: "the union of group sets should contain group key";
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3893) SQL with GROUP_ID may generate wrong plan
[
https://issues.apache.org/jira/browse/CALCITE-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17074213#comment-17074213
]
Feng Zhu commented on CALCITE-3893:
---
The result is correct.
It seems that the rutime can process the illegal Aggregate and sets '_deptno_'
as _null_ since it is not in groupSets.
> SQL with GROUP_ID may generate wrong plan
> -
>
> Key: CALCITE-3893
> URL: https://issues.apache.org/jira/browse/CALCITE-3893
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.22.0
>Reporter: Shuo Cheng
>Assignee: Feng Zhu
>Priority: Major
> Fix For: 1.23.0
>
>
> Consider the following SQL:
> {code:java}
> select
> deptno, group_id() as g, count(*) as c
> from emp
> group by grouping sets (deptno, (), ())
> {code}
> the plan after SqlToRel is:
> {code:java}
> LogicalUnion(all=[true])
> LogicalProject(DEPTNO=[$0], G=[0:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}, {}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalProject(DEPTNO=[$0], G=[1:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> I'm afraid there's some semantic problems here. As `groups` of the second
> Aggregate is empty, then what is `$0` meaning in the Project above it. Maybe
> that we want is:
> {code:java}
> LogicalUnion(all=[true])
> LogicalProject(DEPTNO=[$0], G=[0:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}, {}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalProject(DEPTNO=[$0], G=[1:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}]], C=[COUNT()])
> LogicalProject(DEPTNO=[null])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]]){code}
> I noticed this is introduced by CALCITE-1824, cc [~donnyzone],
> [~vladimirsitnikov].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (CALCITE-3893) SQL with GROUP_ID may generate wrong plan
[
https://issues.apache.org/jira/browse/CALCITE-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu reassigned CALCITE-3893:
-
Assignee: Feng Zhu
> SQL with GROUP_ID may generate wrong plan
> -
>
> Key: CALCITE-3893
> URL: https://issues.apache.org/jira/browse/CALCITE-3893
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.22.0
>Reporter: Shuo Cheng
>Assignee: Feng Zhu
>Priority: Major
>
> Consider the following SQL:
> {code:java}
> select
> deptno, group_id() as g, count(*) as c
> from emp
> group by grouping sets (deptno, (), ())
> {code}
> the plan after SqlToRel is:
> {code:java}
> LogicalUnion(all=[true])
> LogicalProject(DEPTNO=[$0], G=[0:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}, {}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalProject(DEPTNO=[$0], G=[1:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> I'm afraid there's some semantic problems here. As `groups` of the second
> Aggregate is empty, then what is `$0` meaning in the Project above it. Maybe
> that we want is:
> {code:java}
> LogicalUnion(all=[true])
> LogicalProject(DEPTNO=[$0], G=[0:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}, {}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalProject(DEPTNO=[$0], G=[1:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}]], C=[COUNT()])
> LogicalProject(DEPTNO=[null])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]]){code}
> I noticed this is introduced by CALCITE-1824, cc [~donnyzone],
> [~vladimirsitnikov].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3893) SQL with GROUP_ID may generate wrong plan
[
https://issues.apache.org/jira/browse/CALCITE-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu updated CALCITE-3893:
--
Fix Version/s: 1.23.0
> SQL with GROUP_ID may generate wrong plan
> -
>
> Key: CALCITE-3893
> URL: https://issues.apache.org/jira/browse/CALCITE-3893
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.22.0
>Reporter: Shuo Cheng
>Assignee: Feng Zhu
>Priority: Major
> Fix For: 1.23.0
>
>
> Consider the following SQL:
> {code:java}
> select
> deptno, group_id() as g, count(*) as c
> from emp
> group by grouping sets (deptno, (), ())
> {code}
> the plan after SqlToRel is:
> {code:java}
> LogicalUnion(all=[true])
> LogicalProject(DEPTNO=[$0], G=[0:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}, {}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalProject(DEPTNO=[$0], G=[1:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> I'm afraid there's some semantic problems here. As `groups` of the second
> Aggregate is empty, then what is `$0` meaning in the Project above it. Maybe
> that we want is:
> {code:java}
> LogicalUnion(all=[true])
> LogicalProject(DEPTNO=[$0], G=[0:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}, {}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalProject(DEPTNO=[$0], G=[1:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}]], C=[COUNT()])
> LogicalProject(DEPTNO=[null])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]]){code}
> I noticed this is introduced by CALCITE-1824, cc [~donnyzone],
> [~vladimirsitnikov].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3893) SQL with GROUP_ID may generate wrong plan
[
https://issues.apache.org/jira/browse/CALCITE-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073656#comment-17073656
]
Feng Zhu commented on CALCITE-3893:
---
[~icshuo], according to the discussion, we will rewrite the query
{code:java}
select deptno, group_id() as g, count(*) as c
from emp
group by grouping sets (deptno, (), ())
{code}
as
{code:java}
select deptno, 0 as g, count(*) as c
from emp
group by grouping sets (deptno, ())
union all
select null, 1 as g, count(*) as c
from emp
group by grouping sets (())
{code}
Hence, the second sub-query's 'groupSet' should be empty (i.e., *group=[{}]*).
To validate it, you can convert it back to SQL, which will fail the validation
due to 'deptno is not being grouped'.
{code:java}
LogicalAggregate(group=[{0}], groups=[[{}]], C=[COUNT()])
LogicalProject(DEPTNO=[$7])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
select deptno, count(*) as c
from emp
group by grouping sets (())
{code}
> SQL with GROUP_ID may generate wrong plan
> -
>
> Key: CALCITE-3893
> URL: https://issues.apache.org/jira/browse/CALCITE-3893
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.22.0
>Reporter: Shuo Cheng
>Priority: Major
> Fix For: 1.23.0
>
>
> Consider the following SQL:
> {code:java}
> select
> deptno, group_id() as g, count(*) as c
> from emp
> group by grouping sets (deptno, (), ())
> {code}
> the plan after SqlToRel is:
> {code:java}
> LogicalUnion(all=[true])
> LogicalProject(DEPTNO=[$0], G=[0:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}, {}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalProject(DEPTNO=[$0], G=[1:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> I'm afraid there's some semantic problems here. As `groups` of the second
> Aggregate is empty, then what is `$0` meaning in the Project above it. Maybe
> that we want is:
> {code:java}
> LogicalUnion(all=[true])
> LogicalProject(DEPTNO=[$0], G=[0:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}, {}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalProject(DEPTNO=[$0], G=[1:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}]], C=[COUNT()])
> LogicalProject(DEPTNO=[null])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]]){code}
> I noticed this is introduced by CALCITE-1824, cc [~donnyzone],
> [~vladimirsitnikov].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3893) SQL with GROUP_ID may generate wrong plan
[
https://issues.apache.org/jira/browse/CALCITE-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073547#comment-17073547
]
Feng Zhu commented on CALCITE-3893:
---
Hi, [~icshuo], thanks for reporting the problem, you are right.
I'm just curious why the plan can also get the correct result :).
The Aggregate below is illegal. It not safe to only spliting the duplicate
group sets.
{code:java}
LogicalAggregate(group=[{0}], groups=[[{}]], C=[COUNT()])
LogicalProject(DEPTNO=[$7])
LogicalTableScan(table=[[CATALOG, SALES, EMP]])
{code}
Feel free to work on it. If you have no time, I will fix it in this weekend.
> SQL with GROUP_ID may generate wrong plan
> -
>
> Key: CALCITE-3893
> URL: https://issues.apache.org/jira/browse/CALCITE-3893
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.22.0
>Reporter: Shuo Cheng
>Priority: Major
> Fix For: 1.23.0
>
>
> Consider the following SQL:
> {code:java}
> select
> deptno, group_id() as g, count(*) as c
> from emp
> group by grouping sets (deptno, (), ())
> {code}
> the plan after SqlToRel is:
> {code:java}
> LogicalUnion(all=[true])
> LogicalProject(DEPTNO=[$0], G=[0:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}, {}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalProject(DEPTNO=[$0], G=[1:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> I'm afraid there's some semantic problems here. As `groups` of the second
> Aggregate is empty, then what is `$0` meaning in the Project above it. Maybe
> that we want is:
> {code:java}
> LogicalUnion(all=[true])
> LogicalProject(DEPTNO=[$0], G=[0:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}, {}]], C=[COUNT()])
> LogicalProject(DEPTNO=[$7])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalProject(DEPTNO=[$0], G=[1:BIGINT], C=[$1])
> LogicalAggregate(group=[{0}], groups=[[{0}]], C=[COUNT()])
> LogicalProject(DEPTNO=[null])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]]){code}
> I noticed this is introduced by CALCITE-1824, cc [~donnyzone],
> [~vladimirsitnikov].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3866) ReturnTypes.AGG_SUM may cause "numeric field overflow" on PostgreSQL when generate the sql after using the rule AggregateJoinTransposeRule.EXTENDED.
[
https://issues.apache.org/jira/browse/CALCITE-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17070639#comment-17070639
]
Feng Zhu commented on CALCITE-3866:
---
So, the issue is the incorrect type inference of
AggregateJoinTransposeRule.EXTENDED rule?
By the way, could you improve the description from an end user's perspective?
> ReturnTypes.AGG_SUM may cause "numeric field overflow" on PostgreSQL when
> generate the sql after using the rule AggregateJoinTransposeRule.EXTENDED.
> -
>
> Key: CALCITE-3866
> URL: https://issues.apache.org/jira/browse/CALCITE-3866
> Project: Calcite
> Issue Type: Bug
>Reporter: TANG Wen-hui
>Assignee: TANG Wen-hui
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When we try to generate a sql after applying
> AggregateJoinTransposeRule.EXTENDED, the result sql can not run on
> PostgreSQL, and throws the following exception:
> {code:java}
> PSQLException: ERROR: numeric field overflow Detail: A field with precision
> 7, scale 2 must round to an absolute value less than 10^5.
> {code}
> I found that the main reason is that :
> the return types of sum may have wrong precision when the type of
> its operand is the decimal with precision, for example:
> {code:java}
> @Test public void testSum() {
> String query =
> "select sum(e1.\"store_sales\"), sum(e2.\"store_sales\") from
> \"sales_fact_dec_1998\" as "
> + "e1 , \"sales_fact_dec_1998\" as e2 where e1.\"product_id\" =
> e2.\"product_id\"";
> String expect = "";
> HepProgramBuilder builder = new HepProgramBuilder();
> builder.addRuleClass(FilterJoinRule.class);
> builder.addRuleClass(AggregateProjectMergeRule.class);
> builder.addRuleClass(AggregateJoinTransposeRule.class);
> HepPlanner hepPlanner = new HepPlanner(builder.build());
> RuleSet rules = RuleSets.ofList(FilterJoinRule.FILTER_ON_JOIN,
> FilterJoinRule.JOIN,
> AggregateProjectMergeRule.INSTANCE,
> AggregateJoinTransposeRule.EXTENDED);
> sql(query).withPostgresql().optimize(rules, hepPlanner).ok(expect);
> }
> {code}
> the result generated sql of the query is :
> {code:java}
> SELECT SUM(CAST(\"t\".\"EXPR$0\" * \"t0\".\"$f1\" AS DECIMAL(10, 4))),
> SUM(CAST(\"t\".\"$f2\" * \"t0\".\"EXPR$1\" AS DECIMAL(10, 4)))
> FROM (SELECT \"product_id\", SUM(\"store_sales\") AS \"EXPR$0\", COUNT(*) AS
> \"$f2\"
> FROM \"foodmart\".\"sales_fact_dec_1998\"
> GROUP BY \"product_id\") AS \"t\"
> INNER JOIN (SELECT \"product_id\", COUNT(*) AS \"$f1\", SUM(\"store_sales\")
> AS \"EXPR$1\"
> FROM \"foodmart\".\"sales_fact_dec_1998\"
> GROUP BY \"product_id\") AS \"t0\" ON \"t\".\"product_id\" =
> \"t0\".\"product_id\"
> {code}
> AggregateJoinTransposeRule.EXTENDED generates a Aggregate to sum up the
> sub-totals:
> {code:java}
> // Aggregate above to sum up the sub-totals
> final List newAggCalls = new ArrayList<>();
> final int groupCount = aggregate.getGroupCount();
> final int newLeftWidth = sides.get(0).newInput.getRowType().getFieldCount();
> final List projects =
> new ArrayList<>(
> rexBuilder.identityProjects(relBuilder.peek().getRowType()));
> for (Ord aggCall : Ord.zip(aggregate.getAggCallList())) {
> final SqlAggFunction aggregation = aggCall.e.getAggregation();
> final SqlSplittableAggFunction splitter =
> Objects.requireNonNull(
> aggregation.unwrap(SqlSplittableAggFunction.class));
> final Integer leftSubTotal = sides.get(0).split.get(aggCall.i);
> final Integer rightSubTotal = sides.get(1).split.get(aggCall.i);
> newAggCalls.add(
> splitter.topSplit(rexBuilder, registry(projects),
> groupCount, relBuilder.peek().getRowType(), aggCall.e,
> leftSubTotal == null ? -1 : leftSubTotal,
> rightSubTotal == null ? -1 : rightSubTotal + newLeftWidth));
> }
> public AggregateCall topSplit(RexBuilder rexBuilder,
> Registry extra, int offset, RelDataType inputRowType,
> AggregateCall aggregateCall, int leftSubTotal, int rightSubTotal) {
> final List merges = new ArrayList<>();
> final List fieldList = inputRowType.getFieldList();
> if (leftSubTotal >= 0) {
> final RelDataType type = fieldList.get(leftSubTotal).getType();
> merges.add(rexBuilder.makeInputRef(type, leftSubTotal));
> }
> if (rightSubTotal >= 0) {
> final RelDataType type = fieldList.get(rightSubTotal).getType();
> merges.add(rexBuilder.makeInputRef(type, rightSubTotal));
> }
> RexNode node;
> switch (merges.size()) {
> case 1:
> node = merges.get(0);
> break;
> case 2:
> node = rexBuilder.makeCall(SqlStdOperatorTable.MULTIPLY, merges);
> node
[jira] [Resolved] (CALCITE-3864) Supports CONCAT for variable arguments
[
https://issues.apache.org/jira/browse/CALCITE-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu resolved CALCITE-3864.
---
Resolution: Fixed
> Supports CONCAT for variable arguments
> --
>
> Key: CALCITE-3864
> URL: https://issues.apache.org/jira/browse/CALCITE-3864
> Project: Calcite
> Issue Type: Bug
>Reporter: TANG Wen-hui
>Assignee: TANG Wen-hui
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 8h
> Remaining Estimate: 0h
>
> Add the Implementation for SqlLibraryOperators.CONCAT_FUNCTION in
> SqlFunctions rather than register it as a kind of || operator in
> StandardConvertletTable, because it would be better for Jdbc adaptor to
> generate CONCAT for mysql rather than || operator, since CONCAT in mysql is
> not totally same as || operator.
> According to my test, as for PG ,the result of "concat('a', 'b', 'c')" is
> equal to " 'a' || 'b' || 'c' " , which is 'abc', as for Mysql, the result of
> "concat('a', 'b', 'c')" is 'abc', but the result of " 'a'||'b'|| 'c' " is 0,
> as for oracle, concat only have two argument and the the result of
> 'a'||'b'||'c' is 'abc'.
> And the return type inference of SqlLibraryOperators.CONCAT_FUNCTION will
> cause AssertionError when the precsion of its operands are not specified.
> {code:java}
> concat(cast('a' as varchar), cast('b' as varchar),cast('c' as varchar)){code}
> {code:java}
> at
> org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:64)at
>
> org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:64)
> at
> org.apache.calcite.sql.fun.SqlLibraryOperators.lambda$static$1(SqlLibraryOperators.java:291)
> at
> org.apache.calcite.sql.type.SqlTypeTransformCascade.inferReturnType(SqlTypeTransformCascade.java:54)
> at org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:486)
> at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:453)
> at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:321) at
> org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:218) at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5858)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5845)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1800)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1785)
> at org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:260) at
> org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:423) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5552)
> at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:116) at
> org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:259) at
> org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:423) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5552)
> at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:116) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1059)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:766)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.parseAndValidate(AbstractSqlTester.java:175)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.getResultType(AbstractSqlTester.java:163)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.getColumnType(AbstractSqlTester.java:155)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.check(AbstractSqlTester.java:477)
> at
> org.apache.calcite.sql.test.SqlOperatorBaseTest$TesterImpl.check(SqlOperatorBaseTest.java:9489)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.check(AbstractSqlTester.java:462)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.checkString(AbstractSqlTester.java:447)
> at
> org.apache.calcite.sql.test.SqlOperatorBaseTest.testConcatOperator(SqlOperatorBaseTest.java:2184){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (CALCITE-3864) Supports CONCAT for variable arguments
[
https://issues.apache.org/jira/browse/CALCITE-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu closed CALCITE-3864.
-
Resolution: Fixed
> Supports CONCAT for variable arguments
> --
>
> Key: CALCITE-3864
> URL: https://issues.apache.org/jira/browse/CALCITE-3864
> Project: Calcite
> Issue Type: Bug
>Reporter: TANG Wen-hui
>Assignee: TANG Wen-hui
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 8h
> Remaining Estimate: 0h
>
> Add the Implementation for SqlLibraryOperators.CONCAT_FUNCTION in
> SqlFunctions rather than register it as a kind of || operator in
> StandardConvertletTable, because it would be better for Jdbc adaptor to
> generate CONCAT for mysql rather than || operator, since CONCAT in mysql is
> not totally same as || operator.
> According to my test, as for PG ,the result of "concat('a', 'b', 'c')" is
> equal to " 'a' || 'b' || 'c' " , which is 'abc', as for Mysql, the result of
> "concat('a', 'b', 'c')" is 'abc', but the result of " 'a'||'b'|| 'c' " is 0,
> as for oracle, concat only have two argument and the the result of
> 'a'||'b'||'c' is 'abc'.
> And the return type inference of SqlLibraryOperators.CONCAT_FUNCTION will
> cause AssertionError when the precsion of its operands are not specified.
> {code:java}
> concat(cast('a' as varchar), cast('b' as varchar),cast('c' as varchar)){code}
> {code:java}
> at
> org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:64)at
>
> org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:64)
> at
> org.apache.calcite.sql.fun.SqlLibraryOperators.lambda$static$1(SqlLibraryOperators.java:291)
> at
> org.apache.calcite.sql.type.SqlTypeTransformCascade.inferReturnType(SqlTypeTransformCascade.java:54)
> at org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:486)
> at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:453)
> at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:321) at
> org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:218) at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5858)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5845)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1800)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1785)
> at org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:260) at
> org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:423) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5552)
> at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:116) at
> org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:259) at
> org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:423) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5552)
> at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:116) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1059)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:766)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.parseAndValidate(AbstractSqlTester.java:175)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.getResultType(AbstractSqlTester.java:163)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.getColumnType(AbstractSqlTester.java:155)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.check(AbstractSqlTester.java:477)
> at
> org.apache.calcite.sql.test.SqlOperatorBaseTest$TesterImpl.check(SqlOperatorBaseTest.java:9489)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.check(AbstractSqlTester.java:462)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.checkString(AbstractSqlTester.java:447)
> at
> org.apache.calcite.sql.test.SqlOperatorBaseTest.testConcatOperator(SqlOperatorBaseTest.java:2184){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (CALCITE-3864) Supports CONCAT for variable arguments
[
https://issues.apache.org/jira/browse/CALCITE-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu reopened CALCITE-3864:
---
> Supports CONCAT for variable arguments
> --
>
> Key: CALCITE-3864
> URL: https://issues.apache.org/jira/browse/CALCITE-3864
> Project: Calcite
> Issue Type: Bug
>Reporter: TANG Wen-hui
>Assignee: TANG Wen-hui
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 8h
> Remaining Estimate: 0h
>
> Add the Implementation for SqlLibraryOperators.CONCAT_FUNCTION in
> SqlFunctions rather than register it as a kind of || operator in
> StandardConvertletTable, because it would be better for Jdbc adaptor to
> generate CONCAT for mysql rather than || operator, since CONCAT in mysql is
> not totally same as || operator.
> According to my test, as for PG ,the result of "concat('a', 'b', 'c')" is
> equal to " 'a' || 'b' || 'c' " , which is 'abc', as for Mysql, the result of
> "concat('a', 'b', 'c')" is 'abc', but the result of " 'a'||'b'|| 'c' " is 0,
> as for oracle, concat only have two argument and the the result of
> 'a'||'b'||'c' is 'abc'.
> And the return type inference of SqlLibraryOperators.CONCAT_FUNCTION will
> cause AssertionError when the precsion of its operands are not specified.
> {code:java}
> concat(cast('a' as varchar), cast('b' as varchar),cast('c' as varchar)){code}
> {code:java}
> at
> org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:64)at
>
> org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:64)
> at
> org.apache.calcite.sql.fun.SqlLibraryOperators.lambda$static$1(SqlLibraryOperators.java:291)
> at
> org.apache.calcite.sql.type.SqlTypeTransformCascade.inferReturnType(SqlTypeTransformCascade.java:54)
> at org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:486)
> at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:453)
> at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:321) at
> org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:218) at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5858)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5845)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1800)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1785)
> at org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:260) at
> org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:423) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5552)
> at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:116) at
> org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:259) at
> org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:423) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5552)
> at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:116) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1059)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:766)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.parseAndValidate(AbstractSqlTester.java:175)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.getResultType(AbstractSqlTester.java:163)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.getColumnType(AbstractSqlTester.java:155)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.check(AbstractSqlTester.java:477)
> at
> org.apache.calcite.sql.test.SqlOperatorBaseTest$TesterImpl.check(SqlOperatorBaseTest.java:9489)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.check(AbstractSqlTester.java:462)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.checkString(AbstractSqlTester.java:447)
> at
> org.apache.calcite.sql.test.SqlOperatorBaseTest.testConcatOperator(SqlOperatorBaseTest.java:2184){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (CALCITE-3864) Supports CONCAT for variable arguments
[
https://issues.apache.org/jira/browse/CALCITE-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu reopened CALCITE-3864:
---
> Supports CONCAT for variable arguments
> --
>
> Key: CALCITE-3864
> URL: https://issues.apache.org/jira/browse/CALCITE-3864
> Project: Calcite
> Issue Type: Bug
>Reporter: TANG Wen-hui
>Assignee: TANG Wen-hui
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 8h
> Remaining Estimate: 0h
>
> Add the Implementation for SqlLibraryOperators.CONCAT_FUNCTION in
> SqlFunctions rather than register it as a kind of || operator in
> StandardConvertletTable, because it would be better for Jdbc adaptor to
> generate CONCAT for mysql rather than || operator, since CONCAT in mysql is
> not totally same as || operator.
> According to my test, as for PG ,the result of "concat('a', 'b', 'c')" is
> equal to " 'a' || 'b' || 'c' " , which is 'abc', as for Mysql, the result of
> "concat('a', 'b', 'c')" is 'abc', but the result of " 'a'||'b'|| 'c' " is 0,
> as for oracle, concat only have two argument and the the result of
> 'a'||'b'||'c' is 'abc'.
> And the return type inference of SqlLibraryOperators.CONCAT_FUNCTION will
> cause AssertionError when the precsion of its operands are not specified.
> {code:java}
> concat(cast('a' as varchar), cast('b' as varchar),cast('c' as varchar)){code}
> {code:java}
> at
> org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:64)at
>
> org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:64)
> at
> org.apache.calcite.sql.fun.SqlLibraryOperators.lambda$static$1(SqlLibraryOperators.java:291)
> at
> org.apache.calcite.sql.type.SqlTypeTransformCascade.inferReturnType(SqlTypeTransformCascade.java:54)
> at org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:486)
> at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:453)
> at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:321) at
> org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:218) at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5858)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5845)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1800)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1785)
> at org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:260) at
> org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:423) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5552)
> at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:116) at
> org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:259) at
> org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:423) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5552)
> at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:116) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1059)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:766)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.parseAndValidate(AbstractSqlTester.java:175)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.getResultType(AbstractSqlTester.java:163)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.getColumnType(AbstractSqlTester.java:155)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.check(AbstractSqlTester.java:477)
> at
> org.apache.calcite.sql.test.SqlOperatorBaseTest$TesterImpl.check(SqlOperatorBaseTest.java:9489)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.check(AbstractSqlTester.java:462)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.checkString(AbstractSqlTester.java:447)
> at
> org.apache.calcite.sql.test.SqlOperatorBaseTest.testConcatOperator(SqlOperatorBaseTest.java:2184){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3864) Supports CONCAT for variable arguments
[
https://issues.apache.org/jira/browse/CALCITE-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu resolved CALCITE-3864.
---
Fix Version/s: 1.23.0
Resolution: Resolved
Fixed in
[https://github.com/apache/calcite/commit/888dd3a7d20ad04b22434f16d96177b7ca2c28aa],
thanks for your PR [~winipanda]!
> Supports CONCAT for variable arguments
> --
>
> Key: CALCITE-3864
> URL: https://issues.apache.org/jira/browse/CALCITE-3864
> Project: Calcite
> Issue Type: Bug
>Reporter: TANG Wen-hui
>Assignee: TANG Wen-hui
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 8h
> Remaining Estimate: 0h
>
> Add the Implementation for SqlLibraryOperators.CONCAT_FUNCTION in
> SqlFunctions rather than register it as a kind of || operator in
> StandardConvertletTable, because it would be better for Jdbc adaptor to
> generate CONCAT for mysql rather than || operator, since CONCAT in mysql is
> not totally same as || operator.
> According to my test, as for PG ,the result of "concat('a', 'b', 'c')" is
> equal to " 'a' || 'b' || 'c' " , which is 'abc', as for Mysql, the result of
> "concat('a', 'b', 'c')" is 'abc', but the result of " 'a'||'b'|| 'c' " is 0,
> as for oracle, concat only have two argument and the the result of
> 'a'||'b'||'c' is 'abc'.
> And the return type inference of SqlLibraryOperators.CONCAT_FUNCTION will
> cause AssertionError when the precsion of its operands are not specified.
> {code:java}
> concat(cast('a' as varchar), cast('b' as varchar),cast('c' as varchar)){code}
> {code:java}
> at
> org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:64)at
>
> org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:64)
> at
> org.apache.calcite.sql.fun.SqlLibraryOperators.lambda$static$1(SqlLibraryOperators.java:291)
> at
> org.apache.calcite.sql.type.SqlTypeTransformCascade.inferReturnType(SqlTypeTransformCascade.java:54)
> at org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:486)
> at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:453)
> at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:321) at
> org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:218) at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5858)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5845)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1800)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1785)
> at org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:260) at
> org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:423) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5552)
> at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:116) at
> org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:259) at
> org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:423) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5552)
> at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:116) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1059)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:766)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.parseAndValidate(AbstractSqlTester.java:175)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.getResultType(AbstractSqlTester.java:163)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.getColumnType(AbstractSqlTester.java:155)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.check(AbstractSqlTester.java:477)
> at
> org.apache.calcite.sql.test.SqlOperatorBaseTest$TesterImpl.check(SqlOperatorBaseTest.java:9489)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.check(AbstractSqlTester.java:462)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.checkString(AbstractSqlTester.java:447)
> at
> org.apache.calcite.sql.test.SqlOperatorBaseTest.testConcatOperator(SqlOperatorBaseTest.java:2184){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3878) Make ArrayList creation with initial capacity when size is fixed
[ https://issues.apache.org/jira/browse/CALCITE-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17068397#comment-17068397 ] Feng Zhu commented on CALCITE-3878: --- Certainly, changes made by this PR can help to improve the performance. But from personal feeling, I'm just *-0* on this direction. The optimization is a little nitpicking, and brings me some considerations on code style and project evolution. E.g., Should we document this pattern as a mandatory style? If not, how to cope with the later contributions? For example, after reviewing this PR, I noticed another PR (https://github.com/apache/calcite/pull/1870), which uses "new ArrayList<>()" but LGTM. > Make ArrayList creation with initial capacity when size is fixed > > > Key: CALCITE-3878 > URL: https://issues.apache.org/jira/browse/CALCITE-3878 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.22.0 >Reporter: neoremind >Priority: Minor > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > I find many places in Calcite where _new ArrayList<>()_ is used, if the list > is expected to be immutable or not resizing, it is always a good manner to > create with initial capacity, better for memory usage and performance. > I search all occurrences, focus on the core module, to make it safe, I only > update local variables with fixed size and not working in recursive method. > If the local variable reference goes out of scope, if resizing is needed, > things will work normally as well, so no side effect, but for the "escaping" > case, I am very conservative and do not change them. > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3876) Should not combine projects when top project contains window function referencing window function from bottom project
[
https://issues.apache.org/jira/browse/CALCITE-3876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067289#comment-17067289
]
Feng Zhu commented on CALCITE-3876:
---
It duplicates with CALCITE-3079, in which the proposed approach is not thorough.
According to the discussion, we need to prevent merging projects when such
window expressions are detected.
[~swtalbot], would you like to contribute?
> Should not combine projects when top project contains window function
> referencing window function from bottom project
> -
>
> Key: CALCITE-3876
> URL: https://issues.apache.org/jira/browse/CALCITE-3876
> Project: Calcite
> Issue Type: Bug
>Reporter: Steven Talbot
>Priority: Major
>
> Here's a test in RelToSqlConverterTest that should (roughly) pass
> {code:java}
> @Test public void testWindowOnWindowDoesNotCombineProjects() {
> final String query = "SELECT \n" +
> " ROW_NUMBER() OVER (ORDER BY rn) FROM\n" +
> " (SELECT *, \n" +
> " ROW_NUMBER() OVER (ORDER BY \"product_id\") as rn\n" +
> " FROM \"foodmart\".\"product\")";
> final String expected = "SELECT \n" +
> " ROW_NUMBER() OVER (ORDER BY rn) FROM\n" +
> " (SELECT *, \n" +
> " ROW_NUMBER() OVER (ORDER BY \"product_id\") as rn\n" +
> " FROM \"foodmart\".\"product\")";
> sql(query)
> .withPostgresql()
> .ok(expected);
> }{code}
> Instead, we see the result
> {noformat}
> SELECT ROW_NUMBER() OVER (ORDER BY ROW_NUMBER() OVER (ORDER BY
> \"product_id\"))\nFROM \"foodmart\".\"product\"{noformat}
> That's because multiple steps in the pipeline want to squish those two
> projects together, but that's not valid to do (at least in Redshift where I
> tested, but I assume in most other SQL dialects as well).
> So the expected behavior is that SqlToRelConverter, RelBuilder, and
> RelToSqlConverter are all knowledgeable enough about window functions to
> examine the arguments and avoid merging projects (or combining SelectNode[s],
> in the case of RelToSqlConverter), when one window references another window.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3872) Simplify expressions with unary minus
[
https://issues.apache.org/jira/browse/CALCITE-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17066472#comment-17066472
]
Feng Zhu commented on CALCITE-3872:
---
Just of curious, will this simplification bury some exception?
E.g., for a bigint column with min value '-9223372036854775808'
{code:java}
-(-(-9223372036854775808)){code}
In the second step, we will get a number (9223372036854775808) out of bigint's
range.
> Simplify expressions with unary minus
> -
>
> Key: CALCITE-3872
> URL: https://issues.apache.org/jira/browse/CALCITE-3872
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Liya Fan
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Support simplifying expression - ( - ( x ) ) as x.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3864) Add Implementation for SqlLibraryOperators.CONCAT_FUNCTION in SqlFunctions and correct the return type inference of SqlLibraryOperators.CONCAT_FUNCTION
[
https://issues.apache.org/jira/browse/CALCITE-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17066323#comment-17066323
]
Feng Zhu commented on CALCITE-3864:
---
When we converting a function _*f1*_ to another one *_f2_*, there is a common
sense that _*f2*_ is more expressive or powerful than *_f1_*.
For example, we can use *_case-when_* to replace _*IF*_ function.
This PR tries to use the binary "_*||*_" operator to replace *_concat_*
function in validation phase. However, *concat* function is more general than
"*||*".
I have some concerns for this approach.
(1) It may produce complicated nested expressions, bringing burden or even
stackoverflow error. E.g.,
{code:java}
concat('a0', 'a1', ..., 'a') -> (...(('a0'||'a1') || 'a2')...||'a')
{code}
(2) Nested "||" expression generates complex SQL in unparse phase.
[~winipanda], can we make some efforts to implement the CONCAT_FUNCTION in the
runtime, and *reuse* the implementation for "*||*"? Since "*||*" is just a
special case for CONCAT_FUNCTION.
> Add Implementation for SqlLibraryOperators.CONCAT_FUNCTION in SqlFunctions
> and correct the return type inference of SqlLibraryOperators.CONCAT_FUNCTION
> ---
>
> Key: CALCITE-3864
> URL: https://issues.apache.org/jira/browse/CALCITE-3864
> Project: Calcite
> Issue Type: Bug
>Reporter: TANG Wen-hui
>Assignee: TANG Wen-hui
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Add the Implementation for SqlLibraryOperators.CONCAT_FUNCTION in
> SqlFunctions rather than register it as a kind of || operator in
> StandardConvertletTable, because it would be better for Jdbc adaptor to
> generate CONCAT for mysql rather than || operator, since CONCAT in mysql is
> not totally same as || operator.
> According to my test, as for PG ,the result of "concat('a', 'b', 'c')" is
> equal to " 'a' || 'b' || 'c' " , which is 'abc', as for Mysql, the result of
> "concat('a', 'b', 'c')" is 'abc', but the result of " 'a'||'b'|| 'c' " is 0,
> as for oracle, concat only have two argument and the the result of
> 'a'||'b'||'c' is 'abc'.
> And the return type inference of SqlLibraryOperators.CONCAT_FUNCTION will
> cause AssertionError when the precsion of its operands are not specified.
> {code:java}
> concat(cast('a' as varchar), cast('b' as varchar),cast('c' as varchar)){code}
> {code:java}
> at
> org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:64)at
>
> org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:64)
> at
> org.apache.calcite.sql.fun.SqlLibraryOperators.lambda$static$1(SqlLibraryOperators.java:291)
> at
> org.apache.calcite.sql.type.SqlTypeTransformCascade.inferReturnType(SqlTypeTransformCascade.java:54)
> at org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:486)
> at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:453)
> at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:321) at
> org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:218) at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5858)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5845)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1800)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1785)
> at org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:260) at
> org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:423) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5552)
> at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:116) at
> org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:259) at
> org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:423) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5552)
> at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:116) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1059)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:766)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.parseAndValidate(AbstractSqlTester.java:175)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.getResultType(AbstractSqlTester.java:163)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.getColumnType(AbstractSqlTes
[jira] [Commented] (CALCITE-3864) Add Implementation for SqlLibraryOperators.CONCAT_FUNCTION in SqlFunctions and correct the return type inference of SqlLibraryOperators.CONCAT_FUNCTION
[
https://issues.apache.org/jira/browse/CALCITE-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17065554#comment-17065554
]
Feng Zhu commented on CALCITE-3864:
---
Hi, [~winipanda], thanks for your investigation. CONCAT_FUNCTION is introduced
in CALCITE-3235.
>From my understanding, you want to address the following three problems.
(1) *AssertionError* in return type inference for some cases, e.g.,
{code:java}
@Test public void testQuery() throws SQLException {
final String sql = "select concat(cast('a' as varchar), cast('b' as
varchar),cast('c' as varchar))";
CalciteAssert.that()
.with(CalciteConnectionProperty.FUN, "mysql")
.query(sql)
.returns("abc");
}
{code}
The full stacktrace is posted in your description.
(2) *IllegalStateException* because the function is not implemented in runtime.
{code:java}
@Test public void testQuery() throws SQLException {
final String sql = "select concat('a', 'b', 'c')";
CalciteAssert.that()
.with(CalciteConnectionProperty.FUN, "mysql")
.query(sql)
.returns("abc");
}
{code}
The stacktrace is:
{code:java}
Suppressed: java.lang.RuntimeException: cannot translate call CONCAT($t1, $t2,
$t3)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.translateCall(RexToLixTranslator.java:763)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.translate0(RexToLixTranslator.java:737)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.translate(RexToLixTranslator.java:208)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.translate0(RexToLixTranslator.java:691)
at
org.apache.calcite.adapter.enumerable.RexToLixTranslator.translate(RexToLixTranslator.java:208)
{code}
(3) *Unparse* logic for dialect like MySQL.
> Add Implementation for SqlLibraryOperators.CONCAT_FUNCTION in SqlFunctions
> and correct the return type inference of SqlLibraryOperators.CONCAT_FUNCTION
> ---
>
> Key: CALCITE-3864
> URL: https://issues.apache.org/jira/browse/CALCITE-3864
> Project: Calcite
> Issue Type: Bug
>Reporter: TANG Wen-hui
>Assignee: TANG Wen-hui
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Add the Implementation for SqlLibraryOperators.CONCAT_FUNCTION in
> SqlFunctions rather than register it as a kind of || operator in
> StandardConvertletTable, because it would be better for Jdbc adaptor to
> generate CONCAT for mysql rather than || operator, since CONCAT in mysql is
> not totally same as || operator.
> According to my test, as for PG ,the result of "concat('a', 'b', 'c')" is
> equal to " 'a' || 'b' || 'c' " , which is 'abc', as for Mysql, the result of
> "concat('a', 'b', 'c')" is 'abc', but the result of " 'a'||'b'|| 'c' " is 0,
> as for oracle, concat only have two argument and the the result of
> 'a'||'b'||'c' is 'abc'.
> And the return type inference of SqlLibraryOperators.CONCAT_FUNCTION will
> cause AssertionError when the precsion of its operands are not specified.
> {code:java}
> concat(cast('a' as varchar), cast('b' as varchar),cast('c' as varchar)){code}
> {code:java}
> at
> org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:64)at
>
> org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:64)
> at
> org.apache.calcite.sql.fun.SqlLibraryOperators.lambda$static$1(SqlLibraryOperators.java:291)
> at
> org.apache.calcite.sql.type.SqlTypeTransformCascade.inferReturnType(SqlTypeTransformCascade.java:54)
> at org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:486)
> at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:453)
> at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:321) at
> org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:218) at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5858)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5845)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1800)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1785)
> at org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:260) at
> org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:423) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5552)
> at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:116) at
> org.apache.calcite.sql.SqlNode.validateExp
[jira] [Commented] (CALCITE-3864) Add Implementation for SqlLibraryOperators.CONCAT_FUNCTION in SqlFunctions and correct the return type inference of SqlLibraryOperators.CONCAT_FUNCTION
[
https://issues.apache.org/jira/browse/CALCITE-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17063039#comment-17063039
]
Feng Zhu commented on CALCITE-3864:
---
I remember that the concat function differs in many products. Could you please
make a survey? Maybe we also need to take the unparse logic into consideration
when converting to SQL.
> Add Implementation for SqlLibraryOperators.CONCAT_FUNCTION in SqlFunctions
> and correct the return type inference of SqlLibraryOperators.CONCAT_FUNCTION
> ---
>
> Key: CALCITE-3864
> URL: https://issues.apache.org/jira/browse/CALCITE-3864
> Project: Calcite
> Issue Type: Bug
>Reporter: TANG Wen-hui
>Assignee: TANG Wen-hui
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Add the Implementation for SqlLibraryOperators.CONCAT_FUNCTION in
> SqlFunctions rather than register it as a kind of || operator in
> StandardConvertletTable, because it would be better for Jdbc adaptor to
> generate CONCAT for mysql rather than || operator, since CONCAT in mysql is
> not totally same as || operator.
> And the return type inference of SqlLibraryOperators.CONCAT_FUNCTION will
> cause AssertionError when the precsion of its operands are not specified.
> {code:java}
> concat(cast('a' as varchar), cast('b' as varchar),cast('c' as varchar)){code}
> {code:java}
> at
> org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:64)at
>
> org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:64)
> at
> org.apache.calcite.sql.fun.SqlLibraryOperators.lambda$static$1(SqlLibraryOperators.java:291)
> at
> org.apache.calcite.sql.type.SqlTypeTransformCascade.inferReturnType(SqlTypeTransformCascade.java:54)
> at org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:486)
> at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:453)
> at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:321) at
> org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:218) at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5858)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5845)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1800)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1785)
> at org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:260) at
> org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:423) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5552)
> at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:116) at
> org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:259) at
> org.apache.calcite.sql.SqlOperator.validateCall(SqlOperator.java:423) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5552)
> at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:116) at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1059)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:766)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.parseAndValidate(AbstractSqlTester.java:175)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.getResultType(AbstractSqlTester.java:163)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.getColumnType(AbstractSqlTester.java:155)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.check(AbstractSqlTester.java:477)
> at
> org.apache.calcite.sql.test.SqlOperatorBaseTest$TesterImpl.check(SqlOperatorBaseTest.java:9489)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.check(AbstractSqlTester.java:462)
> at
> org.apache.calcite.sql.test.AbstractSqlTester.checkString(AbstractSqlTester.java:447)
> at
> org.apache.calcite.sql.test.SqlOperatorBaseTest.testConcatOperator(SqlOperatorBaseTest.java:2184){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3815) Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
[
https://issues.apache.org/jira/browse/CALCITE-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu resolved CALCITE-3815.
---
Resolution: Resolved
Fixed via
[https://github.com/apache/calcite/commit/f5a2fe9e360f38910f112288581040e0ced5.]
Thanks for your PR, [~hanu.ncr]!
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
> ---
>
> Key: CALCITE-3815
> URL: https://issues.apache.org/jira/browse/CALCITE-3815
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Hanumath Rao Maduri
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION.
> Examples:
> {code}
> SELECT deptno,
> EVERY(sal < 3000) AS es, SOME(sal < 1000) AS ss,
> EVERY(comm < 500) AS ec, SOME(comm < 500) AS sc
> FROM emp
> GROUP BY deptno;
> ++---+---+---+--+
> | DEPTNO | ES | SS | EC | SC |
> ++---+---+---+--+
> | 10 | FALSE | FALSE | | |
> | 20 | FALSE | TRUE | | |
> | 30 | TRUE | TRUE | FALSE | TRUE |
> ++---+---+---+--+
> {code}
> {{EVERY}} and {{SOME}} can be implemented by translating to {{MIN}}, {{MAX}}:
> * {{EVERY(condition)}} is equivalent to {{MIN(condition)}};
> * {{SOME(condition)}} is equivalent to {{MAX(condition)}}
> where {{condition}} is a {{BOOLEAN}} expression, possibly allowing {{NULL}}
> values).
> {{INTERSECTION}} computes the intersection of collections (arrays and
> multisets). (Compare to {{FUSION}}, which computes the union of collections.)
> {{FUSION}} is in the operator table but there is no code-generation for it.
> This task should implement {{FUSION}} and {{INTERSECTION}} so that we can run
> queries in Enumerable mode.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3815) Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
[
https://issues.apache.org/jira/browse/CALCITE-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Zhu updated CALCITE-3815:
--
Fix Version/s: 1.23.0
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
> ---
>
> Key: CALCITE-3815
> URL: https://issues.apache.org/jira/browse/CALCITE-3815
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Hanumath Rao Maduri
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION.
> Examples:
> {code}
> SELECT deptno,
> EVERY(sal < 3000) AS es, SOME(sal < 1000) AS ss,
> EVERY(comm < 500) AS ec, SOME(comm < 500) AS sc
> FROM emp
> GROUP BY deptno;
> ++---+---+---+--+
> | DEPTNO | ES | SS | EC | SC |
> ++---+---+---+--+
> | 10 | FALSE | FALSE | | |
> | 20 | FALSE | TRUE | | |
> | 30 | TRUE | TRUE | FALSE | TRUE |
> ++---+---+---+--+
> {code}
> {{EVERY}} and {{SOME}} can be implemented by translating to {{MIN}}, {{MAX}}:
> * {{EVERY(condition)}} is equivalent to {{MIN(condition)}};
> * {{SOME(condition)}} is equivalent to {{MAX(condition)}}
> where {{condition}} is a {{BOOLEAN}} expression, possibly allowing {{NULL}}
> values).
> {{INTERSECTION}} computes the intersection of collections (arrays and
> multisets). (Compare to {{FUSION}}, which computes the union of collections.)
> {{FUSION}} is in the operator table but there is no code-generation for it.
> This task should implement {{FUSION}} and {{INTERSECTION}} so that we can run
> queries in Enumerable mode.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3724) Presto dialect implementation
[ https://issues.apache.org/jira/browse/CALCITE-3724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058342#comment-17058342 ] Feng Zhu commented on CALCITE-3724: --- Hi [~x1q1j1], could please add some material (e.g., presto's synatax doc, customized logics, special unparse calls and etc.) in the JIRA description. > Presto dialect implementation > - > > Key: CALCITE-3724 > URL: https://issues.apache.org/jira/browse/CALCITE-3724 > Project: Calcite > Issue Type: Improvement >Reporter: Forward Xu >Assignee: Forward Xu >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > Implement PrestoSqlDialect -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3815) Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
[
https://issues.apache.org/jira/browse/CALCITE-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057616#comment-17057616
]
Feng Zhu commented on CALCITE-3815:
---
{quote}_New classes and new SqlKind to be added for these functions. Please let
me know if somehow these can be avoided._
{quote}
[~hanu.ncr], When introducing a new SqlKind, we usually need to change many
places accordingly. This PR creates SqlKind.INTERSECTION. Please also add it
into the corresponding EnumSet in SqlKind.
Since SqlKind.MIN/MAX can map to many functions, can we add a new constructor
to allow the function name for _SqlMinMaxAggFunction_?
E.g.,
{code:java}
public SqlMinMaxAggFunction(String name, SqlKind kind, SqlOperandTypeChecker
inputTypeChecker)
public static final SqlAggFunction EVERY = new SqlMinMaxAggFunction("EVERY",
SqlKind.MIN, OperandTypes.BOOLEAN);
public static final SqlAggFunction SOME = new SqlMinMaxAggFunction("SOME",
SqlKind.MAX, OperandTypes.BOOLEAN);
{code}
{quote}_SqlKind for SOME function interferes with SOME quantifier. To avoid
this "SOME" can passed as argument instead of SqlKind.name() to the registered
class. However, fixing this way might introduce subtle bugs._
_There is a test case in SqlParserTest "select * from emp where name like some
(^select^ name from emp)"; which is passing with this approach._
_Ideally the above test case should pass, I will open another JIRA to track
this issue (test assumes it should fail). This bug is not related to any of the
approach. It just happens that in the ReservedFunctionNames' approach it
succeeds because the parser translates SOME in the query to the function name
instead of a quantifier. Even though this query works fine in the parser test,
it fails in the validation._
{quote}
I agree with you. If SOME can also be recognized as a function, this test case
should pass the parser phase.
I personally incline to make some efforts to put them in ReservedFunctionName.
[~julianhyde] WDYT?
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
> ---
>
> Key: CALCITE-3815
> URL: https://issues.apache.org/jira/browse/CALCITE-3815
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Hanumath Rao Maduri
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION.
> Examples:
> {code}
> SELECT deptno,
> EVERY(sal < 3000) AS es, SOME(sal < 1000) AS ss,
> EVERY(comm < 500) AS ec, SOME(comm < 500) AS sc
> FROM emp
> GROUP BY deptno;
> ++---+---+---+--+
> | DEPTNO | ES | SS | EC | SC |
> ++---+---+---+--+
> | 10 | FALSE | FALSE | | |
> | 20 | FALSE | TRUE | | |
> | 30 | TRUE | TRUE | FALSE | TRUE |
> ++---+---+---+--+
> {code}
> {{EVERY}} and {{SOME}} can be implemented by translating to {{MIN}}, {{MAX}}:
> * {{EVERY(condition)}} is equivalent to {{MIN(condition)}};
> * {{SOME(condition)}} is equivalent to {{MAX(condition)}}
> where {{condition}} is a {{BOOLEAN}} expression, possibly allowing {{NULL}}
> values).
> {{INTERSECTION}} computes the intersection of collections (arrays and
> multisets). (Compare to {{FUSION}}, which computes the union of collections.)
> {{FUSION}} is in the operator table but there is no code-generation for it.
> This task should implement {{FUSION}} and {{INTERSECTION}} so that we can run
> queries in Enumerable mode.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-3815) Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
[
https://issues.apache.org/jira/browse/CALCITE-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057517#comment-17057517
]
Feng Zhu edited comment on CALCITE-3815 at 3/12/20, 2:42 AM:
-
[~julianhyde], yes, you are right. I made something confused.
[~hanu.ncr], as Julian's suggestion, I think we can remove the doc "If no rows
are selected, the result is NULL". Sometimes, it is confusing on "empty
relation" and selective predicate for end-users.
was (Author: donnyzone):
[~julianhyde], yes, you are right. I made something confused.
[~hanu.ncr], as Julian's suggestion, I think we can remove the doc "If no rows
are selected, the result is NULL". It is confusing on "empty relation" and
selective predicate.
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
> ---
>
> Key: CALCITE-3815
> URL: https://issues.apache.org/jira/browse/CALCITE-3815
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Hanumath Rao Maduri
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION.
> Examples:
> {code}
> SELECT deptno,
> EVERY(sal < 3000) AS es, SOME(sal < 1000) AS ss,
> EVERY(comm < 500) AS ec, SOME(comm < 500) AS sc
> FROM emp
> GROUP BY deptno;
> ++---+---+---+--+
> | DEPTNO | ES | SS | EC | SC |
> ++---+---+---+--+
> | 10 | FALSE | FALSE | | |
> | 20 | FALSE | TRUE | | |
> | 30 | TRUE | TRUE | FALSE | TRUE |
> ++---+---+---+--+
> {code}
> {{EVERY}} and {{SOME}} can be implemented by translating to {{MIN}}, {{MAX}}:
> * {{EVERY(condition)}} is equivalent to {{MIN(condition)}};
> * {{SOME(condition)}} is equivalent to {{MAX(condition)}}
> where {{condition}} is a {{BOOLEAN}} expression, possibly allowing {{NULL}}
> values).
> {{INTERSECTION}} computes the intersection of collections (arrays and
> multisets). (Compare to {{FUSION}}, which computes the union of collections.)
> {{FUSION}} is in the operator table but there is no code-generation for it.
> This task should implement {{FUSION}} and {{INTERSECTION}} so that we can run
> queries in Enumerable mode.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-3815) Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
[
https://issues.apache.org/jira/browse/CALCITE-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057517#comment-17057517
]
Feng Zhu edited comment on CALCITE-3815 at 3/12/20, 2:42 AM:
-
[~julianhyde], yes, you are right. I made something confused.
[~hanu.ncr], as Julian's suggestion, I think we can remove the doc "If no rows
are selected, the result is NULL". It is confusing on "empty relation" and
selective predicate.
was (Author: donnyzone):
[~julianhyde], [~hanu.ncr], yes, you are right. I made something confused.
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
> ---
>
> Key: CALCITE-3815
> URL: https://issues.apache.org/jira/browse/CALCITE-3815
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Hanumath Rao Maduri
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION.
> Examples:
> {code}
> SELECT deptno,
> EVERY(sal < 3000) AS es, SOME(sal < 1000) AS ss,
> EVERY(comm < 500) AS ec, SOME(comm < 500) AS sc
> FROM emp
> GROUP BY deptno;
> ++---+---+---+--+
> | DEPTNO | ES | SS | EC | SC |
> ++---+---+---+--+
> | 10 | FALSE | FALSE | | |
> | 20 | FALSE | TRUE | | |
> | 30 | TRUE | TRUE | FALSE | TRUE |
> ++---+---+---+--+
> {code}
> {{EVERY}} and {{SOME}} can be implemented by translating to {{MIN}}, {{MAX}}:
> * {{EVERY(condition)}} is equivalent to {{MIN(condition)}};
> * {{SOME(condition)}} is equivalent to {{MAX(condition)}}
> where {{condition}} is a {{BOOLEAN}} expression, possibly allowing {{NULL}}
> values).
> {{INTERSECTION}} computes the intersection of collections (arrays and
> multisets). (Compare to {{FUSION}}, which computes the union of collections.)
> {{FUSION}} is in the operator table but there is no code-generation for it.
> This task should implement {{FUSION}} and {{INTERSECTION}} so that we can run
> queries in Enumerable mode.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3815) Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
[
https://issues.apache.org/jira/browse/CALCITE-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057517#comment-17057517
]
Feng Zhu commented on CALCITE-3815:
---
[~julianhyde], [~hanu.ncr], yes, you are right. I made something confused.
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
> ---
>
> Key: CALCITE-3815
> URL: https://issues.apache.org/jira/browse/CALCITE-3815
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Hanumath Rao Maduri
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION.
> Examples:
> {code}
> SELECT deptno,
> EVERY(sal < 3000) AS es, SOME(sal < 1000) AS ss,
> EVERY(comm < 500) AS ec, SOME(comm < 500) AS sc
> FROM emp
> GROUP BY deptno;
> ++---+---+---+--+
> | DEPTNO | ES | SS | EC | SC |
> ++---+---+---+--+
> | 10 | FALSE | FALSE | | |
> | 20 | FALSE | TRUE | | |
> | 30 | TRUE | TRUE | FALSE | TRUE |
> ++---+---+---+--+
> {code}
> {{EVERY}} and {{SOME}} can be implemented by translating to {{MIN}}, {{MAX}}:
> * {{EVERY(condition)}} is equivalent to {{MIN(condition)}};
> * {{SOME(condition)}} is equivalent to {{MAX(condition)}}
> where {{condition}} is a {{BOOLEAN}} expression, possibly allowing {{NULL}}
> values).
> {{INTERSECTION}} computes the intersection of collections (arrays and
> multisets). (Compare to {{FUSION}}, which computes the union of collections.)
> {{FUSION}} is in the operator table but there is no code-generation for it.
> This task should implement {{FUSION}} and {{INTERSECTION}} so that we can run
> queries in Enumerable mode.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3815) Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
[
https://issues.apache.org/jira/browse/CALCITE-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17056677#comment-17056677
]
Feng Zhu commented on CALCITE-3815:
---
It seems that we cannot completely reuse _min/max_'s implementation for
_every/some_, at least in the runtime layer.
In min/max, if no rows are selected, the default result is NULL. However, for
every/some, we should get "false".
The query below is validated in PG. Therefore, I'd like to suggest implementing
them seperately.
{code:java}
select every(a>100)
from (VALUES (1, 11), (2, 22), (3, 33)) AS t (a, b)
-
false
{code}
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
> ---
>
> Key: CALCITE-3815
> URL: https://issues.apache.org/jira/browse/CALCITE-3815
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Hanumath Rao Maduri
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION.
> Examples:
> {code}
> SELECT deptno,
> EVERY(sal < 3000) AS es, SOME(sal < 1000) AS ss,
> EVERY(comm < 500) AS ec, SOME(comm < 500) AS sc
> FROM emp
> GROUP BY deptno;
> ++---+---+---+--+
> | DEPTNO | ES | SS | EC | SC |
> ++---+---+---+--+
> | 10 | FALSE | FALSE | | |
> | 20 | FALSE | TRUE | | |
> | 30 | TRUE | TRUE | FALSE | TRUE |
> ++---+---+---+--+
> {code}
> {{EVERY}} and {{SOME}} can be implemented by translating to {{MIN}}, {{MAX}}:
> * {{EVERY(condition)}} is equivalent to {{MIN(condition)}};
> * {{SOME(condition)}} is equivalent to {{MAX(condition)}}
> where {{condition}} is a {{BOOLEAN}} expression, possibly allowing {{NULL}}
> values).
> {{INTERSECTION}} computes the intersection of collections (arrays and
> multisets). (Compare to {{FUSION}}, which computes the union of collections.)
> {{FUSION}} is in the operator table but there is no code-generation for it.
> This task should implement {{FUSION}} and {{INTERSECTION}} so that we can run
> queries in Enumerable mode.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3840) Re-aliasing of VALUES that has column aliases produces wrong SQL in the JDBC adapter
[
https://issues.apache.org/jira/browse/CALCITE-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17053057#comment-17053057
]
Feng Zhu commented on CALCITE-3840:
---
Hey, [~christian.beikov], could you provide a test case?
> Re-aliasing of VALUES that has column aliases produces wrong SQL in the JDBC
> adapter
>
>
> Key: CALCITE-3840
> URL: https://issues.apache.org/jira/browse/CALCITE-3840
> Project: Calcite
> Issue Type: Bug
> Components: jdbc-adapter
>Affects Versions: 1.21.0
>Reporter: Christian Beikov
>Priority: Major
>
> Rendering a VALUES relnode to e.g. PostgreSQL will produce \{{FROM
> (VALUES((1))) AS t(col_alias)}} where "t" is a static alias. When e.g.
> joining with such a VALUES, the RelToSqlConverter tries to re-alias this with
> a unique alias, but fails because it produces \{{FROM (VALUES((1))) AS
> t(col_alias) AS newAlias}}.
> The fix is to replace the static table alias instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3815) Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
[
https://issues.apache.org/jira/browse/CALCITE-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17048717#comment-17048717
]
Feng Zhu commented on CALCITE-3815:
---
[~hanu.ncr], sure! Thanks.
But I found I failed to assigned it to you. You can ask for contributor
permission through mailinglist, then you can assign yourself.
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
> ---
>
> Key: CALCITE-3815
> URL: https://issues.apache.org/jira/browse/CALCITE-3815
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
>
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION.
> Examples:
> {code}
> SELECT deptno,
> EVERY(sal < 3000) AS es, SOME(sal < 1000) AS ss,
> EVERY(comm < 500) AS ec, SOME(comm < 500) AS sc
> FROM emp
> GROUP BY deptno;
> ++---+---+---+--+
> | DEPTNO | ES | SS | EC | SC |
> ++---+---+---+--+
> | 10 | FALSE | FALSE | | |
> | 20 | FALSE | TRUE | | |
> | 30 | TRUE | TRUE | FALSE | TRUE |
> ++---+---+---+--+
> {code}
> {{EVERY}} and {{SOME}} can be implemented by translating to {{MIN}}, {{MAX}}:
> * {{EVERY(condition)}} is equivalent to {{MIN(condition)}};
> * {{SOME(condition)}} is equivalent to {{MAX(condition)}}
> where {{condition}} is a {{BOOLEAN}} expression, possibly allowing {{NULL}}
> values).
> {{INTERSECTION}} computes the intersection of collections (arrays and
> multisets). (Compare to {{FUSION}}, which computes the union of collections.)
> {{FUSION}} is in the operator table but there is no code-generation for it.
> This task should implement {{FUSION}} and {{INTERSECTION}} so that we can run
> queries in Enumerable mode.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (CALCITE-3832) NPE when decimal cast double
[ https://issues.apache.org/jira/browse/CALCITE-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Zhu closed CALCITE-3832. - Resolution: Duplicate The root cause: failing to inline unsafe code. It duplicates with CALCITE-3142. > NPE when decimal cast double > - > > Key: CALCITE-3832 > URL: https://issues.apache.org/jira/browse/CALCITE-3832 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.16.0 >Reporter: Zhixiong Chen >Priority: Major > > sql like: > select > cast(Column1 as DOUBLE) + cast(Column2 as DOUBLE) AS "COLUMN5", > cast(Column3 as DOUBLE) + cast(Column4 as DOUBLE) AS "COLUMN6" > from TABLE > The Column1、Column2、Column3、Column4 type is decimal(19,4) > But it throw npe in decimal cast to double > then I find in calcite plan it doesn't deal with null > /* 24 */ final java.math.BigDecimal inp14_ = current[14] == null ? > (java.math.BigDecimal) null : > org.apache.calcite.runtime.SqlFunctions.toBigDecimal(current[14]); > /* 25 */ final double v = inp14_.doubleValue(); > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3832) NPE when decimal cast double
[ https://issues.apache.org/jira/browse/CALCITE-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046594#comment-17046594 ] Feng Zhu commented on CALCITE-3832: --- Yeah, it's a known problem. That PR can cover this ticket. > NPE when decimal cast double > - > > Key: CALCITE-3832 > URL: https://issues.apache.org/jira/browse/CALCITE-3832 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.16.0 >Reporter: Zhixiong Chen >Priority: Major > > sql like: > select > cast(Column1 as DOUBLE) + cast(Column2 as DOUBLE) AS "COLUMN5", > cast(Column3 as DOUBLE) + cast(Column4 as DOUBLE) AS "COLUMN6" > from TABLE > The Column1、Column2、Column3、Column4 type is decimal(19,4) > But it throw npe in decimal cast to double > then I find in calcite plan it doesn't deal with null > /* 24 */ final java.math.BigDecimal inp14_ = current[14] == null ? > (java.math.BigDecimal) null : > org.apache.calcite.runtime.SqlFunctions.toBigDecimal(current[14]); > /* 25 */ final double v = inp14_.doubleValue(); > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-3815) Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
[
https://issues.apache.org/jira/browse/CALCITE-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043166#comment-17043166
]
Feng Zhu edited comment on CALCITE-3815 at 2/24/20 6:01 AM:
It seems like Intersect/Fusion (including codegen) has already been implemented
in CALCITE-2355.
was (Author: donnyzone):
It seems like Except/Fusion (including codegen) has already been implemented in
CALCITE-2355.
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
> ---
>
> Key: CALCITE-3815
> URL: https://issues.apache.org/jira/browse/CALCITE-3815
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
>
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION.
> Examples:
> {code}
> SELECT deptno,
> EVERY(sal < 3000) AS es, SOME(sal < 1000) AS ss,
> EVERY(comm < 500) AS ec, SOME(comm < 500) AS sc
> FROM emp
> GROUP BY deptno;
> ++---+---+---+--+
> | DEPTNO | ES | SS | EC | SC |
> ++---+---+---+--+
> | 10 | FALSE | FALSE | | |
> | 20 | FALSE | TRUE | | |
> | 30 | TRUE | TRUE | FALSE | TRUE |
> ++---+---+---+--+
> {code}
> {{EVERY}} and {{SOME}} can be implemented by translating to {{MIN}}, {{MAX}}:
> * {{EVERY(condition)}} is equivalent to {{MIN(condition)}};
> * {{SOME(condition)}} is equivalent to {{MAX(condition)}}
> where {{condition}} is a {{BOOLEAN}} expression, possibly allowing {{NULL}}
> values).
> {{INTERSECTION}} computes the intersection of collections (arrays and
> multisets). (Compare to {{FUSION}}, which computes the union of collections.)
> {{FUSION}} is in the operator table but there is no code-generation for it.
> This task should implement {{FUSION}} and {{INTERSECTION}} so that we can run
> queries in Enumerable mode.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3815) Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
[
https://issues.apache.org/jira/browse/CALCITE-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043166#comment-17043166
]
Feng Zhu commented on CALCITE-3815:
---
It seems like Except/Fusion (including codegen) has already been implemented in
CALCITE-2355.
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION
> ---
>
> Key: CALCITE-3815
> URL: https://issues.apache.org/jira/browse/CALCITE-3815
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
>
> Add missing SQL standard aggregate functions: EVERY, SOME, INTERSECTION.
> Examples:
> {code}
> SELECT deptno,
> EVERY(sal < 3000) AS es, SOME(sal < 1000) AS ss,
> EVERY(comm < 500) AS ec, SOME(comm < 500) AS sc
> FROM emp
> GROUP BY deptno;
> ++---+---+---+--+
> | DEPTNO | ES | SS | EC | SC |
> ++---+---+---+--+
> | 10 | FALSE | FALSE | | |
> | 20 | FALSE | TRUE | | |
> | 30 | TRUE | TRUE | FALSE | TRUE |
> ++---+---+---+--+
> {code}
> {{EVERY}} and {{SOME}} can be implemented by translating to {{MIN}}, {{MAX}}:
> * {{EVERY(condition)}} is equivalent to {{MIN(condition)}};
> * {{SOME(condition)}} is equivalent to {{MAX(condition)}}
> where {{condition}} is a {{BOOLEAN}} expression, possibly allowing {{NULL}}
> values).
> {{INTERSECTION}} computes the intersection of collections (arrays and
> multisets). (Compare to {{FUSION}}, which computes the union of collections.)
> {{FUSION}} is in the operator table but there is no code-generation for it.
> This task should implement {{FUSION}} and {{INTERSECTION}} so that we can run
> queries in Enumerable mode.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3224) New RexNode-to-Expression CodeGen Implementation
[
https://issues.apache.org/jira/browse/CALCITE-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17040658#comment-17040658
]
Feng Zhu commented on CALCITE-3224:
---
The codegen component is not intuitive, and it seems that committers are busy
recently.
Therefore, it may be difficult to be resolved in release 1.22. I think we can
change the version to 1.23 first.
> New RexNode-to-Expression CodeGen Implementation
>
>
> Key: CALCITE-3224
> URL: https://issues.apache.org/jira/browse/CALCITE-3224
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Attachments: RexNode-CodeGen.pdf, codegen.png
>
> Time Spent: 6.5h
> Remaining Estimate: 0h
>
> h3. *Background*
> Current RexNode-to-Expression implementation relies on BlockBuilder's
> incorrect “optimizations” to inline unsafe operations. As illustrated in
> CALCITE-3173, when this cooperation is broken in some special cases, it will
> cause exceptions like NPE, such as CALCITE-3142, CALCITE-3143, CALCITE-3150.
> Though we can fix these problems under current implementation framework
> with some efforts like the PR in CALCITE-3142, the logic will become more and
> more complex. To pursue a thorough and elegant solution, we implement a new
> one. Moreover, it also ensures the correctness for non-optimized code.
> h3. *Major Features*
> * *Visitor Pattern*: Each RexNode will be visited only once in a bottom-up
> way, rather than recursively visiting a RexNode many times with different
> NullAs settings.
> * *Conditional Semantic*: It can naturally guarantee the correctness even
> without BlockBuilder’s “optimizings”. Each line of code generated for a
> RexNode is null safe.
> * *Interface Compatibility*: The implementation only updates
> _RexToLixTranslator_ and _RexImpTable_. Interfaces such as CallImplementor
> keep unchanged.
> h3. *Implementation*
> For each RexNode, the visitor will generally generate two declaration
> statements, one for value and one for nullable. The code snippet is like:
> {code:java}
> {valueVariable} = {valueExpression}
> {isNullVariable} = {isNullExpression}
> {code}
> The visitor’s result will be the variable pair (*_isNullVariable_*,
> *_valueVariable_*).
> *Other changes:*
> (1) ReImplement different RexCall implementations (e.g., CastImplementor,
> BinaryImplementor and etc.) as seperated files and remove them into the newly
> created package _org.apache.calcite.adapter.enumerable.rex,_ and organize
> them in RexCallImpTable.
> (2) move some util functions into EnumUtils.
> h3. *Example Demonstration*
> Take a simple test case as example, in which the "commission" column is
> nullable.
> {code:java}
> @Test public void testNPE() {
> CalciteAssert.hr()
> .query("select \"commission\" + 10 as s\n"
> + "from \"hr\".\"emps\"")
> .returns("S=1010\nS=510\nS=null\nS=260\n");
> }
> {code}
> The codegen progress and non-optimized code are demonstrated in the figure
> below.
> !codegen.png!
> # When visiting *RexInputRef (commission)*, the visitor generates three
> lines of code, the result is a pair of ParameterExpression (*_input_isNull_*,
> *_input_value_*).
> # Then the visitor visits *RexLiteral (10)* and generates two lines of code.
> The result is (*_literal_isNull_*, *_literal_value_*).
> # After that, when visiting *RexCall(Add)*, (_*input_isNull*_,
> _*input_value*_) and (_*literal_isNull*_, _*literal_value*_) can be used to
> implement the logic. The visitor also generates two lines of code and return
> the variable pair.
> In the end, the result Expression is constructed based on
> (_*binary_call_isNull*_, _*binary_call_value*_)
> [^RexNode-CodeGen.pdf]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3772) Query returning bad results
[
https://issues.apache.org/jira/browse/CALCITE-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17040230#comment-17040230
]
Feng Zhu commented on CALCITE-3772:
---
I think there are two general ways to fix this issue:
(1) Prevent to trim fields when we find correlated variable in the plan.
(2) Enable _*RelFieldTrimmer*_ to handle correlated variables when trimming
fields. But it may bring more or less duplicate logic and data structurs (e.g.,
maintaining maps) in _*RelDecorrelator*_.
Hi, [~julianhyde], [~vlsi] , sorry to bother you. Since I don't know Calcite's
initial design for correlated-subquery, could you give me some advices?
> Query returning bad results
> ---
>
> Key: CALCITE-3772
> URL: https://issues.apache.org/jira/browse/CALCITE-3772
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0, next
> Environment: [^T1.csv]
>Reporter: Jacob Roldan
>Assignee: Feng Zhu
>Priority: Critical
> Attachments: T1.csv
>
>
> Inspired in tests of sqllite, I have a query that it is returning bad values.
> I've tested using CsvTest in 1.21.0 and the last master
>
> {code:sql}
> SELECT a, (SELECT count(*) FROM t1 AS x WHERE x.b FROM t1
> WHERE (e>100)
> order by a
> {code}
>
> and the result is:
> {code:java}
> 104, 1
> 107, 2
> 111, 2
> 115, 3
> 121, 4 {code}
> Testing the same query in mysql, derby and postgres, the result is:
> {code:java}
> 104 0
> 107 1
> 111 2
> 115 3
> 121 4{code}
> I've attached the csv file I've put in
> ??calcite/example/csv/src/test/resources/bug/T1.csv??
> and the query test in CsvTest.java
> {code:java}
> @Test public void testQuery() throws SQLException {
> sql("bug", "SELECT a, (SELECT count(*) FROM t1 AS x WHERE x.b where e>100 order by a").ok();
> }
> {code}
> The explain plan:
> {code:java}
> EnumerableCalc(expr#0..3=[{inputs}], expr#4=[IS NULL($t3)],
> expr#5=[0:BIGINT], expr#6=[CASE($t4, $t5, $t3)], A=[$t0], EXPR$1=[$t6])
> EnumerableHashJoin(condition=[=($1, $2)], joinType=[left])
> EnumerableCalc(expr#0..4=[{inputs}], A=[$t0], E=[$t4])
> EnumerableSort(sort0=[$0], dir0=[ASC])
> EnumerableCalc(expr#0..4=[{inputs}], expr#5=[100], expr#6=[>($t4,
> $t5)], proj#0..4=[{exprs}], $condition=[$t6])
> EnumerableInterpreter
> BindableTableScan(table=[[BUG, T1]])
> EnumerableAggregate(group=[{1}], EXPR$0=[COUNT()])
> EnumerableNestedLoopJoin(condition=[<($0, $1)], joinType=[inner])
> EnumerableCalc(expr#0..4=[{inputs}], B=[$t1])
> EnumerableInterpreter
> BindableTableScan(table=[[BUG, T1]])
> EnumerableAggregate(group=[{4}])
> EnumerableCalc(expr#0..4=[{inputs}], expr#5=[100], expr#6=[>($t4,
> $t5)], proj#0..4=[{exprs}], $condition=[$t6])
> EnumerableInterpreter
> BindableTableScan(table=[[BUG, T1]])
> {code}
> The T1.csv is very simple:
> {code:java}
> A:int,B:int,C:int,D:int,E:int
> 104,100,102,101,103
> 107,105,106,108,109
> 111,112,113,114,110
> 115,118,119,116,117
> 121,124,123,122,120
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3772) Query returning bad results
[
https://issues.apache.org/jira/browse/CALCITE-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17040212#comment-17040212
]
Feng Zhu commented on CALCITE-3772:
---
It's not safe to trim fields [1] when the Plan contains correlated variables.
*Before trim:*
{code:java}
LogicalProject(A=[$0], EXPR$1=[$SCALAR_QUERY({
LogicalAggregate(group=[{}], EXPR$0=[COUNT()])
LogicalProject($f0=[0])
LogicalFilter(condition=[<($1, $cor0.B)])
LogicalTableScan(table=[[BUG, T1]])
})])
LogicalFilter(condition=[>($4, 100)])
LogicalTableScan(table=[[BUG, T1]])
{code}
*After trim:*
{code:java}
LogicalProject(A=[$0], EXPR$1=[$SCALAR_QUERY({
LogicalAggregate(group=[{}], EXPR$0=[COUNT()])
LogicalProject($f0=[0])
LogicalFilter(condition=[<($1, $cor0.B)])
LogicalTableScan(table=[[BUG, T1]])
})])
LogicalFilter(condition=[>($1, 100)])
LogicalProject(A=[$0], E=[$4])
LogicalTableScan(table=[[BUG, T1]])
{code}
Consequently, when decoralating the plan, it produces the incorrect RelNode for
_*{color:#FF}$cor0.B{color}*_:
{code:java}
LogicalAggregate(group=[{0}])
LogicalProject(E=[$4])
LogicalFilter(condition=[>($4, 100)])
LogicalTableScan(table=[[BUG, T1]])
{code}
[1]https://github.com/apache/calcite/blob/2ffd74abb665f4119ff30926f3944070d8a9d0ac/core/src/main/java/org/apache/calcite/prepare/Prepare.java#L281
> Query returning bad results
> ---
>
> Key: CALCITE-3772
> URL: https://issues.apache.org/jira/browse/CALCITE-3772
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0, next
> Environment: [^T1.csv]
>Reporter: Jacob Roldan
>Assignee: Feng Zhu
>Priority: Critical
> Attachments: T1.csv
>
>
> Inspired in tests of sqllite, I have a query that it is returning bad values.
> I've tested using CsvTest in 1.21.0 and the last master
>
> {code:sql}
> SELECT a, (SELECT count(*) FROM t1 AS x WHERE x.b FROM t1
> WHERE (e>100)
> order by a
> {code}
>
> and the result is:
> {code:java}
> 104, 1
> 107, 2
> 111, 2
> 115, 3
> 121, 4 {code}
> Testing the same query in mysql, derby and postgres, the result is:
> {code:java}
> 104 0
> 107 1
> 111 2
> 115 3
> 121 4{code}
> I've attached the csv file I've put in
> ??calcite/example/csv/src/test/resources/bug/T1.csv??
> and the query test in CsvTest.java
> {code:java}
> @Test public void testQuery() throws SQLException {
> sql("bug", "SELECT a, (SELECT count(*) FROM t1 AS x WHERE x.b where e>100 order by a").ok();
> }
> {code}
> The explain plan:
> {code:java}
> EnumerableCalc(expr#0..3=[{inputs}], expr#4=[IS NULL($t3)],
> expr#5=[0:BIGINT], expr#6=[CASE($t4, $t5, $t3)], A=[$t0], EXPR$1=[$t6])
> EnumerableHashJoin(condition=[=($1, $2)], joinType=[left])
> EnumerableCalc(expr#0..4=[{inputs}], A=[$t0], E=[$t4])
> EnumerableSort(sort0=[$0], dir0=[ASC])
> EnumerableCalc(expr#0..4=[{inputs}], expr#5=[100], expr#6=[>($t4,
> $t5)], proj#0..4=[{exprs}], $condition=[$t6])
> EnumerableInterpreter
> BindableTableScan(table=[[BUG, T1]])
> EnumerableAggregate(group=[{1}], EXPR$0=[COUNT()])
> EnumerableNestedLoopJoin(condition=[<($0, $1)], joinType=[inner])
> EnumerableCalc(expr#0..4=[{inputs}], B=[$t1])
> EnumerableInterpreter
> BindableTableScan(table=[[BUG, T1]])
> EnumerableAggregate(group=[{4}])
> EnumerableCalc(expr#0..4=[{inputs}], expr#5=[100], expr#6=[>($t4,
> $t5)], proj#0..4=[{exprs}], $condition=[$t6])
> EnumerableInterpreter
> BindableTableScan(table=[[BUG, T1]])
> {code}
> The T1.csv is very simple:
> {code:java}
> A:int,B:int,C:int,D:int,E:int
> 104,100,102,101,103
> 107,105,106,108,109
> 111,112,113,114,110
> 115,118,119,116,117
> 121,124,123,122,120
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
