[jira] [Created] (CALCITE-4584) Using function in partition by list of over window cause converting exception
Wang Yanlin created CALCITE-4584: Summary: Using function in partition by list of over window cause converting exception Key: CALCITE-4584 URL: https://issues.apache.org/jira/browse/CALCITE-4584 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin When try to using function expr in the partition by list of over window in sql, got {noformat} java.lang.UnsupportedOperationException: class org.apache.calcite.sql.SqlBasicCall: CHAR_LENGTH(`ENAME`) {noformat} The stack trace is as follow {noformat} java.lang.RuntimeException: while converting CHAR_LENGTH(`EMP`.`ENAME`) at org.apache.calcite.sql2rel.ReflectiveConvertletTable.lambda$registerOpTypeMethod$3(ReflectiveConvertletTable.java:145) at org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:62) at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5232) at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4486) at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:161) at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5088) at org.apache.calcite.sql2rel.SqlToRelConverter.convertOver(SqlToRelConverter.java:2037) at org.apache.calcite.sql2rel.SqlToRelConverter.access$2000(SqlToRelConverter.java:224) at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5081) at org.apache.calcite.sql2rel.StandardConvertletTable.lambda$new$9(StandardConvertletTable.java:209) at org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertCall(SqlNodeToRexConverterImpl.java:62) at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:5232) at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4486) at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:161) at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:5088) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectList(SqlToRelConverter.java:4327) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:707) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:664) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3538) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:589) at org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:638) at org.apache.calcite.test.SqlToRelTestBase$TesterImpl.assertConvertsTo(SqlToRelTestBase.java:799) at org.apache.calcite.test.SqlToRelConverterTest.testFunctionExprInOver(SqlToRelConverterTest.java:4196) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675) at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:125) at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:139) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:131) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:81) at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115) at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:104) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:62) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:43) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:35) at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104) at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke
[jira] [Created] (CALCITE-4376) Materialized view recognition fails when target output different column sequence with GROUP BY
Wang Yanlin created CALCITE-4376: Summary: Materialized view recognition fails when target output different column sequence with GROUP BY Key: CALCITE-4376 URL: https://issues.apache.org/jira/browse/CALCITE-4376 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4374) Support materialized view recognition when query distinct aggregate on target GROUP BY column
Wang Yanlin created CALCITE-4374: Summary: Support materialized view recognition when query distinct aggregate on target GROUP BY column Key: CALCITE-4374 URL: https://issues.apache.org/jira/browse/CALCITE-4374 Project: Calcite Issue Type: Improvement Reporter: Wang Yanlin -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4273) Support get expression lineage for Calc
Wang Yanlin created CALCITE-4273: Summary: Support get expression lineage for Calc Key: CALCITE-4273 URL: https://issues.apache.org/jira/browse/CALCITE-4273 Project: Calcite Issue Type: Improvement Reporter: Wang Yanlin -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4182) MV recognition fails when query has constant filter for group by list in mv, but without group by
Wang Yanlin created CALCITE-4182: Summary: MV recognition fails when query has constant filter for group by list in mv, but without group by Key: CALCITE-4182 URL: https://issues.apache.org/jira/browse/CALCITE-4182 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4177) Throw exception when deserialize SqlOperator fails, do not return null
Wang Yanlin created CALCITE-4177: Summary: Throw exception when deserialize SqlOperator fails, do not return null Key: CALCITE-4177 URL: https://issues.apache.org/jira/browse/CALCITE-4177 Project: Calcite Issue Type: Improvement Reporter: Wang Yanlin Currently, when deserialize a RexNode fails, calcite returns a null value, causing NPE in somewhere else. So we cannot get any information about the SqlOperator in the stacktrace. This makes hard to determine the reason when deserialize a very long json string. {noformat} Caused by: java.lang.NullPointerException at java.util.Objects.requireNonNull(Objects.java:203) at org.apache.calcite.rel.core.AggregateCall.(AggregateCall.java:98) at org.apache.calcite.rel.core.AggregateCall.create(AggregateCall.java:198) at org.apache.calcite.rel.externalize.RelJsonReader.toAggCall(RelJsonReader.java:289) at org.apache.calcite.rel.externalize.RelJsonReader.access$500(RelJsonReader.java:59) at org.apache.calcite.rel.externalize.RelJsonReader$2.getAggregateCalls(RelJsonReader.java:172) at org.apache.calcite.rel.core.Aggregate.(Aggregate.java:220) at org.apache.calcite.rel.logical.LogicalAggregate.(LogicalAggregate.java:105) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.calcite.rel.externalize.RelJsonReader.readRel(RelJsonReader.java:264) at org.apache.calcite.rel.externalize.RelJsonReader.readRels(RelJsonReader.java:91) at org.apache.calcite.rel.externalize.RelJsonReader.read(RelJsonReader.java:85) at org.apache.calcite.plan.RelWriterTest.lambda$deserializeAndDumpToTextFormat$6(RelWriterTest.java:894) at org.apache.calcite.tools.Frameworks.lambda$withPlanner$0(Frameworks.java:131) at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:914) at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:180) ... 67 more {noformat} Maybe it's better to throw exception instead of return null. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4019) Visit SqlInsert with SqlShuttle cause NullPointerException
Wang Yanlin created CALCITE-4019: Summary: Visit SqlInsert with SqlShuttle cause NullPointerException Key: CALCITE-4019 URL: https://issues.apache.org/jira/browse/CALCITE-4019 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin When visit a SqlInsert with SqlShuttle, got NPE, the stack trace is {noformat} java.lang.NullPointerException at com.google.common.collect.Iterators$8.transform(Iterators.java:817) at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48) at org.apache.calcite.sql.parser.SqlParserPos.sum(SqlParserPos.java:274) at org.apache.calcite.sql.parser.SqlParserPos.plusAll(SqlParserPos.java:183) at org.apache.calcite.sql.parser.SqlParserTest.testSqlParserPosPlus(SqlParserTest.java:8793) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675) at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:125) at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:139) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:131) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:81) at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115) at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:104) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:62) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:43) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:35) at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104) at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:202) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:198) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80) at java.util.ArrayList.forEach(ArrayList.java:1257) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125) at org.junit.platform.engine.support.hierarchical.Node.around
Re:Re: [ANNOUNCE] New committer: Wang Yanlin
Thanks Julian, having an entry on the powered by page for Ant Financial would be nice. [1] is the official homepage of the company I'm serving. Yanlin [1] https://www.antfin.com/ <https://www.antfin.com/> -- Best, Wang Yanlin At 2020-05-01 01:54:38, "Julian Hyde" wrote: >Welcome, Yanlin! > >Thanks for your contributions so far, and thanks for introducing yourself. I >often learn so much from committers’ self-introductions about how Calcite is >being used. > >I know we have other Alibaba-related projects on the “powered by” page [1] >(Flink/Ververica, MaxCompute) but it seems that Ant Financial is a distinct >business, so deserves its own entry on the page, and logo. What do you think? > >Julian > >[1] https://calcite.apache.org/docs/powered_by.html#alibaba-maxcompute ><https://calcite.apache.org/docs/powered_by.html#alibaba-maxcompute> > >> On Apr 29, 2020, at 5:50 AM, Wang Yanlin <1989yanlinw...@163.com> wrote: >> >> Hi, guys, thanks for your warm welcome. >> >> >> >> I'm working in Ant Finical, Alibaba Group. Currently my team is working on >> building a system to process big data in form of sql. >> We use calcite to parse sql, optimize Relnode and rewrite SqlNode to execute >> on different engines, like Spark,MaxCompute, HBase and so on. >> Calcite is really a great community, and it's really an honor for me to >> become calcite committer, hops to make more contribution to calcite. >> >> >> Thanks again. >> >> -- >> >> Best, >> Wang Yanlin >> >> >> >> >> >> 在 2020-04-29 13:58:35,"Zoltan Haindrich" 写道: >>> Congratulations! >>> >>> On 4/29/20 7:32 AM, Enrico Olivelli wrote: >>>> Congrats! >>>> >>>> Enrico >>>> >>>> Il Mer 29 Apr 2020, 04:51 Feng Zhu ha scritto: >>>> >>>>> Congrations! Yanlin! >>>>> >>>>> best, >>>>> Feng >>>>> >>>>> Chunwei Lei 于2020年4月29日周三 上午10:16写道: >>>>> >>>>>> Congrats, Yanlin! >>>>>> >>>>>> >>>>>> Best, >>>>>> Chunwei >>>>>> >>>>>> >>>>>> On Wed, Apr 29, 2020 at 10:07 AM Forward Xu >>>>>> wrote: >>>>>> >>>>>>> Congrats >>>>>>> >>>>>>> >>>>>>> Best, >>>>>>> >>>>>>> Forward >>>>>>> >>>>>>> 953396112 <953396...@qq.com> 于2020年4月29日周三 上午8:26写道: >>>>>>> >>>>>>>> Congrats, Wang Yanlin! >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ---Original--- >>>>>>>> From: "Stamatis Zampetakis">>>>>>> Date: Wed, Apr 29, 2020 05:51 AM >>>>>>>> To: "dev">>>>>>> Subject: [ANNOUNCE] New committer: Wang Yanlin >>>>>>>> >>>>>>>> >>>>>>>> Apache Calcite's Project Management Committee (PMC) has invited Wang >>>>>>> Yanlin >>>>>>>> to >>>>>>>> become a committer, and we are pleased to announce that he has >>>>>> accepted. >>>>>>>> >>>>>>>> Wang has pushed numerous fixes and improvements to the project, >>>>> landing >>>>>>> in >>>>>>>> total >>>>>>>> the impressive number of 30 commits to the master. Among other >>>>> things, >>>>>> he >>>>>>>> contributed some important features in the Interpreter. >>>>>>>> >>>>>>>> Wang, welcome, thank you for your contributions, and we look forward >>>>>> your >>>>>>>> further interactions with the community! If you wish, please feel >>>>> free >>>>>> to >>>>>>>> tell >>>>>>>> us more about yourself and what you are working on. >>>>>>>> >>>>>>>> Stamatis (on behalf of the Apache Calcite PMC) >>>>>>> >>>>>> >>>>> >>>> >
Re:Re: [ANNOUNCE] New committer: Jin Xing
Congrats, Jin Xing! -- Best, Wang Yanlin 在 2020-04-29 13:58:53,"Zoltan Haindrich" 写道: >Congrats! > >On 4/29/20 7:32 AM, Enrico Olivelli wrote: >> Congratulations! >> >> Enrico >> >> Il Mer 29 Apr 2020, 04:51 Feng Zhu ha scritto: >> >>> Congrations! >>> >>> best, >>> Feng >>> >>> Chunwei Lei 于2020年4月29日周三 上午10:16写道: >>> >>>> Congrats, Jin! >>>> >>>> >>>> Best, >>>> Chunwei >>>> >>>> >>>> On Wed, Apr 29, 2020 at 10:07 AM Forward Xu >>>> wrote: >>>> >>>>> Congrats >>>>> >>>>> >>>>> best, >>>>> >>>>> Forward >>>>> >>>>> 953396112 <953396...@qq.com> 于2020年4月29日周三 上午8:21写道: >>>>> >>>>>> Congrats, Jin Xing! >>>>>> >>>>>> >>>>>> ---Original--- >>>>>> From: "Stamatis Zampetakis">>>>> Date: Wed, Apr 29, 2020 05:47 AM >>>>>> To: "dev">>>>> Subject: [ANNOUNCE] New committer: Jin Xing >>>>>> >>>>>> >>>>>> Apache Calcite's Project Management Committee (PMC) has invited Jin >>>> Xing >>>>> to >>>>>> become a committer, and we are pleased to announce that he has >>>> accepted. >>>>>> >>>>>> Jin has contributed a lot of code in the project and many >>>>>> recent improvements in >>>>>> materialized view matching have his signature on them. Apart from >>> code >>>>>> contributions, Jin provides valuable help to the community by doing >>>>> reviews >>>>>> and >>>>>> answering questions in the devlist. >>>>>> >>>>>> Jin, welcome, thank you for your contributions, and we look forward >>> to >>>>> your >>>>>> further interactions with the community! If you wish, please feel >>> free >>>> to >>>>>> tell >>>>>> us more about yourself and what you are working on. >>>>>> >>>>>> Stamatis (on behalf of the Apache Calcite PMC) >>>>> >>>> >>> >>
Re:Re: [ANNOUNCE] New committer: Forward Xu
Congrations! Forward!-- Best, Wang Yanlin At 2020-04-29 10:52:25, "Feng Zhu" wrote: > Congrations! Forward! > >best, >Feng > >Chunwei Lei 于2020年4月29日周三 上午10:17写道: > >> Congrats, Forward! >> >> >> >> Best, >> Chunwei >> >> >> On Wed, Apr 29, 2020 at 6:46 AM Rui Wang wrote: >> >> > Congrats! >> > >> > >> > -Rui >> > >> > On Tue, Apr 28, 2020 at 3:04 PM Francis Chuang > > >> > wrote: >> > >> > > Congrats, Forward! >> > > >> > > Francis >> > > >> > > On 29/04/2020 7:53 am, Stamatis Zampetakis wrote: >> > > > Apache Calcite's Project Management Committee (PMC) has invited >> Forward >> > > Xu >> > > > to >> > > > become a committer, and we are pleased to announce that he has >> > accepted. >> > > > >> > > > Forward has been helping the project for some time now. He added many >> > new >> > > > SQL >> > > > functions to the project and is one of our JSON experts. On top of >> > that, >> > > and >> > > > other fixes, he is the one who added the Redis adapter to the >> project. >> > > > >> > > > Forward, welcome, thank you for your contributions, and we look >> forward >> > > to >> > > > your >> > > > further interactions with the community! If you wish, please feel >> free >> > to >> > > > tell >> > > > us more about yourself and what you are working on. >> > > > >> > > > Stamatis (on behalf of the Apache Calcite PMC) >> > > > >> > > >> > >>
Re:Re: [ANNOUNCE] New committer: Wang Yanlin
Hi, guys, thanks for your warm welcome. I'm working in Ant Finical, Alibaba Group. Currently my team is working on building a system to process big data in form of sql. We use calcite to parse sql, optimize Relnode and rewrite SqlNode to execute on different engines, like Spark,MaxCompute, HBase and so on. Calcite is really a great community, and it's really an honor for me to become calcite committer, hops to make more contribution to calcite. Thanks again. -- Best, Wang Yanlin 在 2020-04-29 13:58:35,"Zoltan Haindrich" 写道: >Congratulations! > >On 4/29/20 7:32 AM, Enrico Olivelli wrote: >> Congrats! >> >> Enrico >> >> Il Mer 29 Apr 2020, 04:51 Feng Zhu ha scritto: >> >>> Congrations! Yanlin! >>> >>> best, >>> Feng >>> >>> Chunwei Lei 于2020年4月29日周三 上午10:16写道: >>> >>>> Congrats, Yanlin! >>>> >>>> >>>> Best, >>>> Chunwei >>>> >>>> >>>> On Wed, Apr 29, 2020 at 10:07 AM Forward Xu >>>> wrote: >>>> >>>>> Congrats >>>>> >>>>> >>>>> Best, >>>>> >>>>> Forward >>>>> >>>>> 953396112 <953396...@qq.com> 于2020年4月29日周三 上午8:26写道: >>>>> >>>>>> Congrats, Wang Yanlin! >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ---Original--- >>>>>> From: "Stamatis Zampetakis">>>>> Date: Wed, Apr 29, 2020 05:51 AM >>>>>> To: "dev">>>>> Subject: [ANNOUNCE] New committer: Wang Yanlin >>>>>> >>>>>> >>>>>> Apache Calcite's Project Management Committee (PMC) has invited Wang >>>>> Yanlin >>>>>> to >>>>>> become a committer, and we are pleased to announce that he has >>>> accepted. >>>>>> >>>>>> Wang has pushed numerous fixes and improvements to the project, >>> landing >>>>> in >>>>>> total >>>>>> the impressive number of 30 commits to the master. Among other >>> things, >>>> he >>>>>> contributed some important features in the Interpreter. >>>>>> >>>>>> Wang, welcome, thank you for your contributions, and we look forward >>>> your >>>>>> further interactions with the community! If you wish, please feel >>> free >>>> to >>>>>> tell >>>>>> us more about yourself and what you are working on. >>>>>> >>>>>> Stamatis (on behalf of the Apache Calcite PMC) >>>>> >>>> >>> >>
[jira] [Created] (CALCITE-3929) NullPointerException when deserialize UDAF aggregate call from json string
Wang Yanlin created CALCITE-3929: Summary: NullPointerException when deserialize UDAF aggregate call from json string Key: CALCITE-3929 URL: https://issues.apache.org/jira/browse/CALCITE-3929 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin When serialize a relation algebra which contains udaf aggregate call, it works well and get the json string. But when deserialize from the json string, got NullPointerException as below {noformat} java.lang.RuntimeException: java.lang.NullPointerException at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:182) at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:126) at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:144) at org.apache.calcite.plan.RelWriterTest.deserializeAndDumpToTextFormat(RelWriterTest.java:828) at org.apache.calcite.plan.RelWriterTest.testUDAF(RelWriterTest.java:921) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675) at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:125) at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:139) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:131) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:81) at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115) at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:104) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:62) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:43) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:35) at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104) at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:202) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:198) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80) at java.util.ArrayList.forEach(ArrayList.java:1257) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at
[jira] [Created] (CALCITE-3921) Support serialize TableModify to json string and deserialize from it
Wang Yanlin created CALCITE-3921: Summary: Support serialize TableModify to json string and deserialize from it Key: CALCITE-3921 URL: https://issues.apache.org/jira/browse/CALCITE-3921 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin When trying to serialize TableModify to json string, got {noformat} java.lang.UnsupportedOperationException: type not serializable: INSERT (type org.apache.calcite.rel.core.TableModify.Operation) at org.apache.calcite.rel.externalize.RelJson.toJson(RelJson.java:322) at org.apache.calcite.rel.externalize.RelJsonWriter.put(RelJsonWriter.java:84) at org.apache.calcite.rel.externalize.RelJsonWriter.explain_(RelJsonWriter.java:67) at org.apache.calcite.rel.externalize.RelJsonWriter.done(RelJsonWriter.java:129) at org.apache.calcite.rel.AbstractRelNode.explain(AbstractRelNode.java:303) at org.apache.calcite.plan.RelOptUtil.dumpPlan(RelOptUtil.java:2110) at org.apache.calcite.plan.RelWriterTest.testTableModify(RelWriterTest.java:922) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675) at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:125) at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:139) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:131) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:81) at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115) at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:104) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:62) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:43) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:35) at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104) at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:202) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:198) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80) at java.util.ArrayList.forEach(ArrayList.java:1257) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139) at
[jira] [Created] (CALCITE-3919) Improve ProjectJoinTransposeRule to allow user choose whether to keep join filter
Wang Yanlin created CALCITE-3919: Summary: Improve ProjectJoinTransposeRule to allow user choose whether to keep join filter Key: CALCITE-3919 URL: https://issues.apache.org/jira/browse/CALCITE-3919 Project: Calcite Issue Type: Improvement Reporter: Wang Yanlin Currently, when apply ProjectJoinTransposeRule, the join condition may be also pushed, for example for sql The relation algebra for sql {noformat} select emp.ename, sum(bonus.sal) from emp left outer join bonus on emp.ename = bonus.ename and floor(emp.sal) > 10 group by emp.ename {noformat} is {noformat} LogicalAggregate(group=[{0}], EXPR$1=[SUM($1)]) LogicalProject(ENAME=[$1], SAL0=[$11]) LogicalJoin(condition=[AND(=($1, $9), >(FLOOR($5), 10))], joinType=[left]) LogicalTableScan(table=[[CATALOG, SALES, EMP]]) LogicalTableScan(table=[[CATALOG, SALES, BONUS]]) {noformat} After applying *ProjectJoinTransposeRule*, the join condition 'floor(emp.sal) > 10' is also pushed down, {noformat} LogicalAggregate(group=[{0}], EXPR$1=[SUM($1)]) LogicalProject(ENAME=[$0], SAL0=[$3]) LogicalJoin(condition=[AND(=($0, $2), $1)], joinType=[left]) LogicalProject(ENAME=[$1], >=[>(FLOOR($5), 10)]) LogicalTableScan(table=[[CATALOG, SALES, EMP]]) LogicalProject(ENAME=[$0], SAL=[$2]) LogicalTableScan(table=[[CATALOG, SALES, BONUS]]) {noformat} In some cases, users may want to push Project past Join, but keep the join condition unchanged, so we can improve *ProjectJoinTransposeRule* to allow user choose whether to keep join filter -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3664) Lost sort in subquery when converting SqlNode to Relnode
Wang Yanlin created CALCITE-3664: Summary: Lost sort in subquery when converting SqlNode to Relnode Key: CALCITE-3664 URL: https://issues.apache.org/jira/browse/CALCITE-3664 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin Lost sort in subquery when converting a SqlNode to Relnode, making its semantics changed. The test case to reproduce {code:java} // SqlToRelConverterTest @Test public void testSortInSubquery() { final String sql = "select ename from (select ename from emp order by ename) a"; sql(sql).ok(); } {code} This test case will success with this plan. {code:java} {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3651) NullPointerException when convert relational algebra that correlates TableFunctionScan
Wang Yanlin created CALCITE-3651: Summary: NullPointerException when convert relational algebra that correlates TableFunctionScan Key: CALCITE-3651 URL: https://issues.apache.org/jira/browse/CALCITE-3651 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin Converting RelNode that correlate TableFunctionScan will cause NPE. {noformat} LogicalProject(product_class_id=[$0], product_id=[$1], brand_name=[$2], product_name=[$3], SKU=[$4], SRP=[$5], gross_weight=[$6], net_weight=[$7], recyclable_package=[$8], low_fat=[$9], units_per_case=[$10], cases_per_pallet=[$11], shelf_width=[$12], shelf_height=[$13], shelf_depth=[$14], I=[$15]) LogicalCorrelate(correlation=[$cor0], joinType=[inner], requiredColumns=[{1}]) JdbcTableScan(table=[[foodmart, product]]) LogicalTableFunctionScan(invocation=[RAMP($cor0.product_id)], rowType=[RecordType(INTEGER I)]) {noformat} Add this test case to reproduce {noformat} // RelToSqlConverterTest @Test public void testLateralCorrelate() { final String query = "select * from \"product\", lateral table(RAMP(\"product\".\"product_id\"))"; sql(query).ok(""); } {noformat} got {noformat} java.lang.RuntimeException: While invoking method 'public org.apache.calcite.rel.rel2sql.SqlImplementor$Result org.apache.calcite.rel.rel2sql.RelToSqlConverter.visit(org.apache.calcite.rel.core.Project)' at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527) at org.apache.calcite.rel.rel2sql.RelToSqlConverter.dispatch(RelToSqlConverter.java:126) at org.apache.calcite.rel.rel2sql.RelToSqlConverter.visitChild(RelToSqlConverter.java:132) at org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.toSql(RelToSqlConverterTest.java:190) at org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.access$200(RelToSqlConverterTest.java:94) at org.apache.calcite.rel.rel2sql.RelToSqlConverterTest$Sql.exec(RelToSqlConverterTest.java:4541) at org.apache.calcite.rel.rel2sql.RelToSqlConverterTest$Sql.ok(RelToSqlConverterTest.java:4516) at org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.testLateralCorrelate(RelToSqlConverterTest.java:3741) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675) at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:125) at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:139) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:131) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:81) at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115) at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:104) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:62) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:43) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:35) at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104) at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:202) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:198) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69) at org.junit.platform.engine.support.hierarchical.NodeTestTask.l
[jira] [Created] (CALCITE-3639) JoinConditionPushRule fail to push filter to inputs
Wang Yanlin created CALCITE-3639: Summary: JoinConditionPushRule fail to push filter to inputs Key: CALCITE-3639 URL: https://issues.apache.org/jira/browse/CALCITE-3639 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin The relnode(without applying optimize rules) for the sql {code:java} String sql = "select empno, emp.deptno from emp left join dept\n" + " on emp.deptno = dept.deptno and empno = 10 and dept.deptno = 20"; {code} is {code:java} LogicalProject(EMPNO=[$0], DEPTNO=[$7]) LogicalJoin(condition=[AND(=($7, $9), =($0, 10), =($9, 20))], joinType=[left]) LogicalTableScan(table=[[CATALOG, SALES, EMP]]) LogicalTableScan(table=[[CATALOG, SALES, DEPT]]) {code} After optimized with *JoinConditionPushRule*, the relnode becomes {code:java} LogicalProject(EMPNO=[$0], DEPTNO=[$7]) LogicalJoin(condition=[AND(=($7, $9), =($0, 10))], joinType=[left]) LogicalTableScan(table=[[CATALOG, SALES, EMP]]) LogicalFilter(condition=[=($0, 20)]) LogicalTableScan(table=[[CATALOG, SALES, DEPT]]) {code} The optimize rule failed to push *empno = 10* to the left input, the better relnode should be {code:java} LogicalProject(EMPNO=[$0], DEPTNO=[$7]) LogicalJoin(condition=[AND(=($7, $9))], joinType=[left]) LogicalFilter(condition=[=($0, 10)]) LogicalTableScan(table=[[CATALOG, SALES, EMP]]) LogicalFilter(condition=[=($0, 20)]) LogicalTableScan(table=[[CATALOG, SALES, DEPT]]) {code} Add this test case to reproduce {code:java} // RelOptRulesTest @Test public void testFilterInLeftJoin() { String sql = "select empno, emp.deptno from emp left join dept\n" + " on emp.deptno = dept.deptno and empno = 10 and dept.deptno = 20"; sql(sql).withRule(FilterJoinRule.JOIN).check(); } // data for this case {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3605) Semantics of relation operator changed after decorrelated
Wang Yanlin created CALCITE-3605: Summary: Semantics of relation operator changed after decorrelated Key: CALCITE-3605 URL: https://issues.apache.org/jira/browse/CALCITE-3605 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin For sql {code:java} final String sql = "select deptno from (select deptno, empno from emp) p\n" + "where empno in\n" + "(select count(*) from dept where p.deptno = dept.deptno)"; {code} before decorrelated, the relnode tree is {code:java} LogicalProject(DEPTNO=[$0]) LogicalProject(DEPTNO=[$0], EMPNO=[$1]) LogicalFilter(condition=[=($1, $2)]) LogicalCorrelate(correlation=[$cor0], joinType=[inner], requiredColumns=[{0}]) LogicalProject(DEPTNO=[$7], EMPNO=[$0]) LogicalTableScan(table=[[CATALOG, SALES, EMP]]) LogicalAggregate(group=[{}], EXPR$0=[COUNT()]) LogicalProject($f0=[0]) LogicalFilter(condition=[=($cor0.DEPTNO, $0)]) LogicalTableScan(table=[[CATALOG, SALES, DEPT]]) {code} after decorrelatd, the relnode tree is {code:java} LogicalProject(DEPTNO=[$0]) LogicalJoin(condition=[AND(=($0, $2), =($1, $3))], joinType=[inner]) LogicalProject(DEPTNO=[$7], EMPNO=[$0]) LogicalTableScan(table=[[CATALOG, SALES, EMP]]) LogicalAggregate(group=[{0}], EXPR$0=[COUNT()]) LogicalProject(DEPTNO=[$0], $f0=[0]) LogicalTableScan(table=[[CATALOG, SALES, DEPT]]) {code} however, these two relnode trees have different semantics. Assume, the data in dept and emp is as below {code:java} EMP (deptno, empno) 3, 6 10, 1 8, 0 DEPT (deptno, name) 3, "a" 10, "b" {code} The output of the sql should be +--+ |10| |8| +--+ but the output of the relnode after decorrelated is +--+ |10| +--+ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3596) Sql in annotation of OverScope class has syntax error
Wang Yanlin created CALCITE-3596: Summary: Sql in annotation of OverScope class has syntax error Key: CALCITE-3596 URL: https://issues.apache.org/jira/browse/CALCITE-3596 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin The sql in class of OverScope (https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/validate/OverScope.java#L33) has syntax error, cause SqlParseException {code:java} // SqlToRelConverterTest @Test public void testOver() { /* final String sql = "SELECT * FROM\n" + "(SELECT deptno, count(*) OVER (ORDER BY empno RANGE BETWEEN 2 PRECEDING AND 2 FOLLOWING) FROM emp) "; */ final String sql = "SELECT name FROM\n" + "(SELECT * FROM\n" + "emp OVER (ORDER BY empno RANGE BETWEEN 2 PRECEDING AND 2 FOLLOWING))"; sql(sql).ok(); } {code} got {code:java} Caused by: org.apache.calcite.sql.parser.impl.ParseException: Encountered "OVER" at line 3, column 5. Was expecting one of: "AS" ... "EXCEPT" ... "EXTEND" ... "FETCH" ... "FOR" ... "GROUP" ... "HAVING" ... "INTERSECT" ... "LIMIT" ... "MATCH_RECOGNIZE" ... "OFFSET" ... "ORDER" ... "MINUS" ... "TABLESAMPLE" ... "UNION" ... "WHERE" ... "WINDOW" ... "(" ... ")" ... ... ... ... ... ... ... "NATURAL" ... "JOIN" ... "INNER" ... "LEFT" ... "RIGHT" ... "FULL" ... "CROSS" ... "," ... "OUTER" ... "." ... {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3587) RexBuilder may lose decimal fraction for creating literal with DECIMAL type
Wang Yanlin created CALCITE-3587: Summary: RexBuilder may lose decimal fraction for creating literal with DECIMAL type Key: CALCITE-3587 URL: https://issues.apache.org/jira/browse/CALCITE-3587 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin this test {code:java} // RexBuilderTest @Test public void testDecimal() { final RelDataTypeFactory typeFactory = new SqlTypeFactoryImpl(RelDataTypeSystem.DEFAULT); final RelDataType type = typeFactory.createSqlType(SqlTypeName.DECIMAL, 4, 2); final RexBuilder builder = new RexBuilder(typeFactory); final RexLiteral literal = (RexLiteral) builder.makeLiteral(12.3, type, false); Comparable value = literal.getValue(); assertThat(value.toString(), is("12.3")); } {code} fails with message {code:java} java.lang.AssertionError: Expected: is "12.3" but: was "12" Expected :12.3 Actual :12 {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3583) Support serialized to json and deserialized from json for Exchange relation operator
Wang Yanlin created CALCITE-3583: Summary: Support serialized to json and deserialized from json for Exchange relation operator Key: CALCITE-3583 URL: https://issues.apache.org/jira/browse/CALCITE-3583 Project: Calcite Issue Type: Improvement Reporter: Wang Yanlin Currently, serialize Exchange relnode to json will cause exception {code:java} // RelWriterTest @Test public void testExchange() { final FrameworkConfig config = RelBuilderTest.config().build(); final RelBuilder builder = RelBuilder.create(config); final RelNode rel = builder .scan("EMP") .exchange(RelDistributions.hash(ImmutableList.of(0, 1))) .build(); String relJson = RelOptUtil.dumpPlan("", rel, SqlExplainFormat.JSON, SqlExplainLevel.EXPPLAN_ATTRIBUTES); String s = deserializeAndDumpToTextFormat(getSchema(rel), relJson); final String expected = "" + "LogicalExchange(distribution=[hash[0, 1]])\n" + " LogicalTableScan(table=[[scott, EMP]])\n"; assertThat(s, isLinux(expected)); } {code} got {code:java} java.lang.UnsupportedOperationException: type not serializable: hash[0, 1] (type org.apache.calcite.rel.RelDistributions.RelDistributionImpl) at org.apache.calcite.rel.externalize.RelJson.toJson(RelJson.java:290) at org.apache.calcite.rel.externalize.RelJsonWriter.put(RelJsonWriter.java:83) at org.apache.calcite.rel.externalize.RelJsonWriter.explain_(RelJsonWriter.java:66) at org.apache.calcite.rel.externalize.RelJsonWriter.done(RelJsonWriter.java:128) at org.apache.calcite.rel.AbstractRelNode.explain(AbstractRelNode.java:299) at org.apache.calcite.plan.RelOptUtil.dumpPlan(RelOptUtil.java:1981) at org.apache.calcite.plan.RelWriterTest.testExchange(RelWriterTest.java:772) {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3567) UNNEST(RECORDTYPE(MAP)) not supported
Wang Yanlin created CALCITE-3567: Summary: UNNEST(RECORDTYPE(MAP)) not supported Key: CALCITE-3567 URL: https://issues.apache.org/jira/browse/CALCITE-3567 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin UNNEST(RECORDTYPE(ARRAY)) and UNNEST(RECORDTYPE(MULTISET)) works well, but UNNEST(RECORDTYPE(MAP)) is not supported. {code:java} // JdbcTest @Test public void testUnnestRecordType() { // unnest(RecordType(Array)) CalciteAssert.that() .query("select * from unnest\n" + "(select t.x from (values array[10, 20], array[30, 40]) as t(x))\n" + " with ordinality as t(a, o)") .returnsUnordered("A=10; O=1", "A=20; O=2", "A=30; O=1", "A=40; O=2"); // unnest(RecordType(Multiset)) CalciteAssert.that() .query("select * from unnest\n" + "(select t.x from (values multiset[10, 20], array[30, 40]) as t(x))\n" + " with ordinality as t(a, o)") .returnsUnordered("A=10; O=1", "A=20; O=2", "A=30; O=1", "A=40; O=2"); // unnest(RecordType(Map)) CalciteAssert.that() .query("select * from unnest\n" + "(select t.x from (values map['a', 20, 'b', 30], map['c', 40]) as t(x))\n" + " with ordinality as t(a, b, o)") .returnsUnordered("A=a; B=20; O=1", "A=B; B=30; O=2", "A=c; B=40; O=1"); } {code} In the case, test for *unnest(RecordType(Array))* and *unnest(RecordType(Multiset))* success, but got exception for *unnest(RecordType(Map))*. For the first two test of *unnest(RecordType(Array))* and *unnest(RecordType(Multiset))*, the relnode tree with type is like this {noformat} EnumerableUncollect(withOrdinality=[true]), type = RecordType(INTEGER EXPR$0, INTEGER ORDINALITY) EnumerableUnion(all=[true]), type = RecordType(INTEGER ARRAY EXPR$0) EnumerableCalc(expr#0=[{inputs}], expr#1=[10], expr#2=[20], expr#3=[ARRAY($t1, $t2)], EXPR$0=[$t3]), type = RecordType(INTEGER ARRAY EXPR$0) EnumerableValues(tuples=[[{ 0 }]]), type = RecordType(INTEGER ZERO) EnumerableCalc(expr#0=[{inputs}], expr#1=[30], expr#2=[40], expr#3=[ARRAY($t1, $t2)], EXPR$0=[$t3]), type = RecordType(INTEGER ARRAY EXPR$0) EnumerableValues(tuples=[[{ 0 }]]), type = RecordType(INTEGER ZERO) EnumerableUncollect(withOrdinality=[true]), type = RecordType(INTEGER EXPR$0, INTEGER ORDINALITY) EnumerableUnion(all=[true]), type = RecordType(INTEGER ARRAY EXPR$0) EnumerableCalc(expr#0=[{inputs}], expr#1=[$SLICE($t0)], EXPR$0=[$t1]), type = RecordType(INTEGER MULTISET EXPR$0) EnumerableCollect(field=[EXPR$0]), type = RecordType(RecordType(INTEGER ROW_VALUE) MULTISET EXPR$0) EnumerableValues(tuples=[[{ 10 }, { 20 }]]), type = RecordType(INTEGER ROW_VALUE) EnumerableCalc(expr#0=[{inputs}], expr#1=[30], expr#2=[40], expr#3=[ARRAY($t1, $t2)], EXPR$0=[$t3]), type = RecordType(INTEGER ARRAY EXPR$0) EnumerableValues(tuples=[[{ 0 }]]), type = RecordType(INTEGER ZERO) {noformat} Part of the stacktrace of the exception is: {code:java} Cannot apply 'UNNEST' to arguments of type 'UNNEST()'. Supported form(s): 'UNNEST()' 'UNNEST()' 'UNNEST()' UNNEST() at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463) at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:839) at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:824) at org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:4905) at org.apache.calcite.sql.SqlCallBinding.newValidationSignatureError(SqlCallBinding.java:280) at org.apache.calcite.sql.type.CompositeOperandTypeChecker.checkOperandTypes(CompositeOperandTypeChecker.java:261) at org.apache.calcite.sql.SqlOperator.checkOperandTypes(SqlOperator.java:668) at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:432) at org.apache.calcite.sql.validate.UnnestNamespace.validateImpl(UnnestNamespace.java:66) at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84) at org.apache.calcite.sql.validate
[jira] [Created] (CALCITE-3561) Support using unnest in Interpreter
Wang Yanlin created CALCITE-3561: Summary: Support using unnest in Interpreter Key: CALCITE-3561 URL: https://issues.apache.org/jira/browse/CALCITE-3561 Project: Calcite Issue Type: Improvement Reporter: Wang Yanlin Using unnest in Interpreter will cause exception, {code:java} // InterpreterTest @Test public void testInterpretUnnest() throws Exception { final String sql = "select * from unnest(multiset[1, 2])"; sql(sql).returnsRows("[1]", "[2]"); } {code} got {code:java} java.lang.AssertionError: interpreter: no implementation for class org.apache.calcite.rel.core.Uncollect at org.apache.calcite.interpreter.Interpreter$CompilerImpl.visit(Interpreter.java:463) at org.apache.calcite.interpreter.Nodes$CoreCompiler.visit(Nodes.java:44) at org.apache.calcite.rel.SingleRel.childrenAccept(SingleRel.java:72) at org.apache.calcite.interpreter.Interpreter$CompilerImpl.visit(Interpreter.java:450) at org.apache.calcite.interpreter.Nodes$CoreCompiler.visit(Nodes.java:44) at org.apache.calcite.interpreter.Interpreter$CompilerImpl.visitRoot(Interpreter.java:408) at org.apache.calcite.interpreter.Interpreter.(Interpreter.java:89) at org.apache.calcite.test.InterpreterTest$Sql.returnsRows(InterpreterTest.java:121) at org.apache.calcite.test.InterpreterTest$Sql.returnsRows(InterpreterTest.java:105) at org.apache.calcite.test.InterpreterTest.testInterpretUnnest(InterpreterTest.java:446) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) {code} Add support to use unnest in Interpreter. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3557) ClassCastException for using nested multiset or array inside multiset
Wang Yanlin created CALCITE-3557: Summary: ClassCastException for using nested multiset or array inside multiset Key: CALCITE-3557 URL: https://issues.apache.org/jira/browse/CALCITE-3557 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin {code:java} // JdbcTest @Test public void testNestedMultiset() { CalciteAssert.that() .query("select multiset[ARRAY[1, 2], ARRAY[3, 4]]") .returns("EXPR$0=[[1, 2], [3, 4]]\n"); CalciteAssert.that() .query("select multiset[multiset[1, 2], multiset[3, 4]]") .returns("EXPR$0=[[1, 2], [3, 4]]\n"); } {code} Got {code:java} java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to java.lang.Integer at org.apache.calcite.avatica.util.AbstractCursor$IntAccessor.getInt(AbstractCursor.java:541) at org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.convertValue(AbstractCursor.java:1318) at org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getObject(AbstractCursor.java:1299) at org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getArray(AbstractCursor.java:1352) at org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.convertValue(AbstractCursor.java:1326) at org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getObject(AbstractCursor.java:1299) at org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getArray(AbstractCursor.java:1352) at org.apache.calcite.avatica.AvaticaSite.get(AvaticaSite.java:305) at org.apache.calcite.avatica.AvaticaResultSet.getObject(AvaticaResultSet.java:393) at org.apache.calcite.test.CalciteAssert$ResultSetFormatter.rowToString(CalciteAssert.java:1978) at org.apache.calcite.test.CalciteAssert$ResultSetFormatter.resultSet(CalciteAssert.java:1966) at org.apache.calcite.test.CalciteAssert.lambda$checkResult$2(CalciteAssert.java:303) at org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:544) at org.apache.calcite.test.CalciteAssert$AssertQuery.lambda$returns$1(CalciteAssert.java:1514) at org.apache.calcite.test.CalciteAssert$AssertQuery.withConnection(CalciteAssert.java:1446) at org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1512) at org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1495) at org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1458) at org.apache.calcite.test.JdbcTest.testNestedMultiset(JdbcTest.java:2057) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33) at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230) at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58) Suppressed: org.apache.calcite.util.TestUtil$ExtraInformation: With materializationsEnabled=false, limit=0, sql=select multiset[multiset[1, 2], multiset[3, 4]] at org.apache.calcite.util.TestUtil.rethrow(TestUtil.java:268) at org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:560) ... 28 more {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3554) NullPointerException when deserializing LogicalTableFunctionScan from json
Wang Yanlin created CALCITE-3554: Summary: NullPointerException when deserializing LogicalTableFunctionScan from json Key: CALCITE-3554 URL: https://issues.apache.org/jira/browse/CALCITE-3554 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin Run this test case in *RelWriterTest* {code:java} @Test public void testTableFunctionScan() { final RelBuilder builder = RelBuilder.create(RelBuilderTest.config().build()); final SqlOperator dedupFunction = new MockSqlOperatorTable.DedupFunction(); RelNode rel = builder.scan("EMP") .scan("DEPT") .functionScan(dedupFunction, 2, builder.cursor(2, 0), builder.cursor(2, 1)) .build(); RelJsonWriter jsonWriter = new RelJsonWriter(); rel.explain(jsonWriter); String relJson = jsonWriter.asString(); System.out.println(RelOptUtil.toString(rel)); System.out.println(relJson); String s = deserializeAndDumpToTextFormat(getSchema(rel), relJson); final String expected = "xxx"; assertThat(s, isLinux(expected)); } {code} Got {code:java} Caused by: java.lang.NullPointerException: operator at java.util.Objects.requireNonNull(Objects.java:228) at org.apache.calcite.rex.RexCall.(RexCall.java:83) at org.apache.calcite.rex.RexBuilder.makeCall(RexBuilder.java:237) at org.apache.calcite.rel.externalize.RelJson.toRex(RelJson.java:502) at org.apache.calcite.rel.externalize.RelJsonReader$2.getExpression(RelJsonReader.java:132) at org.apache.calcite.rel.core.TableFunctionScan.(TableFunctionScan.java:97) at org.apache.calcite.rel.logical.LogicalTableFunctionScan.(LogicalTableFunctionScan.java:81) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.calcite.rel.externalize.RelJsonReader.readRel(RelJsonReader.java:261) at org.apache.calcite.rel.externalize.RelJsonReader.readRels(RelJsonReader.java:91) at org.apache.calcite.rel.externalize.RelJsonReader.read(RelJsonReader.java:85) at org.apache.calcite.plan.RelWriterTest.lambda$deserializeAndDumpToTextFormat$6(RelWriterTest.java:793) at org.apache.calcite.tools.Frameworks.lambda$withPlanner$0(Frameworks.java:130) at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:915) at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:179) ... 26 more {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3494) Support decimal type aggregate in Interpreter
Wang Yanlin created CALCITE-3494: Summary: Support decimal type aggregate in Interpreter Key: CALCITE-3494 URL: https://issues.apache.org/jira/browse/CALCITE-3494 Project: Calcite Issue Type: Improvement Reporter: Wang Yanlin Using decimal type aggregate in Interpreter cause exception. Add this case in *InterpreterTest* to reproduce. {code:java} @Test public void testDecimalType() throws Exception { final String sql = "select x, min(y), max(y), sum(y)\n" + "from (values ('a', 1.2), ('a', 2.3), ('a', 15)) as t(x, y)\n" + "group by x"; sql(sql).returnsRows("[a, 1.2, 15.0, 18.5]"); } {code} The stack trace is like this {noformat} java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at org.apache.calcite.interpreter.AggregateNode$UdaAccumulator.send(AggregateNode.java:662) at org.apache.calcite.interpreter.AggregateNode$AccumulatorList.send(AggregateNode.java:398) at org.apache.calcite.interpreter.AggregateNode$Grouping.send(AggregateNode.java:366) at org.apache.calcite.interpreter.AggregateNode.run(AggregateNode.java:96) at org.apache.calcite.interpreter.Interpreter.start(Interpreter.java:130) at org.apache.calcite.interpreter.Interpreter.enumerator(Interpreter.java:107) at org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33) at org.apache.calcite.test.InterpreterTest.assertRows(InterpreterTest.java:173) at org.apache.calcite.test.InterpreterTest.access$100(InterpreterTest.java:51) at org.apache.calcite.test.InterpreterTest$Sql.returnsRows(InterpreterTest.java:122) at org.apache.calcite.test.InterpreterTest$Sql.returnsRows(InterpreterTest.java:105) at org.apache.calcite.test.InterpreterTest.testDecimalType(InterpreterTest.java:441) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.calcite.interpreter.AggregateNode$UdaAccumulator.send(AggregateNode.java:660) ... 35 more Caused by: java.lang.ClassCastException: java.math.BigDecimal cannot be cast to java.lang.Long at org.apache.calcite.interpreter.AggregateNode$NumericComparison.add(AggregateNode.java:494) ... 40 more {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re:Re: [ANNOUNCE] Haisheng Yuan joins Calcite PMC
Congrats Haisheng! Well deserved!-- Best, Wang Yanlin 在 2019-11-11 14:09:58,"Leonard Xu" 写道: >Congratulations! HaiSheng > >Best, >Leonard Xu >> On 2019年11月11日, at 下午1:22, Danny Chan wrote: >> >> Congratulations HaiSheng! >
[jira] [Created] (CALCITE-3481) Support convert TableFunctionScan to SqlNode
Wang Yanlin created CALCITE-3481: Summary: Support convert TableFunctionScan to SqlNode Key: CALCITE-3481 URL: https://issues.apache.org/jira/browse/CALCITE-3481 Project: Calcite Issue Type: Improvement Reporter: Wang Yanlin Currently, converting *TableFunctionScan* to sql is not supported, doing this will cause exception. Add the following test case in *RelToSqlConverterTest* to reproduce. {code:java} @Test public void testTableFunctionScan() { final String query = "SELECT *\n" + "FROM TABLE(DEDUP(CURSOR(select \"product_id\", \"product_name\"\n" + "from \"product\"), CURSOR(select \"employee_id\", \"full_name\"\n" + "from \"employee\"), 'NAME'))"; final String expected = "SELECT *\n" + "FROM TABLE(DEDUP(CURSOR ((SELECT \"product_id\", \"product_name\"\n" + "FROM \"foodmart\".\"product\")), CURSOR ((SELECT \"employee_id\", \"full_name\"\n" + "FROM \"foodmart\".\"employee\")), 'NAME'))"; sql(query).ok(expected); } {code} The stack trace of exception is {noformat} java.lang.RuntimeException: While invoking method 'public org.apache.calcite.rel.rel2sql.SqlImplementor$Result org.apache.calcite.rel.rel2sql.RelToSqlConverter.visit(org.apache.calcite.rel.core.Project)' at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527) at org.apache.calcite.rel.rel2sql.RelToSqlConverter.dispatch(RelToSqlConverter.java:122) at org.apache.calcite.rel.rel2sql.RelToSqlConverter.visitChild(RelToSqlConverter.java:128) at org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.toSql(RelToSqlConverterTest.java:187) at org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.access$100(RelToSqlConverterTest.java:92) at org.apache.calcite.rel.rel2sql.RelToSqlConverterTest$Sql.exec(RelToSqlConverterTest.java:4331) at org.apache.calcite.rel.rel2sql.RelToSqlConverterTest$Sql.ok(RelToSqlConverterTest.java:4306) at org.apache.calcite.rel.rel2sql.RelToSqlConverterTest.testTableFunctionScan(RelToSqlConverterTest.java:4126) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:524) ... 29 more Caused by: java.lang.RuntimeException: While invoking method 'public org.apache.calcite.rel.rel2sql.SqlImplementor$Result org.apache.calcite.rel.rel2sql.RelToSqlConverter.visit(org.apache.calcite.rel.Re
[jira] [Created] (CALCITE-3473) Unique keys of table scan should contain key column(s)
Wang Yanlin created CALCITE-3473: Summary: Unique keys of table scan should contain key column(s) Key: CALCITE-3473 URL: https://issues.apache.org/jira/browse/CALCITE-3473 Project: Calcite Issue Type: Improvement Reporter: Wang Yanlin Currently, *mq.getUniqueKeys()* for *TableScan* returns empty. However, for table with key column(s) defined, we can refer uniqueness on key column(s), so the result should contain key column(s) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3464) RexSimplify simplifies plan having filter with NULL to empty values
Wang Yanlin created CALCITE-3464: Summary: RexSimplify simplifies plan having filter with NULL to empty values Key: CALCITE-3464 URL: https://issues.apache.org/jira/browse/CALCITE-3464 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin When filter by comparing to null in sql, the plan will get empty result {code:java} @Test public void testSimplifyItemEqualNull() { String query = "select * from sales.customer as t1 where name = NULL"; sql(query) .withTester(t -> createDynamicTester()) .withRule(ReduceExpressionsRule.FILTER_INSTANCE) .check(); } {code} The plan after optimization is like this {code:java} LogicalProject(**=[$1]) LogicalValues(tuples=[[]]) {code} The optimized plan will get empty result, is this the result we want? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3459) AssertionError for using Timestamp/Time/Date in table function
Wang Yanlin created CALCITE-3459: Summary: AssertionError for using Timestamp/Time/Date in table function Key: CALCITE-3459 URL: https://issues.apache.org/jira/browse/CALCITE-3459 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin Add the following test case in *TableFunctionTest* to reproduce, you need to add the implementation of user defined table function in *Smalls* before running the test. {code:java} @Test public void testTableFunctionWithTimeRelatedParameter() throws SQLException { try (Connection connection = DriverManager.getConnection("jdbc:calcite:")) { CalciteConnection calciteConnection = connection.unwrap(CalciteConnection.class); SchemaPlus rootSchema = calciteConnection.getRootSchema(); SchemaPlus schema = rootSchema.add("s", new AbstractSchema()); final TableFunction table1 = TableFunctionImpl.create(Smalls.TIMESTAMP_STRING_LENGTH); schema.add("TimestampStringLength", table1); final String sql1 = "select *\n" + "from table(\"s\".\"TimestampStringLength\"(TIMESTAMP '2019-10-12 19:00:35'))\n" + "as t(n, c) where n > 19"; ResultSet resultSet1 = connection.createStatement().executeQuery(sql1); assertThat(CalciteAssert.toString(resultSet1), equalTo("N=20; C=abcdefg\n")); final TableFunction table2 = TableFunctionImpl.create(Smalls.DATE_STRING_LENGTH); schema.add("DateStringLength", table2); final String sql2 = "select *\n" + "from table(\"s\".\"DateStringLength\"(DATE '2019-10-12')) as t(n, c)\n" + "where n > 8"; ResultSet resultSet2 = connection.createStatement().executeQuery(sql2); assertThat(CalciteAssert.toString(resultSet2), equalTo("N=9; C=abcdefghi\n")); final TableFunction table3 = TableFunctionImpl.create(Smalls.TIME_STRING_LENGTH); schema.add("TimeStringLength", table3); final String sql3 = "select *\n" + "from table(\"s\".\"TimeStringLength\"(TIME '19:00:35')) as t(n, c)\n" + "where n > 6"; ResultSet resultSet3 = connection.createStatement().executeQuery(sql3); assertThat(CalciteAssert.toString(resultSet3), equalTo("N=7; C=abcdefg\n")); } } {code} The stack trace of exception {code:java} java.lang.AssertionError: value 2019-10-12 19:00:35 does not match type class java.sql.Timestamp at org.apache.calcite.linq4j.tree.ConstantExpression.(ConstantExpression.java:50) at org.apache.calcite.linq4j.tree.Expressions.constant(Expressions.java:589) at org.apache.calcite.linq4j.tree.OptimizeShuttle.visit(OptimizeShuttle.java:278) at org.apache.calcite.linq4j.tree.UnaryExpression.accept(UnaryExpression.java:37) at org.apache.calcite.linq4j.tree.GotoStatement.accept(GotoStatement.java:60) at org.apache.calcite.linq4j.tree.BlockBuilder.optimize(BlockBuilder.java:437) at org.apache.calcite.linq4j.tree.BlockBuilder.toBlock(BlockBuilder.java:321) at org.apache.calcite.sql.validate.SqlUserDefinedTableMacro.coerce(SqlUserDefinedTableMacro.java:190) at org.apache.calcite.sql.validate.SqlUserDefinedTableMacro.convertArguments(SqlUserDefinedTableMacro.java:110) at org.apache.calcite.sql.validate.SqlUserDefinedTableFunction.getRowType(SqlUserDefinedTableFunction.java:70) at org.apache.calcite.sql.validate.ProcedureNamespace.validateImpl(ProcedureNamespace.java:62) at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1009) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:969) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:3129) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:3111) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3383) at org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60) at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1009) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:969) at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216) at
[jira] [Created] (CALCITE-3429) Refinement for implementation of sql type factory
Wang Yanlin created CALCITE-3429: Summary: Refinement for implementation of sql type factory Key: CALCITE-3429 URL: https://issues.apache.org/jira/browse/CALCITE-3429 Project: Calcite Issue Type: Improvement Reporter: Wang Yanlin 1. The SqlTypeName of data type created with subclass of Map should be SqlTypename.Map, currently not. 2. The SqlTypeName of data type created with subclass of List or Array should be SqlTypename.Array, currently not. 3. SqlTypeName.Map is not a basic type, should not be used to createSqlType, just like SqlTypeName.Array -- This message was sent by Atlassian Jira (v8.3.4#803005)
Apply to be registered in JIRA as a contributor
Hi, community, Follow the direction on Calcite developing page, https://calcite.apache.org/develop/ > If you are going to take on the issue right away assign it to yourself. To > assign issues to yourself you have to be registered in JIRA as a contributor. > In order to do that, send an email to the developers list and provide your > JIRA username. I want to be registered in JIRA as a contributor so that I can assign issues to myself. My JIRA username is: yanlin-Lynn, and my full name is: Wang Yanlin -- Best, Wang Yanlin
[jira] [Created] (CALCITE-3423) Support using CAST operation and bool type value in table macro
Wang Yanlin created CALCITE-3423: Summary: Support using CAST operation and bool type value in table macro Key: CALCITE-3423 URL: https://issues.apache.org/jira/browse/CALCITE-3423 Project: Calcite Issue Type: New Feature Reporter: Wang Yanlin Currently, using bool type or cast operation in table macro, got exception. Add the code snippet in *JdbcTest* to reproduce. {code:java} // check for cast resultSet = connection.createStatement().executeQuery( "select * from table(\"s\".\"Str\"(MAP['a', 1, 'baz', 2], cast(1 as bigint))) as t(n)"); assertThat(CalciteAssert.toString(resultSet), equalTo("N={'a'=1, 'baz'=2}\n" + "N=1 \n")); // check for bool type resultSet = connection.createStatement().executeQuery( "select * from table(\"s\".\"Str\"(MAP['a', 1, 'baz', 2], true)) as t(n)"); assertThat(CalciteAssert.toString(resultSet), equalTo("N={'a'=1, 'baz'=2}\n" + "N=true\n")); // check for nested cast resultSet = connection.createStatement().executeQuery( "select * from table(\"s\".\"Str\"(MAP['a', 1, 'baz', 2]," + "cast(cast(1 as int) as varchar(1 as t(n)"); assertThat(CalciteAssert.toString(resultSet), equalTo("N={'a'=1, 'baz'=2}\n" + "N=1 \n")); {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3413) AssertionError for interpertering union/intersect/minus with null value
Wang Yanlin created CALCITE-3413: Summary: AssertionError for interpertering union/intersect/minus with null value Key: CALCITE-3413 URL: https://issues.apache.org/jira/browse/CALCITE-3413 Project: Calcite Issue Type: New Feature Reporter: Wang Yanlin Add the following test case in *InterpreterTest* {code:java} @Test public void testInterpretUnionWithNullValue() throws Exception { final String sql = "select * from\n" + "(select x, y from (values (cast(NULL as int), cast(NULL as varchar(1))),\n" + "(cast(NULL as int), cast(NULL as varchar(1 as t(x, y))\n" + "union\n" + "(select x, y from (values (cast(NULL as int), cast(NULL as varchar(1 as t2(x, y))\n"; SqlNode validate = planner.validate(planner.parse(sql)); RelNode convert = planner.rel(validate).rel; final Interpreter interpreter = new Interpreter(dataContext, convert); assertRows(interpreter, "[null, null]"); } @Test public void testInterpretUnionAllWithNullValue() throws Exception { final String sql = "select * from\n" + "(select x, y from (values (cast(NULL as int), cast(NULL as varchar(1))),\n" + "(cast(NULL as int), cast(NULL as varchar(1 as t(x, y))\n" + "union all\n" + "(select x, y from (values (cast(NULL as int), cast(NULL as varchar(1 as t2(x, y))\n"; SqlNode validate = planner.validate(planner.parse(sql)); RelNode convert = planner.rel(validate).rel; final Interpreter interpreter = new Interpreter(dataContext, convert); assertRows(interpreter, "[null, null]", "[null, null]", "[null, null]"); } {code} got {code:java} java.lang.AssertionError at org.apache.calcite.interpreter.Interpreter$CompilerImpl.source(Interpreter.java:510) at org.apache.calcite.interpreter.Nodes$CoreCompiler.source(Nodes.java:46) at org.apache.calcite.interpreter.AbstractSingleNode.(AbstractSingleNode.java:33) at org.apache.calcite.interpreter.ProjectNode.(ProjectNode.java:31) at org.apache.calcite.interpreter.Nodes$CoreCompiler.visit(Nodes.java:60) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.calcite.util.ReflectUtil.invokeVisitorInternal(ReflectUtil.java:257) at org.apache.calcite.util.ReflectUtil.invokeVisitor(ReflectUtil.java:214) at org.apache.calcite.util.ReflectUtil$1.invokeVisitor(ReflectUtil.java:464) at org.apache.calcite.interpreter.Interpreter$CompilerImpl.visit(Interpreter.java:451) at org.apache.calcite.interpreter.Nodes$CoreCompiler.visit(Nodes.java:46) at org.apache.calcite.rel.AbstractRelNode.childrenAccept(AbstractRelNode.java:265) at org.apache.calcite.interpreter.Interpreter$CompilerImpl.visit(Interpreter.java:447) at org.apache.calcite.interpreter.Nodes$CoreCompiler.visit(Nodes.java:46) at org.apache.calcite.interpreter.Interpreter$CompilerImpl.visitRoot(Interpreter.java:405) at org.apache.calcite.interpreter.Interpreter.(Interpreter.java:88) at org.apache.calcite.test.InterpreterTest.testInterpretMinusWithNullValue(InterpreterTest.java:418) {code} The cause lies in the optimize of [Interpreter|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/interpreter/Interpreter.java#L85] before optimize, the relnode tree is as follows, no renode with same id exists {code:java} LogicalUnion(all=[false]), id = 19 LogicalProject(X=[$0], Y=[$1]), id = 16 LogicalUnion(all=[true]), id = 15 LogicalProject(EXPR$0=[null:INTEGER], EXPR$1=[null:VARCHAR(1)]), id = 12 LogicalValues(tuples=[[{ 0 }]]), id = 11 LogicalProject(EXPR$0=[null:INTEGER], EXPR$1=[null:VARCHAR(1)]), id = 14 LogicalValues(tuples=[[{ 0 }]]), id = 13 LogicalProject(X=[null:INTEGER], Y=[null:VARCHAR(1)]), id = 18 LogicalValues(tuples=[[{ 0 }]]), id = 17 {code} after optimize, the relnode tree is as follows, {code:java} LogicalUnion(all=[false]), id = 30 LogicalProject(X=[$0], Y=[$1]), id = 26 LogicalUnion(all=[true]), id = 24 LogicalProject(EXPR$0=[null:INTEGER], EXPR$1=[null:VARCHAR(1)]), id = 21 LogicalValues(tuples=[[{ 0 }]]), id = 11 LogicalProject(EXPR$0=[null:INTEGER], EXPR$1=[null:VARCHAR(1)]), id = 21 LogicalValues(tuples=[[{ 0 }]]), id = 11 LogicalProject(X=[null:INTEGER], Y=[null:VARCHAR(1)]), id = 28 LogicalValues(tuples=[[{ 0 }]]), id = 11 {code} there exists rel
[jira] [Created] (CALCITE-3408) Add support for Enumerable Intersect/Minus all
Wang Yanlin created CALCITE-3408: Summary: Add support for Enumerable Intersect/Minus all Key: CALCITE-3408 URL: https://issues.apache.org/jira/browse/CALCITE-3408 Project: Calcite Issue Type: New Feature Reporter: Wang Yanlin When running sql with calcite driver {code:java} final String sql = "select * from\n" + "(select x, y from (values (1, 'a'), (2, 'b'), (2, 'b'), (3, 'c')) as t(x, y))\n" + "except all\n" + "(select x, y from (values (1, 'a'), (2, 'c'), (4, 'x')) as t2(x, y))\n"; {code} got {code:java} java.sql.SQLException: Error while executing SQL "explain plan for select * from (select x, y from (values (1, 'a'), (2, 'b'), (2, 'b'), (3, 'c')) as t(x, y)) except all (select x, y from (values (1, 'a'), (2, 'c'), (4, 'x')) as t2(x, y)) ": There are not enough rules to produce a node with desired properties: convention=ENUMERABLE, sort=[]. Missing conversion is LogicalMinus[convention: NONE -> ENUMERABLE] There is 1 empty subset: rel#27:Subset#4.ENUMERABLE.[], the relevant part of the original plan is as follows 22:LogicalMinus(all=[true]) 1:LogicalValues(subset=[rel#16:Subset#0.NONE.[]], tuples=[[{ 1, 'a' }, { 2, 'b' }, { 2, 'b' }, { 3, 'c' }]]) 5:LogicalValues(subset=[rel#19:Subset#2.NONE.[]], tuples=[[{ 1, 'a' }, { 2, 'c' }, { 4, 'x' }]]) Root: rel#27:Subset#4.ENUMERABLE.[] Original rel: LogicalMinus(all=[true]): rowcount = 3.0, cumulative cost = {17.0 rows, 19.0 cpu, 0.0 io}, id = 14 LogicalProject(X=[$0], Y=[$1]): rowcount = 4.0, cumulative cost = {8.0 rows, 9.0 cpu, 0.0 io}, id = 9 LogicalValues(tuples=[[{ 1, 'a' }, { 2, 'b' }, { 2, 'b' }, { 3, 'c' }]]): rowcount = 4.0, cumulative cost = {4.0 rows, 1.0 cpu, 0.0 io}, id = 1 LogicalProject(X=[$0], Y=[$1]): rowcount = 3.0, cumulative cost = {6.0 rows, 7.0 cpu, 0.0 io}, id = 12 LogicalValues(tuples=[[{ 1, 'a' }, { 2, 'c' }, { 4, 'x' }]]): rowcount = 3.0, cumulative cost = {3.0 rows, 1.0 cpu, 0.0 io}, id = 5 {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3407) Add support for interpretering minus/intersect relational set operators
Wang Yanlin created CALCITE-3407: Summary: Add support for interpretering minus/intersect relational set operators Key: CALCITE-3407 URL: https://issues.apache.org/jira/browse/CALCITE-3407 Project: Calcite Issue Type: Improvement Reporter: Wang Yanlin Currently, for SetOp, only `union` is supported by Interpreter add the test cases in InterpreterTest, and run, they will fail by throwing exception {code:java} @Test public void testInterpretIntersect() throws Exception { final String sql = "select * from\n" + "(select x, y from (values (1, 'a'), (1, 'a'), (2, 'b'), (3, 'c')) as t(x, y))\n" + "intersect\n" + "(select x, y from (values (1, 'a'), (2, 'c'), (4, 'x')) as t2(x, y))\n"; SqlNode validate = planner.validate(planner.parse(sql)); RelNode convert = planner.rel(validate).rel; final Interpreter interpreter = new Interpreter(dataContext, convert); assertRows(interpreter, "[1, a]"); } @Test public void testInterpretIntersectAll() throws Exception { final String sql = "select * from\n" + "(select x, y from (values (1, 'a'), (1, 'a'), (2, 'b'), (3, 'c')) as t(x, y))\n" + "intersect all\n" + "(select x, y from (values (1, 'a'), (2, 'c'), (4, 'x')) as t2(x, y))\n"; SqlNode validate = planner.validate(planner.parse(sql)); RelNode convert = planner.rel(validate).rel; final Interpreter interpreter = new Interpreter(dataContext, convert); assertRows(interpreter, "[1, a]", "[1, a]"); } @Test public void testInterpretMinus() throws Exception { final String sql = "select * from\n" + "(select x, y from (values (1, 'a'), (2, 'b'), (2, 'b'), (3, 'c')) as t(x, y))\n" + "except\n" + "(select x, y from (values (1, 'a'), (2, 'c'), (4, 'x')) as t2(x, y))\n"; SqlNode validate = planner.validate(planner.parse(sql)); RelNode convert = planner.rel(validate).rel; final Interpreter interpreter = new Interpreter(dataContext, convert); assertRows(interpreter, "[2, b]", "[3, c]"); } @Test public void testInterpretMinusAll() throws Exception { final String sql = "select * from\n" + "(select x, y from (values (1, 'a'), (2, 'b'), (2, 'b'), (3, 'c')) as t(x, y))\n" + "except all\n" + "(select x, y from (values (1, 'a'), (2, 'c'), (4, 'x')) as t2(x, y))\n"; SqlNode validate = planner.validate(planner.parse(sql)); RelNode convert = planner.rel(validate).rel; final Interpreter interpreter = new Interpreter(dataContext, convert); assertRows(interpreter, "[2, b]", "[2, b]", "[3, c]"); } {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [QUESTION] Pushing up evaluations from LogicalProjects
OK,I got your point. Thanks for sharing. | | 王炎林 | | 邮箱:1989yanlinw...@163.com | 签名由 网易邮箱大师 定制 On 10/13/2019 06:17, Stamatis Zampetakis wrote: I was thinking that RelFieldTrimmer can be used to transform the plan P1 to plan P2 and then ProjectTableScanRule can be used to introduce the BindableTableScan. The consumer in the case of P1 is the project which only needs $0, $2, $5, $6 so the trimmer could slim down the scan by projecting only these fields. I didn't test if it really does this in this case because in some sense the trimming is a bit redundant given that the project just on top has just a subset of the columns of the table. Note that RelFieldTrimmer could be used also during optimization (not only in sql to rel phase) as a separate planning phase. *P1* LogicalProject(l_orderkey=[$0], l_suppkey=[$2], *=[*($5, -(1, $6))]) LogicalTableScan(table=[[main, lineitem]]) *P2* LogicalProject(l_orderkey=[$0], l_suppkey=[$1], *=[*($2, -(1, $3))]) LogicalProject(l_orderkey=[$0], l_suppkey=[$2], l_extendedprice=[$5], l_discount=[$6]]) LogicalTableScan(table=[[main, lineitem]]) On Sat, Oct 12, 2019 at 1:11 PM Wang Yanlin <1989yanlinw...@163.com> wrote: > Just a little curious, > RelFieldTrimmer is used to 'slimmed down' relational expression that > projects only the columns required by its consumer. > How can it used to do this thing? > > > > > -- > > Best, > Wang Yanlin > > > > 在 2019-10-12 18:03:33,"XING JIN" 写道: > >Sure we can ~ > >If we use BindableTableScanRule to derive a BindableTableScan from > >ProjectableFilterableTable, that would happen during a stage of > >optimization run by RelOptPlanner. But RelFieldTrimmer works right during > >conversion of Sql to Rel. > > > >Wang Yanlin <1989yanlinw...@163.com> 于2019年10月12日周六 下午5:56写道: > > > >> Can we just use Bindables.BINDABLE_TABLE_SCAN_RULE to translate the > >> table scan to BindableTableScan ? > >> > >> > >> > >> -- > >> > >> Best, > >> Wang Yanlin > >> > >> > >> > >> At 2019-10-12 17:12:20, "XING JIN" wrote: > >> >Hi Stamatis, > >> >In current code, BindableTableScan is only created by rules of > >> >ProjectTableScanRule and FilterTableScanRule. I think it's better to > keep > >> >as it is? > >> >I made a PR for CALCITE-3405 -- > >> https://github.com/apache/calcite/pull/1500 > >> > > >> >The idea of the PR is quite straightforward: > >> >1. Analyze the parent Project -- collect all the needed fields; > >> >2. Column pruning by pushing down the needed fields to > BindableTableScan; > >> >3. Adjust RexInputRefs in parent Project > >> > > >> >@Haisheng @Stamatis It would be great if you can give a review when you > >> >have time ~~~ Thanks a lot ! > >> > > >> >Best, > >> >Jin > >> > > >> > > >> >Stamatis Zampetakis 于2019年10月12日周六 下午3:00写道: > >> > > >> >> Hi Rommel, > >> >> > >> >> I was hoping that this could be done at least by RelFieldTrimmer [1]. > >> Are > >> >> you using it already? > >> >> > >> >> Best, > >> >> Stamatis > >> >> > >> >> [1] > >> >> > >> >> > >> > https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java > >> >> > >> >> On Sat, Oct 12, 2019 at 6:06 AM XING JIN > >> wrote: > >> >> > >> >> > Filed a JIRA: > >> >> > https://issues.apache.org/jira/browse/CALCITE-3405 > >> >> > > >> >> > Haisheng Yuan 于2019年10月12日周六 上午4:34写道: > >> >> > > >> >> > > Yes, definitely. > >> >> > > > >> >> > > You can go through the project expression with InputFinder to > find > >> all > >> >> > the > >> >> > > used columns, create a logical project with those columns, and > >> remap > >> >> the > >> >> > > top project with new column indexes. > >> >> > > > >> >> > > On the other hand, instead of creating a new intermidiate pogical > >> >> > project, > >> >> > > we can also update ProjectTableScanRule to accept LogicalProject > >> that > >> >> is > >> >> > >
Re:Re: Re: [QUESTION] Pushing up evaluations from LogicalProjects
Just a little curious, RelFieldTrimmer is used to 'slimmed down' relational expression that projects only the columns required by its consumer. How can it used to do this thing? -- Best, Wang Yanlin 在 2019-10-12 18:03:33,"XING JIN" 写道: >Sure we can ~ >If we use BindableTableScanRule to derive a BindableTableScan from >ProjectableFilterableTable, that would happen during a stage of >optimization run by RelOptPlanner. But RelFieldTrimmer works right during >conversion of Sql to Rel. > >Wang Yanlin <1989yanlinw...@163.com> 于2019年10月12日周六 下午5:56写道: > >> Can we just use Bindables.BINDABLE_TABLE_SCAN_RULE to translate the >> table scan to BindableTableScan ? >> >> >> >> -- >> >> Best, >> Wang Yanlin >> >> >> >> At 2019-10-12 17:12:20, "XING JIN" wrote: >> >Hi Stamatis, >> >In current code, BindableTableScan is only created by rules of >> >ProjectTableScanRule and FilterTableScanRule. I think it's better to keep >> >as it is? >> >I made a PR for CALCITE-3405 -- >> https://github.com/apache/calcite/pull/1500 >> > >> >The idea of the PR is quite straightforward: >> >1. Analyze the parent Project -- collect all the needed fields; >> >2. Column pruning by pushing down the needed fields to BindableTableScan; >> >3. Adjust RexInputRefs in parent Project >> > >> >@Haisheng @Stamatis It would be great if you can give a review when you >> >have time ~~~ Thanks a lot ! >> > >> >Best, >> >Jin >> > >> > >> >Stamatis Zampetakis 于2019年10月12日周六 下午3:00写道: >> > >> >> Hi Rommel, >> >> >> >> I was hoping that this could be done at least by RelFieldTrimmer [1]. >> Are >> >> you using it already? >> >> >> >> Best, >> >> Stamatis >> >> >> >> [1] >> >> >> >> >> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java >> >> >> >> On Sat, Oct 12, 2019 at 6:06 AM XING JIN >> wrote: >> >> >> >> > Filed a JIRA: >> >> > https://issues.apache.org/jira/browse/CALCITE-3405 >> >> > >> >> > Haisheng Yuan 于2019年10月12日周六 上午4:34写道: >> >> > >> >> > > Yes, definitely. >> >> > > >> >> > > You can go through the project expression with InputFinder to find >> all >> >> > the >> >> > > used columns, create a logical project with those columns, and >> remap >> >> the >> >> > > top project with new column indexes. >> >> > > >> >> > > On the other hand, instead of creating a new intermidiate pogical >> >> > project, >> >> > > we can also update ProjectTableScanRule to accept LogicalProject >> that >> >> is >> >> > > not a simple mapping, and do the same task I mentioned above. >> >> > > >> >> > > - Haisheng >> >> > > >> >> > > -- >> >> > > 发件人:Rommel Quintanilla >> >> > > 日 期:2019年10月12日 03:15:31 >> >> > > 收件人: >> >> > > 主 题:[QUESTION] Pushing up evaluations from LogicalProjects >> >> > > >> >> > > Hi, maybe you can help me. >> >> > > I have this portion from a larger logical plan: >> >> > > .. >> >> > > LogicalProject(l_orderkey=[$0], l_suppkey=[$2], *=[*($5, >> -(1, >> >> > > $6))]) >> >> > > LogicalTableScan(table=[[main, lineitem]]) >> >> > > .. >> >> > > >> >> > > Because the LogicalProject above contains an evaluation, the >> >> > > ProjectTableScanRule can't convert it to a BindableTableScan. >> >> > > >> >> > > I wonder if somehow the evaluation could be pushed up more or less >> like >> >> > > this: >> >> > > .. >> >> > > LogicalProject(l_orderkey=[$0], l_suppkey=[$1], *=[*($2, -(1, >> $3))]) >> >> > > LogicalProject(l_orderkey=[$0], l_suppkey=[$2], >> l_extendedprice=[$5], >> >> > > l_discount=[$6]]) >> >> > > LogicalTableScan(table=[[main, lineitem]]) >> >> > > .. >> >> > > >> >> > > Regards. >> >> > > >> >> > >> >> >>
Re:Re: [QUESTION] Pushing up evaluations from LogicalProjects
Can we just use Bindables.BINDABLE_TABLE_SCAN_RULE to translate the table scan to BindableTableScan ? -- Best, Wang Yanlin At 2019-10-12 17:12:20, "XING JIN" wrote: >Hi Stamatis, >In current code, BindableTableScan is only created by rules of >ProjectTableScanRule and FilterTableScanRule. I think it's better to keep >as it is? >I made a PR for CALCITE-3405 -- https://github.com/apache/calcite/pull/1500 > >The idea of the PR is quite straightforward: >1. Analyze the parent Project -- collect all the needed fields; >2. Column pruning by pushing down the needed fields to BindableTableScan; >3. Adjust RexInputRefs in parent Project > >@Haisheng @Stamatis It would be great if you can give a review when you >have time ~~~ Thanks a lot ! > >Best, >Jin > > >Stamatis Zampetakis 于2019年10月12日周六 下午3:00写道: > >> Hi Rommel, >> >> I was hoping that this could be done at least by RelFieldTrimmer [1]. Are >> you using it already? >> >> Best, >> Stamatis >> >> [1] >> >> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java >> >> On Sat, Oct 12, 2019 at 6:06 AM XING JIN wrote: >> >> > Filed a JIRA: >> > https://issues.apache.org/jira/browse/CALCITE-3405 >> > >> > Haisheng Yuan 于2019年10月12日周六 上午4:34写道: >> > >> > > Yes, definitely. >> > > >> > > You can go through the project expression with InputFinder to find all >> > the >> > > used columns, create a logical project with those columns, and remap >> the >> > > top project with new column indexes. >> > > >> > > On the other hand, instead of creating a new intermidiate pogical >> > project, >> > > we can also update ProjectTableScanRule to accept LogicalProject that >> is >> > > not a simple mapping, and do the same task I mentioned above. >> > > >> > > - Haisheng >> > > >> > > -- >> > > 发件人:Rommel Quintanilla >> > > 日 期:2019年10月12日 03:15:31 >> > > 收件人: >> > > 主 题:[QUESTION] Pushing up evaluations from LogicalProjects >> > > >> > > Hi, maybe you can help me. >> > > I have this portion from a larger logical plan: >> > > .. >> > > LogicalProject(l_orderkey=[$0], l_suppkey=[$2], *=[*($5, -(1, >> > > $6))]) >> > > LogicalTableScan(table=[[main, lineitem]]) >> > > .. >> > > >> > > Because the LogicalProject above contains an evaluation, the >> > > ProjectTableScanRule can't convert it to a BindableTableScan. >> > > >> > > I wonder if somehow the evaluation could be pushed up more or less like >> > > this: >> > > .. >> > > LogicalProject(l_orderkey=[$0], l_suppkey=[$1], *=[*($2, -(1, $3))]) >> > > LogicalProject(l_orderkey=[$0], l_suppkey=[$2], l_extendedprice=[$5], >> > > l_discount=[$6]]) >> > > LogicalTableScan(table=[[main, lineitem]]) >> > > .. >> > > >> > > Regards. >> > > >> > >>
[jira] [Created] (CALCITE-3400) Add support for interpretering outer/semi/anti/full join
Wang Yanlin created CALCITE-3400: Summary: Add support for interpretering outer/semi/anti/full join Key: CALCITE-3400 URL: https://issues.apache.org/jira/browse/CALCITE-3400 Project: Calcite Issue Type: Improvement Reporter: Wang Yanlin Currently, [JoinNode|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/interpreter/JoinNode.java#L49] can just run inner type join. We can add support for more join types for JoinNode. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3397) AssertionError for interpreter multiset
Wang Yanlin created CALCITE-3397: Summary: AssertionError for interpreter multiset Key: CALCITE-3397 URL: https://issues.apache.org/jira/browse/CALCITE-3397 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin when interpretering sql got, {code:java} java.lang.AssertionError: interpreter: no implementation for class org.apache.calcite.rel.core.Collect at org.apache.calcite.interpreter.Interpreter$CompilerImpl.visit(Interpreter.java:460) at org.apache.calcite.interpreter.Nodes$CoreCompiler.visit(Nodes.java:43) at org.apache.calcite.rel.BiRel.childrenAccept(BiRel.java:46) at org.apache.calcite.interpreter.Interpreter$CompilerImpl.visit(Interpreter.java:447) at org.apache.calcite.interpreter.Nodes$CoreCompiler.visit(Nodes.java:43) at org.apache.calcite.rel.SingleRel.childrenAccept(SingleRel.java:72) at org.apache.calcite.interpreter.Interpreter$CompilerImpl.visit(Interpreter.java:447) at org.apache.calcite.interpreter.Nodes$CoreCompiler.visit(Nodes.java:43) at org.apache.calcite.interpreter.Interpreter$CompilerImpl.visitRoot(Interpreter.java:405) at org.apache.calcite.interpreter.Interpreter.(Interpreter.java:88) at org.apache.calcite.test.InterpreterTest.testInterpretMultiset(InterpreterTest.java:127) {code} Reproduce this with test case {code:java} @Test public void testInterpretMultiset() throws Exception { final String sql = "select multiset['a', 'b', 'c']"; SqlNode parse = planner.parse(sql); SqlNode validate = planner.validate(parse); RelNode convert = planner.rel(validate).project(); final Interpreter interpreter = new Interpreter(dataContext, convert); assertRows(interpreter, "[[a, b, c]]"); } {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3388) StackOverflowError for creating structured RelDataType from class type
Wang Yanlin created CALCITE-3388: Summary: StackOverflowError for creating structured RelDataType from class type Key: CALCITE-3388 URL: https://issues.apache.org/jira/browse/CALCITE-3388 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin When creating a structured RelDataType from a java type with recursion reference, StackOverflowError occurs, full stack trace {noformat} java.lang.StackOverflowError at java.lang.Class.privateGetPublicFields(Class.java:2600) at java.lang.Class.getFields(Class.java:1557) at org.apache.calcite.rel.type.RelDataTypeFactoryImpl.fieldsOf(RelDataTypeFactoryImpl.java:440) at org.apache.calcite.rel.type.RelDataTypeFactoryImpl.access$700(RelDataTypeFactoryImpl.java:50) at org.apache.calcite.rel.type.RelDataTypeFactoryImpl$JavaType.(RelDataTypeFactoryImpl.java:568) at org.apache.calcite.rel.type.RelDataTypeFactoryImpl$JavaType.(RelDataTypeFactoryImpl.java:560) at org.apache.calcite.rel.type.RelDataTypeFactoryImpl$JavaType.(RelDataTypeFactoryImpl.java:554) at org.apache.calcite.rel.type.RelDataTypeFactoryImpl.createJavaType(RelDataTypeFactoryImpl.java:120) at org.apache.calcite.jdbc.JavaTypeFactoryImpl.createType(JavaTypeFactoryImpl.java:143) at org.apache.calcite.jdbc.JavaTypeFactoryImpl.createStructType(JavaTypeFactoryImpl.java:82) at org.apache.calcite.jdbc.JavaTypeFactoryImpl.createType(JavaTypeFactoryImpl.java:158) at org.apache.calcite.jdbc.JavaTypeFactoryImpl.createStructType(JavaTypeFactoryImpl.java:82) at org.apache.calcite.jdbc.JavaTypeFactoryImpl.createType(JavaTypeFactoryImpl.java:158) at org.apache.calcite.jdbc.JavaTypeFactoryImpl.createStructType(JavaTypeFactoryImpl.java:82) at org.apache.calcite.jdbc.JavaTypeFactoryImpl.createType(JavaTypeFactoryImpl.java:158) at org.apache.calcite.jdbc.JavaTypeFactoryImpl.createStructType(JavaTypeFactoryImpl.java:82) {noformat} Add the test case in JavaTypeFactoryTest to reproduce the exception {noformat} /***/ private static class RecursionStruct { public Integer intField; public RecursionStruct next; } /***/ private static class RecursionStruct1 { public Integer intField; public RecursionStruct2 struct2; } /***/ private static class RecursionStruct2 { public Integer intField; public RecursionStruct1 struct1; } @Test public void testRecursion() { TYPE_FACTORY.createStructType(RecursionStruct.class); TYPE_FACTORY.createStructType(RecursionStruct1.class); } {noformat} Better to check for recursion and throw an exception explicitly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3378) AssertionError for checking RexNode implify
Wang Yanlin created CALCITE-3378: Summary: AssertionError for checking RexNode implify Key: CALCITE-3378 URL: https://issues.apache.org/jira/browse/CALCITE-3378 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin When checking implify for RexNode with CAST, get AssertionError. The test case in *RexImplicationCheckerTest* is like this {code:java} @Test public void testRexImplifyWithCast() { final Fixture f = new Fixture(); final RexNode left = f.rexBuilder.makeCall( SqlStdOperatorTable.AND, f.rexBuilder.makeCall( SqlStdOperatorTable.EQUALS, f.str, f.cast( f.stringDataType, f.literal(1))), f.rexBuilder.makeCall( SqlStdOperatorTable.EQUALS, f.i, f.literal(1))); final RexNode right = f.rexBuilder.makeCall( SqlStdOperatorTable.EQUALS, f.i, f.literal(1)); f.checkImplies(left, right); } {code} got exception as below {code:java} java.lang.AssertionError: cannot convert DECIMAL literal to class java.lang.String at org.apache.calcite.rex.RexLiteral.getValueAs(RexLiteral.java:1067) at org.apache.calcite.plan.VisitorDataContext.getValue(VisitorDataContext.java:147) at org.apache.calcite.plan.VisitorDataContext.of(VisitorDataContext.java:97) at org.apache.calcite.plan.RexImplicationChecker.implies2(RexImplicationChecker.java:236) at org.apache.calcite.plan.RexImplicationChecker.impliesConjunction(RexImplicationChecker.java:148) at org.apache.calcite.plan.RexImplicationChecker.impliesAny(RexImplicationChecker.java:138) at org.apache.calcite.plan.RexImplicationChecker.implies(RexImplicationChecker.java:124) at org.apache.calcite.test.RexImplicationCheckerTest$Fixture.checkImplies(RexImplicationCheckerTest.java:658) at org.apache.calcite.test.RexImplicationCheckerTest.testRexImplifyWithCast(RexImplicationCheckerTest.java:467) {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3374) Error format check result for explain plan as json
Wang Yanlin created CALCITE-3374: Summary: Error format check result for explain plan as json Key: CALCITE-3374 URL: https://issues.apache.org/jira/browse/CALCITE-3374 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin The format check result of explain plan as json is incorrect, as {code:java} public boolean isJson() { return getFormat() == SqlExplainFormat.XML; } {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3362) Implementation for some empty tests of Lattice
Wang Yanlin created CALCITE-3362: Summary: Implementation for some empty tests of Lattice Key: CALCITE-3362 URL: https://issues.apache.org/jira/browse/CALCITE-3362 Project: Calcite Issue Type: Test Reporter: Wang Yanlin When read the code of lattice framework, came across some empty unit tests in [LatticeTest|https://github.com/apache/calcite/blob/67fd318ed755ef975cf31262c96c982f0922a975/core/src/test/java/org/apache/calcite/test/LatticeTest.java#L720]. Trying to implement these empty test cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3350) Keep origin type for RexLiteral when deserialized from json string
Wang Yanlin created CALCITE-3350: Summary: Keep origin type for RexLiteral when deserialized from json string Key: CALCITE-3350 URL: https://issues.apache.org/jira/browse/CALCITE-3350 Project: Calcite Issue Type: Improvement Reporter: Wang Yanlin The json string of sql {noformat} select ename from emp where job = "abd" {noformat} is {noformat} { "rels": [ { "id": "0", "relOp": "LogicalTableScan", "table": [ "scott", "EMP" ], "inputs": [] }, { "id": "1", "relOp": "LogicalFilter", "condition": { "op": { "name": "=", "kind": "EQUALS", "syntax": "BINARY" }, "operands": [ { "input": 2, "name": "$2" }, { "literal": "abd", "type": { "type": "VARCHAR", "nullable": false, "precision": 10 } } ] } }, { "id": "2", "relOp": "LogicalProject", "fields": [ "ENAME" ], "exprs": [ { "input": 1, "name": "$1" } ] } ] } {noformat} The original type of literal "abd" is {noformat} VARCHAR(10) NOT NULL {noformat} When deserialized relnode from this json string, the type of literal "abd" is changed to {noformat} CHAR(3) NOT NULL {noformat} Better to keep the same with original type when deserialized from string. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3348) AssertionError while determining distribution of Calc
Wang Yanlin created CALCITE-3348: Summary: AssertionError while determining distribution of Calc Key: CALCITE-3348 URL: https://issues.apache.org/jira/browse/CALCITE-3348 Project: Calcite Issue Type: Improvement Components: core Reporter: Wang Yanlin When using *EnumerableCalcRule* with *RelDistributionTraitDef* for planner, got {code:java} java.lang.AssertionError at org.apache.calcite.rel.metadata.RelMdDistribution.calc(RelMdDistribution.java:144) at org.apache.calcite.rel.logical.LogicalCalc.lambda$create$1(LogicalCalc.java:108) at org.apache.calcite.plan.RelTraitSet.replaceIf(RelTraitSet.java:249) at org.apache.calcite.rel.logical.LogicalCalc.create(LogicalCalc.java:107) at org.apache.calcite.rel.rules.FilterToCalcRule.onMatch(FilterToCalcRule.java:77) at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:208) at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:631) at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:327) at org.apache.calcite.test.RelOptRulesTest.testEnumerableCalcRule(RelOptRulesTest.java:6346) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) {code} -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3346) able some ignored tests in RelOptRuleTests
Wang Yanlin created CALCITE-3346: Summary: able some ignored tests in RelOptRuleTests Key: CALCITE-3346 URL: https://issues.apache.org/jira/browse/CALCITE-3346 Project: Calcite Issue Type: Improvement Reporter: Wang Yanlin When reading the code, i found that some ignored test case in *RelOptRulesTest* actually can pass the test. Maybe we should try to enable these tests. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3319) AssertionError for ReduceDecimalsRule
Wang Yanlin created CALCITE-3319: Summary: AssertionError for ReduceDecimalsRule Key: CALCITE-3319 URL: https://issues.apache.org/jira/browse/CALCITE-3319 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin When trying to use *ReduceDecimalsRule*, I got {code:java} java.lang.AssertionError at org.apache.calcite.rel.rules.ReduceDecimalsRule$DecimalShuttle.visitCall(ReduceDecimalsRule.java:159) at org.apache.calcite.rel.rules.ReduceDecimalsRule$DecimalShuttle.visitCall(ReduceDecimalsRule.java:124) at org.apache.calcite.rex.RexCall.accept(RexCall.java:191) at org.apache.calcite.rex.RexProgramBuilder.add(RexProgramBuilder.java:653) at org.apache.calcite.rex.RexProgramBuilder.create(RexProgramBuilder.java:601) at org.apache.calcite.rel.rules.ReduceDecimalsRule.onMatch(ReduceDecimalsRule.java:103) at org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:319) at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:560) at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:419) at org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:256) at org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:127) at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:215) at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:202) {code} I read the code, and found this. {code:java} List newOperands = apply(call.getOperands()); if (true) { // FIXME: Operands are now immutable. Create a new call with // new operands? throw new AssertionError(); } {code} After remove this logic, the rel below {code:java} LogicalCalc(expr#0..7=[{inputs}], expr#8=[1.01E1:DOUBLE], expr#9=[15], expr#10=[+($t8, $t9)], expr#11=[>($t5, $t10)], proj#0..7=[{exprs}], $condition=[$t11]) LogicalTableScan(table=[[scott, EMP]]) {code} can be translated into {code:java} LogicalCalc(expr#0..7=[{inputs}], expr#8=[Reinterpret($t5)], expr#9=[CAST($t8):DOUBLE], expr#10=[1E2:DOUBLE], expr#11=[/INT($t9, $t10)], expr#12=[1.01E1:DOUBLE], expr#13=[15], expr#14=[+($t12, $t13)], expr#15=[>($t11, $t14)], proj#0..7=[{exprs}], $condition=[$t15]) LogicalTableScan(table=[[scott, EMP]]) {code} So is this rule not ready for use now, or we should just remove this logic? -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3317) Add public constructor for creating LogicalCalc with RelInput type parameter
Wang Yanlin created CALCITE-3317: Summary: Add public constructor for creating LogicalCalc with RelInput type parameter Key: CALCITE-3317 URL: https://issues.apache.org/jira/browse/CALCITE-3317 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin When trying to deserializing a RelNode contains a LogicalCalc node in the tree, got {code:java} java.lang.RuntimeException: java.lang.RuntimeException: class does not have required constructor, class org.apache.calcite.rel.logical.LogicalCalc(RelInput) at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:181) at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:125) at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:143) at org.apache.calcite.plan.RelWriterTest.testCalc(RelWriterTest.java:688) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) Caused by: java.lang.RuntimeException: class does not have required constructor, class org.apache.calcite.rel.logical.LogicalCalc(RelInput) at org.apache.calcite.rel.externalize.RelJson.getConstructor(RelJson.java:114) at org.apache.calcite.rel.externalize.RelJsonReader.readRel(RelJsonReader.java:98) at org.apache.calcite.rel.externalize.RelJsonReader.readRels(RelJsonReader.java:91) at org.apache.calcite.rel.externalize.RelJsonReader.read(RelJsonReader.java:85) at org.apache.calcite.plan.RelWriterTest.lambda$testCalc$9(RelWriterTest.java:693) at org.apache.calcite.tools.Frameworks.lambda$withPlanner$0(Frameworks.java:130) at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:915) at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:179) ... 25 more {code} Need to add a public constructor for creating LogicalCalc with RelInput type parameter, like this {code:java} public LogicalCalc(RelInput input) {code} -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3316) Exception while deserializing LogicalCorrelate from json string
Wang Yanlin created CALCITE-3316: Summary: Exception while deserializing LogicalCorrelate from json string Key: CALCITE-3316 URL: https://issues.apache.org/jira/browse/CALCITE-3316 Project: Calcite Issue Type: Bug Components: core Reporter: Wang Yanlin When deserializing LogicalCorrelate, we get {code:java} java.lang.RuntimeException: java.lang.NullPointerException at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:181) at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:125) at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:143) at org.apache.calcite.plan.RelWriterTest.testCorrelateQuery(RelWriterTest.java:686) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) Caused by: java.lang.NullPointerException at org.apache.calcite.rel.logical.LogicalCorrelate.(LogicalCorrelate.java:83) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.calcite.rel.externalize.RelJsonReader.readRel(RelJsonReader.java:261) at org.apache.calcite.rel.externalize.RelJsonReader.readRels(RelJsonReader.java:91) at org.apache.calcite.rel.externalize.RelJsonReader.read(RelJsonReader.java:85) at org.apache.calcite.plan.RelWriterTest.lambda$testCorrelateQuery$9(RelWriterTest.java:691) at org.apache.calcite.tools.Frameworks.lambda$withPlanner$0(Frameworks.java:130) at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:915) at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:179) ... 25 more {code} And, after fix the NPE, get {code:java} java.lang.RuntimeException: java.lang.ClassCastException: java.lang.String cannot be cast to java.util.List at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:181) at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:125) at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:143) at org.apache.calcite.plan.RelWriterTest.testCorrelateQuery(RelWriterTest.java:686) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at
[jira] [Created] (CALCITE-3313) AssertionError for an invalid type when using REGEXP_REPLACE
Wang Yanlin created CALCITE-3313: Summary: AssertionError for an invalid type when using REGEXP_REPLACE Key: CALCITE-3313 URL: https://issues.apache.org/jira/browse/CALCITE-3313 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin When using REGEXP_REPLACE function with an invalid type parameter, like this {code:sql} select regexp_replace(12, 'b', 'X', 1, 3, 'i') {code} we got {code:java} java.lang.AssertionError: If you see this, assign operandTypeChecker a value or override this function at org.apache.calcite.sql.SqlOperator.getAllowedSignatures(SqlOperator.java:730) at org.apache.calcite.sql.SqlOperator.getAllowedSignatures(SqlOperator.java:721) at org.apache.calcite.sql.SqlCallBinding.newValidationSignatureError(SqlCallBinding.java:283) at org.apache.calcite.sql.type.FamilyOperandTypeChecker.checkSingleOperandType(FamilyOperandTypeChecker.java:96) at org.apache.calcite.sql.fun.SqlRegexpReplaceFunction.checkOperandTypes(SqlRegexpReplaceFunction.java:56) at org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:432) at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:298) at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:216) at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5626) at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5613) at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139) at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1688) at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1673) at org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:476) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4104) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3392) at org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60) at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1005) at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:965) at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216) {code} Better to give a more detailed message of the allowed signatures. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3306) Add REGEXP_SPLIT_TO_ARRAY function
Wang Yanlin created CALCITE-3306: Summary: Add REGEXP_SPLIT_TO_ARRAY function Key: CALCITE-3306 URL: https://issues.apache.org/jira/browse/CALCITE-3306 Project: Calcite Issue Type: New Feature Reporter: Wang Yanlin In work, I found there is no coresponding *split* function in calcite like the * regexp_split_to_array* in [PostgreSQL|https://www.postgresql.org/docs/9.1/functions-matching.html]. Although we can solve this by creating a udf based on calcite, but a built in function might be better. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3295) Aggregate call name lost in serialized json string.
Wang Yanlin created CALCITE-3295: Summary: Aggregate call name lost in serialized json string. Key: CALCITE-3295 URL: https://issues.apache.org/jira/browse/CALCITE-3295 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin When serialize a relnode to json string, the name of aggregate call was lost. For a sql string like this {code:java} SELECT max(SAL) as max_sal from EMP group by JOB; {code} The serialized json string is like this {code:java} { "rels": [ { "id": "0", "relOp": "LogicalTableScan", "table": [ "scott", "EMP" ], "inputs": [] }, { "id": "1", "relOp": "LogicalProject", "fields": [ "JOB", "SAL" ], "exprs": [ { "input": 2, "name": "$2" }, { "input": 5, "name": "$5" } ] }, { "id": "2", "relOp": "LogicalAggregate", "group": [ 0 ], "aggs": [ { "agg": { "name": "MAX", "kind": "MAX", "syntax": "FUNCTION" }, "type": { "type": "DECIMAL", "nullable": true, "precision": 7, "scale": 2 }, "distinct": false, "operands": [ 1 ], } ] }, { "id": "3", "relOp": "LogicalProject", "fields": [ "max_sal" ], "exprs": [ { "input": 1, "name": "$1" } ] } ] } {code} Lost the name of max call. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3281) Support mixed Primitive types for BinaryExpression evaluate method.
Wang Yanlin created CALCITE-3281: Summary: Support mixed Primitive types for BinaryExpression evaluate method. Key: CALCITE-3281 URL: https://issues.apache.org/jira/browse/CALCITE-3281 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin {code:java} // code placeholder {code} Currently, the value of [expression0 |http://www.baidu.com/] and [expression1|] should be -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3278) Simplify the use to translate RexNode to Expression for evaluating
Wang Yanlin created CALCITE-3278: Summary: Simplify the use to translate RexNode to Expression for evaluating Key: CALCITE-3278 URL: https://issues.apache.org/jira/browse/CALCITE-3278 Project: Calcite Issue Type: Improvement Components: core Reporter: Wang Yanlin The method *forAggregation* of *RexToLixTranslator* is designed to work for translating aggregate functions, with some parameters that we actually do not need, if we just want to translate a single RexNode. We lack a more common sense function to get an instance of RexToLixTranslator. And, the translated expression is a *ParameterExpression*, not fit for evaluating. When evaluating, we get an exception like this {code:java} java.lang.RuntimeException: parameter v not on stack at org.apache.calcite.linq4j.tree.Evaluator.peek(Evaluator.java:51) at org.apache.calcite.linq4j.tree.ParameterExpression.evaluate(ParameterExpression.java:55) at org.apache.calcite.linq4j.tree.GotoStatement.evaluate(GotoStatement.java:97) at org.apache.calcite.linq4j.tree.BlockStatement.evaluate(BlockStatement.java:83) at org.apache.calcite.linq4j.tree.Evaluator.evaluate(Evaluator.java:55) at org.apache.calcite.linq4j.tree.FunctionExpression.lambda$compile$0(FunctionExpression.java:87) at org.apache.calcite.adapter.enumerable.RexToLixTranslatorTest.testRawTranslateRexNode(RexToLixTranslatorTest.java:57) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) {code} -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3260) Add support of evaluate with default Evaluator.
Wang Yanlin created CALCITE-3260: Summary: Add support of evaluate with default Evaluator. Key: CALCITE-3260 URL: https://issues.apache.org/jira/browse/CALCITE-3260 Project: Calcite Issue Type: New Feature Reporter: Wang Yanlin Currently, the public method *evaluate* of AbstractNode need a Evaluator object as parameter, but Evaluator class has default access control. This limit the access of *evaluate* method. So may be we can add an overload *evaluate* method with default Evaluator, thus, allow the evaluate Expression directly. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (CALCITE-3259) Align 'Property' in the serialized xml string of RelXmlWriter.
Wang Yanlin created CALCITE-3259: Summary: Align 'Property' in the serialized xml string of RelXmlWriter. Key: CALCITE-3259 URL: https://issues.apache.org/jira/browse/CALCITE-3259 Project: Calcite Issue Type: Improvement Reporter: Wang Yanlin In the serialized xml string of RelXmlWriter, the 'Property' label is aligned. For the sql below {noformat} select 1 + 2, 3 from (values (true)) {noformat} the serialized xml string is like this: {noformat} +(1, 2) 3 [{ true }] {noformat} It's better to make 'Property' aligned. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (CALCITE-3254) AssertionError while deserializing of interval type.
Wang Yanlin created CALCITE-3254: Summary: AssertionError while deserializing of interval type. Key: CALCITE-3254 URL: https://issues.apache.org/jira/browse/CALCITE-3254 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin When deserializing of logical rel haveing interval type, AssertionError occurs. The exception stacktrace as follow: {code:java} java.lang.RuntimeException: java.lang.RuntimeException: java.lang.AssertionError: use createSqlIntervalType() instead at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:181) at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:125) at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:143) at org.apache.calcite.plan.RelWriterTest.testInterval(RelWriterTest.java:639) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) Caused by: java.lang.RuntimeException: java.lang.AssertionError: use createSqlIntervalType() instead at org.apache.calcite.rel.externalize.RelJsonReader.readRel(RelJsonReader.java:271) at org.apache.calcite.rel.externalize.RelJsonReader.readRels(RelJsonReader.java:91) at org.apache.calcite.rel.externalize.RelJsonReader.read(RelJsonReader.java:85) at org.apache.calcite.plan.RelWriterTest.lambda$testInterval$8(RelWriterTest.java:644) at org.apache.calcite.tools.Frameworks.lambda$withPlanner$0(Frameworks.java:130) at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:915) at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:179) ... 25 more Caused by: java.lang.AssertionError: use createSqlIntervalType() instead at org.apache.calcite.sql.type.SqlTypeFactoryImpl.assertBasic(SqlTypeFactoryImpl.java:222) at org.apache.calcite.sql.type.SqlTypeFactoryImpl.createSqlType(SqlTypeFactoryImpl.java:75) at org.apache.calcite.rel.externalize.RelJson.toType(RelJson.java:222) at org.apache.calcite.rel.externalize.RelJson.toRex(RelJson.java:516) at org.apache.calcite.rel.externalize.RelJson.toRexList(RelJson.java:607) at org.apache.calcite.rel.externalize.RelJson.toRex(RelJson.java:447) at org.apache.calcite.rel.externalize.RelJsonReader$2.getExpressionList(RelJsonReader.java:204) at org.apache.calcite.rel.core.Project.(Project.java:100) at org.apache.calcite.rel.logical.LogicalProject.(LogicalProject.java:88) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.calcite.rel.externalize.RelJsonReader.readRel(RelJsonReader.java:261) ... 31 more {code} -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (CALCITE-3246) NullPointerException while deserializing of udf.
Wang Yanlin created CALCITE-3246: Summary: NullPointerException while deserializing of udf. Key: CALCITE-3246 URL: https://issues.apache.org/jira/browse/CALCITE-3246 Project: Calcite Issue Type: Bug Components: core Reporter: Wang Yanlin when deserializing of logical rel with udf operator, NPE occurs. The exception stacktrace as follow. {code:java} java.lang.RuntimeException: java.lang.NullPointerException at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:181) at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:125) at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:143) at org.apache.calcite.plan.RelWriterTest.testUdf(RelWriterTest.java:598) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) Caused by: java.lang.NullPointerException at java.util.Objects.requireNonNull(Objects.java:203) at org.apache.calcite.rex.RexCall.(RexCall.java:83) at org.apache.calcite.rex.RexBuilder.makeCall(RexBuilder.java:237) at org.apache.calcite.rel.externalize.RelJson.toRex(RelJson.java:485) at org.apache.calcite.rel.externalize.RelJsonReader$2.getExpressionList(RelJsonReader.java:204) at org.apache.calcite.rel.core.Project.(Project.java:100) at org.apache.calcite.rel.logical.LogicalProject.(LogicalProject.java:88) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.calcite.rel.externalize.RelJsonReader.readRel(RelJsonReader.java:261) at org.apache.calcite.rel.externalize.RelJsonReader.readRels(RelJsonReader.java:91) at org.apache.calcite.rel.externalize.RelJsonReader.read(RelJsonReader.java:85) at org.apache.calcite.plan.RelWriterTest.lambda$testUdf$7(RelWriterTest.java:603) at org.apache.calcite.tools.Frameworks.lambda$withPlanner$0(Frameworks.java:130) at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:915) at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:179) ... 25 more {code} -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (CALCITE-3177) Deserialize from json of + operator cause exception
Wang Yanlin created CALCITE-3177: Summary: Deserialize from json of + operator cause exception Key: CALCITE-3177 URL: https://issues.apache.org/jira/browse/CALCITE-3177 Project: Calcite Issue Type: Bug Reporter: Wang Yanlin Attachments: error_info.jpg For a simple sql string "select SAL + 10 FROM SALES.EMP", parse it to RelNode, serialize it to json string works well, but deserialize from the json string cause exception. The error stack is in the error_info.jpg picture. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-3175) Serialize to json of trim function cause exception
Wang Yanlin created CALCITE-3175: Summary: Serialize to json of trim function cause exception Key: CALCITE-3175 URL: https://issues.apache.org/jira/browse/CALCITE-3175 Project: Calcite Issue Type: Bug Components: core Reporter: Wang Yanlin Serialize the RexCall of *trim* function cause *java.lang.AssertionError* exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005)