Re: Calcite - Model - MongoDB view

[Andrei Sereda]

Hi Neha Saini,

I'm afraid currently array access is not translated in mongo adapter.

There is a PR https://github.com/apache/calcite/pull/720 which needs to be
corrected.

Thanks,
Andrei.

On Mon, Oct 14, 2019 at 3:01 AM Danny Chan  wrote:

> Hi, Neha Saini
>
> What do you mean by MongoDB view, can you give more details, so we can
> give as much help if we can.
>
> Best,
> Danny Chan
> 在 2019年10月14日 +0800 PM2:21，Neha Saini ，写道：
> > Hi Team,
> >
> > Gentle reminder!
> >
> > Regards,
> > Neha Saini
> >
> > From: Neha Saini
> > Sent: Thursday, October 10, 2019 3:30 PM
> > To: dev@calcite.apache.org
> > Subject: Calcite - Model - MongoDB view
> >
> > Hi Team,
> >
> > Greetings for the day.
> >
> > I am working as a technical architect in HCL technologies.
> > I am doing a PoC on Apache calcite.
> >
> > Using MongoDB adapter I want to use calcite sql parser on mongo db
> database.
> > I am stuck at a point and need your help in the same.
> >
> > My data in Mongo DB has following structure:-
> > {
> > "_id": "tt0772249",
> > "title": "Palaces of a Queen",
> > "type": "movie",
> > "year": 1967,
> > "crew": [{
> > "name": "Micael Calunga",
> > "role": "actor"
> > }, {
> > "name": "Jordon Zonotti",
> > "role": "actress"
> > }
> > ]
> > }
> >
> > I want to create view with column title, year, crew.name
> > But in modql.json , I am not able to MAP array of objects, like
> crew.name 0th element, or nth element.
> >
> > "tables": [
> > {
> > "name": "movie",
> > "type": "view",
> > "sql": "select cast(_MAP['title'] AS varchar(100)) AS title,
> cast(_MAP['year'] AS varchar(4)) AS year, cast(_MAP['crew.name'][0] AS
> varchar(20)) AS crewname from \"mongo_raw\".\"movie\""
> > },
> >
> >
> > In the GIThib code, we have zip example, but zip file is a flat json,
> but json I am using is embedded , hence not able to use the example.
> >
> > I would really appreciate some help.
> >
> > Thanks!
> >
> > Regards,
> > Neha Saini
> >
> > ::DISCLAIMER::
> > 
> > The contents of this e-mail and any attachment(s) are confidential and
> intended for the named recipient(s) only. E-mail transmission is not
> guaranteed to be secure or error-free as information could be intercepted,
> corrupted, lost, destroyed, arrive late or incomplete, or may contain
> viruses in transmission. The e mail and its contents (with or without
> referred errors) shall therefore not attach any liability on the originator
> or HCL or its affiliates. Views or opinions, if any, presented in this
> email are solely those of the author and may not necessarily reflect the
> views or opinions of HCL or its affiliates. Any form of reproduction,
> dissemination, copying, disclosure, modification, distribution and / or
> publication of this message without the prior written consent of authorized
> representative of HCL is strictly prohibited. If you have received this
> email in error please delete it and notify the sender immediately. Before
> opening any email and/or attachments, please check them for viruses and
> other defects.
> > 
>

Re: Re: [DISCUSSION] Extension of Metadata Query

[Haisheng Yuan]

That's a good question.
Yes. And the method getMetadataQuery from RelOptRuleCall also retrieves 
MetadataQuery from RelOptCluster.
Any other better places?

- Haisheng

--
发件人：Julian Hyde
日　期：2019年10月16日 11:02:33
收件人：dev
主　题：Re: [DISCUSSION] Extension of Metadata Query

Yes, but why RelOptCluster? Besides the fact that there happened to be a method 
there.


> On Oct 15, 2019, at 7:30 PM, Haisheng Yuan  wrote:
> 
> > Is there a good rationale for attaching one to RelOptCluster
> Customized RelMetadataQuery with code generated meta handler for customized 
> metadata, also can provide convenient way to get metadata.
> 
> - Haisheng
> 
> --
> 发件人：Julian Hyde
> 日　期：2019年10月16日 06:52:39
> 收件人：
> 主　题：Re: [DISCUSSION] Extension of Metadata Query
> 
> Having reviewed https://issues.apache.org/jira/browse/CALCITE-3403 
>  and 
> https://issues.apache.org/jira/browse/CALCITE-2018 
>  today I am worried about 
> the direction RelMetadataQuery is taking. Is there a good rationale for 
> attaching one to RelOptCluster? (Besides “it seems to work”?)
> 
> The natural lifespan of a RelMetadataQuery is a RelOptCall (because the 
> structure of the tree is constant for the duration of a call).
> 
> As for sub-classing RelMetadataQuery, I really don’t know. It’s probably 
> harmless, possibly disastrous, and it takes a lot of effort to determine 
> which. We have to be really cautious with RelMetadataQuery because it is one 
> of the few components in Calcite that mutate.
> 
> Julian
> 
> 
> 
> 
> > On Oct 15, 2019, at 1:42 PM, Haisheng Yuan  wrote:
> > 
> > I would like to bring this discussion up again.
> > 
> > We created our own metadata provider, and set it to RelMetadataQuery by 
> > calling
> > RelMetadataQuery.THREAD_PROVIDERS.set()
> > If we just use all the types of metadata provided by Calcite, it should be 
> > fine. But given that we have created new types of metadata, and we want to 
> > get them during rule transformation, we want to have the rule call return 
> > our own RelMetadataQuery. In order to achieve that, we sub-classed 
> > RelMetadataQuery, and sub-classed RelOptCluster to make metadata query 
> > overridable, and return the instance of subclass of RelMetadataQuery when 
> > the mq is null. Indeed, we can achieve that goal by just call 
> > rel.metadata(), but we want a efficient and convenient way to do that. That 
> > is why we want RelMetadataQuery in RelOptCluster customizable. 
> > Is it a good practice? If not, any other better approach to suggest?
> > Thanks.
> > Haisheng
> > 
> > --
> > 发件人：Stamatis Zampetakis
> > 日　期：2019年06月15日 15:47:58
> > 收件人：
> > 主　题：Re: [DISCUSSION] Extension of Metadata Query
> > 
> > I had a look on CALCITE-1147 (which added support for custom providers and
> > metadata) and based on the added tests [1], it seems that the main
> > alternative to RelMetadataQuery is indeed through the MetadataFactory. One
> > other alternative I can think of, is to pass a custom RelMetadataQuery (or
> > a query factory) through the context of the planner [2].
> > 
> > Given that we have already methods to obtain a RelMetadataQuery in
> > RelOptRuleCall and RelOptCluster it may make sense to allow the type of
> > query to be pluggable. However, I have not really used this mechanism in
> > the past so I cannot be sure what's the best way to proceed.
> > 
> > Best,
> > Stamatis
> > 
> > [1]
> > https://github.com/apache/calcite/blob/4e89fddab415a1e04b82c7d69960e399f608949f/core/src/test/java/org/apache/calcite/test/RelMetadataTest.java#L1001
> > [2]
> > https://github.com/apache/calcite/blob/4e89fddab415a1e04b82c7d69960e399f608949f/core/src/main/java/org/apache/calcite/plan/volcano/VolcanoPlanner.java#L256
> > 
> > On Wed, Jun 12, 2019 at 6:07 AM Yuzhao Chen  wrote:
> > 
> >> We have an interface to fetch the RelMetadataQuery in RelOptRuleCall [1],
> >> which I believe is the most common routine to query metadata during
> >> planning, I’m a little confused for your saying
> >> 
> >>> The methods in RelMetadataQuery are for convenience only.
> >> 
> >> If these methods in RelMetadataQuery are only for convenience, what is the
> >> other way I can query a metadata ? The MedataFactory from the RelOptCluster
> >> ? (Users would never expect to use this because it’s based on Java
> >> Reflection and has performance problem)
> >> 
> >> [1]
> >> https://github.com/apache/calcite/blob/a3c56be7bccc58859524ba39e5b30b7078f97d00/core/src/main/java/org/apache/calcite/plan/RelOptRuleCall.java#L200
> >> 
> >> Best,
> >> Danny Chan
> >> 在 2019年6月12日 +0800 AM6:27，Julian Hyde ，写道：
> >>> The goal is that it should be possible to add a new kind of metadata.
> >> The methods in RelMetadataQuery are for convenience 

Re: [DISCUSS] Make Enumerable operators responsive to interrupts

[Julian Hyde]

I didn’t realize that it was cooperative. I now see that we need to call 
Thread.interrupted(), and that no exceptions will be thrown (unless we call a 
method that declares ’throws InterruptedException’).

It seems doable.

Julian


> On Oct 9, 2019, at 3:55 PM, Stamatis Zampetakis  wrote:
> 
> By using interruption, and not forcefully stopping threads, I think we can
> avoid corrupted data-structures etc.
> 
> "Thread interruption is a cooperative mechanism. The cooperative approach
> is required because we rarely want a task, thread, or service to stop
> immediately, since that could leave shared data structures in an
> inconsistent state. Instead, tasks and services can be coded so that, when
> requested, they clean up any work currently in progress and then
> terminate." [1]
> 
> I don't think it will be easy to implement a pause-resume mechanism. I
> tried a couple of times for other use-cases but not with a big success.
> 
> "There is nothing in the API or language specification that ties
> interruption to any specific cancellation semantics, but in
> practice, using interruption for anything but cancellation is fragile and
> difficult to sustain in larger applications." [1]
> 
> Based on the contract of the Enumerator calling close should release any
> resources so if it is implemented right we shouldn't end up with resource
> leaks.
> 
> In my mind we should check for interrupts only at the slowest part(s) of
> the operation. I am hoping that this is not the initialization or tear down
> phase but remains to be verified.
> 
> Regarding tests, I think it depends how we will implement the cancellation.
> For instance, if we decide to throw specialized exceptions then we can
> enrich them with any additional information that could tell us exactly
> where and when the interruption started. We can also inspect the stack
> trace of running threads and verify that there is no Enumerable code
> running after interruption. Sure is that we cannot rely on the correctness
> of the returned results so far (if there are).
> 
> Best,
> Stamatis
> 
> 
> [1] Java Java Concurrency in Practice by Brian Goetz
> 
> On Tue, Oct 8, 2019 at 8:56 PM Julian Hyde  wrote:
> 
>> Is there a possibility that data structures will be corrupted, if a thread
>> is interrupted in the middle of an operation?
>> 
>> Supposing that we allow resume, is it possible to safely resume after an
>> interrupt?
>> 
>> Supposing that we do not allow resume, and instead call close on the root
>> Enumerable, is it possible to guarantee each Enumerator cleans up after
>> itself?
>> 
>> Is there a period during the lifecycle of a tree of Enumerable objects
>> (e.g. initialization, tear down) where we do not allow interrupts?
>> 
>> How would we test this?
>> 
>> Julian
>> 
>> 
>>> On Oct 8, 2019, at 10:48 AM, Haisheng Yuan 
>> wrote:
>>> 
>>> Make sense and quite reasonable.
>>> 
>>> - Haisheng
>>> 
>>> --
>>> 发件人：Stamatis Zampetakis
>>> 日 期：2019年10月08日 18:04:17
>>> 收件人：
>>> 主 题：[DISCUSS] Make Enumerable operators responsive to interrupts
>>> 
>>> Hello,
>>> 
>>> There are many use-cases which require stopping/cancelling the execution
>> of
>>> a query for various reasons. Currently, this can be done by launching the
>>> query in a separate thread and then setting
>>> DataContext.Variable.CANCEL_FLAG [1] accordingly.
>>> 
>>> However if the tread executing the query gets interrupted through the
>> usual
>>> Thread.interrupt() mechanism the query execution will not stop since the
>>> operators are not responsive to interruption.
>>> 
>>> How do you feel about making Enumerable operators responsive to
>> interrupts?
>>> 
>>> Best,
>>> Stamatis
>>> 
>>> [1]
>>> 
>> https://github.com/apache/calcite/blob/3f54108b7dcd4d2b89fc42faab145e2f82883791/core/src/main/java/org/apache/calcite/DataContext.java#L87
>>> 
>> 
>> 

Re: [DISCUSS] Support Sql Hint for Calcite

[Julian Hyde]

By “skeptical” I mean that I think we can come up with a mechanism to copy 
hints when applying planner rules, but even when we have implemented that 
mechanism there will be many cases where people want a hint and that hint is 
not copied to the RelNode where it is needed, and many other cases where we 
spend the effort/memory of copying the hint to a RelNode and the hint is not 
used.

By “reversible” I mean if we come up with an API that does not work, how do we 
change or remove that API without people complaining?

Julian


> On Oct 15, 2019, at 7:11 PM, Danny Chan  wrote:
> 
> Thanks Julian
> 
>> I am skeptical that RelWithHint will work for large queries.
> 
> For “skeptical” do you mean how to transfer the hints during rule planning ? 
> I’m also not that confident yet.
> 
>> How do we introduce it in a reversible way
> Do you mean transform the RelWithHint back into the SqlHint ? I didn’t 
> implement it in current patch, but I think we have the ability to do that 
> because we have a inheritPath for each RelWithHint, we can collect all the 
> hints together and merge them into the SqlHints, then propagate these 
> SqlHints to the SqlNodes.
> 
>> What are the other options?
> Do you mean the way to transfer hints during planning ? I have no other 
> options yet.
> 
> Best,
> Danny Chan
> 在 2019年10月16日 +0800 AM8:03，dev@calcite.apache.org，写道：
>> 
>> I am skeptical that RelWithHint will work for large queries.

Re: [DISCUSSION] Extension of Metadata Query

[Julian Hyde]

Yes, but why RelOptCluster? Besides the fact that there happened to be a method 
there.


> On Oct 15, 2019, at 7:30 PM, Haisheng Yuan  wrote:
> 
> > Is there a good rationale for attaching one to RelOptCluster
> Customized RelMetadataQuery with code generated meta handler for customized 
> metadata, also can provide convenient way to get metadata.
> 
> - Haisheng
> 
> --
> 发件人：Julian Hyde
> 日　期：2019年10月16日 06:52:39
> 收件人：
> 主　题：Re: [DISCUSSION] Extension of Metadata Query
> 
> Having reviewed https://issues.apache.org/jira/browse/CALCITE-3403 
>  and 
> https://issues.apache.org/jira/browse/CALCITE-2018 
>  today I am worried about 
> the direction RelMetadataQuery is taking. Is there a good rationale for 
> attaching one to RelOptCluster? (Besides “it seems to work”?)
> 
> The natural lifespan of a RelMetadataQuery is a RelOptCall (because the 
> structure of the tree is constant for the duration of a call).
> 
> As for sub-classing RelMetadataQuery, I really don’t know. It’s probably 
> harmless, possibly disastrous, and it takes a lot of effort to determine 
> which. We have to be really cautious with RelMetadataQuery because it is one 
> of the few components in Calcite that mutate.
> 
> Julian
> 
> 
> 
> 
> > On Oct 15, 2019, at 1:42 PM, Haisheng Yuan  wrote:
> > 
> > I would like to bring this discussion up again.
> > 
> > We created our own metadata provider, and set it to RelMetadataQuery by 
> > calling
> > RelMetadataQuery.THREAD_PROVIDERS.set()
> > If we just use all the types of metadata provided by Calcite, it should be 
> > fine. But given that we have created new types of metadata, and we want to 
> > get them during rule transformation, we want to have the rule call return 
> > our own RelMetadataQuery. In order to achieve that, we sub-classed 
> > RelMetadataQuery, and sub-classed RelOptCluster to make metadata query 
> > overridable, and return the instance of subclass of RelMetadataQuery when 
> > the mq is null. Indeed, we can achieve that goal by just call 
> > rel.metadata(), but we want a efficient and convenient way to do that. That 
> > is why we want RelMetadataQuery in RelOptCluster customizable. 
> > Is it a good practice? If not, any other better approach to suggest?
> > Thanks.
> > Haisheng
> > 
> > --
> > 发件人：Stamatis Zampetakis
> > 日　期：2019年06月15日 15:47:58
> > 收件人：
> > 主　题：Re: [DISCUSSION] Extension of Metadata Query
> > 
> > I had a look on CALCITE-1147 (which added support for custom providers and
> > metadata) and based on the added tests [1], it seems that the main
> > alternative to RelMetadataQuery is indeed through the MetadataFactory. One
> > other alternative I can think of, is to pass a custom RelMetadataQuery (or
> > a query factory) through the context of the planner [2].
> > 
> > Given that we have already methods to obtain a RelMetadataQuery in
> > RelOptRuleCall and RelOptCluster it may make sense to allow the type of
> > query to be pluggable. However, I have not really used this mechanism in
> > the past so I cannot be sure what's the best way to proceed.
> > 
> > Best,
> > Stamatis
> > 
> > [1]
> > https://github.com/apache/calcite/blob/4e89fddab415a1e04b82c7d69960e399f608949f/core/src/test/java/org/apache/calcite/test/RelMetadataTest.java#L1001
> > [2]
> > https://github.com/apache/calcite/blob/4e89fddab415a1e04b82c7d69960e399f608949f/core/src/main/java/org/apache/calcite/plan/volcano/VolcanoPlanner.java#L256
> > 
> > On Wed, Jun 12, 2019 at 6:07 AM Yuzhao Chen  wrote:
> > 
> >> We have an interface to fetch the RelMetadataQuery in RelOptRuleCall [1],
> >> which I believe is the most common routine to query metadata during
> >> planning, I’m a little confused for your saying
> >> 
> >>> The methods in RelMetadataQuery are for convenience only.
> >> 
> >> If these methods in RelMetadataQuery are only for convenience, what is the
> >> other way I can query a metadata ? The MedataFactory from the RelOptCluster
> >> ? (Users would never expect to use this because it’s based on Java
> >> Reflection and has performance problem)
> >> 
> >> [1]
> >> https://github.com/apache/calcite/blob/a3c56be7bccc58859524ba39e5b30b7078f97d00/core/src/main/java/org/apache/calcite/plan/RelOptRuleCall.java#L200
> >> 
> >> Best,
> >> Danny Chan
> >> 在 2019年6月12日 +0800 AM6:27，Julian Hyde ，写道：
> >>> The goal is that it should be possible to add a new kind of metadata.
> >> The methods in RelMetadataQuery are for convenience only. So you should be
> >> able to use your new kind of metadata, and existing ones, without modifying
> >> RelMetadataQuery.
> >>> 
> >>> If that is not possible, it’s a bug, and you should log a JIRA case.
> >>> 
> >>> Given how the methods re-generate handlers (by catching exceptions and
> >> modifying mutable private members) I 

Re: Re: [DISCUSSION] Extension of Metadata Query

[Haisheng Yuan]

> Is there a good rationale for attaching one to RelOptCluster
Customized RelMetadataQuery with code generated meta handler for customized 
metadata, also can provide convenient way to get metadata.

- Haisheng

--
发件人：Julian Hyde
日　期：2019年10月16日 06:52:39
收件人：
主　题：Re: [DISCUSSION] Extension of Metadata Query

Having reviewed https://issues.apache.org/jira/browse/CALCITE-3403 
 and 
https://issues.apache.org/jira/browse/CALCITE-2018 
 today I am worried about 
the direction RelMetadataQuery is taking. Is there a good rationale for 
attaching one to RelOptCluster? (Besides “it seems to work”?)

The natural lifespan of a RelMetadataQuery is a RelOptCall (because the 
structure of the tree is constant for the duration of a call).

As for sub-classing RelMetadataQuery, I really don’t know. It’s probably 
harmless, possibly disastrous, and it takes a lot of effort to determine which. 
We have to be really cautious with RelMetadataQuery because it is one of the 
few components in Calcite that mutate.

Julian




> On Oct 15, 2019, at 1:42 PM, Haisheng Yuan  wrote:
> 
> I would like to bring this discussion up again.
> 
> We created our own metadata provider, and set it to RelMetadataQuery by 
> calling
> RelMetadataQuery.THREAD_PROVIDERS.set()
> If we just use all the types of metadata provided by Calcite, it should be 
> fine. But given that we have created new types of metadata, and we want to 
> get them during rule transformation, we want to have the rule call return our 
> own RelMetadataQuery. In order to achieve that, we sub-classed 
> RelMetadataQuery, and sub-classed RelOptCluster to make metadata query 
> overridable, and return the instance of subclass of RelMetadataQuery when the 
> mq is null. Indeed, we can achieve that goal by just call rel.metadata(), but 
> we want a efficient and convenient way to do that. That is why we want 
> RelMetadataQuery in RelOptCluster customizable. 
> Is it a good practice? If not, any other better approach to suggest?
> Thanks.
> Haisheng
> 
> --
> 发件人：Stamatis Zampetakis
> 日　期：2019年06月15日 15:47:58
> 收件人：
> 主　题：Re: [DISCUSSION] Extension of Metadata Query
> 
> I had a look on CALCITE-1147 (which added support for custom providers and
> metadata) and based on the added tests [1], it seems that the main
> alternative to RelMetadataQuery is indeed through the MetadataFactory. One
> other alternative I can think of, is to pass a custom RelMetadataQuery (or
> a query factory) through the context of the planner [2].
> 
> Given that we have already methods to obtain a RelMetadataQuery in
> RelOptRuleCall and RelOptCluster it may make sense to allow the type of
> query to be pluggable. However, I have not really used this mechanism in
> the past so I cannot be sure what's the best way to proceed.
> 
> Best,
> Stamatis
> 
> [1]
> https://github.com/apache/calcite/blob/4e89fddab415a1e04b82c7d69960e399f608949f/core/src/test/java/org/apache/calcite/test/RelMetadataTest.java#L1001
> [2]
> https://github.com/apache/calcite/blob/4e89fddab415a1e04b82c7d69960e399f608949f/core/src/main/java/org/apache/calcite/plan/volcano/VolcanoPlanner.java#L256
> 
> On Wed, Jun 12, 2019 at 6:07 AM Yuzhao Chen  wrote:
> 
>> We have an interface to fetch the RelMetadataQuery in RelOptRuleCall [1],
>> which I believe is the most common routine to query metadata during
>> planning, I’m a little confused for your saying
>> 
>>> The methods in RelMetadataQuery are for convenience only.
>> 
>> If these methods in RelMetadataQuery are only for convenience, what is the
>> other way I can query a metadata ? The MedataFactory from the RelOptCluster
>> ? (Users would never expect to use this because it’s based on Java
>> Reflection and has performance problem)
>> 
>> [1]
>> https://github.com/apache/calcite/blob/a3c56be7bccc58859524ba39e5b30b7078f97d00/core/src/main/java/org/apache/calcite/plan/RelOptRuleCall.java#L200
>> 
>> Best,
>> Danny Chan
>> 在 2019年6月12日 +0800 AM6:27，Julian Hyde ，写道：
>>> The goal is that it should be possible to add a new kind of metadata.
>> The methods in RelMetadataQuery are for convenience only. So you should be
>> able to use your new kind of metadata, and existing ones, without modifying
>> RelMetadataQuery.
>>> 
>>> If that is not possible, it’s a bug, and you should log a JIRA case.
>>> 
>>> Given how the methods re-generate handlers (by catching exceptions and
>> modifying mutable private members) I can see that new kinds of metadata
>> might be difficult to add.
>>> 
>>> Julian
>>> 
>>> 
 On Jun 9, 2019, at 7:54 PM, Yuzhao Chen  wrote:
 
 Thanks, Stamatis
 
 You are right, this discussion came up due to CALCITE-2885, it is not
>> about the performance problem, it is the extension of RelMetadataQuery,
>> because we add all kinds 

Re: [QUESTION] Pushing up evaluations from LogicalProjects

[Danny Chan]

Just open it in the SqlToRelConverter [1]

[1] 
https://github.com/apache/calcite/blob/3cbbafa941128dc5097c2a26711f5751f764e12d/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L5692

Best,
Danny Chan
在 2019年10月16日 +0800 AM12:23，Rommel Quintanilla ，写道：
> Hi Stamatis, thank you for your attention and actually many thanks to 
> everyone who is seeing/commenting about this topic.
>
> To be honest I didn't know about the existence of RelFieldTrimmer.
>
> I would like to test this, how could I use RelFieldTrimmer? can you give me a 
> suggestion, please?
>
> On 2019/10/12 07:00:00, Stamatis Zampetakis  wrote:
> > Hi Rommel,
> >
> > I was hoping that this could be done at least by RelFieldTrimmer [1]. Are
> > you using it already?
> >
> > Best,
> > Stamatis
> >
> > [1]
> > https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java
> >
> > On Sat, Oct 12, 2019 at 6:06 AM XING JIN  wrote:
> >
> > > Filed a JIRA:
> > > https://issues.apache.org/jira/browse/CALCITE-3405
> > >
> > > Haisheng Yuan  于2019年10月12日周六 上午4:34写道：
> > >
> > > > Yes, definitely.
> > > >
> > > > You can go through the project expression with InputFinder to find all
> > > the
> > > > used columns, create a logical project with those columns, and remap the
> > > > top project with new column indexes.
> > > >
> > > > On the other hand, instead of creating a new intermidiate pogical
> > > project,
> > > > we can also update ProjectTableScanRule to accept LogicalProject that is
> > > > not a simple mapping, and do the same task I mentioned above.
> > > >
> > > > - Haisheng
> > > >
> > > > --
> > > > 发件人：Rommel Quintanilla
> > > > 日 期：2019年10月12日 03:15:31
> > > > 收件人：
> > > > 主 题：[QUESTION] Pushing up evaluations from LogicalProjects
> > > >
> > > > Hi, maybe you can help me.
> > > > I have this portion from a larger logical plan:
> > > > ..
> > > > LogicalProject(l_orderkey=[$0], l_suppkey=[$2], *=[*($5, -(1,
> > > > $6))])
> > > > LogicalTableScan(table=[[main, lineitem]])
> > > > ..
> > > >
> > > > Because the LogicalProject above contains an evaluation, the
> > > > ProjectTableScanRule can't convert it to a BindableTableScan.
> > > >
> > > > I wonder if somehow the evaluation could be pushed up more or less like
> > > > this:
> > > > ..
> > > > LogicalProject(l_orderkey=[$0], l_suppkey=[$1], *=[*($2, -(1, $3))])
> > > > LogicalProject(l_orderkey=[$0], l_suppkey=[$2], l_extendedprice=[$5],
> > > > l_discount=[$6]])
> > > > LogicalTableScan(table=[[main, lineitem]])
> > > > ..
> > > >
> > > > Regards.
> > > >
> > >
> >

Re: [DISCUSS] Support Sql Hint for Calcite

[Danny Chan]

Thanks Julian

> I am skeptical that RelWithHint will work for large queries.

For “skeptical” do you mean how to transfer the hints during rule planning ? 
I’m also not that confident yet.

> How do we introduce it in a reversible way
Do you mean transform the RelWithHint back into the SqlHint ? I didn’t 
implement it in current patch, but I think we have the ability to do that 
because we have a inheritPath for each RelWithHint, we can collect all the 
hints together and merge them into the SqlHints, then propagate these SqlHints 
to the SqlNodes.

> What are the other options?
Do you mean the way to transfer hints during planning ? I have no other options 
yet.

Best,
Danny Chan
在 2019年10月16日 +0800 AM8:03，dev@calcite.apache.org，写道：
>
> I am skeptical that RelWithHint will work for large queries.

Re: [DISCUSS] Support Sql Hint for Calcite

[Julian Hyde]

I am skeptical that RelWithHint will work for large queries. How do we 
introduce it in a reversible way? What are the other options?


> On Oct 10, 2019, at 1:29 AM, Danny Chan  wrote:
> 
> Hi, Julian, do you think we can make some effort to push this feature into 
> release 1.22 ? The Flink community kind of need this feature in the near 
> future.
> 
> The most controversial part is how to transfer the hints during planning 
> phrase, my initial idea is:
>> To copy the hints automatedly in the method RelOptRuleCall#transformTo for 
>> the new rel node from the >original rel node if both of them are all 
>> RelWithHint.
> 
> I have added some simple planning test cases and it works as expected, but 
> I’m not sure if this can fits all the corner cases(very probably not).
> 
> 
> The other part works great as the design doc describes.
> 
> Best,
> Danny Chan
> 在 2019年9月5日 +0800 AM10:12，Julian Hyde ，写道：
>> Thanks for continuing to push on this!
>> 
>> I don’t much like the MSSQL-style syntax for table hints. It adds a new use 
>> of the WITH keyword that is unrelated to the use of WITH for common-table 
>> expressions. Instead of
>> 
>> select /*+ NO_HASH_JOIN, RESOURCE(mem='128mb', parallelism='24') */
>> from
>> emp with (INDEX(idx1, idx2))
>> join
>> dept with (PROPERTIES(k1='v1', k2='v2'))
>> on
>> emp.deptno = dept.deptno
>> 
>> could we do
>> 
>> select /*+ NO_HASH_JOIN, RESOURCE(mem='128mb', parallelism='24') */
>> from
>> emp /*+ INDEX(idx1, idx2) */
>> join
>> dept /*+ PROPERTIES(k1='v1', k2='v2’) */
>> on
>> emp.deptno = dept.deptno
>> 
>> 
>> In some of the classes you have public fields of type ImmutableList. This 
>> makes it difficult to coexist in an environment that uses a different 
>> version of Guava, or shades Guava. Probably better to make them of type 
>> List. (You don’t need to change ImmutableBitSet; it’s not a Guava class.)
>> 
>> There is one argument where a List is assigned to an 
>> ImmutableBitSet. Make it an Iterable, and people can pass in an 
>> existing ImmutableBitSet without copying.
>> 
>> Julian
>> 
>> 
>>> On Sep 3, 2019, at 1:18 AM, Danny Chan  wrote:
>>> 
>>> Hi, fellows, I’m here again :)
>>> 
>>> This time I want to have a full discussion about CALCITE-482, which is an 
>>> issue fired at 27/Nov/14 by Vladimir Sitnikov.
>>> 
>>> Almost every sql vendor supports sql hint for their production version DB 
>>> engines, so it would be nice if we have a framework in Calcite to support 
>>> this, so that the engines that built based on CALCITE would have the 
>>> ability to extend and have their custom sql hint implementations.
>>> 
>>> In April this year I have fired an initial discussion about this topic[1], 
>>> and I’m glad that we have some agreements for the design.
>>> 
>>> Recently I have fired a PR[3] and write a design doc[2] mostly based on the 
>>> discussion[1], so feel free to give some feedback here so we can make the 
>>> hint framework more flexible.
>>> 
>>> Any suggestions are appreciated.
>>> 
>>> 
>>> [1] 
>>> https://lists.apache.org/list.html?dev@calcite.apache.org:dfr=2019-4-1|dto=2019-4-30:How%20to%20traverse
>>> [2] 
>>> https://docs.google.com/document/d/1mykz-w2t1Yw7CH6NjUWpWqCAf_6YNKxSc59gXafrNCs/edit?usp=sharing
>>> [3] https://github.com/apache/calcite/pull/1354
>>> 
>>> Best,
>>> Danny Chan
>> 

Re: [DISCUSSION] Extension of Metadata Query

[Julian Hyde]

Having reviewed https://issues.apache.org/jira/browse/CALCITE-3403 
 and 
https://issues.apache.org/jira/browse/CALCITE-2018 
 today I am worried about 
the direction RelMetadataQuery is taking. Is there a good rationale for 
attaching one to RelOptCluster? (Besides “it seems to work”?)

The natural lifespan of a RelMetadataQuery is a RelOptCall (because the 
structure of the tree is constant for the duration of a call).

As for sub-classing RelMetadataQuery, I really don’t know. It’s probably 
harmless, possibly disastrous, and it takes a lot of effort to determine which. 
We have to be really cautious with RelMetadataQuery because it is one of the 
few components in Calcite that mutate.

Julian




> On Oct 15, 2019, at 1:42 PM, Haisheng Yuan  wrote:
> 
> I would like to bring this discussion up again.
> 
> We created our own metadata provider, and set it to RelMetadataQuery by 
> calling
> RelMetadataQuery.THREAD_PROVIDERS.set()
> If we just use all the types of metadata provided by Calcite, it should be 
> fine. But given that we have created new types of metadata, and we want to 
> get them during rule transformation, we want to have the rule call return our 
> own RelMetadataQuery. In order to achieve that, we sub-classed 
> RelMetadataQuery, and sub-classed RelOptCluster to make metadata query 
> overridable, and return the instance of subclass of RelMetadataQuery when the 
> mq is null.  Indeed, we can achieve that goal by just call rel.metadata(), 
> but we want a efficient and convenient way to do that. That is why we want 
> RelMetadataQuery in RelOptCluster customizable. 
> Is it a good practice? If not, any other better approach to suggest?
> Thanks.
> Haisheng
> 
> --
> 发件人：Stamatis Zampetakis
> 日　期：2019年06月15日 15:47:58
> 收件人：
> 主　题：Re: [DISCUSSION] Extension of Metadata Query
> 
> I had a look on CALCITE-1147 (which added support for custom providers and
> metadata) and based on the added tests [1], it seems that the main
> alternative to RelMetadataQuery is indeed through the MetadataFactory. One
> other alternative I can think of, is to pass a custom RelMetadataQuery (or
> a query factory) through the context of the planner [2].
> 
> Given that we have already methods to obtain a RelMetadataQuery in
> RelOptRuleCall and RelOptCluster it may make sense to allow the type of
> query to be pluggable. However, I have not really used this mechanism in
> the past so I cannot be sure what's the best way to proceed.
> 
> Best,
> Stamatis
> 
> [1]
> https://github.com/apache/calcite/blob/4e89fddab415a1e04b82c7d69960e399f608949f/core/src/test/java/org/apache/calcite/test/RelMetadataTest.java#L1001
> [2]
> https://github.com/apache/calcite/blob/4e89fddab415a1e04b82c7d69960e399f608949f/core/src/main/java/org/apache/calcite/plan/volcano/VolcanoPlanner.java#L256
> 
> On Wed, Jun 12, 2019 at 6:07 AM Yuzhao Chen  wrote:
> 
>> We have an interface to fetch the RelMetadataQuery in RelOptRuleCall [1],
>> which I believe is the most common routine to query metadata during
>> planning, I’m a little confused for your saying
>> 
>>> The methods in RelMetadataQuery are for convenience only.
>> 
>> If these methods in RelMetadataQuery are only for convenience, what is the
>> other way I can query a metadata ? The MedataFactory from the RelOptCluster
>> ? (Users would never expect to use this because it’s based on Java
>> Reflection and has performance problem)
>> 
>> [1]
>> https://github.com/apache/calcite/blob/a3c56be7bccc58859524ba39e5b30b7078f97d00/core/src/main/java/org/apache/calcite/plan/RelOptRuleCall.java#L200
>> 
>> Best,
>> Danny Chan
>> 在 2019年6月12日 +0800 AM6:27，Julian Hyde ，写道：
>>> The goal is that it should be possible to add a new kind of metadata.
>> The methods in RelMetadataQuery are for convenience only. So you should be
>> able to use your new kind of metadata, and existing ones, without modifying
>> RelMetadataQuery.
>>> 
>>> If that is not possible, it’s a bug, and you should log a JIRA case.
>>> 
>>> Given how the methods re-generate handlers (by catching exceptions and
>> modifying mutable private members) I can see that new kinds of metadata
>> might be difficult to add.
>>> 
>>> Julian
>>> 
>>> 
 On Jun 9, 2019, at 7:54 PM, Yuzhao Chen  wrote:
 
 Thanks, Stamatis
 
 You are right, this discussion came up due to CALCITE-2885, it is not
>> about the performance problem, it is the extension of RelMetadataQuery,
>> because we add all kinds of top interfaces in RelMetadataQuery, e.g. the
>> row count query [1].
 
 When we add a new RelMetadataProvider, a corresponding
>> interface/method should be added into RelMetadataQuery, but for current
>> RelOpeCluster impl, we could not do that(with a always default instance)
 
 As for the methods in RelMetadataProvider, i only saw the usage of
>> 

[jira] [Created] (CALCITE-3418) Grouped Window Function "$TUMBLE" should return "

[Rui Wang (Jira)]

Rui Wang created CALCITE-3418:
-

 Summary: Grouped Window Function "$TUMBLE" should return " 
 Key: CALCITE-3418
 URL: https://issues.apache.org/jira/browse/CALCITE-3418
 Project: Calcite
  Issue Type: Sub-task
Reporter: Rui Wang






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Re: [DISCUSSION] Extension of Metadata Query

[Haisheng Yuan]

I would like to bring this discussion up again.

We created our own metadata provider, and set it to RelMetadataQuery by calling
RelMetadataQuery.THREAD_PROVIDERS.set()
If we just use all the types of metadata provided by Calcite, it should be 
fine. But given that we have created new types of metadata, and we want to get 
them during rule transformation, we want to have the rule call return our own 
RelMetadataQuery. In order to achieve that, we sub-classed RelMetadataQuery, 
and sub-classed RelOptCluster to make metadata query overridable, and return 
the instance of subclass of RelMetadataQuery when the mq is null.  Indeed, we 
can achieve that goal by just call rel.metadata(), but we want a efficient and 
convenient way to do that. That is why we want RelMetadataQuery in 
RelOptCluster customizable. 
Is it a good practice? If not, any other better approach to suggest?
Thanks.
Haisheng

--
发件人：Stamatis Zampetakis
日　期：2019年06月15日 15:47:58
收件人：
主　题：Re: [DISCUSSION] Extension of Metadata Query

I had a look on CALCITE-1147 (which added support for custom providers and
metadata) and based on the added tests [1], it seems that the main
alternative to RelMetadataQuery is indeed through the MetadataFactory. One
other alternative I can think of, is to pass a custom RelMetadataQuery (or
a query factory) through the context of the planner [2].

Given that we have already methods to obtain a RelMetadataQuery in
RelOptRuleCall and RelOptCluster it may make sense to allow the type of
query to be pluggable. However, I have not really used this mechanism in
the past so I cannot be sure what's the best way to proceed.

Best,
Stamatis

[1]
https://github.com/apache/calcite/blob/4e89fddab415a1e04b82c7d69960e399f608949f/core/src/test/java/org/apache/calcite/test/RelMetadataTest.java#L1001
[2]
https://github.com/apache/calcite/blob/4e89fddab415a1e04b82c7d69960e399f608949f/core/src/main/java/org/apache/calcite/plan/volcano/VolcanoPlanner.java#L256

On Wed, Jun 12, 2019 at 6:07 AM Yuzhao Chen  wrote:

> We have an interface to fetch the RelMetadataQuery in RelOptRuleCall [1],
> which I believe is the most common routine to query metadata during
> planning, I’m a little confused for your saying
>
> >The methods in RelMetadataQuery are for convenience only.
>
> If these methods in RelMetadataQuery are only for convenience, what is the
> other way I can query a metadata ? The MedataFactory from the RelOptCluster
> ? (Users would never expect to use this because it’s based on Java
> Reflection and has performance problem)
>
> [1]
> https://github.com/apache/calcite/blob/a3c56be7bccc58859524ba39e5b30b7078f97d00/core/src/main/java/org/apache/calcite/plan/RelOptRuleCall.java#L200
>
> Best,
> Danny Chan
> 在 2019年6月12日 +0800 AM6:27，Julian Hyde ，写道：
> > The goal is that it should be possible to add a new kind of metadata.
> The methods in RelMetadataQuery are for convenience only. So you should be
> able to use your new kind of metadata, and existing ones, without modifying
> RelMetadataQuery.
> >
> > If that is not possible, it’s a bug, and you should log a JIRA case.
> >
> > Given how the methods re-generate handlers (by catching exceptions and
> modifying mutable private members) I can see that new kinds of metadata
> might be difficult to add.
> >
> > Julian
> >
> >
> > > On Jun 9, 2019, at 7:54 PM, Yuzhao Chen  wrote:
> > >
> > > Thanks, Stamatis
> > >
> > > You are right, this discussion came up due to CALCITE-2885, it is not
> about the performance problem, it is the extension of RelMetadataQuery,
> because we add all kinds of top interfaces in RelMetadataQuery, e.g. the
> row count query [1].
> > >
> > > When we add a new RelMetadataProvider, a corresponding
> interface/method should be added into RelMetadataQuery, but for current
> RelOpeCluster impl, we could not do that(with a always default instance)
> > >
> > > As for the methods in RelMetadataProvider, i only saw the usage of
> #handlers in RelMetadataQuery, and saw the reference you pase, it seems not
> that relevant with this topic. What do you think ?
> > >
> > > [1]
> https://github.com/apache/calcite/blob/941cd4e9540e3ef9b7c15daee42831a0c63da8b9/core/src/main/java/org/apache/calcite/rel/metadata/RelMetadataQuery.java#L222
> > >
> > > Best,
> > > Danny Chan
> > > 在 2019年6月5日 +0800 AM6:44，Stamatis Zampetakis ，写道：
> > > > Thanks for bringing this up Danny.
> > > >
> > > > I guess the discussion came up due to CALCITE-2885 [1]. Looking back
> into
> > > > it, I am not sure that the intention there is to make the
> RelMetadataQuery
> > > > pluggable. We could possibly solve the performance problem without
> > > > extending the RelMetadataQuery. We have to look again into this case.
> > > >
> > > > For more details regarding the existence of the two methods in
> > > > RelMetadataProvider have a look in CALCITE-604 [2]. More general for
> the
> > > > design of RelMetadataProvider you may find useful the 

Re: [DISCUSS] Avatica - how efficient is our protocol?

[Michael Mior]

I thought I would resurrect this thread given the announcement of
Flight (link below). I'm not too familiar with Avatica, but it seems
like Flight (essentially a client-server framework for transporting
Arrow data) could be a good fit.

https://arrow.apache.org/blog/2019/10/13/introducing-arrow-flight/
--
Michael Mior
mm...@apache.org

Le jeu. 23 août 2018 à 15:45, Julian Hyde  a écrit :
>
> This is a paper in VLDB 2018, "Don’t Hold My Data Hostage – A Case For Client 
> Protocol Redesign” by Mark Rassveldt and Hannes Muhleisen[1]. It claims that 
> database client protocols (inside ODBC and JDBC drivers) are very 
> inefficient, and has a compelling example where commercial drivers are 10x to 
> 68x slower than net-cat.
>
> One of the goals of Avatica is to do better. How are we doing? Are there any 
> ideas in the paper we could adopt? Would a closer partnership with Apache 
> Arrow help us achieve those goals?
>
> Julian
>
> [1] https://hannes.muehleisen.org/p852-muehleisen.pdf 
> 

[jira] [Created] (CALCITE-3417) The alias is invalid in RelBuilder#join(JoinRelType, RexNode)

[jamie12221 (Jira)]

jamie12221 created CALCITE-3417:
---

 Summary: The alias is invalid in RelBuilder#join(JoinRelType, 
RexNode)
 Key: CALCITE-3417
 URL: https://issues.apache.org/jira/browse/CALCITE-3417
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.21.0, 1.20.0
Reporter: jamie12221


bad code:

RelBuilder relBuilder = RelBuilder.create(config);
RelNode table = relBuilder
 .scan("testdb","travelrecord")
 .as("t")
 .scan("testdb","address")
 .as("a")
 .join(JoinRelType.INNER, relBuilder.equals(relBuilder.field("t","id"),
 relBuilder.field("a","id")))
 
.filter(relBuilder.and(relBuilder.equals(relBuilder.field(1,0,"id"),relBuilder.literal(1
 .project(relBuilder.field(1,0,"id"), relBuilder.field(1,0,"user_id"))
 .build();

 

occur exception:

java.lang.IllegalArgumentException: no aliased field found; fields are: 
[\{aliases=[address, a],fieldName=id}, \{aliases=[address, 
a],fieldName=addressname}]java.lang.IllegalArgumentException: no aliased field 
found; fields are: [\{aliases=[address, a],fieldName=id}, \{aliases=[address, 
a],fieldName=addressname}] at 
org.apache.calcite.tools.RelBuilder.field(RelBuilder.java:511) at 
org.apache.calcite.tools.RelBuilder.field(RelBuilder.java:487)

 

but  it is ok :

RelBuilder relBuilder = RelBuilder.create(config);
RelNode table = relBuilder
 .scan("testdb","travelrecord")
 .as("t")
 .scan("testdb","address")
 .as("a")
 .join(JoinRelType.INNER, 
relBuilder.equals(relBuilder.field({color:#de350b}2{color},"t","id"),
 relBuilder.field(2,"a","id")))
 
.filter(relBuilder.and(relBuilder.equals(relBuilder.field(1,0,"id"),relBuilder.literal(1
 .project(relBuilder.field(1,0,"id"), relBuilder.field(1,0,"user_id"))
 .build();



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Different performance of the same code in two laptops

[Enrico Olivelli]

Can you share the differences between the two machines? RAM, CPU, OS

As far as I know internal java ergonomics make code run very differently
depending on available resources


Enrico

Il mar 15 ott 2019, 15:40 Stamatis Zampetakis  ha
scritto:

> Hi Makis,
>
> Use a profiler (e.g., VisualVM) and check where the time goes. You can also
> compare the snapshots between the two machines.
>
> Best,
> Stamatis
>
> On Tue, Oct 15, 2019 at 3:08 PM Michael Mior  wrote:
>
> > I think it's going to be hard for anyone to give you specific advice
> > without more information. How are you measuring the runtime? You may
> > need to instrument the Calcite code, but I would suggest breaking down
> > the runtime more to find out where the slowdown is happening. On a
> > related note, the optimization process is single-threaded, so this
> > would not be your problem.
> > --
> > Michael Mior
> > mm...@apache.org
> >
> > Le mar. 15 oct. 2019 à 06:11, Serafeim (Makis) Papadias
> >  a écrit :
> > >
> > > Hello everyone,
> > >
> > > I am experimenting with rules and volcano optimiser lately and I am
> > facing something weird. I run my code through Intellij IDE. The databases
> > are set the same way in both machines and since I care only about query
> > rewriting for now I just read the table schemas from the databases; I do
> > not execute anything.
> > >
> > > The same code on one machine runs 10 times slower than on the other
> > machine.
> > >
> > > * I tried to rule out that the IDE is not the problem, so I installed
> > the same versions and picked the exact same vmoptions.
> > > * The database schemas are the same, thus I do not believe it is
> related
> > with the underneath sources.
> > >
> > > Is it possible that calcite uses all the threads available on one
> > machine while running only on thread the other?
> > >
> > > If you have any recommendations, they are more that welcome!
> > >
> > > Thanks in advance.
> > >
> > > Best,
> > > Makis
> >
>

Re: [QUESTION] Pushing up evaluations from LogicalProjects

[Rommel Quintanilla]

Hi Stamatis, thank you for your attention and actually many thanks to everyone 
who is seeing/commenting about this topic.

To be honest I didn't know about the existence of RelFieldTrimmer.

I would like to test this, how could I use RelFieldTrimmer? can you give me a 
suggestion, please?

On 2019/10/12 07:00:00, Stamatis Zampetakis  wrote: 
> Hi Rommel,
> 
> I was hoping that this could be done at least by RelFieldTrimmer [1]. Are
> you using it already?
> 
> Best,
> Stamatis
> 
> [1]
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java
> 
> On Sat, Oct 12, 2019 at 6:06 AM XING JIN  wrote:
> 
> > Filed a JIRA:
> > https://issues.apache.org/jira/browse/CALCITE-3405
> >
> > Haisheng Yuan  于2019年10月12日周六 上午4:34写道：
> >
> > > Yes, definitely.
> > >
> > > You can go through the project expression with InputFinder to find all
> > the
> > > used columns, create a  logical project with those columns, and remap the
> > > top project with new column indexes.
> > >
> > > On the other hand, instead of creating a new intermidiate pogical
> > project,
> > > we can also update ProjectTableScanRule to accept LogicalProject that is
> > > not a simple mapping, and do the same task I mentioned above.
> > >
> > > - Haisheng
> > >
> > > --
> > > 发件人：Rommel Quintanilla
> > > 日 期：2019年10月12日 03:15:31
> > > 收件人：
> > > 主 题：[QUESTION] Pushing up evaluations from LogicalProjects
> > >
> > > Hi, maybe you can help me.
> > > I have this portion from a larger logical plan:
> > > ..
> > > LogicalProject(l_orderkey=[$0], l_suppkey=[$2], *=[*($5, -(1,
> > > $6))])
> > > LogicalTableScan(table=[[main, lineitem]])
> > > ..
> > >
> > > Because the LogicalProject above contains an evaluation, the
> > > ProjectTableScanRule can't convert it to a BindableTableScan.
> > >
> > > I wonder if somehow the evaluation could be pushed up more or less like
> > > this:
> > > ..
> > >  LogicalProject(l_orderkey=[$0], l_suppkey=[$1], *=[*($2, -(1, $3))])
> > >  LogicalProject(l_orderkey=[$0], l_suppkey=[$2], l_extendedprice=[$5],
> > > l_discount=[$6]])
> > >  LogicalTableScan(table=[[main, lineitem]])
> > > ..
> > >
> > > Regards.
> > >
> >
> 

[jira] [Created] (CALCITE-3416) SQL Dialects "DEFAULT"s should be more extensible

[Steven Talbot (Jira)]

Steven Talbot created CALCITE-3416:
--

 Summary: SQL Dialects "DEFAULT"s should be more extensible
 Key: CALCITE-3416
 URL: https://issues.apache.org/jira/browse/CALCITE-3416
 Project: Calcite
  Issue Type: Improvement
Reporter: Steven Talbot


The behavior of SQLDialect is partly governed by the methods defined on the 
given dialect subclass and partly governed by options passed in in a Context 
object. So every dialect subclass exposes a "DEFAULT" instance that has been 
initialized with the correct Context.

However, if you then wish to extend the dialect for one reason or another, you 
must create a new instance to extend, or of course create a whole new subclass. 
In either case, you lose the options from the Context passed into the default, 
which governs important behavior of the dialect. You can copy-paste the 
relevant Context out of the Calcite code, but then you lose future improvements 
or fixes that might land in mainline Calcite.

It would be nice if each dialect exposed the DEFAULT_CONTEXT that it passed 
into its DEFAULT instance as a public final member. Then, when extending the 
dialect, you simply initialize your extension with the DEFAULT_CONTEXT, and if 
any customization needs to happens on the Context options that's easy to do 
with the Context's API.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (CALCITE-3415) Cannot parse REGEXP_SUBSTR in BigQuery

[Pranay Parmar (Jira)]

Pranay Parmar created CALCITE-3415:
--

 Summary: Cannot parse REGEXP_SUBSTR in BigQuery
 Key: CALCITE-3415
 URL: https://issues.apache.org/jira/browse/CALCITE-3415
 Project: Calcite
  Issue Type: Improvement
  Components: core
Reporter: Pranay Parmar
 Fix For: 1.22.0


REGEXP_SUBSTR error :
{code:java}
No match found for function signature REGEXP_SUBSTR(, , 
[, , ]){code}
 

Example query:
{code:sql}
SELECT REGEXP_SUBSTR('chocolate Chip cookies', 'c+.{2}', 1, product_id, 'i')
FROM public.account{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Different performance of the same code in two laptops

[Stamatis Zampetakis]

Hi Makis,

Use a profiler (e.g., VisualVM) and check where the time goes. You can also
compare the snapshots between the two machines.

Best,
Stamatis

On Tue, Oct 15, 2019 at 3:08 PM Michael Mior  wrote:

> I think it's going to be hard for anyone to give you specific advice
> without more information. How are you measuring the runtime? You may
> need to instrument the Calcite code, but I would suggest breaking down
> the runtime more to find out where the slowdown is happening. On a
> related note, the optimization process is single-threaded, so this
> would not be your problem.
> --
> Michael Mior
> mm...@apache.org
>
> Le mar. 15 oct. 2019 à 06:11, Serafeim (Makis) Papadias
>  a écrit :
> >
> > Hello everyone,
> >
> > I am experimenting with rules and volcano optimiser lately and I am
> facing something weird. I run my code through Intellij IDE. The databases
> are set the same way in both machines and since I care only about query
> rewriting for now I just read the table schemas from the databases; I do
> not execute anything.
> >
> > The same code on one machine runs 10 times slower than on the other
> machine.
> >
> > * I tried to rule out that the IDE is not the problem, so I installed
> the same versions and picked the exact same vmoptions.
> > * The database schemas are the same, thus I do not believe it is related
> with the underneath sources.
> >
> > Is it possible that calcite uses all the threads available on one
> machine while running only on thread the other?
> >
> > If you have any recommendations, they are more that welcome!
> >
> > Thanks in advance.
> >
> > Best,
> > Makis
>

Re: [DISCUSS] Automated security fixes via dependabot

[Michael Mior]

Creating a JIRA shouldn't be too painful making use of GitHub Actions.
We should be able to trigger this whenever a dependabot PR is created.
It does add an extra dependency on GitHub, but if we're using
dependabot, we have that anyway.
--
Michael Mior
mm...@apache.org

Le lun. 14 oct. 2019 à 13:36, Vladimir Sitnikov
 a écrit :
>
> I guess the missing bit is the bot that automatically creates JIRA for
> Dependabot issues.
>
> What if we create one?
>
> Vladimir

Re: [QUESTION] Pushing up evaluations from LogicalProjects

[XING JIN]

Hi Rommel ~
I'm very happy you tried the patch ~
Well the stacktrace you provided  seems outdated ~
I updated the the code several hours ago. The current commit id is
05d0e2d9fbadec060e54c622824b3725df36aab0
Could you please try the case again and send me the exception and related
information ?
Or you can just reply on https://github.com/apache/calcite/pull/1500 or
https://issues.apache.org/jira/browse/CALCITE-3405
I will try to fix ASAP

Best,
Jin


Rommel Quintanilla  于2019年10月15日周二 下午7:22写道：

> Hi Jin, your contribution is awesome! thanks.
>
> I tested it and works like a charm in most cases. Definitely clearer than
> the approach that I was trying. However, on the application I'm working on,
> I found a weird issue that I wasn't able to reproduce as a unit test on the
> calcite-core project. Btw, I'm using your changes after Haisheng's revision.
>
> So, it is as follows, I have this simple query: select l_extendedprice-1
> from lineitem
> where the type of l_extendedprice is DOUBLE. Before applying
> ProjectTableScanRule the logical plan is:
>
> LogicalProject(EXPR$0=[-($5, 1)])
>   LogicalTableScan(table=[[main, lineitem]])
>
> So far all is fine, but when the ProjectTableScanRule was applied I got:
>
> java.lang.AssertionError: type mismatch:
> ref:
> DOUBLE
> input:
> BIGINT
> at org.apache.calcite.util.Litmus$1.fail(Litmus.java:31)
> at org.apache.calcite.plan.RelOptUtil.eq(RelOptUtil.java:2000)
> at
> org.apache.calcite.rex.RexChecker.visitInputRef(RexChecker.java:125)
> at
> org.apache.calcite.rex.RexChecker.visitInputRef(RexChecker.java:57)
> at org.apache.calcite.rex.RexInputRef.accept(RexInputRef.java:112)
> at org.apache.calcite.rex.RexChecker.visitCall(RexChecker.java:140)
> at org.apache.calcite.rex.RexChecker.visitCall(RexChecker.java:57)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:191)
> at org.apache.calcite.rel.core.Project.isValid(Project.java:192)
> at org.apache.calcite.rel.core.Project.(Project.java:83)
> at
> org.apache.calcite.rel.logical.LogicalProject.(LogicalProject.java:62)
> at
> org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:112)
> at
> org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:100)
> at
> org.apache.calcite.rel.core.RelFactories$ProjectFactoryImpl.createProject(RelFactories.java:158)
> at
> org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1414)
> at
> org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1252)
> at
> org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1241)
> at
> com.blazingdb.calcite.rules.ProjectTableScanRule.apply(ProjectTableScanRule.java:147)
> at
> com.blazingdb.calcite.rules.ProjectTableScanRule$1.onMatch(ProjectTableScanRule.java:65)
> at
> org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:319)
> at
> org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:560)
> at
> org.apache.calcite.plan.hep.HepPlanner.depthFirstApply(HepPlanner.java:374)
> at
> org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:436)
> at
> org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:256)
> at
> org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:127)
> at
> org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:215)
> at
> org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:202)
> at
> com.blazingdb.calcite.application.RelationalAlgebraGenerator.getOptimizedRelationalAlgebra(RelationalAlgebraGenerator.java:202)
> at
> com.blazingdb.calcite.catalog.BlazingRulesTest.generateSQLTest(BlazingRulesTest.java:274)
>
> Any idea why could this be happening?
>
> Thank you in advance.
>
> On 2019/10/12 09:12:20, XING JIN  wrote:
> > Hi Stamatis,
> > In current code, BindableTableScan is only created by rules of
> > ProjectTableScanRule and FilterTableScanRule. I think it's better to keep
> > as it is?
> > I made a PR for CALCITE-3405  --
> https://github.com/apache/calcite/pull/1500
> >
> > The idea of the PR is quite straightforward:
> > 1. Analyze the parent Project -- collect all the needed fields;
> > 2. Column pruning by pushing down the needed fields to BindableTableScan;
> > 3. Adjust RexInputRefs in parent Project
> >
> > @Haisheng @Stamatis It would be great if you can give a review when you
> > have time ~~~ Thanks a lot !
> >
> > Best,
> > Jin
> >
> >
> > Stamatis Zampetakis  于2019年10月12日周六 下午3:00写道：
> >
> > > Hi Rommel,
> > >
> > > I was hoping that this could be done at least by RelFieldTrimmer [1].
> Are
> > > you using it already?
> > >
> > > Best,
> > > Stamatis
> > >
> > > [1]
> > >
> > >
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java
> > 

Re: Different performance of the same code in two laptops

[Michael Mior]

I think it's going to be hard for anyone to give you specific advice
without more information. How are you measuring the runtime? You may
need to instrument the Calcite code, but I would suggest breaking down
the runtime more to find out where the slowdown is happening. On a
related note, the optimization process is single-threaded, so this
would not be your problem.
--
Michael Mior
mm...@apache.org

Le mar. 15 oct. 2019 à 06:11, Serafeim (Makis) Papadias
 a écrit :
>
> Hello everyone,
>
> I am experimenting with rules and volcano optimiser lately and I am facing 
> something weird. I run my code through Intellij IDE. The databases are set 
> the same way in both machines and since I care only about query rewriting for 
> now I just read the table schemas from the databases; I do not execute 
> anything.
>
> The same code on one machine runs 10 times slower than on the other machine.
>
> * I tried to rule out that the IDE is not the problem, so I installed the 
> same versions and picked the exact same vmoptions.
> * The database schemas are the same, thus I do not believe it is related with 
> the underneath sources.
>
> Is it possible that calcite uses all the threads available on one machine 
> while running only on thread the other?
>
> If you have any recommendations, they are more that welcome!
>
> Thanks in advance.
>
> Best,
> Makis

[jira] [Created] (CALCITE-3414) Unify Expression'type cast and conversion as a robust one

[Feng Zhu (Jira)]

Feng Zhu created CALCITE-3414:
-

 Summary: Unify Expression'type cast and conversion as a robust one
 Key: CALCITE-3414
 URL: https://issues.apache.org/jira/browse/CALCITE-3414
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.21.0
Reporter: Feng Zhu
Assignee: Feng Zhu
 Attachments: RexToLixTranslator.png, TypeConversion.txt, Types.png

 Current now, there are two functions in calcite that can be used to 
cast/convert Expression to a specific Type.
 *_Types.castIfNecessary_* and _*RexToLixTranslator.convert*_.

We make a deep investigation on their implementations and demonstrate them as 
below.

{color:#ff} 
    !RexToLixTranslator.png!{color}

{color:#ff} 
 *RexToLixTranslator.convert*{color}

  !Types.png!

   {color:#ff}  
 *Types.castIfNecessary*{color}

It can be seen that: 
 (1) They have a lot of overlaps; 
 (2) *_RexToLixTranslator.cast_* can cover more cases with tools like 
_SqlFunctions_ and etc.
 (3) Both of them have limitations and may generate incorrect code, which is 
listed in attachment(TypeConversion.txt).

Multiple choices usually bring confusion to developers and resulting to the 
misuse of them. 
 For example, CALCITE-3245 exposes that Types.castIfNecessary cannot cast the 
Expression to BigDecimal.class.


 Fixing the issue in *_Types.castIfNecessary_* directly seems to be not a good 
idea. 
 On one hand, it is not convenient to call _SqlFunctions_ in linq4j. One the 
other hand, it will brings duplicate with _*RexToLixTranslator.cast*_. However, 
due to some unique logic in _*Types.castIfNecessary*_, we cannot replace it as 
_*RexToLixTranslator.cast*_ neither.

Therefore, it is a good idea to integrate implementations into 
RexToLixTranslator.cast.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [QUESTION] Pushing up evaluations from LogicalProjects

[Rommel Quintanilla]

Hi Jin, your contribution is awesome! thanks.

I tested it and works like a charm in most cases. Definitely clearer than the 
approach that I was trying. However, on the application I'm working on, I found 
a weird issue that I wasn't able to reproduce as a unit test on the 
calcite-core project. Btw, I'm using your changes after Haisheng's revision.

So, it is as follows, I have this simple query: select l_extendedprice-1 from 
lineitem
where the type of l_extendedprice is DOUBLE. Before applying 
ProjectTableScanRule the logical plan is:

LogicalProject(EXPR$0=[-($5, 1)])
  LogicalTableScan(table=[[main, lineitem]])

So far all is fine, but when the ProjectTableScanRule was applied I got:

java.lang.AssertionError: type mismatch:
ref:
DOUBLE
input:
BIGINT
at org.apache.calcite.util.Litmus$1.fail(Litmus.java:31)
at org.apache.calcite.plan.RelOptUtil.eq(RelOptUtil.java:2000)
at org.apache.calcite.rex.RexChecker.visitInputRef(RexChecker.java:125)
at org.apache.calcite.rex.RexChecker.visitInputRef(RexChecker.java:57)
at org.apache.calcite.rex.RexInputRef.accept(RexInputRef.java:112)
at org.apache.calcite.rex.RexChecker.visitCall(RexChecker.java:140)
at org.apache.calcite.rex.RexChecker.visitCall(RexChecker.java:57)
at org.apache.calcite.rex.RexCall.accept(RexCall.java:191)
at org.apache.calcite.rel.core.Project.isValid(Project.java:192)
at org.apache.calcite.rel.core.Project.(Project.java:83)
at 
org.apache.calcite.rel.logical.LogicalProject.(LogicalProject.java:62)
at 
org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:112)
at 
org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:100)
at 
org.apache.calcite.rel.core.RelFactories$ProjectFactoryImpl.createProject(RelFactories.java:158)
at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1414)
at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1252)
at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1241)
at 
com.blazingdb.calcite.rules.ProjectTableScanRule.apply(ProjectTableScanRule.java:147)
at 
com.blazingdb.calcite.rules.ProjectTableScanRule$1.onMatch(ProjectTableScanRule.java:65)
at 
org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:319)
at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:560)
at 
org.apache.calcite.plan.hep.HepPlanner.depthFirstApply(HepPlanner.java:374)
at 
org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:436)
at 
org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:256)
at 
org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:127)
at 
org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:215)
at 
org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:202)
at 
com.blazingdb.calcite.application.RelationalAlgebraGenerator.getOptimizedRelationalAlgebra(RelationalAlgebraGenerator.java:202)
at 
com.blazingdb.calcite.catalog.BlazingRulesTest.generateSQLTest(BlazingRulesTest.java:274)

Any idea why could this be happening?

Thank you in advance.

On 2019/10/12 09:12:20, XING JIN  wrote: 
> Hi Stamatis,
> In current code, BindableTableScan is only created by rules of
> ProjectTableScanRule and FilterTableScanRule. I think it's better to keep
> as it is?
> I made a PR for CALCITE-3405  -- https://github.com/apache/calcite/pull/1500
> 
> The idea of the PR is quite straightforward:
> 1. Analyze the parent Project -- collect all the needed fields;
> 2. Column pruning by pushing down the needed fields to BindableTableScan;
> 3. Adjust RexInputRefs in parent Project
> 
> @Haisheng @Stamatis It would be great if you can give a review when you
> have time ~~~ Thanks a lot !
> 
> Best,
> Jin
> 
> 
> Stamatis Zampetakis  于2019年10月12日周六 下午3:00写道：
> 
> > Hi Rommel,
> >
> > I was hoping that this could be done at least by RelFieldTrimmer [1]. Are
> > you using it already?
> >
> > Best,
> > Stamatis
> >
> > [1]
> >
> > https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java
> >
> > On Sat, Oct 12, 2019 at 6:06 AM XING JIN  wrote:
> >
> > > Filed a JIRA:
> > > https://issues.apache.org/jira/browse/CALCITE-3405
> > >
> > > Haisheng Yuan  于2019年10月12日周六 上午4:34写道：
> > >
> > > > Yes, definitely.
> > > >
> > > > You can go through the project expression with InputFinder to find all
> > > the
> > > > used columns, create a  logical project with those columns, and remap
> > the
> > > > top project with new column indexes.
> > > >
> > > > On the other hand, instead of creating a new intermidiate pogical
> > > project,
> > > > we can also update ProjectTableScanRule to accept LogicalProject that
> > is
> > > > not a simple mapping, and do the same task I mentioned 

Different performance of the same code in two laptops

[Serafeim (Makis) Papadias]

Hello everyone,

I am experimenting with rules and volcano optimiser lately and I am facing 
something weird. I run my code through Intellij IDE. The databases are set the 
same way in both machines and since I care only about query rewriting for now I 
just read the table schemas from the databases; I do not execute anything.

The same code on one machine runs 10 times slower than on the other machine.

* I tried to rule out that the IDE is not the problem, so I installed the same 
versions and picked the exact same vmoptions.
* The database schemas are the same, thus I do not believe it is related with 
the underneath sources.

Is it possible that calcite uses all the threads available on one machine while 
running only on thread the other?

If you have any recommendations, they are more that welcome!

Thanks in advance.

Best,
Makis

[jira] [Created] (CALCITE-3413) AssertionError for interpertering union/intersect/minus with null value

[Wang Yanlin (Jira)]

Wang Yanlin created CALCITE-3413:


 Summary: AssertionError for interpertering union/intersect/minus 
with null value
 Key: CALCITE-3413
 URL: https://issues.apache.org/jira/browse/CALCITE-3413
 Project: Calcite
  Issue Type: New Feature
Reporter: Wang Yanlin


Add the following test case in *InterpreterTest*

{code:java}
@Test public void testInterpretUnionWithNullValue() throws Exception {
final String sql = "select * from\n"
+ "(select x, y from (values (cast(NULL as int), cast(NULL as 
varchar(1))),\n"
+ "(cast(NULL as int), cast(NULL as varchar(1 as t(x, y))\n"
+ "union\n"
+ "(select x, y from (values (cast(NULL as int), cast(NULL as 
varchar(1 as t2(x, y))\n";
SqlNode validate = planner.validate(planner.parse(sql));
RelNode convert = planner.rel(validate).rel;
final Interpreter interpreter = new Interpreter(dataContext, convert);
assertRows(interpreter, "[null, null]");
  }

  @Test public void testInterpretUnionAllWithNullValue() throws Exception {
final String sql = "select * from\n"
+ "(select x, y from (values (cast(NULL as int), cast(NULL as 
varchar(1))),\n"
+ "(cast(NULL as int), cast(NULL as varchar(1 as t(x, y))\n"
+ "union all\n"
+ "(select x, y from (values (cast(NULL as int), cast(NULL as 
varchar(1 as t2(x, y))\n";
SqlNode validate = planner.validate(planner.parse(sql));
RelNode convert = planner.rel(validate).rel;
final Interpreter interpreter = new Interpreter(dataContext, convert);
assertRows(interpreter, "[null, null]", "[null, null]", "[null, null]");
  }
{code}

got

{code:java}
java.lang.AssertionError
at 
org.apache.calcite.interpreter.Interpreter$CompilerImpl.source(Interpreter.java:510)
at 
org.apache.calcite.interpreter.Nodes$CoreCompiler.source(Nodes.java:46)
at 
org.apache.calcite.interpreter.AbstractSingleNode.(AbstractSingleNode.java:33)
at 
org.apache.calcite.interpreter.ProjectNode.(ProjectNode.java:31)
at 
org.apache.calcite.interpreter.Nodes$CoreCompiler.visit(Nodes.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.calcite.util.ReflectUtil.invokeVisitorInternal(ReflectUtil.java:257)
at 
org.apache.calcite.util.ReflectUtil.invokeVisitor(ReflectUtil.java:214)
at 
org.apache.calcite.util.ReflectUtil$1.invokeVisitor(ReflectUtil.java:464)
at 
org.apache.calcite.interpreter.Interpreter$CompilerImpl.visit(Interpreter.java:451)
at 
org.apache.calcite.interpreter.Nodes$CoreCompiler.visit(Nodes.java:46)
at 
org.apache.calcite.rel.AbstractRelNode.childrenAccept(AbstractRelNode.java:265)
at 
org.apache.calcite.interpreter.Interpreter$CompilerImpl.visit(Interpreter.java:447)
at 
org.apache.calcite.interpreter.Nodes$CoreCompiler.visit(Nodes.java:46)
at 
org.apache.calcite.interpreter.Interpreter$CompilerImpl.visitRoot(Interpreter.java:405)
at 
org.apache.calcite.interpreter.Interpreter.(Interpreter.java:88)
at 
org.apache.calcite.test.InterpreterTest.testInterpretMinusWithNullValue(InterpreterTest.java:418)
{code}

The cause lies in the optimize of 
[Interpreter|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/interpreter/Interpreter.java#L85]
before optimize, the relnode tree is as follows, no renode with same id exists

{code:java}
LogicalUnion(all=[false]), id = 19
  LogicalProject(X=[$0], Y=[$1]), id = 16
LogicalUnion(all=[true]), id = 15
  LogicalProject(EXPR$0=[null:INTEGER], EXPR$1=[null:VARCHAR(1)]), id = 12
LogicalValues(tuples=[[{ 0 }]]), id = 11
  LogicalProject(EXPR$0=[null:INTEGER], EXPR$1=[null:VARCHAR(1)]), id = 14
LogicalValues(tuples=[[{ 0 }]]), id = 13
  LogicalProject(X=[null:INTEGER], Y=[null:VARCHAR(1)]), id = 18
LogicalValues(tuples=[[{ 0 }]]), id = 17
{code}

after optimize, the relnode tree is as follows,

{code:java}
LogicalUnion(all=[false]), id = 30
  LogicalProject(X=[$0], Y=[$1]), id = 26
LogicalUnion(all=[true]), id = 24
  LogicalProject(EXPR$0=[null:INTEGER], EXPR$1=[null:VARCHAR(1)]), id = 21
LogicalValues(tuples=[[{ 0 }]]), id = 11
  LogicalProject(EXPR$0=[null:INTEGER], EXPR$1=[null:VARCHAR(1)]), id = 21
LogicalValues(tuples=[[{ 0 }]]), id = 11
  LogicalProject(X=[null:INTEGER], Y=[null:VARCHAR(1)]), id = 28
LogicalValues(tuples=[[{ 0 }]]), id = 11
{code}

there exists relnodes with same id, thus breaking the inner structure of