Re: [DISCUSS] Automated security fixes via dependabot

2019-10-12 Thread Julian Hyde
I’ve not looked at the PRs but they sound useful. Keeping software secure these 
days is a moving target; we have to do work just to keep up. All of our 
dependencies are doing that work too, and so we need to keep up to date with 
them.

I think it would be useful to have a task before each release to merge in the 
ones that look important.

If you are concerned that this will result in many “small” changes with no JIRA 
case number, the person merging the PRs could squash them into a single commit 
with its own JIRA case. (I’d use 'git cherry-pick … ; git cherry-pick ; git 
rebase -i origin/master’)

Julian


> On Oct 12, 2019, at 1:01 PM, Muhammad Gelbana  wrote:
> 
> Why would we not merge those PRs or even disable the whole thing ?
> 
> 
> 
> On Fri, Oct 11, 2019 at 12:09 AM Francis Chuang 
> wrote:
> 
>> Dependabot is a bot on Github that opens PRs to automatically upgrade
>> out of date dependencies to fix security issues. Recently, Github
>> acquired dependabot and is gradually enabling the bot on all repositories.
>> 
>> It just opened a PR to upgrade a few dependencies in the Avatica
>> repository: https://github.com/apache/calcite-avatica/pull/114
>> 
>> I'd like to start some discussion as to how we should deal with these
>> PRs. For some background, dependency upgrades should usually have a jira
>> issue number assigned, so that the change is fully trackable. We
>> recently had some discussion regarding trivial fixes to documentation
>> and the consensus was that changes to the code is not considered to be
>> trivial and that an issue should be filed on jira.
>> 
>> If we will not merge these PRs, I think it makes sense to ask infra to
>> disable them. Having these open PRs and then closing them manually seem
>> to generate a lot of noise. According to the documentation for
>> dependabot [1] it appears that we can either opt out of having
>> dependabot opening PRs completely or have it open PRs. There is no
>> middle-ground where dependabot/Github sends members of the repo a
>> notification for security issues, but do not open any PRs.
>> 
>> What do you guys think?
>> 
>> Francis
>> 
>> [1]
>> https://help.github.com/en/articles/configuring-automated-security-fixes
>> 



Re: Re: Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-12 Thread Stamatis Zampetakis
I was thinking that RelFieldTrimmer can be used to transform the plan P1 to
plan P2 and then ProjectTableScanRule can be used to introduce the
BindableTableScan.

The consumer in the case of P1 is the project which only needs $0, $2, $5,
$6 so the trimmer could slim down the scan by projecting only these fields.
I didn't test if it really does this in this case because in some sense the
trimming is a bit redundant given that the project just on top has just a
subset of the columns of the table.

Note that RelFieldTrimmer could be used also during optimization (not only
in sql to rel phase) as a separate planning phase.

*P1*
LogicalProject(l_orderkey=[$0], l_suppkey=[$2], *=[*($5, -(1, $6))])
  LogicalTableScan(table=[[main, lineitem]])

*P2*
LogicalProject(l_orderkey=[$0], l_suppkey=[$1], *=[*($2, -(1, $3))])
  LogicalProject(l_orderkey=[$0], l_suppkey=[$2], l_extendedprice=[$5],
l_discount=[$6]])
LogicalTableScan(table=[[main, lineitem]])


On Sat, Oct 12, 2019 at 1:11 PM Wang Yanlin <1989yanlinw...@163.com> wrote:

> Just a little curious,
> RelFieldTrimmer is used to 'slimmed down' relational expression that
> projects only the columns required by its consumer.
> How can it used to do this thing?
>
>
>
>
> --
>
> Best,
> Wang Yanlin
>
>
>
> 在 2019-10-12 18:03:33,"XING JIN"  写道:
> >Sure we can ~
> >If we use BindableTableScanRule to derive a BindableTableScan from
> >ProjectableFilterableTable, that would happen during a stage of
> >optimization run by RelOptPlanner. But RelFieldTrimmer works right during
> >conversion of Sql to Rel.
> >
> >Wang Yanlin <1989yanlinw...@163.com> 于2019年10月12日周六 下午5:56写道:
> >
> >> Can we just use  Bindables.BINDABLE_TABLE_SCAN_RULE to translate  the
> >> table scan to BindableTableScan ?
> >>
> >>
> >>
> >> --
> >>
> >> Best,
> >> Wang Yanlin
> >>
> >>
> >>
> >> At 2019-10-12 17:12:20, "XING JIN"  wrote:
> >> >Hi Stamatis,
> >> >In current code, BindableTableScan is only created by rules of
> >> >ProjectTableScanRule and FilterTableScanRule. I think it's better to
> keep
> >> >as it is?
> >> >I made a PR for CALCITE-3405  --
> >> https://github.com/apache/calcite/pull/1500
> >> >
> >> >The idea of the PR is quite straightforward:
> >> >1. Analyze the parent Project -- collect all the needed fields;
> >> >2. Column pruning by pushing down the needed fields to
> BindableTableScan;
> >> >3. Adjust RexInputRefs in parent Project
> >> >
> >> >@Haisheng @Stamatis It would be great if you can give a review when you
> >> >have time ~~~ Thanks a lot !
> >> >
> >> >Best,
> >> >Jin
> >> >
> >> >
> >> >Stamatis Zampetakis  于2019年10月12日周六 下午3:00写道:
> >> >
> >> >> Hi Rommel,
> >> >>
> >> >> I was hoping that this could be done at least by RelFieldTrimmer [1].
> >> Are
> >> >> you using it already?
> >> >>
> >> >> Best,
> >> >> Stamatis
> >> >>
> >> >> [1]
> >> >>
> >> >>
> >>
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java
> >> >>
> >> >> On Sat, Oct 12, 2019 at 6:06 AM XING JIN 
> >> wrote:
> >> >>
> >> >> > Filed a JIRA:
> >> >> > https://issues.apache.org/jira/browse/CALCITE-3405
> >> >> >
> >> >> > Haisheng Yuan  于2019年10月12日周六 上午4:34写道:
> >> >> >
> >> >> > > Yes, definitely.
> >> >> > >
> >> >> > > You can go through the project expression with InputFinder to
> find
> >> all
> >> >> > the
> >> >> > > used columns, create a  logical project with those columns, and
> >> remap
> >> >> the
> >> >> > > top project with new column indexes.
> >> >> > >
> >> >> > > On the other hand, instead of creating a new intermidiate pogical
> >> >> > project,
> >> >> > > we can also update ProjectTableScanRule to accept LogicalProject
> >> that
> >> >> is
> >> >> > > not a simple mapping, and do the same task I mentioned above.
> >> >> > >
> >> >> > > - Haisheng
> >> >> > >
> >> >> > >
> --
> >> >> > > 发件人:Rommel Quintanilla
> >> >> > > 日 期:2019年10月12日 03:15:31
> >> >> > > 收件人:
> >> >> > > 主 题:[QUESTION] Pushing up evaluations from LogicalProjects
> >> >> > >
> >> >> > > Hi, maybe you can help me.
> >> >> > > I have this portion from a larger logical plan:
> >> >> > > ..
> >> >> > > LogicalProject(l_orderkey=[$0], l_suppkey=[$2], *=[*($5,
> >> -(1,
> >> >> > > $6))])
> >> >> > > LogicalTableScan(table=[[main, lineitem]])
> >> >> > > ..
> >> >> > >
> >> >> > > Because the LogicalProject above contains an evaluation, the
> >> >> > > ProjectTableScanRule can't convert it to a BindableTableScan.
> >> >> > >
> >> >> > > I wonder if somehow the evaluation could be pushed up more or
> less
> >> like
> >> >> > > this:
> >> >> > > ..
> >> >> > >  LogicalProject(l_orderkey=[$0], l_suppkey=[$1], *=[*($2, -(1,
> >> $3))])
> >> >> > >  LogicalProject(l_orderkey=[$0], l_suppkey=[$2],
> >> l_extendedprice=[$5],
> >> >> > > l_discount=[$6]])
> >> >> > >  LogicalTableScan(table=[[main,
> lineitem]])
> >> >> > > ..
> >> >> > >
> >> >> > > 

Re: Apache Calcite Geode

2019-10-12 Thread Andrei Sereda
Hi Viquar Khan,

Currently write operations (UPDATE / DELETE / INSERT) are not supported by
geode-calcite adapter. You need to load data using native API then query it
using Calcite.

There is some documentation on our website:
https://calcite.apache.org/docs/geode_adapter.html but it is by no means
comprehensive. Let me know if you'd like to improve it and we'll happily
accept your contribution.

Regards,
Andrei.






On Fri, Oct 11, 2019 at 10:04 PM vaquar khan  wrote:

> Hi All,
>
> I am looking good examples or document /tutorials/sample code for Apache
> Calcite Geode,unfortunately noting is available on net , only one  talk
> from  tzolv.
>
> I have raised  following question few month back and now able to connect
> now  .
>
>
> https://stackoverflow.com/questions/49411169/apache-calcite-geode-jdbc-adapte-not-working-with-gemfire-8-x-and-9-x/58343449#58343449
>
>
> I am try to use Apache spark Job to load data on gemfire using Apache
> Calcite Geode adopter .
> I am also interested to create good reference for other users they are
> facing same problems .
>
> Any help greatly appreciated.
>
> Regards,
> Viquar khan
>


Re: [DISCUSS] Automated security fixes via dependabot

2019-10-12 Thread Muhammad Gelbana
Why would we not merge those PRs or even disable the whole thing ?



On Fri, Oct 11, 2019 at 12:09 AM Francis Chuang 
wrote:

> Dependabot is a bot on Github that opens PRs to automatically upgrade
> out of date dependencies to fix security issues. Recently, Github
> acquired dependabot and is gradually enabling the bot on all repositories.
>
> It just opened a PR to upgrade a few dependencies in the Avatica
> repository: https://github.com/apache/calcite-avatica/pull/114
>
> I'd like to start some discussion as to how we should deal with these
> PRs. For some background, dependency upgrades should usually have a jira
> issue number assigned, so that the change is fully trackable. We
> recently had some discussion regarding trivial fixes to documentation
> and the consensus was that changes to the code is not considered to be
> trivial and that an issue should be filed on jira.
>
> If we will not merge these PRs, I think it makes sense to ask infra to
> disable them. Having these open PRs and then closing them manually seem
> to generate a lot of noise. According to the documentation for
> dependabot [1] it appears that we can either opt out of having
> dependabot opening PRs completely or have it open PRs. There is no
> middle-ground where dependabot/Github sends members of the repo a
> notification for security issues, but do not open any PRs.
>
> What do you guys think?
>
> Francis
>
> [1]
> https://help.github.com/en/articles/configuring-automated-security-fixes
>


Re:Re: Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-12 Thread Wang Yanlin
Just a little curious,  
RelFieldTrimmer is used to 'slimmed down' relational expression that projects 
only the columns required by its consumer.
How can it used to do this thing?




--

Best,
Wang Yanlin



在 2019-10-12 18:03:33,"XING JIN"  写道:
>Sure we can ~
>If we use BindableTableScanRule to derive a BindableTableScan from
>ProjectableFilterableTable, that would happen during a stage of
>optimization run by RelOptPlanner. But RelFieldTrimmer works right during
>conversion of Sql to Rel.
>
>Wang Yanlin <1989yanlinw...@163.com> 于2019年10月12日周六 下午5:56写道:
>
>> Can we just use  Bindables.BINDABLE_TABLE_SCAN_RULE to translate  the
>> table scan to BindableTableScan ?
>>
>>
>>
>> --
>>
>> Best,
>> Wang Yanlin
>>
>>
>>
>> At 2019-10-12 17:12:20, "XING JIN"  wrote:
>> >Hi Stamatis,
>> >In current code, BindableTableScan is only created by rules of
>> >ProjectTableScanRule and FilterTableScanRule. I think it's better to keep
>> >as it is?
>> >I made a PR for CALCITE-3405  --
>> https://github.com/apache/calcite/pull/1500
>> >
>> >The idea of the PR is quite straightforward:
>> >1. Analyze the parent Project -- collect all the needed fields;
>> >2. Column pruning by pushing down the needed fields to BindableTableScan;
>> >3. Adjust RexInputRefs in parent Project
>> >
>> >@Haisheng @Stamatis It would be great if you can give a review when you
>> >have time ~~~ Thanks a lot !
>> >
>> >Best,
>> >Jin
>> >
>> >
>> >Stamatis Zampetakis  于2019年10月12日周六 下午3:00写道:
>> >
>> >> Hi Rommel,
>> >>
>> >> I was hoping that this could be done at least by RelFieldTrimmer [1].
>> Are
>> >> you using it already?
>> >>
>> >> Best,
>> >> Stamatis
>> >>
>> >> [1]
>> >>
>> >>
>> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java
>> >>
>> >> On Sat, Oct 12, 2019 at 6:06 AM XING JIN 
>> wrote:
>> >>
>> >> > Filed a JIRA:
>> >> > https://issues.apache.org/jira/browse/CALCITE-3405
>> >> >
>> >> > Haisheng Yuan  于2019年10月12日周六 上午4:34写道:
>> >> >
>> >> > > Yes, definitely.
>> >> > >
>> >> > > You can go through the project expression with InputFinder to find
>> all
>> >> > the
>> >> > > used columns, create a  logical project with those columns, and
>> remap
>> >> the
>> >> > > top project with new column indexes.
>> >> > >
>> >> > > On the other hand, instead of creating a new intermidiate pogical
>> >> > project,
>> >> > > we can also update ProjectTableScanRule to accept LogicalProject
>> that
>> >> is
>> >> > > not a simple mapping, and do the same task I mentioned above.
>> >> > >
>> >> > > - Haisheng
>> >> > >
>> >> > > --
>> >> > > 发件人:Rommel Quintanilla
>> >> > > 日 期:2019年10月12日 03:15:31
>> >> > > 收件人:
>> >> > > 主 题:[QUESTION] Pushing up evaluations from LogicalProjects
>> >> > >
>> >> > > Hi, maybe you can help me.
>> >> > > I have this portion from a larger logical plan:
>> >> > > ..
>> >> > > LogicalProject(l_orderkey=[$0], l_suppkey=[$2], *=[*($5,
>> -(1,
>> >> > > $6))])
>> >> > > LogicalTableScan(table=[[main, lineitem]])
>> >> > > ..
>> >> > >
>> >> > > Because the LogicalProject above contains an evaluation, the
>> >> > > ProjectTableScanRule can't convert it to a BindableTableScan.
>> >> > >
>> >> > > I wonder if somehow the evaluation could be pushed up more or less
>> like
>> >> > > this:
>> >> > > ..
>> >> > >  LogicalProject(l_orderkey=[$0], l_suppkey=[$1], *=[*($2, -(1,
>> $3))])
>> >> > >  LogicalProject(l_orderkey=[$0], l_suppkey=[$2],
>> l_extendedprice=[$5],
>> >> > > l_discount=[$6]])
>> >> > >  LogicalTableScan(table=[[main, lineitem]])
>> >> > > ..
>> >> > >
>> >> > > Regards.
>> >> > >
>> >> >
>> >>
>>


Re: Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-12 Thread XING JIN
Sure we can ~
If we use BindableTableScanRule to derive a BindableTableScan from
ProjectableFilterableTable, that would happen during a stage of
optimization run by RelOptPlanner. But RelFieldTrimmer works right during
conversion of Sql to Rel.

Wang Yanlin <1989yanlinw...@163.com> 于2019年10月12日周六 下午5:56写道:

> Can we just use  Bindables.BINDABLE_TABLE_SCAN_RULE to translate  the
> table scan to BindableTableScan ?
>
>
>
> --
>
> Best,
> Wang Yanlin
>
>
>
> At 2019-10-12 17:12:20, "XING JIN"  wrote:
> >Hi Stamatis,
> >In current code, BindableTableScan is only created by rules of
> >ProjectTableScanRule and FilterTableScanRule. I think it's better to keep
> >as it is?
> >I made a PR for CALCITE-3405  --
> https://github.com/apache/calcite/pull/1500
> >
> >The idea of the PR is quite straightforward:
> >1. Analyze the parent Project -- collect all the needed fields;
> >2. Column pruning by pushing down the needed fields to BindableTableScan;
> >3. Adjust RexInputRefs in parent Project
> >
> >@Haisheng @Stamatis It would be great if you can give a review when you
> >have time ~~~ Thanks a lot !
> >
> >Best,
> >Jin
> >
> >
> >Stamatis Zampetakis  于2019年10月12日周六 下午3:00写道:
> >
> >> Hi Rommel,
> >>
> >> I was hoping that this could be done at least by RelFieldTrimmer [1].
> Are
> >> you using it already?
> >>
> >> Best,
> >> Stamatis
> >>
> >> [1]
> >>
> >>
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java
> >>
> >> On Sat, Oct 12, 2019 at 6:06 AM XING JIN 
> wrote:
> >>
> >> > Filed a JIRA:
> >> > https://issues.apache.org/jira/browse/CALCITE-3405
> >> >
> >> > Haisheng Yuan  于2019年10月12日周六 上午4:34写道:
> >> >
> >> > > Yes, definitely.
> >> > >
> >> > > You can go through the project expression with InputFinder to find
> all
> >> > the
> >> > > used columns, create a  logical project with those columns, and
> remap
> >> the
> >> > > top project with new column indexes.
> >> > >
> >> > > On the other hand, instead of creating a new intermidiate pogical
> >> > project,
> >> > > we can also update ProjectTableScanRule to accept LogicalProject
> that
> >> is
> >> > > not a simple mapping, and do the same task I mentioned above.
> >> > >
> >> > > - Haisheng
> >> > >
> >> > > --
> >> > > 发件人:Rommel Quintanilla
> >> > > 日 期:2019年10月12日 03:15:31
> >> > > 收件人:
> >> > > 主 题:[QUESTION] Pushing up evaluations from LogicalProjects
> >> > >
> >> > > Hi, maybe you can help me.
> >> > > I have this portion from a larger logical plan:
> >> > > ..
> >> > > LogicalProject(l_orderkey=[$0], l_suppkey=[$2], *=[*($5,
> -(1,
> >> > > $6))])
> >> > > LogicalTableScan(table=[[main, lineitem]])
> >> > > ..
> >> > >
> >> > > Because the LogicalProject above contains an evaluation, the
> >> > > ProjectTableScanRule can't convert it to a BindableTableScan.
> >> > >
> >> > > I wonder if somehow the evaluation could be pushed up more or less
> like
> >> > > this:
> >> > > ..
> >> > >  LogicalProject(l_orderkey=[$0], l_suppkey=[$1], *=[*($2, -(1,
> $3))])
> >> > >  LogicalProject(l_orderkey=[$0], l_suppkey=[$2],
> l_extendedprice=[$5],
> >> > > l_discount=[$6]])
> >> > >  LogicalTableScan(table=[[main, lineitem]])
> >> > > ..
> >> > >
> >> > > Regards.
> >> > >
> >> >
> >>
>


Re:Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-12 Thread Wang Yanlin
Can we just use  Bindables.BINDABLE_TABLE_SCAN_RULE to translate  the table 
scan to BindableTableScan ?



--

Best,
Wang Yanlin



At 2019-10-12 17:12:20, "XING JIN"  wrote:
>Hi Stamatis,
>In current code, BindableTableScan is only created by rules of
>ProjectTableScanRule and FilterTableScanRule. I think it's better to keep
>as it is?
>I made a PR for CALCITE-3405  -- https://github.com/apache/calcite/pull/1500
>
>The idea of the PR is quite straightforward:
>1. Analyze the parent Project -- collect all the needed fields;
>2. Column pruning by pushing down the needed fields to BindableTableScan;
>3. Adjust RexInputRefs in parent Project
>
>@Haisheng @Stamatis It would be great if you can give a review when you
>have time ~~~ Thanks a lot !
>
>Best,
>Jin
>
>
>Stamatis Zampetakis  于2019年10月12日周六 下午3:00写道:
>
>> Hi Rommel,
>>
>> I was hoping that this could be done at least by RelFieldTrimmer [1]. Are
>> you using it already?
>>
>> Best,
>> Stamatis
>>
>> [1]
>>
>> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java
>>
>> On Sat, Oct 12, 2019 at 6:06 AM XING JIN  wrote:
>>
>> > Filed a JIRA:
>> > https://issues.apache.org/jira/browse/CALCITE-3405
>> >
>> > Haisheng Yuan  于2019年10月12日周六 上午4:34写道:
>> >
>> > > Yes, definitely.
>> > >
>> > > You can go through the project expression with InputFinder to find all
>> > the
>> > > used columns, create a  logical project with those columns, and remap
>> the
>> > > top project with new column indexes.
>> > >
>> > > On the other hand, instead of creating a new intermidiate pogical
>> > project,
>> > > we can also update ProjectTableScanRule to accept LogicalProject that
>> is
>> > > not a simple mapping, and do the same task I mentioned above.
>> > >
>> > > - Haisheng
>> > >
>> > > --
>> > > 发件人:Rommel Quintanilla
>> > > 日 期:2019年10月12日 03:15:31
>> > > 收件人:
>> > > 主 题:[QUESTION] Pushing up evaluations from LogicalProjects
>> > >
>> > > Hi, maybe you can help me.
>> > > I have this portion from a larger logical plan:
>> > > ..
>> > > LogicalProject(l_orderkey=[$0], l_suppkey=[$2], *=[*($5, -(1,
>> > > $6))])
>> > > LogicalTableScan(table=[[main, lineitem]])
>> > > ..
>> > >
>> > > Because the LogicalProject above contains an evaluation, the
>> > > ProjectTableScanRule can't convert it to a BindableTableScan.
>> > >
>> > > I wonder if somehow the evaluation could be pushed up more or less like
>> > > this:
>> > > ..
>> > >  LogicalProject(l_orderkey=[$0], l_suppkey=[$1], *=[*($2, -(1, $3))])
>> > >  LogicalProject(l_orderkey=[$0], l_suppkey=[$2], l_extendedprice=[$5],
>> > > l_discount=[$6]])
>> > >  LogicalTableScan(table=[[main, lineitem]])
>> > > ..
>> > >
>> > > Regards.
>> > >
>> >
>>


Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-12 Thread XING JIN
Hi Stamatis,
In current code, BindableTableScan is only created by rules of
ProjectTableScanRule and FilterTableScanRule. I think it's better to keep
as it is?
I made a PR for CALCITE-3405  -- https://github.com/apache/calcite/pull/1500

The idea of the PR is quite straightforward:
1. Analyze the parent Project -- collect all the needed fields;
2. Column pruning by pushing down the needed fields to BindableTableScan;
3. Adjust RexInputRefs in parent Project

@Haisheng @Stamatis It would be great if you can give a review when you
have time ~~~ Thanks a lot !

Best,
Jin


Stamatis Zampetakis  于2019年10月12日周六 下午3:00写道:

> Hi Rommel,
>
> I was hoping that this could be done at least by RelFieldTrimmer [1]. Are
> you using it already?
>
> Best,
> Stamatis
>
> [1]
>
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java
>
> On Sat, Oct 12, 2019 at 6:06 AM XING JIN  wrote:
>
> > Filed a JIRA:
> > https://issues.apache.org/jira/browse/CALCITE-3405
> >
> > Haisheng Yuan  于2019年10月12日周六 上午4:34写道:
> >
> > > Yes, definitely.
> > >
> > > You can go through the project expression with InputFinder to find all
> > the
> > > used columns, create a  logical project with those columns, and remap
> the
> > > top project with new column indexes.
> > >
> > > On the other hand, instead of creating a new intermidiate pogical
> > project,
> > > we can also update ProjectTableScanRule to accept LogicalProject that
> is
> > > not a simple mapping, and do the same task I mentioned above.
> > >
> > > - Haisheng
> > >
> > > --
> > > 发件人:Rommel Quintanilla
> > > 日 期:2019年10月12日 03:15:31
> > > 收件人:
> > > 主 题:[QUESTION] Pushing up evaluations from LogicalProjects
> > >
> > > Hi, maybe you can help me.
> > > I have this portion from a larger logical plan:
> > > ..
> > > LogicalProject(l_orderkey=[$0], l_suppkey=[$2], *=[*($5, -(1,
> > > $6))])
> > > LogicalTableScan(table=[[main, lineitem]])
> > > ..
> > >
> > > Because the LogicalProject above contains an evaluation, the
> > > ProjectTableScanRule can't convert it to a BindableTableScan.
> > >
> > > I wonder if somehow the evaluation could be pushed up more or less like
> > > this:
> > > ..
> > >  LogicalProject(l_orderkey=[$0], l_suppkey=[$1], *=[*($2, -(1, $3))])
> > >  LogicalProject(l_orderkey=[$0], l_suppkey=[$2], l_extendedprice=[$5],
> > > l_discount=[$6]])
> > >  LogicalTableScan(table=[[main, lineitem]])
> > > ..
> > >
> > > Regards.
> > >
> >
>


Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-12 Thread Stamatis Zampetakis
Hi Rommel,

I was hoping that this could be done at least by RelFieldTrimmer [1]. Are
you using it already?

Best,
Stamatis

[1]
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java

On Sat, Oct 12, 2019 at 6:06 AM XING JIN  wrote:

> Filed a JIRA:
> https://issues.apache.org/jira/browse/CALCITE-3405
>
> Haisheng Yuan  于2019年10月12日周六 上午4:34写道:
>
> > Yes, definitely.
> >
> > You can go through the project expression with InputFinder to find all
> the
> > used columns, create a  logical project with those columns, and remap the
> > top project with new column indexes.
> >
> > On the other hand, instead of creating a new intermidiate pogical
> project,
> > we can also update ProjectTableScanRule to accept LogicalProject that is
> > not a simple mapping, and do the same task I mentioned above.
> >
> > - Haisheng
> >
> > --
> > 发件人:Rommel Quintanilla
> > 日 期:2019年10月12日 03:15:31
> > 收件人:
> > 主 题:[QUESTION] Pushing up evaluations from LogicalProjects
> >
> > Hi, maybe you can help me.
> > I have this portion from a larger logical plan:
> > ..
> > LogicalProject(l_orderkey=[$0], l_suppkey=[$2], *=[*($5, -(1,
> > $6))])
> > LogicalTableScan(table=[[main, lineitem]])
> > ..
> >
> > Because the LogicalProject above contains an evaluation, the
> > ProjectTableScanRule can't convert it to a BindableTableScan.
> >
> > I wonder if somehow the evaluation could be pushed up more or less like
> > this:
> > ..
> >  LogicalProject(l_orderkey=[$0], l_suppkey=[$1], *=[*($2, -(1, $3))])
> >  LogicalProject(l_orderkey=[$0], l_suppkey=[$2], l_extendedprice=[$5],
> > l_discount=[$6]])
> >  LogicalTableScan(table=[[main, lineitem]])
> > ..
> >
> > Regards.
> >
>