Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-15 Thread Danny Chan
Just open it in the SqlToRelConverter [1] [1]  https://github.com/apache/calcite/blob/3cbbafa941128dc5097c2a26711f5751f764e12d/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L5692 Best, Danny Chan 在 2019年10月16日 +0800 AM12:23,Rommel Quintanilla ,写道: > Hi Stamatis, thank you

Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-15 Thread Rommel Quintanilla
Hi Stamatis, thank you for your attention and actually many thanks to everyone who is seeing/commenting about this topic. To be honest I didn't know about the existence of RelFieldTrimmer. I would like to test this, how could I use RelFieldTrimmer? can you give me a suggestion, please? On

Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-15 Thread XING JIN
Hi Rommel ~ I'm very happy you tried the patch ~ Well the stacktrace you provided seems outdated ~ I updated the the code several hours ago. The current commit id is 05d0e2d9fbadec060e54c622824b3725df36aab0 Could you please try the case again and send me the exception and related information ? Or

Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-15 Thread Rommel Quintanilla
Hi Jin, your contribution is awesome! thanks. I tested it and works like a charm in most cases. Definitely clearer than the approach that I was trying. However, on the application I'm working on, I found a weird issue that I wasn't able to reproduce as a unit test on the calcite-core project.

Re: Re: Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-14 Thread XING JIN
I mean, testPushNonSimpleMappingProject [1] runs with RelFieldTrimmer enabled, which is done by [2] [1] https://github.com/apache/calcite/pull/1500/files#diff-3c834a32d46b821b5241e132f2ae6bfaR324 [2]

Re: Re: Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-14 Thread XING JIN
Yes, that's how testPushNonSimpleMappingProject [1] works [1] https://github.com/apache/calcite/pull/1500/files#diff-3c834a32d46b821b5241e132f2ae6bfaR324 Danny Chan 于2019年10月14日周一 下午3:36写道: > > But why the final BindableTableScan is not pruned ? > > The RelFieldTrimmer default is turned off,

Re: Re: Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-14 Thread Danny Chan
> But why the final BindableTableScan is not pruned ? The RelFieldTrimmer default is turned off, you should open it explicitly. Best, Danny Chan 在 2019年10月14日 +0800 AM11:51,dev@calcite.apache.org,写道: > > BINDABLE_TABLE_SCAN_RULE

Re: Re: Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-13 Thread XING JIN
Also regarding that Projects merging is common in Calcite optimization rules, we should always remember to avoid merging for cases like the one given by Rommel. I think that would be hard. Best, Jin XING JIN 于2019年10月14日周一 上午11:51写道: > Hi, Stamatis, Danny~ > > Thanks for explain ~ > > > "The

Re: Re: Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-13 Thread XING JIN
Hi, Stamatis, Danny~ Thanks for explain ~ > "The consumer in the case of P1 is the project which only needs $0, $2, $5, $6 so the trimmer could slim down the scan by projecting only these fields." I think RelFieldTrimmer is already doing this by [1]. But why the final BindableTableScan is not

Re: Re: Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-13 Thread Danny Chan
+1, RelFieldTrimmer is the role to trim the unused fields. Best, Danny Chan 在 2019年10月13日 +0800 AM6:25,dev@calcite.apache.org,写道: > > RelFieldTrimmer

Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-13 Thread Wang Yanlin
OK,I got your point. Thanks for sharing. | | 王炎林 | | 邮箱:1989yanlinw...@163.com | 签名由 网易邮箱大师 定制 On 10/13/2019 06:17, Stamatis Zampetakis wrote: I was thinking that RelFieldTrimmer can be used to transform the plan P1 to plan P2 and then ProjectTableScanRule can be used to introduce the

Re: Re: Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-12 Thread Stamatis Zampetakis
I was thinking that RelFieldTrimmer can be used to transform the plan P1 to plan P2 and then ProjectTableScanRule can be used to introduce the BindableTableScan. The consumer in the case of P1 is the project which only needs $0, $2, $5, $6 so the trimmer could slim down the scan by projecting

Re:Re: Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-12 Thread Wang Yanlin
Just a little curious, RelFieldTrimmer is used to 'slimmed down' relational expression that projects only the columns required by its consumer. How can it used to do this thing? -- Best, Wang Yanlin 在 2019-10-12 18:03:33,"XING JIN" 写道: >Sure we can ~ >If we use BindableTableScanRule to

Re: Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-12 Thread XING JIN
Sure we can ~ If we use BindableTableScanRule to derive a BindableTableScan from ProjectableFilterableTable, that would happen during a stage of optimization run by RelOptPlanner. But RelFieldTrimmer works right during conversion of Sql to Rel. Wang Yanlin <1989yanlinw...@163.com> 于2019年10月12日周六

Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-12 Thread XING JIN
Hi Stamatis, In current code, BindableTableScan is only created by rules of ProjectTableScanRule and FilterTableScanRule. I think it's better to keep as it is? I made a PR for CALCITE-3405 -- https://github.com/apache/calcite/pull/1500 The idea of the PR is quite straightforward: 1. Analyze the

Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-12 Thread Stamatis Zampetakis
Hi Rommel, I was hoping that this could be done at least by RelFieldTrimmer [1]. Are you using it already? Best, Stamatis [1] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java On Sat, Oct 12, 2019 at 6:06 AM XING JIN wrote: >

Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-11 Thread XING JIN
Filed a JIRA: https://issues.apache.org/jira/browse/CALCITE-3405 Haisheng Yuan 于2019年10月12日周六 上午4:34写道: > Yes, definitely. > > You can go through the project expression with InputFinder to find all the > used columns, create a logical project with those columns, and remap the > top project

Re: [QUESTION] Pushing up evaluations from LogicalProjects

2019-10-11 Thread Haisheng Yuan
Yes, definitely. You can go through the project expression with InputFinder to find all the used columns, create a logical project with those columns, and remap the top project with new column indexes. On the other hand, instead of creating a new intermidiate pogical project, we can also