Hi Rommel ~ I'm very happy you tried the patch ~ Well the stacktrace you provided seems outdated ~ I updated the the code several hours ago. The current commit id is 05d0e2d9fbadec060e54c622824b3725df36aab0 Could you please try the case again and send me the exception and related information ? Or you can just reply on https://github.com/apache/calcite/pull/1500 or https://issues.apache.org/jira/browse/CALCITE-3405 I will try to fix ASAP
Best, Jin Rommel Quintanilla <rom...@blazingdb.com> 于2019年10月15日周二 下午7:22写道： > Hi Jin, your contribution is awesome! thanks. > > I tested it and works like a charm in most cases. Definitely clearer than > the approach that I was trying. However, on the application I'm working on, > I found a weird issue that I wasn't able to reproduce as a unit test on the > calcite-core project. Btw, I'm using your changes after Haisheng's revision. > > So, it is as follows, I have this simple query: select l_extendedprice-1 > from lineitem > where the type of l_extendedprice is DOUBLE. Before applying > ProjectTableScanRule the logical plan is: > > LogicalProject(EXPR$0=[-($5, 1)]) > LogicalTableScan(table=[[main, lineitem]]) > > So far all is fine, but when the ProjectTableScanRule was applied I got: > > java.lang.AssertionError: type mismatch: > ref: > DOUBLE > input: > BIGINT > at org.apache.calcite.util.Litmus$1.fail(Litmus.java:31) > at org.apache.calcite.plan.RelOptUtil.eq(RelOptUtil.java:2000) > at > org.apache.calcite.rex.RexChecker.visitInputRef(RexChecker.java:125) > at > org.apache.calcite.rex.RexChecker.visitInputRef(RexChecker.java:57) > at org.apache.calcite.rex.RexInputRef.accept(RexInputRef.java:112) > at org.apache.calcite.rex.RexChecker.visitCall(RexChecker.java:140) > at org.apache.calcite.rex.RexChecker.visitCall(RexChecker.java:57) > at org.apache.calcite.rex.RexCall.accept(RexCall.java:191) > at org.apache.calcite.rel.core.Project.isValid(Project.java:192) > at org.apache.calcite.rel.core.Project.<init>(Project.java:83) > at > org.apache.calcite.rel.logical.LogicalProject.<init>(LogicalProject.java:62) > at > org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:112) > at > org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:100) > at > org.apache.calcite.rel.core.RelFactories$ProjectFactoryImpl.createProject(RelFactories.java:158) > at > org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1414) > at > org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1252) > at > org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1241) > at > com.blazingdb.calcite.rules.ProjectTableScanRule.apply(ProjectTableScanRule.java:147) > at > com.blazingdb.calcite.rules.ProjectTableScanRule$1.onMatch(ProjectTableScanRule.java:65) > at > org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:319) > at > org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:560) > at > org.apache.calcite.plan.hep.HepPlanner.depthFirstApply(HepPlanner.java:374) > at > org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:436) > at > org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:256) > at > org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:127) > at > org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:215) > at > org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:202) > at > com.blazingdb.calcite.application.RelationalAlgebraGenerator.getOptimizedRelationalAlgebra(RelationalAlgebraGenerator.java:202) > at > com.blazingdb.calcite.catalog.BlazingRulesTest.generateSQLTest(BlazingRulesTest.java:274) > > Any idea why could this be happening? > > Thank you in advance. > > On 2019/10/12 09:12:20, XING JIN <jinxing.co...@gmail.com> wrote: > > Hi Stamatis, > > In current code, BindableTableScan is only created by rules of > > ProjectTableScanRule and FilterTableScanRule. I think it's better to keep > > as it is? > > I made a PR for CALCITE-3405 -- > https://github.com/apache/calcite/pull/1500 > > > > The idea of the PR is quite straightforward: > > 1. Analyze the parent Project -- collect all the needed fields; > > 2. Column pruning by pushing down the needed fields to BindableTableScan; > > 3. Adjust RexInputRefs in parent Project > > > > @Haisheng @Stamatis It would be great if you can give a review when you > > have time ~~~ Thanks a lot ! > > > > Best, > > Jin > > > > > > Stamatis Zampetakis <zabe...@gmail.com> 于2019年10月12日周六 下午3:00写道： > > > > > Hi Rommel, > > > > > > I was hoping that this could be done at least by RelFieldTrimmer [1]. > Are > > > you using it already? > > > > > > Best, > > > Stamatis > > > > > > [1] > > > > > > > https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java > > > > > > On Sat, Oct 12, 2019 at 6:06 AM XING JIN <jinxing.co...@gmail.com> > wrote: > > > > > > > Filed a JIRA: > > > > https://issues.apache.org/jira/browse/CALCITE-3405 > > > > > > > > Haisheng Yuan <h.y...@alibaba-inc.com> 于2019年10月12日周六 上午4:34写道： > > > > > > > > > Yes, definitely. > > > > > > > > > > You can go through the project expression with InputFinder to find > all > > > > the > > > > > used columns, create a logical project with those columns, and > remap > > > the > > > > > top project with new column indexes. > > > > > > > > > > On the other hand, instead of creating a new intermidiate pogical > > > > project, > > > > > we can also update ProjectTableScanRule to accept LogicalProject > that > > > is > > > > > not a simple mapping, and do the same task I mentioned above. > > > > > > > > > > - Haisheng > > > > > > > > > > ------------------------------------------------------------------ > > > > > 发件人：Rommel Quintanilla<rom...@blazingdb.com> > > > > > 日 期：2019年10月12日 03:15:31 > > > > > 收件人：<dev@calcite.apache.org> > > > > > 主 题：[QUESTION] Pushing up evaluations from LogicalProjects > > > > > > > > > > Hi, maybe you can help me. > > > > > I have this portion from a larger logical plan: > > > > > .. > > > > > LogicalProject(l_orderkey=[$0], l_suppkey=[$2], *=[*($5, > -(1, > > > > > $6))]) > > > > > LogicalTableScan(table=[[main, lineitem]]) > > > > > .. > > > > > > > > > > Because the LogicalProject above contains an evaluation, the > > > > > ProjectTableScanRule can't convert it to a BindableTableScan. > > > > > > > > > > I wonder if somehow the evaluation could be pushed up more or less > like > > > > > this: > > > > > .. > > > > > LogicalProject(l_orderkey=[$0], l_suppkey=[$1], *=[*($2, -(1, > $3))]) > > > > > LogicalProject(l_orderkey=[$0], l_suppkey=[$2], > l_extendedprice=[$5], > > > > > l_discount=[$6]]) > > > > > LogicalTableScan(table=[[main, lineitem]]) > > > > > .. > > > > > > > > > > Regards. > > > > > > > > > > > > > > >
