[ https://issues.apache.org/jira/browse/CALCITE-4317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211440#comment-17211440 ]
Rafay A commented on CALCITE-4317: ---------------------------------- [~julianhyde], we did see the above issue both in LoptOptimizeJoinRule and ProjectJoinTransposeRule but i was unable to repro the above issue in LoptOptimizeJoinRule and after some research, i found out that this issue has been fixed already( we need to cherry-pick this change). See [this|https://issues.apache.org/jira/browse/CALCITE-2720]. Since we have already fixed the issue of empty aggregate in LoptOptimizeJoinRule, i think we should also fix the same for ProjectJoinTransposeRule. I could not create the query which would fail for ProjectJoinTransposeRule since we do a lot of transformations, but this simple test will repro the issue: {code:java} @Test void testProjectJoinTransposeRuleOnAggWithNoFields() { final RelBuilder relBuilder = RelBuilder.create(RelBuilderTest.config().build()); // Build a rel equivalent to sql: // SELECT name FROM (SELECT count(*) cnt_star, count(empno) cnt_en FROM sales.emp) // cross join sales.dept // limit 10 RelNode left = relBuilder.scan("DEPT").build(); RelNode right = relBuilder.scan("EMP") .project(ImmutableList.of(relBuilder.getRexBuilder().makeExactLiteral(BigDecimal.ZERO)), ImmutableList.of("DUMMY")) .aggregate(relBuilder.groupKey()).build(); RelNode plan = relBuilder.push(left) .push(right) .join(JoinRelType.INNER, relBuilder.getRexBuilder().makeLiteral(true)) .project(relBuilder.field("DEPTNO")) .build(); HepProgram program = new HepProgramBuilder() .addRuleInstance(CoreRules.PROJECT_JOIN_TRANSPOSE) .build(); HepPlanner hepPlanner = new HepPlanner(program); hepPlanner.setRoot(plan); hepPlanner.findBestExp(); } {code} Regardless, we should fix the RelFieldTrimmer as well, as we discussed above. > Some rules fail to handle Aggregate node if RelFieldTrimmer trims all the > fields > -------------------------------------------------------------------------------- > > Key: CALCITE-4317 > URL: https://issues.apache.org/jira/browse/CALCITE-4317 > Project: Calcite > Issue Type: Bug > Components: core > Reporter: Rafay A > Priority: Major > > Consider this query: > {code:java} > select o_orderkey from > (SELECT count(*) cnt_star, count(l_orderkey) cnt_ok FROM lineitem) > cross join orders > limit 10 > {code} > which generates this plan: > {code:java} > LogicalSort(fetch=[10]) > LogicalProject(o_orderkey=[$2]) > LogicalJoin(condition=[true], joinType=[inner]) > LogicalAggregate(group=[{}], cnt_star=[COUNT()], cnt_ok=[COUNT($0)]) > LogicalProject(l_orderkey=[$0]) > ScanCrel(table=[lineitem], columns=[`l_orderkey`, `l_partkey`, > `l_suppkey`, `l_linenumber`, `l_quantity`, `l_extendedprice`, `l_discount`, > `l_tax`, `l_returnflag`, `l_linestatus`, `l_shipdate`, `l_commitdate`, > `l_receiptdate`, `l_shipinstruct`, `l_shipmode`, `l_comment`]) > ScanCrel(table=[orders], columns=[`o_orderkey`, `o_custkey`, > `o_orderstatus`, `o_totalprice`, `o_orderdate`, `o_orderpriority`, `o_clerk`, > `o_shippriority`, `o_comment`]) > {code} > and after we apply RelFieldTrimmer: > {code:java} > LogicalSort(fetch=[10]) > LogicalProject(o_orderkey=[$0]) > LogicalJoin(condition=[true], joinType=[inner]) > LogicalAggregate(group=[{}]) > LogicalProject(l_orderkey=[$0]) > ScanCrel(table=[lineitem], columns=[`l_orderkey`]) > ScanCrel(table=[orders], columns=[`o_orderkey`]) > {code} > > We see that the LogicalAggregate has no groups/functions after applying the > trimmer. When we apply ProjectJoinTransposeRule or LoptOptimizeJoinRule after > trimming, we are seeing these exceptions: > For LoptOptimizeJoinRule: > {code:java} > (java.lang.ArrayIndexOutOfBoundsException) 0 > com.google.common.collect.RegularImmutableList.get():75 > org.apache.calcite.rel.metadata.RelMdColumnOrigins.getColumnOrigins():77 > sun.reflect.NativeMethodAccessorImpl.invoke0():-2 > sun.reflect.NativeMethodAccessorImpl.invoke():62 > sun.reflect.DelegatingMethodAccessorImpl.invoke():43 > java.lang.reflect.Method.invoke():498 > > org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$1$1.invoke():178 > com.sun.proxy.$Proxy175.getColumnOrigins():-1 > sun.reflect.GeneratedMethodAccessor144.invoke():-1 > sun.reflect.DelegatingMethodAccessorImpl.invoke():43 > java.lang.reflect.Method.invoke():498 > > org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke():139 > com.sun.proxy.$Proxy175.getColumnOrigins():-1 > sun.reflect.GeneratedMethodAccessor144.invoke():-1 > sun.reflect.DelegatingMethodAccessorImpl.invoke():43 > java.lang.reflect.Method.invoke():498 > com.sun.proxy.$Proxy175.getColumnOrigins():-1 > sun.reflect.GeneratedMethodAccessor144.invoke():-1 > sun.reflect.DelegatingMethodAccessorImpl.invoke():43 > java.lang.reflect.Method.invoke():498 > > org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke():139 > com.sun.proxy.$Proxy175.getColumnOrigins():-1 > sun.reflect.GeneratedMethodAccessor144.invoke():-1 > sun.reflect.DelegatingMethodAccessorImpl.invoke():43 > java.lang.reflect.Method.invoke():498 > com.sun.proxy.$Proxy175.getColumnOrigins():-1 > sun.reflect.GeneratedMethodAccessor144.invoke():-1 > sun.reflect.DelegatingMethodAccessorImpl.invoke():43 > java.lang.reflect.Method.invoke():498 > > org.apache.calcite.rel.metadata.RelMetadataQuery$MetadataInvocationHandler.invoke():110 > com.sun.proxy.$Proxy175.getColumnOrigins():-1 > org.apache.calcite.rel.metadata.RelMetadataQuery.getColumnOrigins():293 > org.apache.calcite.rel.metadata.RelMetadataQuery.getTableOrigin():348 > org.apache.calcite.rel.rules.LoptOptimizeJoinRule.getSimpleFactors():377 > > org.apache.calcite.rel.rules.LoptOptimizeJoinRule.findRemovableSelfJoins():297 > org.apache.calcite.rel.rules.LoptOptimizeJoinRule.onMatch():126 > org.apache.calcite.plan.AbstractRelOptPlanner.fireRule():317 > org.apache.calcite.plan.hep.HepPlanner.applyRule():556 > org.apache.calcite.plan.hep.HepPlanner.applyRules():415 > org.apache.calcite.plan.hep.HepPlanner.executeInstruction():280 > org.apache.calcite.plan.hep.HepInstruction$RuleCollection.execute():74 > org.apache.calcite.plan.hep.HepPlanner.executeProgram():211 > org.apache.calcite.plan.hep.HepPlanner.findBestExp():198 > {code} > For ProjectJoinTransposeRule: > {code:java} > (java.lang.ArrayIndexOutOfBoundsException) 0 > com.google.common.collect.RegularImmutableList.get():75 > org.apache.calcite.rel.rules.PushProjector.createProjectRefsAndExprs():477 > org.apache.calcite.rel.rules.ProjectJoinTransposeRule.onMatch():138 > org.apache.calcite.plan.AbstractRelOptPlanner.fireRule():317 > org.apache.calcite.plan.hep.HepPlanner.applyRule():556 > org.apache.calcite.plan.hep.HepPlanner.applyRules():415 > org.apache.calcite.plan.hep.HepPlanner.executeInstruction():280 > org.apache.calcite.plan.hep.HepInstruction$RuleCollection.execute():74 > org.apache.calcite.plan.hep.HepPlanner.executeProgram():211 > org.apache.calcite.plan.hep.HepPlanner.findBestExp():198 > {code} > I think we should either fix the trimmer, or the rules to handle these cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)