Re: RelMetadataQuery.getRowCount stackoverflow
I think ProjectMergeRule related cyclic triggering has been walked around by CALCITE-3774 [1]. Haisheng [1] https://issues.apache.org/jira/browse/CALCITE-3774 On 2020/04/21 21:20:35, Stamatis Zampetakis wrote: > I've seen it also quite a few times and most often the reason was cycles in > the subsets that were triggering some rules (e.g., ProjectMergeRule [1]) to > fire an infinite number of times [2, 3, 4]. > > Best, > Stamatis > > [1] > https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/rules/ProjectMergeRule.java > [2] https://issues.apache.org/jira/browse/CALCITE-3505 > [3] https://issues.apache.org/jira/browse/CALCITE-2223 > [4] https://issues.apache.org/jira/browse/CALCITE-3124 > > On Tue, Apr 21, 2020 at 6:43 AM Haisheng Yuan wrote: > > > Can you add a reproducible test case and log a JIRA? It may be different > > with CALCITE-2057. People who is interested will investigate the issue. > > > > On 2020/04/21 04:24:31, JiaTao Tao wrote: > > > Thanks > > > I didn't add any new rule, just these: > > > > > > CONSTANT_REDUCTION_RULES > > > ABSTRACT_RELATIONAL_RULES > > > BASE_RULES > > > ABSTRACT_RULES > > > ENUMERABLE_RULES > > > > > > So this is a bug, or it just because of the call stack is too deep(if > > this, > > > I can adjust JVM parameter). > > > > > > Regards! > > > > > > Aron Tao > > > > > > > > > Scott Reynolds 于2020年4月21日周二 上午1:10写道： > > > > > > > I have had this happen numerous times when writing new planner rules. > > Most > > > > of the time my rule is missing some boolean logic to prevent itself > > from > > > > transforming the call. This results in the rule continuously > > transforming > > > > it's previous transformations. > > > > > > > > I can usually see this happening when I add a > > > > System.out.println(RelOptUtil.dumpPlan()) to the line before the > > > > call.transformTo(newRelationNode) > > > > > > > > On Mon, Apr 20, 2020 at 3:13 AM JiaTao Tao > > wrote: > > > > > > > > > Hi > > > > > Has anyone encountered this problem before? Just a simple query(no > > more > > > > > than 20 lines, two joins, no union). > > > > > > > > > > And I see this ticket: > > > > https://issues.apache.org/jira/browse/CALCITE-2057, > > > > > but there's no follow up, also I see flink may occur this problem( > > > > > https://developer.aliyun.com/ask/129548) > > > > > > > > > > java.lang.StackOverflowError > > > > > at java.util.HashMap.hash(HashMap.java:339) > > > > > at java.util.HashMap.put(HashMap.java:612) > > > > > at > > > > > > > > > > > com.google.common.collect.StandardTable.getOrCreate(StandardTable.java:165) > > > > > at > > > > com.google.common.collect.StandardTable.put(StandardTable.java:174) > > > > > at > > > > com.google.common.collect.HashBasedTable.put(HashBasedTable.java:55) > > > > > at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) > > > > > at > > > > > > > > > > > org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:208) > > > > > at > > > > > > > > > > > org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:72) > > > > > at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown > > Source) > > > > > at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) > > > > > at ... > > > > > > > > > > Regards! > > > > > > > > > > Aron Tao > > > > > > > > > > > > > > >
Re: RelMetadataQuery.getRowCount stackoverflow
I've seen it also quite a few times and most often the reason was cycles in the subsets that were triggering some rules (e.g., ProjectMergeRule [1]) to fire an infinite number of times [2, 3, 4]. Best, Stamatis [1] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/rules/ProjectMergeRule.java [2] https://issues.apache.org/jira/browse/CALCITE-3505 [3] https://issues.apache.org/jira/browse/CALCITE-2223 [4] https://issues.apache.org/jira/browse/CALCITE-3124 On Tue, Apr 21, 2020 at 6:43 AM Haisheng Yuan wrote: > Can you add a reproducible test case and log a JIRA? It may be different > with CALCITE-2057. People who is interested will investigate the issue. > > On 2020/04/21 04:24:31, JiaTao Tao wrote: > > Thanks > > I didn't add any new rule, just these: > > > > CONSTANT_REDUCTION_RULES > > ABSTRACT_RELATIONAL_RULES > > BASE_RULES > > ABSTRACT_RULES > > ENUMERABLE_RULES > > > > So this is a bug, or it just because of the call stack is too deep(if > this, > > I can adjust JVM parameter). > > > > Regards! > > > > Aron Tao > > > > > > Scott Reynolds 于2020年4月21日周二 上午1:10写道： > > > > > I have had this happen numerous times when writing new planner rules. > Most > > > of the time my rule is missing some boolean logic to prevent itself > from > > > transforming the call. This results in the rule continuously > transforming > > > it's previous transformations. > > > > > > I can usually see this happening when I add a > > > System.out.println(RelOptUtil.dumpPlan()) to the line before the > > > call.transformTo(newRelationNode) > > > > > > On Mon, Apr 20, 2020 at 3:13 AM JiaTao Tao > wrote: > > > > > > > Hi > > > > Has anyone encountered this problem before? Just a simple query(no > more > > > > than 20 lines, two joins, no union). > > > > > > > > And I see this ticket: > > > https://issues.apache.org/jira/browse/CALCITE-2057, > > > > but there's no follow up, also I see flink may occur this problem( > > > > https://developer.aliyun.com/ask/129548) > > > > > > > > java.lang.StackOverflowError > > > > at java.util.HashMap.hash(HashMap.java:339) > > > > at java.util.HashMap.put(HashMap.java:612) > > > > at > > > > > > > > com.google.common.collect.StandardTable.getOrCreate(StandardTable.java:165) > > > > at > > > com.google.common.collect.StandardTable.put(StandardTable.java:174) > > > > at > > > com.google.common.collect.HashBasedTable.put(HashBasedTable.java:55) > > > > at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) > > > > at > > > > > > > > org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:208) > > > > at > > > > > > > > org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:72) > > > > at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown > Source) > > > > at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) > > > > at ... > > > > > > > > Regards! > > > > > > > > Aron Tao > > > > > > > > > >
Re: RelMetadataQuery.getRowCount stackoverflow
Can you add a reproducible test case and log a JIRA? It may be different with CALCITE-2057. People who is interested will investigate the issue. On 2020/04/21 04:24:31, JiaTao Tao wrote: > Thanks > I didn't add any new rule, just these: > > CONSTANT_REDUCTION_RULES > ABSTRACT_RELATIONAL_RULES > BASE_RULES > ABSTRACT_RULES > ENUMERABLE_RULES > > So this is a bug, or it just because of the call stack is too deep(if this, > I can adjust JVM parameter). > > Regards! > > Aron Tao > > > Scott Reynolds 于2020年4月21日周二 上午1:10写道： > > > I have had this happen numerous times when writing new planner rules. Most > > of the time my rule is missing some boolean logic to prevent itself from > > transforming the call. This results in the rule continuously transforming > > it's previous transformations. > > > > I can usually see this happening when I add a > > System.out.println(RelOptUtil.dumpPlan()) to the line before the > > call.transformTo(newRelationNode) > > > > On Mon, Apr 20, 2020 at 3:13 AM JiaTao Tao wrote: > > > > > Hi > > > Has anyone encountered this problem before? Just a simple query(no more > > > than 20 lines, two joins, no union). > > > > > > And I see this ticket: > > https://issues.apache.org/jira/browse/CALCITE-2057, > > > but there's no follow up, also I see flink may occur this problem( > > > https://developer.aliyun.com/ask/129548) > > > > > > java.lang.StackOverflowError > > > at java.util.HashMap.hash(HashMap.java:339) > > > at java.util.HashMap.put(HashMap.java:612) > > > at > > > > > com.google.common.collect.StandardTable.getOrCreate(StandardTable.java:165) > > > at > > com.google.common.collect.StandardTable.put(StandardTable.java:174) > > > at > > com.google.common.collect.HashBasedTable.put(HashBasedTable.java:55) > > > at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) > > > at > > > > > org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:208) > > > at > > > > > org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:72) > > > at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source) > > > at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) > > > at ... > > > > > > Regards! > > > > > > Aron Tao > > > > > >
Re: RelMetadataQuery.getRowCount stackoverflow
Thanks I didn't add any new rule, just these: CONSTANT_REDUCTION_RULES ABSTRACT_RELATIONAL_RULES BASE_RULES ABSTRACT_RULES ENUMERABLE_RULES So this is a bug, or it just because of the call stack is too deep(if this, I can adjust JVM parameter). Regards! Aron Tao Scott Reynolds 于2020年4月21日周二 上午1:10写道： > I have had this happen numerous times when writing new planner rules. Most > of the time my rule is missing some boolean logic to prevent itself from > transforming the call. This results in the rule continuously transforming > it's previous transformations. > > I can usually see this happening when I add a > System.out.println(RelOptUtil.dumpPlan()) to the line before the > call.transformTo(newRelationNode) > > On Mon, Apr 20, 2020 at 3:13 AM JiaTao Tao wrote: > > > Hi > > Has anyone encountered this problem before? Just a simple query(no more > > than 20 lines, two joins, no union). > > > > And I see this ticket: > https://issues.apache.org/jira/browse/CALCITE-2057, > > but there's no follow up, also I see flink may occur this problem( > > https://developer.aliyun.com/ask/129548) > > > > java.lang.StackOverflowError > > at java.util.HashMap.hash(HashMap.java:339) > > at java.util.HashMap.put(HashMap.java:612) > > at > > > com.google.common.collect.StandardTable.getOrCreate(StandardTable.java:165) > > at > com.google.common.collect.StandardTable.put(StandardTable.java:174) > > at > com.google.common.collect.HashBasedTable.put(HashBasedTable.java:55) > > at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) > > at > > > org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:208) > > at > > > org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:72) > > at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source) > > at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) > > at ... > > > > Regards! > > > > Aron Tao > > >
Re: RelMetadataQuery.getRowCount stackoverflow
I have had this happen numerous times when writing new planner rules. Most of the time my rule is missing some boolean logic to prevent itself from transforming the call. This results in the rule continuously transforming it's previous transformations. I can usually see this happening when I add a System.out.println(RelOptUtil.dumpPlan()) to the line before the call.transformTo(newRelationNode) On Mon, Apr 20, 2020 at 3:13 AM JiaTao Tao wrote: > Hi > Has anyone encountered this problem before? Just a simple query(no more > than 20 lines, two joins, no union). > > And I see this ticket: https://issues.apache.org/jira/browse/CALCITE-2057, > but there's no follow up, also I see flink may occur this problem( > https://developer.aliyun.com/ask/129548) > > java.lang.StackOverflowError > at java.util.HashMap.hash(HashMap.java:339) > at java.util.HashMap.put(HashMap.java:612) > at > com.google.common.collect.StandardTable.getOrCreate(StandardTable.java:165) > at com.google.common.collect.StandardTable.put(StandardTable.java:174) > at com.google.common.collect.HashBasedTable.put(HashBasedTable.java:55) > at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) > at > org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:208) > at > org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:72) > at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source) > at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) > at ... > > Regards! > > Aron Tao >
RelMetadataQuery.getRowCount stackoverflow
Hi Has anyone encountered this problem before? Just a simple query(no more than 20 lines, two joins, no union). And I see this ticket: https://issues.apache.org/jira/browse/CALCITE-2057, but there's no follow up, also I see flink may occur this problem( https://developer.aliyun.com/ask/129548) java.lang.StackOverflowError at java.util.HashMap.hash(HashMap.java:339) at java.util.HashMap.put(HashMap.java:612) at com.google.common.collect.StandardTable.getOrCreate(StandardTable.java:165) at com.google.common.collect.StandardTable.put(StandardTable.java:174) at com.google.common.collect.HashBasedTable.put(HashBasedTable.java:55) at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) at org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:208) at org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:72) at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source) at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source) at ... Regards! Aron Tao
