bq . Since we are talking about materialized views, I think in most cases tableRel should be simply a LogicalTableScan.
Stamatis is correct about this, I had not realized tableRel == queryRel in your sample code. Thanks, Jesús On Fri, May 3, 2019 at 6:12 AM Stamatis Zampetakis <zabe...@gmail.com> wrote: > I think the main problem comes from the fact that tableRel == queryRel in > the test case you provided. > Defining the materialized view like that basically says that when you find > a part of the query that satisfies queryRel replace it with itself. > In conjunction with the rule that is used, which allows partial rewritings > using union, you end up with a rule that matches infinite number of times. > Since we are talking about materialized views, I think in most cases > tableRel should be simply a LogicalTableScan. > The idea is that expression represented by queryRel is materialized into a > table so in order to retrieve the results we only need to scan the table. > > Regarding the "if (true)" statements that you observed, most likely they > were introduced as release toggles [1]. > However, since the last commit was in 2013 I think by now it is safe to > refactor that part and remove dead code. > > [1] https://www.martinfowler.com/articles/feature-toggles.html > > Best, > Stamatis > > On Fri, May 3, 2019 at 12:50 PM Mark Pasterkamp < > markpasterkamp1...@gmail.com> wrote: > > > Dear Jesus, > > > > I think your intuition in this regard is correct. > > After executing the main program in the HepPlanner the resulting plan > > contains a lot of circular references. > > Changing the matching order does not influence this behaviour. > > > > > > Mark > > > > On Thu, 2 May 2019 at 22:14, Jesus Camacho Rodriguez > > <jcamachorodrig...@cloudera.com.invalid> wrote: > > > > > Mark, > > > > > > I have an intuition that this happens because the rule creates a > > partially > > > contained rewriting with a union, where one side contains a scan over > the > > > materialized view and the other side contains the query itself with a > > > filter on top excluding the data that is coming from the materialized > > view. > > > Then the rule is triggered on the plan representing the original query > > > again and the process is repeated. Have you tried changing the matching > > > order for your hep program? > > > > > > Thanks, > > > Jesús > > > > > > On Thu, May 2, 2019 at 8:53 AM Mark Pasterkamp < > > > markpasterkamp1...@gmail.com> > > > wrote: > > > > > > > Hi Stamatis, > > > > > > > > I have tried to recreate the issue but I have not been able to do > > that. I > > > > was however able to create a new exception which I don't quite > > > understand. > > > > The error happened when calcite was creating a union rewriting using > > > > materialized views. But trying to recreate this situation gave me > > another > > > > interesting one. > > > > This time, the planner rewrites one of the children nodes into > itself I > > > > would assume which causes a stack overflow. The method itself can be > > > found > > > > here: > > > > > > > > > > > > > > > > > > https://github.com/mpasterkamp/calcite/blob/768b7928dbde5f6f9775a1119e7466d8eafafb4b/core/src/test/java/org/apache/calcite/test/HepPlannerTest.java#L312 > > > > > > > > Perhaps I am doing something wrong, perhaps not? I am not > knowledgeable > > > > enough about this to understand why this is happening. Wish I could > > help > > > > more for that. > > > > > > > > Also, while investigating this issue I found another interesting > > artifact > > > > in de source code of the VolcanoCost. A lot of methods in this class > > have > > > > an "if (true)"-statement like here: > > > > > > > > > > > > > > > > > > https://github.com/apache/calcite/blob/4b4d8037c5073e4eb5702b12bc4ecade31476616/core/src/main/java/org/apache/calcite/plan/volcano/VolcanoCost.java#L100 > > > > > > > > Now I was just curious, is there any reason for this to be there that > > you > > > > know of? > > > > > > > > Thank you for responding and congratulations for your recent > > promotions. > > > > > > > > > > > > With kind regards, > > > > > > > > Mark > > > > > > > > > > > > On Thu, 2 May 2019 at 14:58, Stamatis Zampetakis <zabe...@gmail.com> > > > > wrote: > > > > > > > > > Said like that it looks like a bug. > > > > > > > > > > I think the best would be to reproduce the exception as a unit test > > in > > > > > HepPlannerTest [1], RelOptRulesTest [2], or PlannerTest [3] so that > > we > > > > > could understand better the use case. > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/test/HepPlannerTest.java > > > > > [2] > > > > > > > > > > > > > > > > > > > > https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/test/RelOptRulesTest.java > > > > > [3] > > > > > > > > > > > > > > > > > > > > https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/tools/PlannerTest.java > > > > > > > > > > On Thu, May 2, 2019 at 7:56 AM Mark Pasterkamp < > > > > > markpasterkamp1...@gmail.com> > > > > > wrote: > > > > > > > > > > > I don't, I would assume that the HepPlanner.findBestExp() > > calculates > > > > the > > > > > > cost somewhere down the line > > > > > > > > > > > > On Thu, May 2, 2019, 03:31 Yuzhao Chen <yuzhao....@gmail.com> > > wrote: > > > > > > > > > > > > > Why you care about cost when use HepPlanner ? The HepPlanner is > > > aimed > > > > > for > > > > > > > some deterministic planning rules, we usually do not need cost > in > > > > Hep. > > > > > > Some > > > > > > > exceptions like Join reorder may need a cost. > > > > > > > > > > > > > > What kind of planning promotion you did ? I'm kind of curious > > about > > > > it. > > > > > > > > > > > > > > Best, > > > > > > > Danny Chan > > > > > > > 在 2019年5月1日 +0800 PM9:27，Mark Pasterkamp < > > > > markpasterkamp1...@gmail.com > > > > > > > >，写道： > > > > > > > > Dear all, > > > > > > > > > > > > > > > > While playing around with the HepPlanner I ran into an issue > > > where > > > > > the > > > > > > > > planner wants to rewrite a query with a union rewrite. When > the > > > > > > > > RelMetaDataQuery computes the cost, the cost instance is a > > > > > VolcanoCost. > > > > > > > > Then when it tries to calculate the cost of one of the > union's > > > > > operands > > > > > > > it > > > > > > > > is a RelCostImpl which results in the ClassCastException. > > > > > > > > > > > > > > > > How would I go about solving this issue? As far as my > knowledge > > > > > goes, I > > > > > > > am > > > > > > > > not able to change the costhandler of the RelMetaDataQuery. > > > Another > > > > > > > > approach I could see is removing the cast in the VolcanoCost > > > class, > > > > > > but I > > > > > > > > would hope I do not have to do that. > > > > > > > > > > > > > > > > > > > > > > > > With kind regards, > > > > > > > > > > > > > > > > Mark > > > > > > > > > > > > > > > > > > > > > > > > > > > >
