> MEMO group (RelSet) represents logically equivalent expressions. > All the expressions in one group should share the same logical > properties, e.g. functional dependency, constraint properties etc. > But they are not sharing it. Computation is done for each individual operator.
It's good, and correct, that we compute for each individual operator. Suppose that a RelSubset s contains RelNode r1 and we know that the constraint x > 0 holds. Suppose that we also have r2 with constraint y < 10, and we discover that r1 and r2 are equivalent and belong together in s. Now we can safely say that both constraints (x > 0 and y < 10) apply to both r1 and r2. Deducing additional constraints in this way is a big win. The effort to compute constraints for each RelNode is well-spent. This kind of deduction applies to other logical properties (e.g. unique keys) and it applies to RelSet as well as RelSubset. Julian On Sun, Jan 12, 2020 at 10:10 AM Roman Kondakov <kondako...@mail.ru.invalid> wrote: > > @Haisheng > > > Calcite uses Project operator and all kinds of ProjectXXXTranposeRule to > > prune unused columns. > > I also noticed that in most cases Project-related rules took significant > part of the planning time. But I didn't explore this problem yet. > > > MEMO group (RelSet) represents logically equivalent expressions. All the > > expressions in one group should share the same logical properties, e.g. > > functional dependency, constraint properties etc. But they are not sharing > > it. Computation is done for each individual operator. > > I thought the equivalence of logical properties within groups (RelSets) > are implicit. For example, in RelSet#addInternal it is always verified > that the new added node has the same type as other members of the set. > > Anyway I absolutely agree with you that problems with traits > propagation, rules matching and other that you mentioned in the previous > e-mails should be solved in the first place. We need first to make > Volcano optimizer work right and only then make some improvements like > search space pruning. > > I would love to join this work to improve Volcano planner. Looking > forward for design doc. > > > -- > Kind Regards > Roman Kondakov > > > On 11.01.2020 11:42, Haisheng Yuan wrote: > > Currently, Calcite uses Project operator and all kinds of > > ProjectXXXTranposeRule to prune unused columns. Every operator's output > > columns use index to reference child operators' columns. If there is a > > Project operator with child operator of a Filter, if we push project down > > under Filter, we will have Project-Filter-Project-FilterInput. All the > > newly generated relnodes will trigger rule matches. e.g. If we already did > > ReduceExpressionRule on filter, but due to the new filter RexCall's input > > ref index changed, we have to apply ReduceExpressionRule on the new filter > > again, even there is nothing can be reduced. Similarly new operator > > transpose/merge rule will be triggered. This can trigger a lot of rule > > matches. > > > > MEMO group (RelSet) represents logically equivalent expressions. All the > > expressions in one group should share the same logical properties, e.g. > > functional dependency, constraint properties etc. But they are not sharing > > it. Computation is done for each individual operator. > > > > Without resolving those issue, space pruning won't help much. > > > > There are a lot of room for improvement. Hope the community can join the > > effort to make Calcite better. > > > > - Haisheng > > > > ------------------------------------------------------------------ > > 发件人：Roman Kondakov<kondako...@mail.ru.INVALID> > > 日 期：2020年01月10日 19:39:51 > > 收件人：<dev@calcite.apache.org> > > 主 题：Re: [DISCUSS] Proposal to add API to force rules matching specific rels > > > > @Haisheng, could you please clarify what you mean by these points? > > > >> - the poor-design of column pruning, > >> - lack of group properties etc. > > > > I guess I'm not aware of these problems. > >
