Re: Trait propagation guidelines

Hi, Vladimir. As another topic, it is highly recommended that you split the aggregation in logical stages, not only for traits related matters. It is true that you need to annotate the node with different flags or subclasses and it's a large refactor. But after that, you may find much much bigger

Re: Trait propagation guidelines

> 2) Optimization requests are basically sent to RelSet-s, not RelSubset-s, > as we make pairwise comparisons between the requested RelSubset and other > subsets in the set [5][6]. I agree with you. There could be some waste when the new delivered / required traitset is generated by

Re: Trait propagation guidelines

Hi Vladimir, The top-down optimizer does NOT require implementation rule to generate 1 to 1 physical operator for a logical operator, as you can see, if you generate a 2 phase physical aggregates for the logical aggregate in the implementation rule, it still works. Window is special because we

Re: Top-down optimizer cannot explore the search space because physical rule is treated as transformation rule

Great, that is the correct way to change it and that should be the default implementation. On 2021/05/28 17:41:15, Vladimir Ozerov wrote: > BTW, I tried to change the implementation to: > > 1: protected boolean isTransformationRule(VolcanoRuleCall match) { > 2:return match.getRule()

Re: Top-down optimizer cannot explore the search space because physical rule is treated as transformation rule

BTW, I tried to change the implementation to: 1: protected boolean isTransformationRule(VolcanoRuleCall match) { 2:return match.getRule() instanceof TransformationRule; 3: } It solved my problem - plans returned to normal. In the Apache Calcite repo, only 4 tests in the TopDowOptTest

Re: Top-down optimizer cannot explore the search space because physical rule is treated as transformation rule

Hi Jinpeng, Thank you for the clarification. When I saw the code in question for the first time, my first thought was that it was perhaps designed for gradual migration. The main problem is that the current implementation discards parts of the plan *silently*, which might be difficult to spot. I

Re: Top-down optimizer cannot explore the search space because physical rule is treated as transformation rule

Hi, Vladimir. Good catch! There could be some improvements here. Actually, this problem was discovered early when the top-down rule driver was designed. At that time, no rule was annotated as "TransformationRule". Moreover, it is impossible to ask every calcite user who designed their own rules

Re: [DISCUSS] Towards Calcite 1.27.0

I did a pass over the JIRA issues marked for 1.27.0 release. I moved some that were promising but not close to be resolved in 1.28.0 and for others that there was not much (or any) progress I removed the fix version tag. After the cleanup it seems that there are three/four issues [1] that could

Top-down optimizer cannot explore the search space because physical rule is treated as transformation rule

Hi, I have an optimizer that uses top-down VolcanoPlanner and has a ConverterRule for every LogicalNode. I have a new requirement when one of the physical rules must emit several physical nodes instead of one. I tried to convert a ConverterRule to a normal rule that emits physical nodes. Even

Re: Trait propagation guidelines

Hi Jinpeng, Haisheng, Thank you for your inputs. I really appreciate that. Let me try to address some of your comments and share some experience with the implementation of optimizers for a distributed engine I am currently working with. First of all, I would argue that multiple logical operators

[jira] [Created] (CALCITE-4625) Release Calcite 1.27.0

Stamatis Zampetakis created CALCITE-4625: Summary: Release Calcite 1.27.0 Key: CALCITE-4625 URL: https://issues.apache.org/jira/browse/CALCITE-4625 Project: Calcite Issue Type: Task

Deduplicate correlate variables question.

Hi, calciters ) I am trying to figure out why DeduplicateCorrelateVariables [1] is called only if withExpand [2] flag is true ? Why we don`t need to deduplicate in appropriate case ? [1]