Re: [DISCUSS] Towards Calcite 1.37.0

[Stamatis Zampetakis]

With so many commits it's definitely a good time to cut a new release.

I can be the RM for 1.39.0 or 1.40.0 depending on the timing.

Best,
Stamatis

On Wed, Feb 21, 2024 at 12:10 AM Sergey Nuyanzin  wrote:
>
> Thanks for starting the discussion
>
> > @Sergey, are you still available for being the release manager for 1.37.0?
>
> I think so, in theory I could start with some steps in one/two weeks if
> there is no objections
>
>
>
> On Mon, Feb 19, 2024 at 12:29 PM Benchao Li  wrote:
>
> > It's been a bit more than 3 months since our last release (2023-11-10)
> > [1] and there are currently 100+ new commits in main branch. Per our
> > tradition of producing one release every 2-3 months, I think it's time
> > to consider for next release now.
> >
> > According to [2], the following release managers would be:
> > - 1.37.0 Sergey Nuyanzin
> > - 1.38.0 Julian Hyde
> >
> > @Sergey, are you still available for being the release manager for 1.37.0?
> >
> > Besides, we need more volunteers for the next releases (version >=
> > 1.39.0), anyone who is interested in doing it can reply in this
> > thread.
> >
> > [1] https://calcite.apache.org/docs/history.html#v1-36-0
> > [4] https://lists.apache.org/thread/tm3t42qvpq3db24xtd2g468ofv83l6hk
> >
> >
> > --
> >
> > Best,
> > Benchao Li
> >
>
>
> --
> Best regards,
> Sergey

Re: What's the necessity of the HepRelVertex used in HepPlanner

[Julian Hyde]

Let’s suppose that the information in HepRelVertex could somehow be moved into 
the RelNode. (Since the data structure is a DAG it seems plausible.) Now each 
RelNode would contain mutable state that relates to the HEP planner’s 
algorithm. That state needs to be managed (e.g. reinitialized when starting a 
new planner phase). It also makes RelNodes mutable, which seems to give 
permission for people to start adding more mutable state. 

Also, remember that RelNodes and RelRules can be used, unchanged, by 
VolcanoPlanner. Externalizing the state makes this possible. 

When I started my career writing C, external lists seemed an extravagant use of 
memory. If a particular struct needed to be a member of 3 kinds of list, you 
added 3 pairs of next/previous pointers to each struct. One fewer indirection 
to get your data, but hours of debugging to manage the lifecycle. Internal 
state is usually a false economy. 

> On Feb 20, 2024, at 11:57 PM, Alessandro Solimando 
>  wrote:
> 
> Hi Ayaka,
> I agree with Benchao, you can look at this article to get a sense on why a
> data structure allowing node substitution is useful in query planning:
> https://www.querifylabs.com/blog/memoization-in-cost-based-optimizers
> 
> Best regards,
> Alessandro
> 
>> On Wed, 21 Feb 2024 at 05:55, Benchao Li  wrote:
>> 
>> Ayaka,
>> 
>> Per my understanding, HepRelVertex is used to make RelNode changeable
>> easily, you can see more about HepRelVertex#replaceRel, it is used to
>> change one RelNode to another after some rules applied and new equal
>> node produced. In this way, we can easily change subgraphs of the
>> original DAG after performing some RBO rules.
>> 
>> 
>> Ayaka Kamisato  于2024年2月19日周一 19:29写道：
>>> 
>>> I'm studying Calcite recently and I found that RelNode Tree would be
>>> converted to a DAG composed by HepRelVertex and Edge before the RBO rules
>>> execution, and I don't understand why this step is necessary since
>> RelNode
>>> tree is a DAG essentially.
>> 
>> 
>> 
>> --
>> 
>> Best,
>> Benchao Li
>> 

about optimization rules for over(partition by id, age order by id, age)

[key lou]

HI ALL :
   I have the following sql:
select name, row_number() over(partition by id, age order by id, age) rn
from t; In fact, it can be optimized into
select name ,row_number() over(partition by id,age) rn from t;
Remove order by because order by and partition by are repeated. Since they
are grouped according to the partition by field, the sorting is
meaningless. Is there such a solution in calcite? Optimize rules?
thanks.