I see that if you have a hint on, say, the root node then it would be nice for
its child or grand-child to be able to see that hint.
How about giving each hint an inherit path? Thus given
Filter Hint1
+- Join
+- Scan
+- Project Hint2
+- Scan
Filter would have hints {Hint1[]}
Join would have hints {Hint1[0]}
Scan would have hints {Hint1[0, 0]}
Project would have hints {Hint1[0,1], Hint2}
Scan2 would have hints {[Hint1[0, 0, 1, 0], Hint2[0]}
You could populate the hints and inherit paths with a single visitor pass after
sql-to-rel conversion.
By the way, I still like the idea of having kinds as a kind of RelMetadata, but
I realize that a given RelNode might have more than one hint. So I think that
the getHints(RelNode) method would return a List<Hint>, with Hint as follows:
class Hint {
public final List<Integer> inheritPath; // immutable, not null
public final String type; // not null
public final Object operand; // immutable, may be null, must be JSON data
}
operand must be JSON-style data (null, boolean, number, String, immutable List
of JSON data, or immutable order-preserving Map from String to JSON data).
> On Apr 23, 2019, at 1:25 AM, Yuzhao Chen <yuzhao....@gmail.com> wrote:
>
> Thx, Andrew
>
> I don’t want to have a custom RelNode class, I hope all the work about hints
> would be contributed to the community. I want to find an acceptable way to
> keep and propagate the hints if we use the MetadataHandler to cache and query
> the hints.
>
> I don’t think the hints should be mixed into the cost model, that would make
> the cost computation very complex and hard to maintain, we only need the
> hints in our planning phrase to give suggestions, hints is more like another
> guideline for me and transparent to the planner.
>
> Best,
> Danny Chan
> 在 2019年4月23日 +0800 PM2:24，Андрей Цвелодуб <a.tsvelo...@gmail.com>，写道：
>> Hi Danny,
>>
>> I would also agree with Julian on his position. I've tried to get around
>> this limitation in several different ways, but none of it ended well :)
>>
>> For your idea with hints, if you have custom RelNode classes, you can add
>> hint as an additional field of the class and you can write a simple rule
>> that propagates the hint downwards, step by step. And also include the hint
>> in your cost estimation, so that nodes with hints would be more attractive
>> to the planner. I'm not sure this would be the most correct way to use the
>> cost mechanism, but at least it is straightforward and it works.
>>
>> Best Regards,
>> Andrew Tsvelodub
>>
>> On Tue, 23 Apr 2019 at 08:44, Yuzhao Chen <yuzhao....@gmail.com> wrote:
>>
>>> Julian,
>>>
>>> I want to add hint support for Calcite, the initial idea was to tag a
>>> RelNode(transformed from a SqlNode with hint) with a hit attribute(or
>>> trait), then I hope that the children (inputs) of it can see this hint, so
>>> to make some decisions if it should consume or propagate the hint.
>>>
>>> The problem I got here is the trait propagate from inputs from, which is
>>> the opposite as what I need, can you give some suggestions ? If I use
>>> MetadataHandler to cache and propagate the hints, how to propagate from
>>> parents to children ?
>>>
>>> Best,
>>> Danny Chan
>>> 在 2019年4月23日 +0800 AM3:14，Julian Hyde <jh...@apache.org>，写道：
>>>> TL;DR: RelNodes don’t really have parents. Be careful if you are relying
>>> on the parent concept too much. Rely on rules instead.
>>>>
>>>> In the Volcano model, a RelNode doesn’t really have a parent. It might
>>> be used in several places. (RelSet has a field ‘List<RelNode> parents’ that
>>> is kept up to date as planing progresses. But it’s really for Volcano’s
>>> internal use.)
>>>>
>>>> Even if you are not using Volcano, there are reasons to want the RelNode
>>> graph to be a dag, so again, a RelNode doesn’t have a unique parent.
>>>>
>>>> RelShuttleImpl has a stack. You can use that to find the parent. But the
>>> “parent” is just “where we came from as we traversed the RelNode graph”.
>>> There may be other “parents” that you do not know about.
>>>>
>>>> If you have a Project and want to find all parents that are Filters,
>>> don’t even think about “iterating over the parents” of the Project. Just
>>> write a rule that matches a Filter on a Project, and trust Volcano to do
>>> its job.
>>>>
>>>> Julian
>>>>
>>>>
>>>>
>>>>
>>>>> On Apr 22, 2019, at 6:15 AM, Yuzhao Chen <yuzhao....@gmail.com> wrote:
>>>>>
>>>>> Thx, Stamatis, that somehow make sense, if i pass around the parent
>>> node every time I visit a RelNode and keep the parents in the cache, but it
>>> is still not that intuitive. Actually I what a to add a new RelTrait which
>>> bind to a specific scope, for example:
>>>>>
>>>>> join-rel(trait1)
>>>>> / \
>>>>> join2 join3
>>>>>
>>>>> Join-rel has a trait trait1, and I want all the children of join-rel
>>> can see this trait, with Calcite’s default metadata handler, I can only see
>>> the trait from children nodes(traits propagate from the inputs), and I have
>>> no idea how to propagate a trait reversely?
>>>>>
>>>>>
>>>>> Best,
>>>>> Danny Chan
>>>>> 在 2019年4月22日 +0800 PM8:44，Stamatis Zampetakis <zabe...@gmail.com>，写道：
>>>>>> Hi Danny,
>>>>>>
>>>>>> Apart from RelShuttle there is also RelVisitor which has a visit
>>> method
>>>>>> that provides the parent [1]. Not sure, if it suits your needs.
>>>>>>
>>>>>> Best,
>>>>>> Stamatis
>>>>>>
>>>>>> [1]
>>>>>>
>>> https://github.com/apache/calcite/blob/ee83efd360793ef4201f4cdfc2af8d837b76ca69/core/src/main/java/org/apache/calcite/rel/RelVisitor.java#L43
>>>>>>
>>>>>>
>>>>>> On Mon, Apr 22, 2019 at 2:14 PM Yuzhao Chen <yuzhao....@gmail.com>
>>> wrote:
>>>>>>
>>>>>>> Now for RelNode, we have method getInput()[1] to fetch the input
>>>>>>> RelNodes, but how we fetch the parent ?
>>>>>>>
>>>>>>> For example, we have plan:
>>>>>>>
>>>>>>> join-rel
>>>>>>> / \
>>>>>>> scan1 scan2
>>>>>>>
>>>>>>>
>>>>>>> We can get scan1 and scan2 in join-rel directly with method
>>> getInput, but
>>>>>>> how can we get the join rel in scan1 and scan 2 ?
>>>>>>>
>>>>>>> I know that there is a RelShuttle that can visit every RelNode and
>>> if I
>>>>>>> make a cache for the inputs mapping, finally I can get the
>>> ‘parents’ from
>>>>>>> the cache, but this is boring code and not that intuitive.
>>>>>>>
>>>>>>> Do you guys have any good ideas ?
>>>>>>>
>>>>>>> [1]
>>>>>>>
>>> https://github.com/apache/calcite/blob/ee83efd360793ef4201f4cdfc2af8d837b76ca69/core/src/main/java/org/apache/calcite/rel/RelNode.java#L132
>>>>>>>
>>>>>>>
>>>>>>> Best,
>>>>>>> Danny Chan
>>>>>>>
>>>>
>>>
