Re: Enumerable rules lost hint info

[JiaTao Tao]

+1 With Haisheng
Besides, seems we will lose hint even in the logical
plan,  in RelBuilder#project, we just construct a project without
hints(ImmutableList.of()).

Regards!

Aron Tao


Haisheng Yuan  于2021年5月21日周五 上午9:20写道：

The hint can be used to specify the degree of parallelism (DOP), MIN/MAX
memory allocated for the operator. In that case, we need to keep them in
the physical operators. But I am not sure whether there are downstream
projects that are using hints for physical resource.

On 2021/05/19 17:05:44, Julian Hyde  wrote:
> Sure, there’s a slot to hold the hints. But is there a benefit to keeping
them in the physical algebra? There is certainly a cost to storing them,
propagating them, fixing them up as we apply rules.
>
> The main source of hints is formatted comments in the SQL. As we get
further from the SQL source and the AST, the benefit decreases.
>
> Danny,
>
> Were you envisioning hints being propagated into physical algebra?
There’s no explicit mention in
https://docs.google.com/document/d/1mykz-w2t1Yw7CH6NjUWpWqCAf_6YNKxSc59gXafrNCs/edit
<
https://docs.google.com/document/d/1mykz-w2t1Yw7CH6NjUWpWqCAf_6YNKxSc59gXafrNCs/edit>
or https://issues.apache.org/jira/browse/CALCITE-482 <
https://issues.apache.org/jira/browse/CALCITE-482>.
>
> Julian
>
>
> > On May 18, 2021, at 7:49 PM, JiaTao Tao  wrote:
> >
> > Hi Julian
> >
> > When hints were introduced, was that part of the plan?
> >
> > "hint" is a field in the base class, so the type of the node doesn't
matter
> >
> > public abstract class Project extends SingleRel implements Hintable {
> >  //~ Instance fields

> >
> >  protected final ImmutableList exps;
> >
> >  protected final ImmutableList hints;
> >
> >
> > And in RelBuilder#project, we just construct a project without
> > hints(ImmutableList.of()), so it will lose hints too.
> >
> >  public RelBuilder project(Iterable nodes,
> >  Iterable fieldNames, boolean force) {
> >return project_(nodes, fieldNames, ImmutableList.of(), force);
> >  }
> >
> >
> >
> > Regards!
> >
> > Aron Tao
> >
> >
> > Julian Hyde  于2021年5月19日周三 上午12:30写道：
> >
> >> Do we really need hints in physical operators? When hints were
introduced,
> >> was that part of the plan?
> >>
> >> Julian
> >>
> >>> On May 17, 2021, at 11:50 PM, JiaTao Tao  wrote:
> >>>
> >>> Hi
> >>> Recently, I'm integrating Calcite's hint into our system, and I've
found
> >>> the Enumerable rules just throw the hint info(ImmutableList.of())
away:
> >>>
> >>> public EnumerableProject(
> >>> RelOptCluster cluster,
> >>> RelTraitSet traitSet,
> >>> RelNode input,
> >>> List projects,
> >>> RelDataType rowType) {
> >>>   super(cluster, traitSet, ImmutableList.of(), input, projects,
> >> rowType);
> >>>   assert getConvention() instanceof EnumerableConvention;
> >>> }
> >>>
> >>>
> >>> Regards!
> >>>
> >>> Aron Tao
> >>
>
>

Re: Enumerable rules lost hint info

[Haisheng Yuan]

The hint can be used to specify the degree of parallelism (DOP), MIN/MAX memory 
allocated for the operator. In that case, we need to keep them in the physical 
operators. But I am not sure whether there are downstream projects that are 
using hints for physical resource.

On 2021/05/19 17:05:44, Julian Hyde  wrote: 
> Sure, there’s a slot to hold the hints. But is there a benefit to keeping 
> them in the physical algebra? There is certainly a cost to storing them, 
> propagating them, fixing them up as we apply rules.
> 
> The main source of hints is formatted comments in the SQL. As we get further 
> from the SQL source and the AST, the benefit decreases.
> 
> Danny,
> 
> Were you envisioning hints being propagated into physical algebra? There’s no 
> explicit mention in 
> https://docs.google.com/document/d/1mykz-w2t1Yw7CH6NjUWpWqCAf_6YNKxSc59gXafrNCs/edit
>  
> 
>  or https://issues.apache.org/jira/browse/CALCITE-482 
> .
> 
> Julian
> 
> 
> > On May 18, 2021, at 7:49 PM, JiaTao Tao  wrote:
> > 
> > Hi Julian
> > 
> > When hints were introduced, was that part of the plan?
> > 
> > "hint" is a field in the base class, so the type of the node doesn't matter
> > 
> > public abstract class Project extends SingleRel implements Hintable {
> >  //~ Instance fields 
> > 
> > 
> >  protected final ImmutableList exps;
> > 
> >  protected final ImmutableList hints;
> > 
> > 
> > And in RelBuilder#project, we just construct a project without
> > hints(ImmutableList.of()), so it will lose hints too.
> > 
> >  public RelBuilder project(Iterable nodes,
> >  Iterable fieldNames, boolean force) {
> >return project_(nodes, fieldNames, ImmutableList.of(), force);
> >  }
> > 
> > 
> > 
> > Regards!
> > 
> > Aron Tao
> > 
> > 
> > Julian Hyde  于2021年5月19日周三 上午12:30写道：
> > 
> >> Do we really need hints in physical operators? When hints were introduced,
> >> was that part of the plan?
> >> 
> >> Julian
> >> 
> >>> On May 17, 2021, at 11:50 PM, JiaTao Tao  wrote:
> >>> 
> >>> Hi
> >>> Recently, I'm integrating Calcite's hint into our system, and I've found
> >>> the Enumerable rules just throw the hint info(ImmutableList.of()) away:
> >>> 
> >>> public EnumerableProject(
> >>> RelOptCluster cluster,
> >>> RelTraitSet traitSet,
> >>> RelNode input,
> >>> List projects,
> >>> RelDataType rowType) {
> >>>   super(cluster, traitSet, ImmutableList.of(), input, projects,
> >> rowType);
> >>>   assert getConvention() instanceof EnumerableConvention;
> >>> }
> >>> 
> >>> 
> >>> Regards!
> >>> 
> >>> Aron Tao
> >> 
> 
>