Re: about optimization rules for over(partition by id, age order by id, age)
FWIW, FLINK-12575 is a mechanism that removes redundant shuffle/sort in physical stage, and it's not a a generic rule, but it's in AbstractConverter while we try to satisfy Distribution and Collation trait. I'm not aware of any existing rules/RelBuilder in Calcite that has this optimization, it sounds like a good improvement. key lou 于2024年2月23日周五 11:09写道： > > thanks I also found a rule in fink. > https://issues.apache.org/jira/browse/FLINK-12575 > > Julian Hyde 于2024年2月23日周五 09:04写道： > > > I noticed that DuckDB implemented such an optimization recently. > > > > Can you please log a jira case to remove redundant ORDER BY keys from > > OVER? Yes, it would be either a planner rule or a rewrite by RelBuilder. > > > > Julian > > > > > > > On Feb 21, 2024, at 11:43 PM, key lou wrote: > > > > > > HI ALL : > > > I have the following sql: > > > select name, row_number() over(partition by id, age order by id, age) rn > > > from t; In fact, it can be optimized into > > > select name ,row_number() over(partition by id,age) rn from t; > > > Remove order by because order by and partition by are repeated. Since > > they > > > are grouped according to the partition by field, the sorting is > > > meaningless. Is there such a solution in calcite? Optimize rules? > > > thanks. > > > > -- Best, Benchao Li
Re: about optimization rules for over(partition by id, age order by id, age)
thanks I also found a rule in fink. https://issues.apache.org/jira/browse/FLINK-12575 Julian Hyde 于2024年2月23日周五 09:04写道： > I noticed that DuckDB implemented such an optimization recently. > > Can you please log a jira case to remove redundant ORDER BY keys from > OVER? Yes, it would be either a planner rule or a rewrite by RelBuilder. > > Julian > > > > On Feb 21, 2024, at 11:43 PM, key lou wrote: > > > > HI ALL : > > I have the following sql: > > select name, row_number() over(partition by id, age order by id, age) rn > > from t; In fact, it can be optimized into > > select name ,row_number() over(partition by id,age) rn from t; > > Remove order by because order by and partition by are repeated. Since > they > > are grouped according to the partition by field, the sorting is > > meaningless. Is there such a solution in calcite? Optimize rules? > > thanks. > >
Re: about optimization rules for over(partition by id, age order by id, age)
I noticed that DuckDB implemented such an optimization recently. Can you please log a jira case to remove redundant ORDER BY keys from OVER? Yes, it would be either a planner rule or a rewrite by RelBuilder. Julian > On Feb 21, 2024, at 11:43 PM, key lou wrote: > > HI ALL : > I have the following sql: > select name, row_number() over(partition by id, age order by id, age) rn > from t; In fact, it can be optimized into > select name ,row_number() over(partition by id,age) rn from t; > Remove order by because order by and partition by are repeated. Since they > are grouped according to the partition by field, the sorting is > meaningless. Is there such a solution in calcite? Optimize rules? > thanks.
about optimization rules for over(partition by id, age order by id, age)
HI ALL : I have the following sql: select name, row_number() over(partition by id, age order by id, age) rn from t; In fact, it can be optimized into select name ,row_number() over(partition by id,age) rn from t; Remove order by because order by and partition by are repeated. Since they are grouped according to the partition by field, the sorting is meaningless. Is there such a solution in calcite? Optimize rules? thanks.
