Re: Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-27 Thread Martijn Visser
Hi Xuyang, It's currently the holiday season in Europe so do expect some slow responses. > The key reason the original FLIP is called "Deprecate the Legacy Group Window > Aggregation" is that we also plan to automatically rewrite the group window > agg corresponding the old syntax into the

Re: Re: Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-12 Thread Jane Chan
Hi Xuyang, Thank you for the clarification. I now have a clear understanding of the issue. Actually the work that supports unaligned window in window agg operator is > included in this flip. We will always try to implement > features in window agg operator first and then consider falling back to

Re: Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-11 Thread Jim Hughes
Hi Xuyang, On Sun, Dec 10, 2023 at 10:41 PM Xuyang wrote: > Hi, Jim. > >As a clarification, since FLINK-24204 is finishing up work from > >FLIP-145[1], do we need to discuss anything before you work out the > details > >of FLINK-24024 as a PR? > Which issue do you mean? It seems that

Re: Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-11 Thread Jane Chan
Hi Xuyang, Thanks for driving this discussion. I briefly reviewed the first PoC code[1] you provided for implementing the session window TVF, and I noticed a problem. In the `translateToExecNode` method of `StreamPhysicalWindowAggregate`, the session window TVF is translated into the legacy

Re:Re: Re: Re: Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-10 Thread Xuyang
Hi, Shengkai. > I think we shouldn't remove the operator if we can not give a solution to > help users upgrade their jobs. But I think we can delay the discussion until > we need to remove the operator. +1 for it. -- Best! Xuyang 在 2023-12-08 19:22:40,"Shengkai Fang"

Re: Re: Re: Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-08 Thread Shengkai Fang
Hi, Xuyang. Thanks for your response. I just thought an another way to solve this question instead of introducing > a new configuration. When using legacy syntax like `GROUP BY TUMBLE(xxx), > f0`, the rewritten sql can be GROUP BY f0, window_start, > window_end(window_start and window_end is

Re:Re: Re: Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-08 Thread Xuyang
Hi, Martijn, thanks for your share. >On the topic of syntax for early/late fires, there is existing >configuration for the legacy group windows: > >SET table.exec.emit.early-fire.enabled = true; >SET table.exec.emit.early-fire.delay = 5s; >SET table.exec.emit.late-fire.enabled = true; >SET

Re: Re: Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-07 Thread Martijn Visser
Hi Xuyang, Thanks a lot for starting this discussion. At first, I was a bit confused because the FLIP talks about deprecating the Legacy Group Window Aggregations, but they have already been marked as deprecated in the documentation [1]. My understanding was that the big challenge was that we

Re: Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-04 Thread liu ron
Hi, xuyang Thanks for starting this FLIP discussion, currently there are two types of window aggregation in Flink SQL, namely legacy group window aggregation and window tvf aggregation, these two types of window aggregation are not fully aligned in behavior, which will bring a lot of confusion to

Re: Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-04 Thread Shengkai Fang
Hi, Xuyang. Thanks for your great work. Big +1! After reading the FLIP, I have some questions. Please read the content below. 1. Support SESSION Window TVF Aggregation a. Could you give an example about the pass-through column? A session window may contain multiple rows, which value is