Hi everyone, I have added a section for Performance Optimization to describe how to improve the performance in the short-term and long-term and sketch the future performance potential under the new window API. Introducing the window API is just the first step, we will continuously improve the performance to make it powerful and useful.
Best, Jark On Thu, 1 Oct 2020 at 14:28, Jark Wu <imj...@gmail.com> wrote: > Hi Pengcheng, > > Yes, the window TVF is part of the FLIP. Welcome to contribute and join > the discussion. > Regarding the SESSION window aggregation, users can use the existing > grouped session window function. > > Best, > Jark > > On Sun, 27 Sep 2020 at 21:24, liupengcheng <pengchengliucr...@gmail.com> > wrote: > >> Hi Jark, >> Thanks for reply, yes, I think it's a good feature, it can >> improve the NRT scenarios >> as you mentioned in the FLIP. Also, I think it can improve the >> streaming SQL greatly, >> it can support richer window operations in flink SQL and bring >> great convenience to users. >> (we are now only supported group window in flink). >> >> Regarding the SESSION window, I think it's especially useful for >> user behavior analysis(e.g. >> counting user visits on a news website or social platform), but I >> agree that we can keep it >> out of the FLIP now to catch up 1.12. >> >> Recently, I've done some work on the stream planner with the >> TVFs, and I'm willing to contribute >> to this part. Is it in the plan of this FLIP? >> >> Best, >> PengchengLiu >> >> >> 在 2020/9/26 下午11:09，“Jark Wu”<imj...@gmail.com> 写入: >> >> Hi pengcheng, >> >> That's great to see you also have the need of window join. >> You are right, the windowing TVF is a powerful feature which can >> support >> more operations in the future. >> I think it as of the date time "partition" selection in batch SQL >> jobs, >> with this new syntax, I think it is possible >> to migrate traditional batch SQL jobs to Flink SQL by changing a few >> lines. >> >> Regarding the SESSION window, this is on purpose to keep it out of the >> FLIP, because we want to keep the >> FLIP small to catch up 1.12 and SESSION TVF is rarely useful (e.g. >> session >> window join?). >> >> Best, >> Jark >> >> On Fri, 25 Sep 2020 at 22:59, liupengcheng < >> pengchengliucr...@gmail.com> >> wrote: >> >> > Hi, Jark, >> > I'm very interested in this feature, and I'm also working >> on this >> > recently. >> > I just have a glance at the FLIP, it's good, but I found >> that >> > there is no plan to add SESSION windows. >> > Also, I think there can be more things we can do based on >> this new >> > syntax. For example, >> > - window sort support >> > - window union/intersect/minus support >> > - Improve dimension table join >> > We can have more deep discussion on this new feature later . >> > I've also opened an jira that is related to this feature >> recently: >> > https://issues.apache.org/jira/browse/FLINK-18830 >> > >> > Best! >> > PengchengLiu >> > >> > 在 2020/9/25 下午10:30，“Jark Wu”<imj...@gmail.com> 写入: >> > >> > Hi everyone, >> > >> > I want to start a FLIP about supporting windowing table-valued >> > functions >> > (TVF). >> > The main purpose of this FLIP is to improve the near real-time >> (NRT) >> > experience of Flink. >> > >> > FLIP-145: >> > >> > >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-145%3A+Support+SQL+windowing+table-valued+function >> > >> > We want to introduce TUMBLE, HOP, CUMULATE windowing TVFs, the >> > CUMULATE is >> > a new kind of window. >> > With the windowing TVFs, we can support richer operations on >> windows, >> > including window join, window TopN and so on. >> > This makes things simple: we only need to assign windows at the >> > beginning >> > of the query, and then apply operations after that like >> traditional >> > batch >> > SQL. >> > We hope it can help to reduce the learning curve of windows, >> improve >> > NRT >> > for Flink, and attract more batch users. >> > >> > A simple code snippet for 10 minutes tumbling window aggregate: >> > >> > SELECT window_start, window_end, SUM(price) >> > FROM TABLE( >> > TUMBLE(TABLE Bid, DESCRIPTOR(bidtime), INTERVAL '10' >> MINUTES)) >> > GROUP BY window_start, window_end; >> > >> > I'm looking forward to your feedback. >> > >> > Best, >> > Jark >> > >> > >> > >> >> >>
