Hi Jark, Sounds good and I would love to, thanks! I will involve you and Xingtong on the document after updating.
Best, Shammon FY On Mon, Aug 14, 2023 at 10:39 PM Jark Wu <imj...@gmail.com> wrote: > Hi Shammon, > > Sure, could you help to draft a subsection about this in the google doc? > > Best, > Jark > > 2023年8月14日 20:30，Shammon FY <zjur...@gmail.com> 写道： > > Thanks @Jark for driving the Flink Roadmap. > > As we discussed olap in the thread [1] and according to the suggestions > from @Xingtong Song, could we add a subsection in `Towards Streaming > Warehouses` or `Performance` that the short-lived query in Flink Session > Cluster is one of the future directions for Flink? > > Best, > Shammon FY > > On Mon, Aug 14, 2023 at 8:03 PM Jark Wu <imj...@gmail.com> wrote: > >> Thank you everyone for helping polish the roadmap [1]. >> >> I think I have addressed all the comments and we have included all ongoing >> parts of Flink. >> Please feel free to take a last look. I'm going to prepare the pull >> request >> if there are no more concerns. >> >> Best, >> Jark >> >> [1]: >> >> https://docs.google.com/document/d/12BDiVKEsY-f7HI3suO_IxwzCmR04QcVqLarXgyJAb7c/edit >> >> On Sun, 13 Aug 2023 at 13:04, Yuan Mei <yuanmei.w...@gmail.com> wrote: >> >> > Sorry for taking so long >> > >> > I've added a section about Flink Disaggregated State Management >> Evolution >> > in the attached doc. >> > >> > I found some of the contents might be overlapped with the "large-scale >> > streaming jobs". So that part might need some changes as well. >> > >> > Please let me know what you think. >> > >> > Best >> > Yuan >> > >> > On Mon, Jul 24, 2023 at 12:07 PM Yuan Mei <yuanmei.w...@gmail.com> >> wrote: >> > >> > > Sorry have missed this email and respond a bit late. >> > > >> > > I will put a draft for the long-term vision for the state as well as >> > > large-scale state support into the roadmap. >> > > >> > > Best >> > > Yuan >> > > >> > > On Mon, Jul 17, 2023 at 10:34 AM Jark Wu <imj...@gmail.com> wrote: >> > > >> > >> Hi Jiabao, >> > >> >> > >> Thank you for your suggestions. I have added them to the "Going >> Beyond a >> > >> SQL Stream/Batch Processing Engine" and "Large-Scale State Jobs" >> > sections. >> > >> >> > >> Best, >> > >> Jark >> > >> >> > >> On Thu, 13 Jul 2023 at 16:06, Jiabao Sun <jiabao....@xtransfer.cn >> > >> .invalid> >> > >> wrote: >> > >> >> > >> > Thanks Jark and Martijn for driving this. >> > >> > >> > >> > There are two suggestions about the Table API: >> > >> > >> > >> > - Add the JSON type to adapt to the no sql database type. >> > >> > - Remove changelog normalize operator for upsert stream. >> > >> > >> > >> > >> > >> > Best, >> > >> > Jiabao >> > >> > >> > >> > >> > >> > > 2023年7月13日 下午3:49，Jark Wu <imj...@gmail.com> 写道： >> > >> > > >> > >> > > Hi all, >> > >> > > >> > >> > > Sorry for taking so long back here. >> > >> > > >> > >> > > Martijn and I have drafted the first version of the updated >> roadmap, >> > >> > > including the updated feature radar reflecting the current state >> of >> > >> > > different components. >> > >> > > >> > >> > >> > >> >> > >> https://docs.google.com/document/d/12BDiVKEsY-f7HI3suO_IxwzCmR04QcVqLarXgyJAb7c/edit >> > >> > > >> > >> > > Feel free to leave comments in the thread or the document. >> > >> > > We may miss mentioning something important, so your help in >> > enriching >> > >> > > the content is greatly appreciated. >> > >> > > >> > >> > > Best, >> > >> > > Jark & Martijn >> > >> > > >> > >> > > >> > >> > > On Fri, 2 Jun 2023 at 00:50, Jing Ge <j...@ververica.com.invalid >> > >> > >> wrote: >> > >> > > >> > >> > >> Hi Jark, >> > >> > >> >> > >> > >> Fair enough. Let's do it like you suggested. Thanks! >> > >> > >> >> > >> > >> Best regards, >> > >> > >> Jing >> > >> > >> >> > >> > >> On Thu, Jun 1, 2023 at 6:00 PM Jark Wu <imj...@gmail.com> >> wrote: >> > >> > >> >> > >> > >>> Hi Jing, >> > >> > >>> >> > >> > >>> This thread is for discussing the roadmap for versions 1.18, >> 2.0, >> > >> and >> > >> > >> even >> > >> > >>> more. >> > >> > >>> One of the outcomes of this discussion will be an updated >> version >> > of >> > >> > the >> > >> > >>> current roadmap. >> > >> > >>> Let's work together on refining the roadmap in this thread. >> > >> > >>> >> > >> > >>> Best, >> > >> > >>> Jark >> > >> > >>> >> > >> > >>> On Thu, 1 Jun 2023 at 23:25, Jing Ge >> <j...@ververica.com.invalid> >> > >> > wrote: >> > >> > >>> >> > >> > >>>> Hi Jark, >> > >> > >>>> >> > >> > >>>> Thanks for driving it! For point 2, since we are developing >> 1.18 >> > >> now, >> > >> > >>>> does it make sense to update the roadmap this time while we >> are >> > >> > >> releasing >> > >> > >>>> 1.18? This discussion thread will be focusing on the Flink 2.0 >> > >> > roadmap, >> > >> > >>> as >> > >> > >>>> you mentioned previously. WDYT? >> > >> > >>>> >> > >> > >>>> Best regards, >> > >> > >>>> Jing >> > >> > >>>> >> > >> > >>>> On Thu, Jun 1, 2023 at 3:31 PM Jark Wu <imj...@gmail.com> >> wrote: >> > >> > >>>> >> > >> > >>>>> Hi all, >> > >> > >>>>> >> > >> > >>>>> Martijn and I would like to initiate a discussion on the >> Flink >> > >> > >> roadmap, >> > >> > >>>>> which should cover the project's long-term roadmap and the >> > regular >> > >> > >>> update >> > >> > >>>>> mechanism. >> > >> > >>>>> >> > >> > >>>>> Xintong has already started a discussion about Flink 2.0 >> > planning. >> > >> > >> One >> > >> > >>> of >> > >> > >>>>> the points raised in that discussion is that we should have a >> > >> > >>> high-level >> > >> > >>>>> discussion of the roadmap to present where the project is >> > heading >> > >> > >>> (which >> > >> > >>>>> doesn't necessarily need to block the Flink 2.0 planning). >> > >> Moreover, >> > >> > >>> the >> > >> > >>>>> roadmap on the Flink website [1] hasn't been updated for >> half a >> > >> year, >> > >> > >>> and >> > >> > >>>>> the last update was for the feature radar for the 1.15 >> release. >> > It >> > >> > >> has >> > >> > >>>> been >> > >> > >>>>> 2 years since the community discussed Flink's overall >> roadmap. >> > >> > >>>>> >> > >> > >>>>> I would like to raise two topics for discussion: >> > >> > >>>>> >> > >> > >>>>> 1. The new roadmap. This should be an updated version of the >> > >> current >> > >> > >>>>> roadmap[1]. >> > >> > >>>>> 2. A mechanism to regularly discuss and update the roadmap. >> > >> > >>>>> >> > >> > >>>>> To make the first topic discussion more efficient, Martijn >> and I >> > >> > >>>> volunteer >> > >> > >>>>> to summarize the ongoing big things of different components >> and >> > >> > >>> present a >> > >> > >>>>> roadmap draft to the community in the next few weeks. This >> > should >> > >> be >> > >> > >> a >> > >> > >>>> good >> > >> > >>>>> starting point for a more detailed discussion. >> > >> > >>>>> >> > >> > >>>>> Regarding the regular update mechanism, there was a proposal >> in >> > a >> > >> > >>> thread >> > >> > >>>>> [2] three years ago to make the release manager responsible >> for >> > >> > >>> updating >> > >> > >>>>> the roadmap. However, it appears that this was not documented >> > as a >> > >> > >>>> release >> > >> > >>>>> management task [3], and the roadmap update wasn't performed >> for >> > >> > >>> releases >> > >> > >>>>> 1.16 and 1.17. >> > >> > >>>>> >> > >> > >>>>> In my opinion, making release managers responsible for >> keeping >> > the >> > >> > >>>> roadmap >> > >> > >>>>> up to date is a good idea. Specifically, release managers of >> > >> release >> > >> > >> X >> > >> > >>>> can >> > >> > >>>>> kick off the roadmap update at the beginning of release X, >> which >> > >> can >> > >> > >>> be a >> > >> > >>>>> joint task with collecting a feature list [4]. Additionally, >> > >> release >> > >> > >>>>> managers of release X-1 can help verify and remove the >> > >> accomplished >> > >> > >>> items >> > >> > >>>>> from the roadmap and update the feature radar. >> > >> > >>>>> >> > >> > >>>>> What do you think? Do you have other ideas? >> > >> > >>>>> >> > >> > >>>>> Best, >> > >> > >>>>> Jark & Martijn >> > >> > >>>>> >> > >> > >>>>> [1]: https://flink.apache.org/roadmap.html >> > >> > >>>>> [2]: >> > >> > >> >> https://lists.apache.org/thread/o0l3cg6yphxwrww0k7215jgtw3yfoybv >> > >> > >>>>> [3]: >> > >> > >>>>> >> > >> > >>>> >> > >> > >>> >> > >> > >> >> > >> > >> > >> >> > >> https://cwiki.apache.org/confluence/display/FLINK/Flink+Release+Management >> > >> > >>>>> [4]: >> > >> https://cwiki.apache.org/confluence/display/FLINK/1.18+Release >> > >> > >>>>> >> > >> > >>>> >> > >> > >>> >> > >> > >> >> > >> > >> > >> > >> > >> >> > > >> > >> > >
