Thank you everyone for helping polish the roadmap [1]. I think I have addressed all the comments and we have included all ongoing parts of Flink. Please feel free to take a last look. I'm going to prepare the pull request if there are no more concerns.
Best, Jark [1]: https://docs.google.com/document/d/12BDiVKEsY-f7HI3suO_IxwzCmR04QcVqLarXgyJAb7c/edit On Sun, 13 Aug 2023 at 13:04, Yuan Mei <[email protected]> wrote: > Sorry for taking so long > > I've added a section about Flink Disaggregated State Management Evolution > in the attached doc. > > I found some of the contents might be overlapped with the "large-scale > streaming jobs". So that part might need some changes as well. > > Please let me know what you think. > > Best > Yuan > > On Mon, Jul 24, 2023 at 12:07 PM Yuan Mei <[email protected]> wrote: > > > Sorry have missed this email and respond a bit late. > > > > I will put a draft for the long-term vision for the state as well as > > large-scale state support into the roadmap. > > > > Best > > Yuan > > > > On Mon, Jul 17, 2023 at 10:34 AM Jark Wu <[email protected]> wrote: > > > >> Hi Jiabao, > >> > >> Thank you for your suggestions. I have added them to the "Going Beyond a > >> SQL Stream/Batch Processing Engine" and "Large-Scale State Jobs" > sections. > >> > >> Best, > >> Jark > >> > >> On Thu, 13 Jul 2023 at 16:06, Jiabao Sun <[email protected] > >> .invalid> > >> wrote: > >> > >> > Thanks Jark and Martijn for driving this. > >> > > >> > There are two suggestions about the Table API: > >> > > >> > - Add the JSON type to adapt to the no sql database type. > >> > - Remove changelog normalize operator for upsert stream. > >> > > >> > > >> > Best, > >> > Jiabao > >> > > >> > > >> > > 2023年7月13日 下午3:49，Jark Wu <[email protected]> 写道： > >> > > > >> > > Hi all, > >> > > > >> > > Sorry for taking so long back here. > >> > > > >> > > Martijn and I have drafted the first version of the updated roadmap, > >> > > including the updated feature radar reflecting the current state of > >> > > different components. > >> > > > >> > > >> > https://docs.google.com/document/d/12BDiVKEsY-f7HI3suO_IxwzCmR04QcVqLarXgyJAb7c/edit > >> > > > >> > > Feel free to leave comments in the thread or the document. > >> > > We may miss mentioning something important, so your help in > enriching > >> > > the content is greatly appreciated. > >> > > > >> > > Best, > >> > > Jark & Martijn > >> > > > >> > > > >> > > On Fri, 2 Jun 2023 at 00:50, Jing Ge <[email protected]> > >> wrote: > >> > > > >> > >> Hi Jark, > >> > >> > >> > >> Fair enough. Let's do it like you suggested. Thanks! > >> > >> > >> > >> Best regards, > >> > >> Jing > >> > >> > >> > >> On Thu, Jun 1, 2023 at 6:00 PM Jark Wu <[email protected]> wrote: > >> > >> > >> > >>> Hi Jing, > >> > >>> > >> > >>> This thread is for discussing the roadmap for versions 1.18, 2.0, > >> and > >> > >> even > >> > >>> more. > >> > >>> One of the outcomes of this discussion will be an updated version > of > >> > the > >> > >>> current roadmap. > >> > >>> Let's work together on refining the roadmap in this thread. > >> > >>> > >> > >>> Best, > >> > >>> Jark > >> > >>> > >> > >>> On Thu, 1 Jun 2023 at 23:25, Jing Ge <[email protected]> > >> > wrote: > >> > >>> > >> > >>>> Hi Jark, > >> > >>>> > >> > >>>> Thanks for driving it! For point 2, since we are developing 1.18 > >> now, > >> > >>>> does it make sense to update the roadmap this time while we are > >> > >> releasing > >> > >>>> 1.18? This discussion thread will be focusing on the Flink 2.0 > >> > roadmap, > >> > >>> as > >> > >>>> you mentioned previously. WDYT? > >> > >>>> > >> > >>>> Best regards, > >> > >>>> Jing > >> > >>>> > >> > >>>> On Thu, Jun 1, 2023 at 3:31 PM Jark Wu <[email protected]> wrote: > >> > >>>> > >> > >>>>> Hi all, > >> > >>>>> > >> > >>>>> Martijn and I would like to initiate a discussion on the Flink > >> > >> roadmap, > >> > >>>>> which should cover the project's long-term roadmap and the > regular > >> > >>> update > >> > >>>>> mechanism. > >> > >>>>> > >> > >>>>> Xintong has already started a discussion about Flink 2.0 > planning. > >> > >> One > >> > >>> of > >> > >>>>> the points raised in that discussion is that we should have a > >> > >>> high-level > >> > >>>>> discussion of the roadmap to present where the project is > heading > >> > >>> (which > >> > >>>>> doesn't necessarily need to block the Flink 2.0 planning). > >> Moreover, > >> > >>> the > >> > >>>>> roadmap on the Flink website [1] hasn't been updated for half a > >> year, > >> > >>> and > >> > >>>>> the last update was for the feature radar for the 1.15 release. > It > >> > >> has > >> > >>>> been > >> > >>>>> 2 years since the community discussed Flink's overall roadmap. > >> > >>>>> > >> > >>>>> I would like to raise two topics for discussion: > >> > >>>>> > >> > >>>>> 1. The new roadmap. This should be an updated version of the > >> current > >> > >>>>> roadmap[1]. > >> > >>>>> 2. A mechanism to regularly discuss and update the roadmap. > >> > >>>>> > >> > >>>>> To make the first topic discussion more efficient, Martijn and I > >> > >>>> volunteer > >> > >>>>> to summarize the ongoing big things of different components and > >> > >>> present a > >> > >>>>> roadmap draft to the community in the next few weeks. This > should > >> be > >> > >> a > >> > >>>> good > >> > >>>>> starting point for a more detailed discussion. > >> > >>>>> > >> > >>>>> Regarding the regular update mechanism, there was a proposal in > a > >> > >>> thread > >> > >>>>> [2] three years ago to make the release manager responsible for > >> > >>> updating > >> > >>>>> the roadmap. However, it appears that this was not documented > as a > >> > >>>> release > >> > >>>>> management task [3], and the roadmap update wasn't performed for > >> > >>> releases > >> > >>>>> 1.16 and 1.17. > >> > >>>>> > >> > >>>>> In my opinion, making release managers responsible for keeping > the > >> > >>>> roadmap > >> > >>>>> up to date is a good idea. Specifically, release managers of > >> release > >> > >> X > >> > >>>> can > >> > >>>>> kick off the roadmap update at the beginning of release X, which > >> can > >> > >>> be a > >> > >>>>> joint task with collecting a feature list [4]. Additionally, > >> release > >> > >>>>> managers of release X-1 can help verify and remove the > >> accomplished > >> > >>> items > >> > >>>>> from the roadmap and update the feature radar. > >> > >>>>> > >> > >>>>> What do you think? Do you have other ideas? > >> > >>>>> > >> > >>>>> Best, > >> > >>>>> Jark & Martijn > >> > >>>>> > >> > >>>>> [1]: https://flink.apache.org/roadmap.html > >> > >>>>> [2]: > >> > >> https://lists.apache.org/thread/o0l3cg6yphxwrww0k7215jgtw3yfoybv > >> > >>>>> [3]: > >> > >>>>> > >> > >>>> > >> > >>> > >> > >> > >> > > >> > https://cwiki.apache.org/confluence/display/FLINK/Flink+Release+Management > >> > >>>>> [4]: > >> https://cwiki.apache.org/confluence/display/FLINK/1.18+Release > >> > >>>>> > >> > >>>> > >> > >>> > >> > >> > >> > > >> > > >> > > >
