Re: [VOTE] Release 2.0 must-have work items - Round 2
Thanks all for participating. I'm closing this vote in another thread. Best, Xintong On Thu, Jul 27, 2023 at 11:37 AM Jiabao Sun wrote: > + 1 (non-binding) > > Thanks Xintong for driving this. > > Best, > Jiabao > > > On 2023/07/20 09:22:46 Xintong Song wrote: > > Hi all, > > > > I'd like to start another round of VOTE for the must-have work items for > > release 2.0 [1]. The corresponding discussion thread is [2], and the > > previous voting thread is [3]. All comments from the previous voting > thread > > have been addressed. > > > > Please note that once the vote is approved, any changes to the must-have > > items (adding / removing must-have items, changing the priority) requires > > another vote. Assigning contributors / reviewers, updating descriptions / > > progress, changes to nice-to-have items do not require another vote. > > > > The vote will be open until at least July 25, following the consensus > > voting process. Votes of PMC members are binding. > > > > Best, > > > > Xintong > > > > > > [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > > > [2] https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 > > > > [3] https://lists.apache.org/thread/r0y9syc6k5nmcxvnd0hj33htdpdj9k6m > > >
RE: [VOTE] Release 2.0 must-have work items - Round 2
+ 1 (non-binding) Thanks Xintong for driving this. Best, Jiabao On 2023/07/20 09:22:46 Xintong Song wrote: > Hi all, > > I'd like to start another round of VOTE for the must-have work items for > release 2.0 [1]. The corresponding discussion thread is [2], and the > previous voting thread is [3]. All comments from the previous voting thread > have been addressed. > > Please note that once the vote is approved, any changes to the must-have > items (adding / removing must-have items, changing the priority) requires > another vote. Assigning contributors / reviewers, updating descriptions / > progress, changes to nice-to-have items do not require another vote. > > The vote will be open until at least July 25, following the consensus > voting process. Votes of PMC members are binding. > > Best, > > Xintong > > > [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > [2] https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 > > [3] https://lists.apache.org/thread/r0y9syc6k5nmcxvnd0hj33htdpdj9k6m >
Re: [VOTE] Release 2.0 must-have work items - Round 2
+1 binding Thanks all for your work! Best, Jingsong On Thu, Jul 27, 2023 at 10:52 AM Jark Wu wrote: > > +1 (binding) > > Thanks Xintong for driving this. Thanks all for finalizing the > SourceFunction conclusion. > > Best, > Jark > > On Wed, 26 Jul 2023 at 22:28, Alexander Fedulov > wrote: > > > +1 (non-binding), assuming SourceFunction gets added back to the > > doc as a "nice-to-have". I am glad we've reached a consensus here. > > Extra thanks to Leonard for coordinating this discussion in particular. > > > > Best, > > Alex > > > > On Wed, 26 Jul 2023 at 15:43, Jing Ge wrote: > > > > > +1 (non-binding), glad to see we are now on the same page. Thank you all. > > > > > > Best regards, > > > Jing > > > > > > On Wed, Jul 26, 2023 at 5:18 PM Yun Tang wrote: > > > > > > > +1 (non-binding), thanks @xintong for driving this work. > > > > > > > > > > > > Best > > > > Yun Tang > > > > > > > > From: Zhu Zhu > > > > Sent: Wednesday, July 26, 2023 16:35 > > > > To: dev@flink.apache.org > > > > Subject: Re: [VOTE] Release 2.0 must-have work items - Round 2 > > > > > > > > +1 (binding) > > > > > > > > Thanks, > > > > Zhu > > > > > > > > Leonard Xu 于2023年7月26日周三 15:40写道: > > > > > > > > > > Thanks @xingtong for driving the work. > > > > > > > > > > +1(binding) > > > > > > > > > > Best, > > > > > Leonard > > > > > > > > > > > On Jul 26, 2023, at 3:18 PM, Konstantin Knauf < > > > > knauf.konstan...@gmail.com> wrote: > > > > > > > > > > > > Hi Xingtong, > > > > > > > > > > > > yes, I am fine with the conclusion for SourceFunction. I chatted > > with > > > > > > Leonard a bit last night. Let's continue this vote. > > > > > > > > > > > > Thanks for the clarification, > > > > > > > > > > > > Konstantin > > > > > > > > > > > > > > > > > > > > > > > > Am Mi., 26. Juli 2023 um 04:03 Uhr schrieb Xintong Song < > > > > > > tonysong...@gmail.com>: > > > > > > > > > > > >> Hi Konstantin, > > > > > >> > > > > > >> It seems the offline discussion has already taken place [1], and > > > part > > > > of > > > > > >> the outcome is that removal of SourceFunction would be a > > > > *nice-to-have* > > > > > >> item for release 2.0 which may not block this *must-have* vote. Do > > > > you have > > > > > >> different opinions about the conclusions in [1]? > > > > > >> > > > > > >> If there are still concerns, and the discussion around this topic > > > > needs to > > > > > >> be continued, then I'd suggest (as I mentioned in [2]) not to > > > further > > > > block > > > > > >> this vote (i.e. the decision on other must-have items). Release > > 2.0 > > > > still > > > > > >> has a long way to go, and it is likely we need to review and > > update > > > > the > > > > > >> list every once in a while. We can update the list with another > > vote > > > > if > > > > > >> later we decide to add the removal of SourceFunction to the > > > must-have > > > > list. > > > > > >> > > > > > >> WDYT? > > > > > >> > > > > > >> Best, > > > > > >> > > > > > >> Xintong > > > > > >> > > > > > >> > > > > > >> [1] > > > https://lists.apache.org/thread/yyw52k45x2sp1jszldtdx7hc98n72w7k > > > > > >> [2] > > > https://lists.apache.org/thread/j5d5022ky8k5t088ffm03727o5g9x9jr > > > > > >> > > > > > >> On Tue, Jul 25, 2023 at 8:49 PM Konstantin Knauf < > > kna...@apache.org > > > > > > > > > >> wrote: > > > > > >> > > > > > >>> I assume this vote includes a decision to not
Re: [VOTE] Release 2.0 must-have work items - Round 2
+1 (binding) Thanks Xintong for driving this. Thanks all for finalizing the SourceFunction conclusion. Best, Jark On Wed, 26 Jul 2023 at 22:28, Alexander Fedulov wrote: > +1 (non-binding), assuming SourceFunction gets added back to the > doc as a "nice-to-have". I am glad we've reached a consensus here. > Extra thanks to Leonard for coordinating this discussion in particular. > > Best, > Alex > > On Wed, 26 Jul 2023 at 15:43, Jing Ge wrote: > > > +1 (non-binding), glad to see we are now on the same page. Thank you all. > > > > Best regards, > > Jing > > > > On Wed, Jul 26, 2023 at 5:18 PM Yun Tang wrote: > > > > > +1 (non-binding), thanks @xintong for driving this work. > > > > > > > > > Best > > > Yun Tang > > > ________________ > > > From: Zhu Zhu > > > Sent: Wednesday, July 26, 2023 16:35 > > > To: dev@flink.apache.org > > > Subject: Re: [VOTE] Release 2.0 must-have work items - Round 2 > > > > > > +1 (binding) > > > > > > Thanks, > > > Zhu > > > > > > Leonard Xu 于2023年7月26日周三 15:40写道: > > > > > > > > Thanks @xingtong for driving the work. > > > > > > > > +1(binding) > > > > > > > > Best, > > > > Leonard > > > > > > > > > On Jul 26, 2023, at 3:18 PM, Konstantin Knauf < > > > knauf.konstan...@gmail.com> wrote: > > > > > > > > > > Hi Xingtong, > > > > > > > > > > yes, I am fine with the conclusion for SourceFunction. I chatted > with > > > > > Leonard a bit last night. Let's continue this vote. > > > > > > > > > > Thanks for the clarification, > > > > > > > > > > Konstantin > > > > > > > > > > > > > > > > > > > > Am Mi., 26. Juli 2023 um 04:03 Uhr schrieb Xintong Song < > > > > > tonysong...@gmail.com>: > > > > > > > > > >> Hi Konstantin, > > > > >> > > > > >> It seems the offline discussion has already taken place [1], and > > part > > > of > > > > >> the outcome is that removal of SourceFunction would be a > > > *nice-to-have* > > > > >> item for release 2.0 which may not block this *must-have* vote. Do > > > you have > > > > >> different opinions about the conclusions in [1]? > > > > >> > > > > >> If there are still concerns, and the discussion around this topic > > > needs to > > > > >> be continued, then I'd suggest (as I mentioned in [2]) not to > > further > > > block > > > > >> this vote (i.e. the decision on other must-have items). Release > 2.0 > > > still > > > > >> has a long way to go, and it is likely we need to review and > update > > > the > > > > >> list every once in a while. We can update the list with another > vote > > > if > > > > >> later we decide to add the removal of SourceFunction to the > > must-have > > > list. > > > > >> > > > > >> WDYT? > > > > >> > > > > >> Best, > > > > >> > > > > >> Xintong > > > > >> > > > > >> > > > > >> [1] > > https://lists.apache.org/thread/yyw52k45x2sp1jszldtdx7hc98n72w7k > > > > >> [2] > > https://lists.apache.org/thread/j5d5022ky8k5t088ffm03727o5g9x9jr > > > > >> > > > > >> On Tue, Jul 25, 2023 at 8:49 PM Konstantin Knauf < > kna...@apache.org > > > > > > > >> wrote: > > > > >> > > > > >>> I assume this vote includes a decision to not removing > > > > >>> SourceFunction/SinkFunction in Flink 2.0 (as it has been removed > > > from the > > > > >>> table). If this is the case, I don't think, this discussion has > > > > >> concluded. > > > > >>> There are multiple contributors like myself, Martijn, Alex > Fedulov > > > and > > > > >>> Maximilian Michels, who have indicated they would be in favor of > > > > >>> deprecating/dropping them. This Source/Sink Function discussion > > > seems to > > > > >> go > > > > >>> in circles in
Re: [VOTE] Release 2.0 must-have work items
Hi, I split the blockers [1] from the nice-to-haves [2] and added some missing items. >From [1], the one about support for the ExternallyInducedSource [3] is debatable - AFAIK, it is only used by Pravega, which is not an officially-supported connector. This can, arguably, be something we could hope for them to contribute to the 2.x release line. [1] https://issues.apache.org/jira/browse/FLINK-28045 [2] https://issues.apache.org/jira/browse/FLINK-32692 [3] https://issues.apache.org/jira/browse/FLINK-28051 On Wed, 26 Jul 2023 at 17:59, Jing Ge wrote: > Hi > > I agree with Konstantin that we should have a todo list to provide a clear > picture of when and how to deprecate the SinkFunction. Please let me check > if I can prepare a dedicated thread for it, since I got some feedback and > hints from previous discussions. > > Best regards, > Jing > > > On Wed, Jul 26, 2023 at 3:26 PM Konstantin Knauf > wrote: > > > Hi everyone, > > > > I'd just like to add that we also said, that we would continue the > > discussion to come up and agree on a list of concrete blockers for the > > removal of SourceFunction, so that don't need to have the same discussion > > again in half a year. And while we are add it, we should do the same > thing > > for SinkFunction. > > > > Best, > > > > Konstantin > > > > Am Mi., 26. Juli 2023 um 03:35 Uhr schrieb Xintong Song < > > tonysong...@gmail.com>: > > > > > Thanks Leonard for driving this, and thanks everyone for the > discussion. > > > The back-and-force reflects the importance and complexity around this > > > topic. Glad to see we finally reached consensus. > > > > > > Best, > > > > > > Xintong > > > > > > > > > > > > On Wed, Jul 26, 2023 at 12:42 AM Jing Ge wrote: > > > > > > > Thanks Leonard for driving it. We are now on the same page. > > > > > > > > Best regards, > > > > Jing > > > > > > > > On Tue, Jul 25, 2023 at 9:19 PM Leonard Xu > wrote: > > > > > > > >> We’ve detailed offline discussions with @Alexander and @Jingsong, > > about > > > >> “Remove SourceFunction” item, we’ve reached a consensus as > following: > > > >> > > > >> 1. Deprecate SourceFunction in 1.18 and implement following > > improvement > > > >> subtasks of FLINK-28045[1] later is reasonable for all of us. > > > >> > > > >> 2. Deleting SourceFunction API depends on future’s work progress, > thus > > > >> “Remove SourceFunction APIs” should be a nice to have item. > Alexander > > > has > > > >> volunteered to take these subtasks and would try to finish them > next, > > > >> thanks again. > > > >> > > > >> 3. As a nice to have item, and its READY status depends on future’s > > > work > > > >> progress, this won't block release 2.0 must-have item vote. > > > >> > > > >> Thanks again @Alexander, @Jingsong and @Xintong for driving these > > > things > > > >> forward. > > > >> > > > >> Also CC RMs for 1.18 @QingSheng @Jing @Martijn @Konstantin, I’ve > > > >> communicated with Alexander and would like to help review the > > > deprecation > > > >> PR again. > > > >> > > > >> Best, > > > >> Leonard > > > >> > > > >> [1] https://issues.apache.org/jira/browse/FLINK-28045 > > > >> > > > >> > > > >> On Jul 21, 2023, at 6:09 PM, Chesnay Schepler > > > wrote: > > > >> > > > >> On 21/07/2023 11:45, Leonard Xu wrote: > > > >> > > > >> In this way, the user will see the deprecated API firstly but they > can > > > >> not find a candidate if we can not finish all tasks in one minor > > > version . > > > >> > > > >> > > > >> i'm not convinced that this matters. There will be a whole bunch of > > APIs > > > >> deprecated in 1.18 (that will remain in 1.x!) without a replacement > so > > > we > > > >> can remove them in 2.0. > > > >> We already accepted this scenario. > > > >> > > > >> > > > >> > > > > > > > > > -- > > https://twitter.com/snntrable > > https://github.com/knaufk > > >
Re: [VOTE] Release 2.0 must-have work items
Hi I agree with Konstantin that we should have a todo list to provide a clear picture of when and how to deprecate the SinkFunction. Please let me check if I can prepare a dedicated thread for it, since I got some feedback and hints from previous discussions. Best regards, Jing On Wed, Jul 26, 2023 at 3:26 PM Konstantin Knauf wrote: > Hi everyone, > > I'd just like to add that we also said, that we would continue the > discussion to come up and agree on a list of concrete blockers for the > removal of SourceFunction, so that don't need to have the same discussion > again in half a year. And while we are add it, we should do the same thing > for SinkFunction. > > Best, > > Konstantin > > Am Mi., 26. Juli 2023 um 03:35 Uhr schrieb Xintong Song < > tonysong...@gmail.com>: > > > Thanks Leonard for driving this, and thanks everyone for the discussion. > > The back-and-force reflects the importance and complexity around this > > topic. Glad to see we finally reached consensus. > > > > Best, > > > > Xintong > > > > > > > > On Wed, Jul 26, 2023 at 12:42 AM Jing Ge wrote: > > > > > Thanks Leonard for driving it. We are now on the same page. > > > > > > Best regards, > > > Jing > > > > > > On Tue, Jul 25, 2023 at 9:19 PM Leonard Xu wrote: > > > > > >> We’ve detailed offline discussions with @Alexander and @Jingsong, > about > > >> “Remove SourceFunction” item, we’ve reached a consensus as following: > > >> > > >> 1. Deprecate SourceFunction in 1.18 and implement following > improvement > > >> subtasks of FLINK-28045[1] later is reasonable for all of us. > > >> > > >> 2. Deleting SourceFunction API depends on future’s work progress, thus > > >> “Remove SourceFunction APIs” should be a nice to have item. Alexander > > has > > >> volunteered to take these subtasks and would try to finish them next, > > >> thanks again. > > >> > > >> 3. As a nice to have item, and its READY status depends on future’s > > work > > >> progress, this won't block release 2.0 must-have item vote. > > >> > > >> Thanks again @Alexander, @Jingsong and @Xintong for driving these > > things > > >> forward. > > >> > > >> Also CC RMs for 1.18 @QingSheng @Jing @Martijn @Konstantin, I’ve > > >> communicated with Alexander and would like to help review the > > deprecation > > >> PR again. > > >> > > >> Best, > > >> Leonard > > >> > > >> [1] https://issues.apache.org/jira/browse/FLINK-28045 > > >> > > >> > > >> On Jul 21, 2023, at 6:09 PM, Chesnay Schepler > > wrote: > > >> > > >> On 21/07/2023 11:45, Leonard Xu wrote: > > >> > > >> In this way, the user will see the deprecated API firstly but they can > > >> not find a candidate if we can not finish all tasks in one minor > > version . > > >> > > >> > > >> i'm not convinced that this matters. There will be a whole bunch of > APIs > > >> deprecated in 1.18 (that will remain in 1.x!) without a replacement so > > we > > >> can remove them in 2.0. > > >> We already accepted this scenario. > > >> > > >> > > >> > > > > > -- > https://twitter.com/snntrable > https://github.com/knaufk >
Re: [VOTE] Release 2.0 must-have work items - Round 2
+1 (non-binding), assuming SourceFunction gets added back to the doc as a "nice-to-have". I am glad we've reached a consensus here. Extra thanks to Leonard for coordinating this discussion in particular. Best, Alex On Wed, 26 Jul 2023 at 15:43, Jing Ge wrote: > +1 (non-binding), glad to see we are now on the same page. Thank you all. > > Best regards, > Jing > > On Wed, Jul 26, 2023 at 5:18 PM Yun Tang wrote: > > > +1 (non-binding), thanks @xintong for driving this work. > > > > > > Best > > Yun Tang > > > > From: Zhu Zhu > > Sent: Wednesday, July 26, 2023 16:35 > > To: dev@flink.apache.org > > Subject: Re: [VOTE] Release 2.0 must-have work items - Round 2 > > > > +1 (binding) > > > > Thanks, > > Zhu > > > > Leonard Xu 于2023年7月26日周三 15:40写道: > > > > > > Thanks @xingtong for driving the work. > > > > > > +1(binding) > > > > > > Best, > > > Leonard > > > > > > > On Jul 26, 2023, at 3:18 PM, Konstantin Knauf < > > knauf.konstan...@gmail.com> wrote: > > > > > > > > Hi Xingtong, > > > > > > > > yes, I am fine with the conclusion for SourceFunction. I chatted with > > > > Leonard a bit last night. Let's continue this vote. > > > > > > > > Thanks for the clarification, > > > > > > > > Konstantin > > > > > > > > > > > > > > > > Am Mi., 26. Juli 2023 um 04:03 Uhr schrieb Xintong Song < > > > > tonysong...@gmail.com>: > > > > > > > >> Hi Konstantin, > > > >> > > > >> It seems the offline discussion has already taken place [1], and > part > > of > > > >> the outcome is that removal of SourceFunction would be a > > *nice-to-have* > > > >> item for release 2.0 which may not block this *must-have* vote. Do > > you have > > > >> different opinions about the conclusions in [1]? > > > >> > > > >> If there are still concerns, and the discussion around this topic > > needs to > > > >> be continued, then I'd suggest (as I mentioned in [2]) not to > further > > block > > > >> this vote (i.e. the decision on other must-have items). Release 2.0 > > still > > > >> has a long way to go, and it is likely we need to review and update > > the > > > >> list every once in a while. We can update the list with another vote > > if > > > >> later we decide to add the removal of SourceFunction to the > must-have > > list. > > > >> > > > >> WDYT? > > > >> > > > >> Best, > > > >> > > > >> Xintong > > > >> > > > >> > > > >> [1] > https://lists.apache.org/thread/yyw52k45x2sp1jszldtdx7hc98n72w7k > > > >> [2] > https://lists.apache.org/thread/j5d5022ky8k5t088ffm03727o5g9x9jr > > > >> > > > >> On Tue, Jul 25, 2023 at 8:49 PM Konstantin Knauf > > > > >> wrote: > > > >> > > > >>> I assume this vote includes a decision to not removing > > > >>> SourceFunction/SinkFunction in Flink 2.0 (as it has been removed > > from the > > > >>> table). If this is the case, I don't think, this discussion has > > > >> concluded. > > > >>> There are multiple contributors like myself, Martijn, Alex Fedulov > > and > > > >>> Maximilian Michels, who have indicated they would be in favor of > > > >>> deprecating/dropping them. This Source/Sink Function discussion > > seems to > > > >> go > > > >>> in circles in general. I am wondering if it makes sense to have a > > call > > > >>> about this instead of repeating mailing list discussions. > > > >>> > > > >>> Am Di., 25. Juli 2023 um 13:38 Uhr schrieb Yu Li >: > > > >>> > > > >>>> +1 (binding) > > > >>>> > > > >>>> Thanks for driving this, Xintong! > > > >>>> > > > >>>> Best Regards, > > > >>>> Yu > > > >>>> > > > >>>> > > > >>>> On Sun, 23 Jul 2023 at 18:28, Yuan Mei > > wrote: > > > >>>> > > > >>>>> +1 (binding) >
Re: [VOTE] Release 2.0 must-have work items - Round 2
+1 (non-binding), glad to see we are now on the same page. Thank you all. Best regards, Jing On Wed, Jul 26, 2023 at 5:18 PM Yun Tang wrote: > +1 (non-binding), thanks @xintong for driving this work. > > > Best > Yun Tang > > From: Zhu Zhu > Sent: Wednesday, July 26, 2023 16:35 > To: dev@flink.apache.org > Subject: Re: [VOTE] Release 2.0 must-have work items - Round 2 > > +1 (binding) > > Thanks, > Zhu > > Leonard Xu 于2023年7月26日周三 15:40写道: > > > > Thanks @xingtong for driving the work. > > > > +1(binding) > > > > Best, > > Leonard > > > > > On Jul 26, 2023, at 3:18 PM, Konstantin Knauf < > knauf.konstan...@gmail.com> wrote: > > > > > > Hi Xingtong, > > > > > > yes, I am fine with the conclusion for SourceFunction. I chatted with > > > Leonard a bit last night. Let's continue this vote. > > > > > > Thanks for the clarification, > > > > > > Konstantin > > > > > > > > > > > > Am Mi., 26. Juli 2023 um 04:03 Uhr schrieb Xintong Song < > > > tonysong...@gmail.com>: > > > > > >> Hi Konstantin, > > >> > > >> It seems the offline discussion has already taken place [1], and part > of > > >> the outcome is that removal of SourceFunction would be a > *nice-to-have* > > >> item for release 2.0 which may not block this *must-have* vote. Do > you have > > >> different opinions about the conclusions in [1]? > > >> > > >> If there are still concerns, and the discussion around this topic > needs to > > >> be continued, then I'd suggest (as I mentioned in [2]) not to further > block > > >> this vote (i.e. the decision on other must-have items). Release 2.0 > still > > >> has a long way to go, and it is likely we need to review and update > the > > >> list every once in a while. We can update the list with another vote > if > > >> later we decide to add the removal of SourceFunction to the must-have > list. > > >> > > >> WDYT? > > >> > > >> Best, > > >> > > >> Xintong > > >> > > >> > > >> [1] https://lists.apache.org/thread/yyw52k45x2sp1jszldtdx7hc98n72w7k > > >> [2] https://lists.apache.org/thread/j5d5022ky8k5t088ffm03727o5g9x9jr > > >> > > >> On Tue, Jul 25, 2023 at 8:49 PM Konstantin Knauf > > >> wrote: > > >> > > >>> I assume this vote includes a decision to not removing > > >>> SourceFunction/SinkFunction in Flink 2.0 (as it has been removed > from the > > >>> table). If this is the case, I don't think, this discussion has > > >> concluded. > > >>> There are multiple contributors like myself, Martijn, Alex Fedulov > and > > >>> Maximilian Michels, who have indicated they would be in favor of > > >>> deprecating/dropping them. This Source/Sink Function discussion > seems to > > >> go > > >>> in circles in general. I am wondering if it makes sense to have a > call > > >>> about this instead of repeating mailing list discussions. > > >>> > > >>> Am Di., 25. Juli 2023 um 13:38 Uhr schrieb Yu Li : > > >>> > > >>>> +1 (binding) > > >>>> > > >>>> Thanks for driving this, Xintong! > > >>>> > > >>>> Best Regards, > > >>>> Yu > > >>>> > > >>>> > > >>>> On Sun, 23 Jul 2023 at 18:28, Yuan Mei > wrote: > > >>>> > > >>>>> +1 (binding) > > >>>>> > > >>>>> Thanks for driving the discussion through and for all the efforts > in > > >>>>> resolving the complexities :-) > > >>>>> > > >>>>> Best > > >>>>> Yuan > > >>>>> > > >>>>> On Thu, Jul 20, 2023 at 5:23 PM Xintong Song < > tonysong...@gmail.com> > > >>>>> wrote: > > >>>>> > > >>>>>> Hi all, > > >>>>>> > > >>>>>> I'd like to start another round of VOTE for the must-have work > > >> items > > >>>> for > > >>>>>> release 2.0 [1]. The corresponding discussion thread is [2], and > > >> the > > >>>>>> previous voting thread is [3]. All comments from the previous > > >> voting > > >>>>> thread > > >>>>>> have been addressed. > > >>>>>> > > >>>>>> Please note that once the vote is approved, any changes to the > > >>>> must-have > > >>>>>> items (adding / removing must-have items, changing the priority) > > >>>> requires > > >>>>>> another vote. Assigning contributors / reviewers, updating > > >>>> descriptions / > > >>>>>> progress, changes to nice-to-have items do not require another > > >> vote. > > >>>>>> > > >>>>>> The vote will be open until at least July 25, following the > > >> consensus > > >>>>>> voting process. Votes of PMC members are binding. > > >>>>>> > > >>>>>> Best, > > >>>>>> > > >>>>>> Xintong > > >>>>>> > > >>>>>> > > >>>>>> [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > >>>>>> > > >>>>>> [2] > > >> https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 > > >>>>>> > > >>>>>> [3] > > >> https://lists.apache.org/thread/r0y9syc6k5nmcxvnd0hj33htdpdj9k6m > > >>>>>> > > >>>>> > > >>>> > > >>> > > >>> > > >>> -- > > >>> https://twitter.com/snntrable > > >>> https://github.com/knaufk > > >>> > > >> > > > > > > > > > -- > > > *Konstantin Knauf* > > > knauf.konstan...@gmail.com > > >
Re: [VOTE] Release 2.0 must-have work items - Round 2
+1 (non-binding), thanks @xintong for driving this work. Best Yun Tang From: Zhu Zhu Sent: Wednesday, July 26, 2023 16:35 To: dev@flink.apache.org Subject: Re: [VOTE] Release 2.0 must-have work items - Round 2 +1 (binding) Thanks, Zhu Leonard Xu 于2023年7月26日周三 15:40写道: > > Thanks @xingtong for driving the work. > > +1(binding) > > Best, > Leonard > > > On Jul 26, 2023, at 3:18 PM, Konstantin Knauf > > wrote: > > > > Hi Xingtong, > > > > yes, I am fine with the conclusion for SourceFunction. I chatted with > > Leonard a bit last night. Let's continue this vote. > > > > Thanks for the clarification, > > > > Konstantin > > > > > > > > Am Mi., 26. Juli 2023 um 04:03 Uhr schrieb Xintong Song < > > tonysong...@gmail.com>: > > > >> Hi Konstantin, > >> > >> It seems the offline discussion has already taken place [1], and part of > >> the outcome is that removal of SourceFunction would be a *nice-to-have* > >> item for release 2.0 which may not block this *must-have* vote. Do you have > >> different opinions about the conclusions in [1]? > >> > >> If there are still concerns, and the discussion around this topic needs to > >> be continued, then I'd suggest (as I mentioned in [2]) not to further block > >> this vote (i.e. the decision on other must-have items). Release 2.0 still > >> has a long way to go, and it is likely we need to review and update the > >> list every once in a while. We can update the list with another vote if > >> later we decide to add the removal of SourceFunction to the must-have list. > >> > >> WDYT? > >> > >> Best, > >> > >> Xintong > >> > >> > >> [1] https://lists.apache.org/thread/yyw52k45x2sp1jszldtdx7hc98n72w7k > >> [2] https://lists.apache.org/thread/j5d5022ky8k5t088ffm03727o5g9x9jr > >> > >> On Tue, Jul 25, 2023 at 8:49 PM Konstantin Knauf > >> wrote: > >> > >>> I assume this vote includes a decision to not removing > >>> SourceFunction/SinkFunction in Flink 2.0 (as it has been removed from the > >>> table). If this is the case, I don't think, this discussion has > >> concluded. > >>> There are multiple contributors like myself, Martijn, Alex Fedulov and > >>> Maximilian Michels, who have indicated they would be in favor of > >>> deprecating/dropping them. This Source/Sink Function discussion seems to > >> go > >>> in circles in general. I am wondering if it makes sense to have a call > >>> about this instead of repeating mailing list discussions. > >>> > >>> Am Di., 25. Juli 2023 um 13:38 Uhr schrieb Yu Li : > >>> > >>>> +1 (binding) > >>>> > >>>> Thanks for driving this, Xintong! > >>>> > >>>> Best Regards, > >>>> Yu > >>>> > >>>> > >>>> On Sun, 23 Jul 2023 at 18:28, Yuan Mei wrote: > >>>> > >>>>> +1 (binding) > >>>>> > >>>>> Thanks for driving the discussion through and for all the efforts in > >>>>> resolving the complexities :-) > >>>>> > >>>>> Best > >>>>> Yuan > >>>>> > >>>>> On Thu, Jul 20, 2023 at 5:23 PM Xintong Song > >>>>> wrote: > >>>>> > >>>>>> Hi all, > >>>>>> > >>>>>> I'd like to start another round of VOTE for the must-have work > >> items > >>>> for > >>>>>> release 2.0 [1]. The corresponding discussion thread is [2], and > >> the > >>>>>> previous voting thread is [3]. All comments from the previous > >> voting > >>>>> thread > >>>>>> have been addressed. > >>>>>> > >>>>>> Please note that once the vote is approved, any changes to the > >>>> must-have > >>>>>> items (adding / removing must-have items, changing the priority) > >>>> requires > >>>>>> another vote. Assigning contributors / reviewers, updating > >>>> descriptions / > >>>>>> progress, changes to nice-to-have items do not require another > >> vote. > >>>>>> > >>>>>> The vote will be open until at least July 25, following the > >> consensus > >>>>>> voting process. Votes of PMC members are binding. > >>>>>> > >>>>>> Best, > >>>>>> > >>>>>> Xintong > >>>>>> > >>>>>> > >>>>>> [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > >>>>>> > >>>>>> [2] > >> https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 > >>>>>> > >>>>>> [3] > >> https://lists.apache.org/thread/r0y9syc6k5nmcxvnd0hj33htdpdj9k6m > >>>>>> > >>>>> > >>>> > >>> > >>> > >>> -- > >>> https://twitter.com/snntrable > >>> https://github.com/knaufk > >>> > >> > > > > > > -- > > *Konstantin Knauf* > > knauf.konstan...@gmail.com >
Re: [VOTE] Release 2.0 must-have work items - Round 2
+1 (binding) Thanks, Zhu Leonard Xu 于2023年7月26日周三 15:40写道: > > Thanks @xingtong for driving the work. > > +1(binding) > > Best, > Leonard > > > On Jul 26, 2023, at 3:18 PM, Konstantin Knauf > > wrote: > > > > Hi Xingtong, > > > > yes, I am fine with the conclusion for SourceFunction. I chatted with > > Leonard a bit last night. Let's continue this vote. > > > > Thanks for the clarification, > > > > Konstantin > > > > > > > > Am Mi., 26. Juli 2023 um 04:03 Uhr schrieb Xintong Song < > > tonysong...@gmail.com>: > > > >> Hi Konstantin, > >> > >> It seems the offline discussion has already taken place [1], and part of > >> the outcome is that removal of SourceFunction would be a *nice-to-have* > >> item for release 2.0 which may not block this *must-have* vote. Do you have > >> different opinions about the conclusions in [1]? > >> > >> If there are still concerns, and the discussion around this topic needs to > >> be continued, then I'd suggest (as I mentioned in [2]) not to further block > >> this vote (i.e. the decision on other must-have items). Release 2.0 still > >> has a long way to go, and it is likely we need to review and update the > >> list every once in a while. We can update the list with another vote if > >> later we decide to add the removal of SourceFunction to the must-have list. > >> > >> WDYT? > >> > >> Best, > >> > >> Xintong > >> > >> > >> [1] https://lists.apache.org/thread/yyw52k45x2sp1jszldtdx7hc98n72w7k > >> [2] https://lists.apache.org/thread/j5d5022ky8k5t088ffm03727o5g9x9jr > >> > >> On Tue, Jul 25, 2023 at 8:49 PM Konstantin Knauf > >> wrote: > >> > >>> I assume this vote includes a decision to not removing > >>> SourceFunction/SinkFunction in Flink 2.0 (as it has been removed from the > >>> table). If this is the case, I don't think, this discussion has > >> concluded. > >>> There are multiple contributors like myself, Martijn, Alex Fedulov and > >>> Maximilian Michels, who have indicated they would be in favor of > >>> deprecating/dropping them. This Source/Sink Function discussion seems to > >> go > >>> in circles in general. I am wondering if it makes sense to have a call > >>> about this instead of repeating mailing list discussions. > >>> > >>> Am Di., 25. Juli 2023 um 13:38 Uhr schrieb Yu Li : > >>> > +1 (binding) > > Thanks for driving this, Xintong! > > Best Regards, > Yu > > > On Sun, 23 Jul 2023 at 18:28, Yuan Mei wrote: > > > +1 (binding) > > > > Thanks for driving the discussion through and for all the efforts in > > resolving the complexities :-) > > > > Best > > Yuan > > > > On Thu, Jul 20, 2023 at 5:23 PM Xintong Song > > wrote: > > > >> Hi all, > >> > >> I'd like to start another round of VOTE for the must-have work > >> items > for > >> release 2.0 [1]. The corresponding discussion thread is [2], and > >> the > >> previous voting thread is [3]. All comments from the previous > >> voting > > thread > >> have been addressed. > >> > >> Please note that once the vote is approved, any changes to the > must-have > >> items (adding / removing must-have items, changing the priority) > requires > >> another vote. Assigning contributors / reviewers, updating > descriptions / > >> progress, changes to nice-to-have items do not require another > >> vote. > >> > >> The vote will be open until at least July 25, following the > >> consensus > >> voting process. Votes of PMC members are binding. > >> > >> Best, > >> > >> Xintong > >> > >> > >> [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > >> > >> [2] > >> https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 > >> > >> [3] > >> https://lists.apache.org/thread/r0y9syc6k5nmcxvnd0hj33htdpdj9k6m > >> > > > > >>> > >>> > >>> -- > >>> https://twitter.com/snntrable > >>> https://github.com/knaufk > >>> > >> > > > > > > -- > > *Konstantin Knauf* > > knauf.konstan...@gmail.com >
Re: [VOTE] Release 2.0 must-have work items - Round 2
Thanks @xingtong for driving the work. +1(binding) Best, Leonard > On Jul 26, 2023, at 3:18 PM, Konstantin Knauf > wrote: > > Hi Xingtong, > > yes, I am fine with the conclusion for SourceFunction. I chatted with > Leonard a bit last night. Let's continue this vote. > > Thanks for the clarification, > > Konstantin > > > > Am Mi., 26. Juli 2023 um 04:03 Uhr schrieb Xintong Song < > tonysong...@gmail.com>: > >> Hi Konstantin, >> >> It seems the offline discussion has already taken place [1], and part of >> the outcome is that removal of SourceFunction would be a *nice-to-have* >> item for release 2.0 which may not block this *must-have* vote. Do you have >> different opinions about the conclusions in [1]? >> >> If there are still concerns, and the discussion around this topic needs to >> be continued, then I'd suggest (as I mentioned in [2]) not to further block >> this vote (i.e. the decision on other must-have items). Release 2.0 still >> has a long way to go, and it is likely we need to review and update the >> list every once in a while. We can update the list with another vote if >> later we decide to add the removal of SourceFunction to the must-have list. >> >> WDYT? >> >> Best, >> >> Xintong >> >> >> [1] https://lists.apache.org/thread/yyw52k45x2sp1jszldtdx7hc98n72w7k >> [2] https://lists.apache.org/thread/j5d5022ky8k5t088ffm03727o5g9x9jr >> >> On Tue, Jul 25, 2023 at 8:49 PM Konstantin Knauf >> wrote: >> >>> I assume this vote includes a decision to not removing >>> SourceFunction/SinkFunction in Flink 2.0 (as it has been removed from the >>> table). If this is the case, I don't think, this discussion has >> concluded. >>> There are multiple contributors like myself, Martijn, Alex Fedulov and >>> Maximilian Michels, who have indicated they would be in favor of >>> deprecating/dropping them. This Source/Sink Function discussion seems to >> go >>> in circles in general. I am wondering if it makes sense to have a call >>> about this instead of repeating mailing list discussions. >>> >>> Am Di., 25. Juli 2023 um 13:38 Uhr schrieb Yu Li : >>> +1 (binding) Thanks for driving this, Xintong! Best Regards, Yu On Sun, 23 Jul 2023 at 18:28, Yuan Mei wrote: > +1 (binding) > > Thanks for driving the discussion through and for all the efforts in > resolving the complexities :-) > > Best > Yuan > > On Thu, Jul 20, 2023 at 5:23 PM Xintong Song > wrote: > >> Hi all, >> >> I'd like to start another round of VOTE for the must-have work >> items for >> release 2.0 [1]. The corresponding discussion thread is [2], and >> the >> previous voting thread is [3]. All comments from the previous >> voting > thread >> have been addressed. >> >> Please note that once the vote is approved, any changes to the must-have >> items (adding / removing must-have items, changing the priority) requires >> another vote. Assigning contributors / reviewers, updating descriptions / >> progress, changes to nice-to-have items do not require another >> vote. >> >> The vote will be open until at least July 25, following the >> consensus >> voting process. Votes of PMC members are binding. >> >> Best, >> >> Xintong >> >> >> [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release >> >> [2] >> https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 >> >> [3] >> https://lists.apache.org/thread/r0y9syc6k5nmcxvnd0hj33htdpdj9k6m >> > >>> >>> >>> -- >>> https://twitter.com/snntrable >>> https://github.com/knaufk >>> >> > > > -- > *Konstantin Knauf* > knauf.konstan...@gmail.com
Re: [VOTE] Release 2.0 must-have work items
Hi everyone, I'd just like to add that we also said, that we would continue the discussion to come up and agree on a list of concrete blockers for the removal of SourceFunction, so that don't need to have the same discussion again in half a year. And while we are add it, we should do the same thing for SinkFunction. Best, Konstantin Am Mi., 26. Juli 2023 um 03:35 Uhr schrieb Xintong Song < tonysong...@gmail.com>: > Thanks Leonard for driving this, and thanks everyone for the discussion. > The back-and-force reflects the importance and complexity around this > topic. Glad to see we finally reached consensus. > > Best, > > Xintong > > > > On Wed, Jul 26, 2023 at 12:42 AM Jing Ge wrote: > > > Thanks Leonard for driving it. We are now on the same page. > > > > Best regards, > > Jing > > > > On Tue, Jul 25, 2023 at 9:19 PM Leonard Xu wrote: > > > >> We’ve detailed offline discussions with @Alexander and @Jingsong, about > >> “Remove SourceFunction” item, we’ve reached a consensus as following: > >> > >> 1. Deprecate SourceFunction in 1.18 and implement following improvement > >> subtasks of FLINK-28045[1] later is reasonable for all of us. > >> > >> 2. Deleting SourceFunction API depends on future’s work progress, thus > >> “Remove SourceFunction APIs” should be a nice to have item. Alexander > has > >> volunteered to take these subtasks and would try to finish them next, > >> thanks again. > >> > >> 3. As a nice to have item, and its READY status depends on future’s > work > >> progress, this won't block release 2.0 must-have item vote. > >> > >> Thanks again @Alexander, @Jingsong and @Xintong for driving these > things > >> forward. > >> > >> Also CC RMs for 1.18 @QingSheng @Jing @Martijn @Konstantin, I’ve > >> communicated with Alexander and would like to help review the > deprecation > >> PR again. > >> > >> Best, > >> Leonard > >> > >> [1] https://issues.apache.org/jira/browse/FLINK-28045 > >> > >> > >> On Jul 21, 2023, at 6:09 PM, Chesnay Schepler > wrote: > >> > >> On 21/07/2023 11:45, Leonard Xu wrote: > >> > >> In this way, the user will see the deprecated API firstly but they can > >> not find a candidate if we can not finish all tasks in one minor > version . > >> > >> > >> i'm not convinced that this matters. There will be a whole bunch of APIs > >> deprecated in 1.18 (that will remain in 1.x!) without a replacement so > we > >> can remove them in 2.0. > >> We already accepted this scenario. > >> > >> > >> > -- https://twitter.com/snntrable https://github.com/knaufk
Re: [VOTE] Release 2.0 must-have work items - Round 2
Hi Xingtong, yes, I am fine with the conclusion for SourceFunction. I chatted with Leonard a bit last night. Let's continue this vote. Thanks for the clarification, Konstantin Am Mi., 26. Juli 2023 um 04:03 Uhr schrieb Xintong Song < tonysong...@gmail.com>: > Hi Konstantin, > > It seems the offline discussion has already taken place [1], and part of > the outcome is that removal of SourceFunction would be a *nice-to-have* > item for release 2.0 which may not block this *must-have* vote. Do you have > different opinions about the conclusions in [1]? > > If there are still concerns, and the discussion around this topic needs to > be continued, then I'd suggest (as I mentioned in [2]) not to further block > this vote (i.e. the decision on other must-have items). Release 2.0 still > has a long way to go, and it is likely we need to review and update the > list every once in a while. We can update the list with another vote if > later we decide to add the removal of SourceFunction to the must-have list. > > WDYT? > > Best, > > Xintong > > > [1] https://lists.apache.org/thread/yyw52k45x2sp1jszldtdx7hc98n72w7k > [2] https://lists.apache.org/thread/j5d5022ky8k5t088ffm03727o5g9x9jr > > On Tue, Jul 25, 2023 at 8:49 PM Konstantin Knauf > wrote: > > > I assume this vote includes a decision to not removing > > SourceFunction/SinkFunction in Flink 2.0 (as it has been removed from the > > table). If this is the case, I don't think, this discussion has > concluded. > > There are multiple contributors like myself, Martijn, Alex Fedulov and > > Maximilian Michels, who have indicated they would be in favor of > > deprecating/dropping them. This Source/Sink Function discussion seems to > go > > in circles in general. I am wondering if it makes sense to have a call > > about this instead of repeating mailing list discussions. > > > > Am Di., 25. Juli 2023 um 13:38 Uhr schrieb Yu Li : > > > > > +1 (binding) > > > > > > Thanks for driving this, Xintong! > > > > > > Best Regards, > > > Yu > > > > > > > > > On Sun, 23 Jul 2023 at 18:28, Yuan Mei wrote: > > > > > > > +1 (binding) > > > > > > > > Thanks for driving the discussion through and for all the efforts in > > > > resolving the complexities :-) > > > > > > > > Best > > > > Yuan > > > > > > > > On Thu, Jul 20, 2023 at 5:23 PM Xintong Song > > > > wrote: > > > > > > > > > Hi all, > > > > > > > > > > I'd like to start another round of VOTE for the must-have work > items > > > for > > > > > release 2.0 [1]. The corresponding discussion thread is [2], and > the > > > > > previous voting thread is [3]. All comments from the previous > voting > > > > thread > > > > > have been addressed. > > > > > > > > > > Please note that once the vote is approved, any changes to the > > > must-have > > > > > items (adding / removing must-have items, changing the priority) > > > requires > > > > > another vote. Assigning contributors / reviewers, updating > > > descriptions / > > > > > progress, changes to nice-to-have items do not require another > vote. > > > > > > > > > > The vote will be open until at least July 25, following the > consensus > > > > > voting process. Votes of PMC members are binding. > > > > > > > > > > Best, > > > > > > > > > > Xintong > > > > > > > > > > > > > > > [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > > > > > > > > > [2] > https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 > > > > > > > > > > [3] > https://lists.apache.org/thread/r0y9syc6k5nmcxvnd0hj33htdpdj9k6m > > > > > > > > > > > > > > > > > > -- > > https://twitter.com/snntrable > > https://github.com/knaufk > > > -- *Konstantin Knauf* knauf.konstan...@gmail.com
Re: [VOTE] Release 2.0 must-have work items - Round 2
Hi Konstantin, It seems the offline discussion has already taken place [1], and part of the outcome is that removal of SourceFunction would be a *nice-to-have* item for release 2.0 which may not block this *must-have* vote. Do you have different opinions about the conclusions in [1]? If there are still concerns, and the discussion around this topic needs to be continued, then I'd suggest (as I mentioned in [2]) not to further block this vote (i.e. the decision on other must-have items). Release 2.0 still has a long way to go, and it is likely we need to review and update the list every once in a while. We can update the list with another vote if later we decide to add the removal of SourceFunction to the must-have list. WDYT? Best, Xintong [1] https://lists.apache.org/thread/yyw52k45x2sp1jszldtdx7hc98n72w7k [2] https://lists.apache.org/thread/j5d5022ky8k5t088ffm03727o5g9x9jr On Tue, Jul 25, 2023 at 8:49 PM Konstantin Knauf wrote: > I assume this vote includes a decision to not removing > SourceFunction/SinkFunction in Flink 2.0 (as it has been removed from the > table). If this is the case, I don't think, this discussion has concluded. > There are multiple contributors like myself, Martijn, Alex Fedulov and > Maximilian Michels, who have indicated they would be in favor of > deprecating/dropping them. This Source/Sink Function discussion seems to go > in circles in general. I am wondering if it makes sense to have a call > about this instead of repeating mailing list discussions. > > Am Di., 25. Juli 2023 um 13:38 Uhr schrieb Yu Li : > > > +1 (binding) > > > > Thanks for driving this, Xintong! > > > > Best Regards, > > Yu > > > > > > On Sun, 23 Jul 2023 at 18:28, Yuan Mei wrote: > > > > > +1 (binding) > > > > > > Thanks for driving the discussion through and for all the efforts in > > > resolving the complexities :-) > > > > > > Best > > > Yuan > > > > > > On Thu, Jul 20, 2023 at 5:23 PM Xintong Song > > > wrote: > > > > > > > Hi all, > > > > > > > > I'd like to start another round of VOTE for the must-have work items > > for > > > > release 2.0 [1]. The corresponding discussion thread is [2], and the > > > > previous voting thread is [3]. All comments from the previous voting > > > thread > > > > have been addressed. > > > > > > > > Please note that once the vote is approved, any changes to the > > must-have > > > > items (adding / removing must-have items, changing the priority) > > requires > > > > another vote. Assigning contributors / reviewers, updating > > descriptions / > > > > progress, changes to nice-to-have items do not require another vote. > > > > > > > > The vote will be open until at least July 25, following the consensus > > > > voting process. Votes of PMC members are binding. > > > > > > > > Best, > > > > > > > > Xintong > > > > > > > > > > > > [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > > > > > > > [2] https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 > > > > > > > > [3] https://lists.apache.org/thread/r0y9syc6k5nmcxvnd0hj33htdpdj9k6m > > > > > > > > > > > > -- > https://twitter.com/snntrable > https://github.com/knaufk >
Re: [VOTE] Release 2.0 must-have work items
Thanks Leonard for driving this, and thanks everyone for the discussion. The back-and-force reflects the importance and complexity around this topic. Glad to see we finally reached consensus. Best, Xintong On Wed, Jul 26, 2023 at 12:42 AM Jing Ge wrote: > Thanks Leonard for driving it. We are now on the same page. > > Best regards, > Jing > > On Tue, Jul 25, 2023 at 9:19 PM Leonard Xu wrote: > >> We’ve detailed offline discussions with @Alexander and @Jingsong, about >> “Remove SourceFunction” item, we’ve reached a consensus as following: >> >> 1. Deprecate SourceFunction in 1.18 and implement following improvement >> subtasks of FLINK-28045[1] later is reasonable for all of us. >> >> 2. Deleting SourceFunction API depends on future’s work progress, thus >> “Remove SourceFunction APIs” should be a nice to have item. Alexander has >> volunteered to take these subtasks and would try to finish them next, >> thanks again. >> >> 3. As a nice to have item, and its READY status depends on future’s work >> progress, this won't block release 2.0 must-have item vote. >> >> Thanks again @Alexander, @Jingsong and @Xintong for driving these things >> forward. >> >> Also CC RMs for 1.18 @QingSheng @Jing @Martijn @Konstantin, I’ve >> communicated with Alexander and would like to help review the deprecation >> PR again. >> >> Best, >> Leonard >> >> [1] https://issues.apache.org/jira/browse/FLINK-28045 >> >> >> On Jul 21, 2023, at 6:09 PM, Chesnay Schepler wrote: >> >> On 21/07/2023 11:45, Leonard Xu wrote: >> >> In this way, the user will see the deprecated API firstly but they can >> not find a candidate if we can not finish all tasks in one minor version . >> >> >> i'm not convinced that this matters. There will be a whole bunch of APIs >> deprecated in 1.18 (that will remain in 1.x!) without a replacement so we >> can remove them in 2.0. >> We already accepted this scenario. >> >> >>
Re: [VOTE] Release 2.0 must-have work items
Thanks Leonard for driving it. We are now on the same page. Best regards, Jing On Tue, Jul 25, 2023 at 9:19 PM Leonard Xu wrote: > We’ve detailed offline discussions with @Alexander and @Jingsong, about > “Remove SourceFunction” item, we’ve reached a consensus as following: > > 1. Deprecate SourceFunction in 1.18 and implement following improvement > subtasks of FLINK-28045[1] later is reasonable for all of us. > > 2. Deleting SourceFunction API depends on future’s work progress, thus > “Remove SourceFunction APIs” should be a nice to have item. Alexander has > volunteered to take these subtasks and would try to finish them next, > thanks again. > > 3. As a nice to have item, and its READY status depends on future’s work > progress, this won't block release 2.0 must-have item vote. > > Thanks again @Alexander, @Jingsong and @Xintong for driving these things > forward. > > Also CC RMs for 1.18 @QingSheng @Jing @Martijn @Konstantin, I’ve > communicated with Alexander and would like to help review the deprecation > PR again. > > Best, > Leonard > > [1] https://issues.apache.org/jira/browse/FLINK-28045 > > > On Jul 21, 2023, at 6:09 PM, Chesnay Schepler wrote: > > On 21/07/2023 11:45, Leonard Xu wrote: > > In this way, the user will see the deprecated API firstly but they can not > find a candidate if we can not finish all tasks in one minor version . > > > i'm not convinced that this matters. There will be a whole bunch of APIs > deprecated in 1.18 (that will remain in 1.x!) without a replacement so we > can remove them in 2.0. > We already accepted this scenario. > > >
Re: [VOTE] Release 2.0 must-have work items
We’ve detailed offline discussions with @Alexander and @Jingsong, about “Remove SourceFunction” item, we’ve reached a consensus as following: 1. Deprecate SourceFunction in 1.18 and implement following improvement subtasks of FLINK-28045[1] later is reasonable for all of us. 2. Deleting SourceFunction API depends on future’s work progress, thus “Remove SourceFunction APIs” should be a nice to have item. Alexander has volunteered to take these subtasks and would try to finish them next, thanks again. 3. As a nice to have item, and its READY status depends on future’s work progress, this won't block release 2.0 must-have item vote. Thanks again @Alexander, @Jingsong and @Xintong for driving these things forward. Also CC RMs for 1.18 @QingSheng @Jing @Martijn @Konstantin, I’ve communicated with Alexander and would like to help review the deprecation PR again. Best, Leonard [1] https://issues.apache.org/jira/browse/FLINK-28045 > On Jul 21, 2023, at 6:09 PM, Chesnay Schepler wrote: > > On 21/07/2023 11:45, Leonard Xu wrote: >> In this way, the user will see the deprecated API firstly but they can not >> find a candidate if we can not finish all tasks in one minor version . > > i'm not convinced that this matters. There will be a whole bunch of APIs > deprecated in 1.18 (that will remain in 1.x!) without a replacement so we can > remove them in 2.0. > We already accepted this scenario.
Re: [VOTE] Release 2.0 must-have work items - Round 2
I assume this vote includes a decision to not removing SourceFunction/SinkFunction in Flink 2.0 (as it has been removed from the table). If this is the case, I don't think, this discussion has concluded. There are multiple contributors like myself, Martijn, Alex Fedulov and Maximilian Michels, who have indicated they would be in favor of deprecating/dropping them. This Source/Sink Function discussion seems to go in circles in general. I am wondering if it makes sense to have a call about this instead of repeating mailing list discussions. Am Di., 25. Juli 2023 um 13:38 Uhr schrieb Yu Li : > +1 (binding) > > Thanks for driving this, Xintong! > > Best Regards, > Yu > > > On Sun, 23 Jul 2023 at 18:28, Yuan Mei wrote: > > > +1 (binding) > > > > Thanks for driving the discussion through and for all the efforts in > > resolving the complexities :-) > > > > Best > > Yuan > > > > On Thu, Jul 20, 2023 at 5:23 PM Xintong Song > > wrote: > > > > > Hi all, > > > > > > I'd like to start another round of VOTE for the must-have work items > for > > > release 2.0 [1]. The corresponding discussion thread is [2], and the > > > previous voting thread is [3]. All comments from the previous voting > > thread > > > have been addressed. > > > > > > Please note that once the vote is approved, any changes to the > must-have > > > items (adding / removing must-have items, changing the priority) > requires > > > another vote. Assigning contributors / reviewers, updating > descriptions / > > > progress, changes to nice-to-have items do not require another vote. > > > > > > The vote will be open until at least July 25, following the consensus > > > voting process. Votes of PMC members are binding. > > > > > > Best, > > > > > > Xintong > > > > > > > > > [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > > > > > [2] https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 > > > > > > [3] https://lists.apache.org/thread/r0y9syc6k5nmcxvnd0hj33htdpdj9k6m > > > > > > -- https://twitter.com/snntrable https://github.com/knaufk
Re: [VOTE] Release 2.0 must-have work items - Round 2
+1 (binding) Thanks for driving this, Xintong! Best Regards, Yu On Sun, 23 Jul 2023 at 18:28, Yuan Mei wrote: > +1 (binding) > > Thanks for driving the discussion through and for all the efforts in > resolving the complexities :-) > > Best > Yuan > > On Thu, Jul 20, 2023 at 5:23 PM Xintong Song > wrote: > > > Hi all, > > > > I'd like to start another round of VOTE for the must-have work items for > > release 2.0 [1]. The corresponding discussion thread is [2], and the > > previous voting thread is [3]. All comments from the previous voting > thread > > have been addressed. > > > > Please note that once the vote is approved, any changes to the must-have > > items (adding / removing must-have items, changing the priority) requires > > another vote. Assigning contributors / reviewers, updating descriptions / > > progress, changes to nice-to-have items do not require another vote. > > > > The vote will be open until at least July 25, following the consensus > > voting process. Votes of PMC members are binding. > > > > Best, > > > > Xintong > > > > > > [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > > > [2] https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 > > > > [3] https://lists.apache.org/thread/r0y9syc6k5nmcxvnd0hj33htdpdj9k6m > > >
Re: [VOTE] Release 2.0 must-have work items - Round 2
+1 (binding) Thanks for driving the discussion through and for all the efforts in resolving the complexities :-) Best Yuan On Thu, Jul 20, 2023 at 5:23 PM Xintong Song wrote: > Hi all, > > I'd like to start another round of VOTE for the must-have work items for > release 2.0 [1]. The corresponding discussion thread is [2], and the > previous voting thread is [3]. All comments from the previous voting thread > have been addressed. > > Please note that once the vote is approved, any changes to the must-have > items (adding / removing must-have items, changing the priority) requires > another vote. Assigning contributors / reviewers, updating descriptions / > progress, changes to nice-to-have items do not require another vote. > > The vote will be open until at least July 25, following the consensus > voting process. Votes of PMC members are binding. > > Best, > > Xintong > > > [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > [2] https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 > > [3] https://lists.apache.org/thread/r0y9syc6k5nmcxvnd0hj33htdpdj9k6m >
Re: [VOTE] Release 2.0 must-have work items
On 21/07/2023 11:45, Leonard Xu wrote: In this way, the user will see the deprecated API firstly but they can not find a candidate if we can not finish all tasks in one minor version . i'm not convinced that this matters. There will be a whole bunch of APIs deprecated in 1.18 (that will remain in 1.x!) without a replacement so we can remove them in 2.0. We already accepted this scenario.
Re: [VOTE] Release 2.0 must-have work items
> @Leonard > If we follow the agreed-upon and voted path and do not revert [1], all the > formalities get fulfilled Hi Alexander Sorry for making you uncomfortable even though it was discussed on the dev list and under JIRA before reverting this PR. I'm also +1 for the FLIP as your posted. The reason to revert the PR is not -1 for the voted FLIP, the reason we revert the PR is the incorrect subtask implementation order. We should not mark the API as deprecated firstly and then to implement the necessary subtasks. In this way, the user will see the deprecated API firstly but they can not find a candidate if we can not finish all tasks in one minor version . In this case, there're 7 subtasks to resolve and only 3 days left for 1.18 feature freeze, that means we have a high probability of releasing a deprecated API but not providing a migration path. It looks like that we voted a FLIP for a feature Flink supports XXX, we first add this feature to the document to tell users that Flink supports t XXX, but this feature may needs many subtasks to be completed in one or more versions. So, before we actually support XXX, it is incorrect for users to see this document that Flink supports XXX, we should add the document after we have finished all subtasks of the voted feature. Please correct me if I wrongly understand the order of sub-task. Best, Leonard > >> Hi Alexander >> >>> I see your concerns regarding the complexity of migration, but we still >>> have one year to address them. >> >> Not only the complexity of migration, but also we lack migration path for >> now. We have to deprecate SourceFunction/SinkFunction in 1.18 which feature >> freeze date is 2023/07/24 CEST If we want to remove >> SourceFunction/SinkFunction in 2.0, remove a public API requires at least >> two minor versions. There’re many subtasks to finish before we can mark >> them as deprecated[1], I think we have no time to finish these subtasks >> this week(hint: not this year). That’s why we suggested removing the >> SourceFunction/SinkFunction item from must to have. >> >> Best, >> Leonard >> [1]https://issues.apache.org/jira/browse/FLINK-28045 >> >>
Re: [VOTE] Release 2.0 must-have work items
> How about this, we continue with the vote as is, and keep the discussion on the SourceFunction in Jira or a separate thread. Sure, but I just want to mention two important things here before we switch over to [1]: >Given that eliminating the removal of SourceFunction was proposed 10 days - This is not the case - here is the original discussion thread from a year ago [2] - Here are the vote results [3] agreeing to do exactly the following: > This proposition implies marking the SourceFunction interface > itself as @Deprecated + redirecting to the FLIP-27 Source API > right away, without waiting for all the subtasks to be completed. This was all already discussed and agreed upon, the decision to revert the change comes a bit as a surprise. @Leonard If we follow the agreed-upon and voted path and do not revert [1], all the formalities get fulfilled. The rest of the discussion can happen in [1] [1] https://issues.apache.org/jira/browse/FLINK-28046 [2] https://lists.apache.org/thread/d6cwqw9b3105wcpdkwq7rr4s7x4ywqr9 [3] https://lists.apache.org/thread/hrpsddgz65hjvhjozhg72s0wsmxz145p Thanks, Alex On Fri, 21 Jul 2023 at 04:49, Leonard Xu wrote: > Hi Alexander > > > I see your concerns regarding the complexity of migration, but we still > > have one year to address them. > > Not only the complexity of migration, but also we lack migration path for > now. We have to deprecate SourceFunction/SinkFunction in 1.18 which feature > freeze date is 2023/07/24 CEST If we want to remove > SourceFunction/SinkFunction in 2.0, remove a public API requires at least > two minor versions. There’re many subtasks to finish before we can mark > them as deprecated[1], I think we have no time to finish these subtasks > this week(hint: not this year). That’s why we suggested removing the > SourceFunction/SinkFunction item from must to have. > > Best, > Leonard > [1]https://issues.apache.org/jira/browse/FLINK-28045 > >
Re: [VOTE] Release 2.0 must-have work items
Hi Alexander > I see your concerns regarding the complexity of migration, but we still > have one year to address them. Not only the complexity of migration, but also we lack migration path for now. We have to deprecate SourceFunction/SinkFunction in 1.18 which feature freeze date is 2023/07/24 CEST If we want to remove SourceFunction/SinkFunction in 2.0, remove a public API requires at least two minor versions. There’re many subtasks to finish before we can mark them as deprecated[1], I think we have no time to finish these subtasks this week(hint: not this year). That’s why we suggested removing the SourceFunction/SinkFunction item from must to have. Best, Leonard [1]https://issues.apache.org/jira/browse/FLINK-28045
Re: [VOTE] Release 2.0 must-have work items
> > >> Hey Yun and Xintong, > > >> > > >> (Had a quick offline discussion with Yun) > > >> 1. I agree the current implementation of the queryable state is not a > > >> blocker of anything related to disaggregated state management. They > are > > >> different things. > > >> 2. On the other hand, "queryable snapshot" is not a completely > > equivalent > > >> substitution of "queryable state". > > >> 3. But in whatever way, I think the way how "queryable state" is > > designed > > >> is not the right way to move forward. > > >> 4. "Deprecating queryable state" is put as a must-have because this > > topic > > >> has been raised many times along the way. It seems to reach an > agreement > > >> every time as mentioned by Xingtong, but no one really takes the > action. > > >> > > >> I am suggesting: > > >> > > >> 1. Remove "Deprecating queryable state" from the must-have list (since > > it > > >> does not meet the requirements of "must-have") > > >> 2. But I am still hoping we can move things forward, so let's put > > >> the @Deprecated annotation on it > > >> 3. Removal of the code follows a formal community discussion and vote. > > >> > > >> Best > > >> Yuan > > >> > > >> > > >> > > >> > > >> On Mon, Jul 17, 2023 at 3:40 PM Xintong Song > > >> wrote: > > >> > > >> > Thanks for the clarification. > > >> > > > >> > If the list of "Remove deprecated APIs" means, we must remove the > code > > >> in > > >> > > Flink-2.0 initial release, I would vote -1 for queryable state > > before > > >> we > > >> > > get an alternative. > > >> > > > >> > > > >> > FYI, the removal of queryable state is currently marked as the > > >> `must-have` > > >> > priority. Of course it's not a final decision and that's exactly > why > > we > > >> > are collecting feedback about the list now. > > >> > > > >> > Best, > > >> > > > >> > Xintong > > >> > > > >> > > > >> > > > >> > On Mon, Jul 17, 2023 at 3:15 PM Yun Tang wrote: > > >> > > > >> > > Hi Xintong, > > >> > > > > >> > > If the current implementation of queryable state would not block > the > > >> > > implementation of disaggregated state-backends. > > >> > > I prefer to not removing the implementation until we have a better > > >> > > solution (maybe based on the queryable snapshot) cc @Yuan. > > >> > > > > >> > > If the list of "Remove deprecated APIs" means, we must remove the > > >> code in > > >> > > Flink-2.0 initial release, I would vote -1 for queryable state > > before > > >> we > > >> > > get an alternative. > > >> > > And I will raise the concern in the Flink roadmap discussion. > > >> > > > > >> > > > > >> > > Best > > >> > > Yun Tang > > >> > > > > >> > > From: Xintong Song > > >> > > Sent: Monday, July 17, 2023 10:07 > > >> > > To: dev@flink.apache.org > > >> > > Subject: Re: [VOTE] Release 2.0 must-have work items > > >> > > > > >> > > @Yun, > > >> > > I see your point that the ability queryable states trying to > provide > > >> is > > >> > > meaningful but the current implementation of the feature is > > >> problematic. > > >> > So > > >> > > what's your opinion on deprecating the current queryable state? Do > > you > > >> > > think we need to wait until there is a new implementation of > > queryable > > >> > > state to remove the current one? Or maybe the current > implementation > > >> is > > >> > not > > >> > > well functional anyway and we can treat the removal of it as > > >> > > independent from introducing a new one? > > >> > > > > >> > > However, I don't want
Re: [VOTE] Release 2.0 must-have work items
> >> > >> > >> > >> > >> On Mon, Jul 17, 2023 at 3:40 PM Xintong Song > >> wrote: > >> > >> > Thanks for the clarification. > >> > > >> > If the list of "Remove deprecated APIs" means, we must remove the code > >> in > >> > > Flink-2.0 initial release, I would vote -1 for queryable state > before > >> we > >> > > get an alternative. > >> > > >> > > >> > FYI, the removal of queryable state is currently marked as the > >> `must-have` > >> > priority. Of course it's not a final decision and that's exactly why > we > >> > are collecting feedback about the list now. > >> > > >> > Best, > >> > > >> > Xintong > >> > > >> > > >> > > >> > On Mon, Jul 17, 2023 at 3:15 PM Yun Tang wrote: > >> > > >> > > Hi Xintong, > >> > > > >> > > If the current implementation of queryable state would not block the > >> > > implementation of disaggregated state-backends. > >> > > I prefer to not removing the implementation until we have a better > >> > > solution (maybe based on the queryable snapshot) cc @Yuan. > >> > > > >> > > If the list of "Remove deprecated APIs" means, we must remove the > >> code in > >> > > Flink-2.0 initial release, I would vote -1 for queryable state > before > >> we > >> > > get an alternative. > >> > > And I will raise the concern in the Flink roadmap discussion. > >> > > > >> > > > >> > > Best > >> > > Yun Tang > >> > > > >> > > From: Xintong Song > >> > > Sent: Monday, July 17, 2023 10:07 > >> > > To: dev@flink.apache.org > >> > > Subject: Re: [VOTE] Release 2.0 must-have work items > >> > > > >> > > @Yun, > >> > > I see your point that the ability queryable states trying to provide > >> is > >> > > meaningful but the current implementation of the feature is > >> problematic. > >> > So > >> > > what's your opinion on deprecating the current queryable state? Do > you > >> > > think we need to wait until there is a new implementation of > queryable > >> > > state to remove the current one? Or maybe the current implementation > >> is > >> > not > >> > > well functional anyway and we can treat the removal of it as > >> > > independent from introducing a new one? > >> > > > >> > > However, I don't want to make users feel that this feature cannot be > >> done > >> > > > well, and maybe we can redesign this feature. > >> > > > > >> > > TBH, the impression that I got from the roadmap[1] is that the > >> queryable > >> > > state is retiring and will be replaced by the state processor api. > If > >> > this > >> > > is not the impression we want users to have, you probably also need > to > >> > > raise it in the roadmap discussion [2]. > >> > > > >> > > Best, > >> > > > >> > > Xintong > >> > > > >> > > > >> > > [1] https://flink.apache.org/roadmap > >> > > > >> > > [2] > https://lists.apache.org/thread/szdr4ngrfcmo7zko4917393zbqhgw0v5 > >> > > > >> > > > >> > > > >> > > On Mon, Jul 17, 2023 at 9:53 AM Xintong Song > > >> > > wrote: > >> > > > >> > > > I'd propose to downgrade "Refactor the API modules" to TBD. The > >> > original > >> > > > proposal was based on the condition that we are allowed to > introduce > >> > > > in-place API breaking changes in release 2.0. As the migration > >> period > >> > is > >> > > > introduced, and we are no longer planning to do in-place changes / > >> > > > removal for DataStream (and same for APIs in `flink-core`), we > need > >> to > >> > > > re-evaluate whether it's feasible to do things like moving classes > >> to > >> > > > different module / packages, turning concrete classes into > >
Re: [VOTE] Release 2.0 must-have work items
t; >> > > Hi Xintong, >> > > >> > > If the current implementation of queryable state would not block the >> > > implementation of disaggregated state-backends. >> > > I prefer to not removing the implementation until we have a better >> > > solution (maybe based on the queryable snapshot) cc @Yuan. >> > > >> > > If the list of "Remove deprecated APIs" means, we must remove the >> code in >> > > Flink-2.0 initial release, I would vote -1 for queryable state before >> we >> > > get an alternative. >> > > And I will raise the concern in the Flink roadmap discussion. >> > > >> > > >> > > Best >> > > Yun Tang >> > > >> > > From: Xintong Song >> > > Sent: Monday, July 17, 2023 10:07 >> > > To: dev@flink.apache.org >> > > Subject: Re: [VOTE] Release 2.0 must-have work items >> > > >> > > @Yun, >> > > I see your point that the ability queryable states trying to provide >> is >> > > meaningful but the current implementation of the feature is >> problematic. >> > So >> > > what's your opinion on deprecating the current queryable state? Do you >> > > think we need to wait until there is a new implementation of queryable >> > > state to remove the current one? Or maybe the current implementation >> is >> > not >> > > well functional anyway and we can treat the removal of it as >> > > independent from introducing a new one? >> > > >> > > However, I don't want to make users feel that this feature cannot be >> done >> > > > well, and maybe we can redesign this feature. >> > > > >> > > TBH, the impression that I got from the roadmap[1] is that the >> queryable >> > > state is retiring and will be replaced by the state processor api. If >> > this >> > > is not the impression we want users to have, you probably also need to >> > > raise it in the roadmap discussion [2]. >> > > >> > > Best, >> > > >> > > Xintong >> > > >> > > >> > > [1] https://flink.apache.org/roadmap >> > > >> > > [2] https://lists.apache.org/thread/szdr4ngrfcmo7zko4917393zbqhgw0v5 >> > > >> > > >> > > >> > > On Mon, Jul 17, 2023 at 9:53 AM Xintong Song >> > > wrote: >> > > >> > > > I'd propose to downgrade "Refactor the API modules" to TBD. The >> > original >> > > > proposal was based on the condition that we are allowed to introduce >> > > > in-place API breaking changes in release 2.0. As the migration >> period >> > is >> > > > introduced, and we are no longer planning to do in-place changes / >> > > > removal for DataStream (and same for APIs in `flink-core`), we need >> to >> > > > re-evaluate whether it's feasible to do things like moving classes >> to >> > > > different module / packages, turning concrete classes into >> interfaces >> > on >> > > > the API classes. >> > > > >> > > > Best, >> > > > >> > > > Xintong >> > > > >> > > > >> > > > >> > > > On Mon, Jul 17, 2023 at 1:10 AM Yun Tang wrote: >> > > > >> > > >> I agree that we could downgrade "Eager state declaration" to a >> > > >> nice-to-have feature. >> > > >> >> > > >> For the depreciation of "queryable state", can we just rename to >> > > >> deprecate "current implementation of queryable state"? The feature >> to >> > > query >> > > >> the internal state is actually very useful for debugging and could >> > > provide >> > > >> more possibility to extend FlinkSQL more like a database. >> > > >> >> > > >> Just as Yuan replied in the previous email [1], current >> implementation >> > > of >> > > >> queryable state has many problems in design. However, I don't want >> to >> > > make >> > > >> users feel that this feature cannot be done well, and maybe we can >> > > redesign >> > > >> this feature. As far as I know, risingwave alr
Re: [VOTE] Release 2.0 must-have work items
Let me try to summarize the proposed changes on the list. - "Eager State Declaration" should be nice-to-have - "Remove SourceFunction / SinkFunction / SinkV1" should be changed to "Remove SinkV1", and remain must-to-have - "Remove Queryable State" should be nice-to-have. It will be deprecated in 1.18, but the hard removal requires community discussion and vote, which may or may not happen in 2.0. - "Refactor the API modules" should be TBD. - I also noticed Zhenqiu Huang has already changed "Drop YARN-specific mutating GET REST endpoints" from TBD to must-have, which I'd like to bring up here for attention. I'd leave this discussion open for the next couple of days. If there are no objections, I'll update the list and start another round of voting. In addition, I'd like to cross-post from the other thread [1] that: I'm not aware of any decision that has already been made by the community > regarding after which 1.x minor release will we ship the 2.0 major release. > I also don't think we should push all the deprecation works in 1.18. > > Deciding to deprecate / remove an API definitely deserves thorough > discussions and FLIP, which takes time, and I don't think we should > compromise that for any reason. > Assuming at some point we want to ship the 2.0 release, and there are some > deprecated APIs that we want to remove but have not fulfilled the migration > period required by FLIP-321, I see 3 options to move forward. 1. Not removing the deprecated APIs in 2.0, carrying them until 3.0. 2. Postpone the 2.0 release for another minor release. 3. Considering such APIs as exceptions of FLIP-321. > Trying to deprecate things early is still helpful, because it reduces the > number of APIs we need to consider when deciding between the options. > However, that must not come at the price of rush decisions. I'd suggest > developers to design / discuss / vote the breaking changes at their pace, > and we evaluate the status and choose between the options later (maybe > around the time releasing 1.19). > Best, Xintong [1] https://lists.apache.org/thread/m0b1v2htd0l7oqo6ypf8lnjyjo817bmm On Mon, Jul 17, 2023 at 6:33 PM Yuan Mei wrote: > Hey Yun and Xintong, > > (Had a quick offline discussion with Yun) > 1. I agree the current implementation of the queryable state is not a > blocker of anything related to disaggregated state management. They are > different things. > 2. On the other hand, "queryable snapshot" is not a completely equivalent > substitution of "queryable state". > 3. But in whatever way, I think the way how "queryable state" is designed > is not the right way to move forward. > 4. "Deprecating queryable state" is put as a must-have because this topic > has been raised many times along the way. It seems to reach an agreement > every time as mentioned by Xingtong, but no one really takes the action. > > I am suggesting: > > 1. Remove "Deprecating queryable state" from the must-have list (since it > does not meet the requirements of "must-have") > 2. But I am still hoping we can move things forward, so let's put > the @Deprecated annotation on it > 3. Removal of the code follows a formal community discussion and vote. > > Best > Yuan > > > > > On Mon, Jul 17, 2023 at 3:40 PM Xintong Song > wrote: > > > Thanks for the clarification. > > > > If the list of "Remove deprecated APIs" means, we must remove the code in > > > Flink-2.0 initial release, I would vote -1 for queryable state before > we > > > get an alternative. > > > > > > FYI, the removal of queryable state is currently marked as the > `must-have` > > priority. Of course it's not a final decision and that's exactly why we > > are collecting feedback about the list now. > > > > Best, > > > > Xintong > > > > > > > > On Mon, Jul 17, 2023 at 3:15 PM Yun Tang wrote: > > > > > Hi Xintong, > > > > > > If the current implementation of queryable state would not block the > > > implementation of disaggregated state-backends. > > > I prefer to not removing the implementation until we have a better > > > solution (maybe based on the queryable snapshot) cc @Yuan. > > > > > > If the list of "Remove deprecated APIs" means, we must remove the code > in > > > Flink-2.0 initial release, I would vote -1 for queryable state before > we > > > get an alternative. > > > And I will raise the concern in the Flink roadmap discussion. > > > > > > > > > Best > > > Yun Tang > &g
Re: [VOTE] Release 2.0 must-have work items
Hey Yun and Xintong, (Had a quick offline discussion with Yun) 1. I agree the current implementation of the queryable state is not a blocker of anything related to disaggregated state management. They are different things. 2. On the other hand, "queryable snapshot" is not a completely equivalent substitution of "queryable state". 3. But in whatever way, I think the way how "queryable state" is designed is not the right way to move forward. 4. "Deprecating queryable state" is put as a must-have because this topic has been raised many times along the way. It seems to reach an agreement every time as mentioned by Xingtong, but no one really takes the action. I am suggesting: 1. Remove "Deprecating queryable state" from the must-have list (since it does not meet the requirements of "must-have") 2. But I am still hoping we can move things forward, so let's put the @Deprecated annotation on it 3. Removal of the code follows a formal community discussion and vote. Best Yuan On Mon, Jul 17, 2023 at 3:40 PM Xintong Song wrote: > Thanks for the clarification. > > If the list of "Remove deprecated APIs" means, we must remove the code in > > Flink-2.0 initial release, I would vote -1 for queryable state before we > > get an alternative. > > > FYI, the removal of queryable state is currently marked as the `must-have` > priority. Of course it's not a final decision and that's exactly why we > are collecting feedback about the list now. > > Best, > > Xintong > > > > On Mon, Jul 17, 2023 at 3:15 PM Yun Tang wrote: > > > Hi Xintong, > > > > If the current implementation of queryable state would not block the > > implementation of disaggregated state-backends. > > I prefer to not removing the implementation until we have a better > > solution (maybe based on the queryable snapshot) cc @Yuan. > > > > If the list of "Remove deprecated APIs" means, we must remove the code in > > Flink-2.0 initial release, I would vote -1 for queryable state before we > > get an alternative. > > And I will raise the concern in the Flink roadmap discussion. > > > > > > Best > > Yun Tang > > > > From: Xintong Song > > Sent: Monday, July 17, 2023 10:07 > > To: dev@flink.apache.org > > Subject: Re: [VOTE] Release 2.0 must-have work items > > > > @Yun, > > I see your point that the ability queryable states trying to provide is > > meaningful but the current implementation of the feature is problematic. > So > > what's your opinion on deprecating the current queryable state? Do you > > think we need to wait until there is a new implementation of queryable > > state to remove the current one? Or maybe the current implementation is > not > > well functional anyway and we can treat the removal of it as > > independent from introducing a new one? > > > > However, I don't want to make users feel that this feature cannot be done > > > well, and maybe we can redesign this feature. > > > > > TBH, the impression that I got from the roadmap[1] is that the queryable > > state is retiring and will be replaced by the state processor api. If > this > > is not the impression we want users to have, you probably also need to > > raise it in the roadmap discussion [2]. > > > > Best, > > > > Xintong > > > > > > [1] https://flink.apache.org/roadmap > > > > [2] https://lists.apache.org/thread/szdr4ngrfcmo7zko4917393zbqhgw0v5 > > > > > > > > On Mon, Jul 17, 2023 at 9:53 AM Xintong Song > > wrote: > > > > > I'd propose to downgrade "Refactor the API modules" to TBD. The > original > > > proposal was based on the condition that we are allowed to introduce > > > in-place API breaking changes in release 2.0. As the migration period > is > > > introduced, and we are no longer planning to do in-place changes / > > > removal for DataStream (and same for APIs in `flink-core`), we need to > > > re-evaluate whether it's feasible to do things like moving classes to > > > different module / packages, turning concrete classes into interfaces > on > > > the API classes. > > > > > > Best, > > > > > > Xintong > > > > > > > > > > > > On Mon, Jul 17, 2023 at 1:10 AM Yun Tang wrote: > > > > > >> I agree that we could downgrade "Eager state declaration" to a > > >> nice-to-have feature. > > >> > > >> For the depreciation of "queryable state"
Re: [VOTE] Release 2.0 must-have work items
Thanks for the clarification. If the list of "Remove deprecated APIs" means, we must remove the code in > Flink-2.0 initial release, I would vote -1 for queryable state before we > get an alternative. FYI, the removal of queryable state is currently marked as the `must-have` priority. Of course it's not a final decision and that's exactly why we are collecting feedback about the list now. Best, Xintong On Mon, Jul 17, 2023 at 3:15 PM Yun Tang wrote: > Hi Xintong, > > If the current implementation of queryable state would not block the > implementation of disaggregated state-backends. > I prefer to not removing the implementation until we have a better > solution (maybe based on the queryable snapshot) cc @Yuan. > > If the list of "Remove deprecated APIs" means, we must remove the code in > Flink-2.0 initial release, I would vote -1 for queryable state before we > get an alternative. > And I will raise the concern in the Flink roadmap discussion. > > > Best > Yun Tang > > From: Xintong Song > Sent: Monday, July 17, 2023 10:07 > To: dev@flink.apache.org > Subject: Re: [VOTE] Release 2.0 must-have work items > > @Yun, > I see your point that the ability queryable states trying to provide is > meaningful but the current implementation of the feature is problematic. So > what's your opinion on deprecating the current queryable state? Do you > think we need to wait until there is a new implementation of queryable > state to remove the current one? Or maybe the current implementation is not > well functional anyway and we can treat the removal of it as > independent from introducing a new one? > > However, I don't want to make users feel that this feature cannot be done > > well, and maybe we can redesign this feature. > > > TBH, the impression that I got from the roadmap[1] is that the queryable > state is retiring and will be replaced by the state processor api. If this > is not the impression we want users to have, you probably also need to > raise it in the roadmap discussion [2]. > > Best, > > Xintong > > > [1] https://flink.apache.org/roadmap > > [2] https://lists.apache.org/thread/szdr4ngrfcmo7zko4917393zbqhgw0v5 > > > > On Mon, Jul 17, 2023 at 9:53 AM Xintong Song > wrote: > > > I'd propose to downgrade "Refactor the API modules" to TBD. The original > > proposal was based on the condition that we are allowed to introduce > > in-place API breaking changes in release 2.0. As the migration period is > > introduced, and we are no longer planning to do in-place changes / > > removal for DataStream (and same for APIs in `flink-core`), we need to > > re-evaluate whether it's feasible to do things like moving classes to > > different module / packages, turning concrete classes into interfaces on > > the API classes. > > > > Best, > > > > Xintong > > > > > > > > On Mon, Jul 17, 2023 at 1:10 AM Yun Tang wrote: > > > >> I agree that we could downgrade "Eager state declaration" to a > >> nice-to-have feature. > >> > >> For the depreciation of "queryable state", can we just rename to > >> deprecate "current implementation of queryable state"? The feature to > query > >> the internal state is actually very useful for debugging and could > provide > >> more possibility to extend FlinkSQL more like a database. > >> > >> Just as Yuan replied in the previous email [1], current implementation > of > >> queryable state has many problems in design. However, I don't want to > make > >> users feel that this feature cannot be done well, and maybe we can > redesign > >> this feature. As far as I know, risingwave already support queryable > state > >> with better user experience [2]. > >> > >> > >> [1] https://lists.apache.org/thread/9hmwcjb3q5c24pk3qshjvybfqk62v17m > >> [2] https://syntaxbug.com/06a3e7c554/ > >> > >> Best > >> Yun Tang > >> > >> From: Xintong Song > >> Sent: Friday, July 14, 2023 13:51 > >> To: dev@flink.apache.org > >> Subject: Re: [VOTE] Release 2.0 must-have work items > >> > >> Thanks for the support, Yu. > >> > >> We will have the guideline before removing DataSet. We are currently > >> prioritizing works that need to be done before the 1.18 feature freeze, > >> and > >> will soon get back to working on the guidelines. We expect to get the > >> guideline ready before or soon after the 1.18
Re: [VOTE] Release 2.0 must-have work items
Hi Xintong, If the current implementation of queryable state would not block the implementation of disaggregated state-backends. I prefer to not removing the implementation until we have a better solution (maybe based on the queryable snapshot) cc @Yuan. If the list of "Remove deprecated APIs" means, we must remove the code in Flink-2.0 initial release, I would vote -1 for queryable state before we get an alternative. And I will raise the concern in the Flink roadmap discussion. Best Yun Tang From: Xintong Song Sent: Monday, July 17, 2023 10:07 To: dev@flink.apache.org Subject: Re: [VOTE] Release 2.0 must-have work items @Yun, I see your point that the ability queryable states trying to provide is meaningful but the current implementation of the feature is problematic. So what's your opinion on deprecating the current queryable state? Do you think we need to wait until there is a new implementation of queryable state to remove the current one? Or maybe the current implementation is not well functional anyway and we can treat the removal of it as independent from introducing a new one? However, I don't want to make users feel that this feature cannot be done > well, and maybe we can redesign this feature. > TBH, the impression that I got from the roadmap[1] is that the queryable state is retiring and will be replaced by the state processor api. If this is not the impression we want users to have, you probably also need to raise it in the roadmap discussion [2]. Best, Xintong [1] https://flink.apache.org/roadmap [2] https://lists.apache.org/thread/szdr4ngrfcmo7zko4917393zbqhgw0v5 On Mon, Jul 17, 2023 at 9:53 AM Xintong Song wrote: > I'd propose to downgrade "Refactor the API modules" to TBD. The original > proposal was based on the condition that we are allowed to introduce > in-place API breaking changes in release 2.0. As the migration period is > introduced, and we are no longer planning to do in-place changes / > removal for DataStream (and same for APIs in `flink-core`), we need to > re-evaluate whether it's feasible to do things like moving classes to > different module / packages, turning concrete classes into interfaces on > the API classes. > > Best, > > Xintong > > > > On Mon, Jul 17, 2023 at 1:10 AM Yun Tang wrote: > >> I agree that we could downgrade "Eager state declaration" to a >> nice-to-have feature. >> >> For the depreciation of "queryable state", can we just rename to >> deprecate "current implementation of queryable state"? The feature to query >> the internal state is actually very useful for debugging and could provide >> more possibility to extend FlinkSQL more like a database. >> >> Just as Yuan replied in the previous email [1], current implementation of >> queryable state has many problems in design. However, I don't want to make >> users feel that this feature cannot be done well, and maybe we can redesign >> this feature. As far as I know, risingwave already support queryable state >> with better user experience [2]. >> >> >> [1] https://lists.apache.org/thread/9hmwcjb3q5c24pk3qshjvybfqk62v17m >> [2] https://syntaxbug.com/06a3e7c554/ >> >> Best >> Yun Tang >> >> From: Xintong Song >> Sent: Friday, July 14, 2023 13:51 >> To: dev@flink.apache.org >> Subject: Re: [VOTE] Release 2.0 must-have work items >> >> Thanks for the support, Yu. >> >> We will have the guideline before removing DataSet. We are currently >> prioritizing works that need to be done before the 1.18 feature freeze, >> and >> will soon get back to working on the guidelines. We expect to get the >> guideline ready before or soon after the 1.18 release, which will >> definitely be before removing DataSet in 2.0. >> >> Best, >> >> Xintong >> >> >> >> On Fri, Jul 14, 2023 at 1:06 PM Yu Li wrote: >> >> > It's great to see the discussion about what we need to improve on >> > (completely) switching from DataSet API to DataStream API from the user >> > perspective. I feel that these improvements would happen faster (only) >> when >> > we seriously prepare to remove the DataSet APIs with a target release, >> just >> > like what we are doing now. And the same applies to the SinkV1 related >> > discussions (smile). >> > >> > I support Xintong's opinion on keeping "Remove the DataSet APIs" a >> > must-have item, meantime I support Yuxia's opinion that we should >> > explicitly let our users know how to migrate their existing DataSet API >> > based application
Re: [VOTE] Release 2.0 must-have work items
@Yun, I see your point that the ability queryable states trying to provide is meaningful but the current implementation of the feature is problematic. So what's your opinion on deprecating the current queryable state? Do you think we need to wait until there is a new implementation of queryable state to remove the current one? Or maybe the current implementation is not well functional anyway and we can treat the removal of it as independent from introducing a new one? However, I don't want to make users feel that this feature cannot be done > well, and maybe we can redesign this feature. > TBH, the impression that I got from the roadmap[1] is that the queryable state is retiring and will be replaced by the state processor api. If this is not the impression we want users to have, you probably also need to raise it in the roadmap discussion [2]. Best, Xintong [1] https://flink.apache.org/roadmap [2] https://lists.apache.org/thread/szdr4ngrfcmo7zko4917393zbqhgw0v5 On Mon, Jul 17, 2023 at 9:53 AM Xintong Song wrote: > I'd propose to downgrade "Refactor the API modules" to TBD. The original > proposal was based on the condition that we are allowed to introduce > in-place API breaking changes in release 2.0. As the migration period is > introduced, and we are no longer planning to do in-place changes / > removal for DataStream (and same for APIs in `flink-core`), we need to > re-evaluate whether it's feasible to do things like moving classes to > different module / packages, turning concrete classes into interfaces on > the API classes. > > Best, > > Xintong > > > > On Mon, Jul 17, 2023 at 1:10 AM Yun Tang wrote: > >> I agree that we could downgrade "Eager state declaration" to a >> nice-to-have feature. >> >> For the depreciation of "queryable state", can we just rename to >> deprecate "current implementation of queryable state"? The feature to query >> the internal state is actually very useful for debugging and could provide >> more possibility to extend FlinkSQL more like a database. >> >> Just as Yuan replied in the previous email [1], current implementation of >> queryable state has many problems in design. However, I don't want to make >> users feel that this feature cannot be done well, and maybe we can redesign >> this feature. As far as I know, risingwave already support queryable state >> with better user experience [2]. >> >> >> [1] https://lists.apache.org/thread/9hmwcjb3q5c24pk3qshjvybfqk62v17m >> [2] https://syntaxbug.com/06a3e7c554/ >> >> Best >> Yun Tang >> >> From: Xintong Song >> Sent: Friday, July 14, 2023 13:51 >> To: dev@flink.apache.org >> Subject: Re: [VOTE] Release 2.0 must-have work items >> >> Thanks for the support, Yu. >> >> We will have the guideline before removing DataSet. We are currently >> prioritizing works that need to be done before the 1.18 feature freeze, >> and >> will soon get back to working on the guidelines. We expect to get the >> guideline ready before or soon after the 1.18 release, which will >> definitely be before removing DataSet in 2.0. >> >> Best, >> >> Xintong >> >> >> >> On Fri, Jul 14, 2023 at 1:06 PM Yu Li wrote: >> >> > It's great to see the discussion about what we need to improve on >> > (completely) switching from DataSet API to DataStream API from the user >> > perspective. I feel that these improvements would happen faster (only) >> when >> > we seriously prepare to remove the DataSet APIs with a target release, >> just >> > like what we are doing now. And the same applies to the SinkV1 related >> > discussions (smile). >> > >> > I support Xintong's opinion on keeping "Remove the DataSet APIs" a >> > must-have item, meantime I support Yuxia's opinion that we should >> > explicitly let our users know how to migrate their existing DataSet API >> > based applications afterwards, meaning that the guideline Xintong >> mentioned >> > is a must-have (rather than best efforts) before removing the DataSet >> APIs. >> > >> > Best Regards, >> > Yu >> > >> > >> > On Wed, 12 Jul 2023 at 14:00, yuxia >> wrote: >> > >> > > Thanks Xintong for clarification. A guideline to help users migrating >> > from >> > > DataSet to DataStream will definitely be helpful. >> > > >> > > Best regards, >> > > Yuxia >> > > >> > > - 原始邮件 - >> > > 发件人: "Xintong
Re: [VOTE] Release 2.0 must-have work items
I'd propose to downgrade "Refactor the API modules" to TBD. The original proposal was based on the condition that we are allowed to introduce in-place API breaking changes in release 2.0. As the migration period is introduced, and we are no longer planning to do in-place changes / removal for DataStream (and same for APIs in `flink-core`), we need to re-evaluate whether it's feasible to do things like moving classes to different module / packages, turning concrete classes into interfaces on the API classes. Best, Xintong On Mon, Jul 17, 2023 at 1:10 AM Yun Tang wrote: > I agree that we could downgrade "Eager state declaration" to a > nice-to-have feature. > > For the depreciation of "queryable state", can we just rename to deprecate > "current implementation of queryable state"? The feature to query the > internal state is actually very useful for debugging and could provide more > possibility to extend FlinkSQL more like a database. > > Just as Yuan replied in the previous email [1], current implementation of > queryable state has many problems in design. However, I don't want to make > users feel that this feature cannot be done well, and maybe we can redesign > this feature. As far as I know, risingwave already support queryable state > with better user experience [2]. > > > [1] https://lists.apache.org/thread/9hmwcjb3q5c24pk3qshjvybfqk62v17m > [2] https://syntaxbug.com/06a3e7c554/ > > Best > Yun Tang > ________ > From: Xintong Song > Sent: Friday, July 14, 2023 13:51 > To: dev@flink.apache.org > Subject: Re: [VOTE] Release 2.0 must-have work items > > Thanks for the support, Yu. > > We will have the guideline before removing DataSet. We are currently > prioritizing works that need to be done before the 1.18 feature freeze, and > will soon get back to working on the guidelines. We expect to get the > guideline ready before or soon after the 1.18 release, which will > definitely be before removing DataSet in 2.0. > > Best, > > Xintong > > > > On Fri, Jul 14, 2023 at 1:06 PM Yu Li wrote: > > > It's great to see the discussion about what we need to improve on > > (completely) switching from DataSet API to DataStream API from the user > > perspective. I feel that these improvements would happen faster (only) > when > > we seriously prepare to remove the DataSet APIs with a target release, > just > > like what we are doing now. And the same applies to the SinkV1 related > > discussions (smile). > > > > I support Xintong's opinion on keeping "Remove the DataSet APIs" a > > must-have item, meantime I support Yuxia's opinion that we should > > explicitly let our users know how to migrate their existing DataSet API > > based applications afterwards, meaning that the guideline Xintong > mentioned > > is a must-have (rather than best efforts) before removing the DataSet > APIs. > > > > Best Regards, > > Yu > > > > > > On Wed, 12 Jul 2023 at 14:00, yuxia wrote: > > > > > Thanks Xintong for clarification. A guideline to help users migrating > > from > > > DataSet to DataStream will definitely be helpful. > > > > > > Best regards, > > > Yuxia > > > > > > - 原始邮件 - > > > 发件人: "Xintong Song" > > > 收件人: "dev" > > > 发送时间: 星期三, 2023年 7 月 12日 上午 11:40:12 > > > 主题: Re: [VOTE] Release 2.0 must-have work items > > > > > > @Yuxia, > > > > > > We are aware of the issue that you mentioned. Actually, I don't think > the > > > DataStream API can cover everything in the DataSet API in exactly the > > same > > > way, because the fundamental model, concepts and primitives of the two > > sets > > > of APIs are completely different. Many of the DataSet APIs, especially > > > those accessing the full data set at once, do not fit in the DataStream > > > concepts at all. I think what's important is that users can achieve the > > > same function, even if they may need to code in a different way. > > > > > > We have gone through all the existing DataSet APIs, and categorized > them > > > into 3 kinds: > > > - APIs that are well supported by DataStream API as is. E.g., map, > reduce > > > on grouped dataset, etc. > > > - APIs that can be achieved by DataStream API as is, but with a price > > > (programming complexity, or computation efficiency). E.g., reduce on > full > > > dataset, sort partition, etc. Admittedly, there is room for improvement > > on > > > these. We may ke
Re: [VOTE] Release 2.0 must-have work items
I agree that we could downgrade "Eager state declaration" to a nice-to-have feature. For the depreciation of "queryable state", can we just rename to deprecate "current implementation of queryable state"? The feature to query the internal state is actually very useful for debugging and could provide more possibility to extend FlinkSQL more like a database. Just as Yuan replied in the previous email [1], current implementation of queryable state has many problems in design. However, I don't want to make users feel that this feature cannot be done well, and maybe we can redesign this feature. As far as I know, risingwave already support queryable state with better user experience [2]. [1] https://lists.apache.org/thread/9hmwcjb3q5c24pk3qshjvybfqk62v17m [2] https://syntaxbug.com/06a3e7c554/ Best Yun Tang From: Xintong Song Sent: Friday, July 14, 2023 13:51 To: dev@flink.apache.org Subject: Re: [VOTE] Release 2.0 must-have work items Thanks for the support, Yu. We will have the guideline before removing DataSet. We are currently prioritizing works that need to be done before the 1.18 feature freeze, and will soon get back to working on the guidelines. We expect to get the guideline ready before or soon after the 1.18 release, which will definitely be before removing DataSet in 2.0. Best, Xintong On Fri, Jul 14, 2023 at 1:06 PM Yu Li wrote: > It's great to see the discussion about what we need to improve on > (completely) switching from DataSet API to DataStream API from the user > perspective. I feel that these improvements would happen faster (only) when > we seriously prepare to remove the DataSet APIs with a target release, just > like what we are doing now. And the same applies to the SinkV1 related > discussions (smile). > > I support Xintong's opinion on keeping "Remove the DataSet APIs" a > must-have item, meantime I support Yuxia's opinion that we should > explicitly let our users know how to migrate their existing DataSet API > based applications afterwards, meaning that the guideline Xintong mentioned > is a must-have (rather than best efforts) before removing the DataSet APIs. > > Best Regards, > Yu > > > On Wed, 12 Jul 2023 at 14:00, yuxia wrote: > > > Thanks Xintong for clarification. A guideline to help users migrating > from > > DataSet to DataStream will definitely be helpful. > > > > Best regards, > > Yuxia > > > > ----- 原始邮件 - > > 发件人: "Xintong Song" > > 收件人: "dev" > > 发送时间: 星期三, 2023年 7 月 12日 上午 11:40:12 > > 主题: Re: [VOTE] Release 2.0 must-have work items > > > > @Yuxia, > > > > We are aware of the issue that you mentioned. Actually, I don't think the > > DataStream API can cover everything in the DataSet API in exactly the > same > > way, because the fundamental model, concepts and primitives of the two > sets > > of APIs are completely different. Many of the DataSet APIs, especially > > those accessing the full data set at once, do not fit in the DataStream > > concepts at all. I think what's important is that users can achieve the > > same function, even if they may need to code in a different way. > > > > We have gone through all the existing DataSet APIs, and categorized them > > into 3 kinds: > > - APIs that are well supported by DataStream API as is. E.g., map, reduce > > on grouped dataset, etc. > > - APIs that can be achieved by DataStream API as is, but with a price > > (programming complexity, or computation efficiency). E.g., reduce on full > > dataset, sort partition, etc. Admittedly, there is room for improvement > on > > these. We may keep improving these for the DataStream API, or we can > > concentrate on supporting them better in the new ProcessFunction API. > > Either way, I don't think we should block the retiring of DataSet API on > > them. > > - There are also a few APIs that cannot be supported by the DataStream > API > > as is, unless users write their custom operators from the ground up. Only > > left/rightOuterJoin and combineGroup fall into this category. I think > > combinedGroup is probably not a problem, because this is more like a > > variant of reduceGroup that allows the framework to execute more > > efficiently. As for the outer joins, depending on how badly this is > needed, > > it can be supported by emitting the non-joined entries upon triggering a > > window join. > > > > We are also planning to draft a guideline to help users migrating from > > DataSet to DataStream, which should demonstrate how users can achieve > > things like sort-partition with DataStream API. > > > &g
Re: [VOTE] Release 2.0 must-have work items
Thanks for the support, Yu. We will have the guideline before removing DataSet. We are currently prioritizing works that need to be done before the 1.18 feature freeze, and will soon get back to working on the guidelines. We expect to get the guideline ready before or soon after the 1.18 release, which will definitely be before removing DataSet in 2.0. Best, Xintong On Fri, Jul 14, 2023 at 1:06 PM Yu Li wrote: > It's great to see the discussion about what we need to improve on > (completely) switching from DataSet API to DataStream API from the user > perspective. I feel that these improvements would happen faster (only) when > we seriously prepare to remove the DataSet APIs with a target release, just > like what we are doing now. And the same applies to the SinkV1 related > discussions (smile). > > I support Xintong's opinion on keeping "Remove the DataSet APIs" a > must-have item, meantime I support Yuxia's opinion that we should > explicitly let our users know how to migrate their existing DataSet API > based applications afterwards, meaning that the guideline Xintong mentioned > is a must-have (rather than best efforts) before removing the DataSet APIs. > > Best Regards, > Yu > > > On Wed, 12 Jul 2023 at 14:00, yuxia wrote: > > > Thanks Xintong for clarification. A guideline to help users migrating > from > > DataSet to DataStream will definitely be helpful. > > > > Best regards, > > Yuxia > > > > - 原始邮件 ----- > > 发件人: "Xintong Song" > > 收件人: "dev" > > 发送时间: 星期三, 2023年 7 月 12日 上午 11:40:12 > > 主题: Re: [VOTE] Release 2.0 must-have work items > > > > @Yuxia, > > > > We are aware of the issue that you mentioned. Actually, I don't think the > > DataStream API can cover everything in the DataSet API in exactly the > same > > way, because the fundamental model, concepts and primitives of the two > sets > > of APIs are completely different. Many of the DataSet APIs, especially > > those accessing the full data set at once, do not fit in the DataStream > > concepts at all. I think what's important is that users can achieve the > > same function, even if they may need to code in a different way. > > > > We have gone through all the existing DataSet APIs, and categorized them > > into 3 kinds: > > - APIs that are well supported by DataStream API as is. E.g., map, reduce > > on grouped dataset, etc. > > - APIs that can be achieved by DataStream API as is, but with a price > > (programming complexity, or computation efficiency). E.g., reduce on full > > dataset, sort partition, etc. Admittedly, there is room for improvement > on > > these. We may keep improving these for the DataStream API, or we can > > concentrate on supporting them better in the new ProcessFunction API. > > Either way, I don't think we should block the retiring of DataSet API on > > them. > > - There are also a few APIs that cannot be supported by the DataStream > API > > as is, unless users write their custom operators from the ground up. Only > > left/rightOuterJoin and combineGroup fall into this category. I think > > combinedGroup is probably not a problem, because this is more like a > > variant of reduceGroup that allows the framework to execute more > > efficiently. As for the outer joins, depending on how badly this is > needed, > > it can be supported by emitting the non-joined entries upon triggering a > > window join. > > > > We are also planning to draft a guideline to help users migrating from > > DataSet to DataStream, which should demonstrate how users can achieve > > things like sort-partition with DataStream API. > > > > Last but not least, I'd like to point out that the decision to deprecate > > and eventually remove the DataSet API was approved in FLIP-131, and all > the > > prerequisites mentioned in the FLIP have been completed. > > > > Best, > > > > Xintong > > > > > > [1] > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158866741 > > > > > > > > On Wed, Jul 12, 2023 at 10:20 AM Jingsong Li > > wrote: > > > > > +1 to Leonard and Galen and Jing. > > > > > > About Source and Sink. > > > We're still missing quite a bit of work, including functionality, > > > including ease of use, including bug fixes, and I'm not sure we'll be > > > completely done by 2.0. > > > Until that's done, we won't be in a position to clean up the old APIs. > > > > > > Best, > > > Jingsong > > > > > > On Wed, Jul 12, 2
Re: [VOTE] Release 2.0 must-have work items
It's great to see the discussion about what we need to improve on (completely) switching from DataSet API to DataStream API from the user perspective. I feel that these improvements would happen faster (only) when we seriously prepare to remove the DataSet APIs with a target release, just like what we are doing now. And the same applies to the SinkV1 related discussions (smile). I support Xintong's opinion on keeping "Remove the DataSet APIs" a must-have item, meantime I support Yuxia's opinion that we should explicitly let our users know how to migrate their existing DataSet API based applications afterwards, meaning that the guideline Xintong mentioned is a must-have (rather than best efforts) before removing the DataSet APIs. Best Regards, Yu On Wed, 12 Jul 2023 at 14:00, yuxia wrote: > Thanks Xintong for clarification. A guideline to help users migrating from > DataSet to DataStream will definitely be helpful. > > Best regards, > Yuxia > > - 原始邮件 - > 发件人: "Xintong Song" > 收件人: "dev" > 发送时间: 星期三, 2023年 7 月 12日 上午 11:40:12 > 主题: Re: [VOTE] Release 2.0 must-have work items > > @Yuxia, > > We are aware of the issue that you mentioned. Actually, I don't think the > DataStream API can cover everything in the DataSet API in exactly the same > way, because the fundamental model, concepts and primitives of the two sets > of APIs are completely different. Many of the DataSet APIs, especially > those accessing the full data set at once, do not fit in the DataStream > concepts at all. I think what's important is that users can achieve the > same function, even if they may need to code in a different way. > > We have gone through all the existing DataSet APIs, and categorized them > into 3 kinds: > - APIs that are well supported by DataStream API as is. E.g., map, reduce > on grouped dataset, etc. > - APIs that can be achieved by DataStream API as is, but with a price > (programming complexity, or computation efficiency). E.g., reduce on full > dataset, sort partition, etc. Admittedly, there is room for improvement on > these. We may keep improving these for the DataStream API, or we can > concentrate on supporting them better in the new ProcessFunction API. > Either way, I don't think we should block the retiring of DataSet API on > them. > - There are also a few APIs that cannot be supported by the DataStream API > as is, unless users write their custom operators from the ground up. Only > left/rightOuterJoin and combineGroup fall into this category. I think > combinedGroup is probably not a problem, because this is more like a > variant of reduceGroup that allows the framework to execute more > efficiently. As for the outer joins, depending on how badly this is needed, > it can be supported by emitting the non-joined entries upon triggering a > window join. > > We are also planning to draft a guideline to help users migrating from > DataSet to DataStream, which should demonstrate how users can achieve > things like sort-partition with DataStream API. > > Last but not least, I'd like to point out that the decision to deprecate > and eventually remove the DataSet API was approved in FLIP-131, and all the > prerequisites mentioned in the FLIP have been completed. > > Best, > > Xintong > > > [1] > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158866741 > > > > On Wed, Jul 12, 2023 at 10:20 AM Jingsong Li > wrote: > > > +1 to Leonard and Galen and Jing. > > > > About Source and Sink. > > We're still missing quite a bit of work, including functionality, > > including ease of use, including bug fixes, and I'm not sure we'll be > > completely done by 2.0. > > Until that's done, we won't be in a position to clean up the old APIs. > > > > Best, > > Jingsong > > > > On Wed, Jul 12, 2023 at 9:41 AM yuxia > wrote: > > > > > > Hi,Xintong. > > > Sorry to disturb the voting. I just found an email[1] about DataSet API > > from flink-user-zh channel. And I think it's not just a single case > > according to my observation. > > > > > > Remove DataSet is a must have item in release-2.0. But as the user > email > > said, if we remove DataSet, how users can implement Sort/PartitionBy, etc > > as they did with DataSet? > > > Do we will also provide similar api in datastream or some other thing > > before we remove DataSet? > > > Btw, as far as I see, with regarding to replcaing DataSet with > > Datastream, Datastream are missing many API. I think it may well take > much > > effort to fully cover the missing api. > > > > > > [1] https://lists.apache.org/thread/syjmt8f74gh8ok3z
Re: [VOTE] Release 2.0 must-have work items
Thanks Xintong for clarification. A guideline to help users migrating from DataSet to DataStream will definitely be helpful. Best regards, Yuxia - 原始邮件 - 发件人: "Xintong Song" 收件人: "dev" 发送时间: 星期三, 2023年 7 月 12日 上午 11:40:12 主题: Re: [VOTE] Release 2.0 must-have work items @Yuxia, We are aware of the issue that you mentioned. Actually, I don't think the DataStream API can cover everything in the DataSet API in exactly the same way, because the fundamental model, concepts and primitives of the two sets of APIs are completely different. Many of the DataSet APIs, especially those accessing the full data set at once, do not fit in the DataStream concepts at all. I think what's important is that users can achieve the same function, even if they may need to code in a different way. We have gone through all the existing DataSet APIs, and categorized them into 3 kinds: - APIs that are well supported by DataStream API as is. E.g., map, reduce on grouped dataset, etc. - APIs that can be achieved by DataStream API as is, but with a price (programming complexity, or computation efficiency). E.g., reduce on full dataset, sort partition, etc. Admittedly, there is room for improvement on these. We may keep improving these for the DataStream API, or we can concentrate on supporting them better in the new ProcessFunction API. Either way, I don't think we should block the retiring of DataSet API on them. - There are also a few APIs that cannot be supported by the DataStream API as is, unless users write their custom operators from the ground up. Only left/rightOuterJoin and combineGroup fall into this category. I think combinedGroup is probably not a problem, because this is more like a variant of reduceGroup that allows the framework to execute more efficiently. As for the outer joins, depending on how badly this is needed, it can be supported by emitting the non-joined entries upon triggering a window join. We are also planning to draft a guideline to help users migrating from DataSet to DataStream, which should demonstrate how users can achieve things like sort-partition with DataStream API. Last but not least, I'd like to point out that the decision to deprecate and eventually remove the DataSet API was approved in FLIP-131, and all the prerequisites mentioned in the FLIP have been completed. Best, Xintong [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158866741 On Wed, Jul 12, 2023 at 10:20 AM Jingsong Li wrote: > +1 to Leonard and Galen and Jing. > > About Source and Sink. > We're still missing quite a bit of work, including functionality, > including ease of use, including bug fixes, and I'm not sure we'll be > completely done by 2.0. > Until that's done, we won't be in a position to clean up the old APIs. > > Best, > Jingsong > > On Wed, Jul 12, 2023 at 9:41 AM yuxia wrote: > > > > Hi,Xintong. > > Sorry to disturb the voting. I just found an email[1] about DataSet API > from flink-user-zh channel. And I think it's not just a single case > according to my observation. > > > > Remove DataSet is a must have item in release-2.0. But as the user email > said, if we remove DataSet, how users can implement Sort/PartitionBy, etc > as they did with DataSet? > > Do we will also provide similar api in datastream or some other thing > before we remove DataSet? > > Btw, as far as I see, with regarding to replcaing DataSet with > Datastream, Datastream are missing many API. I think it may well take much > effort to fully cover the missing api. > > > > [1] https://lists.apache.org/thread/syjmt8f74gh8ok3z4lhgt95zl4dzn168 > > > > Best regards, > > Yuxia > > > > - 原始邮件 - > > 发件人: "Jing Ge" > > 收件人: "dev" > > 发送时间: 星期三, 2023年 7 月 12日 上午 1:23:40 > > 主题: Re: [VOTE] Release 2.0 must-have work items > > > > agree with what Leonard said. There are actually more issues wrt the new > > Source and SinkV2[1] > > > > Speaking of must-have vs nice-to-have, I think it depends on the > priority. > > If removing them has higher priority, we should keep related tasks as > > must-have and make sure enough effort will be put to solve those issues > and > > therefore be able to remove those APIs. > > > > Best regards, > > Jing > > > > [1] https://lists.apache.org/thread/90qc9nrlzf0vbvg92klzp9ftxxc43nbk > > > > On Tue, Jul 11, 2023 at 10:26 AM Leonard Xu wrote: > > > > > Thanks Xintong for driving this great work! But I’ve to give my > > > -1(binding) here: > > > > > > -1 to mark "deprecat SourceFunction/SinkFunction/Sinkv1" item as must > to > > > have for release 2.0. > > > > > > I do a lot of connector work in the co
Re: [VOTE] Release 2.0 must-have work items
@Yuxia, We are aware of the issue that you mentioned. Actually, I don't think the DataStream API can cover everything in the DataSet API in exactly the same way, because the fundamental model, concepts and primitives of the two sets of APIs are completely different. Many of the DataSet APIs, especially those accessing the full data set at once, do not fit in the DataStream concepts at all. I think what's important is that users can achieve the same function, even if they may need to code in a different way. We have gone through all the existing DataSet APIs, and categorized them into 3 kinds: - APIs that are well supported by DataStream API as is. E.g., map, reduce on grouped dataset, etc. - APIs that can be achieved by DataStream API as is, but with a price (programming complexity, or computation efficiency). E.g., reduce on full dataset, sort partition, etc. Admittedly, there is room for improvement on these. We may keep improving these for the DataStream API, or we can concentrate on supporting them better in the new ProcessFunction API. Either way, I don't think we should block the retiring of DataSet API on them. - There are also a few APIs that cannot be supported by the DataStream API as is, unless users write their custom operators from the ground up. Only left/rightOuterJoin and combineGroup fall into this category. I think combinedGroup is probably not a problem, because this is more like a variant of reduceGroup that allows the framework to execute more efficiently. As for the outer joins, depending on how badly this is needed, it can be supported by emitting the non-joined entries upon triggering a window join. We are also planning to draft a guideline to help users migrating from DataSet to DataStream, which should demonstrate how users can achieve things like sort-partition with DataStream API. Last but not least, I'd like to point out that the decision to deprecate and eventually remove the DataSet API was approved in FLIP-131, and all the prerequisites mentioned in the FLIP have been completed. Best, Xintong [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158866741 On Wed, Jul 12, 2023 at 10:20 AM Jingsong Li wrote: > +1 to Leonard and Galen and Jing. > > About Source and Sink. > We're still missing quite a bit of work, including functionality, > including ease of use, including bug fixes, and I'm not sure we'll be > completely done by 2.0. > Until that's done, we won't be in a position to clean up the old APIs. > > Best, > Jingsong > > On Wed, Jul 12, 2023 at 9:41 AM yuxia wrote: > > > > Hi,Xintong. > > Sorry to disturb the voting. I just found an email[1] about DataSet API > from flink-user-zh channel. And I think it's not just a single case > according to my observation. > > > > Remove DataSet is a must have item in release-2.0. But as the user email > said, if we remove DataSet, how users can implement Sort/PartitionBy, etc > as they did with DataSet? > > Do we will also provide similar api in datastream or some other thing > before we remove DataSet? > > Btw, as far as I see, with regarding to replcaing DataSet with > Datastream, Datastream are missing many API. I think it may well take much > effort to fully cover the missing api. > > > > [1] https://lists.apache.org/thread/syjmt8f74gh8ok3z4lhgt95zl4dzn168 > > > > Best regards, > > Yuxia > > > > ----- 原始邮件 - > > 发件人: "Jing Ge" > > 收件人: "dev" > > 发送时间: 星期三, 2023年 7 月 12日 上午 1:23:40 > > 主题: Re: [VOTE] Release 2.0 must-have work items > > > > agree with what Leonard said. There are actually more issues wrt the new > > Source and SinkV2[1] > > > > Speaking of must-have vs nice-to-have, I think it depends on the > priority. > > If removing them has higher priority, we should keep related tasks as > > must-have and make sure enough effort will be put to solve those issues > and > > therefore be able to remove those APIs. > > > > Best regards, > > Jing > > > > [1] https://lists.apache.org/thread/90qc9nrlzf0vbvg92klzp9ftxxc43nbk > > > > On Tue, Jul 11, 2023 at 10:26 AM Leonard Xu wrote: > > > > > Thanks Xintong for driving this great work! But I’ve to give my > > > -1(binding) here: > > > > > > -1 to mark "deprecat SourceFunction/SinkFunction/Sinkv1" item as must > to > > > have for release 2.0. > > > > > > I do a lot of connector work in the community, and I have two insights > > > from past experience: > > > > > > 1. Many developers reported that it is very difficult to migrate from > > > SourceFunction to new Source [1]. The migration of existing conenctors > > > after deprecated SourceFu
Re: [VOTE] Release 2.0 must-have work items
+1 to Leonard and Galen and Jing. About Source and Sink. We're still missing quite a bit of work, including functionality, including ease of use, including bug fixes, and I'm not sure we'll be completely done by 2.0. Until that's done, we won't be in a position to clean up the old APIs. Best, Jingsong On Wed, Jul 12, 2023 at 9:41 AM yuxia wrote: > > Hi,Xintong. > Sorry to disturb the voting. I just found an email[1] about DataSet API from > flink-user-zh channel. And I think it's not just a single case according to > my observation. > > Remove DataSet is a must have item in release-2.0. But as the user email > said, if we remove DataSet, how users can implement Sort/PartitionBy, etc as > they did with DataSet? > Do we will also provide similar api in datastream or some other thing before > we remove DataSet? > Btw, as far as I see, with regarding to replcaing DataSet with Datastream, > Datastream are missing many API. I think it may well take much effort to > fully cover the missing api. > > [1] https://lists.apache.org/thread/syjmt8f74gh8ok3z4lhgt95zl4dzn168 > > Best regards, > Yuxia > > - 原始邮件 - > 发件人: "Jing Ge" > 收件人: "dev" > 发送时间: 星期三, 2023年 7 月 12日 上午 1:23:40 > 主题: Re: [VOTE] Release 2.0 must-have work items > > agree with what Leonard said. There are actually more issues wrt the new > Source and SinkV2[1] > > Speaking of must-have vs nice-to-have, I think it depends on the priority. > If removing them has higher priority, we should keep related tasks as > must-have and make sure enough effort will be put to solve those issues and > therefore be able to remove those APIs. > > Best regards, > Jing > > [1] https://lists.apache.org/thread/90qc9nrlzf0vbvg92klzp9ftxxc43nbk > > On Tue, Jul 11, 2023 at 10:26 AM Leonard Xu wrote: > > > Thanks Xintong for driving this great work! But I’ve to give my > > -1(binding) here: > > > > -1 to mark "deprecat SourceFunction/SinkFunction/Sinkv1" item as must to > > have for release 2.0. > > > > I do a lot of connector work in the community, and I have two insights > > from past experience: > > > > 1. Many developers reported that it is very difficult to migrate from > > SourceFunction to new Source [1]. The migration of existing conenctors > > after deprecated SourceFunction is very difficult. Some developers (Flavio > > Pompermaier) reported that they gave up the migration because it was too > > complicated. I believe it's not a few cases. This means that deprecating > > SourceFunction related interfaces require community contributors to reduce > > the migration cost before starting the migration work. > > > > 2. IIRC, the function of SinkV2 cannot currently cover SinkFunction as > > described in FLIP-287[2], it means the migration path after deprecate > > SinkFunction/Sinkv1 does not exist, thus we cannot mark the related > > interfaces of sinkfunction/sinkv1 as deprecated in 1.18. > > > > Based on these two cognitions, I think we should not mark these interfaces > > as must to have in 2.0. Maintaining the two sets of source/sink interfaces > > is not a concern for me, users can choose the interface to implement > > according to their energy and needs. > > > > Btw, some work items in 2.0 are marked as must to have, but no contributor > > has claimed them yet. I think this is a risk and hope the Release Managers > > could pay attention to it. > > > > Thank you all RMs for your work, sorry again for interrupting the vote > > > > Best, > > Leonard > > > > [1] https://lists.apache.org/thread/sqq26s9rorynr4vx4nhxz3fmmxpgtdqp > > [2] > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240880853 > > > > > On Jul 11, 2023, at 4:11 PM, Yuan Mei wrote: > > > > > > As a second thought, I think "Eager State Declaration" is probably not a > > > must-have. > > > > > > I was originally thinking it is a prerequisite for "state querying for > > > disaggregated state management". > > > > > > Since disaggregated state management itself is not a must-have, "Eager > > > State Declaration" is not as well. We can downgrade it to "nice to have" > > if > > > no objection. > > > > > > Best > > > > > > Yuan > > > > > > On Mon, Jul 10, 2023 at 7:02 PM Jing Ge > > wrote: > > > > > >> +1 > > >> > > >> On Mon, Jul 10, 2023 at 12:52 PM Yu Li wrote: > > >> > > >>> +1 (binding) > > >>&g
Re: [VOTE] Release 2.0 must-have work items
Hi,Xintong. Sorry to disturb the voting. I just found an email[1] about DataSet API from flink-user-zh channel. And I think it's not just a single case according to my observation. Remove DataSet is a must have item in release-2.0. But as the user email said, if we remove DataSet, how users can implement Sort/PartitionBy, etc as they did with DataSet? Do we will also provide similar api in datastream or some other thing before we remove DataSet? Btw, as far as I see, with regarding to replcaing DataSet with Datastream, Datastream are missing many API. I think it may well take much effort to fully cover the missing api. [1] https://lists.apache.org/thread/syjmt8f74gh8ok3z4lhgt95zl4dzn168 Best regards, Yuxia - 原始邮件 - 发件人: "Jing Ge" 收件人: "dev" 发送时间: 星期三, 2023年 7 月 12日 上午 1:23:40 主题: Re: [VOTE] Release 2.0 must-have work items agree with what Leonard said. There are actually more issues wrt the new Source and SinkV2[1] Speaking of must-have vs nice-to-have, I think it depends on the priority. If removing them has higher priority, we should keep related tasks as must-have and make sure enough effort will be put to solve those issues and therefore be able to remove those APIs. Best regards, Jing [1] https://lists.apache.org/thread/90qc9nrlzf0vbvg92klzp9ftxxc43nbk On Tue, Jul 11, 2023 at 10:26 AM Leonard Xu wrote: > Thanks Xintong for driving this great work! But I’ve to give my > -1(binding) here: > > -1 to mark "deprecat SourceFunction/SinkFunction/Sinkv1" item as must to > have for release 2.0. > > I do a lot of connector work in the community, and I have two insights > from past experience: > > 1. Many developers reported that it is very difficult to migrate from > SourceFunction to new Source [1]. The migration of existing conenctors > after deprecated SourceFunction is very difficult. Some developers (Flavio > Pompermaier) reported that they gave up the migration because it was too > complicated. I believe it's not a few cases. This means that deprecating > SourceFunction related interfaces require community contributors to reduce > the migration cost before starting the migration work. > > 2. IIRC, the function of SinkV2 cannot currently cover SinkFunction as > described in FLIP-287[2], it means the migration path after deprecate > SinkFunction/Sinkv1 does not exist, thus we cannot mark the related > interfaces of sinkfunction/sinkv1 as deprecated in 1.18. > > Based on these two cognitions, I think we should not mark these interfaces > as must to have in 2.0. Maintaining the two sets of source/sink interfaces > is not a concern for me, users can choose the interface to implement > according to their energy and needs. > > Btw, some work items in 2.0 are marked as must to have, but no contributor > has claimed them yet. I think this is a risk and hope the Release Managers > could pay attention to it. > > Thank you all RMs for your work, sorry again for interrupting the vote > > Best, > Leonard > > [1] https://lists.apache.org/thread/sqq26s9rorynr4vx4nhxz3fmmxpgtdqp > [2] > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240880853 > > > On Jul 11, 2023, at 4:11 PM, Yuan Mei wrote: > > > > As a second thought, I think "Eager State Declaration" is probably not a > > must-have. > > > > I was originally thinking it is a prerequisite for "state querying for > > disaggregated state management". > > > > Since disaggregated state management itself is not a must-have, "Eager > > State Declaration" is not as well. We can downgrade it to "nice to have" > if > > no objection. > > > > Best > > > > Yuan > > > > On Mon, Jul 10, 2023 at 7:02 PM Jing Ge > wrote: > > > >> +1 > >> > >> On Mon, Jul 10, 2023 at 12:52 PM Yu Li wrote: > >> > >>> +1 (binding) > >>> > >>> Thanks for driving this and great to see us moving forward. > >>> > >>> Best Regards, > >>> Yu > >>> > >>> > >>> On Mon, 10 Jul 2023 at 11:59, Feng Wang wrote: > >>> > >>>> +1 > >>>> Thanks for driving this, looking forward to the next stage of flink. > >>>> > >>>> On Fri, Jul 7, 2023 at 5:31 PM Xintong Song > >>> wrote: > >>>> > >>>>> Hi all, > >>>>> > >>>>> I'd like to start the VOTE for the must-have work items for release > >> 2.0 > >>>>> [1]. The corresponding discussion thread is [2]. > >>>>> > >>>>> Please note that once the vote is approved, any changes to the > >>> must-have > >>>>> items (adding / removing must-have items, changing the priority) > >>> requires > >>>>> another vote. Assigning contributors / reviewers, updating > >>> descriptions / > >>>>> progress, changes to nice-to-have items do not require another vote. > >>>>> > >>>>> The vote will be open until at least July 12, following the consensus > >>>>> voting process. Votes of PMC members are binding. > >>>>> > >>>>> Best, > >>>>> > >>>>> Xintong > >>>>> > >>>>> > >>>>> [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > >>>>> > >>>>> [2] https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 > >>>>> > >>>> > >>> > >> > >
Re: [VOTE] Release 2.0 must-have work items
agree with what Leonard said. There are actually more issues wrt the new Source and SinkV2[1] Speaking of must-have vs nice-to-have, I think it depends on the priority. If removing them has higher priority, we should keep related tasks as must-have and make sure enough effort will be put to solve those issues and therefore be able to remove those APIs. Best regards, Jing [1] https://lists.apache.org/thread/90qc9nrlzf0vbvg92klzp9ftxxc43nbk On Tue, Jul 11, 2023 at 10:26 AM Leonard Xu wrote: > Thanks Xintong for driving this great work! But I’ve to give my > -1(binding) here: > > -1 to mark "deprecat SourceFunction/SinkFunction/Sinkv1" item as must to > have for release 2.0. > > I do a lot of connector work in the community, and I have two insights > from past experience: > > 1. Many developers reported that it is very difficult to migrate from > SourceFunction to new Source [1]. The migration of existing conenctors > after deprecated SourceFunction is very difficult. Some developers (Flavio > Pompermaier) reported that they gave up the migration because it was too > complicated. I believe it's not a few cases. This means that deprecating > SourceFunction related interfaces require community contributors to reduce > the migration cost before starting the migration work. > > 2. IIRC, the function of SinkV2 cannot currently cover SinkFunction as > described in FLIP-287[2], it means the migration path after deprecate > SinkFunction/Sinkv1 does not exist, thus we cannot mark the related > interfaces of sinkfunction/sinkv1 as deprecated in 1.18. > > Based on these two cognitions, I think we should not mark these interfaces > as must to have in 2.0. Maintaining the two sets of source/sink interfaces > is not a concern for me, users can choose the interface to implement > according to their energy and needs. > > Btw, some work items in 2.0 are marked as must to have, but no contributor > has claimed them yet. I think this is a risk and hope the Release Managers > could pay attention to it. > > Thank you all RMs for your work, sorry again for interrupting the vote > > Best, > Leonard > > [1] https://lists.apache.org/thread/sqq26s9rorynr4vx4nhxz3fmmxpgtdqp > [2] > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240880853 > > > On Jul 11, 2023, at 4:11 PM, Yuan Mei wrote: > > > > As a second thought, I think "Eager State Declaration" is probably not a > > must-have. > > > > I was originally thinking it is a prerequisite for "state querying for > > disaggregated state management". > > > > Since disaggregated state management itself is not a must-have, "Eager > > State Declaration" is not as well. We can downgrade it to "nice to have" > if > > no objection. > > > > Best > > > > Yuan > > > > On Mon, Jul 10, 2023 at 7:02 PM Jing Ge > wrote: > > > >> +1 > >> > >> On Mon, Jul 10, 2023 at 12:52 PM Yu Li wrote: > >> > >>> +1 (binding) > >>> > >>> Thanks for driving this and great to see us moving forward. > >>> > >>> Best Regards, > >>> Yu > >>> > >>> > >>> On Mon, 10 Jul 2023 at 11:59, Feng Wang wrote: > >>> > +1 > Thanks for driving this, looking forward to the next stage of flink. > > On Fri, Jul 7, 2023 at 5:31 PM Xintong Song > >>> wrote: > > > Hi all, > > > > I'd like to start the VOTE for the must-have work items for release > >> 2.0 > > [1]. The corresponding discussion thread is [2]. > > > > Please note that once the vote is approved, any changes to the > >>> must-have > > items (adding / removing must-have items, changing the priority) > >>> requires > > another vote. Assigning contributors / reviewers, updating > >>> descriptions / > > progress, changes to nice-to-have items do not require another vote. > > > > The vote will be open until at least July 12, following the consensus > > voting process. Votes of PMC members are binding. > > > > Best, > > > > Xintong > > > > > > [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > > > [2] https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 > > > > >>> > >> > >
Re: [VOTE] Release 2.0 must-have work items
Hi Galen, We were aware of the issue and are working on it. StreamingFileSink is a SinkFunction that could not be removed yes as mentioned previously. You can find SinkV1 at [1] Best regards, Jing [1] https://github.com/apache/flink/blob/4cf2124d71a8dd0595e40f07c2dbcc4c85883b82/flink-core/src/main/java/org/apache/flink/api/connector/sink/Sink.java#L55 On Tue, Jul 11, 2023 at 1:59 PM Galen Warren wrote: > Regarding SinkV1 vs. SinkV2: Is StreamingFileSink a SinkV1-related > interface that is proposed to be removed? In a separate thread, it was > discussed how it's important not to remove StreamingFileSink as long as > this critical issue with SinkV2 is still outstanding -- > https://issues.apache.org/jira/plugins/servlet/mobile#issue/FLINK-30238 -- > because of the prospect of data loss when stopping and restarting jobs with > savepoints. > > Thanks, > Galen > > On Tue, Jul 11, 2023 at 7:47 AM Leonard Xu wrote: > > > Hi, Xintong > > > > > Could you please clarify what exact changes you are proposing to make > on > > > the existing list? > > > - Are you suggesting removing the item "Remove deprecated APIs - > > > SourceFunction / SinkFunction / SinkV1", or are you suggesting > > downgrading > > > it as nice-to-have? > > > > I prefer to remove the item as we cannot deprecate SourceFunction / > > SinkFunction related interfaces in 1.18, thus he 2.0 version would not > > satisfy two minor versions condition and would not remove them as well. > > > > > - You said SinkV2 cannot cover SinkFunction. Then how about SinkV1? Is > it > > > covered by SinkV2? Should it be removed or preserved? > > > > SinkV2 related interfaces covers SinkV1 related interfaces well, and > > SinkV1 related interfaces have been deprecated, I think they can be > removed > > in 2.0 safely. > > > > In a word, my proposal is replace must have item "Remove deprecated APIs > - > > SourceFunction / SinkFunction / SinkV1" with must have item "Remove > > deprecated APIs SinkV1" . > > > > Best, > > Leonard > > > > > > > > > > > > > > > > > > > > Best, > > > > > > Xintong > > > > > > > > > > > > On Tue, Jul 11, 2023 at 4:26 PM Leonard Xu wrote: > > > > > >> Thanks Xintong for driving this great work! But I’ve to give my > > >> -1(binding) here: > > >> > > >> -1 to mark "deprecat SourceFunction/SinkFunction/Sinkv1" item as must > to > > >> have for release 2.0. > > >> > > >> I do a lot of connector work in the community, and I have two insights > > >> from past experience: > > >> > > >> 1. Many developers reported that it is very difficult to migrate from > > >> SourceFunction to new Source [1]. The migration of existing conenctors > > >> after deprecated SourceFunction is very difficult. Some developers > > (Flavio > > >> Pompermaier) reported that they gave up the migration because it was > too > > >> complicated. I believe it's not a few cases. This means that > deprecating > > >> SourceFunction related interfaces require community contributors to > > reduce > > >> the migration cost before starting the migration work. > > >> > > >> 2. IIRC, the function of SinkV2 cannot currently cover SinkFunction as > > >> described in FLIP-287[2], it means the migration path after deprecate > > >> SinkFunction/Sinkv1 does not exist, thus we cannot mark the related > > >> interfaces of sinkfunction/sinkv1 as deprecated in 1.18. > > >> > > >> Based on these two cognitions, I think we should not mark these > > interfaces > > >> as must to have in 2.0. Maintaining the two sets of source/sink > > interfaces > > >> is not a concern for me, users can choose the interface to implement > > >> according to their energy and needs. > > >> > > >> Btw, some work items in 2.0 are marked as must to have, but no > > contributor > > >> has claimed them yet. I think this is a risk and hope the Release > > Managers > > >> could pay attention to it. > > >> > > >> Thank you all RMs for your work, sorry again for interrupting the vote > > >> > > >> Best, > > >> Leonard > > >> > > >> [1] https://lists.apache.org/thread/sqq26s9rorynr4vx4nhxz3fmmxpgtdqp > > >> [2] > > >> > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240880853 > > >> > > >>> On Jul 11, 2023, at 4:11 PM, Yuan Mei > wrote: > > >>> > > >>> As a second thought, I think "Eager State Declaration" is probably > not > > a > > >>> must-have. > > >>> > > >>> I was originally thinking it is a prerequisite for "state querying > for > > >>> disaggregated state management". > > >>> > > >>> Since disaggregated state management itself is not a must-have, > "Eager > > >>> State Declaration" is not as well. We can downgrade it to "nice to > > have" > > >> if > > >>> no objection. > > >>> > > >>> Best > > >>> > > >>> Yuan > > >>> > > >>> On Mon, Jul 10, 2023 at 7:02 PM Jing Ge > > >> wrote: > > >>> > > +1 > > > > On Mon, Jul 10, 2023 at 12:52 PM Yu Li wrote: > > > > > +1 (binding) > > > > > > Thanks for driving this and great to see us moving forward. > > > > >
Re: [VOTE] Release 2.0 must-have work items
Regarding SinkV1 vs. SinkV2: Is StreamingFileSink a SinkV1-related interface that is proposed to be removed? In a separate thread, it was discussed how it's important not to remove StreamingFileSink as long as this critical issue with SinkV2 is still outstanding -- https://issues.apache.org/jira/plugins/servlet/mobile#issue/FLINK-30238 -- because of the prospect of data loss when stopping and restarting jobs with savepoints. Thanks, Galen On Tue, Jul 11, 2023 at 7:47 AM Leonard Xu wrote: > Hi, Xintong > > > Could you please clarify what exact changes you are proposing to make on > > the existing list? > > - Are you suggesting removing the item "Remove deprecated APIs - > > SourceFunction / SinkFunction / SinkV1", or are you suggesting > downgrading > > it as nice-to-have? > > I prefer to remove the item as we cannot deprecate SourceFunction / > SinkFunction related interfaces in 1.18, thus he 2.0 version would not > satisfy two minor versions condition and would not remove them as well. > > > - You said SinkV2 cannot cover SinkFunction. Then how about SinkV1? Is it > > covered by SinkV2? Should it be removed or preserved? > > SinkV2 related interfaces covers SinkV1 related interfaces well, and > SinkV1 related interfaces have been deprecated, I think they can be removed > in 2.0 safely. > > In a word, my proposal is replace must have item "Remove deprecated APIs - > SourceFunction / SinkFunction / SinkV1" with must have item "Remove > deprecated APIs SinkV1" . > > Best, > Leonard > > > > > > > > > > > Best, > > > > Xintong > > > > > > > > On Tue, Jul 11, 2023 at 4:26 PM Leonard Xu wrote: > > > >> Thanks Xintong for driving this great work! But I’ve to give my > >> -1(binding) here: > >> > >> -1 to mark "deprecat SourceFunction/SinkFunction/Sinkv1" item as must to > >> have for release 2.0. > >> > >> I do a lot of connector work in the community, and I have two insights > >> from past experience: > >> > >> 1. Many developers reported that it is very difficult to migrate from > >> SourceFunction to new Source [1]. The migration of existing conenctors > >> after deprecated SourceFunction is very difficult. Some developers > (Flavio > >> Pompermaier) reported that they gave up the migration because it was too > >> complicated. I believe it's not a few cases. This means that deprecating > >> SourceFunction related interfaces require community contributors to > reduce > >> the migration cost before starting the migration work. > >> > >> 2. IIRC, the function of SinkV2 cannot currently cover SinkFunction as > >> described in FLIP-287[2], it means the migration path after deprecate > >> SinkFunction/Sinkv1 does not exist, thus we cannot mark the related > >> interfaces of sinkfunction/sinkv1 as deprecated in 1.18. > >> > >> Based on these two cognitions, I think we should not mark these > interfaces > >> as must to have in 2.0. Maintaining the two sets of source/sink > interfaces > >> is not a concern for me, users can choose the interface to implement > >> according to their energy and needs. > >> > >> Btw, some work items in 2.0 are marked as must to have, but no > contributor > >> has claimed them yet. I think this is a risk and hope the Release > Managers > >> could pay attention to it. > >> > >> Thank you all RMs for your work, sorry again for interrupting the vote > >> > >> Best, > >> Leonard > >> > >> [1] https://lists.apache.org/thread/sqq26s9rorynr4vx4nhxz3fmmxpgtdqp > >> [2] > >> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240880853 > >> > >>> On Jul 11, 2023, at 4:11 PM, Yuan Mei wrote: > >>> > >>> As a second thought, I think "Eager State Declaration" is probably not > a > >>> must-have. > >>> > >>> I was originally thinking it is a prerequisite for "state querying for > >>> disaggregated state management". > >>> > >>> Since disaggregated state management itself is not a must-have, "Eager > >>> State Declaration" is not as well. We can downgrade it to "nice to > have" > >> if > >>> no objection. > >>> > >>> Best > >>> > >>> Yuan > >>> > >>> On Mon, Jul 10, 2023 at 7:02 PM Jing Ge > >> wrote: > >>> > +1 > > On Mon, Jul 10, 2023 at 12:52 PM Yu Li wrote: > > > +1 (binding) > > > > Thanks for driving this and great to see us moving forward. > > > > Best Regards, > > Yu > > > > > > On Mon, 10 Jul 2023 at 11:59, Feng Wang > wrote: > > > >> +1 > >> Thanks for driving this, looking forward to the next stage of flink. > >> > >> On Fri, Jul 7, 2023 at 5:31 PM Xintong Song > > wrote: > >> > >>> Hi all, > >>> > >>> I'd like to start the VOTE for the must-have work items for release > 2.0 > >>> [1]. The corresponding discussion thread is [2]. > >>> > >>> Please note that once the vote is approved, any changes to the > > must-have > >>> items (adding / removing must-have items, changing the priority) > > requires > >>> another vote. Assigning
Re: [VOTE] Release 2.0 must-have work items
Hi, Xintong > Could you please clarify what exact changes you are proposing to make on > the existing list? > - Are you suggesting removing the item "Remove deprecated APIs - > SourceFunction / SinkFunction / SinkV1", or are you suggesting downgrading > it as nice-to-have? I prefer to remove the item as we cannot deprecate SourceFunction / SinkFunction related interfaces in 1.18, thus he 2.0 version would not satisfy two minor versions condition and would not remove them as well. > - You said SinkV2 cannot cover SinkFunction. Then how about SinkV1? Is it > covered by SinkV2? Should it be removed or preserved? SinkV2 related interfaces covers SinkV1 related interfaces well, and SinkV1 related interfaces have been deprecated, I think they can be removed in 2.0 safely. In a word, my proposal is replace must have item "Remove deprecated APIs - SourceFunction / SinkFunction / SinkV1" with must have item "Remove deprecated APIs SinkV1" . Best, Leonard > > Best, > > Xintong > > > > On Tue, Jul 11, 2023 at 4:26 PM Leonard Xu wrote: > >> Thanks Xintong for driving this great work! But I’ve to give my >> -1(binding) here: >> >> -1 to mark "deprecat SourceFunction/SinkFunction/Sinkv1" item as must to >> have for release 2.0. >> >> I do a lot of connector work in the community, and I have two insights >> from past experience: >> >> 1. Many developers reported that it is very difficult to migrate from >> SourceFunction to new Source [1]. The migration of existing conenctors >> after deprecated SourceFunction is very difficult. Some developers (Flavio >> Pompermaier) reported that they gave up the migration because it was too >> complicated. I believe it's not a few cases. This means that deprecating >> SourceFunction related interfaces require community contributors to reduce >> the migration cost before starting the migration work. >> >> 2. IIRC, the function of SinkV2 cannot currently cover SinkFunction as >> described in FLIP-287[2], it means the migration path after deprecate >> SinkFunction/Sinkv1 does not exist, thus we cannot mark the related >> interfaces of sinkfunction/sinkv1 as deprecated in 1.18. >> >> Based on these two cognitions, I think we should not mark these interfaces >> as must to have in 2.0. Maintaining the two sets of source/sink interfaces >> is not a concern for me, users can choose the interface to implement >> according to their energy and needs. >> >> Btw, some work items in 2.0 are marked as must to have, but no contributor >> has claimed them yet. I think this is a risk and hope the Release Managers >> could pay attention to it. >> >> Thank you all RMs for your work, sorry again for interrupting the vote >> >> Best, >> Leonard >> >> [1] https://lists.apache.org/thread/sqq26s9rorynr4vx4nhxz3fmmxpgtdqp >> [2] >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240880853 >> >>> On Jul 11, 2023, at 4:11 PM, Yuan Mei wrote: >>> >>> As a second thought, I think "Eager State Declaration" is probably not a >>> must-have. >>> >>> I was originally thinking it is a prerequisite for "state querying for >>> disaggregated state management". >>> >>> Since disaggregated state management itself is not a must-have, "Eager >>> State Declaration" is not as well. We can downgrade it to "nice to have" >> if >>> no objection. >>> >>> Best >>> >>> Yuan >>> >>> On Mon, Jul 10, 2023 at 7:02 PM Jing Ge >> wrote: >>> +1 On Mon, Jul 10, 2023 at 12:52 PM Yu Li wrote: > +1 (binding) > > Thanks for driving this and great to see us moving forward. > > Best Regards, > Yu > > > On Mon, 10 Jul 2023 at 11:59, Feng Wang wrote: > >> +1 >> Thanks for driving this, looking forward to the next stage of flink. >> >> On Fri, Jul 7, 2023 at 5:31 PM Xintong Song > wrote: >> >>> Hi all, >>> >>> I'd like to start the VOTE for the must-have work items for release 2.0 >>> [1]. The corresponding discussion thread is [2]. >>> >>> Please note that once the vote is approved, any changes to the > must-have >>> items (adding / removing must-have items, changing the priority) > requires >>> another vote. Assigning contributors / reviewers, updating > descriptions / >>> progress, changes to nice-to-have items do not require another vote. >>> >>> The vote will be open until at least July 12, following the consensus >>> voting process. Votes of PMC members are binding. >>> >>> Best, >>> >>> Xintong >>> >>> >>> [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release >>> >>> [2] https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 >>> >> > >> >>
Re: [VOTE] Release 2.0 must-have work items
Thanks for the inputs, Yuan and Leonard. I'm canceling this vote, w.r.t. the objections and proposed changes. Meantime, please feel free to raise other concerns and proposed changes in this thread, before we call for another vote. @Leonard, Could you please clarify what exact changes you are proposing to make on the existing list? - Are you suggesting removing the item "Remove deprecated APIs - SourceFunction / SinkFunction / SinkV1", or are you suggesting downgrading it as nice-to-have? - You said SinkV2 cannot cover SinkFunction. Then how about SinkV1? Is it covered by SinkV2? Should it be removed or preserved? Best, Xintong On Tue, Jul 11, 2023 at 4:26 PM Leonard Xu wrote: > Thanks Xintong for driving this great work! But I’ve to give my > -1(binding) here: > > -1 to mark "deprecat SourceFunction/SinkFunction/Sinkv1" item as must to > have for release 2.0. > > I do a lot of connector work in the community, and I have two insights > from past experience: > > 1. Many developers reported that it is very difficult to migrate from > SourceFunction to new Source [1]. The migration of existing conenctors > after deprecated SourceFunction is very difficult. Some developers (Flavio > Pompermaier) reported that they gave up the migration because it was too > complicated. I believe it's not a few cases. This means that deprecating > SourceFunction related interfaces require community contributors to reduce > the migration cost before starting the migration work. > > 2. IIRC, the function of SinkV2 cannot currently cover SinkFunction as > described in FLIP-287[2], it means the migration path after deprecate > SinkFunction/Sinkv1 does not exist, thus we cannot mark the related > interfaces of sinkfunction/sinkv1 as deprecated in 1.18. > > Based on these two cognitions, I think we should not mark these interfaces > as must to have in 2.0. Maintaining the two sets of source/sink interfaces > is not a concern for me, users can choose the interface to implement > according to their energy and needs. > > Btw, some work items in 2.0 are marked as must to have, but no contributor > has claimed them yet. I think this is a risk and hope the Release Managers > could pay attention to it. > > Thank you all RMs for your work, sorry again for interrupting the vote > > Best, > Leonard > > [1] https://lists.apache.org/thread/sqq26s9rorynr4vx4nhxz3fmmxpgtdqp > [2] > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240880853 > > > On Jul 11, 2023, at 4:11 PM, Yuan Mei wrote: > > > > As a second thought, I think "Eager State Declaration" is probably not a > > must-have. > > > > I was originally thinking it is a prerequisite for "state querying for > > disaggregated state management". > > > > Since disaggregated state management itself is not a must-have, "Eager > > State Declaration" is not as well. We can downgrade it to "nice to have" > if > > no objection. > > > > Best > > > > Yuan > > > > On Mon, Jul 10, 2023 at 7:02 PM Jing Ge > wrote: > > > >> +1 > >> > >> On Mon, Jul 10, 2023 at 12:52 PM Yu Li wrote: > >> > >>> +1 (binding) > >>> > >>> Thanks for driving this and great to see us moving forward. > >>> > >>> Best Regards, > >>> Yu > >>> > >>> > >>> On Mon, 10 Jul 2023 at 11:59, Feng Wang wrote: > >>> > +1 > Thanks for driving this, looking forward to the next stage of flink. > > On Fri, Jul 7, 2023 at 5:31 PM Xintong Song > >>> wrote: > > > Hi all, > > > > I'd like to start the VOTE for the must-have work items for release > >> 2.0 > > [1]. The corresponding discussion thread is [2]. > > > > Please note that once the vote is approved, any changes to the > >>> must-have > > items (adding / removing must-have items, changing the priority) > >>> requires > > another vote. Assigning contributors / reviewers, updating > >>> descriptions / > > progress, changes to nice-to-have items do not require another vote. > > > > The vote will be open until at least July 12, following the consensus > > voting process. Votes of PMC members are binding. > > > > Best, > > > > Xintong > > > > > > [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > > > [2] https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 > > > > >>> > >> > >
Re: [VOTE] Release 2.0 must-have work items
Thanks Xintong for driving this great work! But I’ve to give my -1(binding) here: -1 to mark "deprecat SourceFunction/SinkFunction/Sinkv1" item as must to have for release 2.0. I do a lot of connector work in the community, and I have two insights from past experience: 1. Many developers reported that it is very difficult to migrate from SourceFunction to new Source [1]. The migration of existing conenctors after deprecated SourceFunction is very difficult. Some developers (Flavio Pompermaier) reported that they gave up the migration because it was too complicated. I believe it's not a few cases. This means that deprecating SourceFunction related interfaces require community contributors to reduce the migration cost before starting the migration work. 2. IIRC, the function of SinkV2 cannot currently cover SinkFunction as described in FLIP-287[2], it means the migration path after deprecate SinkFunction/Sinkv1 does not exist, thus we cannot mark the related interfaces of sinkfunction/sinkv1 as deprecated in 1.18. Based on these two cognitions, I think we should not mark these interfaces as must to have in 2.0. Maintaining the two sets of source/sink interfaces is not a concern for me, users can choose the interface to implement according to their energy and needs. Btw, some work items in 2.0 are marked as must to have, but no contributor has claimed them yet. I think this is a risk and hope the Release Managers could pay attention to it. Thank you all RMs for your work, sorry again for interrupting the vote Best, Leonard [1] https://lists.apache.org/thread/sqq26s9rorynr4vx4nhxz3fmmxpgtdqp [2] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240880853 > On Jul 11, 2023, at 4:11 PM, Yuan Mei wrote: > > As a second thought, I think "Eager State Declaration" is probably not a > must-have. > > I was originally thinking it is a prerequisite for "state querying for > disaggregated state management". > > Since disaggregated state management itself is not a must-have, "Eager > State Declaration" is not as well. We can downgrade it to "nice to have" if > no objection. > > Best > > Yuan > > On Mon, Jul 10, 2023 at 7:02 PM Jing Ge wrote: > >> +1 >> >> On Mon, Jul 10, 2023 at 12:52 PM Yu Li wrote: >> >>> +1 (binding) >>> >>> Thanks for driving this and great to see us moving forward. >>> >>> Best Regards, >>> Yu >>> >>> >>> On Mon, 10 Jul 2023 at 11:59, Feng Wang wrote: >>> +1 Thanks for driving this, looking forward to the next stage of flink. On Fri, Jul 7, 2023 at 5:31 PM Xintong Song >>> wrote: > Hi all, > > I'd like to start the VOTE for the must-have work items for release >> 2.0 > [1]. The corresponding discussion thread is [2]. > > Please note that once the vote is approved, any changes to the >>> must-have > items (adding / removing must-have items, changing the priority) >>> requires > another vote. Assigning contributors / reviewers, updating >>> descriptions / > progress, changes to nice-to-have items do not require another vote. > > The vote will be open until at least July 12, following the consensus > voting process. Votes of PMC members are binding. > > Best, > > Xintong > > > [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > [2] https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 > >>> >>
Re: [VOTE] Release 2.0 must-have work items
As a second thought, I think "Eager State Declaration" is probably not a must-have. I was originally thinking it is a prerequisite for "state querying for disaggregated state management". Since disaggregated state management itself is not a must-have, "Eager State Declaration" is not as well. We can downgrade it to "nice to have" if no objection. Best Yuan On Mon, Jul 10, 2023 at 7:02 PM Jing Ge wrote: > +1 > > On Mon, Jul 10, 2023 at 12:52 PM Yu Li wrote: > > > +1 (binding) > > > > Thanks for driving this and great to see us moving forward. > > > > Best Regards, > > Yu > > > > > > On Mon, 10 Jul 2023 at 11:59, Feng Wang wrote: > > > > > +1 > > > Thanks for driving this, looking forward to the next stage of flink. > > > > > > On Fri, Jul 7, 2023 at 5:31 PM Xintong Song > > wrote: > > > > > > > Hi all, > > > > > > > > I'd like to start the VOTE for the must-have work items for release > 2.0 > > > > [1]. The corresponding discussion thread is [2]. > > > > > > > > Please note that once the vote is approved, any changes to the > > must-have > > > > items (adding / removing must-have items, changing the priority) > > requires > > > > another vote. Assigning contributors / reviewers, updating > > descriptions / > > > > progress, changes to nice-to-have items do not require another vote. > > > > > > > > The vote will be open until at least July 12, following the consensus > > > > voting process. Votes of PMC members are binding. > > > > > > > > Best, > > > > > > > > Xintong > > > > > > > > > > > > [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > > > > > > > [2] https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 > > > > > > > > > >
Re: [VOTE] Release 2.0 must-have work items
+1 On Mon, Jul 10, 2023 at 12:52 PM Yu Li wrote: > +1 (binding) > > Thanks for driving this and great to see us moving forward. > > Best Regards, > Yu > > > On Mon, 10 Jul 2023 at 11:59, Feng Wang wrote: > > > +1 > > Thanks for driving this, looking forward to the next stage of flink. > > > > On Fri, Jul 7, 2023 at 5:31 PM Xintong Song > wrote: > > > > > Hi all, > > > > > > I'd like to start the VOTE for the must-have work items for release 2.0 > > > [1]. The corresponding discussion thread is [2]. > > > > > > Please note that once the vote is approved, any changes to the > must-have > > > items (adding / removing must-have items, changing the priority) > requires > > > another vote. Assigning contributors / reviewers, updating > descriptions / > > > progress, changes to nice-to-have items do not require another vote. > > > > > > The vote will be open until at least July 12, following the consensus > > > voting process. Votes of PMC members are binding. > > > > > > Best, > > > > > > Xintong > > > > > > > > > [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > > > > > [2] https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 > > > > > >
Re: [VOTE] Release 2.0 must-have work items
+1 (binding) Thanks for driving this and great to see us moving forward. Best Regards, Yu On Mon, 10 Jul 2023 at 11:59, Feng Wang wrote: > +1 > Thanks for driving this, looking forward to the next stage of flink. > > On Fri, Jul 7, 2023 at 5:31 PM Xintong Song wrote: > > > Hi all, > > > > I'd like to start the VOTE for the must-have work items for release 2.0 > > [1]. The corresponding discussion thread is [2]. > > > > Please note that once the vote is approved, any changes to the must-have > > items (adding / removing must-have items, changing the priority) requires > > another vote. Assigning contributors / reviewers, updating descriptions / > > progress, changes to nice-to-have items do not require another vote. > > > > The vote will be open until at least July 12, following the consensus > > voting process. Votes of PMC members are binding. > > > > Best, > > > > Xintong > > > > > > [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > > > [2] https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 > > >
Re: [VOTE] Release 2.0 must-have work items
+1 Thanks for driving this, looking forward to the next stage of flink. On Fri, Jul 7, 2023 at 5:31 PM Xintong Song wrote: > Hi all, > > I'd like to start the VOTE for the must-have work items for release 2.0 > [1]. The corresponding discussion thread is [2]. > > Please note that once the vote is approved, any changes to the must-have > items (adding / removing must-have items, changing the priority) requires > another vote. Assigning contributors / reviewers, updating descriptions / > progress, changes to nice-to-have items do not require another vote. > > The vote will be open until at least July 12, following the consensus > voting process. Votes of PMC members are binding. > > Best, > > Xintong > > > [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > [2] https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 >
Re: [VOTE] Release 2.0 must-have work items
+1 On Mon, Jul 10, 2023 at 10:46 AM Yuan Mei wrote: > > +1 (binding) > > Thanks for driving this! > > Best > Yuan > > On Mon, Jul 10, 2023 at 10:26 AM Jark Wu wrote: > > > +1 (binding) > > > > Thanks for driving this. Looking forward to starting the 2.0 works. > > > > Best, > > Jark > > > > On Fri, 7 Jul 2023 at 17:31, Xintong Song wrote: > > > > > Hi all, > > > > > > I'd like to start the VOTE for the must-have work items for release 2.0 > > > [1]. The corresponding discussion thread is [2]. > > > > > > Please note that once the vote is approved, any changes to the must-have > > > items (adding / removing must-have items, changing the priority) requires > > > another vote. Assigning contributors / reviewers, updating descriptions / > > > progress, changes to nice-to-have items do not require another vote. > > > > > > The vote will be open until at least July 12, following the consensus > > > voting process. Votes of PMC members are binding. > > > > > > Best, > > > > > > Xintong > > > > > > > > > [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > > > > > [2] https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 > > > > >
Re: [VOTE] Release 2.0 must-have work items
+1 (binding) Thanks for driving this! Best Yuan On Mon, Jul 10, 2023 at 10:26 AM Jark Wu wrote: > +1 (binding) > > Thanks for driving this. Looking forward to starting the 2.0 works. > > Best, > Jark > > On Fri, 7 Jul 2023 at 17:31, Xintong Song wrote: > > > Hi all, > > > > I'd like to start the VOTE for the must-have work items for release 2.0 > > [1]. The corresponding discussion thread is [2]. > > > > Please note that once the vote is approved, any changes to the must-have > > items (adding / removing must-have items, changing the priority) requires > > another vote. Assigning contributors / reviewers, updating descriptions / > > progress, changes to nice-to-have items do not require another vote. > > > > The vote will be open until at least July 12, following the consensus > > voting process. Votes of PMC members are binding. > > > > Best, > > > > Xintong > > > > > > [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > > > [2] https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 > > >
Re: [VOTE] Release 2.0 must-have work items
+1 (binding) Thanks for driving this. Looking forward to starting the 2.0 works. Best, Jark On Fri, 7 Jul 2023 at 17:31, Xintong Song wrote: > Hi all, > > I'd like to start the VOTE for the must-have work items for release 2.0 > [1]. The corresponding discussion thread is [2]. > > Please note that once the vote is approved, any changes to the must-have > items (adding / removing must-have items, changing the priority) requires > another vote. Assigning contributors / reviewers, updating descriptions / > progress, changes to nice-to-have items do not require another vote. > > The vote will be open until at least July 12, following the consensus > voting process. Votes of PMC members are binding. > > Best, > > Xintong > > > [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > [2] https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 >