+1 Thanks, Stefan
> On 5. Sep 2023, at 10:45, David Morávek <d...@apache.org> wrote: > > +1 since there is an alternative, more complete implementation available > > Best, > D. > > On Sat, Sep 2, 2023 at 12:07 AM David Anderson <dander...@apache.org > <mailto:dander...@apache.org>> wrote: > >> +1 >> >> Keeping the legacy implementation in place is confusing and encourages >> adoption of something that really shouldn't be used. >> >> Thanks for driving this, >> David >> >> On Fri, Sep 1, 2023 at 8:45 AM Jing Ge <j...@ververica.com.invalid> wrote: >>> >>> Hi Wencong, >>> >>> Thanks for your clarification! +1 >>> >>> Best regards, >>> Jing >>> >>> On Fri, Sep 1, 2023 at 12:36 PM Wencong Liu <liuwencle...@163.com> >> wrote: >>> >>>> Hi Jing, >>>> >>>> >>>> Thanks for your reply！ >>>> >>>> >>>>> Or the "independent module extraction" mentioned in the FLIP does >> mean an >>>> independent module in Flink? >>>> >>>> >>>> Yes. If there are submodules in Flink repository needs the iteration >>>> (currently not), >>>> we could consider extracting them to a new submodule of Flink. >>>> >>>> >>>>> users will have to add one more dependency of Flink ML. If iteration >> is >>>> the >>>> only feature they need, it will look a little bit weird. >>>> >>>> >>>> If users only need to execute iteration jobs, they can simply remove >> the >>>> Flink >>>> dependency and add the necessary dependencies related to Flink ML. >>>> However, >>>> they can still utilize the DataStream API as it is also a dependency of >>>> Flink ML. >>>> >>>> >>>> Keeping an iteration submodule in Flink repository and make Flink ML >>>> depends it >>>> is also another solution. But the current implementation of Iteration >> in >>>> DataStream >>>> should be removed definitely due to its Incompleteness. >>>> >>>> >>>> The placement of the Iteration API in the repository is a topic that >> has >>>> multiple >>>> potential solutions. WDYT? >>>> >>>> >>>> Best, >>>> Wencong >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> At 2023-09-01 17:59:34, "Jing Ge" <j...@ververica.com.INVALID> wrote: >>>>> Hi Wencong, >>>>> >>>>> Thanks for the proposal! >>>>> >>>>> "The Iteration API in DataStream is planned be deprecated in Flink >> 1.19 >>>> and >>>>> then finally removed in Flink 2.0. For the users that rely on the >>>> Iteration >>>>> API in DataStream, they will have to migrate to Flink ML." >>>>> - Does it make sense to migrate the iteration module into Flink >> directly? >>>>> Or the "independent module extraction" mentioned in the FLIP does >> mean an >>>>> independent module in Flink? Since the iteration will be removed in >> Flink, >>>>> users will have to add one more dependency of Flink ML. If iteration >> is >>>> the >>>>> only feature they need, it will look a little bit weird. >>>>> >>>>> >>>>> Best regards, >>>>> Jing >>>>> >>>>> On Fri, Sep 1, 2023 at 11:05 AM weijie guo <guoweijieres...@gmail.com >>> >>>>> wrote: >>>>> >>>>>> Thanks, +1 for this. >>>>>> >>>>>> Best regards, >>>>>> >>>>>> Weijie >>>>>> >>>>>> >>>>>> Yangze Guo <karma...@gmail.com> 于2023年9月1日周五 14:29写道： >>>>>> >>>>>>> +1 >>>>>>> >>>>>>> Thanks for driving this. >>>>>>> >>>>>>> Best, >>>>>>> Yangze Guo >>>>>>> >>>>>>> On Fri, Sep 1, 2023 at 2:00 PM Xintong Song < >> tonysong...@gmail.com> >>>>>> wrote: >>>>>>>> >>>>>>>> +1 >>>>>>>> >>>>>>>> Best, >>>>>>>> >>>>>>>> Xintong >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Sep 1, 2023 at 1:11 PM Dong Lin <lindon...@gmail.com> >>>> wrote: >>>>>>>> >>>>>>>>> Thanks Wencong for initiating the discussion. >>>>>>>>> >>>>>>>>> +1 for the proposal. >>>>>>>>> >>>>>>>>> On Fri, Sep 1, 2023 at 12:00 PM Wencong Liu < >> liuwencle...@163.com >>>>> >>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi devs, >>>>>>>>>> >>>>>>>>>> I would like to start a discussion on FLIP-357: Deprecate >>>> Iteration >>>>>>> API >>>>>>>>> of >>>>>>>>>> DataStream [1]. >>>>>>>>>> >>>>>>>>>> Currently, the Iteration API of DataStream is incomplete. >> For >>>>>>> instance, >>>>>>>>> it >>>>>>>>>> lacks support >>>>>>>>>> for iteration in sync mode and exactly once semantics. >>>>>> Additionally, >>>>>>> it >>>>>>>>>> does not offer the >>>>>>>>>> ability to set iteration termination conditions. As a >> result, >>>> it's >>>>>>> hard >>>>>>>>>> for developers to >>>>>>>>>> build an iteration pipeline by DataStream in the practical >>>>>>> applications >>>>>>>>>> such as machine learning. >>>>>>>>>> >>>>>>>>>> FLIP-176: Unified Iteration to Support Algorithms [2] has >>>>>> introduced >>>>>>> a >>>>>>>>>> unified iteration library >>>>>>>>>> in the Flink ML repository. This library addresses all the >>>> issues >>>>>>> present >>>>>>>>>> in the Iteration API of >>>>>>>>>> DataStream and could provide solution for all the iteration >>>>>>> use-cases. >>>>>>>>>> However, maintaining two >>>>>>>>>> separate implementations of iteration in both the Flink >>>> repository >>>>>>> and >>>>>>>>> the >>>>>>>>>> Flink ML repository >>>>>>>>>> would introduce unnecessary complexity and make it >> difficult to >>>>>>> maintain >>>>>>>>>> the Iteration API. >>>>>>>>>> >>>>>>>>>> As such I propose deprecating the Iteration API of >> DataStream >>>> and >>>>>>>>> removing >>>>>>>>>> it completely in the next >>>>>>>>>> major version. In the future, if other modules in the Flink >>>>>>> repository >>>>>>>>>> require the use of the >>>>>>>>>> Iteration API, we can consider extracting all Iteration >>>>>>> implementations >>>>>>>>>> from the Flink ML repository >>>>>>>>>> into an independent module. >>>>>>>>>> >>>>>>>>>> Looking forward to your feedback. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> [1] >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> >>>> >> https://www.google.com/url?q=https://cwiki.apache.org/confluence/display/FLINK/FLIP-357%253A%2BDeprecate%2BIteration%2BAPI%2Bof%2BDataStream&source=gmail-imap&ust=1694508403000000&usg=AOvVaw1cQxgpHJ7psRVNbB8EIOj_ >>>>>>>>>> [2] >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> >>>> >> https://www.google.com/url?q=https://cwiki.apache.org/confluence/pages/viewpage.action?pageId%3D184615300&source=gmail-imap&ust=1694508403000000&usg=AOvVaw1dNs3y_W5lYqRjauWaOEME >>>>>>>>>> >>>>>>>>>> Best regards, >>>>>>>>>> >>>>>>>>>> Wencong Liu
