+1 Keeping the legacy implementation in place is confusing and encourages adoption of something that really shouldn't be used.
Thanks for driving this, David On Fri, Sep 1, 2023 at 8:45 AM Jing Ge <j...@ververica.com.invalid> wrote: > > Hi Wencong, > > Thanks for your clarification! +1 > > Best regards, > Jing > > On Fri, Sep 1, 2023 at 12:36 PM Wencong Liu <liuwencle...@163.com> wrote: > > > Hi Jing, > > > > > > Thanks for your reply！ > > > > > > > Or the "independent module extraction" mentioned in the FLIP does mean an > > independent module in Flink? > > > > > > Yes. If there are submodules in Flink repository needs the iteration > > (currently not), > > we could consider extracting them to a new submodule of Flink. > > > > > > > users will have to add one more dependency of Flink ML. If iteration is > > the > > only feature they need, it will look a little bit weird. > > > > > > If users only need to execute iteration jobs, they can simply remove the > > Flink > > dependency and add the necessary dependencies related to Flink ML. > > However, > > they can still utilize the DataStream API as it is also a dependency of > > Flink ML. > > > > > > Keeping an iteration submodule in Flink repository and make Flink ML > > depends it > > is also another solution. But the current implementation of Iteration in > > DataStream > > should be removed definitely due to its Incompleteness. > > > > > > The placement of the Iteration API in the repository is a topic that has > > multiple > > potential solutions. WDYT? > > > > > > Best, > > Wencong > > > > > > > > > > > > > > > > > > > > > > > > At 2023-09-01 17:59:34, "Jing Ge" <j...@ververica.com.INVALID> wrote: > > >Hi Wencong, > > > > > >Thanks for the proposal! > > > > > >"The Iteration API in DataStream is planned be deprecated in Flink 1.19 > > and > > >then finally removed in Flink 2.0. For the users that rely on the > > Iteration > > >API in DataStream, they will have to migrate to Flink ML." > > >- Does it make sense to migrate the iteration module into Flink directly? > > >Or the "independent module extraction" mentioned in the FLIP does mean an > > >independent module in Flink? Since the iteration will be removed in Flink, > > >users will have to add one more dependency of Flink ML. If iteration is > > the > > >only feature they need, it will look a little bit weird. > > > > > > > > >Best regards, > > >Jing > > > > > >On Fri, Sep 1, 2023 at 11:05 AM weijie guo <guoweijieres...@gmail.com> > > >wrote: > > > > > >> Thanks, +1 for this. > > >> > > >> Best regards, > > >> > > >> Weijie > > >> > > >> > > >> Yangze Guo <karma...@gmail.com> 于2023年9月1日周五 14:29写道： > > >> > > >> > +1 > > >> > > > >> > Thanks for driving this. > > >> > > > >> > Best, > > >> > Yangze Guo > > >> > > > >> > On Fri, Sep 1, 2023 at 2:00 PM Xintong Song <tonysong...@gmail.com> > > >> wrote: > > >> > > > > >> > > +1 > > >> > > > > >> > > Best, > > >> > > > > >> > > Xintong > > >> > > > > >> > > > > >> > > > > >> > > On Fri, Sep 1, 2023 at 1:11 PM Dong Lin <lindon...@gmail.com> > > wrote: > > >> > > > > >> > > > Thanks Wencong for initiating the discussion. > > >> > > > > > >> > > > +1 for the proposal. > > >> > > > > > >> > > > On Fri, Sep 1, 2023 at 12:00 PM Wencong Liu <liuwencle...@163.com > > > > > >> > wrote: > > >> > > > > > >> > > > > Hi devs, > > >> > > > > > > >> > > > > I would like to start a discussion on FLIP-357: Deprecate > > Iteration > > >> > API > > >> > > > of > > >> > > > > DataStream [1]. > > >> > > > > > > >> > > > > Currently, the Iteration API of DataStream is incomplete. For > > >> > instance, > > >> > > > it > > >> > > > > lacks support > > >> > > > > for iteration in sync mode and exactly once semantics. > > >> Additionally, > > >> > it > > >> > > > > does not offer the > > >> > > > > ability to set iteration termination conditions. As a result, > > it's > > >> > hard > > >> > > > > for developers to > > >> > > > > build an iteration pipeline by DataStream in the practical > > >> > applications > > >> > > > > such as machine learning. > > >> > > > > > > >> > > > > FLIP-176: Unified Iteration to Support Algorithms [2] has > > >> introduced > > >> > a > > >> > > > > unified iteration library > > >> > > > > in the Flink ML repository. This library addresses all the > > issues > > >> > present > > >> > > > > in the Iteration API of > > >> > > > > DataStream and could provide solution for all the iteration > > >> > use-cases. > > >> > > > > However, maintaining two > > >> > > > > separate implementations of iteration in both the Flink > > repository > > >> > and > > >> > > > the > > >> > > > > Flink ML repository > > >> > > > > would introduce unnecessary complexity and make it difficult to > > >> > maintain > > >> > > > > the Iteration API. > > >> > > > > > > >> > > > > As such I propose deprecating the Iteration API of DataStream > > and > > >> > > > removing > > >> > > > > it completely in the next > > >> > > > > major version. In the future, if other modules in the Flink > > >> > repository > > >> > > > > require the use of the > > >> > > > > Iteration API, we can consider extracting all Iteration > > >> > implementations > > >> > > > > from the Flink ML repository > > >> > > > > into an independent module. > > >> > > > > > > >> > > > > Looking forward to your feedback. > > >> > > > > > > >> > > > > > > >> > > > > [1] > > >> > > > > > > >> > > > > > >> > > > >> > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-357%3A+Deprecate+Iteration+API+of+DataStream > > >> > > > > [2] > > >> > > > > > > >> > > > > > >> > > > >> > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=184615300 > > >> > > > > > > >> > > > > Best regards, > > >> > > > > > > >> > > > > Wencong Liu > > >> > > > > > >> > > > >> > >
