Hi Wencong, Thanks for the proposal!
"The Iteration API in DataStream is planned be deprecated in Flink 1.19 and then finally removed in Flink 2.0. For the users that rely on the Iteration API in DataStream, they will have to migrate to Flink ML." - Does it make sense to migrate the iteration module into Flink directly? Or the "independent module extraction" mentioned in the FLIP does mean an independent module in Flink? Since the iteration will be removed in Flink, users will have to add one more dependency of Flink ML. If iteration is the only feature they need, it will look a little bit weird. Best regards, Jing On Fri, Sep 1, 2023 at 11:05 AM weijie guo <guoweijieres...@gmail.com> wrote: > Thanks, +1 for this. > > Best regards, > > Weijie > > > Yangze Guo <karma...@gmail.com> 于2023年9月1日周五 14:29写道： > > > +1 > > > > Thanks for driving this. > > > > Best, > > Yangze Guo > > > > On Fri, Sep 1, 2023 at 2:00 PM Xintong Song <tonysong...@gmail.com> > wrote: > > > > > > +1 > > > > > > Best, > > > > > > Xintong > > > > > > > > > > > > On Fri, Sep 1, 2023 at 1:11 PM Dong Lin <lindon...@gmail.com> wrote: > > > > > > > Thanks Wencong for initiating the discussion. > > > > > > > > +1 for the proposal. > > > > > > > > On Fri, Sep 1, 2023 at 12:00 PM Wencong Liu <liuwencle...@163.com> > > wrote: > > > > > > > > > Hi devs, > > > > > > > > > > I would like to start a discussion on FLIP-357: Deprecate Iteration > > API > > > > of > > > > > DataStream [1]. > > > > > > > > > > Currently, the Iteration API of DataStream is incomplete. For > > instance, > > > > it > > > > > lacks support > > > > > for iteration in sync mode and exactly once semantics. > Additionally, > > it > > > > > does not offer the > > > > > ability to set iteration termination conditions. As a result, it's > > hard > > > > > for developers to > > > > > build an iteration pipeline by DataStream in the practical > > applications > > > > > such as machine learning. > > > > > > > > > > FLIP-176: Unified Iteration to Support Algorithms [2] has > introduced > > a > > > > > unified iteration library > > > > > in the Flink ML repository. This library addresses all the issues > > present > > > > > in the Iteration API of > > > > > DataStream and could provide solution for all the iteration > > use-cases. > > > > > However, maintaining two > > > > > separate implementations of iteration in both the Flink repository > > and > > > > the > > > > > Flink ML repository > > > > > would introduce unnecessary complexity and make it difficult to > > maintain > > > > > the Iteration API. > > > > > > > > > > As such I propose deprecating the Iteration API of DataStream and > > > > removing > > > > > it completely in the next > > > > > major version. In the future, if other modules in the Flink > > repository > > > > > require the use of the > > > > > Iteration API, we can consider extracting all Iteration > > implementations > > > > > from the Flink ML repository > > > > > into an independent module. > > > > > > > > > > Looking forward to your feedback. > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-357%3A+Deprecate+Iteration+API+of+DataStream > > > > > [2] > > > > > > > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=184615300 > > > > > > > > > > Best regards, > > > > > > > > > > Wencong Liu > > > > > > >
