Thanks Wencong for initiating the discussion. +1 for the proposal.
On Fri, Sep 1, 2023 at 12:00 PM Wencong Liu <liuwencle...@163.com> wrote: > Hi devs, > > I would like to start a discussion on FLIP-357: Deprecate Iteration API of > DataStream [1]. > > Currently, the Iteration API of DataStream is incomplete. For instance, it > lacks support > for iteration in sync mode and exactly once semantics. Additionally, it > does not offer the > ability to set iteration termination conditions. As a result, it's hard > for developers to > build an iteration pipeline by DataStream in the practical applications > such as machine learning. > > FLIP-176: Unified Iteration to Support Algorithms [2] has introduced a > unified iteration library > in the Flink ML repository. This library addresses all the issues present > in the Iteration API of > DataStream and could provide solution for all the iteration use-cases. > However, maintaining two > separate implementations of iteration in both the Flink repository and the > Flink ML repository > would introduce unnecessary complexity and make it difficult to maintain > the Iteration API. > > As such I propose deprecating the Iteration API of DataStream and removing > it completely in the next > major version. In the future, if other modules in the Flink repository > require the use of the > Iteration API, we can consider extracting all Iteration implementations > from the Flink ML repository > into an independent module. > > Looking forward to your feedback. > > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-357%3A+Deprecate+Iteration+API+of+DataStream > [2] > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=184615300 > > Best regards, > > Wencong Liu
