Hello Timo and Bechet, @Timo: Thanks a lot for the message forwarding.
@Bechet: Thanks for your answer. I was not aware of *flink-ai-extended* project. Also I was not aware of the fact that ALink is striving to become the new FlinkML. Definitely, I will look into ALink and flink-ai-extended. Alibaba knows better haha. To address your answer: "I guess it depends on what exactly you want to do. If you are doing a training running for hours with a lot of rounds of iterations until it converges, having it trained separately and then porting it to Flink for inference might not lose too much efficiency. However, if you are doing online learning to incrementally update your model as the samples flow by, having such incremental training embedded into Flink would make a lot of sense. Flink-ai-extended was created to support both cases, but it is definitely more attractive in the incremental training case." --> Well, since I focus on streaming I think an online training and serving solution, a.k.a. prequential training, would be more suitable. There are two problems I see in this direction though: 1) To enable *efficient* and *reliable* (super important as well) training in a streaming fashion, Flink DataStream part should support iterations thoroughly. However, I have read from multiple sources that iterations on Flink are not there yet. This is why I searched for other solutions to investigate what they do. Alibaba seems like a good direction as you said also ALink is used in production. 2) Due to lack of reliability, in the majority of use cases prequential training is not chosen by companies, which instead rely on the solution you described; hence, (re)training the model (maybe for hours) and port it to Flink whenever it is ready. Nevertheless, thanks a lot for your answers. @Flinkers: Let's gather any other solutions that exist and are not listed. Best, Max -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
