Hello all, >From our previous discussion started by Stavros, we decided to start a planning document [1] to figure out possible next steps for ML on Flink.
Our concerns where mainly ensuring active development while satisfying the needs of the community. We have listed a number of proposals for future work in the document. In short they are: - Offline learning with the batch API - Online learning - Offline learning with the streaming API - Low-latency prediction serving I saw there is a number of people willing to work on ML for Flink, but the truth is that we cannot cover all of these suggestions without fragmenting the development too much. So my recommendation is to pick out 2 of these options, create design documents and build prototypes for each library. We can then assess their viability and together with the community decide if we should try to include one (or both) of them in the main Flink distribution. So I invite people to express their opinion about which task they would be willing to contribute and hopefully we can settle on two of these options. Once that is done we can decide how we do the actual work. Since this is highly experimental I would suggest we work on repositories where we have complete control. For that purpose I have created an organization [2] on Github which we can use to create repositories and teams that work on them in an organized manner. Once enough work has accumulated we can start discussing contributing the code to the main distribution. Regards, Theodore [1] https://docs.google.com/document/d/1afQbvZBTV15qF3vobVWUjxQc49h3Ud06MIRhahtJ6dw/ [2] https://github.com/flinkml
