Hi, FWIW, there is this JIRA that is tracking a pubsub / broadcast messaging primitive in StateFun: https://issues.apache.org/jira/browse/FLINK-16319
This is probably what you are looking for. And I do agree, in the case that the control stream (which updates the application logic) is high volume, redeploying functions may not work well. I don't think there really is a "recommended" way of doing the "broadcast control stream, join with main stream" pattern with StateFun at the moment, at least without FLINK-16319. On the other hand, it could be possible to use stateful functions to implement a pub-sub model in user space for the time being. I've actually left some ideas for implementing that in the comments of FLINK-16319. Cheers, Gordon On Mon, Feb 22, 2021 at 6:38 AM Miguel Araújo <upwarr...@gmail.com> wrote: > Hi everyone, > > What is the recommended way of achieving the equivalent of a broadcast in > Flink when using Stateful Functions? > > For instance, assume we are implementing something similar to Flink's > demo fraud detection > <https://flink.apache.org/news/2020/03/24/demo-fraud-detection-2.html> but > in Stateful Functions - how can one dynamically update the application's > logic then? > There was a similar question in this mailing list in the past where it was > recommended > moving the dynamic logic to a remote function > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Stateful-Functions-ML-model-prediction-tp38286p38302.html> > so > that one could achieve that by deploying a new container. I think that's > not very realistic as updates might happen with a frequency that's not > compatible with that approach (e.g., sticking to the fraud detection > example, updating fraud detection rules every hour is not unusual), nor > should one be deploying a new container when data (not code) changes. > > Is there a way of, for example, modifying FunctionProviders > <https://ci.apache.org/projects/flink/flink-statefun-docs-master/sdk/java.html#function-providers-and-dependency-injection> > on the fly? > > Thanks, > Miguel >
