Hi Arvid, thx for your reply. We are already using the approach with control streams to propagate business rules through our data-pipeline.
Because all our models are powered by Python, I'm going to use Table API and register UDF functions, where each UDF is a separate model. So my question is - can I update the UDF function on the fly without a job restart ? Because new model versions become available on a daily basis and we should use them as soon as possible. Thx ! пн, 12 окт. 2020 г. в 11:32, Arvid Heise <ar...@ververica.com>: > Hi Rinat, > > Which API are you using? If you use datastream API, the common way to > simulate side inputs (which is what you need) is to use a broadcast. There > is an example on SO [1]. > > [1] > https://stackoverflow.com/questions/54667508/how-to-unit-test-broadcastprocessfunction-in-flink-when-processelement-depends-o > > On Sat, Oct 10, 2020 at 7:12 PM Sharipov, Rinat <r.shari...@cleverdata.ru> > wrote: > >> Hi mates ! >> >> I'm in the beginning of the road of building a recommendation pipeline on >> top of Flink. >> I'm going to register a list of UDF python functions on job >> startups where each UDF is an ML model. >> >> Over time new model versions appear in the ML registry and I would like >> to update my UDF functions on the fly without need to restart the whole job. >> Could you tell me, whether it's possible or not ? Maybe the community can >> give advice on how such tasks can be solved using Flink and what other >> approaches exist. >> >> Thanks a lot for your help and advice ! >> >> >> > > -- > > Arvid Heise | Senior Java Developer > > <https://www.ververica.com/> > > Follow us @VervericaData > > -- > > Join Flink Forward <https://flink-forward.org/> - The Apache Flink > Conference > > Stream Processing | Event Driven | Real Time > > -- > > Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany > > -- > Ververica GmbH > Registered at Amtsgericht Charlottenburg: HRB 158244 B > Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji > (Toni) Cheng >
