Hi, In my flink operators, I need to connect to an external service to update state. I was thinking that the updates to the external service can be synchronized via checkpoint barriers.
The topology of the stream is a source, then a single stage of operator replicas handling different partitions, then all joining in a single sink. Each operator will contact the external service when it receives a checkpoint barrier and uploads local state (which caches the uploads and returns a handle). After upload, it forwards the cache handle to the sink. Once sink receives handles from all such operators, it calls the external service with a list of handles received. This helps ensure that all handles are from the same checkpoint barrier. Is it possible to achieve this in a flink application? Thanks, Gopi
