Hi. Any advice on this? I just hit this too. Some ideas: 1. Manage our own separate cache (disk, Redis, etc). 2. Use two operators (first one a cache one and the second is the RichAsyncFunction). Have a feedback loop by using another Kafka topic or S3 File source/sink.
On Wed, Feb 9, 2022 at 7:11 AM Clayton Wohl <[email protected]> wrote: > I have a RichAsyncFunction that does async queries to an external > database. I'm using a Guava cache within the Flink app. I'd like this Guava > cache to be serialized with the rest of Flink state in > checkpoint/savepoints. However, RichAsyncFunction doesn't support the state > functionality at all. > > There is one Guava cache for the entire Flink app which might make this > scenario simpler. > > Is there a recommended way to handle this situation? > > Also, the Flink MapState interface doesn't support check-and-set type > functionality and doesn't support lock-free concurrent use like > java.util.concurrent.ConcurrentMap and Guava's cache do. I need both of > these features for proper concurrent operation. So even if I could use > Flink MapState, that doesn't seem like a practical solution. >
