Re: RichAsyncFunction + Cache or Map State?

2022-05-09 Thread Dan Hill
Hi. Any advice on this? I just hit this too. Some ideas: 1. Manage our own separate cache (disk, Redis, etc). 2. Use two operators (first one a cache one and the second is the RichAsyncFunction). Have a feedback loop by using another Kafka topic or S3 File source/sink. On Wed, Feb 9, 2022 at

RichAsyncFunction + Cache or Map State?

2022-02-09 Thread Clayton Wohl
I have a RichAsyncFunction that does async queries to an external database. I'm using a Guava cache within the Flink app. I'd like this Guava cache to be serialized with the rest of Flink state in checkpoint/savepoints. However, RichAsyncFunction doesn't support the state functionality at all. The