Checkpointing

Hello, I'm curious about how Flink checkpointing would aid in recovering data if the data source is not Kafka but another system. I understand that checkpoint snapshots are taken at regular time intervals. What happens to the data that were read after the previous successful checkpoint if the sys

Async code inside Flink Sink

Hello, I have a use case where I need to do a cache file deletion after a successful sunk operation(writing to db). My Flink pipeline is built using Java. I am contemplating using Java completableFuture.runasync() to perform the file deletion activity. I am wondering what issues this might cause i

Global connection open and close

Hello, Is there a way in Flink to instantiate or open connections (to cache/db) at global level, so that it can be reused across many process functions rather than doing it in each operator's open()?Along with opening, also wanted to know if there is a way to close them at job level stop, such tha

Fwd: Flink Checkpoint & Offset Commit

Hello, I am implementing proof of concepts based Flink realtime streaming solutions. I came across below lines in out-of-the-box Flink Kafka connector documents. *https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/kafka/*