Hi devs, I would like to initiate a discussion about FLIP-455: Declare async state processing and checkpoint the in-flight requests[1].
FLIP-423[2] and the related sub-FLIPs introduced the disaggregated state and async accessing model of state. However, the in-flight state requests (or records) should be drained at the checkpoint, which leads to an increased checkpoint synchronization delay. The main goal of this FLIP is to snapshot the in-flight requests as part of a checkpoint. This will accelerate the draining process and decouple the mutual impact between checkpoints and data processing. For your ease of understanding, the FLIP covers three key aspects: 1. (Public API) Additional APIs for conditional branchings under `StateFuture` 2. (Internal API) An internal API set for declaring and defining the record processing 3. (Implementation) Method to snapshot the in-flight state requests For the first two parts, there is also a PoC branch[3] provided. Looking forward to hearing from you. [1] https://cwiki.apache.org/confluence/x/C4owEg [2] https://cwiki.apache.org/confluence/x/R4p3EQ [3] https://github.com/apache/flink/pull/24719 Best, Zakelly
