Zakelly Lan created FLINK-34973: ----------------------------------- Summary: FLIP-423: Disaggregated State Storage and Management Key: FLINK-34973 URL: https://issues.apache.org/jira/browse/FLINK-34973 Project: Flink Issue Type: New Feature Reporter: Zakelly Lan Fix For: 2.0.0
The past decade has witnessed a dramatic shift in Flink's deployment mode, workload patterns, and hardware improvements. We've moved from the map-reduce era where workers are computation-storage tightly coupled nodes to a cloud-native world where containerized deployments on Kubernetes become standard. To enable Flink's Cloud-Native future, we introduce Disaggregated State Storage and Management that uses DFS as primary storage in Flink 2.0, as promised in the Flink 2.0 Roadmap. Detailed design and story: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=293046855 Also sub-FLIPs: - Asynchronous State APIs ([FLIP-424|https://cwiki.apache.org/confluence/x/SYp3EQ]): Introduce new APIs for asynchronous state access. - Asynchronous Execution Model ([FLIP-425|https://cwiki.apache.org/confluence/x/S4p3EQ]): Implement a non-blocking execution model leveraging the asynchronous APIs introduced in FLIP-424. - Grouping Remote State Access ([FLIP-426|https://cwiki.apache.org/confluence/x/TYp3EQ]): Enable retrieval of remote state data in batches to avoid unnecessary round-trip costs for remote access. - Disaggregated State Store ([FLIP-427|https://cwiki.apache.org/confluence/x/T4p3EQ]): Introduce the initial version of the ForSt disaggregated state store. - Fault Tolerance/Rescale Integration ([FLIP-428|https://cwiki.apache.org/confluence/x/UYp3EQ]): Integrate checkpointing mechanisms with the disaggregated state store for fault tolerance and fast rescaling. -- This message was sent by Atlassian Jira (v8.20.10#820010)