Zakelly Lan created FLINK-34974:
-----------------------------------
Summary: FLIP-424: Asynchronous State APIs
Key: FLINK-34974
URL: https://issues.apache.org/jira/browse/FLINK-34974
Project: Flink
Issue Type: New Feature
Components: Runtime / State Backends
Reporter: Zakelly Lan
Assignee: Zakelly Lan
Fix For: 2.0.0
This is a sub-FLIP for the disaggregated state management and its related work,
please read the [FLIP-423|https://cwiki.apache.org/confluence/x/R4p3EQ] first
to know the whole story.
To maximize I/O capacity utilization and enhance the use of pre-allocated
computational resources, this FLIP proposes the introduction of asynchronous
state APIs. These APIs permit state access to be executed in threads separate
from the task thread, returning the result when available. Consequently, the
task thread can process another element while awaiting multiple pending state
results. This enables concurrent processing of multiple records, ensuring that
the latency of individual I/O operations no longer has a direct impact on job
performance. This approach is particularly advantageous in scenarios where I/O
bandwidth is underutilized and I/O latency is the limiting factor. The
Disaggregated Storage Architecture, as discussed in FLIP-423, is a prime
example of a scenario characterized by abundant and easily scalable I/O
bandwidth coupled with higher I/O latency. The asynchronous state APIs hold
great promise for significantly enhancing Flink's performance when dealing with
disaggregated state.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
