[
https://issues.apache.org/jira/browse/FLINK-39835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-39835:
-----------------------------------
Labels: pull-request-available (was: )
> AsyncWaitOperator supports soft backpressure to avoid blocking unaligned
> checkpoint
> -----------------------------------------------------------------------------------
>
> Key: FLINK-39835
> URL: https://issues.apache.org/jira/browse/FLINK-39835
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Task
> Reporter: Gen Luo
> Priority: Major
> Labels: pull-request-available
>
> When AsyncWaitOperator processes input, if the async queue is full, it yields
> and waits for a callback to complete to free up a slot. During this time,
> unaligned checkpoints cannot be processed.
> Soft backpressure is used to ensure UC can be processed in time. In the
> non-async path, processInput produces data written to the ResultPartition. If
> the RP is full, processInput can be blocked by soft backpressure so that
> mails can be executed.
> The same idea can apply to AsyncWaitOperator: on the async path, processInput
> produces requests written to the AsyncWaitOperator's queue — when the queue
> is full, processInput should be blocked. Similarly, results from
> AsyncWaitOperator produce data written to downstream (e.g., RP) — if
> downstream is full, AsyncWaitOperator's emitResult should also be blocked.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
