Rui Fan created FLINK-40482:
-------------------------------
Summary: DefaultCheckpointPlanCalculator misclassifies
FAILED/CANCELED tasks as finished
Key: FLINK-40482
URL: https://issues.apache.org/jira/browse/FLINK-40482
Project: Flink
Issue Type: Bug
Components: Runtime / Checkpointing
Reporter: Rui Fan
Assignee: Rui Fan
h3. Problem
DefaultCheckpointPlanCalculator uses Execution.isFinished() (which returns
state.isTerminal()) to decide whether a task is finished. This treats any
terminal task — FINISHED, FAILED, or CANCELED — as finished.
When a failover is in progress, some tasks may be FAILED/CANCELED. The
calculator then classifies them as finished and excludes them from
tasksToWaitFor, producing a checkpoint plan that expects fewer acks than
it should (in the extreme, an empty plan).
Today PendingCheckpoint rejects an empty tasksToWaitFor, so the worst case
is masked. But the classification is wrong at its source: "terminal" is not
the same as "finished", and relying on the downstream guard leaves the issue
latent.
h3. Fix
- Classify explicitly in collectTaskRunningStatus: a non-terminal task is
running; a terminal task must be genuinely FINISHED.
- Abort the checkpoint with a graceful CheckpointException when a task is
FAILED/CANCELED (i.e. a failover is underway) instead of building a plan
from a transient state.
- Remove the misleading Execution.isFinished() (its only caller).
This is condition-based, not mode-based, so it is correct for both streaming
and batch. calculateCheckpointPlan and all task state transitions run on the
JM main thread, so the pre-check and plan computation are atomic.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
