[ https://issues.apache.org/jira/browse/FLINK-21979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flink Jira Bot updated FLINK-21979: ----------------------------------- Labels: stale-critical (was: ) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Critical but is unassigned and neither itself nor its Sub-Tasks have been updated for 7 days. I have gone ahead and marked it "stale-critical". If this ticket is critical, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > Job can be restarted from the beginning after it reached a terminal state > ------------------------------------------------------------------------- > > Key: FLINK-21979 > URL: https://issues.apache.org/jira/browse/FLINK-21979 > Project: Flink > Issue Type: Bug > Components: Runtime / Coordination > Affects Versions: 1.11.3, 1.12.2, 1.13.0 > Reporter: Till Rohrmann > Priority: Critical > Labels: stale-critical > Fix For: 1.14.0 > > > Currently, the {{JobMaster}} removes all checkpoints after a job reaches a > globally terminal state. Then it notifies the {{Dispatcher}} about the > termination of the job. The {{Dispatcher}} then removes the job from the > {{SubmittedJobGraphStore}}. If the {{Dispatcher}} process fails before doing > that it might get restarted. In this case, the {{Dispatcher}} would still > find the job in the {{SubmittedJobGraphStore}} and recover it. Since the > {{CompletedCheckpointStore}} is empty, it would start executing this job from > the beginning. > I think we must not remove job state before the job has not been marked as > done or made inaccessible for any restarted processes. Concretely, we should > first remove the job from the {{SubmittedJobGraphStore}} and only then delete > the checkpoints. Ideally all the job related cleanup operation happens > atomically. -- This message was sent by Atlassian Jira (v8.3.4#803005)