[ https://issues.apache.org/jira/browse/FLINK-35669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17863200#comment-17863200 ]
xingbe commented on FLINK-35669: -------------------------------- I have conducted tests for both scenarios as instructed: 1、I start a batch job consisting of multiple stages in high availability (HA) mode utilizing ZooKeeper. When some stages have already completed, I kill the JobManager process. The job is capable of resuming its execution, with the already completed stages not requiring re-execution. 2、I used a custom FileSource and implemented the SupportsBatchSnapshot interface. During the process, I kill the JobManager and wait for it to recover, and I observed the resumption from the progress. It should be noted that if no task finishes during this period, a snapshot is taken every 3 minutes by default. It can be adjusted by setting the configration `execution.batch.job-recovery.snapshot.min-pause`. > Release Testing: Verify FLIP-383: Support Job Recovery from JobMaster > Failures for Batch Jobs > --------------------------------------------------------------------------------------------- > > Key: FLINK-35669 > URL: https://issues.apache.org/jira/browse/FLINK-35669 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Network > Reporter: Junrui Li > Assignee: xingbe > Priority: Blocker > Labels: release-testing > Fix For: 1.20.0 > > > In 1.20, we introduced a batch job recovery mechanism to enable batch jobs to > recover as much progress as possible after a JobMaster failover, avoiding the > need to rerun tasks that have already been finished. > More information about this feature and how to enable it could be found in: > [https://nightlies.apache.org/flink/flink-docs-master/docs/ops/batch/recovery_from_job_master_failure/] > We may need the following tests: > # Start a batch job with High Availability (HA) enabled, and after it has > progressed to a certain point, kill the JobManager (jm), then observe whether > the job recovers its progress normally. > # Use a custom source and ensure that its SplitEnumerator implements the > SupportsBatchSnapshot interface, submit the job, and after it has progressed > to a certain point, kill the JobManager (jm), then observe whether the job > recovers its progress normally. > > Follow up the test for https://issues.apache.org/jira/browse/FLINK-33892 -- This message was sent by Atlassian Jira (v8.20.10#820010)