Re-start strategy without checkpointing enabled
Hello, If checkpointing is NOT enabled and re-start strategy is configured then flink retries the whole job execution i.e. enabling checkpointing is must for re-try or not? Rgds, Kamal
Re: Re-start strategy without checkpointing enabled
Hi, Kamal. If we don't enable checkpointing, the job will be started with the startup mode each time. For example, the job reads Kafka from the earliest offset and writes to mysql. If the job failover without checkpointing, the tasks will consume Kafka from the earliest offset again. I think it is best to enable checkpointing to restart job from the position where the job stopped reading. Best, Hang Kamal Mittal via user 于2023年8月23日周三 18:46写道： > Hello, > > > > If checkpointing is NOT enabled and re-start strategy is configured then > flink retries the whole job execution i.e. enabling checkpointing is must > for re-try or not? > > > > Rgds, > > Kamal >
Re: Re-start strategy without checkpointing enabled
Hi, Kamal As Hang says, some extra info about job failover strategy for reference in [1] [1] https://nightlies.apache.org/flink/flink-docs-master/docs/ops/state/task_failure_recovery/ Best, Ron Hang Ruan 于2023年8月23日周三 22:27写道： > Hi, Kamal. > > If we don't enable checkpointing, the job will be started with the startup > mode each time. > For example, the job reads Kafka from the earliest offset and writes to > mysql. If the job failover without checkpointing, the tasks will consume > Kafka from the earliest offset again. > > I think it is best to enable checkpointing to restart job from the > position where the job stopped reading. > > Best, > Hang > > Kamal Mittal via user 于2023年8月23日周三 18:46写道： > >> Hello, >> >> >> >> If checkpointing is NOT enabled and re-start strategy is configured then >> flink retries the whole job execution i.e. enabling checkpointing is must >> for re-try or not? >> >> >> >> Rgds, >> >> Kamal >> >
RE: Re-start strategy without checkpointing enabled
Thanks. Only query is that whether checkpointing and job re-start strategy are independent to each other? If checkpointing is not enabled and re-start strategy is given then flink will still re-try the job as per configuration? Checkpointing is not enabled as there is no state to maintain. If checkpointing is not enabled, the “no restart” strategy is used. If checkpointing is activated and the restart strategy has not been configured, the fixed-delay strategy is used with Integer.MAX_VALUE restart attempts. See the following list of available restart strategies to learn what values are supported. Rgds, Kamal From: liu ron Sent: 24 August 2023 07:43 AM To: user@flink.apache.org Subject: Re: Re-start strategy without checkpointing enabled Hi, Kamal As Hang says, some extra info about job failover strategy for reference in [1] [1] https://nightlies.apache.org/flink/flink-docs-master/docs/ops/state/task_failure_recovery/ Best, Ron Hang Ruan mailto:ruanhang1...@gmail.com>> 于2023年8月23日周三 22:27写道： Hi, Kamal. If we don't enable checkpointing, the job will be started with the startup mode each time. For example, the job reads Kafka from the earliest offset and writes to mysql. If the job failover without checkpointing, the tasks will consume Kafka from the earliest offset again. I think it is best to enable checkpointing to restart job from the position where the job stopped reading. Best, Hang Kamal Mittal via user mailto:user@flink.apache.org>> 于2023年8月23日周三 18:46写道： Hello, If checkpointing is NOT enabled and re-start strategy is configured then flink retries the whole job execution i.e. enabling checkpointing is must for re-try or not? Rgds, Kamal
