Hi Zhu Zhu:
Just to clarify - from what I understand, EMR also has by default restart times
(I think it is 3). So if the EMR restarts the job - the job id is the same
since the job graph is the same.
Thanks for the clarification.
On Monday, May 25, 2020, 04:01:17 AM EDT, Yang Wang <[email protected]>
wrote:
Just share some additional information.
When deploying Flink application on Yarn and it exhausted restart policy,
thenthe whole application will failed. If you start another instance(Yarn
application),even the high availability is configured, we could not recover
from the latestcheckpoint because the clusterId(i.e. applicationId) has changed.
Best,Yang
Zhu Zhu <[email protected]> 于2020年5月25日周一 上午11:17写道：
Hi M,
Regarding your questions:1. yes. The id is fixed once the job graph is
generated.2. yes
Regarding yarn mode:1. the job id keeps the same because the job graph will be
generated once at client side and persist in DFS for reuse2. yes if high
availability is enabled
Thanks,Zhu Zhu
M Singh <[email protected]> 于2020年5月23日周六 上午4:06写道：
Hi Flink Folks:
If I have a Flink Application with 10 restarts, if it fails and restarts, then:
1. Does the job have the same id ?2. Does the automatically restarting
application, pickup from the last checkpoint ? I am assuming it does but just
want to confirm.
Also, if it is running on AWS EMR I believe EMR/Yarn is configured to restart
the job 3 times (after it has exhausted it's restart policy) . If that is the
case:1. Does the job get a new id ? I believe it does, but just want to
confirm.2. Does the Yarn restart honor the last checkpoint ? I believe, it
does not, but is there a way to make it restart from the last checkpoint of the
failed job (after it has exhausted its restart policy) ?
Thanks
