[ https://issues.apache.org/jira/browse/FLINK-34007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806410#comment-17806410 ]
Zhenqiu Huang commented on FLINK-34007: --------------------------------------- [~mapohl] Yes, from the observation on the failure case, the ConfigMap was not cleanup when job manager lose the leadership. Even the renewTime field is no longer upgraded by leader elector, it means leader elector already goes out of its run loop. If look into the fabric8 leader elector source code, it looks like only when renew deadline expired, LeaderElector will abort from its run loop. Even through I don't know why renew deadline expired, enlarge the high-availability.kubernetes.leader-election.renew-deadline value could isolate some transient issues. I have started a testing job with debug log of both io.fabric8.kubernetes.client.extended.leaderelection and flink kubernetes leader election modules two days ago. If the job fail, I will post new logs in this thread. [~wangyang0918] Would you please elaborate a little bit why "It seems that the fabric8 Kubernetes client leader elector will not work properly by run() more than once if we do not clean up the leader annotation."? > Flink Job stuck in suspend state after losing leadership in HA Mode > ------------------------------------------------------------------- > > Key: FLINK-34007 > URL: https://issues.apache.org/jira/browse/FLINK-34007 > Project: Flink > Issue Type: Bug > Components: Runtime / Coordination > Affects Versions: 1.16.3, 1.17.2, 1.18.1, 1.18.2 > Reporter: Zhenqiu Huang > Priority: Major > Attachments: Debug.log, job-manager.log > > > The observation is that Job manager goes to suspend state with a failed > container not able to register itself to resource manager after timeout. > JM Log, see attached > -- This message was sent by Atlassian Jira (v8.20.10#820010)