[ https://issues.apache.org/jira/browse/AURORA-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16196186#comment-16196186 ]
Bill Farner commented on AURORA-1944: ------------------------------------- I agree with John's assessment - the retry mechanism here is shutting down, so we should address the inability to shut down. > Aurora is unable to elect leader after losing ZK for an extended period of > time > ------------------------------------------------------------------------------- > > Key: AURORA-1944 > URL: https://issues.apache.org/jira/browse/AURORA-1944 > Project: Aurora > Issue Type: Bug > Components: Scheduler > Environment: Running on 0.17.0 > Reporter: Renan DelValle > Attachments: aurora-0.log, aurora-1.log, aurora-2.log > > > Using Apache Curator as the Zookeeper library causes an issue where Aurora is > unable to elect a leader if Zookeeper loses quorum for an extended period of > time. > Scheduler seems to crash around: > {{W0802 14:01:14.436 [TaskEventBatchWorker, SchedulerLifecycle] Failed to > leave leadership: > org.apache.aurora.common.zookeeper.SingletonService$LeaveException: Failed to > abdicate leadership of group at /aurora/scheduler}} > When the init system brings the scheduler back up, it is unable to elect a > leader if ZK is still down. > Specifically, the redirect monitor fails: > {{E0802 14:09:37.063 [RedirectMonitor STARTING, > GuavaUtils$LifecycleShutdownListener] Service: RedirectMonitor [FAILED] > failed unexpectedly. Triggering shutdown.}} > Leading to every scheduler showing the following: > {{W0802 14:16:34.646 [qtp576711849-43, LeaderRedirect] No serviceGroupMonitor > in host set, will not redirect despite not being leader.}} > Once the scheduler enters this state, it is unable to snap out of it until it > is manually restarted. -- This message was sent by Atlassian JIRA (v6.4.14#64029)