[
https://issues.apache.org/jira/browse/HADOOP-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239725#comment-13239725
]
Bikas Saha commented on HADOOP-8220:
------------------------------------
bq. The new behavior avoids this, since the error handling patch is in
ActiveStandbyElector itself. This makes it easier to get the right semantics.
Ah. Now I get it. The elector should be robust against client code (ZKFC in
this case). I like Hari's proposal of using a return value to inform about
fail/success of becoming active. I am not that familiar with standard practices
in Java - are return values preferred or exceptions?
bq. This generates a lot of log spew and has little actual benefit. If instead
we retry only once a second, then (a) the logs are more readable, and (b) if
there is another StandbyNode in the cluster, it will get a chance to try to
become active.
I did not understand where the tight loop is? Do you mean (Elector gets
lock<->ZKFC fails to becomes active)?
I do not have any data on the trade off between 1) letting the last active
become active again with log spew 2) letting another standby become active by
making the last active sleep. But for arguments sake I would prefer 1). IMO
having continuity in the active node would reduce the overheads of
client/datanode failover etc.
bq. becomeActive() should be protected by a timeout also. If NN is taking far
too long to return, FC should declare failure and give up the lock. Otherwise,
it is a deadlock.
Hari, this seems similar to the alternative proposed in HADOOP-8205 about
trying to make sure that the transition to active is short.
> ZKFailoverController doesn't handle failure to become active correctly
> ----------------------------------------------------------------------
>
> Key: HADOOP-8220
> URL: https://issues.apache.org/jira/browse/HADOOP-8220
> Project: Hadoop Common
> Issue Type: Bug
> Components: ha
> Affects Versions: 0.23.3, 0.24.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Critical
> Attachments: hadoop-8220.txt
>
>
> The ZKFC doesn't properly handle the case where the monitored service fails
> to become active. Currently, it catches the exception and logs a warning, but
> then continues on, after calling quitElection(). This causes a NPE when it
> later tries to use the same zkClient instance while handling that same
> request. There is a test case, but the test case doesn't ensure that the node
> that had the failure is later able to recover properly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
