[
https://issues.apache.org/jira/browse/ZOOKEEPER-2203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14571964#comment-14571964
]
Akihiro Suda commented on ZOOKEEPER-2203:
-----------------------------------------
Comments imported from the former ticket ZOOKEEPER-2189
----
[~shralex] 31/May/15 19:44 GMT
{panel}
Verifying acks as proposed in this JIRA will not solve this issue. Acks from
observers are not required to elect a leader. If standaloneenabled=false server
3 can be elected without seeing any other messages.
Also, suppose you wrote the wrong ports for the other servers ?
It seems that to fix such errors one needs some kind of config registry.
{panel}
----
[~suda] 1/Jun/15 9:27 GMT
{panel}
Hi [~shralex],
Thank you for the feedback.
I understood we cannot solve the issue by verifying acks,
but could you tell me what config registry stands for?
(something like "discovery.etcd.io" or Consul's Atlas integration?)
{panel}
----
[~shralex] 3/Jun/15 21:45 GMT
{panel}
Hi [~suda],
On second thought, although its not going to solve the specific scenario you
describe, it may still be a good idea to add some check(s). For example, if a
server receives a configuration from another server (in
FastLeaderElection.java) with the same configuration version, the configuration
itself must be identical. The check should only be for non-initial configs (see
ZOOKEEPER-1783), so probably (rqv.getVersion() > 0x100000000 and
rqv.getVersion() == curQV.getVersion()).
If you still think its a good idea, would you like to propose a patch ?
Regarding the config registry, I didn't have any specific system in mind,
perhaps others can advise. [~phunt] [~rgs]
Thanks,
Alex
{panel}
----
[~phunt] 3/Jun/15 4:59 GMT
{panel}
Not sure re config registry. What would that entail? Git? :)
{panel}
----
[~hdeng] 3/Jun/15 5:05 GMT
{panel}
I think they are talking about a central registry to contain server list. Just
like "discovery.etcd.io" or DMV...
However, this is like asking the root of root. ZK is the root of knowledge for
most distributed systems. Are we expecting a root for ZK too?
Anyway, JIRA is moved to ZOOKEEPER-2203. Let's move discussion there. Do me a
favor and let me clean up the mess caused by my mistake...
{panel}
> multiple leaders can be elected when configs conflict
> -----------------------------------------------------
>
> Key: ZOOKEEPER-2203
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2203
> Project: ZooKeeper
> Issue Type: Bug
> Components: leaderElection
> Affects Versions: 3.5.0
> Reporter: Akihiro Suda
>
> This sequence leads the ensemble to a split-brain state:
> * Start server 1 (config=1:participant, 2:participant, 3:participant)
> * Start server 2 (config=1:participant, 2:participant, 3:participant)
> * 1 and 2 believe 2 is the leader
> * Start server 3 (config=1:observer, 2:observer, 3:participant)
> * 3 believes 3 is the leader, although 1 and 2 still believe 2 is the leader
> Such a split-brain ensemble is very unstable.
> Znodes can be lost easily:
> * Create some znodes on 2
> * Restart 1 and 2
> * 1, 2 and 3 can think 3 is the leader
> * znodes created on 2 are lost, as 1 and 2 sync with 3
> I consider this behavior as a bug and that ZK should fail gracefully if a
> participant is listed as an observer in the config.
> In current implementation, ZK cannot detect such an invalid config, as
> FastLeaderElection.sendNotification() sends notifications to only voting
> members and hence there is no message from observers(1 and 2) to the new
> voter (3).
> I think FastLeaderElection.sendNotification() should send notifications to
> all the members and FastLeaderElection.Messenger.WorkerReceiver.run() should
> verify acks.
> Any thoughts?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
