HI Lei,
ZooKeeper provides a set of primitives which allows you to do all kinds of
things! You might want to take a look at the api and some examples of
zookeeper recipes to see how it works and probably that will clear things
out for you.
Here are the links:
http://hadoop.apache.org/zookeeper/
I believe Lei's concern is that the leader and all slaves can talk to
ZK, but the slaves cannot talk to the leader. As a result no work can be
done. However nothing will happen on the ZK side since everyone is
heartbeating properly.
Mahadev I think you came up with a pretty good solution. Howe
Hi Mahadev,
First of all, I like to thank you for being patient with me - my questions
seem unclear to many of you who try to help me.
I guess clients have to be smart enough to trigger a new leader election by
trying to delete the znode. But in this case, ZK should not allow any single
or multip
Maybe I jumped the gun here but Ted's response to your query is more
appropriate -
You can then use ZK in your application to pick a lead machine for other
operations. In that case, essentially every failure scenario is handled by
the standard recipe. In your example where the mas
Lei,
I think that Mahadev was correct that there is some confusion here.
Leader election is normally a term used for an operation that is entirely
internal to ZK. It is very robust and you probably don't need to worry
about it.
You can then use ZK in your application to pick a lead machine for
Hi Lei,
Sorry I minsinterpreted your question! The scenario you describe could be
handled in such a way -
You could have a status node in ZooKeeper which every slave will subscribe
to and update! If one of the slave nodes sees that there have been too many
connection refused to the Leader by the
Hi Mahadev,
Why would the leader be disconnected from ZK? ZK is fine communicating with
the leader in this case. We are talking about asymmetric network failure.
Yes. Leader could consider all the slaves being down if it tracks the status
of all slaves himself. But I guess if ZK is used for for me
Hi Lei,
In this case, the Leader will be disconnected from ZK cluster and will give
up its leadership. Since its disconnected, ZK cluster will realize that the
Leader is dead!
When Zk cluster realizes that the Leader is dead (this is because the zk
cluster hasn't heard from the Leader for a c
Thank you all for your answers. It clarifies a lot of my confusion about the
service guarantees of ZK. I am still struggling with one failure case (I am
not trying to be the pain in the neck. But I need to have a full
understanding of what ZK can offer before I make a decision on whether to
used it
If one of your user clients can no longer reach one member of the ZK
cluster, then it will try to reach another. If it succeeds, then it will
continue without any problems as long as the ZK cluster itself is OK.
This applies for all the ZK recipes. You will have to be a little bit
careful to han
It really depends on the type of leader election you have implemented.
You can do simple leader election such as we describe in the recipes, in
that case clients won't notice the leader changed unless they are
connected. You could implement some more complex protocol, but typically
the recipe w
Hi Lei,
In this case, its up to application to decide what to do when this happens.
The application will be notified that its disconnected from the ZooKeeper
cluster. In such a case some of the applications might decide to not proceed
at all, (since it might lead to some state corruption) and some
Hi Lei,
I think you might be confusing Leader Election within ZooKeeper itself with
electing a leader for your application? Is that so?
Leader Election within ZooKeeper is totally internal to zookeeper service
and is not visible to applications.
I am little confused, what your problem statement
Hi Henry,
I am not talking about the leader election within zookeeper cluster. I guess
I didn't make the discussion context clear. In my case, I run a cluster that
uses zookeeper for doing the leader election. Yes, nodes in my cluster are
the clients of zookeeper. Those nodes depend on zookeeper
Hi Lei -
The 'user cluster' (by which I think you mean the set of clients of
ZooKeeper?) plays no part in leader election. If a majority of ZooKeeper
server nodes can talk to each other, a new leader can be elected. Clients of
the minority server partition will be disconnected - if they too cannot
Hi Ted,
I 100% agree with what you said. But my question is more about what if my
zookeeper service cluster is partitioned from a majority of nodes in my USER
CLUSTER. In this case, the majority nodes in one network partition can’t
select a new leader because zookeeper is out of reach.
Anothe
There are a variety of situations that can trigger a new leader election and
a few that can cause the cluster to be unable to elect a new leader.
Isolation of just the leader is one of the situations that will cause a new
leader election. Isolation of nodes into groups smaller than the quorum
wil
Hi,
I have a general question on how zookeeper can maintain its view of the user
cluster (that zookeeper manages) that is consistent with the nodes in the user
cluster. In other words, when zookeeper considers the current leader is
unavailable, does it really guarantee that a majority of nodes
On 04/30/2010 10:16 AM, Aaron Crow wrote:
Hi Patrick, thanks for your time and detailed questions.
No worries. When we hear about an issue we're very interested to
followup and resolve it, regardless of the source. We take the project
goals of high reliability/availablity _very_ seriously,
Hi Patrick, thanks for your time and detailed questions.
We're running on Java build 1.6.0_14-b08, on Ubuntu 4.2.4-1ubuntu3. Below is
output from a recent stat, and a question about node count. For your other
questions, I should save your time with a batch reply: I wasn't tracking
nearly enough th
20 matches
Mail list logo
