Ted, could you elaborate a bit more on this? I was under the (mis) impression that each ZK server in an ensemble only needed connectivity to another member in the ensemble, not to each member in the ensemble. It sounds like you are saying the latter is true.
Could you explain the idea behind the Observers feature, what this concept is supposed to address, and how it applies to the WAN configuration problem in particular? """ The ideas for federating ZK or allowing observers would likely do what you want. I can imagine that an observer would only care that it can see it's local peers and one of the observers would be elected to get updates (and thus would care about the central service). """ This certainly sounds like exactly what I want...Was this introduced in 3.2 in full, or only partially? Here, do you mean the servers will log warnings untill all the ensemble members are visible to each other? """ Any servers that see a minority of the other servers will go tharn until the "partition" is healed. That isn't what you want (at all). """ Given that 3.2 has a serious bug, I've recommended that we proceed with our deploy based 3.1.1. For this version, it sounds like we will have to open up connectivity from each zk server to each zk server, across the various zones in the WAN. Is this correct? -Todd -----Original Message----- From: Ted Dunning [mailto:ted.dunn...@gmail.com] Sent: Friday, July 24, 2009 3:41 PM To: zookeeper-user@hadoop.apache.org Subject: Re: Zookeeper WAN Configuration Vanilla ZK servers will see security constraints as a network partition. Any servers that see a minority of the other servers will go tharn until the "partition" is healed. That isn't what you want (at all). The ideas for federating ZK or allowing observers would likely do what you want. I can imagine that an observer would only care that it can see it's local peers and one of the observers would be elected to get updates (and thus would care about the central service). On Fri, Jul 24, 2009 at 3:32 PM, Todd Greenwood <to...@audiencescience.com>wrote: > Like most folks, our WAN is composed of various zones, some central > processing, some edge, some corp, and some in between (DMZs). In this > model, a given Zookeeper server will not have direct connectivity to all > of it's peers in the ensemble due to various security constraints. Is > this a problem? Are there special configurations for this model? > > Given 3 Zones > ------------- > > A <--> B > B <--> C > > A cannot see C, and vice versa. > B can see A and C. > > 1. Will zookeeper servers function properly even if a given set of > servers can only see some of the servers in the ensemble? For example, > the shared config lists all zk servers in A, B, and C, but A can only > see B, C can only see B, and B can see both A and C. > > 2. Will zookeeper servers flood the log with error messages if only a > subset of the ensemble members are visible? > > 3. Will the zk ensemble function properly if the config used by each > server only lists the servers in the ensemble that are visible? Suppose > that A has a config that only list servers in A and B, C a config for C > and B, and B has a config that lists servers in A, B, and C. Is this the > recommended approach? > > http://hadoop.apache.org/zookeeper/docs/r3.1.1/zookeeperAdmin.html > -- Ted Dunning, CTO DeepDyve
