Thanks for all the comments! My gut feeling is that the quorumListenOnAllIPs should work usually, e.g. even if there is proxy / tunneling between the servers.
But I never operated ZooKeeper in a complex dockerized setup. Maybe you don't want to bind on all IPs locally for security reasons (having separate virtual networks client traffic and for quorum-internal traffic). For more complex cases I like the distinction of "local bind address" and "advertised address" proposed by Enrico. That gives more flexibility, while still keeps the single view of the server address configuration what can be distributed in the whole cluster with dynamic reconfig. If no one propose otherwise, then: - I will start a thread on the user mail list to ask around if it is a problem for many people and also to see if 'quorumListenOnAllIPs' is enough for them, or we should prioritize a more generic approach. - I think someone should ask the guys in the docker community to change the example configuration here, https://hub.docker.com/_/zookeeper not using 0.0.0.0, but include description of quorumListenOnAllIPs. Do we have any contact to them? - I still think for backward-compatibility reason it make sense to go ahead with the PR (https://github.com/apache/zookeeper/pull/1254) as we should support a working 3.4 config also in 3.5+, especially if the fix is this trivial What do you think? Mate On Wed, Feb 19, 2020 at 6:30 AM Andor Molnar <an...@apache.org> wrote: > What does '0.0.0.0:2888:3888' mean exactly in terms of bindind? Listen > on all available interfaces? > > In which case I think it would be better to advise using > quorumListenOnAllIPs to customers instead. We could also detect and > alert if configured with wildcard address. > > Andor > > > > On Tue, 2020-02-18 at 18:00 +0100, Enrico Olivelli wrote: > > I feel it would be better to simply have a "local bind address" and > > an > > "advertised address" > > > > server.1=advertisedaddress:port:port.... > > localBindAddress=0.0.0.0 > > > > it is clearer > > Enrico > > > > Il giorno mar 18 feb 2020 alle ore 11:29 Szalay-Bekő Máté > > <szalay.beko.m...@gmail.com> ha scritto: > > > also, we have the same 0.0.0.0 config suggested in the description > > > of the > > > zookeeper 'docker official' image: > > > https://hub.docker.com/_/zookeeper > > > > > > On Tue, Feb 18, 2020 at 10:59 AM Szalay-Bekő Máté < > > > szalay.beko.m...@gmail.com> wrote: > > > > > > > A quote from the Jira ticket where Sebastian tries to explain > > > > their setup: > > > > (see > > > > > https://issues.apache.org/jira/browse/ZOOKEEPER-2164?focusedCommentId=17037941&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17037941 > > > > ) > > > > > > > > "We are using 0.0.0.0 in our config of the 3-node-cluster as the > > > > nodes are > > > > running in separate docker-instances not directly connected to > > > > each other. > > > > Which means the zookeeper-container doesn't have its real IP > > > > assigned only > > > > using port-forwardings from the docker host to the container to > > > > make it > > > > accessible and so using the external IP or the FQDN doesn't allow > > > > zookeeper > > > > to start as it complains about not finding that IP in the > > > > container. Which > > > > is of course correct. Using the internal Network-IP instead of > > > > 0.0.0.0 > > > > would result in the same problem as using 0.0.0.0 as all internal > > > > networks > > > > of the three docker hosts are using the same network-range in > > > > their > > > > internal networks." > > > > > > > > re-reading it, I think he meant a bit different config than what > > > > I wrote > > > > as an example before... I think he basically connects separate > > > > hosts with > > > > tunnels maybe and running ZK in docker in each host without any > > > > container > > > > orchestration tool. (?) > > > > > > > > On Tue, Feb 18, 2020 at 10:49 AM Enrico Olivelli < > > > > eolive...@gmail.com> > > > > wrote: > > > > > > > > > Honestly I think that using 0.0.0.0 is not so useful. > > > > > I find it very useful that the configuration is the same on > > > > > every peer, > > > > > except from myid file > > > > > What's the real gain in such configuration? > > > > > > > > > > > > > > > Enrico > > > > > > > > > > > > > > > Il Mar 18 Feb 2020, 10:32 Szalay-Bekő Máté < > > > > > szalay.beko.m...@gmail.com> > > > > > ha > > > > > scritto: > > > > > > > > > > > Hi All, > > > > > > > > > > > > in a recent PR I try to fix an issue we found with Suhas > > > > > > Dantkale in > > > > > > ZOOKEEPER-2146 (see > > > > > > https://github.com/apache/zookeeper/pull/1254). The > > > > > > problem is that in ZooKeeper 3.5+ some quorum members can not > > > > > > rejoin to > > > > > the > > > > > > quorum after a restart if the server configs are set like > > > > > > this: > > > > > > > > > > > > zoo.cfg in server 1: > > > > > > server.1=0.0.0.0:2888:3888 > > > > > > server.2=some.fqdn-2.com:2888:3888 > > > > > > server.3=some.fqdn-3.com:2888:3888 > > > > > > > > > > > > zoo.cfg in server 2: > > > > > > server.1=some.fqdn-1.com:2888:3888 > > > > > > server.2=0.0.0.0:2888:3888 > > > > > > server.3=some.fqdn-3.com:2888:3888 > > > > > > > > > > > > I am not exactly sure about the use case behind this config, > > > > > > but people > > > > > > claim they need it for specific dockerized environments (see > > > > > > the > > > > > comments > > > > > > in the jira ticket). Is anyone familiar with such use cases? > > > > > > We never > > > > > used > > > > > > such configs in production as far as I can tell. > > > > > > > > > > > > The above config worked without a problem in ZooKeeper 3.4.x, > > > > > > but not > > > > > > perfectly for 3.5.x. It would be logical to keep supporting > > > > > > it. Still, I > > > > > > think after the introduction of the dynamic reconfig, we kind > > > > > > of assume > > > > > > that all the servers have the same server address > > > > > > configurations. So > > > > > maybe > > > > > > the config is not even valid anymore? > > > > > > > > > > > > Using the 'quorumListenOnAllIPs' config property instead the > > > > > > 0.0.0.0 in > > > > > the > > > > > > configs might solve the issue. But if it is the case, then we > > > > > > definitely > > > > > > should highlight this in the wiki / documentation. Maybe even > > > > > > printing > > > > > out > > > > > > a warning during ZooKeeper startup. > > > > > > > > > > > > What do you think? > > > > > > > > > > > > Kind regards, > > > > > > Mate > > > > > > > >
