we got an other question in the user list just about this, and also there is a duplicate jira ticket opened 6 days ago: https://issues.apache.org/jira/browse/ZOOKEEPER-3725
On Wed, Feb 19, 2020 at 10:04 AM Szalay-Bekő Máté < szalay.beko.m...@gmail.com> wrote: > Thanks for all the comments! > > My gut feeling is that the quorumListenOnAllIPs should work usually, e.g. > even if there is proxy / tunneling between the servers. > > But I never operated ZooKeeper in a complex dockerized setup. Maybe you > don't want to bind on all IPs locally for security reasons (having separate > virtual networks client traffic and for quorum-internal traffic). For more > complex cases I like the distinction of "local bind address" and > "advertised address" proposed by Enrico. That gives more flexibility, while > still keeps the single view of the server address configuration what can be > distributed in the whole cluster with dynamic reconfig. > > If no one propose otherwise, then: > - I will start a thread on the user mail list to ask around if it is a > problem for many people and also to see if 'quorumListenOnAllIPs' is enough > for them, or we should prioritize a more generic approach. > - I think someone should ask the guys in the docker community to change > the example configuration here, https://hub.docker.com/_/zookeeper not > using 0.0.0.0, but include description of quorumListenOnAllIPs. Do we have > any contact to them? > - I still think for backward-compatibility reason it make sense to go > ahead with the PR (https://github.com/apache/zookeeper/pull/1254) as we > should support a working 3.4 config also in 3.5+, especially if the fix is > this trivial > > What do you think? > > Mate > > On Wed, Feb 19, 2020 at 6:30 AM Andor Molnar <an...@apache.org> wrote: > >> What does '0.0.0.0:2888:3888' mean exactly in terms of bindind? Listen >> on all available interfaces? >> >> In which case I think it would be better to advise using >> quorumListenOnAllIPs to customers instead. We could also detect and >> alert if configured with wildcard address. >> >> Andor >> >> >> >> On Tue, 2020-02-18 at 18:00 +0100, Enrico Olivelli wrote: >> > I feel it would be better to simply have a "local bind address" and >> > an >> > "advertised address" >> > >> > server.1=advertisedaddress:port:port.... >> > localBindAddress=0.0.0.0 >> > >> > it is clearer >> > Enrico >> > >> > Il giorno mar 18 feb 2020 alle ore 11:29 Szalay-Bekő Máté >> > <szalay.beko.m...@gmail.com> ha scritto: >> > > also, we have the same 0.0.0.0 config suggested in the description >> > > of the >> > > zookeeper 'docker official' image: >> > > https://hub.docker.com/_/zookeeper >> > > >> > > On Tue, Feb 18, 2020 at 10:59 AM Szalay-Bekő Máté < >> > > szalay.beko.m...@gmail.com> wrote: >> > > >> > > > A quote from the Jira ticket where Sebastian tries to explain >> > > > their setup: >> > > > (see >> > > > >> https://issues.apache.org/jira/browse/ZOOKEEPER-2164?focusedCommentId=17037941&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17037941 >> > > > ) >> > > > >> > > > "We are using 0.0.0.0 in our config of the 3-node-cluster as the >> > > > nodes are >> > > > running in separate docker-instances not directly connected to >> > > > each other. >> > > > Which means the zookeeper-container doesn't have its real IP >> > > > assigned only >> > > > using port-forwardings from the docker host to the container to >> > > > make it >> > > > accessible and so using the external IP or the FQDN doesn't allow >> > > > zookeeper >> > > > to start as it complains about not finding that IP in the >> > > > container. Which >> > > > is of course correct. Using the internal Network-IP instead of >> > > > 0.0.0.0 >> > > > would result in the same problem as using 0.0.0.0 as all internal >> > > > networks >> > > > of the three docker hosts are using the same network-range in >> > > > their >> > > > internal networks." >> > > > >> > > > re-reading it, I think he meant a bit different config than what >> > > > I wrote >> > > > as an example before... I think he basically connects separate >> > > > hosts with >> > > > tunnels maybe and running ZK in docker in each host without any >> > > > container >> > > > orchestration tool. (?) >> > > > >> > > > On Tue, Feb 18, 2020 at 10:49 AM Enrico Olivelli < >> > > > eolive...@gmail.com> >> > > > wrote: >> > > > >> > > > > Honestly I think that using 0.0.0.0 is not so useful. >> > > > > I find it very useful that the configuration is the same on >> > > > > every peer, >> > > > > except from myid file >> > > > > What's the real gain in such configuration? >> > > > > >> > > > > >> > > > > Enrico >> > > > > >> > > > > >> > > > > Il Mar 18 Feb 2020, 10:32 Szalay-Bekő Máté < >> > > > > szalay.beko.m...@gmail.com> >> > > > > ha >> > > > > scritto: >> > > > > >> > > > > > Hi All, >> > > > > > >> > > > > > in a recent PR I try to fix an issue we found with Suhas >> > > > > > Dantkale in >> > > > > > ZOOKEEPER-2146 (see >> > > > > > https://github.com/apache/zookeeper/pull/1254). The >> > > > > > problem is that in ZooKeeper 3.5+ some quorum members can not >> > > > > > rejoin to >> > > > > the >> > > > > > quorum after a restart if the server configs are set like >> > > > > > this: >> > > > > > >> > > > > > zoo.cfg in server 1: >> > > > > > server.1=0.0.0.0:2888:3888 >> > > > > > server.2=some.fqdn-2.com:2888:3888 >> > > > > > server.3=some.fqdn-3.com:2888:3888 >> > > > > > >> > > > > > zoo.cfg in server 2: >> > > > > > server.1=some.fqdn-1.com:2888:3888 >> > > > > > server.2=0.0.0.0:2888:3888 >> > > > > > server.3=some.fqdn-3.com:2888:3888 >> > > > > > >> > > > > > I am not exactly sure about the use case behind this config, >> > > > > > but people >> > > > > > claim they need it for specific dockerized environments (see >> > > > > > the >> > > > > comments >> > > > > > in the jira ticket). Is anyone familiar with such use cases? >> > > > > > We never >> > > > > used >> > > > > > such configs in production as far as I can tell. >> > > > > > >> > > > > > The above config worked without a problem in ZooKeeper 3.4.x, >> > > > > > but not >> > > > > > perfectly for 3.5.x. It would be logical to keep supporting >> > > > > > it. Still, I >> > > > > > think after the introduction of the dynamic reconfig, we kind >> > > > > > of assume >> > > > > > that all the servers have the same server address >> > > > > > configurations. So >> > > > > maybe >> > > > > > the config is not even valid anymore? >> > > > > > >> > > > > > Using the 'quorumListenOnAllIPs' config property instead the >> > > > > > 0.0.0.0 in >> > > > > the >> > > > > > configs might solve the issue. But if it is the case, then we >> > > > > > definitely >> > > > > > should highlight this in the wiki / documentation. Maybe even >> > > > > > printing >> > > > > out >> > > > > > a warning during ZooKeeper startup. >> > > > > > >> > > > > > What do you think? >> > > > > > >> > > > > > Kind regards, >> > > > > > Mate >> > > > > > >> >>
