There are multiple issues around your setup which got fixed recently. The IllegalArgumentException at java.util.concurrent.ThreadPoolExecutor suggest you were hitting this issue: https://issues.apache.org/jira/browse/ZOOKEEPER-3758 This affects ZooKeeper 3.6.0 and we already fixed it, as Enrico mentioned. The 3.6.1 will solve this particular issue. Or you can also set the following config as a workaround: multiAddress.reachabilityCheckEnabled=false. (setting this won't be needed in 3.6.1, but in your case it will most probably help in 3.6.0) This is a 3.6 specific issue and 3.6 specific configuration parameter, you won't see this problem in 3.5.
However... you also using 0.0.0.0 in you server config, which is actually not recommended since 3.5. This leads to other error when peers wish to rejoin to the quorum (see https://issues.apache.org/jira/browse/ZOOKEEPER-2164). This was also fixed and the fix will be released in 3.5.8 and 3.6.1. As a workaround (and this is actually not only a workaround but the more consistent and dynamic re-config compatible way) you can use the following config in all the there servers: quorumListenOnAllIPs=true server.1=< fqdn of server 1 >:2888:3888 server.2=< fqdn of server 2 >:2888:3888 server.3=< fqdn of server 3 >:2888:3888 The 'quorumListenOnAllIPs' config above will have the same effect: all the servers will listen on 0.0.0.0 locally. But it has a benefit that all the members still have the same view of the cluster. And the re-join problem should not happen here. I hope these changes will help. Kind regards, Mate On Tue, Mar 24, 2020 at 2:06 AM rld244 <rld...@cornell.edu> wrote: > Thanks for getting back to me Enrico. > > I'm working within an AWS VPC with all nodes on a private subnet and I > think > Ping isn't enabled, but I can use nc and specific ports to verify that the > servers can talk to each other. > > Interestingly I just noticed that on the node with myid 3 zookeeper is > listening on 2888 and can be reached by the other nodes on 2181, 2888 and > 3888. > > I don't know if that helps. > > Maybe I'll try deploying 3.5.x instead. > > > > -- > Sent from: http://zookeeper-user.578899.n2.nabble.com/ >
