-1 the release artifact looks good generally, however it's failing my manual testing.
I have been running the following test for every release for quite a few years - this tool was initially created in 2009 specifically to enable this type of validation: https://github.com/phunt/zkconf $ python ./zkconf.py --count 23 ~/Downloads/z/apache-zookeeper-3.6.0-bin test23 and then "start.sh" in test23 directory. This starts a ZK cluster with 23 ensemble members. This test has never failed that I can remember - however it's failing with this RC (fwiw I ran it for 3.5.6 most recently and it passed). Note: this passes on 3.6.0rc codebase with 3 members to the quorum created via: $ python ./zkconf.py --count 3 ~/Downloads/z/parent-3.6.0 test3 >From what I see in the test23 ensemble logs there is a warning happening frequently: 2020-01-15 20:02:13,431 [myid:13] - WARN [ListenerHandler-phunt-MBP13.local/192.168.1.91:4193:QuorumCnxManager@691] - None of the addresses (/192.168.1.91:4190) are reachable for sid 10 java.net.NoRouteToHostException: No valid address among [/192.168.1.91:4190] which is new code added in 3.6.0 - ZOOKEEPER-3188. I do not see this warning with the 3 node ensemble using the same codebase. Patrick On Wed, Jan 15, 2020 at 7:45 PM Patrick Hunt <[email protected]> wrote: > wrt the addition of lib - we've had to move to explicitly telling users > _not_ to use the client binaries except as learning aids > > "NOTE: the ZooKeeper C client shell (cli_st and cli_mt) is meant as a > example/sample of ZooKeeper C client API usage. It is not a full > fledged client and not meant for production usage - see the Java > client shell for a fully featured shell." > > due to repeated security issues being raised and attempting to reduce the > burden security reports impose - the libs are fine, but the C clients > themselves were not created with security in mind and users should use the > Java CLIs instead. We could fix that, I assume, but afaik so far that > hasn't been done (harden the c cli/load binaries). > > We shouldn't include the bin/* IMO, although having an artifact for the > libs themselves is fine. > > > I'm sure we all know this, but I also want to clarify that the _source_ is > what we are releasing - the bin/lib/etc... are all "conveniences" > https://www.apache.org/dev/release-publishing.html#valid > language such as "we are releasing a new binary package with the C client > (with OpenSSL support)" concerns me a bit as the binaries are not the > release. Sorry for the nit picking but it is important for the community to > follow and understand established Apache guidance on this. > > Patrick > > > On Wed, Jan 15, 2020 at 7:31 PM Patrick Hunt <[email protected]> wrote: > >> Unzipping the source on mac "unzip apache-zookeeper-3.6.0.zip" results in >> a tld of "parent-3.6.0" - can we fix this? The other artifacts de-archive >> correctly, expect this one. It seems a shame. Given this issue and the >> tar.gz->zip - perhaps we should reconsider and just generate the source >> ourselves? Or is there a way to ensure the tld name is correct? >> (setting/config?) >> >> Patrick >> >> On Wed, Jan 15, 2020 at 8:05 AM Enrico Olivelli <[email protected]> >> wrote: >> >>> Alexander, >>> I have pasted a wrong link in the VOTE email, I am sorry >>> >>> The good link is >>> >>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801&version=12326518 >>> >>> You can also see the staged released notes in the website >>> >>> https://people.apache.org/~eolivelli/zookeeper-3.6.0-candidate-0/website/releasenotes.html >>> >>> Thank you so much for reporting this issue >>> >>> Happy testing >>> >>> Enrico >>> >>> Il giorno mer 15 gen 2020 alle ore 16:55 Alexander Shraer < >>> [email protected]> >>> ha scritto: >>> >>> > Hi Enrico, >>> > >>> > Thank you for driving this release! >>> > >>> > I have a question - i believe that Zookeeper-2024 (an order of >>> magnitude >>> > throughput improvement for mixed workloads) is part of the 3.6.0 >>> release, >>> > but it isn't mentioned in the release notes or the summary. >>> > Could you please clarify ? >>> > >>> > Thanks, >>> > Alex >>> > >>> > >>> > On Wed, Jan 15, 2020 at 7:29 AM Flavio Junqueira <[email protected]> >>> wrote: >>> > >>> > > I can't parse Rudy's message, is it an issue with my mail >>> application? >>> > > >>> > > -Flavio >>> > > >>> > > > On 15 Jan 2020, at 15:00, rudy_steiner <[email protected]> >>> wrote: >>> > > > >>> > > > environment: <br/> * MacOS High Sierra 10.13.1 <br/> * JDK >>> > > 1.8.0_172<br/><br/>I try to run junit test on branch-3.6, and unit >>> test >>> > > thread get stuck, log as follows:<br/><br/>.....<br/>INFO] Running >>> > > org.apache.zookeeper.common.X509UtilTest<br/>[INFO] Tests run: 3, >>> > Failures: >>> > > 0, Errors: 0, Skipped: 0, Time elapsed: 27.797 s - in >>> > > org.apache.zookeeper.server.SnapshotDigestTest<br/>[INFO] Running >>> > > org.apache.zookeeper.common.TimeTest<br/>[INFO] Tests run: 1, >>> Failures: >>> > 0, >>> > > Errors: 0, Skipped: 0, Time elapsed: 0.718 s - in >>> > > org.apache.zookeeper.common.TimeTest<br/>[INFO] Tests run: 352, >>> Failures: >>> > > 0, Errors: 0, Skipped: 0, Time elapsed: 7.425 s - in >>> > > org.apache.zookeeper.common.X509UtilTest<br/>[INFO] Running >>> > > org.apache.zookeeper.common.PEMFileLoaderTest<br/>[INFO] Running >>> > > org.apache.zookeeper.common.KeyStoreFileTypeTest<br/>[INFO] Tests >>> run: 9, >>> > > Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.144 s - in >>> > > org.apache.zookeeper.common.KeyStoreFileTypeTest<br/>[INFO] Running >>> > > org.apache.zookeeper.audit.AuditEventTest<br/>[INFO] Tests run: 2, >>> > > Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.084 s - in >>> > > org.apache.zookeeper.audit.AuditEventTest<br/>[INFO] Running >>> > > org.apache.zookeeper.audit.StandaloneServerAuditTest<br/>[INFO] Tests >>> > run: >>> > > 72, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.027 s - in >>> > > org.apache.zookeeper.common.PEMFileLoaderTest<br/>[INFO] Tests run: >>> 5, >>> > > Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.197 s - in >>> > > org.apache.zookeeper.common.FileChangeWatcherTest<br/>[INFO] Tests >>> run: >>> > 1, >>> > > Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.755 s - in >>> > > org.apache.zookeeper.audit.StandaloneServerAuditTest<br/>[INFO] >>> Running >>> > > org.apache.zookeeper.audit.Log4jAuditLoggerTest<br/>[INFO] Running >>> > > org.apache.zookeeper.ZKUtilTest<br/>[ERROR] Tests run: 4, Failures: >>> 1, >>> > > Errors: 0, Skipped: 0, Time elapsed: 0.194 s <<< FAILURE! - >>> in >>> > > org.apache.zookeeper.ZKUtilTest<br/>[ERROR] >>> > > testUnreadableFileInput(org.apache.zookeeper.ZKUtilTest) Time >>> elapsed: >>> > > 0.014 s <<< FAILURE!<br/>java.lang.AssertionError<br/> >>> at >>> > > >>> > >>> org.apache.zookeeper.ZKUtilTest.testUnreadableFileInput(ZKUtilTest.java:83)<br/><br/>[INFO] >>> > > Running org.apache.zookeeper.PortAssignmentTest<br/>[INFO] Tests >>> run: 13, >>> > > Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.157 s - in >>> > > org.apache.zookeeper.PortAssignmentTest<br/>[INFO] Running >>> > > org.apache.zookeeper.VerGenTest<br/>[INFO] Tests run: 6, Failures: 0, >>> > > Errors: 0, Skipped: 0, Time elapsed: 1.747 s - in >>> > > org.apache.zookeeper.audit.Log4jAuditLoggerTest<br/>[INFO] Tests >>> run: 14, >>> > > Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.327 s - in >>> > > org.apache.zookeeper.VerGenTest<br/>[INFO] Running >>> > > org.apache.zookeeper.ZooKeeperTest<br/>[INFO] Running >>> > > org.apache.zookeeper.GetAllChildrenNumberTest<br/>[INFO] Running >>> > > org.apache.zookeeper.RemoveWatchesCmdTest<br/>[INFO] Tests run: 2, >>> > > Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.511 s - in >>> > > org.apache.zookeeper.GetAllChildrenNumberTest<br/>[INFO] Running >>> > > org.apache.zookeeper.ClientRequestTimeoutTest<br/>[INFO] Tests run: >>> 7, >>> > > Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.004 s - in >>> > > org.apache.zookeeper.RemoveWatchesCmdTest<br/>[INFO] Running >>> > > org.apache.zookeeper.ClientCanonicalizeTest<br/>[INFO] Tests run: 4, >>> > > Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.861 s - in >>> > > org.apache.zookeeper.ClientCanonicalizeTest<br/>[INFO] Running >>> > > org.apache.zookeeper.client.ZKClientConfigTest<br/>[INFO] Tests run: >>> 5, >>> > > Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.155 s - in >>> > > org.apache.zookeeper.client.ZKClientConfigTest<br/>[INFO] Tests run: >>> 35, >>> > > Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.74 s - in >>> > > org.apache.zookeeper.ZooKeeperTest<br/>[INFO] Tests run: 1, >>> Failures: 0, >>> > > Errors: 0, Skipped: 0, Time elapsed: 16.372 s - in >>> > > org.apache.zookeeper.ClientRequestTimeoutTest<br/>[INFO] Tests run: >>> 46, >>> > > Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 61.592 s - in >>> > > org.apache.zookeeper.RemoveWatchesTest<br/>[INFO] Tests run: 24, >>> > Failures: >>> > > 0, Errors: 0, Skipped: 0, Time elapsed: 166.152 s - in >>> > > org.apache.zookeeper.server.quorum.QuorumPeerMainTest<br/>[INFO] >>> Tests >>> > run: >>> > > 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 158.386 s - in >>> > > org.apache.zookeeper.server.quorum.ReconfigRecoveryTest<br/>[INFO] >>> Tests >>> > > run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 416.635 s >>> - in >>> > > org.apache.zookeeper.server.quorum.QuorumSSLTest<br/>..... >>> > > <br/><br/><br/>and I found serveral processes by ps -ef|grep java >>> > > :<br/><br/> 0 6809 87919 0 9:28下午 ?? 2:13.75 >>> > > >>> /Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home/bin/java >>> > > -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:62202 >>> > ,suspend=y,server=n >>> > > -Dvisualvm.id=962684532457553 >>> > > -Dmaven.multiModuleProjectDirectory=/export/workspace/zookeeper >>> > > -Dmaven.home=/Applications/IntelliJ >>> > > IDEA.app/Contents/plugins/maven/lib/maven3 >>> > > -Dclassworlds.conf=/Applications/IntelliJ >>> > > IDEA.app/Contents/plugins/maven/lib/maven3/bin/m2.conf >>> > > >>> > >>> -javaagent:/var/root/Library/Caches/IntelliJIdea2018.2/captureAgent/debugger-agent.jar=file:/private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/capture.props >>> > > -Dfile.encoding=UTF-8 -classpath /Applications/IntelliJ >>> > > >>> > >>> IDEA.app/Contents/plugins/maven/lib/maven3/boot/plexus-classworlds-2.5.2.jar:/Applications/IntelliJ >>> > > IDEA.app/Contents/lib/idea_rt.jar org.codehaus.classworlds.Launcher >>> > > -Didea.version=2018.2.4 test<br/> 0 11582 6809 0 9:35下午 ?? >>> > > 0:00.01 /bin/sh -c cd /export/workspace/zookeeper/zookeeper-server >>> > > && >>> > > >>> > >>> /Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home/jre/bin/java >>> > > -Xmx512m -Dtest.junit.threads=8 -Dzookeeper.junit.threadid=5 >>> > > >>> > >>> -javaagent:/var/root/.m2/repository/org/jmockit/jmockit/1.48/jmockit-1.48.jar >>> > > -jar >>> > > >>> > >>> /export/workspace/zookeeper/zookeeper-server/target/surefire/surefirebooter3509879420266754639.jar >>> > > /export/workspace/zookeeper/zookeeper-server/target/surefire >>> > > 2020-01-15T21-28-59_364-jvmRun5 surefire3587429514371015586tmp >>> > > surefire_2145165207885893729779tmp<br/> 0 11583 11582 0 9:35下午 >>> ?? >>> > > 0:05.16 >>> > > >>> > >>> /Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home/jre/bin/java >>> > > -Xmx512m -Dtest.junit.threads=8 -Dzookeeper.junit.threadid=5 >>> > > >>> > >>> -javaagent:/var/root/.m2/repository/org/jmockit/jmockit/1.48/jmockit-1.48.jar >>> > > -jar >>> > > >>> > >>> /export/workspace/zookeeper/zookeeper-server/target/surefire/surefirebooter3509879420266754639.jar >>> > > /export/workspace/zookeeper/zookeeper-server/target/surefire >>> > > 2020-01-15T21-28-59_364-jvmRun5 surefire3587429514371015586tmp >>> > > surefire_2145165207885893729779tmp<br/><br/><br/>I suspect the >>> processes >>> > > for unit test not close correctly, lead successor test stuck >>> > > > At 2020-01-15 07:18:25, "Enrico Olivelli" <[email protected]> >>> wrote: >>> > > >> This is a release candidate for 3.6.0. >>> > > >> >>> > > >> It is a major release and it introduces a lot of new features, >>> most >>> > > notably: >>> > > >> - Built-in data consistency check inside ZooKeeper >>> > > >> - Allow Followers to host Observers >>> > > >> - A new feature proposal to ZooKeeper: authentication enforcement >>> > > >> - Pluggable metrics system for ZooKeeper (and Prometheus.io >>> > integration) >>> > > >> - TLS Port unification >>> > > >> - Audit logging in ZooKeeper servers >>> > > >> - Improve resilience to network (advertise multiple addresses for >>> > > members >>> > > >> of a Zookeeper cluster) >>> > > >> - Persistent Recursive Watches >>> > > >> - add an API and the corresponding CLI to get total count of >>> recursive >>> > > sub >>> > > >> nodes under a specific path >>> > > >> >>> > > >> The full release notes is available at: >>> > > >> >>> > > >> >>> > > >>> > >>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801&version=12346617 >>> > > >> >>> > > >> *** Please download, test and vote by January 19th 2020, 23:59 >>> UTC+0. >>> > > *** >>> > > >> >>> > > >> Source files: >>> > > >> https://people.apache.org/~eolivelli/zookeeper-3.6.0-candidate-0/ >>> > > >> >>> > > >> Maven staging repo: >>> > > >> >>> > > >>> > >>> https://repository.apache.org/content/repositories/orgapachezookeeper-1045/ >>> > > >> >>> > > >> The release candidate tag in git to be voted upon: release-3.6.0-0 >>> > > >> https://github.com/apache/zookeeper/tree/release-3.6.0-0 >>> > > >> >>> > > >> ZooKeeper's KEYS file containing PGP keys we use to sign the >>> release: >>> > > >> https://www.apache.org/dist/zookeeper/KEYS >>> > > >> >>> > > >> Please note that this new major release introduces these change >>> in the >>> > > >> staging area: >>> > > >> - the source artifacts package is in zip format >>> > > >> - we are releasing a new binary package with the C client (with >>> > OpenSSL >>> > > >> support) >>> > > >> - website preview >>> > > >> >>> > > >> We are also adding a new JAR: zookeeper-metrics-providers >>> > > >> >>> > > >> The staging version of the website is: >>> > > >> >>> > > >>> > >>> https://people.apache.org/~eolivelli/zookeeper-3.6.0-candidate-0/website/ >>> > > >> >>> > > >> Should we release this candidate? >>> > > >> >>> > > >> Enrico Olivelli >>> > > >>> > > >>> > >>> >>
