Damien you can use -Dsurefire-forkcount=1 in order to run only one test at a time. This should reduce flakiness
Enrico Il giorno gio 3 set 2020 alle ore 21:37 Damien Diederen < ddiede...@sinenomine.net> ha scritto: > > Hi Enrico, all, > > TL;DR: Builds of 3.6.2 pass with 8 (!) cores on Ubuntu 18.04, 20.04, and > NixOS 20.03. > > > I wrote: > > >> It took me a number of tries, because that is a VM and the tests are > >> somewhat flaky in that environment. > > You suggested: > > > On 3.6.2 I don't see flaky tests in my local environment, > > can you please start another email thread with your problems ? > > They will deserve JIRA issues and investigations > > Okay; I did a bit more experimentation. > > As I mentioned, the builds which exhibited flakiness were run in a VM. > That was not a "weak" VM, however: KVM, 4 cores, 16 GiB, backed by an > Intel i7 CPU. > > I notably tried with an Ubuntu live DVD and builds on /tmp (tmpfs). > Using "only" 4 cores results in frequent failures, but allocating 8 > cores to the VM seemingly allows builds to SUCCEED. > > I don't know if 8 cores is considered a reasonable requirement for the > test suite. > > > I also tried on a (real) NixOS laptop with 8 logical processors and 16 > GiB of RAM; that build passed. Which means that so far, I have seen > builds succeed on: > > * Ubuntu 20.04; > * Ubuntu 18.04; > * NixOS 20.03. > > > In that previous email, I also wrote: > > >> this is, as far as I know, a long-standing issue and completely > >> unrelated to 3.6.2. > > I have experienced frequent failures when building 3.6.1 with tests on a > 6-core Xeon with 64 GiB RAM. That machine plays the role of a shared > "build server," so other users can cause varying utilization. But > still, it is no slouch, and utilization was relatively low when I > observed these issues—so I was a bit surprised to see such failures. > > > I guess I should try building master on these various configurations (I > haven't seen that kind of trouble there). > > > Best, > Damien > > > P.-S. — For the record, here is the procedure I used (starting from the > bare live Ubuntu DVD): > > sudo add-apt-repository universe > > sudo apt install \ > autoconf \ > build-essential \ > git \ > libcppunit-dev \ > libsasl2-dev \ > libtool-bin \ > maven \ > openjdk-11-jdk-headless \ > pkg-config > > rsync -av --delete /mnt/zookeeper/ /tmp/zookeeper/ > cd /tmp/zookeeper > > export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 > > LC_ALL=C mvn install -Pfull-build 2>&1 | > tee /mnt/ubuntu-mvn-tests.log > > > And here is an example of failure—clearly a time out: > > [ERROR] > testJvmPauseMonitorExceedWarnThreshold(org.apache.zookeeper.server.util.JvmPauseMonitorTest) > Time elapsed: 5.119 s <<< ERROR! > org.junit.runners.model.TestTimedOutException: test timed out after > 5000 milliseconds > at > app//org.apache.zookeeper.server.util.JvmPauseMonitorTest.testJvmPauseMonitorExceedWarnThreshold(JvmPauseMonitorTest.java:54) > > > Here is another example, somewhat more murky (possibly related to some > kind of race condition): > > [ERROR] > testWatchAutoResetWithPending(org.apache.zookeeper.test.WatcherTest) Time > elapsed: 0.083 s <<< FAILURE! > java.lang.AssertionError: Unexpected bean exists! expected:<0> but > was:<1> > at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:84) > > > I can provide more logs if we decide to open a ticket to track this topic. > > > > > >> Il giorno mer 2 set 2020 alle ore 20:42 Damien Diederen < > >> ddiede...@sinenomine.net> ha scritto: > >> > >>> > >>> Hi Enrico, all, > >>> > >>> I was also able to build and successfully run the tests of this release > >>> candidate on Ubuntu 20.04.1, with the provided Java & Maven: > >>> > >>> $ grep VERSION= /etc/os-release > >>> VERSION="20.04.1 LTS (Focal Fossa)" > >>> $ java -version > >>> openjdk version "11.0.8" 2020-07-14 > >>> $ mvn -version > >>> Apache Maven 3.6.3 > >>> > >>> (It took me a number of tries, because that is a VM and the tests are > >>> somewhat flaky in that environment. But this is, as far as I know, a > >>> long-standing issue and completely unrelated to 3.6.2. Please let me > >>> know if you have tips/tricks for avoiding such temporary failures.) > >>> > >>> Cheers, -D > >>> > >>> > >>> > >>> Szalay-Bekő Máté <szalay.beko.m...@gmail.com> writes: > >>> > +1 (non-binding) > >>> > > >>> > - I built the source code (-Pfull-build) on Ubuntu 18.04.3 using > OpenJDK > >>> > 8u242, OpenJDK 11.0.8 and maven 3.6.0. > >>> > - all the unit tests passed (both Java and C-client). > >>> > - I also built and executed unit tests for zkpython > >>> > - checkstyle and spotbugs passed > >>> > - apache-rat passed > >>> > - owasp (CVE check) passed > >>> > - fatjar built (-Pfatjar) > >>> > > >>> > On Tue, Sep 1, 2020 at 11:35 AM Enrico Olivelli <eolive...@gmail.com > > > >>> wrote: > >>> > > >>> >> This is a release candidate for 3.6.2. > >>> >> > >>> >> It is a minor release and it fixes a few critical issues and brings > a > >>> few > >>> >> dependencies upgrades. > >>> >> > >>> >> The full release notes is available at: > >>> >> > >>> >> > >>> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801&version=12347809 > >>> >> > >>> >> *** Please download, test and vote by September 4th 2020, 23:59 > UTC+0. > >>> *** > >>> >> > >>> >> Source files: > >>> >> https://people.apache.org/~eolivelli/zookeeper-3.6.2-candidate-0/ > >>> >> > >>> >> Maven staging repo: > >>> >> > >>> > https://repository.apache.org/content/repositories/orgapachezookeeper-1060/ > >>> >> > >>> >> The release candidate tag in git to be voted upon: release-3.6.2-0 > >>> >> https://github.com/apache/zookeeper/tree/release-3.6.2-0 > >>> >> > >>> >> ZooKeeper's KEYS file containing PGP keys we use to sign the > release: > >>> >> https://www.apache.org/dist/zookeeper/KEYS > >>> >> > >>> >> The staging version of the website is: > >>> >> > >>> > https://people.apache.org/~eolivelli/zookeeper-3.6.2-candidate-0/website/ > >>> >> > >>> >> Should we release this candidate? > >>> >> > >>> >> Enrico Olivelli > >>> >> > >>> >
