it would be nice to make our test suite more stable even with many forks...
when I use docker to build, then I also saw this test to fail frequently
(one which was reported by Damien):
[ERROR]
testWatchAutoResetWithPending(org.apache.zookeeper.test.WatcherTest) Time
elapsed: 0.083 s <<< FAILURE!
java.lang.AssertionError: Unexpected bean exists! expected:<0> but
was:<1>
at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:84)
Also there is a C client test that fails almost every time for me from
docker (maybe the read-only test or the one testing child watchers? I'm not
sure right now)
Kind regards,
Mate
On Fri, Sep 4, 2020 at 1:48 PM Enrico Olivelli <eolive...@gmail.com> wrote:
> Damien
> you can use -Dsurefire-forkcount=1 in order to run only one test at a time.
> This should reduce flakiness
>
> Enrico
>
> Il giorno gio 3 set 2020 alle ore 21:37 Damien Diederen <
> ddiede...@sinenomine.net> ha scritto:
>
>>
>> Hi Enrico, all,
>>
>> TL;DR: Builds of 3.6.2 pass with 8 (!) cores on Ubuntu 18.04, 20.04, and
>> NixOS 20.03.
>>
>>
>> I wrote:
>>
>> >> It took me a number of tries, because that is a VM and the tests are
>> >> somewhat flaky in that environment.
>>
>> You suggested:
>>
>> > On 3.6.2 I don't see flaky tests in my local environment,
>> > can you please start another email thread with your problems ?
>> > They will deserve JIRA issues and investigations
>>
>> Okay; I did a bit more experimentation.
>>
>> As I mentioned, the builds which exhibited flakiness were run in a VM.
>> That was not a "weak" VM, however: KVM, 4 cores, 16 GiB, backed by an
>> Intel i7 CPU.
>>
>> I notably tried with an Ubuntu live DVD and builds on /tmp (tmpfs).
>> Using "only" 4 cores results in frequent failures, but allocating 8
>> cores to the VM seemingly allows builds to SUCCEED.
>>
>> I don't know if 8 cores is considered a reasonable requirement for the
>> test suite.
>>
>>
>> I also tried on a (real) NixOS laptop with 8 logical processors and 16
>> GiB of RAM; that build passed. Which means that so far, I have seen
>> builds succeed on:
>>
>> * Ubuntu 20.04;
>> * Ubuntu 18.04;
>> * NixOS 20.03.
>>
>>
>> In that previous email, I also wrote:
>>
>> >> this is, as far as I know, a long-standing issue and completely
>> >> unrelated to 3.6.2.
>>
>> I have experienced frequent failures when building 3.6.1 with tests on a
>> 6-core Xeon with 64 GiB RAM. That machine plays the role of a shared
>> "build server," so other users can cause varying utilization. But
>> still, it is no slouch, and utilization was relatively low when I
>> observed these issues—so I was a bit surprised to see such failures.
>>
>>
>> I guess I should try building master on these various configurations (I
>> haven't seen that kind of trouble there).
>>
>>
>> Best,
>> Damien
>>
>>
>> P.-S. — For the record, here is the procedure I used (starting from the
>> bare live Ubuntu DVD):
>>
>> sudo add-apt-repository universe
>>
>> sudo apt install \
>> autoconf \
>> build-essential \
>> git \
>> libcppunit-dev \
>> libsasl2-dev \
>> libtool-bin \
>> maven \
>> openjdk-11-jdk-headless \
>> pkg-config
>>
>> rsync -av --delete /mnt/zookeeper/ /tmp/zookeeper/
>> cd /tmp/zookeeper
>>
>> export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
>>
>> LC_ALL=C mvn install -Pfull-build 2>&1 |
>> tee /mnt/ubuntu-mvn-tests.log
>>
>>
>> And here is an example of failure—clearly a time out:
>>
>> [ERROR]
>> testJvmPauseMonitorExceedWarnThreshold(org.apache.zookeeper.server.util.JvmPauseMonitorTest)
>> Time elapsed: 5.119 s <<< ERROR!
>> org.junit.runners.model.TestTimedOutException: test timed out after
>> 5000 milliseconds
>> at
>> app//org.apache.zookeeper.server.util.JvmPauseMonitorTest.testJvmPauseMonitorExceedWarnThreshold(JvmPauseMonitorTest.java:54)
>>
>>
>> Here is another example, somewhat more murky (possibly related to some
>> kind of race condition):
>>
>> [ERROR]
>> testWatchAutoResetWithPending(org.apache.zookeeper.test.WatcherTest) Time
>> elapsed: 0.083 s <<< FAILURE!
>> java.lang.AssertionError: Unexpected bean exists! expected:<0> but
>> was:<1>
>> at
>> org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:84)
>>
>>
>> I can provide more logs if we decide to open a ticket to track this topic.
>>
>>
>>
>>
>> >> Il giorno mer 2 set 2020 alle ore 20:42 Damien Diederen <
>> >> ddiede...@sinenomine.net> ha scritto:
>> >>
>> >>>
>> >>> Hi Enrico, all,
>> >>>
>> >>> I was also able to build and successfully run the tests of this
>> release
>> >>> candidate on Ubuntu 20.04.1, with the provided Java & Maven:
>> >>>
>> >>> $ grep VERSION= /etc/os-release
>> >>> VERSION="20.04.1 LTS (Focal Fossa)"
>> >>> $ java -version
>> >>> openjdk version "11.0.8" 2020-07-14
>> >>> $ mvn -version
>> >>> Apache Maven 3.6.3
>> >>>
>> >>> (It took me a number of tries, because that is a VM and the tests are
>> >>> somewhat flaky in that environment. But this is, as far as I know, a
>> >>> long-standing issue and completely unrelated to 3.6.2. Please let me
>> >>> know if you have tips/tricks for avoiding such temporary failures.)
>> >>>
>> >>> Cheers, -D
>> >>>
>> >>>
>> >>>
>> >>> Szalay-Bekő Máté <szalay.beko.m...@gmail.com> writes:
>> >>> > +1 (non-binding)
>> >>> >
>> >>> > - I built the source code (-Pfull-build) on Ubuntu 18.04.3 using
>> OpenJDK
>> >>> > 8u242, OpenJDK 11.0.8 and maven 3.6.0.
>> >>> > - all the unit tests passed (both Java and C-client).
>> >>> > - I also built and executed unit tests for zkpython
>> >>> > - checkstyle and spotbugs passed
>> >>> > - apache-rat passed
>> >>> > - owasp (CVE check) passed
>> >>> > - fatjar built (-Pfatjar)
>> >>> >
>> >>> > On Tue, Sep 1, 2020 at 11:35 AM Enrico Olivelli <
>> eolive...@gmail.com>
>> >>> wrote:
>> >>> >
>> >>> >> This is a release candidate for 3.6.2.
>> >>> >>
>> >>> >> It is a minor release and it fixes a few critical issues and
>> brings a
>> >>> few
>> >>> >> dependencies upgrades.
>> >>> >>
>> >>> >> The full release notes is available at:
>> >>> >>
>> >>> >>
>> >>>
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801&version=12347809
>> >>> >>
>> >>> >> *** Please download, test and vote by September 4th 2020, 23:59
>> UTC+0.
>> >>> ***
>> >>> >>
>> >>> >> Source files:
>> >>> >> https://people.apache.org/~eolivelli/zookeeper-3.6.2-candidate-0/
>> >>> >>
>> >>> >> Maven staging repo:
>> >>> >>
>> >>>
>> https://repository.apache.org/content/repositories/orgapachezookeeper-1060/
>> >>> >>
>> >>> >> The release candidate tag in git to be voted upon: release-3.6.2-0
>> >>> >> https://github.com/apache/zookeeper/tree/release-3.6.2-0
>> >>> >>
>> >>> >> ZooKeeper's KEYS file containing PGP keys we use to sign the
>> release:
>> >>> >> https://www.apache.org/dist/zookeeper/KEYS
>> >>> >>
>> >>> >> The staging version of the website is:
>> >>> >>
>> >>>
>> https://people.apache.org/~eolivelli/zookeeper-3.6.2-candidate-0/website/
>> >>> >>
>> >>> >> Should we release this candidate?
>> >>> >>
>> >>> >> Enrico Olivelli
>> >>> >>
>> >>>
>>
>
