Hi Enrico, all,
TL;DR: Builds of 3.6.2 pass with 8 (!) cores on Ubuntu 18.04, 20.04, and
NixOS 20.03.
I wrote:
>> It took me a number of tries, because that is a VM and the tests are
>> somewhat flaky in that environment.
You suggested:
> On 3.6.2 I don't see flaky tests in my local environment,
> can you please start another email thread with your problems ?
> They will deserve JIRA issues and investigations
Okay; I did a bit more experimentation.
As I mentioned, the builds which exhibited flakiness were run in a VM.
That was not a "weak" VM, however: KVM, 4 cores, 16 GiB, backed by an
Intel i7 CPU.
I notably tried with an Ubuntu live DVD and builds on /tmp (tmpfs).
Using "only" 4 cores results in frequent failures, but allocating 8
cores to the VM seemingly allows builds to SUCCEED.
I don't know if 8 cores is considered a reasonable requirement for the
test suite.
I also tried on a (real) NixOS laptop with 8 logical processors and 16
GiB of RAM; that build passed. Which means that so far, I have seen
builds succeed on:
* Ubuntu 20.04;
* Ubuntu 18.04;
* NixOS 20.03.
In that previous email, I also wrote:
>> this is, as far as I know, a long-standing issue and completely
>> unrelated to 3.6.2.
I have experienced frequent failures when building 3.6.1 with tests on a
6-core Xeon with 64 GiB RAM. That machine plays the role of a shared
"build server," so other users can cause varying utilization. But
still, it is no slouch, and utilization was relatively low when I
observed these issues—so I was a bit surprised to see such failures.
I guess I should try building master on these various configurations (I
haven't seen that kind of trouble there).
Best,
Damien
P.-S. — For the record, here is the procedure I used (starting from the
bare live Ubuntu DVD):
sudo add-apt-repository universe
sudo apt install \
autoconf \
build-essential \
git \
libcppunit-dev \
libsasl2-dev \
libtool-bin \
maven \
openjdk-11-jdk-headless \
pkg-config
rsync -av --delete /mnt/zookeeper/ /tmp/zookeeper/
cd /tmp/zookeeper
export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
LC_ALL=C mvn install -Pfull-build 2>&1 |
tee /mnt/ubuntu-mvn-tests.log
And here is an example of failure—clearly a time out:
[ERROR]
testJvmPauseMonitorExceedWarnThreshold(org.apache.zookeeper.server.util.JvmPauseMonitorTest)
Time elapsed: 5.119 s <<< ERROR!
org.junit.runners.model.TestTimedOutException: test timed out after 5000
milliseconds
at
app//org.apache.zookeeper.server.util.JvmPauseMonitorTest.testJvmPauseMonitorExceedWarnThreshold(JvmPauseMonitorTest.java:54)
Here is another example, somewhat more murky (possibly related to some
kind of race condition):
[ERROR]
testWatchAutoResetWithPending(org.apache.zookeeper.test.WatcherTest) Time
elapsed: 0.083 s <<< FAILURE!
java.lang.AssertionError: Unexpected bean exists! expected:<0> but was:<1>
at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:84)
I can provide more logs if we decide to open a ticket to track this topic.
>> Il giorno mer 2 set 2020 alle ore 20:42 Damien Diederen <
>> [email protected]> ha scritto:
>>
>>>
>>> Hi Enrico, all,
>>>
>>> I was also able to build and successfully run the tests of this release
>>> candidate on Ubuntu 20.04.1, with the provided Java & Maven:
>>>
>>> $ grep VERSION= /etc/os-release
>>> VERSION="20.04.1 LTS (Focal Fossa)"
>>> $ java -version
>>> openjdk version "11.0.8" 2020-07-14
>>> $ mvn -version
>>> Apache Maven 3.6.3
>>>
>>> (It took me a number of tries, because that is a VM and the tests are
>>> somewhat flaky in that environment. But this is, as far as I know, a
>>> long-standing issue and completely unrelated to 3.6.2. Please let me
>>> know if you have tips/tricks for avoiding such temporary failures.)
>>>
>>> Cheers, -D
>>>
>>>
>>>
>>> Szalay-Bekő Máté <[email protected]> writes:
>>> > +1 (non-binding)
>>> >
>>> > - I built the source code (-Pfull-build) on Ubuntu 18.04.3 using OpenJDK
>>> > 8u242, OpenJDK 11.0.8 and maven 3.6.0.
>>> > - all the unit tests passed (both Java and C-client).
>>> > - I also built and executed unit tests for zkpython
>>> > - checkstyle and spotbugs passed
>>> > - apache-rat passed
>>> > - owasp (CVE check) passed
>>> > - fatjar built (-Pfatjar)
>>> >
>>> > On Tue, Sep 1, 2020 at 11:35 AM Enrico Olivelli <[email protected]>
>>> wrote:
>>> >
>>> >> This is a release candidate for 3.6.2.
>>> >>
>>> >> It is a minor release and it fixes a few critical issues and brings a
>>> few
>>> >> dependencies upgrades.
>>> >>
>>> >> The full release notes is available at:
>>> >>
>>> >>
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801&version=12347809
>>> >>
>>> >> *** Please download, test and vote by September 4th 2020, 23:59 UTC+0.
>>> ***
>>> >>
>>> >> Source files:
>>> >> https://people.apache.org/~eolivelli/zookeeper-3.6.2-candidate-0/
>>> >>
>>> >> Maven staging repo:
>>> >>
>>> https://repository.apache.org/content/repositories/orgapachezookeeper-1060/
>>> >>
>>> >> The release candidate tag in git to be voted upon: release-3.6.2-0
>>> >> https://github.com/apache/zookeeper/tree/release-3.6.2-0
>>> >>
>>> >> ZooKeeper's KEYS file containing PGP keys we use to sign the release:
>>> >> https://www.apache.org/dist/zookeeper/KEYS
>>> >>
>>> >> The staging version of the website is:
>>> >>
>>> https://people.apache.org/~eolivelli/zookeeper-3.6.2-candidate-0/website/
>>> >>
>>> >> Should we release this candidate?
>>> >>
>>> >> Enrico Olivelli
>>> >>
>>>
