> It seems forkCount is overriden by the surefire-forkcount property. > This seems to do what you expect: > > mvn test -Dsurefire.rerunFailingTestsCount=3 -fae -Pfull-build -Dsurefire-forkcount=1
indeed... this seems to be the case on all active branches. my mistake :) Thanks! Mate On Thu, Jan 28, 2021 at 9:38 PM Damien Diederen <ddiede...@apache.org> wrote: > > Hi Máté, all, > > I managed to fight my way though some Apple ID insanity to get a Mac > running with Catalina, Java, Maven and a C compiler. This part of the > puzzle was particularly fun: > > https://dd.crosstwine.com/tmp/apple-id.png > > Anyway. > > > So I prefer to execute the java unit tests using the following command > when > > checking RCs: > > > > mvn test -Dsurefire.rerunFailingTestsCount=3 -fae -Pfull-build > -DforkCount=1 > > This command does not work, at least on Catalina—where it forks eight > runners: > > % pstree 7846 > -+= 07846 administrator /Library/Java/JavaVir.../adoptopenjdk-8... > |-+- 09342 administrator /bin/sh -c cd /Users/... > | \--- 09343 administrator /Library/Java/JavaVir.../adoptopenjdk-8... > |-+- 09640 administrator /bin/sh -c cd /Users/... > | \--- 09641 administrator /Library/Java/JavaVir.../adoptopenjdk-8... > |-+- 09797 administrator /bin/sh -c cd /Users/... > | \--- 09798 administrator /Library/Java/JavaVir.../adoptopenjdk-8... > |-+- 09893 administrator /bin/sh -c cd /Users/... > | \--- 09898 administrator /Library/Java/JavaVir.../adoptopenjdk-8... > |-+- 09912 administrator /bin/sh -c cd /Users/... > | \--- 09913 administrator /Library/Java/JavaVir.../adoptopenjdk-8... > |-+- 09942 administrator /bin/sh -c cd /Users/... > | \--- 09944 administrator /Library/Java/JavaVir.../adoptopenjdk-8... > |-+- 09955 administrator /bin/sh -c cd /Users/... > | \--- 09956 administrator /Library/Java/JavaVir.../adoptopenjdk-8... > \-+- 09960 administrator /bin/sh -c cd /Users/... > \--- 09961 administrator /Library/Java/JavaVir.../adoptopenjdk-8... > > It seems forkCount is overriden by the surefire-forkcount property. > This seems to do what you expect: > > mvn test -Dsurefire.rerunFailingTestsCount=3 -fae -Pfull-build > -Dsurefire-forkcount=1 > > But independently of that, I can reproduce the issue—even with a "real" > sequential run. It does not always result in the stack trace you saw, > but often hangs with "funny" errors such as: > > [WARNING] Corrupted STDOUT by directly writing to native stream in > forked JVM 1. See FAQ web page and the dump file > /Users/administrator/zookeeper/zookeeper-server/target/surefire-reports/2021-01-28T17-56-26_094-jvmRun1.dumpstream > > # Created at 2021-01-28T18:37:51.381 > Corrupted STDOUT by directly writing to native stream in forked JVM 1. > Stream '[59.370s][warning][os,thread] Failed to start thread - > pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: > 4k, detached.'. > > I'm planning to have a closer look soon. > > Cu, -D > > P.-S. — Enrico: I am also able to reproduce the C compilation issues you > reported. > > > --8<---------------original message------------->8--- > > Szalay-Bekő Máté <szalay.beko.m...@gmail.com> writes: > > this is not specific to 3.7, but in general: > > from time to time I see different tests failing intermittently. I assume > > this can be caused by other tests running in parallel. (because > re-running > > those tests alone usually makes them succeed) > > So I prefer to execute the java unit tests using the following command > when > > checking RCs: > > > > mvn test -Dsurefire.rerunFailingTestsCount=3 -fae -Pfull-build > -DforkCount=1 > > > > As far as I remember, by default the fork count is set to 8 in maven, and > > the CI is using (or was using before the migration) -DforkCount=4. > > > > > > Regards, > > Mate > > > > On Mon, Jan 25, 2021 at 1:51 PM Flavio Junqueira <f...@apache.org> wrote: > > > >> I don't want to mess up with the vote thread, so I'm responding to this > >> separately. I have been trying to build locally too unsuccessfully. I've > >> been trying on an Ubuntu VM, Java 8 (build 1.8.0_181-b13), Maven 3.6.0. > The > >> set of tests failing varies from build to build, if it makes sense, I > can > >> try to collect all test failures I have seen and post. > >> > >> -Flavio > >> > >> > On 25 Jan 2021, at 13:38, Szalay-Bekő Máté < > szalay.beko.m...@gmail.com> > >> wrote: > >> > > >> > +0 (and not even binding :) ) > >> > > >> > - I built the source code (-Pfull-build) on Ubuntu 18.04.3 using > OpenJDK > >> > 8u265 and maven 3.6.3. > >> > - I also built and executed unit tests for zkpython > >> > - the unit tests passed for the C-client and for python client > >> > - checkstyle and spotbugs passed > >> > - apache-rat passed > >> > - owasp (CVE check) passed > >> > - fatjar built (-Pfatjar) > >> > - I executed a quick rolling-upgrade test from 3.5.9 and from 3.6.2. > >> (using > >> > https://github.com/symat/zk-rolling-upgrade-test) > >> > > >> > for some reason the java unit tests failed for me. > >> > > >> > On mac (jdk 1.8.212 and maven 3.6.3), I got all the unit tests > executed > >> > successfully, but then the maven job still failed for hbase-server > test > >> > with error message (with -DforkCount=4 and even with -DforkCount=1) > like: > >> > ------------- > >> > [ERROR] ExecutionException There was an error in the forked process > >> > [ERROR] unable to create new native thread > >> > [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: > >> > ExecutionException There was an error in the forked process > >> > [ERROR] unable to create new native thread > >> > [ERROR] at > >> > > >> > org.apache.maven.plugin.surefire.booterclient.ForkStarter.awaitResultsDone(ForkStarter.java:510) > >> > ------------- > >> > > >> > > >> > Then I tried on a dockerized environment (ubuntu 18.4, OpenJDK 8u265 > and > >> > maven 3.6.3) and I got other kinds of strange maven errors: > >> > ------------------- > >> > [ERROR] Caused by: > >> > org.apache.maven.surefire.booter.SurefireBooterForkException: The > forked > >> VM > >> > terminated without properly saying goodbye. VM crash or System.exit > >> called? > >> > [ERROR] Command was /bin/sh -c cd > >> > /tmp/zk/apache-zookeeper-3.7.0/zookeeper-server && > >> > /home/symat/.sdkman/candidates/java/8.0.265-open/jre/bin/java -Xmx512m > >> > -Dtest.junit.threads=8 -Dzookeeper.junit.threadid=3 > >> > > >> > -javaagent:/home/symat/.m2/repository/org/jmockit/jmockit/1.48/jmockit-1.48.jar > >> > -jar > >> > > >> > /tmp/zk/apache-zookeeper-3.7.0/zookeeper-server/target/surefire/surefirebooter8828313385463488429.jar > >> > /tmp/zk/apache-zookeeper-3.7.0/zookeeper-server/target/surefire > >> > 2021-01-25T11-54-03_621-jvmRun3 surefire4024538135165099286tmp > >> > surefire_37800399112966511000tmp > >> > [ERROR] Process Exit Code: 0 > >> > [ERROR] at > >> > > >> > org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:669) > >> > [ERROR] at > >> > > >> > org.apache.maven.plugin.surefire.booterclient.ForkStarter.access$600(ForkStarter.java:115) > >> > [ERROR] at > >> > > >> > org.apache.maven.plugin.surefire.booterclient.ForkStarter$2.call(ForkStarter.java:444) > >> > [ERROR] at > >> > > >> > org.apache.maven.plugin.surefire.booterclient.ForkStarter$2.call(ForkStarter.java:420) > >> > [ERROR] at > >> java.util.concurrent.FutureTask.run(FutureTask.java:266) > >> > [ERROR] at > >> > > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > >> > [ERROR] at > >> > > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > >> > [ERROR] at java.lang.Thread.run(Thread.java:748) > >> > [ERROR] > >> > -------------------------------------- > >> > > >> > > >> > These issues might be very well specific to my local (mac or docker on > >> mac) > >> > environments. This is why I didn't vote with -1 > >> > Can someone else run the java unit tests successfully locally? > >> > > >> > I also tried to check if the CI was green for the last PR on 3.7.0 ( > >> > https://github.com/apache/zookeeper/pull/1586/checks), but it looks > the > >> CI > >> > haven't even started to execute the tests, due to errors in the > "install > >> C > >> > dependencies" step. > >> > > >> > Regards, > >> > Mate > >> > > >> > On Sun, Jan 24, 2021 at 11:39 PM Patrick Hunt <ph...@apache.org> > wrote: > >> > > >> >> +1. xsum/sig verified. rat ran clean. built and dependency checks are > >> fine. > >> >> Tried running some manual clusters and it was successful. > >> >> > >> >> Regards, > >> >> > >> >> Patrick > >> >> > >> >> > >> >> On Sun, Jan 24, 2021 at 12:11 PM Damien Diederen < > ddiede...@apache.org> > >> >> wrote: > >> >> > >> >>> > >> >>> Dear all, > >> >>> > >> >>> This is a second release candidate for ZooKeeper 3.7.0. Compared to > >> >>> RC0, it fixes a tarball generation issue, includes a description of > the > >> >>> 'whoami' CLI command, and incorporates a contribution to > ZooInspector. > >> >>> > >> >>> ZooKeeper 3.7.0 introduces a number of new features, notably: > >> >>> > >> >>> * An API to start a ZooKeeper server from Java (ZOOKEEPER-3874); > >> >>> > >> >>> * Quota enforcement (ZOOKEEPER-3301); > >> >>> > >> >>> * Host name canonicalization in quorum SASL authentication > >> >>> (ZOOKEEPER-4030); > >> >>> > >> >>> * Support for BCFKS key/trust store format (ZOOKEEPER-3950); > >> >>> > >> >>> * A choice of mandatory authentication scheme(s) (ZOOKEEPER-3561); > >> >>> > >> >>> * A "whoami" API and CLI command (ZOOKEEPER-3969); > >> >>> > >> >>> * The possibility of disabling digest authentication > (ZOOKEEPER-3979); > >> >>> > >> >>> * Multiple SASL "superUsers" (ZOOKEEPER-3959); > >> >>> > >> >>> * Fast-tracking of throttled requests (ZOOKEEPER-3683); > >> >>> > >> >>> * Additional security metrics (ZOOKEEPER-3978); > >> >>> > >> >>> * SASL support in the C and Perl clients (ZOOKEEPER-1112, > >> >>> ZOOKEEPER-3714); > >> >>> > >> >>> * A new zkSnapshotComparer.sh tool (ZOOKEEPER-3427); > >> >>> > >> >>> * Notes on how to benchmark ZooKeeper with the YCSB tool > >> >>> (ZOOKEEPER-3264). > >> >>> > >> >>> The release notes are available here: > >> >>> > >> >>> > >> >>> > >> >> > >> > https://people.apache.org/~ddiederen/zookeeper-3.7.0-candidate-1/website/releasenotes.html > >> >>> > >> >>> > >> >> > >> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801&version=12346617 > >> >>> > >> >>> *** Please download, test and vote by January 31st 2020, 23:59 > UTC+0. > >> *** > >> >>> > >> >>> Source files: > >> >>> > >> >>> https://people.apache.org/~ddiederen/zookeeper-3.7.0-candidate-1/ > >> >>> > >> >>> Maven staging repo: > >> >>> > >> >>> > >> >>> > >> >> > >> > https://repository.apache.org/content/repositories/orgapachezookeeper-1066/ > >> >>> > >> >>> The release candidate tag in git to be voted upon: release-3.7.0-1 > >> >>> > >> >>> https://github.com/apache/zookeeper/tree/release-3.7.0-1 > >> >>> > >> >>> ZooKeeper's KEYS file containing PGP keys we use to sign the > release: > >> >>> > >> >>> https://www.apache.org/dist/zookeeper/KEYS > >> >>> > >> >>> The staging version of the website is: > >> >>> > >> >>> > >> >>> > >> >> > >> > https://people.apache.org/~ddiederen/zookeeper-3.7.0-candidate-1/website/ > >> >>> > >> >>> Should we release this candidate? > >> >>> > >> >>> Damien Diederen > >> >>> > >> >> > >> > >> >
