ndimiduk commented on a change in pull request #1461: HBASE-23779 Up the default fork count to make builds complete faster;… URL: https://github.com/apache/hbase/pull/1461#discussion_r405871037
########## File path: dev-support/hbase-personality.sh ########## @@ -140,7 +140,15 @@ function personality_modules clear_personality_queue - extra="-DHBasePatchProcess" + # Set a fork count based off the host cpu count. Pass maven a -T argument. + # Default -T is one thread. 0.5C on an apache box of 16 cores and 2 jenkins + # 'executors' per host should make for 8 threads. Setting this here for yetus + # to pick up. See + # https://yetus.apache.org/documentation/0.11.1/precommit-advanced/#global-definitions + # See below for more on -T: + # https://cwiki.apache.org/confluence/display/MAVEN/Parallel+builds+in+Maven+3 + forkcount="0.5C" + extra="-T${forkcount} -Dsurefire.firstPartForkCount=${forkcount} -Dsurefire.secondPartForkCount=${forkcount} -DHBasePatchProcess" Review comment: How does maven `--threads` interact with surefire forks? Isn't it the case that we get `forkCount` maven process thread each building a module simultaneously, and each of those modules is launching up to `forkCount` concurrent test jvms? Means we're looking at 64 concurrent test processes? Seems high to me. If it is indeed launching like this, I would want `-T{forkCount}` when running `compile/install`, but i want just a single core when running `test`. I took a read through the [surefire docs](https://maven.apache.org/surefire/maven-surefire-plugin/examples/fork-options-and-parallel-execution.html) but I don't have a better handle on this than when I started. If Maven is instead managing an upper-bound on concurrent test jvms at `${forkcount}`, I say +1. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
