[ https://issues.apache.org/jira/browse/HBASE-23779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17078905#comment-17078905 ]
Michael Stack commented on HBASE-23779: --------------------------------------- The new subissue HBASE-24150 takes a different tack. It allows two modules when possible to run their tests in parallel. It leaves the forkcount at 0.25C but ups the mvn --threads argument from default of '1' to '2'. In tests running local builds, a machine completed full test suite in 1hr 15mins w/ forkcount of 1.0C and --threads=1. When same machine ran full suite of tests with forkcount of 0.5C and --threads=2, it completed in 53mins. Lets see what we get. > Up the default fork count to make builds complete faster; make count relative > to CPU count > ------------------------------------------------------------------------------------------ > > Key: HBASE-23779 > URL: https://issues.apache.org/jira/browse/HBASE-23779 > Project: HBase > Issue Type: Task > Components: test > Reporter: Michael Stack > Assignee: Michael Stack > Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: Screen Shot 2020-04-08 at 9.19.53 AM.png, > addendum2.patch, test_yetus_934.0.patch > > > Tests take a long time. Our fork count running all tests are conservative -- > 1 (small) for first part and 5 for second part (medium and large). Rather > than hardcoding we should set the fork count to be relative to machine size. > Suggestion here is 0.75C where C is CPU count. This ups the CPU use on my box. > Looking up at jenkins, it seems like the boxes are 24 cores... at least going > by my random survey. The load reported on a few seems low though this not > representative (looking at machine/uptime). > More parallelism willl probably mean more test failure. Let me take a look > see. -- This message was sent by Atlassian Jira (v8.3.4#803005)