Great to see this Benjamin! Looking forward to seeing the parallel test runner turn green, I'll help file tickets under the epic (I see there are a lot of test failures for me).
Once we clear the issues and turn it green, shall we make this the default? I would be in favor of that. Ben On Thu, Oct 13, 2016 at 2:28 PM, Benjamin Bannier < benjamin.bann...@mesosphere.io> wrote: > > Hi, > > Since most tests in the Mesos, libprocess, and stout test suites can > be executed in parallel (the exception being some `ROOT` tests with > global side effects in Mesos), we recently added a parallel test > runner `support/mesos-gtest-runner.py`. This should allow to > potentially significantly speed up running of test suites. > > To enable automatic parallel execution of tests for test targets > executed during `make check`, configure Mesos with the option > `--enable-parallel-test-execution`. This will configure the test runner > to run all tests but the `ROOT` tests in parallel; `ROOT` tests will > be run in a separate, sequential step. > > * * * > > We use the environment variable `TEST_DRIVER` to drive parallel test > execution. By setting this variable to an empty string you can > temporarily disable configured parallel execution, e.g., > > % make check TEST_DRIVER= > > By setting this environment variable you have control over the test > runner itself and its arguments, even without enabling parallel test > during `./configure` time. Be aware that many `ROOT` tests cannot be > run in parallel. > > > The current settings oversubscribe the machine by running `#cores*1.5` > parallel jobs. This was driven by the observation that currently our > tests by and large do not make extended use of even a single core. > The number of parallel jobs can by controlled with the `-j` flag of > the test runner. > > Since making more use of the machine will likely increase machine load > during test execution, running tests in parallel might expose test > flakiness. Tests might also fail to run in parallel if testcases e.g., > write data to hardcoded locations or use hardcoded ports. Please file > JIRA tickets for such tests if they do not yet exist. > > > There is still some work needed to improve reporting from parallel > tests. We currently use a very silent mode if tests are running > without failures, and just report the logs of failed jobs in case of > failure. MESOS-6387 sketches out possible future improvements in this > area. > > > Happy testing, > > Benjamin with help from Kevin & Till > >