Fabiano Rosas <[email protected]> writes: > make check is currently by default running tests tagged > "slow". Running make check with no SPEED variable set should be > equivalent to SPEED=quick and run the meson 'quick' test setup, which > explicitly excludes slow tests: > > add_test_setup('quick', exclude_suites: ['slow', 'thorough'] ... > > The problem is that the mtest2make.py script is effectively overriding > the "meson test --setup" setting by using --suite which takes > precedence over --setup according to meson documentation. > > Fix by excluding suites directly with the "--no-suite" option while > generating Makefile.mtest. > > Although the 'exclude_suites' in meson.build are useless for make > check, they still help when running 'meson test' from the > command-line, so keep them around. > > For the configure line I'm using: > > make check goes from 1100 to 996 tests > make SPEED=slow check stays at 2158 tests > make SPEED=thorough check stays at 2279 tests > > (the rest of the test targets look sane, but maintainers please > double-check) > > Signed-off-by: Fabiano Rosas <[email protected]> > --- > - With this change we obviously lose coverage in make check. I suggest > we use this opportunity to review what gets put in the quick > suite. > > For migration, I want to keep only the smoke tests in make > check. I'll send a patch adding them back if this change is > accepted. > > - Some of the suffixes from functional tests could probably be removed > and rely on having the tests in the proper suite instead. I.e. use > 'slow' instead of 'foobar_slow'. > --- > scripts/mtest2make.py | 11 +++++++++-- > tests/qemu-iotests/meson.build | 2 +- > 2 files changed, 10 insertions(+), 3 deletions(-) > > diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py > index 383ea68b16..a61a83f775 100644 > --- a/scripts/mtest2make.py > +++ b/scripts/mtest2make.py > @@ -26,6 +26,11 @@ def names(self, base): > .speed.slow = $(sort $(filter-out %-thorough, $1)) > .speed.thorough = $(sort $1) > > +.speed_exclude_suites.quick = --no-suite slow --no-suite thorough \ > + --no-suite optional > +.speed_exclude_suites.slow = --no-suite thorough > +.speed_exclude_suites.thorough = > + > TIMEOUT_MULTIPLIER ?= 1 > .mtestargs = --no-rebuild -t $(TIMEOUT_MULTIPLIER) > ifneq ($(SPEED), quick) > @@ -34,7 +39,8 @@ def names(self, base): > .mtestargs += $(subst -j,--num-processes , $(filter-out -j, $(lastword -j1 > $(filter -j%, $(MAKEFLAGS))))) > > .check.mtestargs = $(MTESTARGS) $(.mtestargs) $(if > $(V),--verbose,--print-errorlogs) \ > - $(foreach s, $(sort $(.check.mtest-suites)), --suite $s) > + $(foreach s, $(sort $(.check.mtest-suites)), --suite $s) \ > + $(.speed_exclude_suites.$(SPEED)) > .bench.mtestargs = $(MTESTARGS) $(.mtestargs) --benchmark --verbose \ > $(foreach s, $(sort $(.bench.mtest-suites)), --suite $s)''') > > @@ -102,7 +108,8 @@ def emit_suite(name, suite, prefix): > targets += f' {prefix} {prefix}-report.junit.xml' > print(f'ifneq ($(filter {targets}, $(MAKECMDGOALS)),)') > # for the "base" suite possibly add FOO-slow and FOO-thorough > - print(f".{prefix}.mtest-suites += {name} $(call .speed.$(SPEED), > {names})") > + print(f".{prefix}.mtest-suites += {name} $(call .speed.$(SPEED), > {names}) " > + ".speed_exclude_suites.$(SPEED)") > print(f'endif') > > targets = {t['id']: [os.path.relpath(f) for f in t['filename']] > diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build > index 8857f25f29..34cfcdc53b 100644 > --- a/tests/qemu-iotests/meson.build > +++ b/tests/qemu-iotests/meson.build > @@ -68,7 +68,7 @@ foreach driver, speed: qemu_iotests_drivers > endif > > # Every driver gets put in the driver specific suite > - suites = ['block-' + driver + '-optional'] > + suites = ['block-' + driver + '-optional', 'optional'] > # Any driver tagged quick or slow also gets added to slow > # otherwise its tagged thorough > if speed != 'thorough'
Oops, this one sneaked in, please ignore it. I sent it as a separate patch already. https://lore.kernel.org/r/[email protected]
