Original (very broken) v1 posting: https://lists.nongnu.org/archive/html/qemu-devel/2025-10/msg01650.html
The recent set of regressions identified in the LUKS block driver re-inforced that despite having a hugely useful set of I/O tests, our CI coverage is still letting through too many bugs. The core goals of this series were/are: * Add LUKS and NBD to the tested formats/protocols integrated into "make check-block SPEED=thorough" * Ensure that all qcow2 tests can be run with "SPEED=slow", not staying limited to only the 'auto' group used by the 'make check-block' target in its default 'quick' mode * Add 'make check-block-$FORMAT' to expose a standalone target for running all tests for a given format (or equivalently a protocol) * Add GitLab CI jobs for exercising tests for all formats A sample pipeline for this is https://gitlab.com/berrange/qemu/-/pipelines/2258731804/ Showing the 'block-FORMAT-centos' jobs which test the 10 block formats/protocols currently enabled. This series fixes a couple of bugs * sendmsg generates a deprecation warning on centos * Test 128 check for device mapper usage doesn't work sufficiently well to detect gitlab's containe env can't do dynamic devfs * Test 185 is reliably failing on gitlab I'm unclear if there are other non-deterministic failures that would only appear if we start running the tests regularly, or against QEMU's private runners instead of gitlab shared runners. The ones I've fixed have been consistent across quite a few test pipelines so far with the shared runners. At this point it is a chicken & egg problem though. If we don't bite the bullet and enable block I/O tests in GitLab we'll never find out if they're stable enough to rely on. As a mitigation against instability a patch in this series adds a QEMU_TEST_IO_SKIP env variable that we are able to set as a GitLab CI env variable. This lets us skip broken tests without waiting for a git commit to disable them. Changed in v3: - Test whether dmsetup create device appeared or not, instead of unconditionally skipping test 128 in GitLab - Use 'driver' term instead of 'format' or 'protocol' - Fix misc typos & rephrasing Daniel P. Berrangé (14): Revert "python/aqmp: fix send_fd_scm for python 3.6.x" tests: print reason when I/O test is skipped in TAP mode tests: remove redundant meson suite for iotests tests: ensure all qcow2 I/O tests are able to be run via make scripts/mtest2make: ensure output has stable sorting scripts/mtest2make: support optional tests grouping tests: add a meson suite / make target per block I/O tests format docs/devel/testing: expand documentation for 'make check-block' tests: add nbd and luks to the I/O test suites tests: use 'driver' as collective term for either format or protocol tests: validate dmsetup result in test 128 tests: fix check for sudo access in LUKS I/O test tests: add QEMU_TEST_IO_SKIP for skipping I/O tests gitlab: add jobs for thorough block tests .gitlab-ci.d/buildtest.yml | 92 ++++++++++++++++++++++++++++++++ docs/devel/testing/main.rst | 33 ++++++++++-- python/qemu/qmp/qmp_client.py | 9 ++-- scripts/mtest2make.py | 30 +++++++---- tests/Makefile.include | 3 +- tests/qemu-iotests/128 | 6 +++ tests/qemu-iotests/149 | 13 +++-- tests/qemu-iotests/meson.build | 59 +++++++++++++++++--- tests/qemu-iotests/testrunner.py | 18 ++++++- 9 files changed, 229 insertions(+), 34 deletions(-) -- 2.52.0
