On Thu, Jul 11, 2024 at 01:55:42PM +0200, Thomas Huth wrote:
> From: Ani Sinha <a...@anisinha.ca>
>
> Integrate the pytest framework with the meson build system. This
> will make meson run all the pytests under the pytest directory.
Lets add a note about the compelling benefit of this new approach
With this change, each functional test becomes subject
to an individual execution timeout, defaulting to 60
seconds, but overridable per-test.
>
> Signed-off-by: Ani Sinha <a...@anisinha.ca>
> [thuth: Removed the acpi-bits and adjusted for converted avocado tests
> instead]
> Signed-off-by: Thomas Huth <th...@redhat.com>
> ---
> tests/Makefile.include | 4 ++-
> tests/meson.build | 1 +
> tests/pytest/meson.build | 53 ++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 57 insertions(+), 1 deletion(-)
> create mode 100644 tests/pytest/meson.build
For CI purposes we'll need to add 'python3-pytest' to
tests/lcitool/projects/qemu.yml, and re-generate the
the dockerfiles. Some of the other non-gitlab CI
integrations probably need manual additions of pytest
packages.
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index d39d5dd6a4..68151717d7 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -3,12 +3,14 @@
> .PHONY: check-help
> check-help:
> @echo "Regression testing targets:"
> - @echo " $(MAKE) check Run block, qapi-schema, unit,
> softfloat, qtest and decodetree tests"
> + @echo " $(MAKE) check Run block, qapi-schema, unit,
> softfloat, qtest, pytest and decodetree tests"
> @echo " $(MAKE) bench Run speed tests"
> @echo
> @echo "Individual test suites:"
> @echo " $(MAKE) check-qtest-TARGET Run qtest tests for given target"
> @echo " $(MAKE) check-qtest Run qtest tests"
> + @echo " $(MAKE) check-pytest Run pytest tests"
> + @echo " $(MAKE) check-pytest-TARGET Run pytest for a given target"
Or name it after the type of test rather than harness ?
eg check-functional / check-functional-TARGET
For that matter perhaps also for the dir name ?
tests/functional/*.py
> @echo " $(MAKE) check-unit Run qobject tests"
> @echo " $(MAKE) check-qapi-schema Run QAPI schema tests"
> @echo " $(MAKE) check-block Run block tests"
> diff --git a/tests/meson.build b/tests/meson.build
> index acb6807094..17510a468e 100644
> --- a/tests/meson.build
> +++ b/tests/meson.build
> @@ -85,3 +85,4 @@ subdir('unit')
> subdir('qapi-schema')
> subdir('qtest')
> subdir('migration')
> +subdir('pytest')
> diff --git a/tests/pytest/meson.build b/tests/pytest/meson.build
> new file mode 100644
> index 0000000000..1486628d45
> --- /dev/null
> +++ b/tests/pytest/meson.build
> @@ -0,0 +1,53 @@
> +slow_pytests = {
> + 'mem_addr_space' : 90,
> +}
> +
> +pytests_generic = [
> + 'empty_cpu_model',
> + 'info_usernet',
> + 'version',
> +]
> +
> +pytests_x86_64 = [
> + 'cpu_queries',
> + 'mem_addr_space',
> + 'virtio_version',
> +]
> +
> +pytest = find_program('pytest', required: false)
> +if not pytest.found()
> + message('pytest not available ==> Disabled the qemu-pytests.')
> + subdir_done()
> +endif
> +
> +foreach dir : target_dirs
> + if not dir.endswith('-softmmu')
> + continue
> + endif
> +
> + target_base = dir.split('-')[0]
> + pytest_emulator = emulators['qemu-system-' + target_base]
> + target_pytests = get_variable('pytests_' + target_base, []) +
> pytests_generic
> +
> + test_deps = roms
> + pytest_env = environment()
> + if have_tools
> + pytest_env.set('PYTEST_QEMU_IMG', './qemu-img')
> + test_deps += [qemu_img]
> + endif
> + pytest_env.set('PYTEST_QEMU_BINARY', meson.global_build_root() /
> 'qemu-system-' + target_base)
> + pytest_env.set('PYTEST_SOURCE_ROOT', meson.project_source_root())
> + pytest_env.set('PYTEST_BUILD_ROOT', meson.project_build_root())
> + pytest_env.set('PYTHONPATH', meson.project_source_root() / 'python')
> +
> + foreach test : target_pytests
> + test('pytest-@0@/@1@'.format(target_base, test),
> + pytest,
> + depends: [test_deps, pytest_emulator, emulator_modules],
> + env: pytest_env,
> + args: [meson.current_source_dir() / 'test_' + test + '.py'],
> + timeout: slow_pytests.get(test, 60),
> + priority: slow_pytests.get(test, 60),
> + suite: ['pytest', 'pytest-' + target_base])
> + endforeach
> +endforeach
> --
> 2.45.2
>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
