The Avocado v88 that we use in QEMU is already on a life support
system: It is not supported by upstream anymore, and with the latest
versions of Python, it won't work anymore since it depends on the
"imp" module that has been removed in Python 3.12.
There have been several attempts to update the test suite in QEMU
to a newer version of Avocado, but so far no attempt has successfully
been merged yet.
Additionally, the whole "make check" test suite in QEMU is using the
meson test runner nowadays, so running the python-based tests via the
Avocodo test runner looks and feels quite like an oddball, requiring
the users to deal with the knowledge of multiple test runners in
parallel.
So instead of trying to update the python-based test suite in QEMU
to a newer version of Avocado, we should maybe try to better integrate
it with the meson test runner instead. Indeed most tests work quite
nicely without the Avocado framework already, as you can see with
this patch series - it does not convert all tests, just a subset since
it is just an RFC so far, but as you can see, many tests only need
small modifications to work without Avocado.
If you want to try it: Apply the patches, make sure that you have the
"pytest" program installed, then recompile and then run:
make check-pytest
Things that need further attention though:
- All tests that use the LinuxTest / LinuxDistro classes (e.g. based
on cloud-init images) really depend on the Avocado framework,
thus we'd need a solution for those if we want to continue with
this approach
- Same for all tests that require the LinuxSSHMixIn class - we'd
need to provide a solution for ssh-based tests, too.
- We lose the way of running tests via the avocado tags this way...
single targets can still be tested by running "make check-pytest-arm"
for example, but running selected tests by other tags does not
work anymore.
- I haven't looked into logging yet ... this still needs some work
so that you could e.g. inspect the console output of the guests
somewhere
- I did not work on documentation updates yet (will do that if we
agree to continue with this patch series)
What's your thoughts? Is it worth to continue with this approach?
Or shall I rather forget about it and wait for the Avocado version
update?
Thomas
Ani Sinha (1):
tests/pytest: add pytest to the meson build system
Thomas Huth (7):
tests/pytest: Add base classes for the upcoming pytest-based tests
tests/pytest: Convert some simple avocado tests into pytests
tests/pytest: Convert info_usernet and version test with small
adjustments
tests_pytest: Implement fetch_asset() method for downloading assets
tests/pytest: Convert some tests that download files via fetch_asset()
tests/pytest: Add a function for extracting files from an archive
tests/pytest: Convert avocado test that needed avocado.utils.archive
tests/Makefile.include | 4 +-
tests/meson.build | 1 +
tests/pytest/meson.build | 74 ++++
tests/pytest/qemu_pytest/__init__.py | 362 ++++++++++++++++++
tests/pytest/qemu_pytest/utils.py | 21 +
.../test_arm_canona1100.py} | 16 +-
.../test_cpu_queries.py} | 2 +-
.../test_empty_cpu_model.py} | 2 +-
.../test_info_usernet.py} | 6 +-
.../test_machine_arm_n8x0.py} | 20 +-
.../test_machine_avr6.py} | 7 +-
.../test_machine_loongarch.py} | 11 +-
.../test_machine_mips_loongson3v.py} | 19 +-
.../test_mem_addr_space.py} | 3 +-
.../test_ppc_bamboo.py} | 18 +-
.../version.py => pytest/test_version.py} | 8 +-
.../test_virtio_version.py} | 2 +-
17 files changed, 502 insertions(+), 74 deletions(-)
create mode 100644 tests/pytest/meson.build
create mode 100644 tests/pytest/qemu_pytest/__init__.py
create mode 100644 tests/pytest/qemu_pytest/utils.py
rename tests/{avocado/machine_arm_canona1100.py =>
pytest/test_arm_canona1100.py} (74%)
rename tests/{avocado/cpu_queries.py => pytest/test_cpu_queries.py} (96%)
rename tests/{avocado/empty_cpu_model.py => pytest/test_empty_cpu_model.py}
(94%)
rename tests/{avocado/info_usernet.py => pytest/test_info_usernet.py} (91%)
rename tests/{avocado/machine_arm_n8x0.py => pytest/test_machine_arm_n8x0.py}
(71%)
rename tests/{avocado/machine_avr6.py => pytest/test_machine_avr6.py} (91%)
rename tests/{avocado/machine_loongarch.py =>
pytest/test_machine_loongarch.py} (89%)
rename tests/{avocado/machine_mips_loongson3v.py =>
pytest/test_machine_mips_loongson3v.py} (59%)
rename tests/{avocado/mem-addr-space-check.py =>
pytest/test_mem_addr_space.py} (99%)
rename tests/{avocado/ppc_bamboo.py => pytest/test_ppc_bamboo.py} (75%)
rename tests/{avocado/version.py => pytest/test_version.py} (82%)
rename tests/{avocado/virtio_version.py => pytest/test_virtio_version.py} (99%)
--
2.45.2
