On Wed, Jul 17, 2024 at 10:04:19AM +0200, Thomas Huth wrote: > On 16/07/2024 19.03, Thomas Huth wrote: > > On 16/07/2024 18.51, Daniel P. Berrangé wrote: > > > On Tue, Jul 16, 2024 at 01:26:03PM +0200, Thomas Huth wrote: > > ... > > > > So instead of trying to update the python-based test suite in QEMU > > > > to a newer version of Avocado, we should maybe try to better integrate > > > > it with the meson test runner instead. Indeed most tests work quite > > > > nicely without the Avocado framework already, as you can see with > > > > this patch series - it does not convert all tests, just a subset so > > > > far, but this already proves that many tests only need small modifi- > > > > cations to work without Avocado. > > ... > > > > Now if you want to try out these patches: Apply the patches, then > > > > recompile and then run: > > > > > > > > make check-functional > > > > > > > > You can also run single targets e.g. with: > > > > > > > > make check-functional-ppc > > > > > > > > You can also run the tests without any test runner now by > > > > setting the PYTHONPATH environment variable to the "python" folder > > > > of your source tree, and by specifying the build directory via > > > > QEMU_BUILD_ROOT (if autodetection fails) and by specifying the > > > > QEMU binary via QEMU_TEST_QEMU_BINARY. For example: > > > > > > > > export PYTHONPATH=$HOME/qemu/python > > > > export QEMU_TEST_QEMU_BINARY=qemu-system-x86_64 > > > > export PYTHONPATH=$HOME/qemu/build > > > > ~/qemu/tests/functional/test_virtio_version.py > > > > > > For the whole series as is > > > > > > Tested-by: Daniel P. Berrangé <berra...@redhat.com> > > > > > > as it does what you claim it does here when I tried it. > > > > Thanks! > > > > > > The logs of the tests can be found in the build directory under > > > > tests/functional/<testname> - console log and general logs will > > > > be put in separate files there. > > > > > > As an example, one dir name appears to be: > > > > > > __main__.MemAddrCheck.test_phybits_ok_pentium_pae > > > > > > I'd rather prefer it if the dir name matched the test script > > > file name - in this case test_mem_addr_space.py, as I don't > > > want to have to lookup which class names were defined inside > > > each test script. We could drop the "test_" prefix from the > > > method name too > > > > > > IOW, could we make this dir name be: > > > > > > test_mem_addr_space.phybits_ok_pentium_pae > > > > I can try to change that, indeed ... but the boilerplate code will > > increase a little bit, I guess, since I cannot simply rely on the > > unittest.id() function in that case anymore... > > After looking at this for a while, I think it's maybe best to ditch the idea > of making the .py files directly runnable and run the tests via a simple > pycotap runner instead. Then you get proper module names:
I'd really not want to loose that. To me, eliminating the test harness entirely when debugging is the single biggest improvement of this new approach, especially when I want to 'strace' the test without extraneous processes. > $ pyvenv/bin/python3 -m pycotap test_virtio_version > TAP version 13 > ok 1 test_virtio_version.VirtioVersionCheck.test_conventional_devs > ok 2 test_virtio_version.VirtioVersionCheck.test_modern_only_devs > 1.. With the following change, you get the same output with direct execution, by making argv look the same as you'd get when running your pycotap example. diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py index cc49fd4c94..3a3e65252d 100644 --- a/tests/functional/qemu_test/__init__.py +++ b/tests/functional/qemu_test/__init__.py @@ -266,7 +266,10 @@ def fetch_asset(self, url, asset_hash): def main(): tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError, test_output_log = pycotap.LogMode.LogToError) - unittest.main(testRunner = tr) + import sys + import os.path + path = os.path.basename(sys.argv[0])[:-3] + unittest.main(module = None, testRunner = tr, argv=["__dummy__", path]) class QemuSystemTest(QemuBaseTest): With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|