[PULL 35/42] tests/functional: Add QemuUserTest class

2024-09-04 Thread Thomas Huth
tion. Signed-off-by: Philippe Mathieu-Daudé Message-ID: <20240822104238.75045-2-phi...@linaro.org> Reviewed-by: Thomas Huth Message-ID: <20240830133841.142644-38-th...@redhat.com> Signed-off-by: Thomas Huth --- tests/functional/qemu_test/__init__.py | 2 +- tests/functional/qemu_test/testc

[PULL 20/42] tests/functional: Convert the s390x avocado tests into standalone tests

2024-09-04 Thread Thomas Huth
ymore nowadays. Reviewed-by: Philippe Mathieu-Daudé Message-ID: <20240830133841.142644-21-th...@redhat.com> Signed-off-by: Thomas Huth --- MAINTAINERS | 4 +- tests/functional/meson.build | 6 ++ tests/functional/qemu_test/utils.py

[PULL 32/42] tests/functional: Convert Aarch64 SBSA-Ref avocado tests

2024-09-04 Thread Thomas Huth
-Daudé Message-ID: <20240823131614.10269-1-phi...@linaro.org> Reviewed-by: Thomas Huth Reviewed-by: Marcin Juszkiewicz Message-ID: <20240830133841.142644-35-th...@redhat.com> [thuth: remove remaining Avocado tags] Signed-off-by: Thomas Huth --- MAINTAINERS

[PULL 13/42] tests/functional: Convert avocado tests that just need a small adjustment

2024-09-04 Thread Thomas Huth
ed-by: Daniel P. Berrangé Tested-by: Philippe Mathieu-Daudé Reviewed-by: Philippe Mathieu-Daudé Message-ID: <20240830133841.142644-14-th...@redhat.com> Signed-off-by: Thomas Huth --- MAINTAINERS | 3 + tests/functional/meson.build |

[PULL 27/42] tests/functional: Convert the m68k nextcube test with tesseract

2024-09-04 Thread Thomas Huth
thieu-Daudé Tested-by: Philippe Mathieu-Daudé Message-ID: <20240830133841.142644-29-th...@redhat.com> Signed-off-by: Thomas Huth --- MAINTAINERS | 1 + tests/avocado/tesseract_utils.py | 46 --- tests/functional/

[PULL 38/42] gitlab-ci: Add "check-functional" to the build tests

2024-09-04 Thread Thomas Huth
h...@redhat.com> Signed-off-by: Thomas Huth --- .gitlab-ci.d/buildtest-template.yml | 14 +-- .gitlab-ci.d/buildtest.yml | 60 ++--- 2 files changed, 41 insertions(+), 33 deletions(-) diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/bu

[PULL 11/42] tests/functional: Prepare the meson build system for the functional tests

2024-09-04 Thread Thomas Huth
ed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Message-ID: <20240830133841.142644-12-th...@redhat.com> Signed-off-by: Thomas Huth --- tests/Makefile.include | 11 + tests/functional/meson.build | 81 tests/meson.build

[PULL 01/42] tests/avocado: machine aarch64: standardize location and RO access

2024-09-04 Thread Thomas Huth
y: Cleber Rosa Tested-by: Marcin Juszkiewicz Message-ID: <20240806173119.582857-5-cr...@redhat.com> Reviewed-by: Philippe Mathieu-Daudé Message-ID: <20240830133841.142644-2-th...@redhat.com> Signed-off-by: Thomas Huth --- tests/avocado/machine_aarch64_sbsaref.py | 2 +- tests/av

[PULL 06/42] tests/avocado/boot_linux_console: Remove the s390x subtest

2024-09-04 Thread Thomas Huth
y. Reviewed-by: Daniel P. Berrangé Message-ID: <20240830133841.142644-7-th...@redhat.com> Signed-off-by: Thomas Huth --- tests/avocado/boot_linux_console.py | 20 tests/avocado/machine_s390_ccw_virtio.py | 2 ++ 2 files changed, 2 insertions(+), 20 deletions(-

[PULL 42/42] docs/devel/testing: Add documentation for functional tests

2024-09-04 Thread Thomas Huth
Document the new functional testing framework. The text is originally based on the Avocado documentation, but heavily modified to match the new framework. Message-ID: <20240830133841.142644-45-th...@redhat.com> Signed-off-by: Thomas Huth --- docs/devel/testing/functional.rst

[PULL 15/42] tests/functional: enable pre-emptive caching of assets

2024-09-04 Thread Thomas Huth
ys done by make check-functional Signed-off-by: Daniel P. Berrangé Tested-by: Richard Henderson [thuth: Remove the duplicated "path = os.path.basename(...)" line] Message-ID: <20240830133841.142644-16-th...@redhat.com> Signed-off-by: Thomas Huth --- tests/Makefile.include

[PULL 14/42] tests/functional: add a module for handling asset download & caching

2024-09-04 Thread Thomas Huth
tent for naming instead of URL, add the possibility to specify the cache dir via environment variable] Signed-off-by: Daniel P. Berrangé Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Message-ID: <20240830133841.142644-15-th...@redhat.com> Signed-off-by: Th

[PULL 05/42] tests/avocado/avocado_qemu: Fix the "from" statements in linuxtest.py

2024-09-04 Thread Thomas Huth
Without this change, the new Avocado v103 fails to find the tests that are based on the LinuxTest class. Suggested-by: Cleber Rosa Tested-by: Philippe Mathieu-Daudé Message-ID: <20240830133841.142644-6-th...@redhat.com> Signed-off-by: Thomas Huth --- tests/avocado/avocado_qemu/linuxt

[PULL 29/42] tests/functional: Convert the rx_gdbsim avocado test into a standalone test

2024-09-04 Thread Thomas Huth
Provide a "gzip_uncompress" function based on the standard "gzip" module to avoid the usage of avocado.utils here. Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Message-ID: <20240830133841.142644-31-th...@redhat.com> Signed-off-by: Tho

[PULL 21/42] tests/functional: Convert the x86_cpu_model_versions test

2024-09-04 Thread Thomas Huth
Nothing thrilling in here, it's just a straight forward conversion. Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Message-ID: <20240830133841.142644-22-th...@redhat.com> Signed-off-by: Thomas Huth --- MAINTAINERS |

[PULL 31/42] tests/functional: Convert ARM Integrator/CP avocado tests

2024-09-04 Thread Thomas Huth
OK 3.90s 2 subtests passed Signed-off-by: Philippe Mathieu-Daudé Message-ID: <20240822110238.82312-1-phi...@linaro.org> Reviewed-by: Thomas Huth Message-ID: <20240830133841.142644-34-th...@redhat.com> Signed-off-by: Thomas Huth --- MAINTAINERS

[PULL 26/42] tests/functional: Convert the ppc_hv avocado test into a standalone test

2024-09-04 Thread Thomas Huth
and there are stale read-only files already around. Message-ID: <20240830133841.142644-28-th...@redhat.com> Signed-off-by: Thomas Huth --- MAINTAINERS | 2 +- tests/functional/meson.build | 2 + .../test_ppc64_hv.py}

[PULL 09/42] tests/functional: Set up logging

2024-09-04 Thread Thomas Huth
Create log files for each test separately, one file that contains the basic logging and one that contains the console output. Reviewed-by: Daniel P. Berrangé Message-ID: <20240830133841.142644-10-th...@redhat.com> Signed-off-by: Thomas Huth --- tests/functional/qemu_test/testcase.p

[PULL 00/42] Introduce new functional test framework

2024-09-04 Thread Thomas Huth
vert ARM bFLT linux-user avocado test tests/avocado: Remove unused QemuUserTest class Thomas Huth (29): tests/avocado/avocado_qemu: Fix the "from" statements in linuxtest.py tests/avocado/boot_linux_console: Remove the s390x subtest python: Install pycotap

[PULL 07/42] python: Install pycotap in our venv if necessary

2024-09-04 Thread Thomas Huth
ere should not really be a problem). The wheel file has been obtained with: pip download --only-binary :all: --dest . --no-cache pycotap Tested-by: Philippe Mathieu-Daudé Reviewed-by: Daniel P. Berrangé Message-ID: <20240830133841.142644-8-th...@redhat.com> Signed-off-by: Thomas Huth --

[PULL 04/42] Bump avocado to 103.0

2024-09-04 Thread Thomas Huth
://avocado-framework.readthedocs.io/en/103.0/releases/lts/103_0.html Signed-off-by: Cleber Rosa Tested-by: Marcin Juszkiewicz Message-ID: <20240806173119.582857-2-cr...@redhat.com> Tested-by: Philippe Mathieu-Daudé Message-ID: <20240830133841.142644-5-th...@redhat.com> Signed-off-by:

[PULL 02/42] tests/avocado/boot_xen.py: fetch kernel during test setUp()

2024-09-04 Thread Thomas Huth
9.582857-7-cr...@redhat.com> Reviewed-by: Akihiko Odaki Tested-by: Marcin Juszkiewicz Message-ID: <20240830133841.142644-3-th...@redhat.com> Signed-off-by: Thomas Huth --- tests/avocado/boot_xen.py | 13 ++--- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/tests/avoca

Re: [PATCH for-9.2 34/53] hw/usb: Remove MUSB USB host controller

2024-09-04 Thread Thomas Huth
| 1553 - hw/usb/Kconfig|4 - hw/usb/meson.build|1 - 4 files changed, 1607 deletions(-) delete mode 100644 include/hw/usb/hcd-musb.h delete mode 100644 hw/usb/hcd-musb.c Reviewed-by: Thomas Huth

Re: [PATCH for-9.2 33/53] hw/usb: Remove tusb6010 USB controller

2024-09-04 Thread Thomas Huth
files changed, 855 deletions(-) delete mode 100644 hw/usb/tusb6010.c Reviewed-by: Thomas Huth

Re: [PATCH for-9.2 32/53] hw/block: Remove OneNAND device

2024-09-04 Thread Thomas Huth
- hw/block/meson.build | 1 - 4 files changed, 879 deletions(-) delete mode 100644 hw/block/onenand.c Reviewed-by: Thomas Huth

Re: [PATCH for-9.2 31/53] hw/input: Remove lm832x device

2024-09-04 Thread Thomas Huth
hw/input/lm832x.c | 528 -- hw/input/Kconfig | 4 - hw/input/meson.build | 1 - 5 files changed, 568 deletions(-) delete mode 100644 include/hw/input/lm832x.h delete mode 100644 hw/input/lm832x.c Reviewed-by: Thomas Huth

Re: [PATCH for-9.2 30/53] hw/rtc: Remove twl92230 device

2024-09-04 Thread Thomas Huth
/meson.build | 1 - 4 files changed, 888 deletions(-) delete mode 100644 hw/rtc/twl92230.c Reviewed-by: Thomas Huth

Re: [PATCH for-9.2 29/53] hw/input: Remove tsc210x device

2024-09-04 Thread Thomas Huth
On 03/09/2024 18.07, Peter Maydell wrote: Remove the tsc210x touchscreen controller device, which was only used by the n800 and n810. ... and the cheetah palm machine ... but you removed that earlier in the series already. Anyway: Reviewed-by: Thomas Huth The uWireSlave struct is still

Re: [PATCH for-9.2 28/53] hw/input: Remove tsc2005 touchscreen controller

2024-09-04 Thread Thomas Huth
- hw/input/Kconfig | 3 - hw/input/meson.build | 1 - hw/input/trace-events | 3 - 6 files changed, 584 deletions(-) delete mode 100644 hw/input/tsc2005.c Reviewed-by: Thomas Huth

Re: [PATCH for-9.2 27/53] hw/display: Remove Blizzard display device

2024-09-04 Thread Thomas Huth
- hw/display/Kconfig|3 - hw/display/meson.build|1 - 5 files changed, 1053 deletions(-) delete mode 100644 include/hw/display/blizzard.h delete mode 100644 hw/display/blizzard.c Reviewed-by: Thomas Huth

Re: [PATCH v5 17/44] tests/functional: Convert some tests that download files via fetch_asset()

2024-09-04 Thread Thomas Huth
On 30/08/2024 15.38, Thomas Huth wrote: Now that we've got the Asset class with pre-caching, we can convert some Avocado tests that use fetch_asset() for downloading their required files. Signed-off-by: Thomas Huth --- MAINTAINERS | 12 ++--- tests/av

Re: [PATCH for-9.2 25/53] hw/arm: Remove 'n800' and 'n810' machines

2024-09-04 Thread Thomas Huth
/machine_arm_n8x0.py | 49 - 7 files changed, 1561 deletions(-) delete mode 100644 docs/system/arm/nseries.rst delete mode 100644 hw/arm/nseries.c delete mode 100644 tests/avocado/machine_arm_n8x0.py I think tests/qtest/libqos/arm-n800-machine.c should now get removed, too? Thomas

Re: [PATCH for-9.2 02/53] hw/input: Drop ADS7846 device

2024-09-04 Thread Thomas Huth
/input/ads7846.c | 186 --- hw/input/Kconfig | 3 - hw/input/meson.build | 1 - 4 files changed, 191 deletions(-) delete mode 100644 hw/input/ads7846.c Reviewed-by: Thomas Huth

[PATCH] tests/qtest/meson.build: Add more CONFIG switches checks for the x86 tests

2024-09-04 Thread Thomas Huth
ice is not available. Signed-off-by: Thomas Huth --- tests/qtest/meson.build | 16 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build index 486974cad7..587e1dc47e 100644 --- a/tests/qtest/meson.build +++ b/tests/qtest/m

Re: [PATCH v2 01/11] hw/s390/ccw-device: Convert to three-phase reset

2024-09-03 Thread Thomas Huth
insertions(+), 3 deletions(-) Acked-by: Thomas Huth

Re: [PATCH] tests/functional/test_vnc: Reduce raciness in find_free_ports()

2024-09-03 Thread Thomas Huth
aybe think of a logic that avoids clashes between different tests, too. Thomas

[PATCH] tests/qtest/hd-geo-test: Check for availability of "pc" machine before using it

2024-09-03 Thread Thomas Huth
In case QEMU has been configured with "--without-default-devices", the "pc" machine type might be missing in the binary. We should check for its availability before using it. Signed-off-by: Thomas Huth --- tests/qtest/hd-geo-test.c | 71 +--

[PATCH] tests/qtest/boot-order-test: Make the machine name mandatory in this test

2024-09-03 Thread Thomas Huth
Let's make sure that we always pass a machine name to the test_boot_orders() function, so we can check whether the machine is available in the binary and skip the test in case it is not included in the build. Signed-off-by: Thomas Huth --- tests/qtest/boot-order-test.c | 4 ++-- 1 file ch

[PATCH] tests/qtest/cdrom-test: Improve the machine detection in the cdrom test

2024-09-03 Thread Thomas Huth
When configuring QEMU with the --without-default-devices switch, these tests are currently failing since they assume that the "pc" and "q35" machines are always available. Add some proper checks to make the test work without these machines, too. Signed-off-by: Thomas Huth --

[PATCH] tests/qtest/migration: Add a check for the availability of the "pc" machine

2024-09-03 Thread Thomas Huth
The test_vcpu_dirty_limit is the only test that does not check for the availability of the machine before starting the test, so it fails when QEMU has been configured with --without-default-devices. Add a check for the "pc" machine type to fix it. Signed-off-by: Thomas Huth --- t

[Bug 2078790] Re: jammy qemu x86 int3: 0000 [#1] SMP NOPTI

2024-09-03 Thread Thomas Huth
** Project changed: qemu => qemu (Ubuntu) -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/2078790 Title: jammy qemu x86 int3: [#1] SMP NOPTI Status in linux-lowlatency-hwe-6.8 package in Ubuntu

Re: tests/avocado/riscv_opensbi.py does not work reliable

2024-09-02 Thread Thomas Huth
On 03/09/2024 01.54, Alistair Francis wrote: On Sat, Aug 31, 2024 at 1:35 AM Thomas Huth wrote: Hi! While running a lot of tests (i.e. with a very loaded machine), I noticed that tests/avocado/riscv_opensbi.py is very flaky when the host machine is slow. I can easily reproduce the

Re: [PATCH v2 2/2] hw/char: sifive_uart: Print uart charecters async

2024-09-02 Thread Thomas Huth
qemu_chr_fe_write(&s->chr, charecters, numptr); + +if (ret >= 0) { +/* We wrote the data, actuallly pop the fifo */ s/actuallly/actually/ I recommend running checkpatch.pl with the --codespell flag ;-) Anyway, this indeed seems to fix the problem with the avocado test, thank you very much for tackling this! Tested-by: Thomas Huth

Re: [PATCH v2 1/2] hw/char: riscv_htif: Use blocking qemu_chr_fe_write_all

2024-09-02 Thread Thomas Huth
+-- 1 file changed, 10 insertions(+), 2 deletions(-) With above typos fixed: Reviewed-by: Thomas Huth

[PATCH] tests/functional: Convert the multiprocess avocado test into a standalone test

2024-09-02 Thread Thomas Huth
This test handles both, aarch64 and x86_64, with the same test code (apart from some initial setup), so don't split this file by target but add a check for self.arch in the main test function. Signed-off-by: Thomas Huth --- Based-on: 20240830133841.142644-1-th...@redhat.com tests/av

[PATCH] contrib/plugins/Makefile: Add a 'distclean' target

2024-09-02 Thread Thomas Huth
ith "make distclean", add this command to one of the build jobs, too. Signed-off-by: Thomas Huth --- .gitlab-ci.d/buildtest.yml | 2 ++ contrib/plugins/Makefile | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml i

Re: [PATCH v5 32/44] tests/functional: Convert the vnc test

2024-09-02 Thread Thomas Huth
On 02/09/2024 11.48, Philippe Mathieu-Daudé wrote: On 30/8/24 15:38, Thomas Huth wrote: Nothing thrilling in here, it's just a straight forward conversion. Signed-off-by: Thomas Huth ---   tests/functional/meson.build |  1 +   tests/{avocado/vnc.py => fu

Re: [PATCH 3/3] hw/nubus/nubus-device: Range check 'slot' property

2024-08-30 Thread Thomas Huth
evice.c | 7 +++ > 1 file changed, 7 insertions(+) Reviewed-by: Thomas Huth

Re: [PATCH 2/3] hw/m68k/mcf5208: Add URLs for datasheets

2024-08-30 Thread Thomas Huth
8 > 1 file changed, 8 insertions(+) Reviewed-by: Thomas Huth

Re: [PATCH 1/3] hw/m68k/mcf5208: Avoid shifting off end of integer

2024-08-30 Thread Thomas Huth
*opaque, hwaddr addr, > { > int n; > for (n = 0; n < 32; n++) { > -if (current_machine->ram_size < (2u << n)) { > +if (current_machine->ram_size < (2ULL << n)) { > break; > } > } Reviewed-by: Thomas Huth

tests/avocado/riscv_opensbi.py does not work reliable

2024-08-30 Thread Thomas Huth
does the interface not allow this?)? Otherwise, I think it might be best to mark the spike and sifive_u tests with QEMU_TEST_FLAKY_TESTS here to make it clear that these tests are not reliable by default...? Thomas

[PATCH v5 29/44] tests/functional: Convert the acpi-bits test into a standalone test

2024-08-30 Thread Thomas Huth
Mostly a straight-forward conversion. Looks like we can simply drop the avocado datadrainer stuff when not using the avocado framework anymore. Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- MAINTAINERS

[PATCH v5 22/44] tests/functional: Convert the microblaze avocado tests into standalone tests

2024-08-30 Thread Thomas Huth
Signed-off-by: Thomas Huth --- MAINTAINERS | 2 +- tests/avocado/machine_microblaze.py | 61 --- tests/functional/meson.build | 8 +++ .../functional/test_microblaze_s3adsp1800.py | 40

[PATCH v5 38/44] tests/functional: Convert ARM bFLT linux-user avocado test

2024-08-30 Thread Thomas Huth
phi...@linaro.org> [thuth: Add test to meson.build] Signed-off-by: Thomas Huth --- tests/avocado/load_bflt.py | 54 - tests/functional/meson.build| 4 +++ tests/functional/qemu_test/utils.py | 9 + tests/functional/test_arm_bflt.py

[PATCH v5 43/44] docs/devel/testing: Rename avocado_qemu.Test class

2024-08-30 Thread Thomas Huth
The avocado_qemu.Test class has been renamed a while back in commit 2283b627bc ("tests/avocado: Rename avocado_qemu.Test -> QemuSystemTest"), so we should reflect this now in the documentation, too. Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- docs/devel/testin

[PATCH v5 04/44] Bump avocado to 103.0

2024-08-30 Thread Thomas Huth
://avocado-framework.readthedocs.io/en/103.0/releases/lts/103_0.html Signed-off-by: Cleber Rosa Tested-by: Marcin Juszkiewicz Message-ID: <20240806173119.582857-2-cr...@redhat.com> Tested-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- pythondeps.toml| 2 +- tests/Makefile.i

[PATCH v5 15/44] tests/functional: enable pre-emptive caching of assets

2024-08-30 Thread Thomas Huth
ys done by make check-functional Signed-off-by: Daniel P. Berrangé Tested-by: Richard Henderson [thuth: Remove the duplicated "path = os.path.basename(...)" line] Signed-off-by: Thomas Huth --- tests/Makefile.include | 3 ++- tests/fu

[PATCH v5 24/44] tests/functional: Convert the virtio_gpu avocado test into a standalone test

2024-08-30 Thread Thomas Huth
Nothing thrilling in here, it's just a straight forward conversion. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- tests/functional/meson.build | 1 + .../test_virtio_gpu.py} | 64 +-- 2 files changed, 30 inser

[PATCH v5 39/44] tests/avocado: Remove unused QemuUserTest class

2024-08-30 Thread Thomas Huth
From: Philippe Mathieu-Daudé The single test that was using the QemuUserTest class has been converted to the functional test framework. This class is now unused, remove it. Signed-off-by: Philippe Mathieu-Daudé Message-ID: <20240822104238.75045-4-phi...@linaro.org> Signed-off-by: Thoma

[PATCH v5 23/44] tests/functional: Convert the riscv_opensbi avocado test into a standalone test

2024-08-30 Thread Thomas Huth
ned-off-by: Thomas Huth --- MAINTAINERS| 1 + tests/avocado/riscv_opensbi.py | 63 -- tests/functional/meson.build | 8 tests/functional/test_riscv_opensbi.py | 40 4 files changed, 49 insertions(+), 63

[PATCH v5 31/44] tests/functional: Convert the linux_initrd avocado test into a standalone test

2024-08-30 Thread Thomas Huth
While we're at it, update the SHA1 hashes to SHA256 hashes since SHA1 should not be used anymore nowadays. Signed-off-by: Thomas Huth --- tests/functional/meson.build | 1 + .../test_linux_initrd.py} | 37 +++ 2 files changed, 22 inser

[PATCH v5 42/44] docs/devel/testing: Split the Avocado documentation into a separate file

2024-08-30 Thread Thomas Huth
The main testing documentation file got very overloaded already. Thus let's split the Avocado information into a separate file. Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- docs/devel/testing/avocado.rst | 581 + docs/devel/testing/inde

[PATCH v5 26/44] tests/functional: Convert the ppc_amiga avocado test into a standalone test

2024-08-30 Thread Thomas Huth
Use the Python standard zipfile module instead of avocado.utils for extracting the ZIP file that we download here, and use the standard subprocess module for running the "tail" command. Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Signed-off-by: T

[PATCH v5 40/44] gitlab-ci: Add "check-functional" to the build tests

2024-08-30 Thread Thomas Huth
Now that we converted many tests from the "check-avocado" test suite to the "check-functional" test suite, we should make sure that these also get tested in the CI. Reviewed-by: Daniel P. Berrangé Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- .g

[PATCH v5 41/44] docs/devel: Split testing docs from the build docs and move to separate folder

2024-08-30 Thread Thomas Huth
Building and testing are two separate topics, so let's split the testing into a separate category and move the related files into a separate folder. Signed-off-by: Thomas Huth --- docs/devel/index-build.rst | 14 -- docs/devel/inde

[PATCH v5 37/44] tests/functional: Add QemuUserTest class

2024-08-30 Thread Thomas Huth
tion. Signed-off-by: Philippe Mathieu-Daudé Message-ID: <20240822104238.75045-2-phi...@linaro.org> Signed-off-by: Thomas Huth --- tests/functional/qemu_test/__init__.py | 2 +- tests/functional/qemu_test/testcase.py | 17 + 2 files changed, 18 insertions(+), 1 deletion(-) dif

[PATCH v5 33/44] tests/functional: Convert ARM Integrator/CP avocado tests

2024-08-30 Thread Thomas Huth
OK 3.90s 2 subtests passed Signed-off-by: Philippe Mathieu-Daudé Reviewed-by: Thomas Huth Message-ID: <20240822110238.82312-1-phi...@linaro.org> Signed-off-by: Thomas Huth --- MAINTAINERS | 2 +- tests/functional/meson.build

[PATCH v5 11/44] tests/functional: Prepare the meson build system for the functional tests

2024-08-30 Thread Thomas Huth
ed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- tests/Makefile.include | 11 + tests/functional/meson.build | 81 tests/meson.build| 1 + 3 files changed, 93 insertions(+) create mode 10064

[PATCH v5 44/44] docs/devel/testing: Add documentation for functional tests

2024-08-30 Thread Thomas Huth
Document the new functional testing framework. The text is originally based on the Avocado documentation, but heavily modified to match the new framework. Signed-off-by: Thomas Huth --- docs/devel/testing/functional.rst | 338 ++ docs/devel/testing/index.rst

[PATCH v5 32/44] tests/functional: Convert the vnc test

2024-08-30 Thread Thomas Huth
Nothing thrilling in here, it's just a straight forward conversion. Signed-off-by: Thomas Huth --- tests/functional/meson.build | 1 + tests/{avocado/vnc.py => functional/test_vnc.py} | 12 +++- 2 files changed, 8 insertions(+), 5 deletions(-) rename tests/

[PATCH v5 19/44] tests/functional: Convert some avocado tests that needed avocado.utils.archive

2024-08-30 Thread Thomas Huth
Instead of using the "archive" module from avocado.utils, switch these tests to use the new wrapper function that is based on the "tarfile" module instead. Reviewed-by: Daniel P. Berrangé Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Signed

[PATCH v5 36/44] tests/functional: Convert mips64el Fuloong2e avocado test (1/2)

2024-08-30 Thread Thomas Huth
ssed 3/4 qemu:func-quick+func-mips64el / func-mips64el-info_usernet OK 0.15s 1 subtests passed 4/4 qemu:func-quick+func-mips64el / func-mips64el-mips64el_fuloong2e OK 0.19s 1 subtests passed Signed-off-by: Philippe Mathieu-Daudé Reviewed-by: Thomas Huth Message-ID: <202408270

[PATCH v5 00/44] Convert avocado tests to normal Python unittests

2024-08-30 Thread Thomas Huth
ocado tests tests/functional: Convert mips64el Fuloong2e avocado test (1/2) tests/functional: Add QemuUserTest class tests/functional: Convert ARM bFLT linux-user avocado test tests/avocado: Remove unused QemuUserTest class Thomas Huth (31): tests/avocado/avocado_qemu: Fix the "from

[PATCH v5 18/44] tests/functional: Add a function for extracting files from an archive

2024-08-30 Thread Thomas Huth
with this functionality, so let's just add a nice wrapper function around that. Reviewed-by: Daniel P. Berrangé Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Tested-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- tests/functional/qemu_test/utils.py | 21 ++

[PATCH v5 35/44] tests/functional: Convert Aarch64 Virt machine avocado tests

2024-08-30 Thread Thomas Huth
subtests passed ... Signed-off-by: Philippe Mathieu-Daudé Message-ID: <20240826103634.52384-1-phi...@linaro.org> Signed-off-by: Thomas Huth --- MAINTAINERS | 2 +- tests/functional/meson.build | 2 + tests/functional/qemu_test/__ini

[PATCH v5 34/44] tests/functional: Convert Aarch64 SBSA-Ref avocado tests

2024-08-30 Thread Thomas Huth
-Daudé Reviewed-by: Thomas Huth Reviewed-by: Marcin Juszkiewicz Message-ID: <20240823131614.10269-1-phi...@linaro.org> Signed-off-by: Thomas Huth --- MAINTAINERS | 2 +- tests/functional/meson.build | 5 + .../test_aarch64_sbsa

[PATCH v5 16/44] tests/functional: Allow asset downloading with concurrent threads

2024-08-30 Thread Thomas Huth
When running "make -j$(nproc) check-functional", tests that use the same asset might be running in parallel. Improve the downloading to detect this situation and wait for the other thread to finish the download. Signed-off-by: Thomas Huth --- tests/functional/qemu_test/ass

[PATCH v5 20/44] tests/functional: Convert the s390x avocado tests into standalone tests

2024-08-30 Thread Thomas Huth
ymore nowadays. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- MAINTAINERS | 4 +- tests/functional/meson.build | 6 ++ tests/functional/qemu_test/utils.py | 14 +++ .../test_s390x_ccw_virtio.py}

[PATCH v5 12/44] tests/functional: Convert simple avocado tests into standalone python tests

2024-08-30 Thread Thomas Huth
em directly. Reviewed-by: Daniel P. Berrangé Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- MAINTAINERS | 5 ++ tests/functional/meson.build | 5 ++ .../test_cpu_queries.py}

[PATCH v5 03/44] tests/avocado/machine_aarch64_sbsaref.py: allow for rw usage of image

2024-08-30 Thread Thomas Huth
ewicz Message-ID: <20240806173119.582857-9-cr...@redhat.com> Signed-off-by: Thomas Huth --- tests/avocado/machine_aarch64_sbsaref.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/avocado/machine_aarch64_sbsaref.py b/tests/avocado/machine_aarch64_sbsaref.py index

[PATCH v5 25/44] tests/functional: Convert most ppc avocado tests into standalone tests

2024-08-30 Thread Thomas Huth
Nothing thrilling in here, just straight forward conversions. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- MAINTAINERS | 9 +-- tests/functional/meson.build | 11 .../test_ppc64_powernv.py}| 56

[PATCH v5 30/44] tests/functional: Convert the rx_gdbsim avocado test into a standalone test

2024-08-30 Thread Thomas Huth
Provide a "gzip_uncompress" function based on the standard "gzip" module to avoid the usage of avocado.utils here. Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- MAINTAINERS | 2

[PATCH v5 21/44] tests/functional: Convert the x86_cpu_model_versions test

2024-08-30 Thread Thomas Huth
Nothing thrilling in here, it's just a straight forward conversion. Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- MAINTAINERS | 1 + tests/functional/meson.build

[PATCH v5 01/44] tests/avocado: machine aarch64: standardize location and RO access

2024-08-30 Thread Thomas Huth
y: Cleber Rosa Tested-by: Marcin Juszkiewicz Message-ID: <20240806173119.582857-5-cr...@redhat.com> Signed-off-by: Thomas Huth --- tests/avocado/machine_aarch64_sbsaref.py | 2 +- tests/avocado/machine_aarch64_virt.py| 14 +++--- 2 files changed, 8 insertions(+), 8 deletions

[PATCH v5 05/44] tests/avocado/avocado_qemu: Fix the "from" statements in linuxtest.py

2024-08-30 Thread Thomas Huth
Without this change, the new Avocado v103 fails to find the tests that are based on the LinuxTest class. Suggested-by: Cleber Rosa Tested-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- tests/avocado/avocado_qemu/linuxtest.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions

[PATCH v5 28/44] tests/functional: Convert the m68k nextcube test with tesseract

2024-08-30 Thread Thomas Huth
thieu-Daudé Tested-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- MAINTAINERS | 1 + tests/avocado/tesseract_utils.py | 46 --- tests/functional/meson.build | 4 ++ tests/functional/qemu_test/tessera

[PATCH v5 08/44] tests/functional: Add base classes for the upcoming pytest-based tests

2024-08-30 Thread Thomas Huth
will be set via meson.build later. Signed-off-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- MAINTAINERS| 5 + tests/functional/qemu_test/__init__.py | 13 ++ tests/functional/qemu_test/cmd.py | 178 + tests/functional/qemu_test

[PATCH v5 27/44] tests/functional: Convert the ppc_hv avocado test into a standalone test

2024-08-30 Thread Thomas Huth
and there are stale read-only files already around. Signed-off-by: Thomas Huth --- MAINTAINERS | 2 +- tests/functional/meson.build | 2 + .../test_ppc64_hv.py} | 52 --- 3 files changed, 24 inserti

[PATCH v5 06/44] tests/avocado/boot_linux_console: Remove the s390x subtest

2024-08-30 Thread Thomas Huth
y. Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- tests/avocado/boot_linux_console.py | 20 tests/avocado/machine_s390_ccw_virtio.py | 2 ++ 2 files changed, 2 insertions(+), 20 deletions(-) diff --git a/tests/avocado/boot_linux_console.py

[PATCH v5 17/44] tests/functional: Convert some tests that download files via fetch_asset()

2024-08-30 Thread Thomas Huth
Now that we've got the Asset class with pre-caching, we can convert some Avocado tests that use fetch_asset() for downloading their required files. Signed-off-by: Thomas Huth --- MAINTAINERS | 12 ++--- tests/avocado/machine_mips_loongson3v.py

[PATCH v5 10/44] tests/Makefile.include: Increase the level of indentation in the help text

2024-08-30 Thread Thomas Huth
The next patch is going to add some entries that need more space between the command and the help text, so let's increase the indentation here first. Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Reviewed-by: Alex Bennée Signed-off-by: Thomas Huth ---

[PATCH v5 07/44] python: Install pycotap in our venv if necessary

2024-08-30 Thread Thomas Huth
ere should not really be a problem). The wheel file has been obtained with: pip download --only-binary :all: --dest . --no-cache pycotap Tested-by: Philippe Mathieu-Daudé Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- python/wheels/pycotap-1.3.1-py3-none-any.whl | Bin 0 -&g

[PATCH v5 09/44] tests/functional: Set up logging

2024-08-30 Thread Thomas Huth
Create log files for each test separately, one file that contains the basic logging and one that contains the console output. Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- tests/functional/qemu_test/testcase.py | 27 +- 1 file changed, 26 insertions

[PATCH v5 14/44] tests/functional: add a module for handling asset download & caching

2024-08-30 Thread Thomas Huth
hilippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé [thuth: Drop sha1 support, use hash on file content for naming instead of URL, add the possibility to specify the cache dir via environment variable] Signed-off-by: Thomas Huth --- tests/functional/qemu_test/__init__.py | 1 + t

[PATCH v5 13/44] tests/functional: Convert avocado tests that just need a small adjustment

2024-08-30 Thread Thomas Huth
ed-by: Daniel P. Berrangé Tested-by: Philippe Mathieu-Daudé Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- MAINTAINERS | 3 + tests/functional/meson.build | 6 ++ .../test_info_usernet.py} | 11

[PATCH v5 02/44] tests/avocado/boot_xen.py: fetch kernel during test setUp()

2024-08-30 Thread Thomas Huth
by: Marcin Juszkiewicz Message-ID: <20240806173119.582857-7-cr...@redhat.com> Signed-off-by: Thomas Huth --- tests/avocado/boot_xen.py | 13 ++--- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/tests/avocado/boot_xen.py b/tests/avocado/boot_xen.py index f29bc58b9e..4

Re: [PATCH for-9.2 00/10] s390: Convert virtio-ccw, cpu to three-phase reset, and followup cleanup

2024-08-30 Thread Thomas Huth
you take the s390 specific patches, and I take the remaining reset-cleanup patches once those have landed Any preference? I'm fine if you take the patches through your tree along with the other reset-related patches. Thomas

Re: [PATCH v4 15/35] tests/functional: enable pre-emptive caching of assets

2024-08-30 Thread Thomas Huth
On 30/08/2024 09.42, Daniel P. Berrangé wrote: On Fri, Aug 30, 2024 at 09:38:17AM +0200, Thomas Huth wrote: On 29/08/2024 12.15, Daniel P. Berrangé wrote: On Tue, Aug 27, 2024 at 04:24:59PM +0200, Thomas Huth wrote: On 27/08/2024 15.16, Thomas Huth wrote: On 23/08/2024 09.28, Philippe

Re: [PATCH v4 15/35] tests/functional: enable pre-emptive caching of assets

2024-08-30 Thread Thomas Huth
On 29/08/2024 12.15, Daniel P. Berrangé wrote: On Tue, Aug 27, 2024 at 04:24:59PM +0200, Thomas Huth wrote: On 27/08/2024 15.16, Thomas Huth wrote: On 23/08/2024 09.28, Philippe Mathieu-Daudé wrote: Hi, On 21/8/24 10:27, Thomas Huth wrote: From: Daniel P. Berrangé Many tests need to

Re: [PATCH v4 35/35] docs/devel/testing: Add documentation for functional tests

2024-08-29 Thread Thomas Huth
On 29/08/2024 12.34, Daniel P. Berrangé wrote: On Wed, Aug 21, 2024 at 10:27:36AM +0200, Thomas Huth wrote: Document the new functional testing framework. The text is originally based on the Avocado documentation, but heavily modified to match the new framework. Signed-off-by: Thomas Huth

<    1   2   3   4   5   6   7   8   9   10   >