[PATCH v7 0/8] Acceptance test: Add "boot_linux" acceptance test
This acceptance test, validates that a full blown Linux guest can successfully boot in QEMU. In this specific case, the guest chosen is Fedora version 31. * x86_64, pc and q35 machine types, with and without kvm as an accellerator * aarch64 and virt machine type, with and without kvm as an accellerator * ppc64 and pseries machine type * s390x and s390-ccw-virtio machine type This has been tested on x86_64 and ppc64le hosts and has been running reliably (in my experience) on Travis CI. Some hopefully useful pointers for reviewers: = Git Info: - URI: https://github.com/clebergnu/qemu/tree/test_boot_linux_v7 - Remote: https://github.com/clebergnu/qemu - Branch: test_boot_linux_v7 Travis CI Info: - Build: https://travis-ci.org/clebergnu/qemu/builds/606191094 - Job: https://travis-ci.org/clebergnu/qemu/jobs/606191120 Previous version: - v6: https://lists.gnu.org/archive/html/qemu-devel/2019-06/msg01202.html - v5: https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg04652.html - v4: https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg02032.html - v3: https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg01677.html - v2: https://lists.gnu.org/archive/html/qemu-devel/2018-11/msg04318.html - v1: http://lists.nongnu.org/archive/html/qemu-devel/2018-09/msg02530.html Changes from v6: * Bumped Fedora to most recently released version (31). * Included new architectures (ppc64 and s390x), consolidating all tests into the same commit. * New commit: "Acceptance tests: use avocado tags for machine type" * New commit: "Acceptance tests: introduce utility method for tags unique vals" * New commit: "Acceptance test x86_cpu_model_versions: use default vm", needed to normalize the use of the machine type tags * Added a lot of leniency to the test setup (and reliability to the test/job), canceling the test if there are any failures while downloading/preparing the boot images. * Made use of Avocado's data drainer a regular feature (dropped the commit with RFC) and squashed it. * Bumped pycdlib version to 1.8.0 * Dropped explicit "--enable-slirp=git" (added on v5) to Travis CI configure line, as the default configuration on Travis CI now results in user networking capabilities. Changes from v5: * Added explicit "--enable-slirp=git" to Travis CI configure line, as these tests depend on "-netdev user" like networking. * Bumped Fedora to most recently released version (30). * Changed "checksum" parameter to 'sha256' and use the same hashes as provided by the Fedora project (instead of using Avocado's default sha1 and compute and use a different hash value). * New commit: Add "boot_linux" test for aarch64 and virt machine type * New commit: [RFC]: use Avocado data drainer for console logging Changes from v4: * New commit "Acceptance tests: use relative location for tests" * New commit "Acceptance tests: keep a stable reference to the QEMU build dir" * Pinned the Fedora 29 image by adding a checksum. The goal is to never allow more than one component to change at a time (the one allowed to change is QEMU itself). Updates to the image should be manual. (Based on comments from Cornelia) * Moved the downloading of the Fedora 29 cloud image to the test setUp() method, canceling the test if the image can not be downloaded. * Removed the ":avocado: enable" tag, given that Avocado versions 68.0 and later operate on a "recursive by default" manner, that is able to correctly identify this as an Avocado test. Changes from v3: * New patch "Acceptance tests: depend on qemu-img" Known Issues on v3 (no longer applicable): == * A recent TCG performance regression[1] affects this test in a number of ways: - The test execution may timeout by itself - The generation of SSH host keys in the guest's first boot is also affected (possibly also a timeout) - The cloud-init "phone home" feature attempts to read the host keys and fails, causing the test to timeout and fail These are not observed anymore once the fix[2] is applied. [1] - https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg00338.html [2] - https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg01129.html Changes from v2: * Updated the tag to include the "arch:" key, in a similar fashion as to the tests in the "Acceptance Tests: target architecture support": - https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg00369.html * Renamed the test method name to test_x86_64_pc, again, similarly to the boot_linux_console.py tests in the series mentioned before. * Set the machine type explicitly, again similarly to the boot_linux_console.py tests in the series mentioned before. * Added messages after the launch of the VM, to let
Re: [PATCH v7 0/8] Acceptance test: Add "boot_linux" acceptance test
Patchew URL: https://patchew.org/QEMU/20191104151323.9883-1-cr...@redhat.com/
Hi,
This series seems to have some coding style problems. See output below for
more information:
Subject: [PATCH v7 0/8] Acceptance test: Add "boot_linux" acceptance test
Type: series
Message-id: 20191104151323.9883-1-cr...@redhat.com
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
8675afa Acceptance test: add "boot_linux" tests
409affc Acceptance tests: depend on qemu-img
58c0f47 Acceptance tests: add the build directory to the system PATH
236ecfa Acceptance tests: keep a stable reference to the QEMU build dir
6a147b4 Acceptance tests: use relative location for tests
bcd92a3 Acceptance tests: use avocado tags for machine type
0f52135 Acceptance tests: introduce utility method for tags unique vals
050f1d1 Acceptance test x86_cpu_model_versions: use default vm
=== OUTPUT BEGIN ===
1/8 Checking commit 050f1d1ea660 (Acceptance test x86_cpu_model_versions: use
default vm)
ERROR: line over 90 characters
#47: FILE: tests/acceptance/x86_cpu_model_versions.py:248:
+self.vm.add_args('-cpu',
'Cascadelake-Server,x-force-features=on,check=off,enforce=off')
ERROR: line over 90 characters
#61: FILE: tests/acceptance/x86_cpu_model_versions.py:256:
+self.vm.add_args('-cpu',
'Cascadelake-Server,x-force-features=on,check=off,enforce=off')
ERROR: line over 90 characters
#76: FILE: tests/acceptance/x86_cpu_model_versions.py:265:
+self.vm.add_args('-cpu',
'Cascadelake-Server,x-force-features=on,check=off,enforce=off,+arch-capabilities')
ERROR: line over 90 characters
#90: FILE: tests/acceptance/x86_cpu_model_versions.py:273:
+self.vm.add_args('-cpu',
'Cascadelake-Server,x-force-features=on,check=off,enforce=off,-arch-capabilities')
ERROR: line over 90 characters
#105: FILE: tests/acceptance/x86_cpu_model_versions.py:282:
+self.vm.add_args('-cpu',
'Cascadelake-Server-v1,x-force-features=on,check=off,enforce=off')
ERROR: line over 90 characters
#120: FILE: tests/acceptance/x86_cpu_model_versions.py:290:
+self.vm.add_args('-cpu',
'Cascadelake-Server-v2,x-force-features=on,check=off,enforce=off')
ERROR: line over 90 characters
#123: FILE: tests/acceptance/x86_cpu_model_versions.py:293:
+'pc-i440fx-4.0 + Cascadelake-Server-v2 should have
arch-capabilities')
ERROR: line over 90 characters
#136: FILE: tests/acceptance/x86_cpu_model_versions.py:299:
+self.vm.add_args('-cpu',
'Cascadelake-Server-v1,x-force-features=on,check=off,enforce=off,+arch-capabilities')
ERROR: line over 90 characters
#139: FILE: tests/acceptance/x86_cpu_model_versions.py:302:
+'pc-i440fx-4.0 +
Cascadelake-Server-v1,+arch-capabilities should have arch-capabilities')
ERROR: line over 90 characters
#150: FILE: tests/acceptance/x86_cpu_model_versions.py:307:
+self.vm.add_args('-cpu',
'Cascadelake-Server-v2,x-force-features=on,check=off,enforce=off,-arch-capabilities')
total: 10 errors, 0 warnings, 134 lines checked
Patch 1/8 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
2/8 Checking commit 0f5213583c78 (Acceptance tests: introduce utility method
for tags unique vals)
3/8 Checking commit bcd92a334692 (Acceptance tests: use avocado tags for
machine type)
WARNING: line over 80 characters
#59: FILE: tests/acceptance/avocado_qemu/__init__.py:119:
+
default=self._get_unique_tag_val('machine'))
total: 0 errors, 1 warnings, 450 lines checked
Patch 3/8 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
4/8 Checking commit 6a147b4b2eb1 (Acceptance tests: use relative location for
tests)
5/8 Checking commit 236ecfa68ee0 (Acceptance tests: keep a stable reference to
the QEMU build dir)
ERROR: line over 90 characters
#48: FILE: tests/acceptance/avocado_qemu/__init__.py:19:
+SRC_ROOT_DIR =
os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirname(__file__
total: 1 errors, 0 warnings, 8 lines checked
Patch 5/8 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
6/8 Checking commit 58c0f475d5ec (Acceptance tests: add the build directory to
the system PATH)
7/8 Checking commit 409affc034ce (Acceptance tests: depend on qemu-im
Re: [PATCH v7 0/8] Acceptance test: Add "boot_linux" acceptance test
Hi Cleber, On 11/4/19 1:13 PM, Cleber Rosa wrote: This acceptance test, validates that a full blown Linux guest can successfully boot in QEMU. In this specific case, the guest chosen is Fedora version 31. * x86_64, pc and q35 machine types, with and without kvm as an accellerator * aarch64 and virt machine type, with and without kvm as an accellerator * ppc64 and pseries machine type * s390x and s390-ccw-virtio machine type This has been tested on x86_64 and ppc64le hosts and has been running reliably (in my experience) on Travis CI. Some hopefully useful pointers for reviewers: = Git Info: - URI: https://github.com/clebergnu/qemu/tree/test_boot_linux_v7 - Remote: https://github.com/clebergnu/qemu - Branch: test_boot_linux_v7 Travis CI Info: - Build: https://travis-ci.org/clebergnu/qemu/builds/606191094 - Job: https://travis-ci.org/clebergnu/qemu/jobs/606191120 Previous version: - v6: https://lists.gnu.org/archive/html/qemu-devel/2019-06/msg01202.html - v5: https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg04652.html - v4: https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg02032.html - v3: https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg01677.html - v2: https://lists.gnu.org/archive/html/qemu-devel/2018-11/msg04318.html - v1: http://lists.nongnu.org/archive/html/qemu-devel/2018-09/msg02530.html Changes from v6: * Bumped Fedora to most recently released version (31). * Included new architectures (ppc64 and s390x), consolidating all tests into the same commit. * New commit: "Acceptance tests: use avocado tags for machine type" * New commit: "Acceptance tests: introduce utility method for tags unique vals" It seems 02 and 03 are re-sending of patches originally sent in the series with Message-Id: 20190924194501.9303-1-cr...@redhat.com. On the original thread you can find my reviews. - Wainer * New commit: "Acceptance test x86_cpu_model_versions: use default vm", needed to normalize the use of the machine type tags * Added a lot of leniency to the test setup (and reliability to the test/job), canceling the test if there are any failures while downloading/preparing the boot images. * Made use of Avocado's data drainer a regular feature (dropped the commit with RFC) and squashed it. * Bumped pycdlib version to 1.8.0 * Dropped explicit "--enable-slirp=git" (added on v5) to Travis CI configure line, as the default configuration on Travis CI now results in user networking capabilities. Changes from v5: * Added explicit "--enable-slirp=git" to Travis CI configure line, as these tests depend on "-netdev user" like networking. * Bumped Fedora to most recently released version (30). * Changed "checksum" parameter to 'sha256' and use the same hashes as provided by the Fedora project (instead of using Avocado's default sha1 and compute and use a different hash value). * New commit: Add "boot_linux" test for aarch64 and virt machine type * New commit: [RFC]: use Avocado data drainer for console logging Changes from v4: * New commit "Acceptance tests: use relative location for tests" * New commit "Acceptance tests: keep a stable reference to the QEMU build dir" * Pinned the Fedora 29 image by adding a checksum. The goal is to never allow more than one component to change at a time (the one allowed to change is QEMU itself). Updates to the image should be manual. (Based on comments from Cornelia) * Moved the downloading of the Fedora 29 cloud image to the test setUp() method, canceling the test if the image can not be downloaded. * Removed the ":avocado: enable" tag, given that Avocado versions 68.0 and later operate on a "recursive by default" manner, that is able to correctly identify this as an Avocado test. Changes from v3: * New patch "Acceptance tests: depend on qemu-img" Known Issues on v3 (no longer applicable): == * A recent TCG performance regression[1] affects this test in a number of ways: - The test execution may timeout by itself - The generation of SSH host keys in the guest's first boot is also affected (possibly also a timeout) - The cloud-init "phone home" feature attempts to read the host keys and fails, causing the test to timeout and fail These are not observed anymore once the fix[2] is applied. [1] - https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg00338.html [2] - https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg01129.html Changes from v2: * Updated the tag to include the "arch:" key, in a similar fashion as to the tests in the "Acceptance Tests: target architecture support": - https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg00369.htm
