from:"Wainer dos Santos Moschetta"

Re: [PATCH 01/15] tests/avocado: add RISC-V opensbi boot test

2022-12-27 Thread Wainer dos Santos Moschetta


Hi Daniel,

On 12/21/22 15:22, Daniel Henrique Barboza wrote:

This test is used to do a quick sanity check to ensure that we're able
to run the existing QEMU FW image.

'sifive_u', 'spike' and 'virt' riscv64 machines, and 'sifive_u' and
'virt' 32 bit machines are able to run the default RISCV64_BIOS_BIN |
RISCV32_BIOS_BIN firmware with minimal options.

Cc: Cleber Rosa 
Cc: Philippe Mathieu-Daudé 
Cc: Wainer dos Santos Moschetta 
Cc: Beraldo Leal 
Signed-off-by: Daniel Henrique Barboza 
---
  tests/avocado/riscv_opensbi.py | 65 ++
  1 file changed, 65 insertions(+)
  create mode 100644 tests/avocado/riscv_opensbi.py


It looks good to me. Thanks!

Reviewed-by: Wainer dos Santos Moschetta 



diff --git a/tests/avocado/riscv_opensbi.py b/tests/avocado/riscv_opensbi.py
new file mode 100644
index 00..abc99ced30
--- /dev/null
+++ b/tests/avocado/riscv_opensbi.py
@@ -0,0 +1,65 @@
+# opensbi boot test for RISC-V machines
+#
+# Copyright (c) 2022, Ventana Micro
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+
+from avocado_qemu import QemuSystemTest
+from avocado_qemu import wait_for_console_pattern
+
+class RiscvOpensbi(QemuSystemTest):
+"""
+:avocado: tags=accel:tcg
+"""
+timeout = 5
+
+def test_riscv64_virt(self):
+"""
+:avocado: tags=arch:riscv64
+:avocado: tags=machine:virt
+"""
+self.vm.set_console()
+self.vm.launch()
+wait_for_console_pattern(self, 'Platform Name')
+wait_for_console_pattern(self, 'Boot HART MEDELEG')
+
+def test_riscv64_spike(self):
+"""
+:avocado: tags=arch:riscv64
+:avocado: tags=machine:spike
+"""
+self.vm.set_console()
+self.vm.launch()
+wait_for_console_pattern(self, 'Platform Name')
+wait_for_console_pattern(self, 'Boot HART MEDELEG')
+
+def test_riscv64_sifive_u(self):
+"""
+:avocado: tags=arch:riscv64
+:avocado: tags=machine:sifive_u
+"""
+self.vm.set_console()
+self.vm.launch()
+wait_for_console_pattern(self, 'Platform Name')
+wait_for_console_pattern(self, 'Boot HART MEDELEG')
+
+def test_riscv32_virt(self):
+"""
+:avocado: tags=arch:riscv32
+:avocado: tags=machine:virt
+"""
+self.vm.set_console()
+self.vm.launch()
+wait_for_console_pattern(self, 'Platform Name')
+wait_for_console_pattern(self, 'Boot HART MEDELEG')
+
+def test_riscv32_sifive_u(self):
+"""
+:avocado: tags=arch:riscv32
+:avocado: tags=machine:sifive_u
+"""
+self.vm.set_console()
+self.vm.launch()
+wait_for_console_pattern(self, 'Platform Name')
+wait_for_console_pattern(self, 'Boot HART MEDELEG')

Re: [PATCH v2] Acceptance Tests: add standard clean up at test tearDown()

2021-09-20 Thread Wainer dos Santos Moschetta




On 9/20/21 12:18 PM, Willian Rampazzo wrote:

From: Cleber Rosa 

The avocado.Test class, used as the basis of the avocado_qemu.Test
class, performs a clean of temporary directories up as part of its own
tearDown() implementation.

But the avocado_qemu.Test class is currently missing the same clean
up, as it implemented its own tearDown() method without resorting to
the upper class behavior.

This brings avocado_qemu.Test behavior in sync with the standard
avocado.Test behavior and prevents temporary directories from
cluttering the test results directory (unless instructed to do so with
Avocado's "--keep-tmp" option).

Reported-by: Peter Maydell 
Signed-off-by: Cleber Rosa 
[willianr: respin to new Python super format]
Signed-off-by: Willian Rampazzo 

---
v2
   - adjust super call to conform with PEP3135
---
  tests/acceptance/avocado_qemu/__init__.py | 1 +
  1 file changed, 1 insertion(+)



Reviewed-by: Wainer dos Santos Moschetta 




diff --git a/tests/acceptance/avocado_qemu/__init__.py 
b/tests/acceptance/avocado_qemu/__init__.py
index 2c4fef3e14..d9e1b32aa1 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -276,6 +276,7 @@ def tearDown(self):
  for vm in self._vms.values():
  vm.shutdown()
  self._sd = None
+super().tearDown()
  
  def fetch_asset(self, name,

  asset_hash=None, algorithm=None,

Re: [PATCH 1/2] Acceptance Tests: add standard clean up at test tearDown()

2021-08-16 Thread Wainer dos Santos Moschetta


Hi,

On 8/3/21 4:34 PM, Cleber Rosa wrote:

The avocado.Test class, used as the basis of the avocado_qemu.Test
class, performs a clean of temporary directories up as part of its own
tearDown() implementation.

But the avocado_qemu.Test class is currently missing the same clean
up, as it implemented its own tearDown() method without resorting to
the upper class behavior.

Good catch Peter!


This brings avocado_qemu.Test behavior in sync with the standard
avocado.Test behavior and prevents temporary directories from
cluttering the test results directory (unless instructed to do so with
Avocado's "--keep-tmp" option).

Reported-by: Peter Maydell 
Signed-off-by: Cleber Rosa 
---
  tests/acceptance/avocado_qemu/__init__.py | 1 +
  1 file changed, 1 insertion(+)

diff --git a/tests/acceptance/avocado_qemu/__init__.py 
b/tests/acceptance/avocado_qemu/__init__.py
index 2c4fef3e14..1e807e2e55 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -276,6 +276,7 @@ def tearDown(self):
  for vm in self._vms.values():
  vm.shutdown()
  self._sd = None
+super(Test, self).tearDown()


Could you use the new form of super [1] to make the pylint happier?

[1] https://www.python.org/dev/peps/pep-3135/

Thanks!

- Wainer

  
  def fetch_asset(self, name,

  asset_hash=None, algorithm=None,

Re: [PATCH 2/2] Acceptance Tests: updates to the MAINTAINERS file

2021-08-16 Thread Wainer dos Santos Moschetta


Hi,

On 8/3/21 4:34 PM, Cleber Rosa wrote:

The tests/acceptance directory is currently lacking a maintainer
entry, even though I've been performing that role (of course with help
from many others).  Thus, its status is, even more now, Maintained.

yay!


This also removes the currently broken Trello board link, which was
make unavailable unintentionally by a third party.
I can still access the link. Anyway, maybe we could migrate the cards 
(not everything, just those which still make sense) to the GitLab's 
Issues tracker?


Signed-off-by: Cleber Rosa 
---
  MAINTAINERS | 6 +++---
  1 file changed, 3 insertions(+), 3 deletions(-)


Acked-by: Wainer dos Santos Moschetta 



diff --git a/MAINTAINERS b/MAINTAINERS
index 37b1a8e442..d35b948e8d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3418,11 +3418,11 @@ F: tests/tcg/Makefile
  F: tests/tcg/Makefile.include
  
  Acceptance (Integration) Testing with the Avocado framework

-W: https://trello.com/b/6Qi1pxVn/avocado-qemu
-R: Cleber Rosa 
+M: Cleber Rosa 
  R: Philippe Mathieu-Daudé 
  R: Wainer dos Santos Moschetta 
-S: Odd Fixes
+S: Maintained
+F: tests/Makefile.include
  F: tests/acceptance/
  
  Documentation

Re: [PATCH] Acceptance tests: add myself as a reviewer for the acceptance tests.

2021-08-16 Thread Wainer dos Santos Moschetta


Hi Willian,

The "Acceptance (Integration) Testing with the Avocado framework" 
section hasn't any formal maintainer but I always considered Cleber the 
de-facto one. In this case I don't know whether my acknowledgement is 
taken in account or not; anyway, you got it from me:


Acked-by: Wainer dos Santos Moschetta 

Thanks!

On 8/9/21 4:29 PM, Willian Rampazzo wrote:

Signed-off-by: Willian Rampazzo 
---
  MAINTAINERS | 1 +
  1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 37b1a8e442..3f8ad63165 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3422,6 +3422,7 @@ W: https://trello.com/b/6Qi1pxVn/avocado-qemu
  R: Cleber Rosa 
  R: Philippe Mathieu-Daudé 
  R: Wainer dos Santos Moschetta 
+R: Willian Rampazzo 
  S: Odd Fixes
  F: tests/acceptance/

Re: [PATCH] tests/acceptance: Allow overwrite smp and memory

2021-08-16 Thread Wainer dos Santos Moschetta


Hi Ahmed,

Sorry for taking longer to reply you as I just returned from vacation.

On 8/2/21 7:22 PM, Ahmed Abouzied wrote:

Removes the hard-coded values in setUp(). Class inheriting from
avocado_qemu.LinuxTest can overwrite the default smp and memory instead.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/453

Thanks for working on that issue!

Signed-off-by: Ahmed Abouzied 
---
  tests/acceptance/avocado_qemu/__init__.py | 6 --
  1 file changed, 4 insertions(+), 2 deletions(-)


I would change the patch's subject as suggested by Philippe in another 
email. Apart from that, the change looks good to me. So:


Reviewed-by: Wainer dos Santos Moschetta 



diff --git a/tests/acceptance/avocado_qemu/__init__.py 
b/tests/acceptance/avocado_qemu/__init__.py
index 2c4fef3e14..2639b89c84 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -441,6 +441,8 @@ class LinuxTest(Test, LinuxSSHMixIn):
  distro = None
  username = 'root'
  password = 'password'
+smp = '2'
+memory = '1024'
  
  def _set_distro(self):

  distro_name = self.params.get(
@@ -471,8 +473,8 @@ def _set_distro(self):
  def setUp(self, ssh_pubkey=None, network_device_type='virtio-net'):
  super(LinuxTest, self).setUp()
  self._set_distro()
-self.vm.add_args('-smp', '2')
-self.vm.add_args('-m', '1024')
+self.vm.add_args('-smp', self.smp)
+self.vm.add_args('-m', self.memory)
  # The following network device allows for SSH connections
  self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22',
   '-device', '%s,netdev=vnet' % network_device_type)

Re: [PATCH] tests/acceptance: Test powernv machines

2021-08-16 Thread Wainer dos Santos Moschetta


Hi Cédric,

On 8/11/21 11:54 AM, Cédric Le Goater wrote:

On 8/11/21 11:07 AM, Thomas Huth wrote:

On 10/08/2021 11.09, Cédric Le Goater wrote:

On 8/10/21 10:36 AM, Joel Stanley wrote:

On Tue, 10 Aug 2021 at 08:34, Cédric Le Goater  wrote:

Fetch the OpenPOWER images to boot the powernv8 and powernv9 machines
with a simple PCI layout.

Cc: Cleber Rosa 
Cc: Wainer dos Santos Moschetta 
Signed-off-by: Cédric Le Goater 
---
   tests/acceptance/boot_linux_console.py | 42 ++
   1 file changed, 42 insertions(+)

diff --git a/tests/acceptance/boot_linux_console.py 
b/tests/acceptance/boot_linux_console.py
index 5248c8097df9..da93a475ca87 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -1176,6 +1176,48 @@ def test_ppc64_e500(self):
   tar_hash = '6951d86d644b302898da2fd701739c9406527fe1'
   self.do_test_advcal_2018('19', tar_hash, 'uImage')

+    def do_test_ppc64_powernv(self, proc):
+
+    images_url = 
('https://github.com/open-power/op-build/releases/download/v2.7/')
+
+    skiboot_url = images_url + 'skiboot.lid'
+    skiboot_hash = 
'a9ffcddbf238f86cda4b2cae2882d6bd13cff8489109758a4980efaf154f4a29'
+    skiboot_path = self.fetch_asset(skiboot_url, asset_hash=skiboot_hash,
+   algorithm='sha256')

What's the thought that led you to using this instead of the one that
gets packaged with qemu?

Good question.

I considered that the skiboot.lid shipped with QEMU was somewhat a default
to make things work. The official released versions are the ones used by
the outside world on real systems and were a better target for tests.

That said, using the default version might be enough. Maintainers, please
advise !

IMHO:

- We want to test the things that *we* ship

- We want to download as few things as possible, since downloads
   often slow down the tests and break CI runs if the network to
   the download site is not available

  ==> I'd prefer to use the internal skiboot.lid unless there is
  really a compelling reason to use the external one.

OK. I changed the test to use the internal skiboot.lid.


The code of patch v1 looks good to me but I will wait the v2 to put my 
reviewed-by stamp.


Thanks!

- Wainer



Thanks,

C.

Re: [PATCH v5 4/4] avocado_qemu: Add Intel iommu tests

2021-07-09 Thread Wainer dos Santos Moschetta




On 7/6/21 10:17 AM, Eric Auger wrote:

Add Intel IOMMU functional tests based on fedora 31.
Different configs are checked:
- strict
- caching mode, strict
- passthrough.

Signed-off-by: Eric Auger 
Signed-off-by: Willian Rampazzo 
Acked-by: Peter Xu 

---

v4 -> v5:
- import cleanup (William)
- IntelIOMMU becomes the class name (William)
- Added William's Sob
---
  tests/acceptance/intel_iommu.py | 115 
  1 file changed, 115 insertions(+)
  create mode 100644 tests/acceptance/intel_iommu.py


I successfully ran this test.

Tested-by: Wainer dos Santos Moschetta 



diff --git a/tests/acceptance/intel_iommu.py b/tests/acceptance/intel_iommu.py
new file mode 100644
index 00..ad10491d89
--- /dev/null
+++ b/tests/acceptance/intel_iommu.py
@@ -0,0 +1,115 @@
+# INTEL_IOMMU Functional tests
+#
+# Copyright (c) 2021 Red Hat, Inc.
+#
+# Author:
+#  Eric Auger 
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+import os
+
+from avocado import skipIf
+from avocado_qemu import LinuxTest
+
+@skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
+class IntelIOMMU(LinuxTest):
+"""
+:avocado: tags=arch:x86_64
+:avocado: tags=distro:fedora
+:avocado: tags=distro_version:31
+:avocado: tags=machine:q35
+:avocado: tags=accel:kvm
+:avocado: tags=intel_iommu
+"""
+
+IOMMU_ADDON = ',iommu_platform=on,disable-modern=off,disable-legacy=on'
+kernel_path = None
+initrd_path = None
+kernel_params = None
+
+def set_up_boot(self):
+path = self.download_boot()
+self.vm.add_args('-device', 'virtio-blk-pci,bus=pcie.0,scsi=off,' +
+ 'drive=drv0,id=virtio-disk0,bootindex=1,'
+ 'werror=stop,rerror=stop' + self.IOMMU_ADDON)
+self.vm.add_args('-device', 'virtio-gpu-pci' + self.IOMMU_ADDON)
+self.vm.add_args('-drive',
+ 'file=%s,if=none,cache=writethrough,id=drv0' % path)
+
+def setUp(self):
+super(IntelIOMMU, self).setUp(None, 'virtio-net-pci' + 
self.IOMMU_ADDON)
+
+def add_common_args(self):
+self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0')
+self.vm.add_args('-object',
+ 'rng-random,id=rng0,filename=/dev/urandom')
+
+def common_vm_setup(self, custom_kernel=None):
+self.require_accelerator("kvm")
+self.add_common_args()
+self.vm.add_args("-accel", "kvm")
+
+if custom_kernel is None:
+return
+
+kernel_url = self.distro.pxeboot_url + 'vmlinuz'
+initrd_url = self.distro.pxeboot_url + 'initrd.img'
+self.kernel_path = self.fetch_asset(kernel_url)
+self.initrd_path = self.fetch_asset(initrd_url)
+
+def run_and_check(self):
+if self.kernel_path:
+self.vm.add_args('-kernel', self.kernel_path,
+ '-append', self.kernel_params,
+ '-initrd', self.initrd_path)
+self.launch_and_wait()
+self.ssh_command('cat /proc/cmdline')
+self.ssh_command('dmesg | grep -e DMAR -e IOMMU')
+self.ssh_command('find /sys/kernel/iommu_groups/ -type l')
+self.ssh_command('dnf -y install numactl-devel')
+
+def test_intel_iommu(self):
+"""
+:avocado: tags=intel_iommu_intremap
+"""
+
+self.common_vm_setup(True)
+self.vm.add_args('-device', 'intel-iommu,intremap=on')
+self.vm.add_args('-machine', 'kernel_irqchip=split')
+
+self.kernel_params = self.distro.default_kernel_params + ' quiet 
intel_iommu=on'
+self.run_and_check()
+
+def test_intel_iommu_strict(self):
+"""
+:avocado: tags=intel_iommu_strict
+"""
+
+self.common_vm_setup(True)
+self.vm.add_args('-device', 'intel-iommu,intremap=on')
+self.vm.add_args('-machine', 'kernel_irqchip=split')
+self.kernel_params = self.distro.default_kernel_params + ' quiet 
intel_iommu=on,strict'
+self.run_and_check()
+
+def test_intel_iommu_strict_cm(self):
+"""
+:avocado: tags=intel_iommu_strict_cm
+"""
+
+self.common_vm_setup(True)
+self.vm.add_args('-device', 'intel-iommu,intremap=on,caching-mode=on')
+self.vm.add_args('-machine', 'kernel_irqchip=split')
+self.kernel_params = self.distro.default_kernel_params + ' quiet 
intel_iommu=on,strict'
+self.run_and_check()
+
+def test_intel_iommu_pt(self):
+"""
+:avocado: tags=intel_iommu_pt
+"""
+
+self.common_vm_setup(True)
+self.vm.add_args('-device', 'intel-iommu,intremap=on')
+self.vm.add_args('-machine', 'kernel_irqchip=split')
+self.kernel_params = self.distro.default_kernel_params + ' quiet 
intel_iommu=on iommu=pt'
+self.run_and_check()

Re: [PATCH v5 1/4] avocado_qemu: Fix KNOWN_DISTROS map into the LinuxDistro class

2021-07-09 Thread Wainer dos Santos Moschetta




On 7/8/21 2:34 PM, Cleber Rosa wrote:


On 7/8/21 4:56 AM, Eric Auger wrote:


I am not sufficiently expert on the test infra and python to be really
efficient fixing that. Can anyone help quickly to target the soft
freeze? Otherwise, today I will drop that patch and restore the code I
had in v4, just based on Cleber series. I think the refactoring can
happen later...



Hi Eric,


The following diff works for me:


diff --git a/tests/acceptance/avocado_qemu/__init__.py 
b/tests/acceptance/avocado_qemu/__init__.py

index af93cd63ea..b3bed00062 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -310,6 +310,8 @@ class LinuxDistro:
 '31': {
 'x86_64':
 {'checksum': 
'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'},

+    'aarch64':
+    {'checksum': 
'1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49'},

 'ppc64':
 {'checksum': 
'7c3528b85a3df4b2306e892199a9e1e43f991c506f2cc390dc4efa2026ad2f58'},

 's390x':
@@ -323,10 +325,11 @@ def __init__(self, name, version, arch):
 self.version = version
 self.arch = arch
 try:
-    self._info = 
self.KNOWN_DISTROS.get(name).get(version).get(arch)

+    info = self.KNOWN_DISTROS.get(name).get(version).get(arch)
 except AttributeError:
 # Unknown distro
-    self._info = {}
+    info = None
+    self._info = info or {}

 @property
 def checksum(self):




Thanks for that fix, Cleber.

Acked-by: Wainer dos Santos Moschetta 



I've tested it with both existing and the newly introduced tests.


Cheers,

- Cleber.

Re: [PATCH v2] tests/acceptance: allow control over tags during check-acceptance

2021-07-09 Thread Wainer dos Santos Moschetta


Hi,

On 7/7/21 12:24 PM, Willian Rampazzo wrote:

Although it is possible to run a specific test using the avocado
command-line, a user may want to use a specific tag while running the
``make check-acceptance`` during the development or debugging.

I always wanted that feature, thanks for implementing it!


This allows using the AVOCADO_TAGS environment variable where the user
takes total control of which tests should run based on the tags defined.

This also makes the check-acceptance command flexible to restrict tests
based on tags while running on CI.

e.g.:

AVOCADO_TAGS="foo bar baz" make check-acceptance


If I run `AVOCADO_TAGS="machine:malta arch:mips" make check-acceptance` 
it is going to get the tests with 'machine:malta' *or* 'arch:mips' tags, 
whereas with AVOCADO_TAGS="machine:malta,arch:mips" it gets those with 
'machine:malta' *and* 'arch:mips', i.e., the semantic is different. I 
think it deserves a mention in the docs/devel/testing.rst, maybe linking 
to Avocado's doc [1] as well.


[1] 
https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/tags.html




Signed-off-by: Willian Rampazzo 
---
  docs/devel/testing.rst |  8 
  tests/Makefile.include | 12 +---
  2 files changed, 17 insertions(+), 3 deletions(-)


Tested-by: Wainer dos Santos Moschetta 

Reviewed-by: Wainer dos Santos Moschetta 



diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 4e42392810..f6e4fc08a7 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -760,6 +760,14 @@ in the current directory, tagged as "quick", run:
  
avocado run -t quick .
  
+It is also possible to run tests based on tags using the

+``make check-acceptance`` command and the ``AVOCADO_TAGS`` environment
+variable:
+
+.. code::
+
+   AVOCADO_TAGS=quick make check-acceptance
+
  The ``avocado_qemu.Test`` base test class
  -
  
diff --git a/tests/Makefile.include b/tests/Makefile.include

index 8f220e15d1..d464f06e4a 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -92,7 +92,12 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
  # Any number of command separated loggers are accepted.  For more
  # information please refer to "avocado --help".
  AVOCADO_SHOW=app
-AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGETS)))
+ifndef AVOCADO_TAGS
+   AVOCADO_CMDLINE_TAGS=$(patsubst %-softmmu,-t arch:%, \
+$(filter %-softmmu,$(TARGETS)))
+else
+   AVOCADO_CMDLINE_TAGS=$(addprefix -t , $(AVOCADO_TAGS))
+endif
  
  $(TESTS_VENV_DIR): $(TESTS_VENV_REQ)

$(call quiet-command, \
@@ -128,8 +133,9 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) 
get-vm-images
$(call quiet-command, \
  $(TESTS_VENV_DIR)/bin/python -m avocado \
  --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) 
\
---filter-by-tags-include-empty --filter-by-tags-include-empty-key \
-$(AVOCADO_TAGS) \
+$(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \
+   --filter-by-tags-include-empty-key) \
+$(AVOCADO_CMDLINE_TAGS) \
  $(if $(GITLAB_CI),,--failfast) tests/acceptance, \
  "AVOCADO", "tests/acceptance")

Re: [PATCH v4 2/4] avocado_qemu: Add SMMUv3 tests

2021-07-06 Thread Wainer dos Santos Moschetta




On 7/5/21 5:00 AM, Eric Auger wrote:

Hi Wainer,

On 7/1/21 8:13 PM, Wainer dos Santos Moschetta wrote:

Hi,

On 6/29/21 11:36 AM, Eric Auger wrote:

Add new tests checking the good behavior of the SMMUv3 protecting
2 virtio pci devices (block and net). We check the guest boots and
we are able to install a package. Different guest configs are tested:
standard, passthrough an strict=0. This is tested with both fedora 31
and
33. The former uses a 5.3 kernel without range invalidation whereas the
latter uses a 5.8 kernel that features range invalidation.

Signed-off-by: Eric Auger 

---

v3 -> v4:
- add tags for machine, distro in the class
- removed smp and memory overrides
- set default param value of common_vm_setup to False

v1 -> v2:
- removed ssh import
- combined add_command_args() and common_vm_setup()
- moved tags in class' docstring and added tags=arch:aarch64
- use self.get_default_kernel_params()
- added RIL tests with fed33 + introduce new tags
---
   tests/acceptance/smmu.py | 132 +++
   1 file changed, 132 insertions(+)
   create mode 100644 tests/acceptance/smmu.py

Reviewed-by: Wainer dos Santos Moschetta 

Tested-by: Wainer dos Santos Moschetta 

I tested it in a Fedora 32 aarch64 host. The execution output:

# ./tests/venv/bin/avocado run tests/acceptance/smmu.py
JOB ID : 1625038f5a2ae17c8ba6c503d3df8661ff528942
JOB LOG    :
/root/avocado/job-results/job-2021-07-01T13.38-1625038/job.log
  (1/6) tests/acceptance/smmu.py:SMMU.test_smmu_noril: PASS (175.54 s)
  (2/6) tests/acceptance/smmu.py:SMMU.test_smmu_noril_passthrough:
WARN: Test passed but there were warnings during execution. Check the
log for details. (168.39 s)
  (3/6) tests/acceptance/smmu.py:SMMU.test_smmu_noril_nostrict: WARN:
Test passed but there were warnings during execution. Check the log
for details. (161.58 s)
  (4/6) tests/acceptance/smmu.py:SMMU.test_smmu_ril: PASS (150.85 s)
  (5/6) tests/acceptance/smmu.py:SMMU.test_smmu_ril_passthrough: WARN:
Test passed but there were warnings during execution. Check the log
for details. (177.56 s)
  (6/6) tests/acceptance/smmu.py:SMMU.test_smmu_ril_nostrict: WARN:
Test passed but there were warnings during execution. Check the log
for details. (190.86 s)
RESULTS    : PASS 2 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 4 | INTERRUPT 0
| CANCEL 0
JOB TIME   : 1026.50 s

One thing that caught my attention was the amount of time spent on
each test. It spend more than 2 minutes on the package installation
(`self.ssh_command('dnf -y install numactl-devel')`) in the guest.

Without that operation, it runs way faster:

# ./tests/venv/bin/avocado run tests/acceptance/smmu.py
JOB ID : 24f22f99169ece37df64d72d2eb373921f378aac
JOB LOG    :
/root/avocado/job-results/job-2021-07-01T13.28-24f22f9/job.log
  (1/6) tests/acceptance/smmu.py:SMMU.test_smmu_noril: PASS (39.61 s)
  (2/6) tests/acceptance/smmu.py:SMMU.test_smmu_noril_passthrough:
WARN: Test passed but there were warnings during execution. Check the
log for details. (48.32 s)
  (3/6) tests/acceptance/smmu.py:SMMU.test_smmu_noril_nostrict: WARN:
Test passed but there were warnings during execution. Check the log
for details. (48.10 s)
  (4/6) tests/acceptance/smmu.py:SMMU.test_smmu_ril: PASS (39.22 s)
  (5/6) tests/acceptance/smmu.py:SMMU.test_smmu_ril_passthrough: WARN:
Test passed but there were warnings during execution. Check the log
for details. (52.92 s)
  (6/6) tests/acceptance/smmu.py:SMMU.test_smmu_ril_nostrict: WARN:
Test passed but there were warnings during execution. Check the log
for details. (50.96 s)
RESULTS    : PASS 2 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 4 | INTERRUPT 0
| CANCEL 0
JOB TIME   : 280.62 s

Install a package seems a good exerciser for disk I/O and networking,
but maybe you can use another method for the sake of speed up the tests?

As discussed earlier with Cleber, I am aware the test duration is long
but it was useful finding bugs for SMMU with range invalidation. such a
bug could not be hit with a single boot + ping for instance.

Maybe we should have a mechanism that allows to put some tests out of
the automatic CI?


Sorry Eric, I missed that discussion. I will review your v5 series very 
soon.


Thanks!

- Wainer



Thanks

Eric

- Wainer


diff --git a/tests/acceptance/smmu.py b/tests/acceptance/smmu.py
new file mode 100644
index 00..c1d4b88e5f
--- /dev/null
+++ b/tests/acceptance/smmu.py
@@ -0,0 +1,132 @@
+# SMMUv3 Functional tests
+#
+# Copyright (c) 2021 Red Hat, Inc.
+#
+# Author:
+#  Eric Auger 
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+
+import os
+
+from avocado_qemu import LinuxTest, BUILD_DIR
+
+class SMMU(LinuxTest):
+    """
+    :avocado: tags=accel:kvm
+    :avocado: tags=cpu:host
+    :avocado: tags=arch:aarch64
+    :avocado: tags=machine:virt
+    :avocado: tags=distro:fedora
+    :avocado: tags=smmu
+    """
+

Re: [PATCH v4 2/4] avocado_qemu: Add SMMUv3 tests

2021-07-01 Thread Wainer dos Santos Moschetta


Hi,

On 6/29/21 11:36 AM, Eric Auger wrote:

Add new tests checking the good behavior of the SMMUv3 protecting
2 virtio pci devices (block and net). We check the guest boots and
we are able to install a package. Different guest configs are tested:
standard, passthrough an strict=0. This is tested with both fedora 31 and
33. The former uses a 5.3 kernel without range invalidation whereas the
latter uses a 5.8 kernel that features range invalidation.

Signed-off-by: Eric Auger 

---

v3 -> v4:
- add tags for machine, distro in the class
- removed smp and memory overrides
- set default param value of common_vm_setup to False

v1 -> v2:
- removed ssh import
- combined add_command_args() and common_vm_setup()
- moved tags in class' docstring and added tags=arch:aarch64
- use self.get_default_kernel_params()
- added RIL tests with fed33 + introduce new tags
---
  tests/acceptance/smmu.py | 132 +++
  1 file changed, 132 insertions(+)
  create mode 100644 tests/acceptance/smmu.py


Reviewed-by: Wainer dos Santos Moschetta 

Tested-by: Wainer dos Santos Moschetta 

I tested it in a Fedora 32 aarch64 host. The execution output:

# ./tests/venv/bin/avocado run tests/acceptance/smmu.py
JOB ID : 1625038f5a2ae17c8ba6c503d3df8661ff528942
JOB LOG    : /root/avocado/job-results/job-2021-07-01T13.38-1625038/job.log
 (1/6) tests/acceptance/smmu.py:SMMU.test_smmu_noril: PASS (175.54 s)
 (2/6) tests/acceptance/smmu.py:SMMU.test_smmu_noril_passthrough: WARN: 
Test passed but there were warnings during execution. Check the log for 
details. (168.39 s)
 (3/6) tests/acceptance/smmu.py:SMMU.test_smmu_noril_nostrict: WARN: 
Test passed but there were warnings during execution. Check the log for 
details. (161.58 s)

 (4/6) tests/acceptance/smmu.py:SMMU.test_smmu_ril: PASS (150.85 s)
 (5/6) tests/acceptance/smmu.py:SMMU.test_smmu_ril_passthrough: WARN: 
Test passed but there were warnings during execution. Check the log for 
details. (177.56 s)
 (6/6) tests/acceptance/smmu.py:SMMU.test_smmu_ril_nostrict: WARN: Test 
passed but there were warnings during execution. Check the log for 
details. (190.86 s)
RESULTS    : PASS 2 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 4 | INTERRUPT 0 | 
CANCEL 0

JOB TIME   : 1026.50 s

One thing that caught my attention was the amount of time spent on each 
test. It spend more than 2 minutes on the package installation 
(`self.ssh_command('dnf -y install numactl-devel')`) in the guest.


Without that operation, it runs way faster:

# ./tests/venv/bin/avocado run tests/acceptance/smmu.py
JOB ID : 24f22f99169ece37df64d72d2eb373921f378aac
JOB LOG    : /root/avocado/job-results/job-2021-07-01T13.28-24f22f9/job.log
 (1/6) tests/acceptance/smmu.py:SMMU.test_smmu_noril: PASS (39.61 s)
 (2/6) tests/acceptance/smmu.py:SMMU.test_smmu_noril_passthrough: WARN: 
Test passed but there were warnings during execution. Check the log for 
details. (48.32 s)
 (3/6) tests/acceptance/smmu.py:SMMU.test_smmu_noril_nostrict: WARN: 
Test passed but there were warnings during execution. Check the log for 
details. (48.10 s)

 (4/6) tests/acceptance/smmu.py:SMMU.test_smmu_ril: PASS (39.22 s)
 (5/6) tests/acceptance/smmu.py:SMMU.test_smmu_ril_passthrough: WARN: 
Test passed but there were warnings during execution. Check the log for 
details. (52.92 s)
 (6/6) tests/acceptance/smmu.py:SMMU.test_smmu_ril_nostrict: WARN: Test 
passed but there were warnings during execution. Check the log for 
details. (50.96 s)
RESULTS    : PASS 2 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 4 | INTERRUPT 0 | 
CANCEL 0

JOB TIME   : 280.62 s

Install a package seems a good exerciser for disk I/O and networking, 
but maybe you can use another method for the sake of speed up the tests?


- Wainer



diff --git a/tests/acceptance/smmu.py b/tests/acceptance/smmu.py
new file mode 100644
index 00..c1d4b88e5f
--- /dev/null
+++ b/tests/acceptance/smmu.py
@@ -0,0 +1,132 @@
+# SMMUv3 Functional tests
+#
+# Copyright (c) 2021 Red Hat, Inc.
+#
+# Author:
+#  Eric Auger 
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+
+import os
+
+from avocado_qemu import LinuxTest, BUILD_DIR
+
+class SMMU(LinuxTest):
+"""
+:avocado: tags=accel:kvm
+:avocado: tags=cpu:host
+:avocado: tags=arch:aarch64
+:avocado: tags=machine:virt
+:avocado: tags=distro:fedora
+:avocado: tags=smmu
+"""
+
+IOMMU_ADDON = ',iommu_platform=on,disable-modern=off,disable-legacy=on'
+kernel_path = None
+initrd_path = None
+kernel_params = None
+
+def set_up_boot(self):
+path = self.download_boot()
+self.vm.add_args('-device', 'virtio-blk-pci,bus=pcie.0,scsi=off,' +
+ 'drive=drv0,id=virtio-disk0,bootindex=1,'
+ 'werror=stop,rerror=stop' + self.IOMMU_ADDON)
+self.vm.add_args('-drive',
+ 'file=%s,if=non

Re: [PATCH] tests/vm: update NetBSD to 9.2

2021-07-01 Thread Wainer dos Santos Moschetta




On 6/2/21 2:21 PM, Brad Smith wrote:

tests/vm: update NetBSD to 9.2

Signed-off-by: Brad Smith 
---
  tests/vm/netbsd | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)

Reviewed-by: Wainer dos Santos Moschetta 


diff --git a/tests/vm/netbsd b/tests/vm/netbsd
index b9efc269d2..4cc58df130 100755
--- a/tests/vm/netbsd
+++ b/tests/vm/netbsd
@@ -22,8 +22,8 @@ class NetBSDVM(basevm.BaseVM):
  name = "netbsd"
  arch = "x86_64"
  
-link = "https://cdn.netbsd.org/pub/NetBSD/NetBSD-9.1/images/NetBSD-9.1-amd64.iso;

-csum = 
"65bddc95945991c3b2021f9c8ded7f34c25f0a7611b7aa15a15fe23399e902307e926ae97fcd01dc1662ac67b5f6e4be643c6a2b581692ddcb616d30125066f9"
+link = 
"https://cdn.netbsd.org/pub/NetBSD/NetBSD-9.2/images/NetBSD-9.2-amd64.iso;
+csum = 
"5ee0ea101f73386b9b424f5d1041e371db3c42fdd6f4e4518dc79c4a08f31d43091ebe93425c9f0dcaaed2b51131836fe6774f33f89030b58d64709b35fda72f"
  size = "20G"
  pkgs = [
  # tools

Re: [PATCH v4 0/4] avocado-qemu: New SMMUv3 and intel IOMMU tests

2021-06-30 Thread Wainer dos Santos Moschetta


Hi,

On 6/29/21 5:17 PM, Eric Auger wrote:

Hi Cleber, all,

On 6/29/21 4:36 PM, Eric Auger wrote:

This series adds ARM SMMU and Intel IOMMU functional
tests using Fedora cloud-init images.

ARM SMMU tests feature guests with and without RIL
(range invalidation support) using respectively fedora 33
and 31.  For each, we test the protection of virtio-net-pci
and virtio-block-pci devices. Also strict=no and passthrough
modes are tested. So there is a total of 6 tests.

The series applies on top of Cleber's series:
- [PATCH 0/3] Acceptance Tests: support choosing specific

Note:
- SMMU tests 2, 3, 5, 6 (resp. test_smmu_noril_passthrough and
test_smmu_noril_nostrict) pass but the log reports:
"WARN: Test passed but there were warnings during execution."
This seems due to the lack of hash when fetching the kernel and
initrd through fetch_asset():
WARNI| No hash provided. Cannot check the asset file integrity.

I wanted to emphasize that point and wondered how we could fix that
issue. Looks a pity the tests get tagged as WARN due to a lack of sha1.
Any advice?


As Willian mentioned somewhere, to supress the WARN you can pass the 
kernel and initrd checksums (sha1) to the fetch_asset() method.


Below is an draft implementation. It would need to fill out the 
remaining checksums and adjust the `smmu.py` tests.


- Wainer



diff --git a/tests/acceptance/avocado_qemu/__init__.py 
b/tests/acceptance/avocado_qemu/__init__.py

index 00eb0bfcc8..83637e2654 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -312,6 +312,8 @@ class LinuxDistro:
 {'checksum': 
'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0',
 'pxeboot_url': 
"https://archives.fedoraproject.org/pub/archive/fedora/;

"linux/releases/31/Everything/x86_64/os/images/pxeboot/",
+    'pxeboot_initrd_chksum': 
'dd0340a1b39bd28f88532babd4581c67649ec5b1',
+    'pxeboot_vmlinuz_chksum': 
'5b6f6876e1b5bda314f93893271da0d5777b1f3c',
 'kernel_params': 
"root=UUID=b1438b9b-2cab-4065-a99a-08a96687f73c ro "

   "no_timer_check net.ifnames=0 "
   "console=tty1 console=ttyS0,115200n8"},
@@ -371,6 +373,16 @@ def pxeboot_url(self):
 """Gets the repository url where pxeboot files can be found"""
 return self._info.get('pxeboot_url', None)

+    @property
+    def pxeboot_initrd_chksum(self):
+    """Gets the pxeboot initrd file checksum"""
+    return self._info.get('pxeboot_initrd_chksum', None)
+
+    @property
+    def pxeboot_vmlinuz_chksum(self):
+    """Gets the pxeboot vmlinuz file checksum"""
+    return self._info.get('pxeboot_vmlinuz_chksum', None)
+
 @property
 def checksum(self):
 """Gets the cloud-image file checksum"""
diff --git a/tests/acceptance/intel_iommu.py 
b/tests/acceptance/intel_iommu.py

index bf8dea6e4f..a2f38ee2e9 100644
--- a/tests/acceptance/intel_iommu.py
+++ b/tests/acceptance/intel_iommu.py
@@ -55,8 +55,10 @@ def common_vm_setup(self, custom_kernel=None):

 kernel_url = self.distro.pxeboot_url + 'vmlinuz'
 initrd_url = self.distro.pxeboot_url + 'initrd.img'
-    self.kernel_path = self.fetch_asset(kernel_url)
-    self.initrd_path = self.fetch_asset(initrd_url)
+    self.kernel_path = self.fetch_asset(kernel_url,
+ asset_hash=self.distro.pxeboot_vmlinuz_chksum)
+    self.initrd_path = self.fetch_asset(initrd_url,
+ asset_hash=self.distro.pxeboot_initrd_chksum)

 def run_and_check(self):
 if self.kernel_path:



Best Regards

Eric

History:
v3 -> v4:
- I added Wainer's refactoring of KNOWN_DISTROS
into a class (last patch) and took into account his comments.

v2 -> v3:
- Added Intel IOMMU tests were added. Different
operating modes are tested such as strict, caching mode, pt.

Best Regards

Eric

The series and its dependencies can be found at:
https://github.com/eauger/qemu/tree/avocado-qemu-v4

Eric Auger (3):
   Acceptance Tests: Add default kernel params and pxeboot url to the
 KNOWN_DISTROS collection
   avocado_qemu: Add SMMUv3 tests
   avocado_qemu: Add Intel iommu tests

Wainer dos Santos Moschetta (1):
   avocado_qemu: Fix KNOWN_DISTROS map into the LinuxDistro class

  tests/acceptance/avocado_qemu/__init__.py | 118 +--
  tests/acceptance/intel_iommu.py   | 115 +++
  tests/acceptance/smmu.py  | 132 ++
  3 files changed, 332 insertions(+), 33 deletions(-)
  create mode 100644 tests/acceptance/intel_iommu.py
  create mode 100644 tests/acceptance/smmu.py

Re: [PATCH v7 4/4] Jobs based on custom runners: add job definitions for QEMU's machines

2021-06-30 Thread Wainer dos Santos Moschetta




On 6/29/21 10:26 PM, Cleber Rosa wrote:

The QEMU project has two machines (aarch64 and s390x) that can be used
for jobs that do build and run tests.  This introduces those jobs,
which are a mapping of custom scripts used for the same purpose.

Signed-off-by: Cleber Rosa 
Reviewed-by: Willian Rampazzo 
---
  .gitlab-ci.d/custom-runners.yml | 208 
  1 file changed, 208 insertions(+)

Reviewed-by: Wainer dos Santos Moschetta 


diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
index a07b27384c..061d3cdfed 100644
--- a/.gitlab-ci.d/custom-runners.yml
+++ b/.gitlab-ci.d/custom-runners.yml
@@ -12,3 +12,211 @@
  # guarantees a fresh repository on each job run.
  variables:
GIT_STRATEGY: clone
+
+# All ubuntu-18.04 jobs should run successfully in an environment
+# setup by the scripts/ci/setup/build-environment.yml task
+# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
+ubuntu-18.04-s390x-all-linux-static:
+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ # --disable-libssh is needed because of 
https://bugs.launchpad.net/qemu/+bug/1838763
+ # --disable-glusterfs is needed because there's no static version of those 
libs in distro supplied packages
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --static --disable-system --disable-glusterfs 
--disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check-tcg V=1
+
+ubuntu-18.04-s390x-all:
+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-18.04-s390x-alldbg:
+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --disable-libssh
+ - make clean
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-18.04-s390x-clang:
+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+   when: manual
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-18.04-s390x-tci:
+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --enable-tcg-interpreter
+ - make --output-sync -j`nproc`
+
+ubuntu-18.04-s390x-notcg:
+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+   when: manual
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --disable-tcg
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+# All ubuntu-20.04 jobs should run successfully in an environment
+# setup by the scripts/ci/setup/qemu/build-environment.yml task
+# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
+ubuntu-20.04-aarch64-all-linux-static:
+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ # --disable-libssh is needed because of 
https://bugs.launchpad.net/qemu/+bug/1838763
+ # --disable-glusterfs is needed because there's no static version of those 
libs in distro supplied packages
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --static --disable-system --disable-glusterfs 
--disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check-tcg V=1
+
+ubuntu-20.04-aarch64-all:
+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-20.04-aarch64-alldbg:
+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --disable-libssh
+ - make clean
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-20.04-aarch64-clang:
+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+   when: manual
+ script:
+ - mkdir build
+ - cd bu

Re: [PATCH v7 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook

2021-06-30 Thread Wainer dos Santos Moschetta




On 6/29/21 10:26 PM, Cleber Rosa wrote:

To have the jobs dispatched to custom runners, gitlab-runner must
be installed, active as a service and properly configured.  The
variables file and playbook introduced here should help with those
steps.

The playbook introduced here covers the Linux distributions and
has been primarily tested on OS/machines that the QEMU project
has available to act as runners, namely:

  * Ubuntu 20.04 on aarch64
  * Ubuntu 18.04 on s390x

But, it should work on all other Linux distributions.  Earlier
versions were tested on FreeBSD too, so chances of success are
high.

Signed-off-by: Cleber Rosa 
Reviewed-by: Willian Rampazzo 
Tested-by: Willian Rampazzo 
---
  docs/devel/ci.rst  | 55 +++
  scripts/ci/setup/.gitignore|  2 +-
  scripts/ci/setup/gitlab-runner.yml | 71 ++
  scripts/ci/setup/vars.yml.template | 12 +
  4 files changed, 139 insertions(+), 1 deletion(-)
  create mode 100644 scripts/ci/setup/gitlab-runner.yml
  create mode 100644 scripts/ci/setup/vars.yml.template

Reviewed-by: Wainer dos Santos Moschetta 


diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
index bfedbb1025..b3bf3ef615 100644
--- a/docs/devel/ci.rst
+++ b/docs/devel/ci.rst
@@ -70,3 +70,58 @@ privileges, such as those from the ``root`` account or those 
obtained
  by ``sudo``.  If necessary, please refer to ``ansible-playbook``
  options such as ``--become``, ``--become-method``, ``--become-user``
  and ``--ask-become-pass``.
+
+gitlab-runner setup and registration
+
+
+The gitlab-runner agent needs to be installed on each machine that
+will run jobs.  The association between a machine and a GitLab project
+happens with a registration token.  To find the registration token for
+your repository/project, navigate on GitLab's web UI to:
+
+ * Settings (the gears-like icon at the bottom of the left hand side
+   vertical toolbar), then
+ * CI/CD, then
+ * Runners, and click on the "Expand" button, then
+ * Under "Set up a specific Runner manually", look for the value under
+   "And this registration token:"
+
+Copy the ``scripts/ci/setup/vars.yml.template`` file to
+``scripts/ci/setup/vars.yml``.  Then, set the
+``gitlab_runner_registration_token`` variable to the value obtained
+earlier.
+
+To run the playbook, execute::
+
+  cd scripts/ci/setup
+  ansible-playbook -i inventory gitlab-runner.yml
+
+Following the registration, it's necessary to configure the runner tags,
+and optionally other configurations on the GitLab UI.  Navigate to:
+
+ * Settings (the gears like icon), then
+ * CI/CD, then
+ * Runners, and click on the "Expand" button, then
+ * "Runners activated for this project", then
+ * Click on the "Edit" icon (next to the "Lock" Icon)
+
+Tags are very important as they are used to route specific jobs to
+specific types of runners, so it's a good idea to double check that
+the automatically created tags are consistent with the OS and
+architecture.  For instance, an Ubuntu 20.04 aarch64 system should
+have tags set as::
+
+  ubuntu_20.04,aarch64
+
+Because the job definition at ``.gitlab-ci.d/custom-runners.yml``
+would contain::
+
+  ubuntu-20.04-aarch64-all:
+   tags:
+   - ubuntu_20.04
+   - aarch64
+
+It's also recommended to:
+
+ * increase the "Maximum job timeout" to something like ``2h``
+ * give it a better Description
diff --git a/scripts/ci/setup/.gitignore b/scripts/ci/setup/.gitignore
index ee088604d1..f4a6183f1f 100644
--- a/scripts/ci/setup/.gitignore
+++ b/scripts/ci/setup/.gitignore
@@ -1,2 +1,2 @@
  inventory
-
+vars.yml
diff --git a/scripts/ci/setup/gitlab-runner.yml 
b/scripts/ci/setup/gitlab-runner.yml
new file mode 100644
index 00..1127db516f
--- /dev/null
+++ b/scripts/ci/setup/gitlab-runner.yml
@@ -0,0 +1,71 @@
+# Copyright (c) 2021 Red Hat, Inc.
+#
+# Author:
+#  Cleber Rosa 
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+#
+# This is an ansible playbook file.  Run it to set up systems with the
+# gitlab-runner agent.
+---
+- name: Installation of gitlab-runner
+  hosts: all
+  vars_files:
+- vars.yml
+  tasks:
+- debug:
+msg: 'Checking for a valid GitLab registration token'
+  failed_when: "gitlab_runner_registration_token == 
'PLEASE_PROVIDE_A_VALID_TOKEN'"
+
+- name: Create a group for the gitlab-runner service
+  group:
+name: gitlab-runner
+
+- name: Create a user for the gitlab-runner service
+  user:
+user: gitlab-runner
+group: gitlab-runner
+comment: GitLab Runner
+home: /home/gitlab-runner
+shell: /bin/bash
+
+- name: Remove the .bash_logout file when on Ubuntu systems
+  file:
+path: /home/gitlab-runner/.bash_logout
+state: absent
+  when: "ansib

Re: [PATCH v7 2/4] Jobs based on custom runners: build environment docs and playbook

2021-06-30 Thread Wainer dos Santos Moschetta




On 6/29/21 10:26 PM, Cleber Rosa wrote:

To run basic jobs on custom runners, the environment needs to be
properly set up.  The most common requirement is having the right
packages installed.

The playbook introduced here covers the QEMU's project s390x and
aarch64 machines.  At the time this is being proposed, those machines
have already had this playbook applied to them.

Signed-off-by: Cleber Rosa 
---
  docs/devel/ci.rst  |  40 +
  scripts/ci/setup/.gitignore|   2 +
  scripts/ci/setup/build-environment.yml | 116 +
  scripts/ci/setup/inventory.template|   1 +
  4 files changed, 159 insertions(+)
  create mode 100644 scripts/ci/setup/.gitignore
  create mode 100644 scripts/ci/setup/build-environment.yml
  create mode 100644 scripts/ci/setup/inventory.template


Reviewed-by: Wainer dos Santos Moschetta 



diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
index 064ffa9988..bfedbb1025 100644
--- a/docs/devel/ci.rst
+++ b/docs/devel/ci.rst
@@ -30,3 +30,43 @@ The GitLab CI jobs definition for the custom runners are 
located under::
  Custom runners entail custom machines.  To see a list of the machines
  currently deployed in the QEMU GitLab CI and their maintainers, please
  refer to the QEMU `wiki <https://wiki.qemu.org/AdminContacts>`__.
+
+Machine Setup Howto
+---
+
+For all Linux based systems, the setup can be mostly automated by the
+execution of two Ansible playbooks.  Create an ``inventory`` file
+under ``scripts/ci/setup``, such as this::
+
+  fully.qualified.domain
+  other.machine.hostname
+
+You may need to set some variables in the inventory file itself.  One
+very common need is to tell Ansible to use a Python 3 interpreter on
+those hosts.  This would look like::
+
+  fully.qualified.domain ansible_python_interpreter=/usr/bin/python3
+  other.machine.hostname ansible_python_interpreter=/usr/bin/python3
+
+Build environment
+~
+
+The ``scripts/ci/setup/build-environment.yml`` Ansible playbook will
+set up machines with the environment needed to perform builds and run
+QEMU tests.  This playbook consists on the installation of various
+required packages (and a general package update while at it).  It
+currently covers a number of different Linux distributions, but it can
+be expanded to cover other systems.
+
+The minimum required version of Ansible successfully tested in this
+playbook is 2.8.0 (a version check is embedded within the playbook
+itself).  To run the playbook, execute::
+
+  cd scripts/ci/setup
+  ansible-playbook -i inventory build-environment.yml
+
+Please note that most of the tasks in the playbook require superuser
+privileges, such as those from the ``root`` account or those obtained
+by ``sudo``.  If necessary, please refer to ``ansible-playbook``
+options such as ``--become``, ``--become-method``, ``--become-user``
+and ``--ask-become-pass``.
diff --git a/scripts/ci/setup/.gitignore b/scripts/ci/setup/.gitignore
new file mode 100644
index 00..ee088604d1
--- /dev/null
+++ b/scripts/ci/setup/.gitignore
@@ -0,0 +1,2 @@
+inventory
+
diff --git a/scripts/ci/setup/build-environment.yml 
b/scripts/ci/setup/build-environment.yml
new file mode 100644
index 00..581c1c75d1
--- /dev/null
+++ b/scripts/ci/setup/build-environment.yml
@@ -0,0 +1,116 @@
+# Copyright (c) 2021 Red Hat, Inc.
+#
+# Author:
+#  Cleber Rosa 
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+#
+# This is an ansible playbook file.  Run it to set up systems with the
+# environment needed to build QEMU.
+---
+- name: Installation of basic packages to build QEMU
+  hosts: all
+  tasks:
+- name: Check for suitable ansible version
+  delegate_to: localhost
+  assert:
+that:
+  - '((ansible_version.major == 2) and (ansible_version.minor >= 8)) or 
(ansible_version.major >= 3)'
+msg: "Unsuitable ansible version, please use version 2.8.0 or later"
+
+- name: Update apt cache / upgrade packages via apt
+  apt:
+update_cache: yes
+upgrade: yes
+  when:
+- ansible_facts['distribution'] == 'Ubuntu'
+
+- name: Install basic packages to build QEMU on Ubuntu 18.04/20.04
+  package:
+name:
+# Originally from tests/docker/dockerfiles/ubuntu1804.docker
+  - ccache
+  - gcc
+  - gettext
+  - git
+  - glusterfs-common
+  - libaio-dev
+  - libattr1-dev
+  - libbrlapi-dev
+  - libbz2-dev
+  - libcacard-dev
+  - libcap-ng-dev
+  - libcurl4-gnutls-dev
+  - libdrm-dev
+  - libepoxy-dev
+  - libfdt-dev
+  - libgbm-dev
+  - libgtk-3-dev
+  - libibverbs-dev
+  - libiscsi-dev
+  - libjemalloc-dev
+  - libjpeg-turbo8-dev
+  - liblzo2-dev
+

Re: [PATCH 3/3] cirrus: delete FreeBSD and macOS jobs

2021-06-30 Thread Wainer dos Santos Moschetta




On 6/25/21 2:22 PM, Daniel P. Berrangé wrote:

The builds for these two platforms can now be performed from GitLab CI
using cirrus-run.

Signed-off-by: Daniel P. Berrangé 
---
  .cirrus.yml | 55 -
  1 file changed, 55 deletions(-)


Reviewed-by: Wainer dos Santos Moschetta 



diff --git a/.cirrus.yml b/.cirrus.yml
index f4bf49b704..02c43a074a 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -1,61 +1,6 @@
  env:
CIRRUS_CLONE_DEPTH: 1
  
-freebsd_12_task:

-  freebsd_instance:
-image_family: freebsd-12-2
-cpu: 8
-memory: 8G
-  install_script:
-- ASSUME_ALWAYS_YES=yes pkg bootstrap -f ;
-- pkg install -y bash curl cyrus-sasl git glib gmake gnutls gsed
-  nettle perl5 pixman pkgconf png usbredir ninja
-  script:
-- mkdir build
-- cd build
-# TODO: Enable gnutls again once FreeBSD's libtasn1 got fixed
-# See: https://gitlab.com/gnutls/libtasn1/-/merge_requests/71
-- ../configure --enable-werror --disable-gnutls
-  || { cat config.log meson-logs/meson-log.txt; exit 1; }
-- gmake -j$(sysctl -n hw.ncpu)
-- gmake -j$(sysctl -n hw.ncpu) check V=1
-
-macos_task:
-  osx_instance:
-image: catalina-base
-  install_script:
-- brew install pkg-config python gnu-sed glib pixman make sdl2 bash ninja
-  script:
-- mkdir build
-- cd build
-- ../configure --python=/usr/local/bin/python3 --enable-werror
-   --extra-cflags='-Wno-error=deprecated-declarations'
-   || { cat config.log meson-logs/meson-log.txt; exit 1; }
-- gmake -j$(sysctl -n hw.ncpu)
-- gmake check-unit V=1
-- gmake check-block V=1
-- gmake check-qapi-schema V=1
-- gmake check-softfloat V=1
-- gmake check-qtest-x86_64 V=1
-
-macos_xcode_task:
-  osx_instance:
-# this is an alias for the latest Xcode
-image: catalina-xcode
-  install_script:
-- brew install pkg-config gnu-sed glib pixman make sdl2 bash ninja
-  script:
-- mkdir build
-- cd build
-- ../configure --extra-cflags='-Wno-error=deprecated-declarations' 
--enable-modules
-   --enable-werror --cc=clang || { cat config.log 
meson-logs/meson-log.txt; exit 1; }
-- gmake -j$(sysctl -n hw.ncpu)
-- gmake check-unit V=1
-- gmake check-block V=1
-- gmake check-qapi-schema V=1
-- gmake check-softfloat V=1
-- gmake check-qtest-x86_64 V=1
-
  windows_msys2_task:
timeout_in: 90m
windows_container:

Re: [PATCH 2/3] gitlab: support for FreeBSD 12, 13 and macOS 11 via cirrus-run

2021-06-30 Thread Wainer dos Santos Moschetta


Hi,

On 6/25/21 2:22 PM, Daniel P. Berrangé wrote:

This adds support for running 4 jobs via Cirrus CI runners:

  * FreeBSD 12
  * FreeBSD 13
  * macOS 11 with default XCode
  * macOS 11 with latest XCode

The gitlab job uses a container published by the libvirt-ci
project (https://gitlab.com/libvirt/libvirt-ci) that contains
the 'cirrus-run' command. This accepts a short yaml file that
describes a single Cirrus CI job, runs it using the Cirrus CI
REST API, and reports any output to the console.

In this way Cirrus CI is effectively working as an indirect
custom runner for GitLab CI pipelines. The key benefit is that
Cirrus CI job results affect the GitLab CI pipeline result and
so the user only has look at one CI dashboard.

Signed-off-by: Daniel P. Berrangé 
---
  .gitlab-ci.d/cirrus.yml | 103 
  .gitlab-ci.d/cirrus/README.rst  |  54 +++
  .gitlab-ci.d/cirrus/build.yml   |  35 ++
  .gitlab-ci.d/cirrus/freebsd-12.vars |  13 
  .gitlab-ci.d/cirrus/freebsd-13.vars |  13 
  .gitlab-ci.d/cirrus/macos-11.vars   |  15 
  .gitlab-ci.d/qemu-project.yml   |   1 +
  7 files changed, 234 insertions(+)
  create mode 100644 .gitlab-ci.d/cirrus.yml
  create mode 100644 .gitlab-ci.d/cirrus/README.rst
  create mode 100644 .gitlab-ci.d/cirrus/build.yml
  create mode 100644 .gitlab-ci.d/cirrus/freebsd-12.vars
  create mode 100644 .gitlab-ci.d/cirrus/freebsd-13.vars
  create mode 100644 .gitlab-ci.d/cirrus/macos-11.vars

diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
new file mode 100644
index 00..d7b4cce79b
--- /dev/null
+++ b/.gitlab-ci.d/cirrus.yml
@@ -0,0 +1,103 @@
+# Jobs that we delegate to Cirrus CI because they require an operating
+# system other than Linux. These jobs will only run if the required
+# setup has been performed on the GitLab account.
+#
+# The Cirrus CI configuration is generated by replacing target-specific
+# variables in a generic template: some of these variables are provided
+# when the GitLab CI job is defined, others are taken from a shell
+# snippet generated using lcitool.
+#
+# Note that the $PATH environment variable has to be treated with
+# special care, because we can't just override it at the GitLab CI job
+# definition level or we risk breaking it completely.
+.cirrus_build_job:
+  stage: build
+  image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:master
+  needs: []
+  script:
+- source .gitlab-ci.d/cirrus/$NAME.vars
+- sed -e "s|[@]CI_REPOSITORY_URL@|$CI_REPOSITORY_URL|g"
+  -e "s|[@]CI_COMMIT_REF_NAME@|$CI_COMMIT_REF_NAME|g"
+  -e "s|[@]CI_COMMIT_SHA@|$CI_COMMIT_SHA|g"
+  -e "s|[@]CIRRUS_VM_INSTANCE_TYPE@|$CIRRUS_VM_INSTANCE_TYPE|g"
+  -e "s|[@]CIRRUS_VM_IMAGE_SELECTOR@|$CIRRUS_VM_IMAGE_SELECTOR|g"
+  -e "s|[@]CIRRUS_VM_IMAGE_NAME@|$CIRRUS_VM_IMAGE_NAME|g"
+  -e "s|[@]CIRRUS_VM_CPUS@|$CIRRUS_VM_CPUS|g"
+  -e "s|[@]CIRRUS_VM_RAM@|$CIRRUS_VM_RAM|g"
+  -e "s|[@]UPDATE_COMMAND@|$UPDATE_COMMAND|g"
+  -e "s|[@]INSTALL_COMMAND@|$INSTALL_COMMAND|g"
+  -e "s|[@]PATH@|$PATH_EXTRA${PATH_EXTRA:+:}\$PATH|g"
+  -e "s|[@]PKG_CONFIG_PATH@|$PKG_CONFIG_PATH|g"
+  -e "s|[@]PKGS@|$PKGS|g"
+  -e "s|[@]MAKE@|$MAKE|g"
+  -e "s|[@]PYTHON@|$PYTHON|g"
+  -e "s|[@]PIP3@|$PIP3|g"
+  -e "s|[@]PYPI_PKGS@|$PYPI_PKGS|g"
+  -e "s|[@]CONFIGURE_ARGS@|$CONFIGURE_ARGS|g"
+  -e "s|[@]TEST_TARGETSS@|$TEST_TARGETSS|g"
+  <.gitlab-ci.d/cirrus/build.yml >.gitlab-ci.d/cirrus/$NAME.yml
+- cat .gitlab-ci.d/cirrus/$NAME.yml
+- cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml
+  rules:
+- if: "$TEMPORARILY_DISABLED"


Reading 'TEMPORARILY_DISABLED' I immediately think the job is 
malfunctioning or under maintenance.


But since the plan is to keep it running as 'non-gate' until it proves 
reliable, so maybe you could rename the variable to 'NON_GATE' or 
'STAGING_JOB' (i.e. some words to better express the intent).


Thanks!

- Wainer


+  allow_failure: true
+- if: "$CIRRUS_GITHUB_REPO && $CIRRUS_API_TOKEN"
+
+x64-freebsd-12-build:
+  extends: .cirrus_build_job
+  variables:
+NAME: freebsd-12
+CIRRUS_VM_INSTANCE_TYPE: freebsd_instance
+CIRRUS_VM_IMAGE_SELECTOR: image_family
+CIRRUS_VM_IMAGE_NAME: freebsd-12-2
+CIRRUS_VM_CPUS: 8
+CIRRUS_VM_RAM: 8G
+UPDATE_COMMAND: pkg update
+INSTALL_COMMAND: pkg install -y
+# TODO: Enable gnutls again once FreeBSD's libtasn1 got fixed
+# See: https://gitlab.com/gnutls/libtasn1/-/merge_requests/71
+CONFIGURE_ARGS: --disable-gnutls
+TEST_TARGETS: check
+
+x64-freebsd-13-build:
+  extends: .cirrus_build_job
+  variables:
+NAME: freebsd-13
+CIRRUS_VM_INSTANCE_TYPE: freebsd_instance
+CIRRUS_VM_IMAGE_SELECTOR: image_family
+CIRRUS_VM_IMAGE_NAME: freebsd-13-0
+CIRRUS_VM_CPUS: 8
+CIRRUS_VM_RAM: 8G
+UPDATE_COMMAND: pkg

[PATCH] python: Configure tox to skip missing interpreters

2021-06-30 Thread Wainer dos Santos Moschetta

Currently tox tests against the installed interpreters, however if any
supported interpreter is absent then it will return fail. It seems not
reasonable to expect developers to have all supported interpreters
installed on their systems. Luckily tox can be configured to skip
missing interpreters.

This changed the tox setup so that missing interpreters are skipped by
default. On the CI, however, we still want to enforce it tests
against all supported. This way on CI the
--skip-missing-interpreters=false option is passed to tox.

Signed-off-by: Wainer dos Santos Moschetta 
---
Tested locally with `make check-tox` and where I only Python 3.6 and 3.9
installed.
Tested on CI: https://gitlab.com/wainersm/qemu/-/jobs/1390010988
Still on CI, but I deliberately removed Python 3.8: 
https://gitlab.com/wainersm/qemu/-/jobs/1390046531

 .gitlab-ci.d/static_checks.yml | 1 +
 python/Makefile| 5 -
 python/setup.cfg   | 1 +
 3 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.d/static_checks.yml b/.gitlab-ci.d/static_checks.yml
index b01f6ec231..96dbd9e310 100644
--- a/.gitlab-ci.d/static_checks.yml
+++ b/.gitlab-ci.d/static_checks.yml
@@ -43,6 +43,7 @@ check-python-tox:
 - make -C python check-tox
   variables:
 GIT_DEPTH: 1
+QEMU_TOX_EXTRA_ARGS: --skip-missing-interpreters=false
   needs:
 job: python-container
   allow_failure: true
diff --git a/python/Makefile b/python/Makefile
index ac46ae33e7..fe27a3e12e 100644
--- a/python/Makefile
+++ b/python/Makefile
@@ -1,4 +1,5 @@
 QEMU_VENV_DIR=.dev-venv
+QEMU_TOX_EXTRA_ARGS ?=
 
 .PHONY: help
 help:
@@ -15,6 +16,8 @@ help:
@echo "These tests use the newest dependencies."
@echo "Requires: Python 3.6 - 3.10, and tox."
@echo "Hint (Fedora): 'sudo dnf install python3-tox python3.10'"
+   @echo "The variable QEMU_TOX_EXTRA_ARGS can be use to pass extra"
+   @echo "arguments to tox".
@echo ""
@echo "make check-dev:"
@echo "Run tests in a venv against your default python3 version."
@@ -87,7 +90,7 @@ check:
 
 .PHONY: check-tox
 check-tox:
-   @tox
+   @tox $(QEMU_TOX_EXTRA_ARGS)
 
 .PHONY: clean
 clean:
diff --git a/python/setup.cfg b/python/setup.cfg
index 11f71d5312..14bab90288 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -121,6 +121,7 @@ multi_line_output=3
 
 [tox:tox]
 envlist = py36, py37, py38, py39, py310
+skip_missing_interpreters = true
 
 [testenv]
 allowlist_externals = make
-- 
2.31.1

Re: [PATCH v2 07/12] python: update help text for check-tox

2021-06-30 Thread Wainer dos Santos Moschetta

On 6/29/21 6:27 PM, John Snow wrote:

On Tue, Jun 29, 2021 at 4:25 PM Wainer dos Santos Moschetta 
mailto:waine...@redhat.com>> wrote:

Hi John,

On 6/29/21 1:42 PM, John Snow wrote:
> Move it up near the check-pipenv help text, and update it to
suggest parity.
>
> (At the time I first added it, I wasn't sure if I would be
keeping it,
> but I've come to appreciate it as it has actually helped uncover
bugs I
> would not have noticed without it. It should stay.)
>
> Signed-off-by: John Snow mailto:js...@redhat.com>>
> Reviewed-by: Willian Rampazzo mailto:willi...@redhat.com>>
> ---
>   python/Makefile | 8 ++--
>   1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/python/Makefile b/python/Makefile
> index 07ad73ccd0..d2cfa6ad8f 100644
> --- a/python/Makefile
> +++ b/python/Makefile
> @@ -9,13 +9,17 @@ help:
>       @echo "    Requires: Python 3.6 and pipenv."
>       @echo "    Hint (Fedora): 'sudo dnf install python3.6 pipenv'"
>       @echo ""
> +     @echo "make check-tox:"
> +     @echo "    Run tests against multiple python versions."
> +     @echo "    These tests use the newest dependencies."
> +     @echo "    Requires: Python 3.6 - 3.10, and tox."
> +     @echo "    Hint (Fedora): 'sudo dnf install python3-tox
python3.10'"
> +     @echo ""

Somewhat related... in my system I don't have all supported python
versions installed, thus check-tox fails.

Instead, maybe, you could configure tox (as below) to test to
whatever
supported versions the developer have installed in the system; and on
absence of some versions it won't fail the tests entirely.

diff --git a/python/setup.cfg b/python/setup.cfg
index e730f208d3..1db8aaf340 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -123,6 +123,7 @@ multi_line_output=3

  [tox:tox]
  envlist = py36, py37, py38, py39, py310
+skip_missing_interpreters=true

Didn't know this was an option, to be honest ... I wonder if it can be 
toggled on/off easily? I like the idea that it will fail if we don't 
set up the CI environment correctly instead of succeeding quietly.

Though, you're right, some is better than none. Send a patch if you want?

I just sent a patch. Message-Id: 
<20210630184546.456582-1-waine...@redhat.com>

- Wainer

--js

Re: [PATCH v3 15/15] python: Fix broken ReST docstrings

2021-06-30 Thread Wainer dos Santos Moschetta




On 6/29/21 6:43 PM, John Snow wrote:

This patch *doesn't* update all of the docstring standards across the
QEMU package directory to make our docstring usage consistent. It
*doesn't* fix the formatting to make it look pretty or reasonable in
generated output. It *does* fix a few small instances where Sphinx would
emit a build warning because of malformed ReST -- If we built our Python
docs with Sphinx.

Signed-off-by: John Snow 

---

You'll have to take my word for it for now, or, to test that (ugly
though it may be) a theoretical Sphinx build would produce no build
errors:


cd ~/src/qemu/python
sphinx-apidoc --separate --private --no-toc --module-first \

 --implicit-namespaces --full --ext-intersphinx --ext-coverage \
 --ext-viewcode qemu/ -o docs/

sed -i '1s|^|import os; import sys; sys.path.insert(0, 
os.path.abspath("../"))\n|' docs/conf.py
make -C docs html
rm -rf docs/

I am preparing to add Sphinx, but need to fix these annoyances first so
that regressions are easy to spot as fixes are applied across the
tree. I plan to use my forthcoming Asynchronous QMP series as a test
pilot for documenting our docstring standards. Assuming it goes well, I
will update the docstrings elsewhere in this package at that time.

Signed-off-by: John Snow 
---
  python/qemu/machine/__init__.py | 6 +++---
  python/qemu/machine/machine.py  | 3 ++-
  python/qemu/qmp/__init__.py | 1 +
  python/qemu/qmp/qom_common.py   | 2 +-
  python/qemu/utils/accel.py  | 2 +-
  5 files changed, 8 insertions(+), 6 deletions(-)

Reviewed-by: Wainer dos Santos Moschetta 


diff --git a/python/qemu/machine/__init__.py b/python/qemu/machine/__init__.py
index 728f27adbe..9ccd58ef14 100644
--- a/python/qemu/machine/__init__.py
+++ b/python/qemu/machine/__init__.py
@@ -4,10 +4,10 @@
  This library provides a few high-level classes for driving QEMU from a
  test suite, not intended for production use.
  
-- QEMUMachine: Configure and Boot a QEMU VM

- - QEMUQtestMachine: VM class, with a qtest socket.
+ | QEMUQtestProtocol: send/receive qtest messages.
+ | QEMUMachine: Configure and Boot a QEMU VM
+ | +-- QEMUQtestMachine: VM class, with a qtest socket.
  
-- QEMUQtestProtocol: Connect to, send/receive qtest messages.

  """
  
  # Copyright (C) 2020-2021 John Snow for Red Hat Inc.

diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py
index e3345dfa1b..d47ab3d896 100644
--- a/python/qemu/machine/machine.py
+++ b/python/qemu/machine/machine.py
@@ -545,7 +545,8 @@ def set_qmp_monitor(self, enabled: bool = True) -> None:
  @param enabled: if False, qmp monitor options will be removed from
  the base arguments of the resulting QEMU command
  line. Default is True.
-@note: call this function before launch().
+
+.. note:: Call this function before launch().
  """
  self._qmp_set = enabled
  
diff --git a/python/qemu/qmp/__init__.py b/python/qemu/qmp/__init__.py

index 376954cb6d..269516a79b 100644
--- a/python/qemu/qmp/__init__.py
+++ b/python/qemu/qmp/__init__.py
@@ -279,6 +279,7 @@ def accept(self, timeout: Optional[float] = 15.0) -> 
QMPMessage:
  None). The value passed will set the behavior of the
  underneath QMP socket as described in [1].
  Default value is set to 15.0.
+
  @return QMP greeting dict
  @raise OSError on socket connection errors
  @raise QMPConnectError if the greeting is not received
diff --git a/python/qemu/qmp/qom_common.py b/python/qemu/qmp/qom_common.py
index f82b16772d..a59ae1a2a1 100644
--- a/python/qemu/qmp/qom_common.py
+++ b/python/qemu/qmp/qom_common.py
@@ -156,7 +156,7 @@ def command_runner(
  """
  Run a fully-parsed subcommand, with error-handling for the CLI.
  
-:return: The return code from `.run()`.

+:return: The return code from `run()`.
  """
  try:
  cmd = cls(args)
diff --git a/python/qemu/utils/accel.py b/python/qemu/utils/accel.py
index 297933df2a..386ff640ca 100644
--- a/python/qemu/utils/accel.py
+++ b/python/qemu/utils/accel.py
@@ -36,7 +36,7 @@ def list_accel(qemu_bin: str) -> List[str]:
  List accelerators enabled in the QEMU binary.
  
  @param qemu_bin (str): path to the QEMU binary.

-@raise Exception: if failed to run `qemu -accel help`
+@raise Exception: if failed to run ``qemu -accel help``
  @return a list of accelerator names.
  """
  if not qemu_bin:

Re: [PATCH v3 11/15] python: add 'make check-dev' invocation

2021-06-30 Thread Wainer dos Santos Moschetta



On 6/30/21 11:58 AM, John Snow wrote:



On Wed, Jun 30, 2021 at 10:41 AM Willian Rampazzo <mailto:wramp...@redhat.com>> wrote:


On Tue, Jun 29, 2021 at 6:43 PM John Snow mailto:js...@redhat.com>> wrote:
>
> This is a *third* way to run the Python tests. Unlike the first two
> (check-pipenv, check-tox), this version does not require any
specific
> interpreter version -- making it a lot easier to tell people to
run it
> as a quick smoketest prior to submission to GitLab CI.
>
> Summary:
>
>   Checked via GitLab CI:
>     - check-pipenv: tests our oldest python & dependencies
>     - check-tox: tests newest dependencies on all non-EOL python
versions
>   Executed only incidentally:
>     - check-dev: tests newest dependencies on whichever python
version
>
> ('make check' does not set up any environment at all, it just
runs the
> tests in your current environment. All four invocations perform the
> exact same tests, just in different execution environments.)
>
> Signed-off-by: John Snow mailto:js...@redhat.com>>
> ---
>  python/Makefile | 35 +--
>  1 file changed, 33 insertions(+), 2 deletions(-)
>

Reviewed-by: Willian Rampazzo mailto:willi...@redhat.com>>


Thanks. I am squashing in a hotfix here to add .dev-venv to 
.gitignore, too. Not worth an entire respin for that.

(Assuming that's gonna be A-OK with both of you.)


Acked-by: Wainer dos Santos Moschetta

Re: [PATCH v3 10/15] python: only check qemu/ subdir with flake8

2021-06-30 Thread Wainer dos Santos Moschetta




On 6/29/21 6:43 PM, John Snow wrote:

flake8 is a little eager to check everything it can. Limit it to
checking inside the qemu namespace directory only. Update setup.cfg now
that the exclude patterns are no longer necessary.

Signed-off-by: John Snow 
---
  python/setup.cfg   | 2 --
  python/tests/flake8.sh | 2 +-
  2 files changed, 1 insertion(+), 3 deletions(-)


Tested-by: Wainer dos Santos Moschetta 

Reviewed-by: Wainer dos Santos Moschetta 

How I tested it:

$ make check-dev
VENV .dev-venv
ACTIVATE .dev-venv
INSTALL qemu[devel] .dev-venv
ACTIVATE .dev-venv
make[1]: Entering directory '/home/wmoschet/src/qemu/python'
JOB ID : 41162d4a9e93095abb66c219227b6af27edbcf43
JOB LOG    : 
/home/wmoschet/avocado/job-results/job-2021-06-30T11.50-41162d4/job.log

 (1/4) tests/flake8.sh: PASS (0.30 s)
 (2/4) tests/isort.sh: PASS (0.12 s)
 (3/4) tests/mypy.sh: PASS (0.32 s)
 (4/4) tests/pylint.sh: PASS (4.46 s)
RESULTS    : PASS 4 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | 
CANCEL 0

JOB TIME   : 8.46 s
make[1]: Leaving directory '/home/wmoschet/src/qemu/python'



diff --git a/python/setup.cfg b/python/setup.cfg
index e730f208d3..11f71d5312 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -62,8 +62,6 @@ console_scripts =
  [flake8]
  extend-ignore = E722  # Prefer pylint's bare-except checks to flake8's
  exclude = __pycache__,
-  .venv,
-  .tox,
  
  [mypy]

  strict = True
diff --git a/python/tests/flake8.sh b/python/tests/flake8.sh
index 51e0788462..1cd7d40fad 100755
--- a/python/tests/flake8.sh
+++ b/python/tests/flake8.sh
@@ -1,2 +1,2 @@
  #!/bin/sh -e
-python3 -m flake8
+python3 -m flake8 qemu/

Re: [PATCH v3 11/15] python: add 'make check-dev' invocation

2021-06-30 Thread Wainer dos Santos Moschetta




On 6/29/21 6:43 PM, John Snow wrote:

This is a *third* way to run the Python tests. Unlike the first two
(check-pipenv, check-tox), this version does not require any specific
interpreter version -- making it a lot easier to tell people to run it
as a quick smoketest prior to submission to GitLab CI.

Summary:

   Checked via GitLab CI:
 - check-pipenv: tests our oldest python & dependencies
 - check-tox: tests newest dependencies on all non-EOL python versions
   Executed only incidentally:
 - check-dev: tests newest dependencies on whichever python version

('make check' does not set up any environment at all, it just runs the
tests in your current environment. All four invocations perform the
exact same tests, just in different execution environments.)

Signed-off-by: John Snow 
---
  python/Makefile | 35 +--
  1 file changed, 33 insertions(+), 2 deletions(-)


Tested-by: Wainer dos Santos Moschetta 

Reviewed-by: Wainer dos Santos Moschetta 

It was tested as I mentioned on patch 10.



diff --git a/python/Makefile b/python/Makefile
index d34c4e35d9..8f8e1999c0 100644
--- a/python/Makefile
+++ b/python/Makefile
@@ -1,3 +1,5 @@
+QEMU_VENV_DIR=.dev-venv
+
  .PHONY: help
  help:
@echo "python packaging help:"
@@ -14,6 +16,11 @@ help:
@echo "Requires: Python 3.6 - 3.10, and tox."
@echo "Hint (Fedora): 'sudo dnf install python3-tox python3.10'"
@echo ""
+   @echo "make check-dev:"
+   @echo "Run tests in a venv against your default python3 version."
+   @echo "These tests use the newest dependencies."
+   @echo "Requires: Python 3.x"
+   @echo ""
@echo "make develop:Install deps for 'make check', and"
@echo " the qemu libs in editable/development mode."
@echo ""
@@ -22,6 +29,9 @@ help:
@echo "make pipenv"
@echo "Creates pipenv's virtual environment (.venv)"
@echo ""
+   @echo "make dev-venv"
+   @echo "Creates a simple venv for check-dev. ($(QEMU_VENV_DIR))"
+   @echo ""
@echo "make clean:  remove package build output."
@echo ""
@echo "make distclean:  remove venv files, qemu package forwarder,"
@@ -38,9 +48,30 @@ pipenv: .venv
  check-pipenv: pipenv
@pipenv run make check
  
+.PHONY: dev-venv

+dev-venv: $(QEMU_VENV_DIR) $(QEMU_VENV_DIR)/bin/activate
+$(QEMU_VENV_DIR) $(QEMU_VENV_DIR)/bin/activate: setup.cfg
+   @echo "VENV $(QEMU_VENV_DIR)"
+   @python3 -m venv $(QEMU_VENV_DIR)
+   @(  \
+   echo "ACTIVATE $(QEMU_VENV_DIR)"; \
+   . $(QEMU_VENV_DIR)/bin/activate;\
+   echo "INSTALL qemu[devel] $(QEMU_VENV_DIR)";  \
+   make develop 1>/dev/null;\
+   )
+   @touch $(QEMU_VENV_DIR)
+
+.PHONY: check-dev
+check-dev: dev-venv
+   @(  \
+   echo "ACTIVATE $(QEMU_VENV_DIR)"; \
+   . $(QEMU_VENV_DIR)/bin/activate;\
+   make check; \
+   )
+
  .PHONY: develop
  develop:
-   pip3 install -e .[devel]
+   pip3 install --disable-pip-version-check -e .[devel]
  
  .PHONY: check

  check:
@@ -56,4 +87,4 @@ clean:
  
  .PHONY: distclean

  distclean: clean
-   rm -rf qemu.egg-info/ .venv/ .tox/ dist/
+   rm -rf qemu.egg-info/ .venv/ .tox/ $(QEMU_VENV_DIR) dist/

Re: [PATCH v3 01/15] python/qom: Do not use 'err' name at module scope

2021-06-30 Thread Wainer dos Santos Moschetta




On 6/29/21 6:43 PM, John Snow wrote:

Pylint updated to 2.9.0 upstream, adding new warnings for things that
re-use the 'err' variable. Luckily, this only breaks the
python-check-tox job, which is allowed to fail as a warning.

Signed-off-by: John Snow 

---

I guess that's good enough evidence that check-pipenv and check-tox both
have their place :)

Signed-off-by: John Snow 
---
  python/qemu/qmp/qom.py | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)

Reviewed-by: Wainer dos Santos Moschetta 


diff --git a/python/qemu/qmp/qom.py b/python/qemu/qmp/qom.py
index 7ec7843d57..8ff28a8343 100644
--- a/python/qemu/qmp/qom.py
+++ b/python/qemu/qmp/qom.py
@@ -38,8 +38,8 @@
  
  try:

  from .qom_fuse import QOMFuse
-except ModuleNotFoundError as err:
-if err.name != 'fuse':
+except ModuleNotFoundError as _err:
+if _err.name != 'fuse':
  raise
  else:
  assert issubclass(QOMFuse, QOMCommand)

Re: [PATCH v2 07/12] python: update help text for check-tox

2021-06-29 Thread Wainer dos Santos Moschetta


Hi John,

On 6/29/21 1:42 PM, John Snow wrote:

Move it up near the check-pipenv help text, and update it to suggest parity.

(At the time I first added it, I wasn't sure if I would be keeping it,
but I've come to appreciate it as it has actually helped uncover bugs I
would not have noticed without it. It should stay.)

Signed-off-by: John Snow 
Reviewed-by: Willian Rampazzo 
---
  python/Makefile | 8 ++--
  1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/python/Makefile b/python/Makefile
index 07ad73ccd0..d2cfa6ad8f 100644
--- a/python/Makefile
+++ b/python/Makefile
@@ -9,13 +9,17 @@ help:
@echo "Requires: Python 3.6 and pipenv."
@echo "Hint (Fedora): 'sudo dnf install python3.6 pipenv'"
@echo ""
+   @echo "make check-tox:"
+   @echo "Run tests against multiple python versions."
+   @echo "These tests use the newest dependencies."
+   @echo "Requires: Python 3.6 - 3.10, and tox."
+   @echo "Hint (Fedora): 'sudo dnf install python3-tox python3.10'"
+   @echo ""


Somewhat related... in my system I don't have all supported python 
versions installed, thus check-tox fails.


Instead, maybe, you could configure tox (as below) to test to whatever 
supported versions the developer have installed in the system; and on 
absence of some versions it won't fail the tests entirely.


diff --git a/python/setup.cfg b/python/setup.cfg
index e730f208d3..1db8aaf340 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -123,6 +123,7 @@ multi_line_output=3

 [tox:tox]
 envlist = py36, py37, py38, py39, py310
+skip_missing_interpreters=true

 [testenv]
 allowlist_externals = make


@echo "make develop:Install deps for 'make check', and"
@echo " the qemu libs in editable/development mode."
@echo ""
@echo "make check:  run linters using the current environment."
@echo ""
-   @echo "make check-tox:  run linters using multiple python versions."
-   @echo ""
@echo "make pipenv"
@echo "Creates pipenv's virtual environment (.venv)"
@echo ""

Re: [PATCH v2 09/12] python: add 'make check-dev' invocation

2021-06-29 Thread Wainer dos Santos Moschetta




On 6/29/21 1:42 PM, John Snow wrote:

This is a *third* way to run the Python tests. Unlike the first two
(check-pipenv, check-tox), this version does not require any specific
interpreter version -- making it a lot easier to tell people to run it
as a quick smoketest prior to submission to GitLab CI.

Summary:

   Checked via GitLab CI:
 - check-pipenv: tests our oldest python & dependencies
 - check-tox: tests newest dependencies on all non-EOL python versions
   Executed only incidentally:
 - check-dev: tests newest dependencies on whichever python version

('make check' does not set up any environment at all, it just runs the
tests in your current environment. All four invocations perform the
exact same tests, just in different execution environments.)

Signed-off-by: John Snow 
---
  python/Makefile | 35 +--
  1 file changed, 33 insertions(+), 2 deletions(-)

diff --git a/python/Makefile b/python/Makefile
index d34c4e35d9..8f8e1999c0 100644
--- a/python/Makefile
+++ b/python/Makefile
@@ -1,3 +1,5 @@
+QEMU_VENV_DIR=.dev-venv
+


I gave `make check-dev` a try and `tests/flake8.sh` failed, because it 
is checking the python sources from the $QEMU_VENV_DIR itself.


Either we keep $QEMU_VENV_DIR outside (as in the v1 series) or ...

  diff --git a/python/tests/flake8.sh b/python/tests/flake8.sh
  index 51e0788462..1cd7d40fad 100755
  --- a/python/tests/flake8.sh
  +++ b/python/tests/flake8.sh
  @@ -1,2 +1,2 @@
   #!/bin/sh -e
  -python3 -m flake8
  +python3 -m flake8 qemu/

... to ensure flake8 parses only the files from the 'qemu' directory 
(btw, this is how tests/pylint.sh is configured). I prefer the later.



  .PHONY: help
  help:
@echo "python packaging help:"
@@ -14,6 +16,11 @@ help:
@echo "Requires: Python 3.6 - 3.10, and tox."
@echo "Hint (Fedora): 'sudo dnf install python3-tox python3.10'"
@echo ""
+   @echo "make check-dev:"
+   @echo "Run tests in a venv against your default python3 version."
+   @echo "These tests use the newest dependencies."
+   @echo "Requires: Python 3.x"
+   @echo ""
@echo "make develop:Install deps for 'make check', and"
@echo " the qemu libs in editable/development mode."
@echo ""
@@ -22,6 +29,9 @@ help:
@echo "make pipenv"
@echo "Creates pipenv's virtual environment (.venv)"
@echo ""
+   @echo "make dev-venv"
+   @echo "Creates a simple venv for check-dev. ($(QEMU_VENV_DIR))"
+   @echo ""
@echo "make clean:  remove package build output."
@echo ""
@echo "make distclean:  remove venv files, qemu package forwarder,"
@@ -38,9 +48,30 @@ pipenv: .venv
  check-pipenv: pipenv
@pipenv run make check
  
+.PHONY: dev-venv

+dev-venv: $(QEMU_VENV_DIR) $(QEMU_VENV_DIR)/bin/activate
+$(QEMU_VENV_DIR) $(QEMU_VENV_DIR)/bin/activate: setup.cfg
+   @echo "VENV $(QEMU_VENV_DIR)"
+   @python3 -m venv $(QEMU_VENV_DIR)
+   @(  \
+   echo "ACTIVATE $(QEMU_VENV_DIR)"; \
+   . $(QEMU_VENV_DIR)/bin/activate;\
+   echo "INSTALL qemu[devel] $(QEMU_VENV_DIR)";  \
+   make develop 1>/dev/null;\
+   )
+   @touch $(QEMU_VENV_DIR)
+
+.PHONY: check-dev
+check-dev: dev-venv
+   @(  \
+   echo "ACTIVATE $(QEMU_VENV_DIR)"; \
+   . $(QEMU_VENV_DIR)/bin/activate;\
+   make check; \
+   )
+
  .PHONY: develop
  develop:
-   pip3 install -e .[devel]
+   pip3 install --disable-pip-version-check -e .[devel]
  
  .PHONY: check

  check:
@@ -56,4 +87,4 @@ clean:
  
  .PHONY: distclean

  distclean: clean
-   rm -rf qemu.egg-info/ .venv/ .tox/ dist/
+   rm -rf qemu.egg-info/ .venv/ .tox/ $(QEMU_VENV_DIR) dist/

Re: [PATCH v2 08/12] python: Fix .PHONY Make specifiers

2021-06-29 Thread Wainer dos Santos Moschetta




On 6/29/21 1:42 PM, John Snow wrote:

I missed the 'check-tox' target. Add that, but split the large .PHONY
specifier at the top into its component pieces and move them near the
targets they describe so that they're much harder to forget to update.

Signed-off-by: John Snow 
---
  python/Makefile | 10 --
  1 file changed, 8 insertions(+), 2 deletions(-)


I didn't know .PHONY could be split. Nice!

Reviewed-by: Wainer dos Santos Moschetta 



diff --git a/python/Makefile b/python/Makefile
index d2cfa6ad8f..d34c4e35d9 100644
--- a/python/Makefile
+++ b/python/Makefile
@@ -1,5 +1,4 @@
-.PHONY: help pipenv check-pipenv check clean distclean develop
-
+.PHONY: help
  help:
@echo "python packaging help:"
@echo ""
@@ -29,25 +28,32 @@ help:
@echo " built distribution files, and everything"
@echo " from 'make clean'."
  
+.PHONY: pipenv

  pipenv: .venv
  .venv: Pipfile.lock
@PIPENV_VENV_IN_PROJECT=1 pipenv sync --dev --keep-outdated
@touch .venv
  
+.PHONY: check-pipenv

  check-pipenv: pipenv
@pipenv run make check
  
+.PHONY: develop

  develop:
pip3 install -e .[devel]
  
+.PHONY: check

  check:
@avocado --config avocado.cfg run tests/
  
+.PHONY: check-tox

  check-tox:
@tox
  
+.PHONY: clean

  clean:
python3 setup.py clean --all
  
+.PHONY: distclean

  distclean: clean
rm -rf qemu.egg-info/ .venv/ .tox/ dist/

Re: [PATCH 07/11] python: update help text for check-tox

2021-06-29 Thread Wainer dos Santos Moschetta



On 6/29/21 12:40 PM, John Snow wrote:



On Mon, Jun 28, 2021 at 5:16 PM Wainer dos Santos Moschetta 
mailto:waine...@redhat.com>> wrote:


Hi,

On 6/25/21 12:45 PM, John Snow wrote:
> Move it up near the check-pipenv help text, and update it to
suggest parity.
>
> (At the time I first added it, I wasn't sure if I would be
keeping it,
> but I've come to appreciate it as it has actually helped uncover
bugs I
> would not have noticed without it. It should stay.)
> Signed-off-by: John Snow mailto:js...@redhat.com>>
> ---
>   python/Makefile | 8 ++--
>   1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/python/Makefile b/python/Makefile
> index 07ad73ccd0..5cb8378b81 100644
> --- a/python/Makefile
> +++ b/python/Makefile
> @@ -9,13 +9,17 @@ help:
>       @echo "    Requires: Python 3.6 and pipenv."
>       @echo "    Hint (Fedora): 'sudo dnf install python3.6 pipenv'"
>       @echo ""
> +     @echo "make check-tox:"
> +     @echo "    Run tests against multiple python versions."
> +     @echo "    These tests use the newest dependencies."
> +     @echo "    Requires: Python 3.6-3.10 and tox."

Nit: I'm not sure it is clear '3.6-3.10' is a range rather than a
specific version.


It was meant to be a range. Would you prefer "3.6 - 3.10" instead, then?


I think it's better.

- Wainer



- Wainer

Re: [PATCH v2 07/12] python: update help text for check-tox

2021-06-29 Thread Wainer dos Santos Moschetta




On 6/29/21 1:42 PM, John Snow wrote:

Move it up near the check-pipenv help text, and update it to suggest parity.

(At the time I first added it, I wasn't sure if I would be keeping it,
but I've come to appreciate it as it has actually helped uncover bugs I
would not have noticed without it. It should stay.)

Signed-off-by: John Snow 
Reviewed-by: Willian Rampazzo 
---
  python/Makefile | 8 ++--
  1 file changed, 6 insertions(+), 2 deletions(-)


Reviewed-by: Wainer dos Santos Moschetta 



diff --git a/python/Makefile b/python/Makefile
index 07ad73ccd0..d2cfa6ad8f 100644
--- a/python/Makefile
+++ b/python/Makefile
@@ -9,13 +9,17 @@ help:
@echo "Requires: Python 3.6 and pipenv."
@echo "Hint (Fedora): 'sudo dnf install python3.6 pipenv'"
@echo ""
+   @echo "make check-tox:"
+   @echo "Run tests against multiple python versions."
+   @echo "These tests use the newest dependencies."
+   @echo "Requires: Python 3.6 - 3.10, and tox."
+   @echo "Hint (Fedora): 'sudo dnf install python3-tox python3.10'"
+   @echo ""
@echo "make develop:Install deps for 'make check', and"
@echo " the qemu libs in editable/development mode."
@echo ""
@echo "make check:  run linters using the current environment."
@echo ""
-   @echo "make check-tox:  run linters using multiple python versions."
-   @echo ""
@echo "make pipenv"
@echo "Creates pipenv's virtual environment (.venv)"
@echo ""

Re: [PATCH 03/11] python: Re-lock pipenv at oldest supported versions

2021-06-29 Thread Wainer dos Santos Moschetta

On 6/28/21 5:38 PM, John Snow wrote:

On Mon, Jun 28, 2021 at 4:31 PM Wainer dos Santos Moschetta 
mailto:waine...@redhat.com>> wrote:

On 6/25/21 12:45 PM, John Snow wrote:
> tox is already testing the most recent versions. Let's use pipenv to
> test the oldest versions we claim to support. This matches the
stylistic
> choice to have pipenv always test our oldest supported Python
version, 3.6.
Currently tox is testing with 3.6 too. Should we remove that version
from the test matrix?

Tox will test with 3.6, but against the very latest pylint/mypy. This 
test will also use 3.6, but against the oldest pylint/mypy we claim to 
support in the package metadata. I think they are both useful.

The pipenv test will make sure we don't accidentally increase our 
dependency requirements without noticing. This has some value for 
other QEMU developers that are used to running and managing their own 
linters, such as we did for a long time. I don't want to break their 
environments by accident. (It also makes sure that no 3.7+ isms sneak 
into the code by accident, too. This has historically been, and I 
anticipate it will continue to be, a prominent problem in the Python 
library.)

The tox test gives us a heads up if there are incompatible changes 
arriving in the bleeding edge for pylint/mypy et al that I will need 
to be aware of before they are rolled out and wind up on random 
developer's systems and start causing 'make check' to fail.

I think it's worth keeping both for now, provided the run cost isn't 
too great. At the moment it seems rather minimal.

While I was reviewing the next patches I started to understand the tox 
and pipenv testing strategies. I also think they are useful.

Thanks for the details explanation above.

- Wainer

>
> The effect of this is that the python-check-pipenv CI job on
gitlab will
> now test against much older versions of these linters, which
will help
> highlight incompatible changes that might otherwise go unnoticed.
>
> Update instructions for adding and bumping versions in
setup.cfg. The
> reason for deleting the line that gets added to Pipfile is
largely just
> to avoid having the version minimums specified in multiple places in
> config checked into the tree.
>
> (This patch was written by deleting Pipfile and Pipfile.lock, then
> explicitly installing each dependency manually at a specific
> version. Then, I restored the prior Pipfile and re-ran `pipenv lock
> --dev --keep-outdated` to re-add the qemu dependency back to the
pipenv
> environment while keeping the "old" packages. It's annoying,
yes, but I
> think the improvement to test coverage is worthwhile.)
>
> Signed-off-by: John Snow mailto:js...@redhat.com>>
> ---
>   python/Pipfile.lock | 113
+---
>   python/setup.cfg    |   4 +-
    >   2 files changed, 56 insertions(+), 61 deletions(-)

Reviewed-by: Wainer dos Santos Moschetta mailto:waine...@redhat.com>>

Thanks!

Re: [PATCH 11/11] python: remove auto-generated pyproject.toml file

2021-06-28 Thread Wainer dos Santos Moschetta




On 6/25/21 12:45 PM, John Snow wrote:

For reasons that at-present escape me, pipenv insists on creating a stub
pyproject.toml file. This file is a nuisance, because its mere presence
changes the behavior of various tools.

For instance, this stub file will cause "pip install --user -e ." to
fail in spectacular fashion with misleading errors. "pip install -e ."
works okay, but for some reason pip does not support editable installs
to the user directory when using PEP517.

References:
   https://github.com/pypa/pip/pull/9990
   https://github.com/pypa/pip/issues/7953

As outlined in ea1213b7ccc, it is still too early for us to consider
moving to a PEP-517 exclusive package. We must support support older
distributions, so squash the annoyance for now. (Python 3.6 shipped Dec
2016, PEP517 support showed up in pip sometime in 2019 or so.)

Add 'pyproject.toml' to the 'make clean' target, and also delete it
after every pipenv invocation issued by the Makefile.

Signed-off-by: John Snow 
---
  python/Makefile | 2 ++
  1 file changed, 2 insertions(+)

Reviewed-by: Wainer dos Santos Moschetta 


diff --git a/python/Makefile b/python/Makefile
index 06f78f760a..758ce2c8d8 100644
--- a/python/Makefile
+++ b/python/Makefile
@@ -49,6 +49,7 @@ help:
  pipenv: .venv
  .venv: Pipfile.lock
@PIPENV_VENV_IN_PROJECT=1 pipenv sync --dev --keep-outdated
+   rm -f pyproject.toml
@touch .venv
  
  check-pipenv: pipenv

@@ -84,6 +85,7 @@ check-tox:
  
  clean:

python3 setup.py clean --all
+   rm -f pyproject.toml
  
  distclean: clean

rm -rf qemu.egg-info/ .venv/ .tox/ $(QEMU_VENV_DIR) dist/

Re: [PATCH 10/11] python: Update help text on 'make clean', 'make distclean'

2021-06-28 Thread Wainer dos Santos Moschetta




On 6/25/21 12:45 PM, John Snow wrote:

Just for visual parity with everything else.

Signed-off-by: John Snow 
---
  python/Makefile | 11 +++
  1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/python/Makefile b/python/Makefile
index 4ed37c29f0..06f78f760a 100644
--- a/python/Makefile
+++ b/python/Makefile
@@ -37,11 +37,14 @@ help:
@echo "make venv"
@echo "Creates a simple venv for check-venv. ($(QEMU_VENV_DIR))"
@echo ""
-   @echo "make clean:  remove package build output."
+   @echo "make clean:"
+   @echo "Remove package build output."
@echo ""
-   @echo "make distclean:  remove venv files, qemu package forwarder,"
-   @echo " built distribution files, and everything"
-   @echo " from 'make clean'."
+   @echo "make distclean:"
+   @echo "remove pipenv/venv files, qemu package forwarder,"
+   @echo "built distribution files, and everything from 'make clean'."
+   @echo ""
+   @echo -e "Have a nice day ^_^\n"


Devs will like the last message. ;)

Reviewed-by: Wainer dos Santos Moschetta 

  
  pipenv: .venv

  .venv: Pipfile.lock

Re: [PATCH 09/11] python: Update help text on 'make check', 'make develop'

2021-06-28 Thread Wainer dos Santos Moschetta




On 6/25/21 12:45 PM, John Snow wrote:

Update for visual parity with the other targets.

Signed-off-by: John Snow 
---
  python/Makefile | 10 +++---
  1 file changed, 7 insertions(+), 3 deletions(-)

Reviewed-by: Wainer dos Santos Moschetta 


diff --git a/python/Makefile b/python/Makefile
index 76bb24e671..4ed37c29f0 100644
--- a/python/Makefile
+++ b/python/Makefile
@@ -22,10 +22,14 @@ help:
@echo "These tests use the newest dependencies."
@echo "Requires: Python 3.x"
@echo ""
-   @echo "make develop:Install deps for 'make check', and"
-   @echo " the qemu libs in editable/development mode."
+   @echo "make check:"
+   @echo "Run tests in your *current environment*."
+   @echo "Performs no environment setup of any kind."
@echo ""
-   @echo "make check:  run linters using the current environment."
+   @echo "make develop:"
+   @echo "Install deps needed for for 'make check',"
+   @echo "and install the qemu package in editable mode."
+   @echo "(Can be used in or outside of a venv.)"
@echo ""
@echo "make pipenv"
@echo "Creates pipenv's virtual environment (.venv)"

Re: [PATCH 08/11] python: add 'make check-venv' invocation

2021-06-28 Thread Wainer dos Santos Moschetta




On 6/25/21 12:45 PM, John Snow wrote:

This is a *third* way to run the Python tests. Unlike the first two
(check-pipenv, check-tox), this version does not require any specific
interpreter version -- making it a lot easier to tell people to run it
as a quick smoketest prior to submission to GitLab CI.

Summary:

   Checked via GitLab CI:
 - check-pipenv: tests our oldest python & dependencies
 - check-tox: tests newest dependencies on all non-EOL python versions
   Executed only incidentally:
 - check-venv: tests newest dependencies on whichever python version

('make check' does not set up any environment at all, it just runs the
tests in your current environment. All four invocations perform the
exact same tests, just in different execution environments.)

Signed-off-by: John Snow 
---
  python/Makefile | 35 ---
  1 file changed, 32 insertions(+), 3 deletions(-)

diff --git a/python/Makefile b/python/Makefile
index 5cb8378b81..76bb24e671 100644
--- a/python/Makefile
+++ b/python/Makefile
@@ -1,4 +1,6 @@
-.PHONY: help pipenv check-pipenv check clean distclean develop
+.PHONY: help pipenv venv check-venv check-pipenv check clean distclean develop
+

btw, check-tox is missed here ^

+QEMU_VENV_DIR=~/.cache/qemu-pyvenv


A few suggestions:

1. For the sake of consistence with others temporary directories 
created, use QEMU_VENV_DIR=/.devvenv


2. Reword to 'devvenv' or 'dev-venv' (or something similar), instead of 
'venv', the directories and Make targets. IMHO it will make the purpose 
of the targets a bit clear.


What do you think John?

- Wainer

  
  help:

@echo "python packaging help:"
@@ -15,6 +17,11 @@ help:
@echo "Requires: Python 3.6-3.10 and tox."
@echo "Hint (Fedora): 'sudo dnf install python3-tox python3.10'"
@echo ""
+   @echo "make check-venv:"
+   @echo "Run tests in a venv against your default python3 version."
+   @echo "These tests use the newest dependencies."
+   @echo "Requires: Python 3.x"
+   @echo ""
@echo "make develop:Install deps for 'make check', and"
@echo " the qemu libs in editable/development mode."
@echo ""
@@ -23,6 +30,9 @@ help:
@echo "make pipenv"
@echo "Creates pipenv's virtual environment (.venv)"
@echo ""
+   @echo "make venv"
+   @echo "Creates a simple venv for check-venv. ($(QEMU_VENV_DIR))"
+   @echo ""
@echo "make clean:  remove package build output."
@echo ""
@echo "make distclean:  remove venv files, qemu package forwarder,"
@@ -37,8 +47,27 @@ pipenv: .venv
  check-pipenv: pipenv
@pipenv run make check
  
+venv: $(QEMU_VENV_DIR) $(QEMU_VENV_DIR)/bin/activate

+$(QEMU_VENV_DIR) $(QEMU_VENV_DIR)/bin/activate: setup.cfg
+   @echo "VENV $(QEMU_VENV_DIR)"
+   @python3 -m venv $(QEMU_VENV_DIR)
+   @(  \
+   echo "ACTIVATE $(QEMU_VENV_DIR)"; \
+   . $(QEMU_VENV_DIR)/bin/activate;\
+   echo "INSTALL qemu[devel] $(QEMU_VENV_DIR)";  \
+   make develop 1>/dev/null;\
+   )
+   @touch $(QEMU_VENV_DIR)
+
+check-venv: venv
+   @(  \
+   echo "ACTIVATE $(QEMU_VENV_DIR)"; \
+   . $(QEMU_VENV_DIR)/bin/activate;\
+   make check; \
+   )
+
  develop:
-   pip3 install -e .[devel]
+   pip3 install --disable-pip-version-check -e .[devel]
  
  check:

@avocado --config avocado.cfg run tests/
@@ -50,4 +79,4 @@ clean:
python3 setup.py clean --all
  
  distclean: clean

-   rm -rf qemu.egg-info/ .venv/ .tox/ dist/
+   rm -rf qemu.egg-info/ .venv/ .tox/ $(QEMU_VENV_DIR) dist/

Re: [PATCH 07/11] python: update help text for check-tox

2021-06-28 Thread Wainer dos Santos Moschetta


Hi,

On 6/25/21 12:45 PM, John Snow wrote:

Move it up near the check-pipenv help text, and update it to suggest parity.

(At the time I first added it, I wasn't sure if I would be keeping it,
but I've come to appreciate it as it has actually helped uncover bugs I
would not have noticed without it. It should stay.)
Signed-off-by: John Snow 
---
  python/Makefile | 8 ++--
  1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/python/Makefile b/python/Makefile
index 07ad73ccd0..5cb8378b81 100644
--- a/python/Makefile
+++ b/python/Makefile
@@ -9,13 +9,17 @@ help:
@echo "Requires: Python 3.6 and pipenv."
@echo "Hint (Fedora): 'sudo dnf install python3.6 pipenv'"
@echo ""
+   @echo "make check-tox:"
+   @echo "Run tests against multiple python versions."
+   @echo "These tests use the newest dependencies."
+   @echo "Requires: Python 3.6-3.10 and tox."


Nit: I'm not sure it is clear '3.6-3.10' is a range rather than a 
specific version.


- Wainer


+   @echo "Hint (Fedora): 'sudo dnf install python3-tox python3.10'"
+   @echo ""
@echo "make develop:Install deps for 'make check', and"
@echo " the qemu libs in editable/development mode."
@echo ""
@echo "make check:  run linters using the current environment."
@echo ""
-   @echo "make check-tox:  run linters using multiple python versions."
-   @echo ""
@echo "make pipenv"
@echo "Creates pipenv's virtual environment (.venv)"
@echo ""

Re: [PATCH 06/11] python: rename 'venv-check' target to 'check-pipenv'

2021-06-28 Thread Wainer dos Santos Moschetta




On 6/25/21 12:45 PM, John Snow wrote:

Well, Cleber was right, this is a better name.

In preparation for adding a different kind of virtual environment check
(One that simply uses whichever version of Python you happen to have),
rename this test 'check-pipenv' so that it matches the CI job
'check-python-pipenv'.

Remove the "If you don't know which test to run" hint, because it's not
actually likely you have Python 3.6 installed to be able to run the
test. It's still the test I'd most prefer you to run, but it's not the
test you are most likely to be able to run.

Rename the 'venv' target to 'pipenv' as well, and move the more
pertinent help text under the 'check-pipenv' target.
Signed-off-by: John Snow 
---
  python/README.rst  |  2 +-
  .gitlab-ci.d/static_checks.yml |  2 +-
  python/Makefile| 21 +++--
  3 files changed, 13 insertions(+), 12 deletions(-)


I also agree pipenv/check-pipenvit is a better name.

Reviewed-by: Wainer dos Santos Moschetta 



diff --git a/python/README.rst b/python/README.rst
index d4502fdb60..9c1fceaee7 100644
--- a/python/README.rst
+++ b/python/README.rst
@@ -79,7 +79,7 @@ Files in this directory
  - ``PACKAGE.rst`` is used as the README file that is visible on PyPI.org.
  - ``Pipfile`` is used by Pipenv to generate ``Pipfile.lock``.
  - ``Pipfile.lock`` is a set of pinned package dependencies that this package
-  is tested under in our CI suite. It is used by ``make venv-check``.
+  is tested under in our CI suite. It is used by ``make check-pipenv``.
  - ``README.rst`` you are here!
  - ``VERSION`` contains the PEP-440 compliant version used to describe
this package; it is referenced by ``setup.cfg``.
diff --git a/.gitlab-ci.d/static_checks.yml b/.gitlab-ci.d/static_checks.yml
index 7e685c6a65..8c7dc41abd 100644
--- a/.gitlab-ci.d/static_checks.yml
+++ b/.gitlab-ci.d/static_checks.yml
@@ -33,7 +33,7 @@ check-python-pipenv:
stage: test
image: $CI_REGISTRY_IMAGE/qemu/python:latest
script:
-- make -C python venv-check
+- make -C python check-pipenv
variables:
  GIT_DEPTH: 1
needs:
diff --git a/python/Makefile b/python/Makefile
index b5621b0d54..07ad73ccd0 100644
--- a/python/Makefile
+++ b/python/Makefile
@@ -1,15 +1,13 @@
-.PHONY: help venv venv-check check clean distclean develop
+.PHONY: help pipenv check-pipenv check clean distclean develop
  
  help:

@echo "python packaging help:"
@echo ""
-   @echo "make venv:   Create pipenv's virtual environment."
-   @echo "NOTE: Requires Python 3.6 and pipenv."
-   @echo "  Will download packages from PyPI."
-   @echo "Hint: (On Fedora): 'sudo dnf install python36 pipenv'"
-   @echo ""
-   @echo "make venv-check: run linters using pipenv's virtual environment."
-   @echo "Hint: If you don't know which test to run, run this one!"
+   @echo "make check-pipenv:"
+   @echo "Run tests in pipenv's virtual environment."
+   @echo "These tests use the oldest dependencies."
+   @echo "Requires: Python 3.6 and pipenv."
+   @echo "Hint (Fedora): 'sudo dnf install python3.6 pipenv'"
@echo ""
@echo "make develop:Install deps for 'make check', and"
@echo " the qemu libs in editable/development mode."
@@ -18,18 +16,21 @@ help:
@echo ""
@echo "make check-tox:  run linters using multiple python versions."
@echo ""
+   @echo "make pipenv"
+   @echo "Creates pipenv's virtual environment (.venv)"
+   @echo ""
@echo "make clean:  remove package build output."
@echo ""
@echo "make distclean:  remove venv files, qemu package forwarder,"
@echo " built distribution files, and everything"
@echo " from 'make clean'."
  
-venv: .venv

+pipenv: .venv
  .venv: Pipfile.lock
@PIPENV_VENV_IN_PROJECT=1 pipenv sync --dev --keep-outdated
@touch .venv
  
-venv-check: venv

+check-pipenv: pipenv
@pipenv run make check
  
  develop:

Re: [PATCH 05/11] python: Add no-install usage instructions

2021-06-28 Thread Wainer dos Santos Moschetta




On 6/25/21 12:45 PM, John Snow wrote:

It's not encouraged, but it's legitimate to want to know how to do.

Signed-off-by: John Snow 
---
  python/README.rst | 28 
  1 file changed, 28 insertions(+)

Reviewed-by: Wainer dos Santos Moschetta 


diff --git a/python/README.rst b/python/README.rst
index 107786ffdc..d4502fdb60 100644
--- a/python/README.rst
+++ b/python/README.rst
@@ -37,6 +37,34 @@ See `Installing packages using pip and virtual environments
  for more information.
  
  
+Using these packages without installing them

+
+
+These packages may be used without installing them first, by using one
+of two tricks:
+
+1. Set your PYTHONPATH environment variable to include this source
+   directory, e.g. ``~/src/qemu/python``. See
+   https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPATH
+
+2. Inside a Python script, use ``sys.path`` to forcibly include a search
+   path prior to importing the ``qemu`` namespace. See
+   https://docs.python.org/3/library/sys.html#sys.path
+
+A strong downside to both approaches is that they generally interfere
+with static analysis tools being able to locate and analyze the code
+being imported.
+
+Package installation also normally provides executable console scripts,
+so that tools like ``qmp-shell`` are always available via $PATH. To
+invoke them without installation, you can invoke e.g.:
+
+``> PYTHONPATH=~/src/qemu/python python3 -m qemu.qmp.qmp_shell``
+
+The mappings between console script name and python module path can be
+found in ``setup.cfg``.
+
+
  Files in this directory
  ---

Re: [PATCH 04/11] python: README.rst touchups

2021-06-28 Thread Wainer dos Santos Moschetta




On 6/25/21 12:45 PM, John Snow wrote:

Clarifying a few points; removing the reference to 'setuptools' because
it referenced anywhere else in this document and doesn't really provide
any useful information to a Python newcomer.

Adjusting the language elsewhere to be less ambiguous and have fewer
run-on sentences.

Signed-off-by: John Snow 
---
  python/README.rst | 17 +
  1 file changed, 9 insertions(+), 8 deletions(-)

Reviewed-by: Wainer dos Santos Moschetta 


diff --git a/python/README.rst b/python/README.rst
index dcf993819d..107786ffdc 100644
--- a/python/README.rst
+++ b/python/README.rst
@@ -7,8 +7,7 @@ then by package (e.g. ``qemu/machine``, ``qemu/qmp``, etc).
  
  ``setup.py`` is used by ``pip`` to install this tooling to the current

  environment. ``setup.cfg`` provides the packaging configuration used by
-``setup.py`` in a setuptools specific format. You will generally invoke
-it by doing one of the following:
+``setup.py``. You will generally invoke it by doing one of the following:
  
  1. ``pip3 install .`` will install these packages to your current

 environment. If you are inside a virtual environment, they will
@@ -17,12 +16,13 @@ it by doing one of the following:
  
  2. ``pip3 install --user .`` will install these packages to your user's

 local python packages. If you are inside of a virtual environment,
-   this will fail; you likely want the first invocation above.
+   this will fail; you want the first invocation above.
  
-If you append the ``-e`` argument, pip will install in "editable" mode;

-which installs a version of the package that installs a forwarder
-pointing to these files, such that the package always reflects the
-latest version in your git tree.
+If you append the ``--editable`` or ``-e`` argument to either invocation
+above, pip will install in "editable" mode. This installs the package as
+a forwarder ("qemu.egg-link") that points to the source tree. In so
+doing, the installed package always reflects the latest version in your
+source tree.
  
  Installing ".[devel]" instead of "." will additionally pull in required

  packages for testing this package. They are not runtime requirements,
@@ -30,6 +30,7 @@ and are not needed to simply use these libraries.
  
  Running ``make develop`` will pull in all testing dependencies and

  install QEMU in editable mode to the current environment.
+(It is a shortcut for ``pip3 install -e .[devel]``.)
  
  See `Installing packages using pip and virtual environments

  
<https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/>`_
@@ -39,7 +40,7 @@ for more information.
  Files in this directory
  ---
  
-- ``qemu/`` Python package source directory.

+- ``qemu/`` Python 'qemu' namespace package source directory.
  - ``tests/`` Python package tests directory.
  - ``avocado.cfg`` Configuration for the Avocado test-runner.
Used by ``make check`` et al.

Re: [PATCH 03/11] python: Re-lock pipenv at oldest supported versions

2021-06-28 Thread Wainer dos Santos Moschetta




On 6/25/21 12:45 PM, John Snow wrote:

tox is already testing the most recent versions. Let's use pipenv to
test the oldest versions we claim to support. This matches the stylistic
choice to have pipenv always test our oldest supported Python version, 3.6.
Currently tox is testing with 3.6 too. Should we remove that version 
from the test matrix?


The effect of this is that the python-check-pipenv CI job on gitlab will
now test against much older versions of these linters, which will help
highlight incompatible changes that might otherwise go unnoticed.

Update instructions for adding and bumping versions in setup.cfg. The
reason for deleting the line that gets added to Pipfile is largely just
to avoid having the version minimums specified in multiple places in
config checked into the tree.

(This patch was written by deleting Pipfile and Pipfile.lock, then
explicitly installing each dependency manually at a specific
version. Then, I restored the prior Pipfile and re-ran `pipenv lock
--dev --keep-outdated` to re-add the qemu dependency back to the pipenv
environment while keeping the "old" packages. It's annoying, yes, but I
think the improvement to test coverage is worthwhile.)

Signed-off-by: John Snow 
---
  python/Pipfile.lock | 113 +---
  python/setup.cfg|   4 +-
  2 files changed, 56 insertions(+), 61 deletions(-)


Reviewed-by: Wainer dos Santos Moschetta 



diff --git a/python/Pipfile.lock b/python/Pipfile.lock
index 5bb3f1b635..8ab41a3f60 100644
--- a/python/Pipfile.lock
+++ b/python/Pipfile.lock
@@ -31,19 +31,19 @@
  },
  "astroid": {
  "hashes": [
-
"sha256:4db03ab5fc3340cf619dbc25e42c2cc3755154ce6009469766d7143d1fc2ee4e",
-
"sha256:8a398dfce302c13f14bab13e2b14fe385d32b73f4e4853b9bdfb64598baa1975"
+
"sha256:09bdb456e02564731f8b5957cdd0c98a7f01d2db5e90eb1d794c353c28bfd705",
+
"sha256:6a8a51f64dae307f6e0c9db752b66a7951e282389d8362cc1d39a56f3feeb31d"
  ],
  "markers": "python_version ~= '3.6'",
-"version": "==2.5.6"
+"version": "==2.6.0"
  },
  "avocado-framework": {
  "hashes": [
-
"sha256:42aa7962df98d6b78d4efd9afa2177226dc630f3d83a2a7d5baf7a0a7da7fa1b",
-
"sha256:d96ae343abf890e1ef3b3a6af5ce49e35f6bded0715770c4acb325bca555c515"
+
"sha256:3fca7226d7d164f124af8a741e7fa658ff4345a0738ddc32907631fd688b38ed",
+
"sha256:48ac254c0ae2ef0c0ceeb38e3d3df0388718eda8f48b3ab55b30b252839f42b1"
  ],
-"markers": "python_version >= '3.6'",
-"version": "==88.1"
+"index": "pypi",
+"version": "==87.0"
  },
  "distlib": {
  "hashes": [
@@ -61,25 +61,27 @@
  },
  "flake8": {
  "hashes": [
-
"sha256:07528381786f2a6237b061f6e96610a4167b226cb926e2aa2b6b1d78057c576b",
-
"sha256:bf8fd46d844f616e8d47905ef3a3384edae6b4e9beb0c5101e25e3110907"
+
"sha256:6a35f5b8761f45c5513e3405f110a86bea57982c3b75b766ce7b65217abe1670",
+
"sha256:c01f8a3963b3571a8e6bd7a4063359aff90749e160778e03817cd9b71c9e07d2"
  ],
-"markers": "python_version >= '2.7' and python_version not in '3.0, 
3.1, 3.2, 3.3, 3.4'",
-"version": "==3.9.2"
+"index": "pypi",
+"version": "==3.6.0"
  },
  "fusepy": {
  "hashes": [
-
"sha256:72ff783ec2f43de3ab394e3f7457605bf04c8cf288a2f4068b4cde141d4ee6bd"
+
"sha256:10f5c7f5414241bffecdc333c4d3a725f1d6605cae6b4eaf86a838ff49cdaf6c",
+
"sha256:a9f3a3699080ddcf0919fd1eb2cf743e1f5859ca54c2018632f939bdfac269ee"
  ],
-"version": "==3.0.1"
+"index": "pypi",
+"version": "==2.0.4"
  },
  "importlib-metadata": {
  "hashes": [
-
"sha256:8c501196e49fb9df5df43833bdb1e4328f64847763ec8a50703148b73784d581",
-
"sha256:d7eb1dea6d6a6086f8be21784cc9e3bcfa55872b52309bc5fad53a8ea65d"
+
"sha256:90bb658cdbbf6d1735b6341ce708fc7024a3e14e99ffdc5783edea9f9b077f83",
+
"sha256:dc15b2969b4ce36305c51eebe62d418ac7791e9a157911d58bfb1f9ccd

Re: [PATCH 02/11] python: Remove global pylint suppressions

2021-06-28 Thread Wainer dos Santos Moschetta




On 6/25/21 12:45 PM, John Snow wrote:

These suppressions only apply to a small handful of places. Instead of
disabling them globally, disable them just in the cases where we
need. The design of the machine class grew quite organically with tons
of constructor and class instance variables -- there's little chance of
meaningfully refactoring it in the near term, so just suppress the
warnings for that class.

Signed-off-by: John Snow 
---
  python/qemu/machine/machine.py | 3 +++
  python/qemu/machine/qtest.py   | 2 ++
  python/setup.cfg   | 4 +---
  3 files changed, 6 insertions(+), 3 deletions(-)

Reviewed-by: Wainer dos Santos Moschetta 


diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py
index b62435528e..e3345dfa1b 100644
--- a/python/qemu/machine/machine.py
+++ b/python/qemu/machine/machine.py
@@ -84,6 +84,7 @@ class QEMUMachine:
  ...
  # vm is guaranteed to be shut down here
  """
+# pylint: disable=too-many-instance-attributes, too-many-public-methods
  
  def __init__(self,

   binary: str,
@@ -111,6 +112,8 @@ def __init__(self,
  @param console_log: (optional) path to console log file
  @note: Qemu process is not started until launch() is used.
  '''
+# pylint: disable=too-many-arguments
+
  # Direct user configuration
  
  self._binary = binary

diff --git a/python/qemu/machine/qtest.py b/python/qemu/machine/qtest.py
index 93700684d1..d6d9c6a34a 100644
--- a/python/qemu/machine/qtest.py
+++ b/python/qemu/machine/qtest.py
@@ -116,6 +116,8 @@ def __init__(self,
   base_temp_dir: str = "/var/tmp",
   socket_scm_helper: Optional[str] = None,
   sock_dir: Optional[str] = None):
+# pylint: disable=too-many-arguments
+
  if name is None:
  name = "qemu-%d" % os.getpid()
  if sock_dir is None:
diff --git a/python/setup.cfg b/python/setup.cfg
index db1639c1f2..524789d6e0 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -87,9 +87,7 @@ ignore_missing_imports = True
  # --enable=similarities". If you want to run only the classes checker, but 
have
  # no Warning level messages displayed, use "--disable=all --enable=classes
  # --disable=W".
-disable=too-many-arguments,
-too-many-instance-attributes,
-too-many-public-methods,
+disable=
  
  [pylint.basic]

  # Good variable names which should always be accepted, separated by a comma.

Re: [PATCH 01/11] python: expose typing information via PEP 561

2021-06-28 Thread Wainer dos Santos Moschetta


Hi,

On 6/25/21 12:45 PM, John Snow wrote:

https://www.python.org/dev/peps/pep-0561/#specification
Create 'py.typed' files in each subpackage that indicate to mypy that
this is a typed module, so that users of any of these packages can use
mypy to check their code as well.

Note: Theoretically it's possible to ditch MANIFEST.in in favor of using
package_data in setup.cfg, but I genuinely could not figure out how to
get it to include things from the *source root* into the *package root*;
only how to include things from each subpackage. I tried!

Signed-off-by: John Snow 
---
  python/qemu/machine/py.typed | 0
  python/qemu/qmp/py.typed | 0
  python/qemu/utils/py.typed   | 0
  python/setup.cfg | 4 
  4 files changed, 4 insertions(+)
  create mode 100644 python/qemu/machine/py.typed
  create mode 100644 python/qemu/qmp/py.typed
  create mode 100644 python/qemu/utils/py.typed


Just double-checking that you did not add a single py.typed on the root 
"python" dir, so that all modules inherit from it, because this is a 
namespace package. Right?


Anyway,

Reviewed-by: Wainer dos Santos Moschetta 



diff --git a/python/qemu/machine/py.typed b/python/qemu/machine/py.typed
new file mode 100644
index 00..e69de29bb2
diff --git a/python/qemu/qmp/py.typed b/python/qemu/qmp/py.typed
new file mode 100644
index 00..e69de29bb2
diff --git a/python/qemu/utils/py.typed b/python/qemu/utils/py.typed
new file mode 100644
index 00..e69de29bb2
diff --git a/python/setup.cfg b/python/setup.cfg
index 85cecbb41b..db1639c1f2 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -19,6 +19,7 @@ classifiers =
  Programming Language :: Python :: 3.8
  Programming Language :: Python :: 3.9
  Programming Language :: Python :: 3.10
+Typing :: Typed
  
  [options]

  python_requires = >= 3.6
@@ -27,6 +28,9 @@ packages =
  qemu.machine
  qemu.utils
  
+[options.package_data]

+* = py.typed
+
  [options.extras_require]
  # Run `pipenv lock --dev` when changing these requirements.
  devel =

Re: [PATCH 5/5] tests/acceptance: Test PMON on the Fuloong 2E machine

2021-06-28 Thread Wainer dos Santos Moschetta


Hi,

On 6/24/21 5:27 PM, Philippe Mathieu-Daudé wrote:

Test the PMON firmware. As the firmware is not redistributable,
it has to be downloaded manually first. Then it can be used by
providing its path via the PMON_BIN_PATH environment variable:

In the code it is used PMON2E_BIN_PATH.


   $ PMON2E_BIN_PATH=~/images/fuloong2e/pmon_2e.bin \
 AVOCADO_ALLOW_UNTRUSTED_CODE=1 \
 avocado --show=app,console run tests/acceptance/machine_mips_fuloong2e.py
   Fetching asset from 
tests/acceptance/machine_mips_fuloong2e.py:MipsFuloong2e.test_linux_kernel_isa_serial
(1/3) 
tests/acceptance/machine_mips_fuloong2e.py:MipsFuloong2e.test_pmon_serial_console:
   console: PMON2000 MIPS Initializing. Standby...
   console: ERRORPC= CONFIG=00030932
   console: PRID=6302
   console: Init SDRAM Done!
   console: Sizing caches...
   console: Init caches...
   console: godson2 caches found
   console: Init caches done, cfg = 00030932
   console: Copy PMON to execute location...
   console: copy text section done.
   console: Copy PMON to execute location done.
   Uncompressing BiosOK,Booting Bios
   PASS (0.25 s)
(2/3) 
tests/acceptance/machine_mips_fuloong2e.py:MipsFuloong2e.test_pmon_framebuffer_console:
   [...]
   Uncompressing BiosOK,Booting Bios
   console: FREQ
   console: FREI
   console: DONE
   console: TTYI
   console: TTYD
   console: ENVI
   console: MAPV
   console: Mfg  0, Id 60
   console: STDV
   console: SBDD
   console: PPCIH
   console: PCIS
   console: PCIR
   console: PCIW
   console: NETI
   console: RTCL
   console: PCID
   console: VGAI
   console: Default MODE_ID 2
   console: starting radeon init...
   console: radeon init done
   console: FRBI
   console: cfb_console init,fb=b400
   console: Video: Drawing the logo ...
   console: CONSOLE_SIZE 450560HSTI
   PASS (4.10 s)
(3/3) 
tests/acceptance/machine_mips_fuloong2e.py:MipsFuloong2e.test_linux_kernel_isa_serial:
   console: Linux version 2.6.27.7lemote (root@debian) (gcc version 4.1.3 
20080623 (prerelease) (Debian 4.1.2-23)) #6 Fri Dec 12 00:11:25 CST 2008
   console: busclock=3300, cpuclock=-2145008360,memsize=256,highmemsize=0
   console: console [early0] enabled
   console: CPU revision is: 6302 (ICT Loongson-2)
   PASS (0.19 s)
   RESULTS: PASS 3 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | 
CANCEL 0
   JOB TIME   : 5.10 s

Signed-off-by: Philippe Mathieu-Daudé 
---
  tests/acceptance/machine_mips_fuloong2e.py | 62 ++
  1 file changed, 62 insertions(+)


Need to fix the commit message (or the code) as pointed above. With that,

Reviewed-by: Wainer dos Santos Moschetta 



diff --git a/tests/acceptance/machine_mips_fuloong2e.py 
b/tests/acceptance/machine_mips_fuloong2e.py
index 0ac285e2af1..4854ba98560 100644
--- a/tests/acceptance/machine_mips_fuloong2e.py
+++ b/tests/acceptance/machine_mips_fuloong2e.py
@@ -8,15 +8,77 @@
  # SPDX-License-Identifier: GPL-2.0-or-later
  
  import os

+import time
  
  from avocado import skipUnless

  from avocado_qemu import Test
  from avocado_qemu import wait_for_console_pattern
  
+from tesseract_utils import tesseract_available, tesseract_ocr

+
  class MipsFuloong2e(Test):
  
  timeout = 60
  
+@skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')

+@skipUnless(os.getenv('PMON2E_BIN_PATH'), 'PMON2E_BIN_PATH not available')
+def test_pmon_serial_console(self):
+"""
+:avocado: tags=arch:mips64el
+:avocado: tags=machine:fuloong2e
+:avocado: tags=endian:little
+:avocado: tags=device:bonito64
+:avocado: tags=device:via686b
+"""
+pmon_hash = 'c812e1695d7b2320036f3ef494976969' # v1.1.2
+pmon_path = self.fetch_asset('file://' + os.getenv('PMON2E_BIN_PATH'),
+ asset_hash=pmon_hash, algorithm='md5')
+
+self.vm.set_console()
+self.vm.add_args('-bios', pmon_path)
+self.vm.launch()
+wait_for_console_pattern(self, 'PMON2000 MIPS Initializing. 
Standby...')
+wait_for_console_pattern(self, 'Booting Bios')
+
+@skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
+# Tesseract 4 adds a new OCR engine based on LSTM neural networks. The
+# new version is faster and more accurate than version 3. The drawback is
+# that it is still alpha-level software.
+@skipUnless(tesseract_available(4), 'tesseract v4 OCR tool not available')
+@skipUnless(os.getenv('PMON2E_BIN_PATH'), 'PMON2E_BIN_PATH not available')
+def test_pmon_framebuffer_console(self):
+"""
+:avocado: tags=arch:mips64el
+:avocado: tags=machine:fuloong2e
+:avocado: tags=endian:little
+:avocado: tags=device:bonito64
+:avocado: tags=device:ati-vga
+"""
+screenshot_path =

Re: [PATCH 4/5] tests/acceptance: Test Linux on the Fuloong 2E machine

2021-06-28 Thread Wainer dos Santos Moschetta




On 6/24/21 5:27 PM, Philippe Mathieu-Daudé wrote:

Test the kernel from Lemote rescue image:
http://dev.lemote.com/files/resource/download/rescue/rescue-yl
Once downloaded, set the RESCUE_YL_PATH environment variable
to point to the downloaded image and test as:

   $ RESCUE_YL_PATH=~/images/fuloong2e/rescue-yl \
 AVOCADO_ALLOW_UNTRUSTED_CODE=1 \
 avocado --show=app,console run tests/acceptance/machine_mips_fuloong2e.py
   Fetching asset from 
tests/acceptance/machine_mips_fuloong2e.py:MipsFuloong2e.test_linux_kernel_isa_serial
(1/1) 
tests/acceptance/machine_mips_fuloong2e.py:MipsFuloong2e.test_linux_kernel_isa_serial:
   console: Linux version 2.6.27.7lemote (root@debian) (gcc version 4.1.3 
20080623 (prerelease) (Debian 4.1.2-23)) #6 Fri Dec 12 00:11:25 CST 2008
   console: busclock=3300, cpuclock=-2145008360,memsize=256,highmemsize=0
   console: console [early0] enabled
   console: CPU revision is: 6302 (ICT Loongson-2)
   PASS (0.16 s)
   JOB TIME   : 0.51 s

Signed-off-by: Philippe Mathieu-Daudé 
---
  MAINTAINERS|  1 +
  tests/acceptance/machine_mips_fuloong2e.py | 42 ++
  2 files changed, 43 insertions(+)
  create mode 100644 tests/acceptance/machine_mips_fuloong2e.py


Reviewed-by: Wainer dos Santos Moschetta 



diff --git a/MAINTAINERS b/MAINTAINERS
index 1a041eaf864..1c515b4ba14 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1176,6 +1176,7 @@ F: hw/isa/vt82c686.c
  F: hw/pci-host/bonito.c
  F: hw/usb/vt82c686-uhci-pci.c
  F: include/hw/isa/vt82c686.h
+F: tests/acceptance/machine_mips_fuloong2e.py
  
  Loongson-3 virtual platforms

  M: Huacai Chen 
diff --git a/tests/acceptance/machine_mips_fuloong2e.py 
b/tests/acceptance/machine_mips_fuloong2e.py
new file mode 100644
index 000..0ac285e2af1
--- /dev/null
+++ b/tests/acceptance/machine_mips_fuloong2e.py
@@ -0,0 +1,42 @@
+# Functional tests for the Lemote Fuloong-2E machine.
+#
+# Copyright (c) 2019 Philippe Mathieu-Daudé 
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+import os
+
+from avocado import skipUnless
+from avocado_qemu import Test
+from avocado_qemu import wait_for_console_pattern
+
+class MipsFuloong2e(Test):
+
+timeout = 60
+
+@skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
+@skipUnless(os.getenv('RESCUE_YL_PATH'), 'RESCUE_YL_PATH not available')
+def test_linux_kernel_isa_serial(self):
+"""
+:avocado: tags=arch:mips64el
+:avocado: tags=machine:fuloong2e
+:avocado: tags=endian:little
+:avocado: tags=device:bonito64
+:avocado: tags=device:via686b
+"""
+# Recovery system for the Yeeloong laptop
+# (enough to test the fuloong2e southbridge, accessing its ISA bus)
+# http://dev.lemote.com/files/resource/download/rescue/rescue-yl
+kernel_hash = 'ec4d1bd89a8439c41033ca63db60160cc6d6f09a'
+kernel_path = self.fetch_asset('file://' + os.getenv('RESCUE_YL_PATH'),
+   asset_hash=kernel_hash)
+
+self.vm.set_console()
+self.vm.add_args('-kernel', kernel_path)
+self.vm.launch()
+wait_for_console_pattern(self, 'Linux version 2.6.27.7lemote')
+cpu_revision = 'CPU revision is: 6302 (ICT Loongson-2)'
+wait_for_console_pattern(self, cpu_revision)

Re: [PATCH v3 2/3] avocado_qemu: Add SMMUv3 tests

2021-06-28 Thread Wainer dos Santos Moschetta


Hi,

On 6/21/21 5:08 AM, Eric Auger wrote:

Add new tests checking the good behavior of the SMMUv3 protecting
2 virtio pci devices (block and net). We check the guest boots and
we are able to install a package. Different guest configs are tested:
standard, passthrough an strict=0. This is tested with both fedora 31 and
33. The former uses a 5.3 kernel without range invalidation whereas the
latter uses a 5.8 kernel that features range invalidation.

Signed-off-by: Eric Auger 

---

v1 -> v2:
- removed ssh import
- combined add_command_args() and common_vm_setup()
- moved tags in class' docstring and added tags=arch:aarch64
- use self.get_default_kernel_params()
- added RIL tests with fed33 + introduce new tags
---
  tests/acceptance/smmu.py | 133 +++
  1 file changed, 133 insertions(+)
  create mode 100644 tests/acceptance/smmu.py

diff --git a/tests/acceptance/smmu.py b/tests/acceptance/smmu.py
new file mode 100644
index 00..bcb5416a56
--- /dev/null
+++ b/tests/acceptance/smmu.py
@@ -0,0 +1,133 @@
+# SMMUv3 Functional tests
+#
+# Copyright (c) 2021 Red Hat, Inc.
+#
+# Author:
+#  Eric Auger 
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+
+import os
+
+from avocado_qemu import LinuxTest, BUILD_DIR
+
+class SMMU(LinuxTest):
+"""
+:avocado: tags=accel:kvm
+:avocado: tags=cpu:host
+:avocado: tags=arch:aarch64
+:avocado: tags=smmu
+"""
+
+IOMMU_ADDON = ',iommu_platform=on,disable-modern=off,disable-legacy=on'
+kernel_path = None
+initrd_path = None
+kernel_params = None
+
+def set_up_boot(self):
+path = self.download_boot()
+self.vm.add_args('-device', 'virtio-blk-pci,bus=pcie.0,scsi=off,' +
+ 'drive=drv0,id=virtio-disk0,bootindex=1,'
+ 'werror=stop,rerror=stop' + self.IOMMU_ADDON)
+self.vm.add_args('-drive',
+ 'file=%s,if=none,cache=writethrough,id=drv0' % path)
+
+def setUp(self):
+super(SMMU, self).setUp(None, 'virtio-net-pci' + self.IOMMU_ADDON)
+
+def common_vm_setup(self, custom_kernel=None):
As some tests call `self.common_vm_setup(True)`, I think `custom_kernel` 
should be `True|False`.

+self.require_accelerator("kvm")
+self.vm.add_args("-machine", "virt")
FYI, if you have the "machine" tag (e.g. "tags=machine:virt") to your 
tests then avocado_qemu will set "-machine" automatically to any created VM.

+self.vm.add_args("-accel", "kvm")
+self.vm.add_args("-cpu", "host")
+self.vm.add_args("-smp", "8")
+self.vm.add_args("-m", "4096")


The `avocado_qemu.LinuxTest` provides default values for smp and memory 
which cannot be properly overwritten. The resulting command line will 
have -smp and -m twice.


I created an issue to track that improvement: 
https://gitlab.com/qemu-project/qemu/-/issues/453



+self.vm.add_args("-machine", "iommu=smmuv3")
+self.vm.add_args("-d", "guest_errors")
+self.vm.add_args('-bios', os.path.join(BUILD_DIR, 'pc-bios',
+ 'edk2-aarch64-code.fd'))
+self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0')
+self.vm.add_args('-object',
+ 'rng-random,id=rng0,filename=/dev/urandom')
+
+if custom_kernel is None:
+return
+
+kernel_url = self.get_pxeboot_url() + 'vmlinuz'
+initrd_url = self.get_pxeboot_url() + 'initrd.img'
+self.kernel_path = self.fetch_asset(kernel_url)
+self.initrd_path = self.fetch_asset(initrd_url)
+
+def run_and_check(self):
+if self.kernel_path:
+self.vm.add_args('-kernel', self.kernel_path,
+ '-append', self.kernel_params,
+ '-initrd', self.initrd_path)
+self.launch_and_wait()
+self.ssh_command('cat /proc/cmdline')
+self.ssh_command('dnf -y install numactl-devel')
+
+
+# 5.3 kernel without RIL #
+
+def test_smmu_noril(self):
+"""
+:avocado: tags=smmu_noril
+:avocado: tags=smmu_noril_tests
+:avocado: tags=distro_version:31
+"""
+self.common_vm_setup()
+self.run_and_check()
+
+def test_smmu_noril_passthrough(self):
+"""
+:avocado: tags=smmu_noril_passthrough
+:avocado: tags=smmu_noril_tests
+:avocado: tags=distro_version:31


Maybe you should "distro" tag the tests (or better, tag the class), so 
that a readers of this don't need to browse the `LinuxTest` class 
looking for the distro to be used (although it may be clear to some, 
based on the distro_version)...


Thanks for contributing those tests!

- Wainer


+"""
+self.common_vm_setup(True)
+self.kernel_params = self.get_default_kernel_params() + ' 
iommu.passthrough=on'
+self.run_and_check()
+
+def

Re: [PATCH v3 1/3] Acceptance Tests: Add default kernel params and pxeboot url to the KNOWN_DISTROS collection

2021-06-28 Thread Wainer dos Santos Moschetta


Hi Eric,

On 6/21/21 5:08 AM, Eric Auger wrote:

When running LinuxTests we may need to run the guest with
custom params. It is practical to store the pxeboot URL
and the default kernel params so that the
tests just need to fetch those and augment the kernel params.

Signed-off-by: Eric Auger 

---

v2 -> v3:
- add fed32 and fed33 checksums
---
  tests/acceptance/avocado_qemu/__init__.py | 52 ++-
  1 file changed, 50 insertions(+), 2 deletions(-)

diff --git a/tests/acceptance/avocado_qemu/__init__.py 
b/tests/acceptance/avocado_qemu/__init__.py
index 81ac90bebb..8152420fa5 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -305,17 +305,59 @@ def ssh_command(self, command):
  'fedora': {
  '31': {
  'x86_64':
-{'checksum': 
'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'},
+{'checksum': 
'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0',
+ 'pxeboot_url': 
"https://archives.fedoraproject.org/pub/archive/fedora/;
+
"linux/releases/31/Everything/x86_64/os/images/pxeboot/",
+ 'kernel_params': "root=UUID=b1438b9b-2cab-4065-a99a-08a96687f73c ro 
"
+  "no_timer_check net.ifnames=0 "
+  "console=tty1 console=ttyS0,115200n8"},
  'aarch64':
-{'checksum': 
'1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49'},
+{'checksum': 
'1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49',
+ 'pxeboot_url': 
"https://archives.fedoraproject.org/pub/archive/fedora/;
+
"linux/releases/31/Everything/aarch64/os/images/pxeboot/",
+ 'kernel_params': "root=UUID=b6950a44-9f3c-4076-a9c2-355e8475b0a7 ro 
"
+  "earlyprintk=pl011,0x900 ignore_loglevel "
+  "no_timer_check printk.time=1 rd_NO_PLYMOUTH "
+  "console=ttyAMA0 "},
  'ppc64':
  {'checksum': 
'7c3528b85a3df4b2306e892199a9e1e43f991c506f2cc390dc4efa2026ad2f58'},
  's390x':
  {'checksum': 
'4caaab5a434fd4d1079149a072fdc7891e354f834d355069ca982fdcaf5a122d'},
  }
+,
+'32': {
+'aarch64':
+{ 'kernel_params': "root=UUID=3df75b65-be8d-4db4-8655-14d95c0e90c5 ro 
"
+  "no_timer_check net.ifnames=0 console=tty1 "
+  "console=ttyS0,115200n8 ",

Nit: indentation ^

+  'checksum': 
'b367755c664a2d7a26955bbfff985855adfa2ca15e908baf15b4b176d68d3967',
+  'pxeboot_url':  
"https://ftp.lip6.fr/ftp/pub/linux/distributions/fedora/releases/;

Maybe use dl.fedoraproject.org instead of a mirror server?

+  "32/Server/aarch64/os/images/pxeboot/"},
+}
+,
+'33': {
+'aarch64':
+{ 'kernel_params': "root=UUID=d20b3ffa-6397-4a63-a734-1126a0208f8a ro 
"
+  "no_timer_check net.ifnames=0 console=tty1 "
+  "console=ttyS0,115200n8 console=tty0 ",
+  'checksum': 
'e7f75cdfd523fe5ac2ca9eeece68edc1a81f386a17f969c1d1c7c87031008a6b',
+  'pxeboot_url':  
"https://ftp.lip6.fr/ftp/pub/linux/distributions/fedora/releases/;

Likewise.

+  "33/Server/aarch64/os/images/pxeboot/"},
+}
  }
  }
  
+def get_known_distro_kernel_params(distro, distro_version, arch):

+try:
+return 
KNOWN_DISTROS.get(distro).get(distro_version).get(arch).get('kernel_params')
+except AttributeError:
+return None
+
+def get_known_distro_pxeboot_url(distro, distro_version, arch):
+try:
+return 
KNOWN_DISTROS.get(distro).get(distro_version).get(arch).get('pxeboot_url')
+except AttributeError:
+return None
  
  def get_known_distro_checksum(distro, distro_version, arch):

  try:
@@ -449,6 +491,12 @@ def set_up_cloudinit(self, ssh_pubkey=None):
  cloudinit_iso = self.prepare_cloudinit(ssh_pubkey)
  self.vm.add_args('-drive', 'file=%s,format=raw' % cloudinit_iso)
  
+def get_default_kernel_params(self):

+return get_known_distro_kernel_params(self.distro, 
self.distro_version, self.arch)
+
+def get_pxeboot_url(self):
+return get_known_distro_pxeboot_url(self.distro, self.distro_version, 
self.arch)
+


As the KNOWN_DISTROS grows, more loosely methods will be created in the 
avocado_qemu/__init__.py file.


I refactored the code so that KNOWN_DISTROS and related methods are 
packaged in a class. See in:


https://github.com/wainersm/qemu/tree/eauger_avocado-qemu-v3-refactor

Maybe you could incorporate that in your series? Otherwise I can send as 
a follow up series. Whatever you prefer.


Thanks!

- Wainer


  def

Re: [PATCH] tests/vm/freebsd: Increase code coverage

2021-06-15 Thread Wainer dos Santos Moschetta


Hi,

On 6/15/21 2:19 PM, Daniel P. Berrangé wrote:

On Tue, Jun 15, 2021 at 06:02:02PM +0100, Daniel P. Berrangé wrote:

On Mon, May 31, 2021 at 05:53:25PM -0300, Wainer dos Santos Moschetta wrote:

Hi,

On 5/31/21 7:03 AM, Philippe Mathieu-Daudé wrote:

Install more dependencies to increase code coverage.

Signed-off-by: Philippe Mathieu-Daudé 
---
   tests/vm/freebsd | 5 +
   1 file changed, 5 insertions(+)

With or without this patch I got an error when `make vm-build-freebsd`. It
fails to install packages.

For example, with this patch I got:

< Output omitted>

### Installing packages ...
Failed to prepare guest environment
Traceback (most recent call last):
   File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 634, in main
     return vm.build_image(args.image)
   File "/home/wmoschet/src/qemu/tests/vm/freebsd", line 206, in build_image
     self.ssh_root_check("pkg install -y %s\n" % " ".join(self.pkgs))
   File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 255, in
ssh_root_check
     self._ssh_do(self._config["root_user"], cmd, True)
   File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 242, in _ssh_do
     raise Exception("SSH command failed: %s" % cmd)
Exception: SSH command failed: pkg install -y git pkgconf bzip2 python37
ninja bash gmake gsed gettext cyrus-sasl gnutls nettle jpeg-turbo png sdl2
gtk3 libxkbcommon pixman libepoxy mesa-libs zstd usbredir

Is it a known issue?

Hard to actually tell what the error really is. This message is
only giving the command that was invoked, but seems to have thrown
away stdout/stderr which would have the messages telling us what
went wrong.  This lack of error reporting in basevm.py so badly
needs to be fixed, otherwise we're working blind when debugging
failures.

Hmm, when I try the same command, it *does* print all the output
so you can see what's going on, but it doesn't actually fail for
me either (without this patch).


Thomas sent me a message in private while ago that the error could be 
related with the amount of ssh keys in my agent-ssh. I didn' t check 
that hypothesis yet; will debug a littler further and let you all posted.


ah, if this problem cannot be reproduced on other machinesI see no 
reason to hold this patch.


- Wainer



Regards,
Daniel

Re: [PATCH v3 0/7] tests/acceptance: Handle tests with "cpu" tag

2021-06-11 Thread Wainer dos Santos Moschetta


Ping.

Only patches 06 and 07 did not get any review.

The series touches many files and it was last rebased months ago, so 
likely I will need to resolve rebase conflicts. But I would like to have 
the reviews of those patches before.


Thanks!

- Wainer

On 4/30/21 10:34 AM, Wainer dos Santos Moschetta wrote:

Currently the acceptance tests tagged with "machine" have the "-M TYPE"
automatically added to the list of arguments of the QEMUMachine object.
In other words, that option is passed to the launched QEMU. On this
series it is implemented the same feature but instead for tests marked
with "cpu".

There is a caveat, however, in case the test needs additional arguments to
the CPU type they cannot be passed via tag, because the tags parser split
values by comma (limitation which Avocado plans to address, see
https://github.com/avocado-framework/avocado/issues/45410). For example, in
tests/acceptance/x86_cpu_model_versions.py, there are cases where:

   * -cpu is set to 
"Cascadelake-Server,x-force-features=on,check=off,enforce=off"
   * if it was tagged like 
"cpu:Cascadelake-Server,x-force-features=on,check=off,enforce=off"
 then the parser would break it into 4 tags ("cpu:Cascadelake-Server",
 "x-force-features=on", "check=off", "enforce=off")
   * resulting on "-cpu Cascadelake-Server" and the remaining arguments are 
ignored.

It was introduced the avocado_qemu.Test.set_vm_arg() method to deal with
cases like the example above, so that one can tag it as "cpu:Cascadelake-Server"
AND call self.set_vm_args('-cpu', 
"Cascadelake-Server,x-force-features=on,check=off,enforce=off"),
and that results on the reset of the initial value of -cpu.

This series was tested on CI 
(https://gitlab.com/wainersm/qemu/-/pipelines/294640198)
and with the following code:

from avocado_qemu import Test

class CPUTest(Test):
 def test_cpu(self):
 """
 :avocado: tags=cpu:host
 """
 # The cpu property is set to the tag value, or None on its absence
 self.assertEqual(self.cpu, "host")
 # The created VM has the '-cpu host' option
 self.assertIn("-cpu host", " ".join(self.vm._args))
 self.vm.launch()

 def test_cpu_none(self):
 self.assertEqual(self.cpu, None)
 self.assertNotIn('-cpu', self.vm._args)

 def test_cpu_reset(self):
 """
 :avocado: tags=cpu:host
 """
 self.assertIn("-cpu host", " ".join(self.vm._args))
 self.set_vm_arg("-cpu", "Cascadelake-Server,x-force-features=on")
 self.assertNotIn("-cpu host", " ".join(self.vm._args))
 self.assertIn("-cpu Cascadelake-Server,x-force-features=on", " 
".join(self.vm._args))

Changes:
  - v2 -> v3:
- The arg and value parameters of set_vm_arg() are now mandatories and
  fixed an index out of bounds bug [crosa]
- Rebased. Needed to adapt the (new) boot_xen.py test (patch 03)
  - v1 -> v2:
- Recognize the cpu value passed via test parameter [crosa]
- Fixed tags (patch 02) on preparation to patch 03 [crosa]
- Added QEMUMachine.args property (patch 04) so that _args could be handled
  without pylint complaining (protected property)
- Added Test.set_vm_arg() (patch 05) to handle the corner case [crosa]

Wainer dos Santos Moschetta (7):
   tests/acceptance: Automatic set -cpu to the test vm
   tests/acceptance: Fix mismatch on cpu tagged tests
   tests/acceptance: Let the framework handle "cpu:VALUE" tagged tests
   tests/acceptance: Tagging tests with "cpu:VALUE"
   python/qemu: Add args property to the QEMUMachine class
   tests/acceptance: Add set_vm_arg() to the Test class
   tests/acceptance: Handle cpu tag on x86_cpu_model_versions tests

  docs/devel/testing.rst | 17 +
  python/qemu/machine.py |  5 +++
  tests/acceptance/avocado_qemu/__init__.py  | 26 ++
  tests/acceptance/boot_linux.py |  3 --
  tests/acceptance/boot_linux_console.py | 16 +
  tests/acceptance/boot_xen.py   |  1 -
  tests/acceptance/machine_mips_malta.py |  7 ++--
  tests/acceptance/pc_cpu_hotplug_props.py   |  2 +-
  tests/acceptance/replay_kernel.py  | 17 -
  tests/acceptance/reverse_debugging.py  |  2 +-
  tests/acceptance/tcg_plugins.py| 15 
  tests/acceptance/virtio-gpu.py |  4 +--
  tests/acceptance/x86_cpu_model_versions.py | 40 +-
  13 files changed, 112 insertions(+), 43 deletions(-)

Re: [PATCH 2/4] Python QEMU utils: introduce a generic feature list

2021-06-10 Thread Wainer dos Santos Moschetta


Hi,

On 6/8/21 8:55 PM, Cleber Rosa Junior wrote:



On Tue, Jun 8, 2021 at 5:42 PM Wainer dos Santos Moschetta 
mailto:waine...@redhat.com>> wrote:


Hi,

On 6/8/21 11:09 AM, Cleber Rosa wrote:
> Which can be used to check for any "feature" that is available as a
> QEMU command line option, and that will return its list of available
> options.
>
> This is a generalization of the list_accel() utility function, which
> is itself re-implemented in terms of the more generic feature.
>
> Signed-off-by: Cleber Rosa mailto:cr...@redhat.com>>
> ---
>   python/qemu/utils/__init__.py |  2 ++
>   python/qemu/utils/accel.py    | 15 ++--
>   python/qemu/utils/feature.py  | 44
+++
>   3 files changed, 48 insertions(+), 13 deletions(-)
>   create mode 100644 python/qemu/utils/feature.py
>
> diff --git a/python/qemu/utils/__init__.py
b/python/qemu/utils/__init__.py
> index 7f1a5138c4..1d0789eaa2 100644
> --- a/python/qemu/utils/__init__.py
> +++ b/python/qemu/utils/__init__.py
> @@ -20,12 +20,14 @@
>
>   # pylint: disable=import-error
>   from .accel import kvm_available, list_accel, tcg_available
> +from .feature import list_feature
>
>
>   __all__ = (
>       'get_info_usernet_hostfwd_port',
>       'kvm_available',
>       'list_accel',
> +    'list_feature',
>       'tcg_available',
>   )
>
> diff --git a/python/qemu/utils/accel.py b/python/qemu/utils/accel.py
> index 297933df2a..b5bb80c6d3 100644
> --- a/python/qemu/utils/accel.py
> +++ b/python/qemu/utils/accel.py
> @@ -14,13 +14,11 @@
>   # the COPYING file in the top-level directory.
>   #
>
> -import logging
>   import os
> -import subprocess
>   from typing import List, Optional
>
> +from qemu.utils.feature import list_feature
>
> -LOG = logging.getLogger(__name__)
>
>   # Mapping host architecture to any additional architectures it can
>   # support which often includes its 32 bit cousin.
> @@ -39,16 +37,7 @@ def list_accel(qemu_bin: str) -> List[str]:
>       @raise Exception: if failed to run `qemu -accel help`
>       @return a list of accelerator names.
>       """
> -    if not qemu_bin:
> -        return []
> -    try:
> -        out = subprocess.check_output([qemu_bin, '-accel', 'help'],
> - universal_newlines=True)
> -    except:
> -        LOG.debug("Failed to get the list of accelerators in
%s", qemu_bin)
> -        raise
> -    # Skip the first line which is the header.
> -    return [acc.strip() for acc in out.splitlines()[1:]]
> +    return list_feature(qemu_bin, 'accel')
>
>
>   def kvm_available(target_arch: Optional[str] = None,
> diff --git a/python/qemu/utils/feature.py
b/python/qemu/utils/feature.py
> new file mode 100644
> index 00..b4a5f929ab
> --- /dev/null
> +++ b/python/qemu/utils/feature.py
> @@ -0,0 +1,44 @@
> +"""
> +QEMU feature module:
> +
> +This module provides a utility for discovering the availability of
> +generic features.
> +"""
> +# Copyright (C) 2022 Red Hat Inc.
Cleber, please, tell me how is the future like! :)


I grabbed a sports almanac.  That's all I can say. :)

Now seriously, thanks for spotting the typo.

> +#
> +# Authors:
> +#  Cleber Rosa mailto:cr...@redhat.com>>
> +#
> +# This work is licensed under the terms of the GNU GPL, version
2.  See
> +# the COPYING file in the top-level directory.
> +#
> +
> +import logging
> +import subprocess
> +from typing import List
> +
> +
> +LOG = logging.getLogger(__name__)
> +
> +
> +def list_feature(qemu_bin: str, feature: str) -> List[str]:
> +    """
> +    List available options the QEMU binary for a given feature
type.
> +
> +    By calling a "qemu $feature -help" and parsing its output.

I understand we need a mean to easily cancel the test if given
feature
is not present. However, I'm unsure this generic list_feature() is
what
we need.

The `-accel help` returns a simple list of strings (besides the
header,
which is dismissed). Whereas `-machine help` returns what could be
parsed as a tuple of (name, description).

Another example is `-cpu help` which will print a similar list as
`-machine`, plus a se

Re: [PATCH 2/4] Python QEMU utils: introduce a generic feature list

2021-06-08 Thread Wainer dos Santos Moschetta


Hi,

On 6/8/21 11:09 AM, Cleber Rosa wrote:

Which can be used to check for any "feature" that is available as a
QEMU command line option, and that will return its list of available
options.

This is a generalization of the list_accel() utility function, which
is itself re-implemented in terms of the more generic feature.

Signed-off-by: Cleber Rosa 
---
  python/qemu/utils/__init__.py |  2 ++
  python/qemu/utils/accel.py| 15 ++--
  python/qemu/utils/feature.py  | 44 +++
  3 files changed, 48 insertions(+), 13 deletions(-)
  create mode 100644 python/qemu/utils/feature.py

diff --git a/python/qemu/utils/__init__.py b/python/qemu/utils/__init__.py
index 7f1a5138c4..1d0789eaa2 100644
--- a/python/qemu/utils/__init__.py
+++ b/python/qemu/utils/__init__.py
@@ -20,12 +20,14 @@
  
  # pylint: disable=import-error

  from .accel import kvm_available, list_accel, tcg_available
+from .feature import list_feature
  
  
  __all__ = (

  'get_info_usernet_hostfwd_port',
  'kvm_available',
  'list_accel',
+'list_feature',
  'tcg_available',
  )
  
diff --git a/python/qemu/utils/accel.py b/python/qemu/utils/accel.py

index 297933df2a..b5bb80c6d3 100644
--- a/python/qemu/utils/accel.py
+++ b/python/qemu/utils/accel.py
@@ -14,13 +14,11 @@
  # the COPYING file in the top-level directory.
  #
  
-import logging

  import os
-import subprocess
  from typing import List, Optional
  
+from qemu.utils.feature import list_feature
  
-LOG = logging.getLogger(__name__)
  
  # Mapping host architecture to any additional architectures it can

  # support which often includes its 32 bit cousin.
@@ -39,16 +37,7 @@ def list_accel(qemu_bin: str) -> List[str]:
  @raise Exception: if failed to run `qemu -accel help`
  @return a list of accelerator names.
  """
-if not qemu_bin:
-return []
-try:
-out = subprocess.check_output([qemu_bin, '-accel', 'help'],
-  universal_newlines=True)
-except:
-LOG.debug("Failed to get the list of accelerators in %s", qemu_bin)
-raise
-# Skip the first line which is the header.
-return [acc.strip() for acc in out.splitlines()[1:]]
+return list_feature(qemu_bin, 'accel')
  
  
  def kvm_available(target_arch: Optional[str] = None,

diff --git a/python/qemu/utils/feature.py b/python/qemu/utils/feature.py
new file mode 100644
index 00..b4a5f929ab
--- /dev/null
+++ b/python/qemu/utils/feature.py
@@ -0,0 +1,44 @@
+"""
+QEMU feature module:
+
+This module provides a utility for discovering the availability of
+generic features.
+"""
+# Copyright (C) 2022 Red Hat Inc.

Cleber, please, tell me how is the future like! :)

+#
+# Authors:
+#  Cleber Rosa 
+#
+# This work is licensed under the terms of the GNU GPL, version 2.  See
+# the COPYING file in the top-level directory.
+#
+
+import logging
+import subprocess
+from typing import List
+
+
+LOG = logging.getLogger(__name__)
+
+
+def list_feature(qemu_bin: str, feature: str) -> List[str]:
+"""
+List available options the QEMU binary for a given feature type.
+
+By calling a "qemu $feature -help" and parsing its output.


I understand we need a mean to easily cancel the test if given feature 
is not present. However, I'm unsure this generic list_feature() is what 
we need.


The `-accel help` returns a simple list of strings (besides the header, 
which is dismissed). Whereas `-machine help` returns what could be 
parsed as a tuple of (name, description).


Another example is `-cpu help` which will print a similar list as 
`-machine`, plus a section with CPUID flags.


If confess I still don't have a better idea, although I feel it will 
require a OO design.


Thanks!

- Wainer


+
+@param qemu_bin (str): path to the QEMU binary.
+@param feature (str): feature name, matching the command line option.
+@raise Exception: if failed to run `qemu -feature help`
+@return a list of available options for the given feature.
+"""
+if not qemu_bin:
+return []
+try:
+out = subprocess.check_output([qemu_bin, '-%s' % feature, 'help'],
+  universal_newlines=True)
+except:
+LOG.debug("Failed to get the list of %s(s) in %s", feature, qemu_bin)
+raise
+# Skip the first line which is the header.
+return [item.split(' ', 1)[0] for item in out.splitlines()[1:]]

Re: [PATCH v6 4/4] Jobs based on custom runners: add job definitions for QEMU's machines

2021-06-08 Thread Wainer dos Santos Moschetta

Hi,

On 6/8/21 10:36 AM, Cleber Rosa Junior wrote:

On Tue, Jun 8, 2021 at 2:30 AM Philippe Mathieu-Daudé > wrote:

Hi Alex, Stefan,

On 6/8/21 5:14 AM, Cleber Rosa wrote:
> The QEMU project has two machines (aarch64 and s390x) that can
be used
> for jobs that do build and run tests.

AFAIK there is more hardware available to the project, so I'm
wondering
what happened to the rest, is it a deliberate choice to start small?

Hi Phil,

Yes, this series was deliberately focused on the first two machines 
owned and available to QEMU.

What will happen with the rest, since we are wasting resources?

The plan is to allow all machines (currently available and to-be 
available) to be connected as custom runners. This hopefully gets that 
started.

About "more hardware available to the project", there's one VM from 
fosshost which was made available not long ago, and which I set up 
even more recently, which could be used as a gitlab runner too.  But, 
even though some new hardware resource is available (and wasted?), the 
human resources to maintain them have not been properly determined, so 
I believe it's a good decision to start with the machines that have 
been operational for long, and that already have to the best of my 
knowledge, people maintaining them.

I also see a "Debian unstable mips64el (Debian) @ cipunited.cn 
" registered as a runner, but I don't have extra 
information about it.

Besides that, I'll send another series shortly, that builds upon this 
series, and adds a Red Hat focused job, on a Red Hat managed machine.  
This should be what other entities should be capable of doing and 
allowed to do.

Who has access to what and should do what (setup)? How is this list of
hw managed btw? Should there be some public visibility (i.e. Wiki)?

These are good questions, and I believe Alex can answer them about 
those two machines.  Even though I hooked them up to GitLab, AFAICT he 
is the ultimate admin (maybe Peter too?).

About hardware management, it has been suggested to use either the 
Wiki or a MAINTAINERS entry.  This is still unresolved and feedback 
would be appreciated.  For me to propose a MAINTAINERS entry, say, on 
a v7, I'd need the information on who is managing them.

Is there a document explaining what are the steps to follow for an
entity to donate / sponsor hardware? Where would it be stored, should
this hw be shipped somewhere? What documentation should be
provided for
its system administration?

In case an entity manages hosting and maintenance, can the QEMU
project
share the power bill? Up to which amount?
Similar question if a sysadmin has to be paid.

If the QEMU project can't spend money on CI, what is expected from
resource donators? Simply hardware + power (electricity) + network
traffic? Also sysadmining and monitoring? Do we expect some kind of
reliability on the data stored here or can we assume disposable /
transient runners?
Should donators also provide storage? Do we have a minimum storage
requirement?

Should we provide some guideline explaining any code might be run by
our contributors on these runners and some security measures have to
be taken / considered?

Sponsors usually expect some advertising to thanks them, and often
regular reports on how their resources are used, else they might not
renew their offer. Who should care to keep the relationship with
sponsors?

Where is defined what belong to the project, who is responsible,
who can
request access to this hardware, what resource can be used?

You obviously directed the question towards Alex and Stefan 
(rightfully so), so I won't attempt to answer these ones at this point.

More generically, what is the process for a private / corporate entity
to register a runner to the project? (how did it work for this aarch64
and s390x one?)

The steps are listed on the documentation.  Basically anyone with 
knowledge of the "registration token" can add new machines to GitLab 
as runners.  For the two aarch64 and s390x, it was a matter of 
following the documentation steps which basically involve:

1) providing the hostname(s) in the inventory file
2) provide the "registration token" in the vars.yml file
3) running the playbooks

What else am I missing?

I think you're missing the answers to all your good questions :).

And I understand that are a lot of them (from everyone, including 
myself).  The dilemma here is: should we activate the machines already 
available, and learn in practice, what's missing?  I honestly believe 
we should.

IMHO we should merge the minimum possible to start using the existing 
machines, then address the questions (good questions, btw) raised by 
Philippe as needed.

Thanks!

- Wainer

Thanks,
- Clr.

Thanks,

Phil.

> This introduces those jobs,

Re: [PATCH v6 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook

2021-06-08 Thread Wainer dos Santos Moschetta


Hi,

On 6/8/21 12:14 AM, Cleber Rosa wrote:

To have the jobs dispatched to custom runners, gitlab-runner must
be installed, active as a service and properly configured.  The
variables file and playbook introduced here should help with those
steps.

The playbook introduced here covers the Linux distributions and
has been primarily tested on OS/machines that the QEMU project
has available to act as runners, namely:

  * Ubuntu 20.04 on aarch64
  * Ubuntu 18.04 on s390x

But, it should work on all other Linux distributions.  Earlier
versions were tested on FreeBSD too, so chances of success are
high.

Signed-off-by: Cleber Rosa 
---
  docs/devel/ci.rst  | 57 
  scripts/ci/setup/.gitignore|  1 +
  scripts/ci/setup/gitlab-runner.yml | 61 ++
  scripts/ci/setup/vars.yml.template | 12 ++
  4 files changed, 131 insertions(+)
  create mode 100644 scripts/ci/setup/.gitignore
  create mode 100644 scripts/ci/setup/gitlab-runner.yml
  create mode 100644 scripts/ci/setup/vars.yml.template

diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
index 35c6b5e269..bbd89e54d7 100644
--- a/docs/devel/ci.rst
+++ b/docs/devel/ci.rst
@@ -56,3 +56,60 @@ To run the playbook, execute::
  
cd scripts/ci/setup

ansible-playbook -i inventory build-environment.yml
+
+gitlab-runner setup and registration
+
+
+The gitlab-runner agent needs to be installed on each machine that
+will run jobs.  The association between a machine and a GitLab project
+happens with a registration token.  To find the registration token for
+your repository/project, navigate on GitLab's web UI to:
+
+ * Settings (the gears like icon), then
+ * CI/CD, then
+ * Runners, and click on the "Expand" button, then
+ * Under "Set up a specific Runner manually", look for the value under
+   "Use the following registration token during setup"
+
+Copy the ``scripts/ci/setup/vars.yml.template`` file to
+``scripts/ci/setup/vars.yml``.  Then, set the
+``gitlab_runner_registration_token`` variable to the value obtained
+earlier.
+
+.. note:: gitlab-runner is not available from the standard location
+  for all OS and architectures combinations.  For some systems,
+  a custom build may be necessary.  Some builds are avaiable
+  at https://cleber.fedorapeople.org/gitlab-runner/ and this
+  URI may be used as a value on ``vars.yml``
I think you can remove the information about the gitlab-running being 
not available for some systems.

+
+To run the playbook, execute::
+
+  cd scripts/ci/setup
+  ansible-playbook -i inventory gitlab-runner.yml
+
+Following the registration, it's necessary to configure the runner tags,
+and optionally other configurations on the GitLab UI.  Navigate to:
+
+ * Settings (the gears like icon), then
+ * CI/CD, then
+ * Runners, and click on the "Expand" button, then
+ * "Runners activated for this project", then
+ * Click on the "Edit" icon (next to the "Lock" Icon)
+
+Under tags, add values matching the jobs a runner should run.  For a
+Ubuntu 20.04 aarch64 system, the tags should be set as::
+
+  ubuntu_20.04,aarch64


Also users no longer need manually create the tags.

Remaining of the file looks good to me.


+
+Because the job definition at ``.gitlab-ci.d/custom-runners.yml``
+would contain::
+
+  ubuntu-20.04-aarch64-all:
+   tags:
+   - ubuntu_20.04
+   - aarch64
+
+It's also recommended to:
+
+ * increase the "Maximum job timeout" to something like ``2h``
+ * give it a better Description
diff --git a/scripts/ci/setup/.gitignore b/scripts/ci/setup/.gitignore
new file mode 100644
index 00..f112d05dd0
--- /dev/null
+++ b/scripts/ci/setup/.gitignore
@@ -0,0 +1 @@
+vars.yml
\ No newline at end of file
diff --git a/scripts/ci/setup/gitlab-runner.yml 
b/scripts/ci/setup/gitlab-runner.yml
new file mode 100644
index 00..98dab92bb5
--- /dev/null
+++ b/scripts/ci/setup/gitlab-runner.yml
@@ -0,0 +1,61 @@
+---
+- name: Installation of gitlab-runner
+  hosts: all
+  vars_files:
+- vars.yml
+  tasks:
+- debug:
+msg: 'Checking for a valid GitLab registration token'
+  failed_when: "gitlab_runner_registration_token == 
'PLEASE_PROVIDE_A_VALID_TOKEN'"
+
+- name: Create a group for the gitlab-runner service
+  group:
+name: gitlab-runner
+
+- name: Create a user for the gitlab-runner service
+  user:
+user: gitlab-runner
+group: gitlab-runner
+comment: GitLab Runner
+home: /home/gitlab-runner
+shell: /bin/bash
+
+- name: Remove the .bash_logout file when on Ubuntu systems
+  file:
+path: /home/gitlab-runner/.bash_logout
+state: absent
+  when: "ansible_facts['distribution'] == 'Ubuntu'"
+
+- name: Set the Operating System for gitlab-runner
+  set_fact:
+gitlab_runner_os: "{{ ansible_facts[\"system\"]|lower }}"
+- debug:
+msg: gitlab-runner OS is {{

Re: [PATCH v6 2/4] Jobs based on custom runners: build environment docs and playbook

2021-06-08 Thread Wainer dos Santos Moschetta


Hi,

On 6/8/21 12:14 AM, Cleber Rosa wrote:

To run basic jobs on custom runners, the environment needs to be
properly set up.  The most common requirement is having the right
packages installed.

The playbook introduced here covers the QEMU's project s390x and
aarch64 machines.  At the time this is being proposed, those machines
have already had this playbook applied to them.

Signed-off-by: Cleber Rosa 
---
  docs/devel/ci.rst  | 30 
  scripts/ci/setup/build-environment.yml | 98 ++
  scripts/ci/setup/inventory.template|  1 +
  3 files changed, 129 insertions(+)
  create mode 100644 scripts/ci/setup/build-environment.yml
  create mode 100644 scripts/ci/setup/inventory.template

diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
index 585b7bf4b8..35c6b5e269 100644
--- a/docs/devel/ci.rst
+++ b/docs/devel/ci.rst
@@ -26,3 +26,33 @@ gitlab-runner, is called a "custom runner".
  The GitLab CI jobs definition for the custom runners are located under::
  
.gitlab-ci.d/custom-runners.yml

+
+Machine Setup Howto
+---
+
+For all Linux based systems, the setup can be mostly automated by the
+execution of two Ansible playbooks.  Create an ``inventory`` file
+under ``scripts/ci/setup``, such as this::

Missing to mention the template file.

+
+  fully.qualified.domain
+  other.machine.hostname
+
+You may need to set some variables in the inventory file itself.  One
+very common need is to tell Ansible to use a Python 3 interpreter on
+those hosts.  This would look like::
+
+  fully.qualified.domain ansible_python_interpreter=/usr/bin/python3
+  other.machine.hostname ansible_python_interpreter=/usr/bin/python3
+
+Build environment
+~
+
+The ``scripts/ci/setup/build-environment.yml`` Ansible playbook will
+set up machines with the environment needed to perform builds and run
+QEMU tests.  It covers a number of different Linux distributions and
+FreeBSD.
+
+To run the playbook, execute::
+
+  cd scripts/ci/setup
+  ansible-playbook -i inventory build-environment.yml
diff --git a/scripts/ci/setup/build-environment.yml 
b/scripts/ci/setup/build-environment.yml
new file mode 100644
index 00..664f2f0519
--- /dev/null
+++ b/scripts/ci/setup/build-environment.yml
@@ -0,0 +1,98 @@
+---
+- name: Installation of basic packages to build QEMU
+  hosts: all


You will need to "become: yes" if the login user is not privileged, right?

Or mention on the documentation how the user should configure the 
connection for privileged login.


About privilege escalation with Ansible: 
https://docs.ansible.com/ansible/latest/user_guide/become.html



+  tasks:


Just a tip: you can put all those task under a block 
(https://docs.ansible.com/ansible/latest/user_guide/playbooks_blocks.html) 
so check if "ansible_facts['distribution'] == 'Ubuntu'" only once.


I reviewed the remain of the playbook; it looks good to me.


+- name: Update apt cache
+  apt:
+update_cache: yes
+  when:
+- ansible_facts['distribution'] == 'Ubuntu'
+
+- name: Install basic packages to build QEMU on Ubuntu 18.04/20.04
+  package:
+name:
+# Originally from tests/docker/dockerfiles/ubuntu1804.docker
+  - ccache
+  - gcc
+  - gettext
+  - git
+  - glusterfs-common
+  - libaio-dev
+  - libattr1-dev
+  - libbrlapi-dev
+  - libbz2-dev
+  - libcacard-dev
+  - libcap-ng-dev
+  - libcurl4-gnutls-dev
+  - libdrm-dev
+  - libepoxy-dev
+  - libfdt-dev
+  - libgbm-dev
+  - libgtk-3-dev
+  - libibverbs-dev
+  - libiscsi-dev
+  - libjemalloc-dev
+  - libjpeg-turbo8-dev
+  - liblzo2-dev
+  - libncurses5-dev
+  - libncursesw5-dev
+  - libnfs-dev
+  - libnss3-dev
+  - libnuma-dev
+  - libpixman-1-dev
+  - librados-dev
+  - librbd-dev
+  - librdmacm-dev
+  - libsasl2-dev
+  - libsdl2-dev
+  - libseccomp-dev
+  - libsnappy-dev
+  - libspice-protocol-dev
+  - libssh-dev
+  - libusb-1.0-0-dev
+  - libusbredirhost-dev
+  - libvdeplug-dev
+  - libvte-2.91-dev
+  - libzstd-dev
+  - make
+  - python3-yaml
+  - python3-sphinx
+  - python3-sphinx-rtd-theme
+  - ninja-build
+  - sparse
+  - xfslibs-dev
+state: present
+  when:
+- ansible_facts['distribution'] == 'Ubuntu'
+
+- name: Install packages to build QEMU on Ubuntu 18.04/20.04 on non-s390x
+  package:
+name:
+  - libspice-server-dev
+  - libxen-dev
+state: present
+  when:
+- ansible_facts['distribution'] == 'Ubuntu'
+- ansible_facts['architecture'] != 's390x'
+
+- name: Install basic packages to build QEMU on Ubuntu 18.04

Re: [PATCH v6 1/4] Jobs based on custom runners: documentation and configuration placeholder

2021-06-08 Thread Wainer dos Santos Moschetta




On 6/8/21 12:14 AM, Cleber Rosa wrote:

As described in the included documentation, the "custom runner" jobs
extend the GitLab CI jobs already in place.  One of their primary
goals of catching and preventing regressions on a wider number of host
systems than the ones provided by GitLab's shared runners.

This sets the stage in which other community members can add their own
machine configuration documentation/scripts, and accompanying job
definitions.  As a general rule, those newly added contributed jobs
should run as "non-gating", until their reliability is verified (AKA
"allow_failure: true").

Signed-off-by: Cleber Rosa 
---
  .gitlab-ci.d/custom-runners.yml | 14 ++
  .gitlab-ci.d/qemu-project.yml   |  1 +
  docs/devel/ci.rst   | 28 
  docs/devel/index.rst|  1 +
  4 files changed, 44 insertions(+)
  create mode 100644 .gitlab-ci.d/custom-runners.yml
  create mode 100644 docs/devel/ci.rst



Reviewed-by: Wainer dos Santos Moschetta 




diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
new file mode 100644
index 00..a07b27384c
--- /dev/null
+++ b/.gitlab-ci.d/custom-runners.yml
@@ -0,0 +1,14 @@
+# The CI jobs defined here require GitLab runners installed and
+# registered on machines that match their operating system names,
+# versions and architectures.  This is in contrast to the other CI
+# jobs that are intended to run on GitLab's "shared" runners.
+
+# Different than the default approach on "shared" runners, based on
+# containers, the custom runners have no such *requirement*, as those
+# jobs should be capable of running on operating systems with no
+# compatible container implementation, or no support from
+# gitlab-runner.  To avoid problems that gitlab-runner can cause while
+# reusing the GIT repository, let's enable the clone strategy, which
+# guarantees a fresh repository on each job run.
+variables:
+  GIT_STRATEGY: clone
diff --git a/.gitlab-ci.d/qemu-project.yml b/.gitlab-ci.d/qemu-project.yml
index 64cb2ba1da..dde8270301 100644
--- a/.gitlab-ci.d/qemu-project.yml
+++ b/.gitlab-ci.d/qemu-project.yml
@@ -9,3 +9,4 @@ include:
- local: '/.gitlab-ci.d/crossbuilds.yml'
- local: '/.gitlab-ci.d/buildtest.yml'
- local: '/.gitlab-ci.d/static_checks.yml'
+  - local: '/.gitlab-ci.d/custom-runners.yml'
diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
new file mode 100644
index 00..585b7bf4b8
--- /dev/null
+++ b/docs/devel/ci.rst
@@ -0,0 +1,28 @@
+==
+CI
+==
+
+QEMU has configurations enabled for a number of different CI services.
+The most up to date information about them and their status can be
+found at::
+
+   https://wiki.qemu.org/Testing/CI
+
+Jobs on Custom Runners
+==
+
+Besides the jobs run under the various CI systems listed before, there
+are a number additional jobs that will run before an actual merge.
+These use the same GitLab CI's service/framework already used for all
+other GitLab based CI jobs, but rely on additional systems, not the
+ones provided by GitLab as "shared runners".
+
+The architecture of GitLab's CI service allows different machines to
+be set up with GitLab's "agent", called gitlab-runner, which will take
+care of running jobs created by events such as a push to a branch.
+Here, the combination of a machine, properly configured with GitLab's
+gitlab-runner, is called a "custom runner".
+
+The GitLab CI jobs definition for the custom runners are located under::
+
+  .gitlab-ci.d/custom-runners.yml
diff --git a/docs/devel/index.rst b/docs/devel/index.rst
index 791925dcda..c9a02e786e 100644
--- a/docs/devel/index.rst
+++ b/docs/devel/index.rst
@@ -27,6 +27,7 @@ Contents:
 migration
 atomics
 stable-process
+   ci
 qtest
 decodetree
 secure-coding-practices

Re: [PATCH v6 4/4] Jobs based on custom runners: add job definitions for QEMU's machines

2021-06-08 Thread Wainer dos Santos Moschetta


Hi,

On 6/8/21 12:14 AM, Cleber Rosa wrote:

The QEMU project has two machines (aarch64 and s390x) that can be used
for jobs that do build and run tests.  This introduces those jobs,
which are a mapping of custom scripts used for the same purpose.

Signed-off-by: Cleber Rosa 
---
  .gitlab-ci.d/custom-runners.yml | 208 
  1 file changed, 208 insertions(+)

diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
index a07b27384c..061d3cdfed 100644
--- a/.gitlab-ci.d/custom-runners.yml
+++ b/.gitlab-ci.d/custom-runners.yml
@@ -12,3 +12,211 @@
  # guarantees a fresh repository on each job run.
  variables:
GIT_STRATEGY: clone
+
+# All ubuntu-18.04 jobs should run successfully in an environment
+# setup by the scripts/ci/setup/build-environment.yml task
+# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
+ubuntu-18.04-s390x-all-linux-static:
+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'


Should it restrict the job for pushes to qemu-project only? If yes, then 
it probably needs the statement:


'$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'

If you change that here, you will end it changing all over the jobs. In 
general, there are many boilerplates in this file. I'm ok to merge it as 
is as long as it is followed by another series to refactor the code.



+ script:
+ # --disable-libssh is needed because of 
https://bugs.launchpad.net/qemu/+bug/1838763
+ # --disable-glusterfs is needed because there's no static version of those 
libs in distro supplied packages
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --static --disable-system --disable-glusterfs 
--disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check-tcg V=1
+
+ubuntu-18.04-s390x-all:
+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-18.04-s390x-alldbg:
Maybe we don't need both ubuntu-18.04-s390x-all and 
ubuntu-18.04-s390x-alldbg jobs.

+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --disable-libssh
+ - make clean
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+ubuntu-18.04-s390x-clang:
+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+   when: manual
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-18.04-s390x-tci:
+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --enable-tcg-interpreter
+ - make --output-sync -j`nproc`
I think it needs to `make check-tcg` at least. See "build-tci" in 
`.gitlab-ci.d/buildtest.yml` for other tests being executed on shared 
runners.

+
+ubuntu-18.04-s390x-notcg:
The "build-tcg-disabled" in `.gitlab-ci.d/buildtest.yml` could be 
mimic-ed here too.

+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+   when: manual
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --disable-tcg
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+# All ubuntu-20.04 jobs should run successfully in an environment
+# setup by the scripts/ci/setup/qemu/build-environment.yml task
+# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
+ubuntu-20.04-aarch64-all-linux-static:
+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ # --disable-libssh is needed because of 
https://bugs.launchpad.net/qemu/+bug/1838763
+ # --disable-glusterfs is needed because there's no static version of those 
libs in distro supplied packages
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --static --disable-system --disable-glusterfs 
--disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check-tcg V=1
+
+ubuntu-20.04-aarch64-all:
+ allow_failure: true
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh
+ - make

Re: [RFC PATCH] scripts/checkpatch: roll diff tweaking into checkpatch itself

2021-06-08 Thread Wainer dos Santos Moschetta




On 6/7/21 2:18 PM, Alex Bennée wrote:

Rather than relying on external tweaks lets just do it inside
checkpatch's direct commitish handling which is QEMU specific code
anyway.

Suggested-by: Daniel P. Berrangé 
Signed-off-by: Alex Bennée 
---
  .gitlab-ci.d/static_checks.yml | 3 ---
  .patchew.yml   | 3 ---

ah, there is the patchew configuration file.

  scripts/checkpatch.pl  | 7 ++-
  3 files changed, 6 insertions(+), 7 deletions(-)


Reviewed-by: Wainer dos Santos Moschetta 



diff --git a/.gitlab-ci.d/static_checks.yml b/.gitlab-ci.d/static_checks.yml
index 7e685c6a65..c5fa4fce26 100644
--- a/.gitlab-ci.d/static_checks.yml
+++ b/.gitlab-ci.d/static_checks.yml
@@ -4,9 +4,6 @@ check-patch:
needs:
  job: amd64-centos8-container
script:
-- git config --local diff.renamelimit 0
-- git config --local diff.renames True
-- git config --local diff.algorithm histogram
  - .gitlab-ci.d/check-patch.py
variables:
  GIT_DEPTH: 1000
diff --git a/.patchew.yml b/.patchew.yml
index 2638b7f564..1b78262ce5 100644
--- a/.patchew.yml
+++ b/.patchew.yml
@@ -138,9 +138,6 @@ testing:
script: |
  #!/bin/bash
  git rev-parse base > /dev/null || exit 0
-git config --local diff.renamelimit 0
-git config --local diff.renames True
-git config --local diff.algorithm histogram
  ./scripts/checkpatch.pl --mailback base..
  docker-mingw@fedora:
enabled: true
diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index bbcd25ae05..cb8eff233e 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -399,7 +399,12 @@ if ($chk_branch) {
my $num_patches = @patches;
for my $hash (@patches) {
my $FILE;
-   open($FILE, '-|', "git", "show", "--patch-with-stat", $hash) ||
+   open($FILE, '-|', "git",
+ "-c", "diff.renamelimit=0",
+ "-c", "diff.renames=True",
+ "-c", "diff.algorithm=histogram",
+ "show",
+ "--patch-with-stat", $hash) ||
die "$P: git show $hash - $!\n";
while (<$FILE>) {
chomp;

Re: [RFC PATCH] gitlab: work harder to avoid false positives in checkpatch

2021-06-04 Thread Wainer dos Santos Moschetta


Hi,

On 6/2/21 12:32 PM, Alex Bennée wrote:

This copies the behaviour of patchew's configuration to make the diff
algorithm generate a minimal diff.


I was intrigued on where you had found that configuration. Then I found 
it myself here:


https://patchew.org/logs/20210604155532.1499282-1-js...@redhat.com/testing.checkpatch/?type=message

Assuming patchew has been doing the right thing, I think it is a good 
idea to configure Git alike on CI.


Thus:

Reviewed-by: Wainer dos Santos Moschetta 




Signed-off-by: Alex Bennée 
---
  .gitlab-ci.d/static_checks.yml | 6 +-
  1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.d/static_checks.yml b/.gitlab-ci.d/static_checks.yml
index 91247a6f67..ec09da799d 100644
--- a/.gitlab-ci.d/static_checks.yml
+++ b/.gitlab-ci.d/static_checks.yml
@@ -3,7 +3,11 @@ check-patch:
image: $CI_REGISTRY_IMAGE/qemu/centos8:latest
needs:
  job: amd64-centos8-container
-  script: .gitlab-ci.d/check-patch.py
+  script:
+- git config --local diff.renamelimit 0
+- git config --local diff.renames True
+- git config --local diff.algorithm histogram
+- .gitlab-ci.d/check-patch.py
variables:
  GIT_DEPTH: 1000
rules:

Re: [PATCH] tests/vm/freebsd: Increase code coverage

2021-05-31 Thread Wainer dos Santos Moschetta


Hi,

On 5/31/21 7:03 AM, Philippe Mathieu-Daudé wrote:

Install more dependencies to increase code coverage.

Signed-off-by: Philippe Mathieu-Daudé 
---
  tests/vm/freebsd | 5 +
  1 file changed, 5 insertions(+)


With or without this patch I got an error when `make vm-build-freebsd`. 
It fails to install packages.


For example, with this patch I got:

< Output omitted>

### Installing packages ...
Failed to prepare guest environment
Traceback (most recent call last):
  File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 634, in main
    return vm.build_image(args.image)
  File "/home/wmoschet/src/qemu/tests/vm/freebsd", line 206, in build_image
    self.ssh_root_check("pkg install -y %s\n" % " ".join(self.pkgs))
  File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 255, in 
ssh_root_check

    self._ssh_do(self._config["root_user"], cmd, True)
  File "/home/wmoschet/src/qemu/tests/vm/basevm.py", line 242, in _ssh_do
    raise Exception("SSH command failed: %s" % cmd)
Exception: SSH command failed: pkg install -y git pkgconf bzip2 python37 
ninja bash gmake gsed gettext cyrus-sasl gnutls nettle jpeg-turbo png 
sdl2 gtk3 libxkbcommon pixman libepoxy mesa-libs zstd usbredir


Is it a known issue?



diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index 6e20e843228..85049b43136 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -46,7 +46,9 @@ class FreeBSDVM(basevm.BaseVM):
  "gettext",
  
  # libs: crypto

+"cyrus-sasl",
  "gnutls",
+"nettle",
  
  # libs: images

  "jpeg-turbo",
@@ -56,6 +58,7 @@ class FreeBSDVM(basevm.BaseVM):
  "sdl2",
  "gtk3",
  "libxkbcommon",
+"pixman",
  
  # libs: opengl

  "libepoxy",
@@ -63,6 +66,8 @@ class FreeBSDVM(basevm.BaseVM):
  
  # libs: migration

  "zstd",
+
+"usbredir",
  ]
  
  # TODO: Enable gnutls again once FreeBSD's libtasn1 got fixed

Re: [PATCH] tests: Boot and halt a Linux guest on the Raspberry Pi 2 machine

2021-05-31 Thread Wainer dos Santos Moschetta




On 5/31/21 8:38 AM, Philippe Mathieu-Daudé wrote:

Add a test booting and quickly shutdown a raspi2 machine,
to test the power management model:

(1/1) 
tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_raspi2_initrd:
   console: [0.00] Booting Linux on physical CPU 0xf00
   console: [0.00] Linux version 4.14.98-v7+ (dom@dom-XPS-13-9370) (gcc 
version 4.9.3 (crosstool-NG crosstool-ng-1.22.0-88-g8460611)) #1200 SMP Tue Feb 
12 20:27:48 GMT 2019
   console: [0.00] CPU: ARMv7 Processor [410fc075] revision 5 (ARMv7), 
cr=10c5387d
   console: [0.00] CPU: div instructions available: patching division 
code
   console: [0.00] CPU: PIPT / VIPT nonaliasing data cache, VIPT 
aliasing instruction cache
   console: [0.00] OF: fdt: Machine model: Raspberry Pi 2 Model B
   ...
   console: Boot successful.
   console: cat /proc/cpuinfo
   console: / # cat /proc/cpuinfo
   ...
   console: processor  : 3
   console: model name : ARMv7 Processor rev 5 (v7l)
   console: BogoMIPS   : 125.00
   console: Features   : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 
idiva idivt vfpd32 lpae evtstrm
   console: CPU implementer: 0x41
   console: CPU architecture: 7
   console: CPU variant: 0x0
   console: CPU part   : 0xc07
   console: CPU revision   : 5
   console: Hardware   : BCM2835
   console: Revision   : 
   console: Serial : 
   console: cat /proc/iomem
   console: / # cat /proc/iomem
   console: -3bff : System RAM
   console: 8000-00af : Kernel code
   console: 00c0-00d468ef : Kernel data
   console: 3f006000-3f006fff : dwc_otg
   console: 3f007000-3f007eff : /soc/dma@7e007000
   console: 3f00b880-3f00b8bf : /soc/mailbox@7e00b880
   console: 3f10-3f100027 : /soc/watchdog@7e10
   console: 3f101000-3f102fff : /soc/cprman@7e101000
   console: 3f20-3f2000b3 : /soc/gpio@7e20
   PASS (24.59 s)
   RESULTS: PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | 
CANCEL 0
   JOB TIME   : 25.02 s

Signed-off-by: Philippe Mathieu-Daudé 
---
Based-on: <162137039825.30884.244582579824080919...@git.sr.ht>
---
  tests/acceptance/boot_linux_console.py | 43 ++
  1 file changed, 43 insertions(+)


Even though I did not patch my sources with the fix, I ran the test and 
it failed on the VM's shutdown (as expected, I suppose). The test code 
looks good to me, so:


Reviewed-by: Wainer dos Santos Moschetta 



diff --git a/tests/acceptance/boot_linux_console.py 
b/tests/acceptance/boot_linux_console.py
index 276a53f1464..19d328203c7 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -16,6 +16,7 @@
  from avocado import skip
  from avocado import skipUnless
  from avocado_qemu import Test
+from avocado_qemu import exec_command
  from avocado_qemu import exec_command_and_wait_for_pattern
  from avocado_qemu import interrupt_interactive_console_until_pattern
  from avocado_qemu import wait_for_console_pattern
@@ -467,6 +468,48 @@ def test_arm_raspi2_uart0(self):
  """
  self.do_test_arm_raspi2(0)
  
+def test_arm_raspi2_initrd(self):

+"""
+:avocado: tags=arch:arm
+:avocado: tags=machine:raspi2
+"""
+deb_url = ('http://archive.raspberrypi.org/debian/'
+   'pool/main/r/raspberrypi-firmware/'
+   'raspberrypi-kernel_1.20190215-1_armhf.deb')
+deb_hash = 'cd284220b32128c5084037553db3c482426f3972'
+deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
+kernel_path = self.extract_from_deb(deb_path, '/boot/kernel7.img')
+dtb_path = self.extract_from_deb(deb_path, '/boot/bcm2709-rpi-2-b.dtb')
+
+initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
+  '2eb0a73b5d5a28df3170c546ddaaa9757e1e0848/rootfs/'
+  'arm/rootfs-armv7a.cpio.gz')
+initrd_hash = '604b2e45cdf35045846b8bbfbf2129b1891bdc9c'
+initrd_path_gz = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
+initrd_path = os.path.join(self.workdir, 'rootfs.cpio')
+archive.gzip_uncompress(initrd_path_gz, initrd_path)
+
+self.vm.set_console()
+kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
+   'earlycon=pl011,0x3f201000 console=ttyAMA0 '
+   'panic=-1 noreboot ' +
+   'dwc_otg.fiq_fsm_enable=0')
+self.vm.add_args('-kernel', kernel_path,
+ '-dtb', dtb_path,
+ '-initrd', initrd_path,
+ '-append', kernel_command_line,
+ '-no-reboot')
+self.vm.launch()
+self.wait_for_console_pattern('Boot successful.')
+
+exec_co

Re: [PATCH 2/2] tests/acceptance: Add tests for the Pegasos2 machine

2021-05-27 Thread Wainer dos Santos Moschetta


Hi,

On 5/15/21 10:45 AM, Philippe Mathieu-Daudé wrote:

Add a pair of tests for the Pegasos2 machine following the steps from:
https://lists.nongnu.org/archive/html/qemu-devel/2021-01/msg01553.html

   $ PEGASOS2_ROM_PATH=/tmp/pegasos2.rom AVOCADO_ALLOW_UNTRUSTED_CODE=1 \
 avocado --show=app,console,tesseract \
   run -t machine:pegasos2 tests/acceptance/
(1/2) 
tests/acceptance/machine_ppc_pegasos.py:PPCPegasos2.test_rom_serial_console:
   console: PegasosII Boot Strap (c) 2002-2003 bplan GmbH
   console: Running on CPU PVR:000C0209
   console: Enable L1 ICache... 
   Done.
   console: Reading W83194 :
   FAILED.
   console: Setting Front Side Bus to 133MHz... 
   FAILED.
   console: Configuring DDR...  
   Done.
   console: Configuring PCI0... 
   Done.
   console: Configuring PCI1... 
   Done.
   console: Configuring ETH...  
   Done.
   console: Releasing IDE reset ... 
   Done.
   console: Configuring Legacy Devices
   console: Initializing KBD... 
   Done.
   console: Testing 1000 Bytes, Pass:  Failed: 
   console: RAM TEST (fill linear)...   
   Done.
   console: 
   console: SmartFirmware:
   console: cpu0: PowerPC,G4 CPUClock 599 Mhz BUSClock 133 Mhz (Version 
0x000C,0x0209)
   console: no/bad nvramrc - performing default startup script
   console: channel 1 unit 0 : atapi | QEMU DVD-ROM 
| 2.5+
   console: ATA device not present or not responding
   console: Welcome to SmartFirmware(tm) for bplan Pegasos2 version 1.1 
(20040405172512)
   PASS (5.23 s)
(2/2) 
tests/acceptance/machine_ppc_pegasos.py:PPCPegasos2.test_morphos_cdrom_vga:
   ...
   console: Welcome to SmartFirmware(tm) for bplan Pegasos2 version 1.1 
(20040405172512)
   console: SmartFirmware(tm) Copyright 1996-2001 by CodeGen, Inc.
   console: All Rights Reserved.
   console: Pegasos BIOS Extensions Copyright 2001-2003 by bplan GmbH.
   console: All Rights Reserved.
   console: entering main read/eval loop...
   console: ok boot cd boot.img
   console: ISO-9660 filesystem:  System-ID: "MORPHOS"  Volume-ID: "MorphOSBoot"
   console: " flags=0x2 extent=0x20 size=0x1800
   console: Memory used before SYS_Init: 9MB
   console: PCI ATA/ATAPI Driver@2: PIO Mode 4
   console: PCI ATA/ATAPI Driver@2: UDMA Mode 5
   console: ide.device@2: QEMU QEMU DVD-ROM 
   console: ide.device@2:  CDRom , found, bootable
   tesseract: Ambient Screen 4: Saturday, 15 May 2021, 13:36:06 &
   tesseract: keymap
   tesseract: Albanian keyboard with 101/104 keys
   tesseract: ‘American keyboard with Greek input extension, 105 keys
   tesseract: Belarusian keyboard with 105 keys
   tesseract: Belgian keyboard with 105 keys J
   tesseract: British Apple keyboard
   tesseract: British keyboard with 105 keys
   tesseract: Bulgarian keyboard with 104 keys
   tesseract: Canadian keyboard with 105 keys
   tesseract: Colemak layout for keyboards with 101/104 keys
   tesseract: Croatian keyboard with 101/108 keys
   tesseract: Czech keyboard (QWERTY) with 101/104 keys
   tesseract: Czech keyboard (QWERTZ) with 101/104 keys
   tesseract: Danish keyboard with 105 keys
   PASS (28.56 s)
   RESULTS: PASS 2 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | 
CANCEL 0
   JOB TIME   : 34.42 s

Signed-off-by: Philippe Mathieu-Daudé 
---
  tests/acceptance/machine_ppc_pegasos.py | 98 +
  1 file changed, 98 insertions(+)
  create mode 100644 tests/acceptance/machine_ppc_pegasos.py

diff --git a/tests/acceptance/machine_ppc_pegasos.py 
b/tests/acceptance/machine_ppc_pegasos.py
new file mode 100644
index 000..d36e920ebde
--- /dev/null
+++ b/tests/acceptance/machine_ppc_pegasos.py
@@ -0,0 +1,98 @@
+# Functional tests for the Pegasos2 machine.
+#
+# Copyright (c) 2021 Philippe Mathieu-Daudé 
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+import os
+import time
+
+from avocado import skipUnless
+from avocado_qemu import Test
+from avocado_qemu import exec_command_and_wait_for_pattern
+from avocado_qemu import wait_for_console_pattern
+from tesseract_utils import tesseract_available, tesseract_ocr
+
+PIL_AVAILABLE = True
+try:
+from PIL import Image
+except ImportError:
+PIL_AVAILABLE = False
+
+
+@skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
+@skipUnless(os.getenv('PEGASOS2_ROM_PATH'), 'PEGASOS2_ROM_PATH not available')
+class PPCPegasos2(Test):
+

Re: [PATCH 1/2] tests/acceptance: Ignore binary data sent on serial console

2021-05-27 Thread Wainer dos Santos Moschetta




On 5/15/21 10:45 AM, Philippe Mathieu-Daudé wrote:

If a guest sends binary data on the serial console, we get:

  File "tests/acceptance/avocado_qemu/__init__.py", line 92,
in _console_interaction msg = console.readline().strip()
  File "/usr/lib64/python3.8/codecs.py", line 322,
in decode (result, consumed) = self._buffer_decode(data, self.errors, final)
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa9 in position 2: 
invalid start byte



Good catch!




Since we use the console with readline(), fix it the easiest
way possible: ignore binary data (all current tests compare
text string anyway).

Signed-off-by: Philippe Mathieu-Daudé 
---
  tests/acceptance/avocado_qemu/__init__.py | 7 +--
  1 file changed, 5 insertions(+), 2 deletions(-)


Reviewed-by: Wainer dos Santos Moschetta 

Tested-by: Wainer dos Santos Moschetta 



diff --git a/tests/acceptance/avocado_qemu/__init__.py 
b/tests/acceptance/avocado_qemu/__init__.py
index 83b1741ec85..b55578e1cca 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -82,14 +82,17 @@ def _console_interaction(test, success_message, 
failure_message,
  assert not keep_sending or send_string
  if vm is None:
  vm = test.vm
-console = vm.console_socket.makefile()
+console = vm.console_socket.makefile(mode='rb', encoding='utf-8')
  console_logger = logging.getLogger('console')
  while True:
  if send_string:
  vm.console_socket.sendall(send_string.encode())
  if not keep_sending:
  send_string = None # send only once
-msg = console.readline().strip()
+try:
+msg = console.readline().decode().strip()
+except UnicodeDecodeError:
+msg = None
  if not msg:
  continue
  console_logger.debug(msg)

Re: [PATCH 1/1] tests/acceptance: change armbian archive to a faster host

2021-05-27 Thread Wainer dos Santos Moschetta


Hi,

On 5/26/21 5:56 PM, Willian Rampazzo wrote:

The current host for the image
Armbian_20.08.1_Orangepipc_bionic_current_5.8.5.img.xz
(archive.armbian.com) is extremely slow in the last couple of weeks,
making the job running the test
tests/system/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic_20_08
for the first time when the image is not yet on GitLab cache, time out
while the image is being downloaded.

This changes the host to one faster, so new users with an empty cache
are not impacted.


Here the old host performed slightly better: download time of 0:17:36 vs 
0:19:44. Maybe it was a temporary issue with the old host or maybe 
GitLab's runner network?


Anyway,

Reviewed-by: Wainer dos Santos Moschetta 




Signed-off-by: Willian Rampazzo 
---
  tests/acceptance/boot_linux_console.py | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/acceptance/boot_linux_console.py 
b/tests/acceptance/boot_linux_console.py
index 276a53f146..51c23b822c 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -804,7 +804,8 @@ def test_arm_orangepi_bionic_20_08(self):
  # to 1036 MiB, but the underlying filesystem is 1552 MiB...
  # As we expand it to 2 GiB we are safe.
  
-image_url = ('https://archive.armbian.com/orangepipc/archive/'

+image_url = ('https://armbian.systemonachip.net/'
+ 'archive/orangepipc/archive/'
   'Armbian_20.08.1_Orangepipc_bionic_current_5.8.5.img.xz')
  image_hash = ('b4d6775f5673486329e45a0586bf06b6'
'dbe792199fd182ac6b9c7bb6c7d3e6dd')

Re: [PATCH v3 3/6] gitlab-ci: Run GNU make via the $MAKE variable

2021-05-20 Thread Wainer dos Santos Moschetta




On 5/19/21 3:45 PM, Philippe Mathieu-Daudé wrote:

Add the $MAKE variable to call GNU make, and set it to 'gmake'
on FreeBSD to avoid:

   $ make -j"$JOBS"
   make: Unknown modifier ','
   make: "/builds/dTyar424/0/qemu/build/Makefile" line 3: Need an operator
   make: "/builds/dTyar424/0/qemu/build/Makefile" line 4: Missing dependency 
operator

Signed-off-by: Philippe Mathieu-Daudé 
---
  .gitlab-ci.d/buildtest-template.yml | 9 ++---
  1 file changed, 6 insertions(+), 3 deletions(-)



Reviewed-by: Wainer dos Santos Moschetta 




diff --git a/.gitlab-ci.d/buildtest-template.yml 
b/.gitlab-ci.d/buildtest-template.yml
index fe4f18595ac..f284d7a0eec 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -5,9 +5,11 @@
  ;
then
  JOBS=$(sysctl -n hw.ncpu)
+MAKE=gmake
  ;
else
  JOBS=$(expr $(nproc) + 1)
+MAKE=make
  ;
fi
  - echo "=== Using $JOBS simultaneous jobs ==="
@@ -33,22 +35,23 @@
then
  ../meson/meson.py configure . -Dbackend_max_links="$LD_JOBS" ;
fi || exit 1;
-- make -j"$JOBS"
+- $MAKE -j"$JOBS"
  - if test -n "$MAKE_CHECK_ARGS";
then
-make -j"$JOBS" $MAKE_CHECK_ARGS ;
+$MAKE -j"$JOBS" $MAKE_CHECK_ARGS ;
fi
  
  .native_test_job_template:

stage: test
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+  extends: .environment_variables_template
script:
  - scripts/git-submodule.sh update
  $(sed -n '/GIT_SUBMODULES=/ s/.*=// p' build/config-host.mak)
  - cd build
  - find . -type f -exec touch {} +
  # Avoid recompiling by hiding ninja with NINJA=":"
-- make NINJA=":" $MAKE_CHECK_ARGS
+- $MAKE NINJA=":" $MAKE_CHECK_ARGS
  
  .integration_test_job_template:

extends: .native_test_job_template

Re: [PATCH v3 2/6] gitlab-ci: Adapt JOBS variable for FreeBSD runners

2021-05-20 Thread Wainer dos Santos Moschetta


Hi,

On 5/19/21 3:45 PM, Philippe Mathieu-Daudé wrote:

'nproc' is not available on FreeBSD:

   $ JOBS=$(expr $(nproc) + 1)
   bash: line 119: nproc: command not found
   expr: syntax error

Instead, use 'sysctl -n hw.ncpu'.

Signed-off-by: Philippe Mathieu-Daudé 
---
  .gitlab-ci.d/buildtest-template.yml | 12 +++-
  1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.d/buildtest-template.yml 
b/.gitlab-ci.d/buildtest-template.yml
index 58b01744751..fe4f18595ac 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -1,6 +1,16 @@
  .environment_variables_template:
before_script:
-- JOBS=$(expr $(nproc) + 1)
+- if
+test $(uname) = "FreeBSD"
+;
+  then
+JOBS=$(sysctl -n hw.ncpu)
+;
+  else
+JOBS=$(expr $(nproc) + 1)
+;
+  fi
+- echo "=== Using $JOBS simultaneous jobs ==="


It has grown to beyond variables definitions. I suggest to rename it to 
something like `.before_script_template` (pure creativity, see?).


- Wainer

  
  .native_build_job_template:

stage: build

Re: [PATCH v3 1/6] gitlab-ci: Extract _variables template

2021-05-20 Thread Wainer dos Santos Moschetta


Hi,

On 5/19/21 3:45 PM, Philippe Mathieu-Daudé wrote:

To be able to set the same environment variables to multiple jobs,
extract what we currently have as a template.

Signed-off-by: Philippe Mathieu-Daudé 
---
  .gitlab-ci.d/buildtest-template.yml | 7 +--
  1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/.gitlab-ci.d/buildtest-template.yml 
b/.gitlab-ci.d/buildtest-template.yml
index 32aaef1a213..58b01744751 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -1,8 +1,11 @@
+.environment_variables_template:
+  before_script:
+- JOBS=$(expr $(nproc) + 1)
+


I wish we could wrap environment variables definitions in `variables` 
(https://docs.gitlab.com/ee/ci/yaml/README.html#variables) instead of 
`before_scripts`, but it wouldn't evaluate the shell expressions.


Unless I am wrong, if the job definition has its own `before_script` 
then including `.environment_variables_template` won't take effect. 
Shouldn't it be documented?


- Wainer


  .native_build_job_template:
stage: build
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
-  before_script:
-- JOBS=$(expr $(nproc) + 1)
+  extends: .environment_variables_template
script:
  - if test -n "$LD_JOBS";
then

Re: [PATCH 05/10] python/machine: Disable pylint warning for open() in _pre_launch

2021-05-14 Thread Wainer dos Santos Moschetta


Hi,

On 5/12/21 6:46 PM, John Snow wrote:

Shift the open() call later so that the pylint pragma applies *only* to
that one open() call. Add a note that suggests why this is safe: the
resource is unconditionally cleaned up in _post_shutdown().



You can also put it in a pylint disable/enable block. E.g.:

    # pylint: disable=consider-using-with

    self._qemu_log_file = open(self._qemu_log_path, 'wb')

    # pylint: enable=consider-using-with

However I don't know if this is bad practice. :)

Reviewed-by: Wainer dos Santos Moschetta 



_post_shutdown is called after failed launches (see launch()), and
unconditionally after every call to shutdown(), and therefore also on
__exit__.

Signed-off-by: John Snow 
---
  python/qemu/machine.py | 6 +-
  1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index c13ff9b32bf..8f86303b48f 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -308,7 +308,6 @@ def _pre_launch(self) -> None:
  self._temp_dir = tempfile.mkdtemp(prefix="qemu-machine-",
dir=self._test_dir)
  self._qemu_log_path = os.path.join(self._temp_dir, self._name + 
".log")
-self._qemu_log_file = open(self._qemu_log_path, 'wb')
  
  if self._console_set:

  self._remove_files.append(self._console_address)
@@ -323,6 +322,11 @@ def _pre_launch(self) -> None:
  nickname=self._name
  )
  
+# NOTE: Make sure any opened resources are *definitely* freed in

+# _post_shutdown()!
+# pylint: disable=consider-using-with
+self._qemu_log_file = open(self._qemu_log_path, 'wb')
+
  def _post_launch(self) -> None:
  if self._qmp_connection:
  self._qmp.accept()

Re: [PATCH 03/10] python/machine: use subprocess.run instead of subprocess.Popen

2021-05-14 Thread Wainer dos Santos Moschetta


Hi,

On 5/12/21 6:46 PM, John Snow wrote:

use run() instead of Popen() -- to assert to pylint that we are not
forgetting to close a long-running program.

Signed-off-by: John Snow 
---
  python/qemu/machine.py | 15 +--
  1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index 41f51bd27d0..c13ff9b32bf 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -223,13 +223,16 @@ def send_fd_scm(self, fd: Optional[int] = None,
  assert fd is not None
  fd_param.append(str(fd))
  
-proc = subprocess.Popen(

-fd_param, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE,
-stderr=subprocess.STDOUT, close_fds=False
+proc = subprocess.run(
+fd_param,
+stdin=subprocess.DEVNULL,
+stdout=subprocess.PIPE,
+stderr=subprocess.STDOUT,
+check=True,
+close_fds=False,
  )


Now it might throw a CalledProcessError given that `check=True`. 
Shouldn't it capture the exception and (possible) re-throw as an 
QEMUMachineError?


- Wainer


-output = proc.communicate()[0]
-if output:
-LOG.debug(output)
+if proc.stdout:
+LOG.debug(proc.stdout)
  
  return proc.returncode

[PATCH 6/7] tests/acceptance: Move wait_for_console_pattern to ConsoleMixIn

2021-05-03 Thread Wainer dos Santos Moschetta

This moved wait_for_console_pattern() to ConsoleMixIn.

By far this change required the most adaptations on tests.

Notice that:

 1) Some tests from boot_linux_console.py were using the 
wait_for_console_pattern()
from the avocado_qemu package rather than the overloaded method on the
LinuxKernelTest class, and that explains the explict calls to
ConsoleMixIn.wait_for_console_pattern().

Likewise in boot_xen.py file.

 2) In virtiofs_submounts.py, wait_for_console_pattern() was imported but not 
used.

Signed-off-by: Wainer dos Santos Moschetta 
---
 tests/acceptance/avocado_qemu/__init__.py| 23 +---
 tests/acceptance/boot_linux_console.py   | 14 ++--
 tests/acceptance/boot_xen.py |  5 +++--
 tests/acceptance/linux_ssh_mips_malta.py |  8 +++
 tests/acceptance/machine_arm_canona1100.py   |  6 ++---
 tests/acceptance/machine_arm_integratorcp.py |  8 +++
 tests/acceptance/machine_arm_n8x0.py |  6 ++---
 tests/acceptance/machine_microblaze.py   |  8 +++
 tests/acceptance/machine_mips_loongson3v.py  |  6 ++---
 tests/acceptance/machine_mips_malta.py   |  6 ++---
 tests/acceptance/machine_ppc.py  | 10 -
 tests/acceptance/machine_rx_gdbsim.py|  7 +++---
 tests/acceptance/machine_s390_ccw_virtio.py  |  7 +++---
 tests/acceptance/machine_sparc64_sun4u.py|  6 ++---
 tests/acceptance/machine_sparc_leon3.py  |  8 +++
 tests/acceptance/multiprocess.py |  5 ++---
 tests/acceptance/ppc_prep_40p.py | 16 +++---
 tests/acceptance/virtio-gpu.py   |  4 +---
 tests/acceptance/virtiofs_submounts.py   |  1 -
 19 files changed, 73 insertions(+), 81 deletions(-)

diff --git a/tests/acceptance/avocado_qemu/__init__.py 
b/tests/acceptance/avocado_qemu/__init__.py
index 4a0129c0eb..b21f9ea3ff 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -101,19 +101,6 @@ def _console_interaction(test, success_message, 
failure_message,
 (failure_message, success_message)
 test.fail(fail)
 
-def wait_for_console_pattern(test, success_message, failure_message=None,
- vm=None):
-"""
-Waits for messages to appear on the console, while logging the content
-
-:param test: an Avocado test containing a VM that will have its console
- read and probed for a success or failure message
-:type test: :class:`avocado_qemu.Test`
-:param success_message: if this message appears, test succeeds
-:param failure_message: if this message appears, test fails
-"""
-_console_interaction(test, success_message, failure_message, None, vm=vm)
-
 class ConsoleMixIn():
 """Contains utilities for interacting with a guest via Console."""
 
@@ -163,6 +150,16 @@ def interrupt_interactive_console_until_pattern(self, 
success_message,
 _console_interaction(self, success_message, failure_message,
  interrupt_string, True)
 
+def wait_for_console_pattern(self, success_message, failure_message=None,
+ vm=None):
+"""
+Waits for messages to appear on the console, while logging the content
+
+:param success_message: if this message appears, test succeeds
+:param failure_message: if this message appears, test fails
+"""
+_console_interaction(self, success_message, failure_message, None, 
vm=vm)
+
 class Test(avocado.Test):
 def _get_unique_tag_val(self, tag_name):
 """
diff --git a/tests/acceptance/boot_linux_console.py 
b/tests/acceptance/boot_linux_console.py
index 50e0a3fe79..e8d7a127fe 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -17,7 +17,6 @@
 from avocado import skipUnless
 from avocado_qemu import Test
 from avocado_qemu import ConsoleMixIn
-from avocado_qemu import wait_for_console_pattern
 from avocado.utils import process
 from avocado.utils import archive
 from avocado.utils.path import find_command, CmdNotFoundError
@@ -48,7 +47,7 @@ class LinuxKernelTest(Test, ConsoleMixIn):
 KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
 
 def wait_for_console_pattern(self, success_message, vm=None):
-wait_for_console_pattern(self, success_message,
+super().wait_for_console_pattern(success_message,
  failure_message='Kernel panic - not syncing',
  vm=vm)
 
@@ -262,7 +261,7 @@ def test_mips64el_malta_5KEc_cpio(self):
  '-append', kernel_command_line,
  '-no-reboot')
 self.vm.launch()
-wait_for_console_pattern(self, 'Boot successful.')
+ConsoleMixIn.wait_for_conso

[PATCH 7/7] tests/acceptance: Move _console_interaction to ConsoleMixIn

2021-05-03 Thread Wainer dos Santos Moschetta

This moved the last remaining _console_interaction() to ConsoleMixIn.

None tests call it directly, so only the other methods in ConsoleMixIn
needed to be adapted.

Signed-off-by: Wainer dos Santos Moschetta 
---
 tests/acceptance/avocado_qemu/__init__.py | 57 +++
 1 file changed, 28 insertions(+), 29 deletions(-)

diff --git a/tests/acceptance/avocado_qemu/__init__.py 
b/tests/acceptance/avocado_qemu/__init__.py
index b21f9ea3ff..a6de3fe11a 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -76,34 +76,33 @@ def pick_default_qemu_bin(arch=None):
 if is_readable_executable_file(qemu_bin_from_bld_dir_path):
 return qemu_bin_from_bld_dir_path
 
-
-def _console_interaction(test, success_message, failure_message,
- send_string, keep_sending=False, vm=None):
-assert not keep_sending or send_string
-if vm is None:
-vm = test.vm
-console = vm.console_socket.makefile()
-console_logger = logging.getLogger('console')
-while True:
-if send_string:
-vm.console_socket.sendall(send_string.encode())
-if not keep_sending:
-send_string = None # send only once
-msg = console.readline().strip()
-if not msg:
-continue
-console_logger.debug(msg)
-if success_message is None or success_message in msg:
-break
-if failure_message and failure_message in msg:
-console.close()
-fail = 'Failure message found in console: "%s". Expected: "%s"' % \
-(failure_message, success_message)
-test.fail(fail)
-
 class ConsoleMixIn():
 """Contains utilities for interacting with a guest via Console."""
 
+def _console_interaction(self, success_message, failure_message,
+ send_string, keep_sending=False, vm=None):
+assert not keep_sending or send_string
+if vm is None:
+vm = self.vm
+console = vm.console_socket.makefile()
+console_logger = logging.getLogger('console')
+while True:
+if send_string:
+vm.console_socket.sendall(send_string.encode())
+if not keep_sending:
+send_string = None # send only once
+msg = console.readline().strip()
+if not msg:
+continue
+console_logger.debug(msg)
+if success_message is None or success_message in msg:
+break
+if failure_message and failure_message in msg:
+console.close()
+fail = 'Failure message found in console: "%s". Expected: 
"%s"' % \
+(failure_message, success_message)
+self.fail(fail)
+
 def exec_command(self, command):
 """
 Send a command to a console (appending CRLF characters), while logging
@@ -112,7 +111,7 @@ def exec_command(self, command):
 :param command: the command to send
 :type command: str
 """
-_console_interaction(self, None, None, command + '\r')
+self._console_interaction(None, None, command + '\r')
 
 def exec_command_and_wait_for_pattern(self, command,
   success_message, 
failure_message=None):
@@ -125,7 +124,7 @@ def exec_command_and_wait_for_pattern(self, command,
 :param success_message: if this message appears, test succeeds
 :param failure_message: if this message appears, test fails
 """
-_console_interaction(self, success_message, failure_message, command + 
'\r')
+self._console_interaction(success_message, failure_message, command + 
'\r')
 
 def interrupt_interactive_console_until_pattern(self, success_message,
 failure_message=None,
@@ -147,7 +146,7 @@ def interrupt_interactive_console_until_pattern(self, 
success_message,
 :param interrupt_string: a string to send to the console before trying
 to read a new line
 """
-_console_interaction(self, success_message, failure_message,
+self._console_interaction(success_message, failure_message,
  interrupt_string, True)
 
 def wait_for_console_pattern(self, success_message, failure_message=None,
@@ -158,7 +157,7 @@ def wait_for_console_pattern(self, success_message, 
failure_message=None,
 :param success_message: if this message appears, test succeeds
 :param failure_message: if this message appears, test fails
 """
-_console_interaction(self, success_message, failure_message, None, 
vm=vm)
+self._console_interaction(success_message, failure_message, None, 
vm=vm)
 
 class Test(avocado.Test):
 def _get_unique_tag_val(self, tag_name):
-- 
2.29.2

[PATCH 3/7] tests/acceptance: Move exec_command_and_wait_for_pattern to ConsoleMixIn

2021-05-03 Thread Wainer dos Santos Moschetta

It was the time of exec_command_and_wait_for_pattern() to find a new
home at ConsoleMixIn. This time various tests needed to be adapted.

Signed-off-by: Wainer dos Santos Moschetta 
---
 tests/acceptance/avocado_qemu/__init__.py   |  29 +++---
 tests/acceptance/boot_linux_console.py  | 107 ++--
 tests/acceptance/machine_rx_gdbsim.py   |   8 +-
 tests/acceptance/machine_s390_ccw_virtio.py |  72 ++---
 tests/acceptance/multiprocess.py|   3 +-
 tests/acceptance/virtio-gpu.py  |  12 +--
 6 files changed, 111 insertions(+), 120 deletions(-)

diff --git a/tests/acceptance/avocado_qemu/__init__.py 
b/tests/acceptance/avocado_qemu/__init__.py
index 4d3b869765..4a0129c0eb 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -114,22 +114,6 @@ def wait_for_console_pattern(test, success_message, 
failure_message=None,
 """
 _console_interaction(test, success_message, failure_message, None, vm=vm)
 
-def exec_command_and_wait_for_pattern(test, command,
-  success_message, failure_message=None):
-"""
-Send a command to a console (appending CRLF characters), then wait
-for success_message to appear on the console, while logging the.
-content. Mark the test as failed if failure_message is found instead.
-
-:param test: an Avocado test containing a VM that will have its console
- read and probed for a success or failure message
-:type test: :class:`avocado_qemu.Test`
-:param command: the command to send
-:param success_message: if this message appears, test succeeds
-:param failure_message: if this message appears, test fails
-"""
-_console_interaction(test, success_message, failure_message, command + 
'\r')
-
 class ConsoleMixIn():
 """Contains utilities for interacting with a guest via Console."""
 
@@ -143,6 +127,19 @@ def exec_command(self, command):
 """
 _console_interaction(self, None, None, command + '\r')
 
+def exec_command_and_wait_for_pattern(self, command,
+  success_message, 
failure_message=None):
+"""
+Send a command to a console (appending CRLF characters), then wait
+for success_message to appear on the console, while logging the.
+content. Mark the test as failed if failure_message is found instead.
+
+:param command: the command to send
+:param success_message: if this message appears, test succeeds
+:param failure_message: if this message appears, test fails
+"""
+_console_interaction(self, success_message, failure_message, command + 
'\r')
+
 def interrupt_interactive_console_until_pattern(self, success_message,
 failure_message=None,
 interrupt_string='\r'):
diff --git a/tests/acceptance/boot_linux_console.py 
b/tests/acceptance/boot_linux_console.py
index 10317b232b..50e0a3fe79 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -16,7 +16,6 @@
 from avocado import skip
 from avocado import skipUnless
 from avocado_qemu import Test
-from avocado_qemu import exec_command_and_wait_for_pattern
 from avocado_qemu import ConsoleMixIn
 from avocado_qemu import wait_for_console_pattern
 from avocado.utils import process
@@ -223,12 +222,12 @@ def test_mips_malta_cpio(self):
 self.vm.launch()
 self.wait_for_console_pattern('Boot successful.')
 
-exec_command_and_wait_for_pattern(self, 'cat /proc/cpuinfo',
-'BogoMIPS')
-exec_command_and_wait_for_pattern(self, 'uname -a',
-'Debian')
-exec_command_and_wait_for_pattern(self, 'reboot',
-'reboot: Restarting system')
+self.exec_command_and_wait_for_pattern('cat /proc/cpuinfo',
+   'BogoMIPS')
+self.exec_command_and_wait_for_pattern('uname -a',
+   'Debian')
+self.exec_command_and_wait_for_pattern('reboot',
+   'reboot: Restarting system')
 # Wait for VM to shut down gracefully
 self.vm.wait()
 
@@ -265,12 +264,12 @@ def test_mips64el_malta_5KEc_cpio(self):
 self.vm.launch()
 wait_for_console_pattern(self, 'Boot successful.')
 
-exec_command_and_wait_for_pattern(self, 'cat /proc/cpuinfo',
-'MIPS 5KE')
-exec_command_and_wait_for_pattern(self, 'uname -a',
-'3.19.3.mtoman.201504

[PATCH 0/7] tests/acceptance: Introducing the ConsoleMixIn

2021-05-03 Thread Wainer dos Santos Moschetta

The avocado_qemu package provides the following methods to interact with the
guest via console, which are mainly used on the acceptance boot tests:

 exec_command(), exec_command_and_wait_for_pattern(), 
wait_for_console_pattern(),
 interrupt_interactive_console_until_pattern()

Those methods are loosely defined in avocado_qemu/__init__.py. Because that 
file is expected to
grow, I thought that for the sake of keeping it organized it would be better to 
logically group
them. So in this series I create the ConsoleMixIn class to be the new home for 
the console
methods. An alternative approach could be to create a separated package, 
however because
they are just a few methods at the moment, I prefered not to break avocado_qemu 
into smaller pieces.

As the "MixIn" in the name implies, the class is meant to be used as a mixin on 
the test class. Here
I am following an idea introduced by Cleber in [1].

This series was tested on CI 
(https://gitlab.com/wainersm/qemu/-/pipelines/296412039)

[1] http://next.patchew.org/QEMU/20210412044644.55083-1-cr...@redhat.com/

Wainer dos Santos Moschetta (7):
  tests/acceptance: Introduce the ConsoleMixIn class
  tests/acceptance: Move exec_command to ConsoleMixIn
  tests/acceptance: Move exec_command_and_wait_for_pattern to
ConsoleMixIn
  tests/acceptance: Sun4uMachine: Remove dependency to LinuxKernelTest
  tests/acceptance: replay_kernel: Remove unused
wait_for_console_pattern
  tests/acceptance: Move wait_for_console_pattern to ConsoleMixIn
  tests/acceptance: Move _console_interaction to ConsoleMixIn

 tests/acceptance/avocado_qemu/__init__.py| 161 +--
 tests/acceptance/boot_linux_console.py   | 133 ---
 tests/acceptance/boot_xen.py |   5 +-
 tests/acceptance/linux_ssh_mips_malta.py |   8 +-
 tests/acceptance/machine_arm_canona1100.py   |   6 +-
 tests/acceptance/machine_arm_integratorcp.py |   8 +-
 tests/acceptance/machine_arm_n8x0.py |   6 +-
 tests/acceptance/machine_microblaze.py   |   8 +-
 tests/acceptance/machine_mips_loongson3v.py  |   6 +-
 tests/acceptance/machine_mips_malta.py   |   6 +-
 tests/acceptance/machine_ppc.py  |  10 +-
 tests/acceptance/machine_rx_gdbsim.py|  15 +-
 tests/acceptance/machine_s390_ccw_virtio.py  |  79 +
 tests/acceptance/machine_sparc64_sun4u.py|  13 +-
 tests/acceptance/machine_sparc_leon3.py  |   8 +-
 tests/acceptance/multiprocess.py |  14 +-
 tests/acceptance/ppc_prep_40p.py |  16 +-
 tests/acceptance/replay_kernel.py|   1 -
 tests/acceptance/virtio-gpu.py   |  16 +-
 tests/acceptance/virtiofs_submounts.py   |   1 -
 20 files changed, 249 insertions(+), 271 deletions(-)

-- 
2.29.2

[PATCH 1/7] tests/acceptance: Introduce the ConsoleMixIn class

2021-05-03 Thread Wainer dos Santos Moschetta

This created the ConsoleMixIn class to wrap the methods related with console
interaction with the guest that currently are loose in the avocado_qemu
package. It should be used as a mixin on the test classes.

At this point only the interrupt_interactive_console_until_pattern() was moved
to ConsoleMixIn. This method is only used in boot_linux_console.py tests, so
there was needed to adapt them.

Signed-off-by: Wainer dos Santos Moschetta 
---
 tests/acceptance/avocado_qemu/__init__.py | 52 +++
 tests/acceptance/boot_linux_console.py| 10 ++---
 2 files changed, 31 insertions(+), 31 deletions(-)

diff --git a/tests/acceptance/avocado_qemu/__init__.py 
b/tests/acceptance/avocado_qemu/__init__.py
index 83b1741ec8..6f4e0edfa3 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -101,32 +101,6 @@ def _console_interaction(test, success_message, 
failure_message,
 (failure_message, success_message)
 test.fail(fail)
 
-def interrupt_interactive_console_until_pattern(test, success_message,
-failure_message=None,
-interrupt_string='\r'):
-"""
-Keep sending a string to interrupt a console prompt, while logging the
-console output. Typical use case is to break a boot loader prompt, such:
-
-Press a key within 5 seconds to interrupt boot process.
-5
-4
-3
-2
-1
-Booting default image...
-
-:param test: an Avocado test containing a VM that will have its console
- read and probed for a success or failure message
-:type test: :class:`avocado_qemu.Test`
-:param success_message: if this message appears, test succeeds
-:param failure_message: if this message appears, test fails
-:param interrupt_string: a string to send to the console before trying
- to read a new line
-"""
-_console_interaction(test, success_message, failure_message,
- interrupt_string, True)
-
 def wait_for_console_pattern(test, success_message, failure_message=None,
  vm=None):
 """
@@ -168,6 +142,32 @@ def exec_command_and_wait_for_pattern(test, command,
 """
 _console_interaction(test, success_message, failure_message, command + 
'\r')
 
+class ConsoleMixIn():
+"""Contains utilities for interacting with a guest via Console."""
+
+def interrupt_interactive_console_until_pattern(self, success_message,
+failure_message=None,
+interrupt_string='\r'):
+"""
+Keep sending a string to interrupt a console prompt, while logging the
+console output. Typical use case is to break a boot loader prompt, 
such:
+
+Press a key within 5 seconds to interrupt boot process.
+5
+4
+3
+2
+1
+Booting default image...
+
+:param success_message: if this message appears, test succeeds
+:param failure_message: if this message appears, test fails
+:param interrupt_string: a string to send to the console before trying
+to read a new line
+"""
+_console_interaction(self, success_message, failure_message,
+ interrupt_string, True)
+
 class Test(avocado.Test):
 def _get_unique_tag_val(self, tag_name):
 """
diff --git a/tests/acceptance/boot_linux_console.py 
b/tests/acceptance/boot_linux_console.py
index 1ca32ecf25..10317b232b 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -17,7 +17,7 @@
 from avocado import skipUnless
 from avocado_qemu import Test
 from avocado_qemu import exec_command_and_wait_for_pattern
-from avocado_qemu import interrupt_interactive_console_until_pattern
+from avocado_qemu import ConsoleMixIn
 from avocado_qemu import wait_for_console_pattern
 from avocado.utils import process
 from avocado.utils import archive
@@ -45,7 +45,7 @@ def image_pow2ceil_expand(path):
 with open(path, 'ab+') as fd:
 fd.truncate(size_aligned)
 
-class LinuxKernelTest(Test):
+class LinuxKernelTest(Test, ConsoleMixIn):
 KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
 
 def wait_for_console_pattern(self, success_message, vm=None):
@@ -626,8 +626,8 @@ def test_arm_quanta_gsj(self):
 self.wait_for_console_pattern('>Device: Poleg BMC NPCM730')
 self.wait_for_console_pattern('>Skip DDR init.')
 self.wait_for_console_pattern('U-Boot ')
-interrupt_interactive_console_until_pattern(
-self, 'Hit any key t

[PATCH 5/7] tests/acceptance: replay_kernel: Remove unused wait_for_console_pattern

2021-05-03 Thread Wainer dos Santos Moschetta

The ReplayKernelBase class uses the wait_for_console_pattern from its
parent LinuxKernelTest class, thus it doesn't need to import that method
from avocado_qemu.

Signed-off-by: Wainer dos Santos Moschetta 
---
 tests/acceptance/replay_kernel.py | 1 -
 1 file changed, 1 deletion(-)

diff --git a/tests/acceptance/replay_kernel.py 
b/tests/acceptance/replay_kernel.py
index 71facdaa75..6d4e202723 100644
--- a/tests/acceptance/replay_kernel.py
+++ b/tests/acceptance/replay_kernel.py
@@ -17,7 +17,6 @@
 from avocado import skip
 from avocado import skipIf
 from avocado import skipUnless
-from avocado_qemu import wait_for_console_pattern
 from avocado.utils import archive
 from avocado.utils import process
 from boot_linux_console import LinuxKernelTest
-- 
2.29.2

[PATCH 2/7] tests/acceptance: Move exec_command to ConsoleMixIn

2021-05-03 Thread Wainer dos Santos Moschetta

This moved exec_command() to ConsoleMixIn class.

Only the multiprocess.py file were touched by that change, so its tests
were adapted.

Signed-off-by: Wainer dos Santos Moschetta 
---
 tests/acceptance/avocado_qemu/__init__.py | 22 ++
 tests/acceptance/multiprocess.py  |  6 +++---
 2 files changed, 13 insertions(+), 15 deletions(-)

diff --git a/tests/acceptance/avocado_qemu/__init__.py 
b/tests/acceptance/avocado_qemu/__init__.py
index 6f4e0edfa3..4d3b869765 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -114,18 +114,6 @@ def wait_for_console_pattern(test, success_message, 
failure_message=None,
 """
 _console_interaction(test, success_message, failure_message, None, vm=vm)
 
-def exec_command(test, command):
-"""
-Send a command to a console (appending CRLF characters), while logging
-the content.
-
-:param test: an Avocado test containing a VM.
-:type test: :class:`avocado_qemu.Test`
-:param command: the command to send
-:type command: str
-"""
-_console_interaction(test, None, None, command + '\r')
-
 def exec_command_and_wait_for_pattern(test, command,
   success_message, failure_message=None):
 """
@@ -145,6 +133,16 @@ def exec_command_and_wait_for_pattern(test, command,
 class ConsoleMixIn():
 """Contains utilities for interacting with a guest via Console."""
 
+def exec_command(self, command):
+"""
+Send a command to a console (appending CRLF characters), while logging
+the content.
+
+:param command: the command to send
+:type command: str
+"""
+_console_interaction(self, None, None, command + '\r')
+
 def interrupt_interactive_console_until_pattern(self, success_message,
 failure_message=None,
 interrupt_string='\r'):
diff --git a/tests/acceptance/multiprocess.py b/tests/acceptance/multiprocess.py
index 96627f022a..41d3e51164 100644
--- a/tests/acceptance/multiprocess.py
+++ b/tests/acceptance/multiprocess.py
@@ -9,10 +9,10 @@
 
 from avocado_qemu import Test
 from avocado_qemu import wait_for_console_pattern
-from avocado_qemu import exec_command
+from avocado_qemu import ConsoleMixIn
 from avocado_qemu import exec_command_and_wait_for_pattern
 
-class Multiprocess(Test):
+class Multiprocess(Test, ConsoleMixIn):
 """
 :avocado: tags=multiprocess
 """
@@ -59,7 +59,7 @@ def do_test(self, kernel_url, initrd_url, kernel_command_line,
 self.vm.launch()
 wait_for_console_pattern(self, 'as init process',
  'Kernel panic - not syncing')
-exec_command(self, 'mount -t sysfs sysfs /sys')
+self.exec_command('mount -t sysfs sysfs /sys')
 exec_command_and_wait_for_pattern(self,
   'cat /sys/bus/pci/devices/*/uevent',
   'PCI_ID=1000:0012')
-- 
2.29.2

[PATCH 4/7] tests/acceptance: Sun4uMachine: Remove dependency to LinuxKernelTest

2021-05-03 Thread Wainer dos Santos Moschetta

The Sun4uMachine class inherit from LinuxKernelTest to effectively only use
the KERNEL_COMMON_COMMAND_LINE attribute. This change remove that unneeded
dependency, making Sun4uMachine self-content.

I took the occasion to delint the code: the unused os import was
removed, imports were reordered, and the module has a docstring now.

Signed-off-by: Wainer dos Santos Moschetta 
---
 tests/acceptance/machine_sparc64_sun4u.py | 11 +--
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/tests/acceptance/machine_sparc64_sun4u.py 
b/tests/acceptance/machine_sparc64_sun4u.py
index 458165500e..c7ad474bdc 100644
--- a/tests/acceptance/machine_sparc64_sun4u.py
+++ b/tests/acceptance/machine_sparc64_sun4u.py
@@ -1,4 +1,4 @@
-# Functional test that boots a Linux kernel and checks the console
+"""Functional test that boots a Linux kernel and checks the console"""
 #
 # Copyright (c) 2020 Red Hat, Inc.
 #
@@ -8,16 +8,15 @@
 # This work is licensed under the terms of the GNU GPL, version 2 or
 # later. See the COPYING file in the top-level directory.
 
-import os
-
-from avocado_qemu import wait_for_console_pattern
 from avocado.utils import archive
-from boot_linux_console import LinuxKernelTest
+from avocado_qemu import Test
+from avocado_qemu import wait_for_console_pattern
 
-class Sun4uMachine(LinuxKernelTest):
+class Sun4uMachine(Test):
 """Boots the Linux kernel and checks that the console is operational"""
 
 timeout = 90
+KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
 
 def test_sparc64_sun4u(self):
 """
-- 
2.29.2

[PATCH v3 4/7] tests/acceptance: Tagging tests with "cpu:VALUE"

2021-04-30 Thread Wainer dos Santos Moschetta

The existing tests which are passing "-cpu VALUE" argument to the vm object
are now properly "cpu:VALUE" tagged, so letting the avocado_qemu framework to
handle that automatically.

Reviewed-by: Cleber Rosa 
Reviewed-by: Willian Rampazzo 
Signed-off-by: Wainer dos Santos Moschetta 
---
 tests/acceptance/boot_linux_console.py   | 16 +---
 tests/acceptance/pc_cpu_hotplug_props.py |  2 +-
 tests/acceptance/replay_kernel.py|  9 ++---
 tests/acceptance/virtio-gpu.py   |  4 ++--
 4 files changed, 18 insertions(+), 13 deletions(-)

diff --git a/tests/acceptance/boot_linux_console.py 
b/tests/acceptance/boot_linux_console.py
index 1ca32ecf25..b7a856d871 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -238,6 +238,7 @@ def test_mips64el_malta_5KEc_cpio(self):
 :avocado: tags=arch:mips64el
 :avocado: tags=machine:malta
 :avocado: tags=endian:little
+:avocado: tags=cpu:5KEc
 """
 kernel_url = ('https://github.com/philmd/qemu-testing-blob/'
   'raw/9ad2df38/mips/malta/mips64el/'
@@ -257,8 +258,7 @@ def test_mips64el_malta_5KEc_cpio(self):
 kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
+ 'console=ttyS0 console=tty '
+ 'rdinit=/sbin/init noreboot')
-self.vm.add_args('-cpu', '5KEc',
- '-kernel', kernel_path,
+self.vm.add_args('-kernel', kernel_path,
  '-initrd', initrd_path,
  '-append', kernel_command_line,
  '-no-reboot')
@@ -286,7 +286,6 @@ def do_test_mips_malta32el_nanomips(self, kernel_url, 
kernel_hash):
+ 'mem=256m@@0x0 '
+ 'console=ttyS0')
 self.vm.add_args('-no-reboot',
- '-cpu', 'I7200',
  '-kernel', kernel_path,
  '-append', kernel_command_line)
 self.vm.launch()
@@ -298,6 +297,7 @@ def test_mips_malta32el_nanomips_4k(self):
 :avocado: tags=arch:mipsel
 :avocado: tags=machine:malta
 :avocado: tags=endian:little
+:avocado: tags=cpu:I7200
 """
 kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
   'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
@@ -310,6 +310,7 @@ def test_mips_malta32el_nanomips_16k_up(self):
 :avocado: tags=arch:mipsel
 :avocado: tags=machine:malta
 :avocado: tags=endian:little
+:avocado: tags=cpu:I7200
 """
 kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
   'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
@@ -322,6 +323,7 @@ def test_mips_malta32el_nanomips_64k_dbg(self):
 :avocado: tags=arch:mipsel
 :avocado: tags=machine:malta
 :avocado: tags=endian:little
+:avocado: tags=cpu:I7200
 """
 kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/'
   'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/'
@@ -333,6 +335,7 @@ def test_aarch64_virt(self):
 """
 :avocado: tags=arch:aarch64
 :avocado: tags=machine:virt
+:avocado: tags=cpu:cortex-a53
 """
 kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
   '/linux/releases/29/Everything/aarch64/os/images/pxeboot'
@@ -343,8 +346,7 @@ def test_aarch64_virt(self):
 self.vm.set_console()
 kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
'console=ttyAMA0')
-self.vm.add_args('-cpu', 'cortex-a53',
- '-kernel', kernel_path,
+self.vm.add_args('-kernel', kernel_path,
  '-append', kernel_command_line)
 self.vm.launch()
 console_pattern = 'Kernel command line: %s' % kernel_command_line
@@ -1038,9 +1040,9 @@ def test_ppc64_e500(self):
 """
 :avocado: tags=arch:ppc64
 :avocado: tags=machine:ppce500
+:avocado: tags=cpu:e5500
 """
 tar_hash = '6951d86d644b302898da2fd701739c9406527fe1'
-self.vm.add_args('-cpu', 'e5500')
 self.do_test_advcal_2018('19', tar_hash, 'uImage')
 
 def test_ppc_g3beige(self):
@@ -1082,7 +1084,7 @@ def test_xtensa_lx60(self):
 """
 :avocado: tags=arch:xtensa
 :avocado: tags=machine:lx60
+:avocado: tags=cpu:dc233c
 """
 tar_hash = '49e88d9933742f0164b60839886c9739cb7a0d34'
-self.vm.add_args('-cpu', 'dc233c')
 self.do_test_advcal_2018('02', tar_hash, 'santas-sleigh-ride.elf')
diff --git

[PATCH v3 0/7] tests/acceptance: Handle tests with "cpu" tag

2021-04-30 Thread Wainer dos Santos Moschetta

Currently the acceptance tests tagged with "machine" have the "-M TYPE"
automatically added to the list of arguments of the QEMUMachine object.
In other words, that option is passed to the launched QEMU. On this
series it is implemented the same feature but instead for tests marked
with "cpu".

There is a caveat, however, in case the test needs additional arguments to
the CPU type they cannot be passed via tag, because the tags parser split
values by comma (limitation which Avocado plans to address, see
https://github.com/avocado-framework/avocado/issues/45410). For example, in
tests/acceptance/x86_cpu_model_versions.py, there are cases where:

  * -cpu is set to 
"Cascadelake-Server,x-force-features=on,check=off,enforce=off"
  * if it was tagged like 
"cpu:Cascadelake-Server,x-force-features=on,check=off,enforce=off"
then the parser would break it into 4 tags ("cpu:Cascadelake-Server",
"x-force-features=on", "check=off", "enforce=off")
  * resulting on "-cpu Cascadelake-Server" and the remaining arguments are 
ignored.

It was introduced the avocado_qemu.Test.set_vm_arg() method to deal with
cases like the example above, so that one can tag it as "cpu:Cascadelake-Server"
AND call self.set_vm_args('-cpu', 
"Cascadelake-Server,x-force-features=on,check=off,enforce=off"),
and that results on the reset of the initial value of -cpu.

This series was tested on CI 
(https://gitlab.com/wainersm/qemu/-/pipelines/294640198)
and with the following code:

from avocado_qemu import Test

class CPUTest(Test):
def test_cpu(self):
"""
:avocado: tags=cpu:host
"""
# The cpu property is set to the tag value, or None on its absence
self.assertEqual(self.cpu, "host")
# The created VM has the '-cpu host' option
self.assertIn("-cpu host", " ".join(self.vm._args))
self.vm.launch()

def test_cpu_none(self):
self.assertEqual(self.cpu, None)
self.assertNotIn('-cpu', self.vm._args)

def test_cpu_reset(self):
"""
:avocado: tags=cpu:host
"""
self.assertIn("-cpu host", " ".join(self.vm._args))
self.set_vm_arg("-cpu", "Cascadelake-Server,x-force-features=on")
self.assertNotIn("-cpu host", " ".join(self.vm._args))
self.assertIn("-cpu Cascadelake-Server,x-force-features=on", " 
".join(self.vm._args))

Changes:
 - v2 -> v3:
   - The arg and value parameters of set_vm_arg() are now mandatories and
 fixed an index out of bounds bug [crosa]
   - Rebased. Needed to adapt the (new) boot_xen.py test (patch 03)
 - v1 -> v2:
   - Recognize the cpu value passed via test parameter [crosa]
   - Fixed tags (patch 02) on preparation to patch 03 [crosa]
   - Added QEMUMachine.args property (patch 04) so that _args could be handled
 without pylint complaining (protected property)
   - Added Test.set_vm_arg() (patch 05) to handle the corner case [crosa]

Wainer dos Santos Moschetta (7):
  tests/acceptance: Automatic set -cpu to the test vm
  tests/acceptance: Fix mismatch on cpu tagged tests
  tests/acceptance: Let the framework handle "cpu:VALUE" tagged tests
  tests/acceptance: Tagging tests with "cpu:VALUE"
  python/qemu: Add args property to the QEMUMachine class
  tests/acceptance: Add set_vm_arg() to the Test class
  tests/acceptance: Handle cpu tag on x86_cpu_model_versions tests

 docs/devel/testing.rst | 17 +
 python/qemu/machine.py |  5 +++
 tests/acceptance/avocado_qemu/__init__.py  | 26 ++
 tests/acceptance/boot_linux.py |  3 --
 tests/acceptance/boot_linux_console.py | 16 +
 tests/acceptance/boot_xen.py   |  1 -
 tests/acceptance/machine_mips_malta.py |  7 ++--
 tests/acceptance/pc_cpu_hotplug_props.py   |  2 +-
 tests/acceptance/replay_kernel.py  | 17 -
 tests/acceptance/reverse_debugging.py  |  2 +-
 tests/acceptance/tcg_plugins.py| 15 
 tests/acceptance/virtio-gpu.py |  4 +--
 tests/acceptance/x86_cpu_model_versions.py | 40 +-
 13 files changed, 112 insertions(+), 43 deletions(-)

-- 
2.29.2

[PATCH v3 3/7] tests/acceptance: Let the framework handle "cpu:VALUE" tagged tests

2021-04-30 Thread Wainer dos Santos Moschetta

The tests that are already tagged with "cpu:VALUE" don't need to add
"-cpu VALUE" to the list of arguments of the vm object because the avocado_qemu
framework is able to handle it automatically.

Reviewed-by: Cleber Rosa 
Tested-by: Cleber Rosa 
Reviewed-by: Willian Rampazzo 
Signed-off-by: Wainer dos Santos Moschetta 
---
 tests/acceptance/boot_linux.py | 3 ---
 tests/acceptance/boot_xen.py   | 1 -
 tests/acceptance/machine_mips_malta.py | 1 -
 tests/acceptance/replay_kernel.py  | 8 +++-
 tests/acceptance/reverse_debugging.py  | 2 +-
 tests/acceptance/tcg_plugins.py| 9 -
 6 files changed, 8 insertions(+), 16 deletions(-)

diff --git a/tests/acceptance/boot_linux.py b/tests/acceptance/boot_linux.py
index 0d178038a0..55637d126e 100644
--- a/tests/acceptance/boot_linux.py
+++ b/tests/acceptance/boot_linux.py
@@ -82,7 +82,6 @@ def test_virt_tcg(self):
 """
 self.require_accelerator("tcg")
 self.vm.add_args("-accel", "tcg")
-self.vm.add_args("-cpu", "max")
 self.vm.add_args("-machine", "virt,gic-version=2")
 self.add_common_args()
 self.launch_and_wait()
@@ -95,7 +94,6 @@ def test_virt_kvm_gicv2(self):
 """
 self.require_accelerator("kvm")
 self.vm.add_args("-accel", "kvm")
-self.vm.add_args("-cpu", "host")
 self.vm.add_args("-machine", "virt,gic-version=2")
 self.add_common_args()
 self.launch_and_wait()
@@ -108,7 +106,6 @@ def test_virt_kvm_gicv3(self):
 """
 self.require_accelerator("kvm")
 self.vm.add_args("-accel", "kvm")
-self.vm.add_args("-cpu", "host")
 self.vm.add_args("-machine", "virt,gic-version=3")
 self.add_common_args()
 self.launch_and_wait()
diff --git a/tests/acceptance/boot_xen.py b/tests/acceptance/boot_xen.py
index 75c2d44492..3479b5233b 100644
--- a/tests/acceptance/boot_xen.py
+++ b/tests/acceptance/boot_xen.py
@@ -48,7 +48,6 @@ def launch_xen(self, xen_path):
 
 xen_command_line = self.XEN_COMMON_COMMAND_LINE
 self.vm.add_args('-machine', 'virtualization=on',
- '-cpu', 'cortex-a57',
  '-m', '768',
  '-kernel', xen_path,
  '-append', xen_command_line,
diff --git a/tests/acceptance/machine_mips_malta.py 
b/tests/acceptance/machine_mips_malta.py
index b1fd075f51..b67d8cb141 100644
--- a/tests/acceptance/machine_mips_malta.py
+++ b/tests/acceptance/machine_mips_malta.py
@@ -62,7 +62,6 @@ def do_test_i6400_framebuffer_logo(self, cpu_cores_count):
 kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
'clocksource=GIC console=tty0 console=ttyS0')
 self.vm.add_args('-kernel', kernel_path,
- '-cpu', 'I6400',
  '-smp', '%u' % cpu_cores_count,
  '-vga', 'std',
  '-append', kernel_command_line)
diff --git a/tests/acceptance/replay_kernel.py 
b/tests/acceptance/replay_kernel.py
index 71facdaa75..75f80506c1 100644
--- a/tests/acceptance/replay_kernel.py
+++ b/tests/acceptance/replay_kernel.py
@@ -156,8 +156,7 @@ def test_aarch64_virt(self):
'console=ttyAMA0')
 console_pattern = 'VFS: Cannot open root device'
 
-self.run_rr(kernel_path, kernel_command_line, console_pattern,
-args=('-cpu', 'cortex-a53'))
+self.run_rr(kernel_path, kernel_command_line, console_pattern)
 
 def test_arm_virt(self):
 """
@@ -301,7 +300,7 @@ def test_ppc64_e500(self):
 tar_url = ('https://www.qemu-advent-calendar.org'
'/2018/download/day19.tar.xz')
 file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
-self.do_test_advcal_2018(file_path, 'uImage', ('-cpu', 'e5500'))
+self.do_test_advcal_2018(file_path, 'uImage')
 
 def test_ppc_g3beige(self):
 """
@@ -348,8 +347,7 @@ def test_xtensa_lx60(self):
 tar_url = ('https://www.qemu-advent-calendar.org'
'/2018/download/day02.tar.xz')
 file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
-self.do_test_advcal_2018(file_path, 'santas-sleigh-ride.elf',
- args=('-cpu', 'dc233c'))
+self.do_test_advcal_2018(file_path, 'santas-sleigh-ride.elf')
 
 @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might timeout')
 class ReplayKernelSlow(ReplayKernelBase):
diff --git a/tests/acceptance/reverse_debugging.py 
b/tests/acceptance/reverse_debugging.py
index be01aca217..d2921e70c3 100644
-

[PATCH v3 7/7] tests/acceptance: Handle cpu tag on x86_cpu_model_versions tests

2021-04-30 Thread Wainer dos Santos Moschetta

Some test cases on x86_cpu_model_versions.py are corner cases because they
need to pass extra options to the -cpu argument. Once the avocado_qemu
framework will set -cpu automatically, the value should be reset. This changed
those tests so to call set_vm_arg() to overwrite the -cpu value.

Signed-off-by: Wainer dos Santos Moschetta 
---
 tests/acceptance/x86_cpu_model_versions.py | 40 +-
 1 file changed, 32 insertions(+), 8 deletions(-)

diff --git a/tests/acceptance/x86_cpu_model_versions.py 
b/tests/acceptance/x86_cpu_model_versions.py
index 77ed8597a4..0e9feda62d 100644
--- a/tests/acceptance/x86_cpu_model_versions.py
+++ b/tests/acceptance/x86_cpu_model_versions.py
@@ -252,10 +252,13 @@ def get_cpu_prop(self, prop):
 def test_4_1(self):
 """
 :avocado: tags=machine:pc-i440fx-4.1
+:avocado: tags=cpu:Cascadelake-Server
 """
 # machine-type only:
 self.vm.add_args('-S')
-self.vm.add_args('-cpu', 
'Cascadelake-Server,x-force-features=on,check=off,enforce=off')
+self.set_vm_arg('-cpu',
+'Cascadelake-Server,x-force-features=on,check=off,'
+'enforce=off')
 self.vm.launch()
 self.assertFalse(self.get_cpu_prop('arch-capabilities'),
  'pc-i440fx-4.1 + Cascadelake-Server should not have 
arch-capabilities')
@@ -263,9 +266,12 @@ def test_4_1(self):
 def test_4_0(self):
 """
 :avocado: tags=machine:pc-i440fx-4.0
+:avocado: tags=cpu:Cascadelake-Server
 """
 self.vm.add_args('-S')
-self.vm.add_args('-cpu', 
'Cascadelake-Server,x-force-features=on,check=off,enforce=off')
+self.set_vm_arg('-cpu',
+'Cascadelake-Server,x-force-features=on,check=off,'
+'enforce=off')
 self.vm.launch()
 self.assertFalse(self.get_cpu_prop('arch-capabilities'),
  'pc-i440fx-4.0 + Cascadelake-Server should not have 
arch-capabilities')
@@ -273,10 +279,13 @@ def test_4_0(self):
 def test_set_4_0(self):
 """
 :avocado: tags=machine:pc-i440fx-4.0
+:avocado: tags=cpu:Cascadelake-Server
 """
 # command line must override machine-type if CPU model is not 
versioned:
 self.vm.add_args('-S')
-self.vm.add_args('-cpu', 
'Cascadelake-Server,x-force-features=on,check=off,enforce=off,+arch-capabilities')
+self.set_vm_arg('-cpu',
+'Cascadelake-Server,x-force-features=on,check=off,'
+'enforce=off,+arch-capabilities')
 self.vm.launch()
 self.assertTrue(self.get_cpu_prop('arch-capabilities'),
 'pc-i440fx-4.0 + Cascadelake-Server,+arch-capabilities 
should have arch-capabilities')
@@ -284,9 +293,12 @@ def test_set_4_0(self):
 def test_unset_4_1(self):
 """
 :avocado: tags=machine:pc-i440fx-4.1
+:avocado: tags=cpu:Cascadelake-Server
 """
 self.vm.add_args('-S')
-self.vm.add_args('-cpu', 
'Cascadelake-Server,x-force-features=on,check=off,enforce=off,-arch-capabilities')
+self.set_vm_arg('-cpu',
+'Cascadelake-Server,x-force-features=on,check=off,'
+'enforce=off,-arch-capabilities')
 self.vm.launch()
 self.assertFalse(self.get_cpu_prop('arch-capabilities'),
  'pc-i440fx-4.1 + 
Cascadelake-Server,-arch-capabilities should not have arch-capabilities')
@@ -294,10 +306,13 @@ def test_unset_4_1(self):
 def test_v1_4_0(self):
 """
 :avocado: tags=machine:pc-i440fx-4.0
+:avocado: tags=cpu:Cascadelake-Server
 """
 # versioned CPU model overrides machine-type:
 self.vm.add_args('-S')
-self.vm.add_args('-cpu', 
'Cascadelake-Server-v1,x-force-features=on,check=off,enforce=off')
+self.set_vm_arg('-cpu',
+'Cascadelake-Server-v1,x-force-features=on,check=off,'
+'enforce=off')
 self.vm.launch()
 self.assertFalse(self.get_cpu_prop('arch-capabilities'),
  'pc-i440fx-4.0 + Cascadelake-Server-v1 should not 
have arch-capabilities')
@@ -305,9 +320,12 @@ def test_v1_4_0(self):
 def test_v2_4_0(self):
 """
 :avocado: tags=machine:pc-i440fx-4.0
+:avocado: tags=cpu:Cascadelake-Server
 """
 self.vm.add_args('-S')
-self.vm.add_args('-cpu', 
'Cascadelake-Server-v2,x-force-features=on,check=off,enforce=off')
+self.set_vm_arg('-cpu',
+'Cascadelake-Server-v2,x-force-features=on,check=off,'
+'enforce=off')
 self.vm.launch()

[PATCH v3 5/7] python/qemu: Add args property to the QEMUMachine class

2021-04-30 Thread Wainer dos Santos Moschetta

This added the args property to QEMUMachine so that users of the class
can access and handle the list of arguments to be given to the QEMU
binary.

Reviewed-by: Cleber Rosa 
Reviewed-by: Willian Rampazzo 
Signed-off-by: Wainer dos Santos Moschetta 
---
 python/qemu/machine.py | 5 +
 1 file changed, 5 insertions(+)

diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index 6e44bda337..1c30bde99d 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -302,6 +302,11 @@ def _base_args(self) -> List[str]:
 args.extend(['-device', device])
 return args
 
+@property
+def args(self) -> List[str]:
+"""Returns the list of arguments given to the QEMU binary."""
+return self._args
+
 def _pre_launch(self) -> None:
 self._temp_dir = tempfile.mkdtemp(prefix="qemu-machine-",
   dir=self._test_dir)
-- 
2.29.2

[PATCH v3 1/7] tests/acceptance: Automatic set -cpu to the test vm

2021-04-30 Thread Wainer dos Santos Moschetta

This introduces a new feature to the functional tests: automatic setting of
the '-cpu VALUE' option to the created vm if the test is tagged with
'cpu:VALUE'. The 'cpu' property is made available to the test object as well.

For example, for a simple test as:

def test(self):
"""
:avocado: tags=cpu:host
"""
self.assertEqual(self.cpu, "host")
self.vm.launch()

The resulting QEMU evocation will be like:

qemu-system-x86_64 -display none -vga none \
-chardev 
socket,id=mon,path=/var/tmp/avo_qemu_sock_pdgzbgd_/qemu-1135557-monitor.sock \
-mon chardev=mon,mode=control -cpu host

Reviewed-by: Cleber Rosa 
Tested-by: Cleber Rosa 
Reviewed-by: Willian Rampazzo 
Signed-off-by: Wainer dos Santos Moschetta 
---
 docs/devel/testing.rst| 17 +
 tests/acceptance/avocado_qemu/__init__.py |  5 +
 2 files changed, 22 insertions(+)

diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 1da4c4e4c4..e139a618f5 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -878,6 +878,17 @@ name.  If one is not given explicitly, it will either be 
set to
 ``None``, or, if the test is tagged with one (and only one)
 ``:avocado: tags=arch:VALUE`` tag, it will be set to ``VALUE``.
 
+cpu
+~~~
+
+The cpu model that will be set to all QEMUMachine instances created
+by the test.
+
+The ``cpu`` attribute will be set to the test parameter of the same
+name. If one is not given explicitly, it will either be set to
+``None ``, or, if the test is tagged with one (and only one)
+``:avocado: tags=cpu:VALUE`` tag, it will be set to ``VALUE``.
+
 machine
 ~~~
 
@@ -924,6 +935,12 @@ architecture of a kernel or disk image to boot a VM with.
 This parameter has a direct relation with the ``arch`` attribute.  If
 not given, it will default to None.
 
+cpu
+~~~
+
+The cpu model that will be set to all QEMUMachine instances created
+by the test.
+
 machine
 ~~~
 
diff --git a/tests/acceptance/avocado_qemu/__init__.py 
b/tests/acceptance/avocado_qemu/__init__.py
index 83b1741ec8..7f8e703757 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -206,6 +206,9 @@ def setUp(self):
 self.arch = self.params.get('arch',
 default=self._get_unique_tag_val('arch'))
 
+self.cpu = self.params.get('cpu',
+   default=self._get_unique_tag_val('cpu'))
+
 self.machine = self.params.get('machine',

default=self._get_unique_tag_val('machine'))
 
@@ -231,6 +234,8 @@ def get_vm(self, *args, name=None):
 name = str(uuid.uuid4())
 if self._vms.get(name) is None:
 self._vms[name] = self._new_vm(*args)
+if self.cpu is not None:
+self._vms[name].add_args('-cpu', self.cpu)
 if self.machine is not None:
 self._vms[name].set_machine(self.machine)
 return self._vms[name]
-- 
2.29.2

[PATCH v3 2/7] tests/acceptance: Fix mismatch on cpu tagged tests

2021-04-30 Thread Wainer dos Santos Moschetta

There are test cases on machine_mips_malta.py and tcg_plugins.py files
where the cpu tag does not correspond to the value actually given to the QEMU
binary. This fixed those tests tags.

Reviewed-by: Cleber Rosa 
Tested-by: Cleber Rosa 
Reviewed-by: Willian Rampazzo 
Signed-off-by: Wainer dos Santos Moschetta 
---
 tests/acceptance/machine_mips_malta.py | 6 +++---
 tests/acceptance/tcg_plugins.py| 6 +++---
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/tests/acceptance/machine_mips_malta.py 
b/tests/acceptance/machine_mips_malta.py
index 7c9a4ee4d2..b1fd075f51 100644
--- a/tests/acceptance/machine_mips_malta.py
+++ b/tests/acceptance/machine_mips_malta.py
@@ -96,7 +96,7 @@ def test_mips_malta_i6400_framebuffer_logo_1core(self):
 """
 :avocado: tags=arch:mips64el
 :avocado: tags=machine:malta
-:avocado: tags=cpu:i6400
+:avocado: tags=cpu:I6400
 """
 self.do_test_i6400_framebuffer_logo(1)
 
@@ -105,7 +105,7 @@ def test_mips_malta_i6400_framebuffer_logo_7cores(self):
 """
 :avocado: tags=arch:mips64el
 :avocado: tags=machine:malta
-:avocado: tags=cpu:i6400
+:avocado: tags=cpu:I6400
 :avocado: tags=mips:smp
 """
 self.do_test_i6400_framebuffer_logo(7)
@@ -115,7 +115,7 @@ def test_mips_malta_i6400_framebuffer_logo_8cores(self):
 """
 :avocado: tags=arch:mips64el
 :avocado: tags=machine:malta
-:avocado: tags=cpu:i6400
+:avocado: tags=cpu:I6400
 :avocado: tags=mips:smp
 """
 self.do_test_i6400_framebuffer_logo(8)
diff --git a/tests/acceptance/tcg_plugins.py b/tests/acceptance/tcg_plugins.py
index c21bf9e52a..aa6e18b62d 100644
--- a/tests/acceptance/tcg_plugins.py
+++ b/tests/acceptance/tcg_plugins.py
@@ -68,7 +68,7 @@ def test_aarch64_virt_insn(self):
 :avocado: tags=accel:tcg
 :avocado: tags=arch:aarch64
 :avocado: tags=machine:virt
-:avocado: tags=cpu:cortex-a57
+:avocado: tags=cpu:cortex-a53
 """
 kernel_path = self._grab_aarch64_kernel()
 kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
@@ -95,7 +95,7 @@ def test_aarch64_virt_insn_icount(self):
 :avocado: tags=accel:tcg
 :avocado: tags=arch:aarch64
 :avocado: tags=machine:virt
-:avocado: tags=cpu:cortex-a57
+:avocado: tags=cpu:cortex-a53
 """
 kernel_path = self._grab_aarch64_kernel()
 kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
@@ -121,7 +121,7 @@ def test_aarch64_virt_mem_icount(self):
 :avocado: tags=accel:tcg
 :avocado: tags=arch:aarch64
 :avocado: tags=machine:virt
-:avocado: tags=cpu:cortex-a57
+:avocado: tags=cpu:cortex-a53
 """
 kernel_path = self._grab_aarch64_kernel()
 kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
-- 
2.29.2

[PATCH v3 6/7] tests/acceptance: Add set_vm_arg() to the Test class

2021-04-30 Thread Wainer dos Santos Moschetta

The set_vm_arg method is added to avocado_qemu.Test class on this
change. Use that method to set (or replace) an argument to the list of
arguments given to the QEMU binary.

Suggested-by: Cleber Rosa 
Signed-off-by: Wainer dos Santos Moschetta 
---
 tests/acceptance/avocado_qemu/__init__.py | 21 +
 1 file changed, 21 insertions(+)

diff --git a/tests/acceptance/avocado_qemu/__init__.py 
b/tests/acceptance/avocado_qemu/__init__.py
index 7f8e703757..14c6ae70c8 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -240,6 +240,27 @@ def get_vm(self, *args, name=None):
 self._vms[name].set_machine(self.machine)
 return self._vms[name]
 
+def set_vm_arg(self, arg, value):
+"""
+Set an argument to list of extra arguments to be given to the QEMU
+binary. If the argument already exists then its value is replaced.
+
+:param arg: the QEMU argument, such as "-cpu" in "-cpu host"
+:type arg: str
+:param value: the argument value, such as "host" in "-cpu host"
+:type value: str
+"""
+if not arg or not value:
+return
+if arg not in self.vm.args:
+self.vm.args.extend([arg, value])
+else:
+idx = self.vm.args.index(arg) + 1
+if idx < len(self.vm.args):
+self.vm.args[idx] = value
+else:
+self.vm.args.append(value)
+
 def tearDown(self):
 for vm in self._vms.values():
 vm.shutdown()
-- 
2.29.2

Re: [PATCH v2 2/2] gitlab-ci: Add a job to build virtiofsd standalone

2021-04-29 Thread Wainer dos Santos Moschetta


Hi,

On 4/29/21 5:33 AM, Philippe Mathieu-Daudé wrote:

Add a job which builds virtiofsd without any emulation or tool.

Signed-off-by: Philippe Mathieu-Daudé 
---
https://gitlab.com/philmd/qemu/-/jobs/1222007991
Duration: 7 minutes 48 seconds
---
  .gitlab-ci.yml | 13 +
  1 file changed, 13 insertions(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 52d65d6c04f..ba3c7ade6ca 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -800,6 +800,19 @@ build-libvhost-user:
  - meson
  - ninja
  
+build-virtiofsd-fedora:

+  <<: *native_build_job_definition
+  needs:
+job: amd64-fedora-container
+  variables:
+IMAGE: fedora
+CONFIGURE_ARGS: --enable-virtiofsd
+--disable-system --disable-user --disable-tools --disable-docs
+  artifacts:
+expire_in: 2 days
+paths:
+  - build/tools/virtiofsd/virtiofsd
+


Why it needs the executable archived?

- Wainer


  # No targets are built here, just tools, docs, and unit tests. This
  # also feeds into the eventual documentation deployment steps later
  build-tools-and-docs-debian:

Re: [PATCH] tests/migration: fix unix socket migration

2021-04-28 Thread Wainer dos Santos Moschetta


Cleber,

Maybe you could review then queue this one?

- Wainer

On 4/20/21 10:16 PM, Hyman Huang wrote:


在 2021/3/10 0:55, Philippe Mathieu-Daudé 写道:

On 3/9/21 5:00 PM, huang...@chinatelecom.cn wrote:

From: Hyman 

The test aborts and error message as the following be throwed:
"No such file or directory: '/var/tmp/qemu-migrate-{pid}.migrate",
when the unix socket migration test nearly done. The reason is
qemu removes the unix socket file after migration before
guestperf.py script do it. So pre-check if the socket file exists
when removing it to prevent the guestperf program from aborting.

Signed-off-by: Hyman 
---
  tests/migration/guestperf/engine.py | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)


Interesting, we have in MAINTAINERS:

Python scripts
M: Eduardo Habkost 
M: Cleber Rosa 
S: Odd Fixes
F: scripts/*.py
F: tests/*.py

However:

./scripts/get_maintainer.pl -f tests/migration/guestperf/engine.py
get_maintainer.pl: No maintainers found, printing recent contributors.
get_maintainer.pl: Do not blindly cc: them on patches!  Use common 
sense.


Ping

The following patch has fixed it
https://patchew.org/QEMU/91d5978357fb8709ef61d2030984f7142847037d.1616141556.git.huang...@chinatelecom.cn/ 





diff --git a/tests/migration/guestperf/engine.py 
b/tests/migration/guestperf/engine.py

index 83bfc3b..86d4f21 100644
--- a/tests/migration/guestperf/engine.py
+++ b/tests/migration/guestperf/engine.py
@@ -405,7 +405,7 @@ def run(self, hardware, scenario, 
result_dir=os.getcwd()):

  progress_history = ret[0]
  qemu_timings = ret[1]
  vcpu_timings = ret[2]
-    if uri[0:5] == "unix:":
+    if uri[0:5] == "unix:" and os.path.exists(uri[5:]):
  os.remove(uri[5:])
  if self._verbose:
  print("Finished migration")



Reviewed-by: Philippe Mathieu-Daudé

Re: [PATCH v2 1/7] tests/acceptance: Automatic set -cpu to the test vm

2021-04-28 Thread Wainer dos Santos Moschetta


Hi,

On 4/21/21 5:16 PM, Cleber Rosa wrote:

On Thu, Apr 08, 2021 at 04:52:31PM -0300, Wainer dos Santos Moschetta wrote:

This introduces a new feature to the functional tests: automatic setting of
the '-cpu VALUE' option to the created vm if the test is tagged with
'cpu:VALUE'. The 'cpu' property is made available to the test object as well.

For example, for a simple test as:

 def test(self):
 """
 :avocado: tags=cpu:host
 """
 self.assertEqual(self.cpu, "host")
 self.vm.launch()


So I tried a few tests with different CPU models and it works as
expected.  One minor caveat is that using "host" has side effects
in some cases, causing tests to fail because they may also require
KVM to be enabled.

But this is a generic mechanism so I don't think it should be
concerned with that.



Good point. Certainly I will consider this when reviewing new tests.





The resulting QEMU evocation will be like:

 qemu-system-x86_64 -display none -vga none -chardev 
socket,id=mon,path=/var/tmp/avo_qemu_sock_pdgzbgd_/qemu-1135557-monitor.sock 
-mon chardev=mon,mode=control -cpu host

Only thing is: can we please just break this line (I could not ignore
a 174 character line :).


Signed-off-by: Wainer dos Santos Moschetta 
---
  docs/devel/testing.rst| 17 +
  tests/acceptance/avocado_qemu/__init__.py |  5 +
  2 files changed, 22 insertions(+)

With the line broken mentioned above (which I can take care of when
queueing this patch):



I will send a v3 to address your review for patch 06, so I can take care 
of it.





Reviewed-by: Cleber Rosa 
Tested-by: Cleber Rosa 



Thanks for the reviews!

- Wainer

Re: [PATCH 2/3] Acceptance Tests: move definition of distro checksums to the framework

2021-04-19 Thread Wainer dos Santos Moschetta


Hi,

On 4/19/21 3:35 PM, Cleber Rosa wrote:

On Mon, Apr 19, 2021 at 12:25:44PM -0300, Wainer dos Santos Moschetta wrote:

Hi,

On 4/14/21 7:14 PM, Cleber Rosa wrote:

Instead of having, by default, the checksum in the tests, and the
definition of tests in the framework, let's keep them together.

A central definition for distributions is available, and it should
allow other known distros to be added more easily.

No behavior change is expected here, and tests can still define
a distro_checksum value if for some reason they want to override
the known distribution information.

Signed-off-by: Cleber Rosa 
---
   tests/acceptance/avocado_qemu/__init__.py | 34 +--
   tests/acceptance/boot_linux.py|  8 --
   2 files changed, 32 insertions(+), 10 deletions(-)

diff --git a/tests/acceptance/avocado_qemu/__init__.py 
b/tests/acceptance/avocado_qemu/__init__.py
index aae1e5bbc9..97093614d9 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -299,6 +299,30 @@ def ssh_command(self, command):
   return stdout_lines, stderr_lines
+#: A collection of known distros and their respective image checksum
+KNOWN_DISTROS = {

Do you plan to expand that mapping to record values other than checksums?
Otherwise it could be named KNOWN_DISTROS_CHECKSUMS.


Let's just say I had an intuition about it being used for other
purposes.  Talking to Eric Auger earlier this morning, he will expand
this mapping with default kernel args distros, so that he can *add*
to the common args.

Ok



+'fedora': {
+'31': {
+'x86_64':
+{'checksum': 
'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'},
+'aarch64':
+{'checksum': 
'1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49'},
+'ppc64':
+{'checksum': 
'7c3528b85a3df4b2306e892199a9e1e43f991c506f2cc390dc4efa2026ad2f58'},
+'s390x':
+{'checksum': 
'4caaab5a434fd4d1079149a072fdc7891e354f834d355069ca982fdcaf5a122d'},
+}
+}
+}
+
+
+def get_known_distro_checksum(distro, distro_version, arch):
+try:
+return 
KNOWN_DISTROS.get(distro).get(distro_version).get(arch).get('checksum')
+except AttributeError:
+return None
+
+

Currently we have a few loose methods on avocado_qemu/__init__.py, and I'm
about to send a series to wrap them in a mixin class. This series will
introduce more loose code on the file; so would you consider moving
KNOWN_DISTROS and get_known_distro_checksum() to the LinuxTest class, and
possibly making the latest a class method?


Some of our experience in "avocado.Test" revealed that users would:

   1) find it confusing to have so many methods in the class that are not useful
  to them

   2) would conflict with variables/attributes of their own

About #2, we end up turning a lot of variables atttributes into
properties so that errors would be explicit when users tried to
overwrite them unknowingly.

But, in the specific example of KNOWN_DISTROS and its expansion I
mentioned before, it may indeed make sense to have a Test or LinuxTest
method that test writers can use.  It'd probably need to be a bit more
generic and evolved than this current version though.

Maybe wait for Eric's input based on real world use case here?

Sure, we can wait on Eric's input.



   class LinuxTest(Test, LinuxSSHMixIn):
   """Facilitates having a cloud-image Linux based available.
@@ -348,14 +372,20 @@ def download_boot(self):
   vmimage.QEMU_IMG = qemu_img
   self.log.info('Downloading/preparing boot image')
+distro = 'fedora'
+distro_version = '31'
+known_distro_checksum = get_known_distro_checksum(distro,
+  distro_version,
+  self.arch)
+distro_checksum = self.distro_checksum or known_distro_checksum


distro_checksum may be None. In this case vmimage.get() will silently skip
the check? I suggest to log a warn message.



Yes, good point.  But, I think adding that warning to Avocado's
vmimage.get() method itself is even better.  What do you think?


I like the idea. IMHO avocado_qemu should be a thin layer upon Avocado, 
so such as kind of code should live on avocado's side.


I just opened the PR: https://github.com/avocado-framework/avocado/pull/4539

- Wainer



Thanks for the review!
- Cleber.

Re: [PATCH 7/8] tests/acceptance/migration.py: cancel test on s390x

2021-04-19 Thread Wainer dos Santos Moschetta




On 4/15/21 6:51 PM, Cleber Rosa wrote:

Because s390x targets it can not currently migrate without a guest
running.

Future work may provide a proper guest, but for now, it's safer to
cancel the test.

Signed-off-by: Cleber Rosa 
---
  tests/acceptance/migration.py | 6 ++
  1 file changed, 6 insertions(+)



Reviewed-by: Wainer dos Santos Moschetta 




diff --git a/tests/acceptance/migration.py b/tests/acceptance/migration.py
index b4d46becc6..4174d55c81 100644
--- a/tests/acceptance/migration.py
+++ b/tests/acceptance/migration.py
@@ -48,6 +48,12 @@ def do_migrate(self, dest_uri, src_uri=None):
  dest_vm = self.get_vm('-incoming', dest_uri)
  dest_vm.add_args('-nodefaults', '-no-shutdown')
  dest_vm.launch()
+
+cpus = dest_vm.command('query-cpus-fast')
+if cpus:
+if cpus[0].get('target') == 's390x':
+self.cancel('Migration without a guest not possible on s390')
+
  if src_uri is None:
  src_uri = dest_uri
  source_vm = self.get_vm()

Re: [PATCH 4/8] tests/acceptance/migration.py: cancel test if migration is not supported

2021-04-19 Thread Wainer dos Santos Moschetta


Hi,

On 4/15/21 6:51 PM, Cleber Rosa wrote:

FIXME: check if there's a way to query migration support before
actually requesting migration.

Some targets/machines contain devices that do not support migration.
Let's acknowledge that and cancel the test as early as possible.

Signed-off-by: Cleber Rosa 
---
  tests/acceptance/migration.py | 6 +-
  1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/tests/acceptance/migration.py b/tests/acceptance/migration.py
index 792639cb69..25ee55f36a 100644
--- a/tests/acceptance/migration.py
+++ b/tests/acceptance/migration.py
@@ -53,7 +53,11 @@ def do_migrate(self, dest_uri, src_uri=None):
  source_vm = self.get_vm()
  source_vm.add_args('-nodefaults')
  source_vm.launch()
-source_vm.qmp('migrate', uri=src_uri)
+response = source_vm.qmp('migrate', uri=src_uri)
+if 'error' in response:
+if 'desc' in response['error']:
+msg = response['error']['desc']
+self.cancel('Migration does not seem to be supported: %s' % msg)


My concern is about that cancellation actually covering up a real bug.

Cleber, have you seen the test failing on CI?

- Wainer


  self.assert_migration(source_vm, dest_vm)
  
  def _get_free_port(self):

Re: [PATCH 6/8] Acceptance tests: prevent shutdown on non-specific target tests

2021-04-19 Thread Wainer dos Santos Moschetta


Hi,

On 4/15/21 6:51 PM, Cleber Rosa wrote:

When running tests that are not target specific with various target
binaries, some specific behavior appears.  For s390x, when there's no
guest code running, it will produce GUEST_PANICKED events as the
firmware will shutdown the machine.

With this change, no GUEST_PANICKED *event* will be generated.

For some QMP commands, such as "query-migrate", a proper response
("guest-panicked" for the s390x target) will still be given.

Signed-off-by: Cleber Rosa 
---
  tests/acceptance/migration.py | 4 ++--
  tests/acceptance/version.py   | 2 +-
  2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/tests/acceptance/migration.py b/tests/acceptance/migration.py
index 25ee55f36a..b4d46becc6 100644
--- a/tests/acceptance/migration.py
+++ b/tests/acceptance/migration.py
@@ -46,12 +46,12 @@ def assert_migration(self, src_vm, dst_vm):
  
  def do_migrate(self, dest_uri, src_uri=None):

  dest_vm = self.get_vm('-incoming', dest_uri)
-dest_vm.add_args('-nodefaults')
+dest_vm.add_args('-nodefaults', '-no-shutdown')


On the other hand, can't that new argument introduce unwanted behavior 
on other targets? Maybe the argument should be set only for s390 because 
we know it prevents the test failure on that target only.


- Wainer


  dest_vm.launch()
  if src_uri is None:
  src_uri = dest_uri
  source_vm = self.get_vm()
-source_vm.add_args('-nodefaults')
+source_vm.add_args('-nodefaults', '-no-shutdown')
  source_vm.launch()
  response = source_vm.qmp('migrate', uri=src_uri)
  if 'error' in response:
diff --git a/tests/acceptance/version.py b/tests/acceptance/version.py
index 79b923d4fc..3cf18c9878 100644
--- a/tests/acceptance/version.py
+++ b/tests/acceptance/version.py
@@ -17,7 +17,7 @@ class Version(Test):
  :avocado: tags=quick
  """
  def test_qmp_human_info_version(self):
-self.vm.add_args('-nodefaults')
+self.vm.add_args('-nodefaults', '-no-shutdown')
  self.vm.launch()
  res = self.vm.command('human-monitor-command',
command_line='info version')

Re: [PATCH 5/8] tests/acceptance/cpu_queries.py: use the proper logging channels

2021-04-19 Thread Wainer dos Santos Moschetta


Hi,

On 4/15/21 6:51 PM, Cleber Rosa wrote:

The test contains methods for the proper log of test related
information.  Let's use that and remove the print and the unused
logging import.

Reference: 
https://avocado-framework.readthedocs.io/en/87.0/api/test/avocado.html#avocado.Test.log
Signed-off-by: Cleber Rosa 
---
  tests/acceptance/cpu_queries.py | 4 +---
  1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/tests/acceptance/cpu_queries.py b/tests/acceptance/cpu_queries.py
index 293dccb89a..cc9e380cc7 100644
--- a/tests/acceptance/cpu_queries.py
+++ b/tests/acceptance/cpu_queries.py
@@ -8,8 +8,6 @@
  # This work is licensed under the terms of the GNU GPL, version 2 or
  # later.  See the COPYING file in the top-level directory.
  
-import logging

-
  from avocado_qemu import Test
  
  class QueryCPUModelExpansion(Test):

@@ -27,7 +25,7 @@ def test(self):
  
  cpus = self.vm.command('query-cpu-definitions')

  for c in cpus:
-print(repr(c))
+self.log.info("Checking CPU: %s", c)
  self.assertNotIn('', c['unavailable-features'], c['name'])
  
  for c in cpus:


While you are here, maybe merge the two `for c in cpus` loops? :D

Anyway,

Reviewed-by: Wainer dos Santos Moschetta

Re: [PATCH 3/8] tests/acceptance/linux_ssh_mips_malta.py: drop identical setUp

2021-04-19 Thread Wainer dos Santos Moschetta




On 4/16/21 2:46 PM, Philippe Mathieu-Daudé wrote:

On 4/16/21 5:43 PM, Cleber Rosa wrote:

On Fri, Apr 16, 2021 at 07:26:05AM +0200, Philippe Mathieu-Daudé wrote:

On 4/15/21 11:51 PM, Cleber Rosa wrote:

These tests' setUp do not do anything beyong what their base class do.
And while they do decorate the setUp() we can decorate the classes
instead, so no functionality is lost here.

This is what I did first when adding this test, but it was not working,
so I had to duplicate it to each method. Did something change so now
this is possible?


It did, but quite a while ago:

   
https://avocado-framework.readthedocs.io/en/87.0/releases/76_0.html#users-test-writers

OK, the test is older. Do you mind adding a comment?

"Since Avocado 76.0 we can decorate setUp() directly, ..."


Ditto.

Also you may want to adjust VirtiofsSubmountsTest.setUp() in 
tests/acceptance/virtiofs_submounts.py as well.


- Wainer



Reviewed-by: Philippe Mathieu-Daudé 


It could have been updated much earlier, but, better late than never.

Sure :)

Thanks,

Phil.

Re: [PATCH 2/8] Acceptance tests: do not try to reuse packages from the system

2021-04-19 Thread Wainer dos Santos Moschetta




On 4/15/21 6:51 PM, Cleber Rosa wrote:

The premise behind the original behavior is that it would save people
from downloading Avocado (and other dependencies) if already installed
on the system.  To be honest, I think it's extremely rare that the
same versions described as dependencies will be available on most
systems.  But, the biggest motivations here are that:

  1) Hacking on QEMU in the same system used to develop Avocado leads
 to confusion with regards to the exact bits that are being used;

  2) Not reusing Python packages from system wide installations gives
 extra assurance that the same behavior will be seen from tests run
 on different machines;

With regards to downloads, pip already caches the downloaded wheels
and tarballs under ~/.cache/pip, so there should not be more than
one download even if the venv is destroyed and recreated.

Signed-off-by: Cleber Rosa 
---
  tests/Makefile.include | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)



Reviewed-by: Wainer dos Santos Moschetta 




diff --git a/tests/Makefile.include b/tests/Makefile.include
index 8f220e15d1..63477c8b4b 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -96,7 +96,7 @@ AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter 
%-softmmu,$(TARGETS)))
  
  $(TESTS_VENV_DIR): $(TESTS_VENV_REQ)

$(call quiet-command, \
-$(PYTHON) -m venv --system-site-packages $@, \
+$(PYTHON) -m venv $@, \
  VENV, $@)
$(call quiet-command, \
  $(TESTS_VENV_DIR)/bin/python -m pip -q install -r 
$(TESTS_VENV_REQ), \

Re: [PATCH 2/3] Acceptance Tests: move definition of distro checksums to the framework

2021-04-19 Thread Wainer dos Santos Moschetta


Hi,

On 4/14/21 7:14 PM, Cleber Rosa wrote:

Instead of having, by default, the checksum in the tests, and the
definition of tests in the framework, let's keep them together.

A central definition for distributions is available, and it should
allow other known distros to be added more easily.

No behavior change is expected here, and tests can still define
a distro_checksum value if for some reason they want to override
the known distribution information.

Signed-off-by: Cleber Rosa 
---
  tests/acceptance/avocado_qemu/__init__.py | 34 +--
  tests/acceptance/boot_linux.py|  8 --
  2 files changed, 32 insertions(+), 10 deletions(-)

diff --git a/tests/acceptance/avocado_qemu/__init__.py 
b/tests/acceptance/avocado_qemu/__init__.py
index aae1e5bbc9..97093614d9 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -299,6 +299,30 @@ def ssh_command(self, command):
  return stdout_lines, stderr_lines
  
  
+#: A collection of known distros and their respective image checksum

+KNOWN_DISTROS = {


Do you plan to expand that mapping to record values other than 
checksums? Otherwise it could be named KNOWN_DISTROS_CHECKSUMS.



+'fedora': {
+'31': {
+'x86_64':
+{'checksum': 
'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'},
+'aarch64':
+{'checksum': 
'1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49'},
+'ppc64':
+{'checksum': 
'7c3528b85a3df4b2306e892199a9e1e43f991c506f2cc390dc4efa2026ad2f58'},
+'s390x':
+{'checksum': 
'4caaab5a434fd4d1079149a072fdc7891e354f834d355069ca982fdcaf5a122d'},
+}
+}
+}
+
+
+def get_known_distro_checksum(distro, distro_version, arch):
+try:
+return 
KNOWN_DISTROS.get(distro).get(distro_version).get(arch).get('checksum')
+except AttributeError:
+return None
+
+


Currently we have a few loose methods on avocado_qemu/__init__.py, and 
I'm about to send a series to wrap them in a mixin class. This series 
will introduce more loose code on the file; so would you consider moving 
KNOWN_DISTROS and get_known_distro_checksum() to the LinuxTest class, 
and possibly making the latest a class method?



  class LinuxTest(Test, LinuxSSHMixIn):
  """Facilitates having a cloud-image Linux based available.
  
@@ -348,14 +372,20 @@ def download_boot(self):

  vmimage.QEMU_IMG = qemu_img
  
  self.log.info('Downloading/preparing boot image')

+distro = 'fedora'
+distro_version = '31'
+known_distro_checksum = get_known_distro_checksum(distro,
+  distro_version,
+  self.arch)
+distro_checksum = self.distro_checksum or known_distro_checksum



distro_checksum may be None. In this case vmimage.get() will silently 
skip the check? I suggest to log a warn message.




  # Fedora 31 only provides ppc64le images
  image_arch = self.arch
  if image_arch == 'ppc64':
  image_arch = 'ppc64le'
  try:
  boot = vmimage.get(
-'fedora', arch=image_arch, version='31',
-checksum=self.distro_checksum,
+distro, arch=image_arch, version=distro_version,
+checksum=distro_checksum,
  algorithm='sha256',
  cache_dir=self.cache_dirs[0],
  snapshot_dir=self.workdir)
diff --git a/tests/acceptance/boot_linux.py b/tests/acceptance/boot_linux.py
index c7bc3a589e..9e618c6daa 100644
--- a/tests/acceptance/boot_linux.py
+++ b/tests/acceptance/boot_linux.py
@@ -20,8 +20,6 @@ class BootLinuxX8664(LinuxTest):
  :avocado: tags=arch:x86_64
  """
  
-distro_checksum = 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'

-
  def test_pc_i440fx_tcg(self):
  """
  :avocado: tags=machine:pc
@@ -66,8 +64,6 @@ class BootLinuxAarch64(LinuxTest):
  :avocado: tags=machine:gic-version=2
  """
  
-distro_checksum = '1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49'

-
  def add_common_args(self):
  self.vm.add_args('-bios',
   os.path.join(BUILD_DIR, 'pc-bios',
@@ -119,8 +115,6 @@ class BootLinuxPPC64(LinuxTest):
  :avocado: tags=arch:ppc64
  """
  
-distro_checksum = '7c3528b85a3df4b2306e892199a9e1e43f991c506f2cc390dc4efa2026ad2f58'

-
  def test_pseries_tcg(self):
  """
  :avocado: tags=machine:pseries
@@ -136,8 +130,6 @@ class BootLinuxS390X(LinuxTest):
  :avocado: tags=arch:s390x
  """
  
-distro_checksum = '4caaab5a434fd4d1079149a072fdc7891e354f834d355069ca982fdcaf5a122d'

-
  @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
  def test_s390_ccw_virtio_tcg(self):
  """

Re: [PATCH 1/3] Acceptance Tests: rename attribute holding the distro image checksum

2021-04-19 Thread Wainer dos Santos Moschetta




On 4/14/21 7:14 PM, Cleber Rosa wrote:

This renames the attribute that holds the checksum for the image Linux
distribution image used.

The current name of the attribute is not very descriptive.  Also, in
preparation for making the distribution used configurable, which will
add distro related parameters, attributes and tags, let's make the
naming of those more uniform.

Signed-off-by: Cleber Rosa 
---
  tests/acceptance/avocado_qemu/__init__.py | 4 ++--
  tests/acceptance/boot_linux.py| 8 
  2 files changed, 6 insertions(+), 6 deletions(-)



Reviewed-by: Wainer dos Santos Moschetta 




diff --git a/tests/acceptance/avocado_qemu/__init__.py 
b/tests/acceptance/avocado_qemu/__init__.py
index 1062a851b9..aae1e5bbc9 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -307,7 +307,7 @@ class LinuxTest(Test, LinuxSSHMixIn):
  """
  
  timeout = 900

-chksum = None
+distro_checksum = None
  username = 'root'
  password = 'password'
  
@@ -355,7 +355,7 @@ def download_boot(self):

  try:
  boot = vmimage.get(
  'fedora', arch=image_arch, version='31',
-checksum=self.chksum,
+checksum=self.distro_checksum,
  algorithm='sha256',
  cache_dir=self.cache_dirs[0],
  snapshot_dir=self.workdir)
diff --git a/tests/acceptance/boot_linux.py b/tests/acceptance/boot_linux.py
index 314370fd1f..c7bc3a589e 100644
--- a/tests/acceptance/boot_linux.py
+++ b/tests/acceptance/boot_linux.py
@@ -20,7 +20,7 @@ class BootLinuxX8664(LinuxTest):
  :avocado: tags=arch:x86_64
  """
  
-chksum = 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'

+distro_checksum = 
'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'
  
  def test_pc_i440fx_tcg(self):

  """
@@ -66,7 +66,7 @@ class BootLinuxAarch64(LinuxTest):
  :avocado: tags=machine:gic-version=2
  """
  
-chksum = '1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49'

+distro_checksum = 
'1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49'
  
  def add_common_args(self):

  self.vm.add_args('-bios',
@@ -119,7 +119,7 @@ class BootLinuxPPC64(LinuxTest):
  :avocado: tags=arch:ppc64
  """
  
-chksum = '7c3528b85a3df4b2306e892199a9e1e43f991c506f2cc390dc4efa2026ad2f58'

+distro_checksum = 
'7c3528b85a3df4b2306e892199a9e1e43f991c506f2cc390dc4efa2026ad2f58'
  
  def test_pseries_tcg(self):

  """
@@ -136,7 +136,7 @@ class BootLinuxS390X(LinuxTest):
  :avocado: tags=arch:s390x
  """
  
-chksum = '4caaab5a434fd4d1079149a072fdc7891e354f834d355069ca982fdcaf5a122d'

+distro_checksum = 
'4caaab5a434fd4d1079149a072fdc7891e354f834d355069ca982fdcaf5a122d'
  
  @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')

  def test_s390_ccw_virtio_tcg(self):

Re: [PATCH 08/15] gitlab-ci: Extract DCO/style check jobs to checks.yml

2021-04-19 Thread Wainer dos Santos Moschetta




On 4/19/21 2:26 AM, Thomas Huth wrote:

On 19/04/2021 01.34, Philippe Mathieu-Daudé wrote:

Extract the DCO / checkpatch jobs to a new file (checks.yml)
to be able to run them without having to run all the jobs
included in the default .gitlab-ci.yml, which are mainly useful
for mainstream CI.

Signed-off-by: Philippe Mathieu-Daudé 
---
  .gitlab-ci.d/checks.yml | 24 
  .gitlab-ci.yml  |  1 +
  2 files changed, 25 insertions(+)
  create mode 100644 .gitlab-ci.d/checks.yml

diff --git a/.gitlab-ci.d/checks.yml b/.gitlab-ci.d/checks.yml
new file mode 100644
index 000..f695627b7cd
--- /dev/null
+++ b/.gitlab-ci.d/checks.yml
@@ -0,0 +1,24 @@
+check-patch:
+  stage: build
+  image: $CI_REGISTRY_IMAGE/qemu/centos8:latest
+  needs:
+    job: amd64-centos8-container
+  script: .gitlab-ci.d/check-patch.py
+  except:
+    variables:
+  - $CI_PROJECT_NAMESPACE == 'qemu-project' && $CI_COMMIT_BRANCH 
== 'master'

+  variables:
+    GIT_DEPTH: 1000
+  allow_failure: true
+
+check-dco:
+  stage: build
+  image: $CI_REGISTRY_IMAGE/qemu/centos8:latest
+  needs:
+    job: amd64-centos8-container
+  script: .gitlab-ci.d/check-dco.py
+  except:
+    variables:
+  - $CI_PROJECT_NAMESPACE == 'qemu-project' && $CI_COMMIT_BRANCH 
== 'master'

+  variables:
+    GIT_DEPTH: 1000
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index bf4862a78de..ba35b3917e7 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -12,6 +12,7 @@ include:
    - local: '/.gitlab-ci.d/opensbi.yml'
    - local: '/.gitlab-ci.d/containers.yml'
    - local: '/.gitlab-ci.d/crossbuilds.yml'
+  - local: '/.gitlab-ci.d/checks.yml'
    .native_build_job_template:
    stage: build



You added the new file, but it looks like you did not remove the jobs 
from the main .gitlab-ci.yml file ? ... that looks like a mistake?


Also, "checks.yml" sounds a little bit too generic. Maybe rather name 
it "style-check.yml" or something similar?



Maybe "static_checks.yml".

- Wainer




 Thomas

Re: [PATCH 1/1] Acceptance Tests: bump Avocado version requirement to 87.0

2021-04-14 Thread Wainer dos Santos Moschetta




On 4/14/21 1:11 PM, Cleber Rosa wrote:

This version (and 86.0) contain improvements that address specific
QEMU use cases, including:

* Fix to the error message given when downloading assets

* Asset listing/purging capabilities

Signed-off-by: Cleber Rosa 
---
  tests/requirements.txt | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)



Acked-by: Wainer dos Santos Moschetta 




diff --git a/tests/requirements.txt b/tests/requirements.txt
index 91f3a343b9..f57e895c4a 100644
--- a/tests/requirements.txt
+++ b/tests/requirements.txt
@@ -1,5 +1,5 @@
  # Add Python module requirements, one per line, to be installed
  # in the tests/venv Python virtual environment. For more info,
  # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
-avocado-framework==85.0
+avocado-framework==87.0
  pycdlib==1.11.0

Re: [PATCH v3 11/11] tests/acceptance/virtiofs_submounts.py: fix setup of SSH pubkey

2021-04-12 Thread Wainer dos Santos Moschetta




On 4/12/21 1:46 AM, Cleber Rosa wrote:

The public key argument should be a path to a file, and not the
public key data.

Reported-by: Wainer dos Santos Moschetta 
Signed-off-by: Cleber Rosa 
---
  tests/acceptance/virtiofs_submounts.py | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)



Reviewed-by: Wainer dos Santos Moschetta 




diff --git a/tests/acceptance/virtiofs_submounts.py 
b/tests/acceptance/virtiofs_submounts.py
index d77ee356740..21ad7d792e7 100644
--- a/tests/acceptance/virtiofs_submounts.py
+++ b/tests/acceptance/virtiofs_submounts.py
@@ -195,7 +195,7 @@ def setUp(self):
  
  self.run(('ssh-keygen', '-N', '', '-t', 'ed25519', '-f', self.ssh_key))
  
-pubkey = open(self.ssh_key + '.pub').read()

+pubkey = self.ssh_key + '.pub'
  
  super(VirtiofsSubmountsTest, self).setUp(pubkey)

Re: [PATCH 0/3] tests/acceptance: Handle tests with "cpu" tag

2021-04-09 Thread Wainer dos Santos Moschetta


Hi,

On 4/7/21 5:01 PM, Eduardo Habkost wrote:

On Tue, Mar 23, 2021 at 05:01:09PM -0400, John Snow wrote:

On 3/17/21 3:16 PM, Wainer dos Santos Moschetta wrote:

Added John and Eduardo,

On 3/9/21 3:52 PM, Cleber Rosa wrote:

On Wed, Feb 24, 2021 at 06:26:51PM -0300, Wainer dos Santos
Moschetta wrote:

Currently the acceptance tests tagged with "machine" have the "-M TYPE"
automatically added to the list of arguments of the QEMUMachine object.
In other words, that option is passed to the launched QEMU. On this
series it is implemented the same feature but instead for tests marked
with "cpu".


Good!


There is a caveat, however, in case the test needs additional
arguments to
the CPU type they cannot be passed via tag, because the tags
parser split
values by comma. For example, in
tests/acceptance/x86_cpu_model_versions.py,
there are cases where:

    * -cpu is set to
"Cascadelake-Server,x-force-features=on,check=off,enforce=off"
    * if it was tagged like
"cpu:Cascadelake-Server,x-force-features=on,check=off,enforce=off"
  then the parser would break it into 4 tags
("cpu:Cascadelake-Server",
  "x-force-features=on", "check=off", "enforce=off")
    * resulting on "-cpu Cascadelake-Server" and the remaining
arguments are ignored.

For the example above, one should tag it (or not at all) as
"cpu:Cascadelake-Server"
AND self.vm.add_args('-cpu',
"Cascadelake-Server,x-force-features=on,check=off,enforce=off"),
and that results on something like:

    "qemu-system-x86_64 (...) -cpu Cascadelake-Server -cpu
Cascadelake-Server,x-force-features=on,check=off,enforce=off".


There are clearly two problems here:

1) the tag is meant to be succinct, so that it can be used by users
     selecting which tests to run.  At the same time, it's a waste
     to throw away the other information or keep it duplicate or
     incosistent.

2) QEMUMachine doesn't keep track of command line arguments
     (add_args() makes it pretty clear what's doing).  But, on this type
     of use case, a "set_args()" is desirable, in which case it would
     overwrite the existing arguments for a given command line option.

I like the idea of a "set_args()" to QEMUMachine as you describe above
but it needs further discussion because I can see at least one corner
case; for example, one can set the machine type as either -machine or
-M, then what key it should be searched-and-replaced (if any) on the
list of args?

Unlike your suggestion, I thought on implement the method to deal with a
single argument at time, as:

      def set_arg(self, arg: Union[str, list], value: str) -> None:
      """
      Set the value of an argument from the list of extra arguments
to be
      given to the QEMU binary. If the argument does not exist then
it is
      added to the list.

      If the ``arg`` parameter is a list then it will search and
replace all
      occurencies (if any). Otherwise a new argument is added and it is
      used the first value of the ``arg`` list.
      """
      pass

Does it sound good to you?

Thanks!

Wainer


A little hokey, but I suppose that's true of our CLI interface in general.

I'd prefer not get into the business of building a "config" inside the
python module if we can help it right now, but if "setting" individual args
is something you truly need to do, I won't stand in the way.

Do what's least-gross.

I don't have any specific suggestions on how the API should look
like, but I'm having trouble understanding the documentation
above.

I don't know what "it will search and replace all occurrences"
means.  Occurrences of what?

I don't understand what "it is used the first value of the `arg`
list" means, either.  I understand you are going to use the first
value of the list, but you don't say what you are going to do
with it.



The documentation was indeed confusing but, please, disregard it. Based 
on John's comments on this thread I decided to not introduce yet another 
specialized function to QEMUMachine class. Instead I added the "args" 
property so that users will have access to QEMUMachine._args to change 
it whatever they like. You will find that implemented on the v2 of this 
series:


'[PATCH v2 0/7] tests/acceptance: Handle tests with "cpu" tag'

Thanks!

- Wainer

1 2 3 4 5 6 >

1 - 100 of 595 matches

Mail list logo