[libvirt PATCH v2 29/81] qemu: Preserve error in qemuMigrationDstFinish

2022-06-01 Thread Jiri Denemark
We want to prevent our error path that can potentially kill the domain on the destination host from overwriting an error reported earlier, but we were only doing so in one specific path when starting vCPUs fails. Let's do it in all paths. Signed-off-by: Jiri Denemark Reviewed-by: Peter Krempa Re

[libvirt PATCH v2 44/81] qemu: Separate protocol checks from qemuMigrationJobSetPhase

2022-06-01 Thread Jiri Denemark
Into a new qemuMigrationCheckPhase helper, which can be reused in other places. Signed-off-by: Jiri Denemark Reviewed-by: Peter Krempa Reviewed-by: Pavel Hrdina --- Notes: Version 2: - no change src/qemu/qemu_migration.c | 19 +++ 1 file changed, 15 insertions(+), 4 d

[libvirt PATCH v2 46/81] qemu: Refactor qemuDomainObjSetJobPhase

2022-06-01 Thread Jiri Denemark
We will want to update migration phase without affecting job ownership. Either in the thread that already owns the job or from an event handler which only changes the phase (of a job no-one owns) without assuming it. Let's move the ownership change to a new qemuDomainObjStartJobPhase helper and le

[libvirt PATCH v2 17/81] qemu: Don't wait for migration job when migration is running

2022-06-01 Thread Jiri Denemark
Migration is a job which takes some time and if it succeeds, there's nothing to call another migration on. If a migration fails, it might make sense to rerun it with different arguments, but this would only be done once the first migration fails rather than while it is still running. If this was n

[libvirt PATCH v2 43/81] qemu: Add new migration phases for post-copy recovery

2022-06-01 Thread Jiri Denemark
When recovering from a failed post-copy migration, we need to go through all migration phases again, but don't need to repeat all the steps in each phase. Let's create a new set of migration phases dedicated to post-copy recovery so that we can easily distinguish between normal and recovery code.

[libvirt PATCH v2 36/81] qemu: Make sure migrationPort is released even in callbacks

2022-06-01 Thread Jiri Denemark
Normally migrationPort is released in the Finish phase, but we need to make sure it is properly released also in case qemuMigrationDstFinish is not called at all. Currently the only callback which is called in this situation qemuMigrationDstPrepareCleanup which already releases migrationPort. This

[libvirt PATCH v2 00/81] Add support for post-copy recovery

2022-06-01 Thread Jiri Denemark
This series implements a new VIR_MIGRATE_POSTCOPY_RESUME flag (virsh migrate --resume) for recovering from a failed post-copy migration. You can also fetch the series from my gitlab fork (the last RFC patch is missing there): git fetch https://gitlab.com/jirkade/libvirt.git post-copy-recovery

[libvirt PATCH v2 30/81] qemu: Introduce qemuMigrationDstFinishFresh

2022-06-01 Thread Jiri Denemark
Refactors qemuMigrationDstFinish by moving some parts to a dedicated function for easier introduction of postcopy resume code without duplicating common parts of the Finish phase. The goal is to have the following call graph: - qemuMigrationDstFinish - qemuMigrationDstFinishOffline

[libvirt PATCH v2 24/81] qemu: Separate success and failure path in qemuMigrationDstFinish

2022-06-01 Thread Jiri Denemark
Most of the code in "endjob" label is executed only on failure. Let's duplicate the rest so that the label can be used only in error path making the success path easier to follow and refactor. Signed-off-by: Jiri Denemark Reviewed-by: Peter Krempa Reviewed-by: Pavel Hrdina --- Notes: Versi

[libvirt PATCH v2 33/81] qemu: Introduce qemuMigrationDstFinishActive

2022-06-01 Thread Jiri Denemark
Refactors qemuMigrationDstFinish by moving some parts to a dedicated function for easier introduction of postcopy resume code without duplicating common parts of the Finish phase. The goal is to have the following call graph: - qemuMigrationDstFinish - qemuMigrationDstFinishOffline

[libvirt PATCH v2 37/81] qemu: Pass qemuDomainJobObj to qemuMigrationDstComplete

2022-06-01 Thread Jiri Denemark
When reconnecting to an active domain we need to use a different job structure than the one referenced from the VM object. Signed-off-by: Jiri Denemark Reviewed-by: Peter Krempa Reviewed-by: Pavel Hrdina --- Notes: Version 2: - fixed a typo in the commit message src/qemu/qemu_migrati

[libvirt PATCH v2 77/81] virsh: Add --postcopy option for domjobabort command

2022-06-01 Thread Jiri Denemark
Signed-off-by: Jiri Denemark Reviewed-by: Peter Krempa Reviewed-by: Pavel Hrdina --- Notes: Version 2: - no change docs/manpages/virsh.rst | 8 +++- tools/virsh-domain.c| 16 +++- 2 files changed, 22 insertions(+), 2 deletions(-) diff --git a/docs/manpages/virsh.

[libvirt PATCH v2 14/81] qemu: Restore failed migration job on reconnect

2022-06-01 Thread Jiri Denemark
Since we keep the migration job active when post-copy migration fails, we need to restore it when reconnecting to running domains. Signed-off-by: Jiri Denemark Reviewed-by: Peter Krempa Reviewed-by: Pavel Hrdina --- Notes: Version 2: - no change src/qemu/qemu_process.c | 128

[libvirt PATCH v2 16/81] qemu: Drop forward declarations in migration code

2022-06-01 Thread Jiri Denemark
Signed-off-by: Jiri Denemark Reviewed-by: Peter Krempa Reviewed-by: Pavel Hrdina --- Notes: Version 2: - no change src/qemu/qemu_migration.c | 176 -- 1 file changed, 76 insertions(+), 100 deletions(-) diff --git a/src/qemu/qemu_migration.c b/src/q

[libvirt PATCH v2 15/81] qemu: Restore async job start timestamp on reconnect

2022-06-01 Thread Jiri Denemark
Jobs that are supposed to remain active even when libvirt daemon restarts were reported as started at the time the daemon was restarted. This is not very helpful, we should restore the original timestamp. Signed-off-by: Jiri Denemark --- Notes: Version 2: - use current time as a fallback

[libvirt PATCH v2 63/81] qemu: Refactor qemuMigrationAnyConnectionClosed

2022-06-01 Thread Jiri Denemark
To prepare the code for handling incoming migration too. Signed-off-by: Jiri Denemark Reviewed-by: Peter Krempa Reviewed-by: Pavel Hrdina --- Notes: Version 2: - moved relevant cases directly to "all done; unreachable" section to avoid unnecessary churn src/qemu/qemu_migration.

[libvirt PATCH v2 20/81] qemu: Handle 'postcopy-paused' migration state

2022-06-01 Thread Jiri Denemark
When connection breaks during post-copy migration, QEMU enters 'postcopy-paused' state. We need to handle this state and make the situation visible to upper layers. Signed-off-by: Jiri Denemark --- Notes: Version 2: - rebased on top of newly added "qemu: Use switch in qemuProcessHa

[libvirt PATCH v2 19/81] qemu: Use switch in qemuProcessHandleMigrationStatus

2022-06-01 Thread Jiri Denemark
Signed-off-by: Jiri Denemark --- Notes: Version 2: - new patch src/qemu/qemu_process.c | 44 ++--- 1 file changed, 32 insertions(+), 12 deletions(-) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index a87dc9a1fb..f1264909de 100644 --

[libvirt PATCH v2 22/81] qemu: Create domain object at the end of qemuMigrationDstFinish

2022-06-01 Thread Jiri Denemark
We don't need the object until we get to the "endjob" label. Signed-off-by: Jiri Denemark Reviewed-by: Peter Krempa Reviewed-by: Pavel Hrdina --- Notes: Version 2: - no change src/qemu/qemu_migration.c | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/src/qemu

[libvirt PATCH v2 13/81] qemu: Abort failed post-copy when we haven't called Finish yet

2022-06-01 Thread Jiri Denemark
When migration fails after it already switched to post-copy phase on the source, but early enough that we haven't called Finish on the destination yet, we know the vCPUs were not started on the destination and the source host still has a complete state of the domain. Thus we can just ignore the fac

[libvirt PATCH v2 12/81] qemu: Keep migration job active after failed post-copy

2022-06-01 Thread Jiri Denemark
When post-copy migration fails, we can't just abort the migration and resume the domain on the source host as it is already running on the destination host and no host has a complete state of the domain memory. Instead of the current approach of just marking the domain on both ends as paused/runnin

[libvirt PATCH v2 28/81] qemu: Drop obsolete comment in qemuMigrationDstFinish

2022-06-01 Thread Jiri Denemark
The comment about QEMU < 0.10.6 has been irrelevant for years. Signed-off-by: Jiri Denemark Reviewed-by: Peter Krempa Reviewed-by: Pavel Hrdina --- Notes: Version 2: - no change src/qemu/qemu_migration.c | 4 1 file changed, 4 deletions(-) diff --git a/src/qemu/qemu_migration.c

[libvirt PATCH v2 23/81] qemu: Move success-only code out of endjob in qemuMigrationDstFinish

2022-06-01 Thread Jiri Denemark
Code executed only when dom != NULL can be moved before "endjob" label, to the only place where dom is set. Signed-off-by: Jiri Denemark Reviewed-by: Peter Krempa Reviewed-by: Pavel Hrdina --- Notes: Version 2: - fixed indentation src/qemu/qemu_migration.c | 50 ++

[libvirt PATCH v2 03/81] Introduce VIR_DOMAIN_RUNNING_POSTCOPY_FAILED

2022-06-01 Thread Jiri Denemark
This new "post-copy failed" reason for the running state will be used on the destination host when post-copy migration fails while the domain is already running there. Signed-off-by: Jiri Denemark --- Notes: Version 2: - documented both VIR_DOMAIN_RUNNING_POSTCOPY_FAILED and VIR_DO

[libvirt PATCH v2 11/81] qemu: Add qemuDomainObjRestoreAsyncJob

2022-06-01 Thread Jiri Denemark
The code for setting up a previously active backup job in qemuProcessRecoverJob is generalized into a dedicated function so that it can be later reused in other places. Signed-off-by: Jiri Denemark Reviewed-by: Pavel Hrdina --- Notes: Version 2: - s/priv->job./job->/ src/qemu/qemu_dom

[libvirt PATCH v2 05/81] qemu: Explicitly emit events on post-copy failure

2022-06-01 Thread Jiri Denemark
The events would normally be triggered only if we're changing domain state. But most of the time the domain is already in the right state and we're just changing its substate from {PAUSED,RUNNING}_POSTCOPY to *_POSTCOPY_FAILED. Let's emit lifecycle events explicitly when post-copy migration fails t

[libvirt PATCH v2 10/81] qemu: Rename qemuDomainObjRestoreJob as qemuDomainObjPreserveJob

2022-06-01 Thread Jiri Denemark
It is used for saving job out of domain object. Just like virErrorPreserveLast is used for errors. Let's make the naming consistent as Restore would suggest different semantics. Signed-off-by: Jiri Denemark Reviewed-by: Peter Krempa Reviewed-by: Pavel Hrdina --- Notes: Version 2: - no

[libvirt PATCH v2 07/81] conf: Introduce virDomainObjIsFailedPostcopy helper

2022-06-01 Thread Jiri Denemark
Signed-off-by: Jiri Denemark Reviewed-by: Peter Krempa Reviewed-by: Pavel Hrdina --- Notes: Version 2: - no change src/conf/domain_conf.c | 10 ++ src/conf/domain_conf.h | 4 src/libvirt_private.syms | 1 + 3 files changed, 15 insertions(+) diff --git a/src/conf/do

[libvirt PATCH v2 04/81] qemu: Keep domain running on dst on failed post-copy migration

2022-06-01 Thread Jiri Denemark
There's no need to artificially pause a domain when post-copy fails from our point of view unless QEMU connection is broken too as migration may still be progressing well. Signed-off-by: Jiri Denemark --- Notes: Version 2: - commit message and warning text updated - dropped dead code

[libvirt PATCH v2 06/81] qemu: Make qemuDomainCleanupAdd return void

2022-06-01 Thread Jiri Denemark
The function never returns anything but zero. Signed-off-by: Jiri Denemark Reviewed-by: Peter Krempa Reviewed-by: Pavel Hrdina --- Notes: Version 2: - no change src/qemu/qemu_domain.c| 5 ++--- src/qemu/qemu_domain.h| 4 ++-- src/qemu/qemu_migration.c | 3 +-- 3 files changed

[libvirt PATCH v2 08/81] conf: Introduce virDomainObjIsPostcopy helper

2022-06-01 Thread Jiri Denemark
Signed-off-by: Jiri Denemark Reviewed-by: Peter Krempa Reviewed-by: Pavel Hrdina --- Notes: Version 2: - no change src/conf/domain_conf.c | 22 ++ src/conf/domain_conf.h | 4 src/libvirt_private.syms | 1 + 3 files changed, 27 insertions(+) diff --git a

[libvirt PATCH v2 01/81] qemu: Add debug messages to job recovery code

2022-06-01 Thread Jiri Denemark
Signed-off-by: Jiri Denemark Reviewed-by: Peter Krempa Reviewed-by: Pavel Hrdina --- Notes: Version 2: - no change src/qemu/qemu_process.c | 13 + 1 file changed, 13 insertions(+) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 1593ca7933..e8936cd623

[libvirt PATCH v2 02/81] qemumonitorjsontest: Test more migration capabilities

2022-06-01 Thread Jiri Denemark
Signed-off-by: Jiri Denemark Reviewed-by: Peter Krempa Reviewed-by: Pavel Hrdina --- Notes: Version 2: - no change tests/qemumonitorjsontest.c | 30 -- 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/tests/qemumonitorjsontest.c b/tests/qemumo

Re: [libvirt PATCH v2 5/7] ci: Don't mark any Fedora 36 job as optional

2022-05-26 Thread Andrea Bolognani
On Thu, May 26, 2022 at 04:23:44PM +0200, Erik Skultety wrote: > On Thu, May 26, 2022 at 04:01:50PM +0200, Andrea Bolognani wrote: > > +++ b/ci/manifest.yml > > @@ -157,7 +157,6 @@ targets: > >- arch: x86_64 > > > >- arch: mingw32 > > -allow-failure: true > > There must be a

Re: [libvirt PATCH v2 0/7] ci: Drop Fedora 34, add Fedora 36

2022-05-26 Thread Erik Skultety
On Thu, May 26, 2022 at 04:01:45PM +0200, Andrea Bolognani wrote: > Test pipeline: > > https://gitlab.com/abologna/libvirt/-/pipelines/548848259 > > Only patches 1-5 should be pushed until the issues outlined in Go ahead and push 1-5 ;). Erik

Re: [libvirt PATCH v2 5/7] ci: Don't mark any Fedora 36 job as optional

2022-05-26 Thread Erik Skultety
On Thu, May 26, 2022 at 04:01:50PM +0200, Andrea Bolognani wrote: > It's a stable distro, so we expect all jobs to succeed. > > Signed-off-by: Andrea Bolognani > --- > ci/gitlab/containers.yml | 2 +- > ci/manifest.yml | 1 - > 2 files changed, 1 insertion(+), 2 deletions(-) > > diff -

Re: [libvirt PATCH v2 4/7] ci: Move MinGW jobs to Fedora 36

2022-05-26 Thread Erik Skultety
On Thu, May 26, 2022 at 04:01:49PM +0200, Andrea Bolognani wrote: > Signed-off-by: Andrea Bolognani > --- Reviewed-by: Erik Skultety

Re: [libvirt PATCH v2 3/7] ci: Add Fedora 36

2022-05-26 Thread Erik Skultety
On Thu, May 26, 2022 at 04:01:48PM +0200, Andrea Bolognani wrote: > The target is intentionally not added to the integration tests > at this time, because the corresponding VM template is not yet > available on the runner. A later patch will take care of that. > > Signed-off-by: Andrea Bolognani

[libvirt PATCH v2 6/7] ci: Add Fedora 36 to integration tests

2022-05-26 Thread Andrea Bolognani
This requires publishing the RPMs as artifacts from the regular build job. Signed-off-by: Andrea Bolognani --- ci/gitlab/builds.yml | 4 ci/integration.yml | 16 ci/manifest.yml | 4 3 files changed, 24 insertions(+) diff --git a/ci/gitlab/builds.yml b/ci/git

[libvirt PATCH v2 4/7] ci: Move MinGW jobs to Fedora 36

2022-05-26 Thread Andrea Bolognani
Signed-off-by: Andrea Bolognani --- ...gw32.Dockerfile => fedora-36-cross-mingw32.Dockerfile} | 2 +- ...gw64.Dockerfile => fedora-36-cross-mingw64.Dockerfile} | 2 +- ci/gitlab/builds.yml | 6 +++--- ci/gitlab/containers.yml |

[libvirt PATCH v2 7/7] ci: Move upstream QEMU integration test to Fedora 36

2022-05-26 Thread Andrea Bolognani
Signed-off-by: Andrea Bolognani --- ci/integration.yml | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/ci/integration.yml b/ci/integration.yml index 5b60b41e85..43f22813c9 100644 --- a/ci/integration.yml +++ b/ci/integration.yml @@ -49,24 +49,24 @@ fedora-35-tes

[libvirt PATCH v2 1/7] ci: Drop Fedora 34

2022-05-26 Thread Andrea Bolognani
Signed-off-by: Andrea Bolognani Reviewed-by: Erik Skultety --- ci/containers/fedora-34.Dockerfile | 109 - ci/gitlab.yml | 20 -- ci/integration.yml | 16 - ci/manifest.yml| 8 --- 4 files changed, 1

[libvirt PATCH v2 5/7] ci: Don't mark any Fedora 36 job as optional

2022-05-26 Thread Andrea Bolognani
It's a stable distro, so we expect all jobs to succeed. Signed-off-by: Andrea Bolognani --- ci/gitlab/containers.yml | 2 +- ci/manifest.yml | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/ci/gitlab/containers.yml b/ci/gitlab/containers.yml index 5b8a91c295..37c179c

[libvirt PATCH v2 2/7] ci: Refresh generated files

2022-05-26 Thread Andrea Bolognani
Notable changes: * 'lcitool manifest' now generates GitLab CI rules spread across a bunch of files; * container images are built less frequently for the main repository. Signed-off-by: Andrea Bolognani Reviewed-by: Erik Skultety --- ci/gitlab.yml | 747 +---

[libvirt PATCH v2 3/7] ci: Add Fedora 36

2022-05-26 Thread Andrea Bolognani
The target is intentionally not added to the integration tests at this time, because the corresponding VM template is not yet available on the runner. A later patch will take care of that. Signed-off-by: Andrea Bolognani --- ci/containers/fedora-36.Dockerfile | 108 +

[libvirt PATCH v2 0/7] ci: Drop Fedora 34, add Fedora 36

2022-05-26 Thread Andrea Bolognani
Test pipeline: https://gitlab.com/abologna/libvirt/-/pipelines/548848259 Only patches 1-5 should be pushed until the issues outlined in https://listman.redhat.com/archives/libvir-list/2022-May/231851.html have been addressed. Changes from [v1] * handle integration tests separately; *

Re: [libvirt PATCH v2 0/7] po: Various fixes and cleanups

2022-05-23 Thread Michal Prívozník
On 5/18/22 11:52, Andrea Bolognani wrote: > Changes from [v1] > > * instead of checking that the potfile doesn't contain unwanted > comments at syntax-check time, prevent them from being added by > passing all xgettext options explicitly ourselves. > > [v1] https://listman.redhat.com/ar

[libvirt PATCH v2 5/7] syntax-check: Don't exclude src/false.c from sc_po_check

2022-05-18 Thread Andrea Bolognani
This is something that certainly made sense in the context of gnulib, but we don't have a use for it. Signed-off-by: Andrea Bolognani --- build-aux/syntax-check.mk | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk index 35

[libvirt PATCH v2 4/7] po: Don't generate POTFILES

2022-05-18 Thread Andrea Bolognani
Now that we have dropped prefixes from the file, it no longer needs to go through configure_file() and we can use it directly. Signed-off-by: Andrea Bolognani --- build-aux/syntax-check.mk| 6 +++--- po/{POTFILES.in => POTFILES} | 0 po/meson.build | 13 + 3 files

[libvirt PATCH v2 7/7] syntax-check: Introduce sc_linguas_sorting

2022-05-18 Thread Andrea Bolognani
Make sure LINGUAS remains sorted correctly. Signed-off-by: Andrea Bolognani --- build-aux/syntax-check.mk | 8 1 file changed, 8 insertions(+) diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk index 513c147cec..6b54f830f1 100644 --- a/build-aux/syntax-check.mk +++ b/bu

[libvirt PATCH v2 6/7] po: Sort LINGUAS

2022-05-18 Thread Andrea Bolognani
Signed-off-by: Andrea Bolognani --- po/LINGUAS | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/po/LINGUAS b/po/LINGUAS index 889ef80aba..35acd33a87 100644 --- a/po/LINGUAS +++ b/po/LINGUAS @@ -28,17 +28,17 @@ nl or pa pl -pt_BR pt +pt_BR ru -sr@latin +si sr +sr@l

[libvirt PATCH v2 3/7] po: Drop prefixes from POTFILES.in

2022-05-18 Thread Andrea Bolognani
Commit 8beb7fdd0e23 changed the handling of POTFILES so that it could cope with files being located in either the source or build directory: it did so by adding @SRCDIR@ and @BUILDDIR@ respectively at the beginning of each line, and then converting them back to the actual values when generating POT

[libvirt PATCH v2 0/7] po: Various fixes and cleanups

2022-05-18 Thread Andrea Bolognani
Changes from [v1] * instead of checking that the potfile doesn't contain unwanted comments at syntax-check time, prevent them from being added by passing all xgettext options explicitly ourselves. [v1] https://listman.redhat.com/archives/libvir-list/2022-May/231526.html Andrea Bolognan

[libvirt PATCH v2 2/7] po: Stop using 'glib' preset for i18n.gettext()

2022-05-18 Thread Andrea Bolognani
Due to a bug in meson versions earlier than 0.60, the --add-comments that's part of the 'glib' preset and the --add-comments=TRANSLATORS: that we add ourselves might be passed to xgettext in the wrong order, resulting in a bunch of comments that we don't care about being added to the potfile. Most

[libvirt PATCH v2 1/7] po: Drop unwanted comments from potfile

2022-05-18 Thread Andrea Bolognani
These managed to sneak in as part of ec02f5719a87, when the potfile was last refreshed, but are not supposed to be there. Signed-off-by: Andrea Bolognani --- po/libvirt.pot | 216 - 1 file changed, 216 deletions(-) diff --git a/po/libvirt.pot b/po

Re: [libvirt PATCH v2 3/5] ci: Introduce a template for upstream QEMU build

2022-05-12 Thread Erik Skultety
On Thu, May 12, 2022 at 10:04:07AM +0100, Daniel P. Berrangé wrote: > On Thu, May 12, 2022 at 10:50:57AM +0200, Erik Skultety wrote: > > This was heavily inspired by QEMU's upstream CI buildtest-template.yml. > > Rather than referencing QEMU's template directly (which GitLab can do), > > this patch

Re: [libvirt PATCH v2 3/5] ci: Introduce a template for upstream QEMU build

2022-05-12 Thread Daniel P . Berrangé
On Thu, May 12, 2022 at 10:50:57AM +0200, Erik Skultety wrote: > This was heavily inspired by QEMU's upstream CI buildtest-template.yml. > Rather than referencing QEMU's template directly (which GitLab can do), > this patch resorts to hard-coding the build steps ourselves, solely > because there's

[libvirt PATCH v2 5/5] ci: Add a Fedora integration test job utilizing upstream QEMU

2022-05-12 Thread Erik Skultety
Signed-off-by: Erik Skultety Reviewed-by: Michal Privoznik Reviewed-by: Daniel P. Berrangé --- ci/integration.yml | 16 1 file changed, 16 insertions(+) diff --git a/ci/integration.yml b/ci/integration.yml index a71df41694..4978d3cf02 100644 --- a/ci/integration.yml +++ b/ci/i

[libvirt PATCH v2 4/5] ci: Add a new integration job template for the upstream QEMU scenario

2022-05-12 Thread Erik Skultety
YAML anchors don't work with Shell condition structures, so we cannot simply reference the QEMU build template YAML anchor conditionally and hence have everything as part of a single job template. Instead, we have to "subclass" the .integration_tests template and inject the QEMU building bits expli

[libvirt PATCH v2 3/5] ci: Introduce a template for upstream QEMU build

2022-05-12 Thread Erik Skultety
This was heavily inspired by QEMU's upstream CI buildtest-template.yml. Rather than referencing QEMU's template directly (which GitLab can do), this patch resorts to hard-coding the build steps ourselves, solely because there's no guarantee QEMU will keep either the template file name or the templa

[libvirt PATCH v2 2/5] ci: Break off the integration_tests template into more templates

2022-05-12 Thread Erik Skultety
There's quite a lot happening in the .integration_tests template already even without adding upstream QEMU build into the mix. Let's break the template into more pieces which can then reference in the .integration_tests template when putting all the pieces back together using YAML anchors. Signed-

[libvirt PATCH v2 1/5] ci: Separate the integration job template to a separate file

2022-05-12 Thread Erik Skultety
Future patches will do more code extraction from the existing template using YAML anchors so it'd be better that the templates would live separately from job definitions. Signed-off-by: Erik Skultety Reviewed-by: Michal Privoznik Reviewed-by: Daniel P. Berrangé --- ci/integration-template.yml

[libvirt PATCH v2 0/5] ci: Add an integration test job utilizing upstream QEMU

2022-05-12 Thread Erik Skultety
Since QEMU doesn't maintain a spec file in upstream, we cannot build RPM artifacts as part of the CI as we do for libvirt. Instead, we build QEMU from source manually. Since v1: - hard-code the QEMU build steps rather than pulling the build template from upstream QEMU's CI due to a possibility of

Re: [libvirt PATCH v2 0/4] Enable copy/paste for vnc displays

2022-05-10 Thread Michal Prívozník
On 5/9/22 20:53, Jonathon Jongsma wrote: > On 5/9/22 11:06 AM, Marc-André Lureau wrote: >> Hi Jonathon >> >> On Thu, Mar 24, 2022 at 11:26 PM Jonathon Jongsma > > wrote: >> >>     This patch series enables support for the qemu-vdagent character >>     device which >>    

Re: [libvirt PATCH v2 0/4] Enable copy/paste for vnc displays

2022-05-09 Thread Jonathon Jongsma
On 5/9/22 11:06 AM, Marc-André Lureau wrote: Hi Jonathon On Thu, Mar 24, 2022 at 11:26 PM Jonathon Jongsma > wrote: This patch series enables support for the qemu-vdagent character device which enables copy/paste support between guest and client when usi

Re: [libvirt PATCH v2 0/4] Enable copy/paste for vnc displays

2022-05-09 Thread Marc-André Lureau
Hi Jonathon On Thu, Mar 24, 2022 at 11:26 PM Jonathon Jongsma wrote: > This patch series enables support for the qemu-vdagent character device > which > enables copy/paste support between guest and client when using vnc > graphics. > > The guest must be configured with something like the followi

Re: [libvirt PATCH v2] iohelper: some refactoring

2022-04-21 Thread Claudio Fontana
On 4/21/22 5:42 PM, Daniel P. Berrangé wrote: > On Sat, Mar 26, 2022 at 06:06:37PM +0100, Claudio Fontana wrote: >> while doing research with alternative implementations of runIO, >> it seemed necessary to do some refactoring, in order to separate >> parameter setting from the actual copy, so that

Re: [libvirt PATCH v2] iohelper: some refactoring

2022-04-21 Thread Daniel P . Berrangé
On Sat, Mar 26, 2022 at 06:06:37PM +0100, Claudio Fontana wrote: > while doing research with alternative implementations of runIO, > it seemed necessary to do some refactoring, in order to separate > parameter setting from the actual copy, so that alternative > copy methods can be researched and ho

Re: [libvirt PATCH v2] iohelper: some refactoring

2022-04-21 Thread Claudio Fontana
A ping on this one, if it is something interesting. My research has not identified great benefits in using other alternative APIs, but this refactoring might be better looking code anyway? Thanks, Claudio On 3/26/22 6:06 PM, Claudio Fontana wrote: > while doing research with alternative impleme

[libvirt PATCH v2] iohelper: some refactoring

2022-03-26 Thread Claudio Fontana
while doing research with alternative implementations of runIO, it seemed necessary to do some refactoring, in order to separate parameter setting from the actual copy, so that alternative copy methods can be researched and hopefully eventually implemented. No functional changes are expected. Sig

Re: [libvirt PATCH v2 4/4] qemu: add support for qemu-vdagent channel

2022-03-25 Thread Marc-André Lureau
Hi, On Thu, Mar 24, 2022 at 11:26 PM Jonathon Jongsma wrote: > > Implement the qemu-vdagent channel introduced in the previous commit. > > Signed-off-by: Jonathon Jongsma lgtm, Reviewed-by: Marc-André Lureau > --- > src/qemu/qemu_command.c | 21 ++- > src/qemu/qemu

Re: [libvirt PATCH v2 3/4] conf: add qemu-vdagent channel

2022-03-25 Thread Marc-André Lureau
Hi On Thu, Mar 24, 2022 at 11:26 PM Jonathon Jongsma wrote: > > Add the ability to configure a qemu-vdagent in guest domains. This > device is similar to the spice vdagent channel except that qemu handles > the spice-vdagent protocol messages itself rather than routing them over > a spice protoc

Re: [libvirt PATCH v2 2/4] Rename virDomainGraphicsSpiceMouseMode to virDomainMouseMode

2022-03-25 Thread Marc-André Lureau
On Thu, Mar 24, 2022 at 11:26 PM Jonathon Jongsma wrote: > > This enumeration will be useful for vnc with the upcoming qemu-vdagent > device so make the name more generic. > > Signed-off-by: Jonathon Jongsma Reviewed-by: Marc-André Lureau > --- > src/conf/domain_conf.c | 8 > src/

Re: [libvirt PATCH v2 1/4] qemu: add capability for qemu-vdagent chardev

2022-03-25 Thread Marc-André Lureau
On Thu, Mar 24, 2022 at 11:26 PM Jonathon Jongsma wrote: > > Detect whether qemu supports the qemu-vdagent character device. This > enables support for copy/paste with VNC graphics. > > Signed-off-by: Jonathon Jongsma Reviewed-by: Marc-André Lureau > --- > src/qemu/qemu_capabilities.c

[libvirt PATCH v2 1/4] qemu: add capability for qemu-vdagent chardev

2022-03-24 Thread Jonathon Jongsma
Detect whether qemu supports the qemu-vdagent character device. This enables support for copy/paste with VNC graphics. Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_capabilities.c | 2 ++ src/qemu/qemu_capabilities.h | 1 + tests/qemucapabilitiesdata

[libvirt PATCH v2 4/4] qemu: add support for qemu-vdagent channel

2022-03-24 Thread Jonathon Jongsma
Implement the qemu-vdagent channel introduced in the previous commit. Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_command.c | 21 ++- src/qemu/qemu_monitor_json.c | 28 - src/qemu/qemu_validate.c | 10 +++- ...l-qemu-v

[libvirt PATCH v2 2/4] Rename virDomainGraphicsSpiceMouseMode to virDomainMouseMode

2022-03-24 Thread Jonathon Jongsma
This enumeration will be useful for vnc with the upcoming qemu-vdagent device so make the name more generic. Signed-off-by: Jonathon Jongsma --- src/conf/domain_conf.c | 8 src/conf/domain_conf.h | 14 +++--- src/libvirt_private.syms | 4 ++-- src/libxl/libxl_conf.c | 8

[libvirt PATCH v2 3/4] conf: add qemu-vdagent channel

2022-03-24 Thread Jonathon Jongsma
Add the ability to configure a qemu-vdagent in guest domains. This device is similar to the spice vdagent channel except that qemu handles the spice-vdagent protocol messages itself rather than routing them over a spice protocol channel. The qemu-vdagent device has two notable configuration option

[libvirt PATCH v2 0/4] Enable copy/paste for vnc displays

2022-03-24 Thread Jonathon Jongsma
This patch series enables support for the qemu-vdagent character device which enables copy/paste support between guest and client when using vnc graphics. The guest must be configured with something like the following: Copy/paste sync requires a v

Re: [libvirt PATCH v2] schemas: move out of docs, fix no-docs build

2022-03-22 Thread Andrea Bolognani
On Tue, Mar 22, 2022 at 09:18:57AM +0100, Claudio Fontana wrote: > schemas are used for more than just documentation, > virsh edit fails if schemas are not available. > > Therefore, fix the no-docs build by moving schemas/ > to the parsing code inside src/conf/. > > Signed-off-by: Claudio Fontana

[libvirt PATCH v2] schemas: move out of docs, fix no-docs build

2022-03-22 Thread Claudio Fontana
schemas are used for more than just documentation, virsh edit fails if schemas are not available. Therefore, fix the no-docs build by moving schemas/ to the parsing code inside src/conf/. Signed-off-by: Claudio Fontana --- changes v1 -> v2: * move to src/conf/ instead of top_srcdir (Daniel) *

[libvirt PATCH v2 17/17] news: Document virtio-iommu

2022-03-18 Thread Andrea Bolognani
Signed-off-by: Andrea Bolognani Reviewed-by: Ján Tomko --- NEWS.rst | 4 1 file changed, 4 insertions(+) diff --git a/NEWS.rst b/NEWS.rst index b9b9807625..9ee3a77427 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -17,6 +17,10 @@ v8.2.0 (unreleased) * **New features** + * qemu: Introduce su

[libvirt PATCH v2 10/17] qemu: Validate use of ACPI with virtio-iommu

2022-03-18 Thread Andrea Bolognani
virtio-iommu doesn't work without ACPI, so we need to make sure the latter is enabled. Signed-off-by: Andrea Bolognani Reviewed-by: Ján Tomko --- src/qemu/qemu_validate.c | 6 ++ .../virtio-iommu-no-acpi.x86_64-latest.err| 1 + tests/qemuxml2argvdata/virti

[libvirt PATCH v2 04/17] qemu: Introduce QEMU_CAPS_DEVICE_VIRTIO_IOMMU_PCI

2022-03-18 Thread Andrea Bolognani
This capability detects the availability of the virtio-iommu-pci device. Note that, while this device is present even in somewhat old versions of QEMU, it's only some recent changes that made it actually usable for our purposes. Signed-off-by: Andrea Bolognani Reviewed-by: Ján Tomko --- src/qe

[libvirt PATCH v2 16/17] docs: Document virtio-iommu

2022-03-18 Thread Andrea Bolognani
Signed-off-by: Andrea Bolognani Reviewed-by: Ján Tomko --- docs/formatdomain.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index 9b1b69bb4d..4e5008f8b7 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -

[libvirt PATCH v2 14/17] tests: Add test for virtio-iommu address

2022-03-18 Thread Andrea Bolognani
virtio-iommu needs to be an integrated device, and our address assignment code will make sure that is the case. If the user has provided an explicit address, however, we should make sure any addresses pointing to a different bus are rejected. Signed-off-by: Andrea Bolognani Reviewed-by: Ján Tomko

[libvirt PATCH v2 15/17] qemu: Generate command line for virtio-iommu

2022-03-18 Thread Andrea Bolognani
https://bugzilla.redhat.com/show_bug.cgi?id=1653327 Signed-off-by: Andrea Bolognani --- src/qemu/qemu_command.c | 12 .../virtio-iommu-aarch64.aarch64-latest.args | 1 + .../virtio-iommu-x86_64.x86_64-latest.args | 1 + 3 files change

[libvirt PATCH v2 09/17] qemu: Validate capabilities for virtio-iommu

2022-03-18 Thread Andrea Bolognani
Signed-off-by: Andrea Bolognani Reviewed-by: Ján Tomko --- src/qemu/qemu_validate.c | 8 .../qemuxml2argvdata/virtio-iommu-x86_64.x86_64-6.1.0.err | 1 + tests/qemuxml2argvtest.c | 1 + 3 files changed, 10 insertions(+)

[libvirt PATCH v2 11/17] conf: Add virDomainDeviceInfo to virDomainIOMMUDef

2022-03-18 Thread Andrea Bolognani
This is needed so that IOMMU devices can have addresses. Existing IOMMU devices (intel-iommu and SMMUv3) are system devices and as such don't have an address associated to them, but virtio-iommu is a PCI device and needs one. Signed-off-by: Andrea Bolognani Reviewed-by: Ján Tomko --- docs/sche

[libvirt PATCH v2 12/17] qemu: Assign PCI address to virtio-iommu

2022-03-18 Thread Andrea Bolognani
The device is configured to be an integrated endpoint, as is necessary for it to function correctly. Signed-off-by: Andrea Bolognani Reviewed-by: Ján Tomko --- src/qemu/qemu_domain_address.c | 6 +- .../virtio-iommu-aarch64.aarch64-latest.xml | 4

[libvirt PATCH v2 07/17] tests: Add test cases for virtio-iommu

2022-03-18 Thread Andrea Bolognani
These represent valid uses of the device. Signed-off-by: Andrea Bolognani Reviewed-by: Ján Tomko --- .../virtio-iommu-aarch64.aarch64-latest.args | 35 +++ .../qemuxml2argvdata/virtio-iommu-aarch64.xml | 20 +++ .../virtio-iommu-x86_64.x86_64-latest.args| 31 +++

[libvirt PATCH v2 13/17] qemu: Validate address type for virtio-iommu

2022-03-18 Thread Andrea Bolognani
virtio-iommu is a PCI device and attempts to use a different address type should be rejected. Signed-off-by: Andrea Bolognani Reviewed-by: Ján Tomko --- src/qemu/qemu_validate.c | 7 +++ ...mmu-invalid-address-type.x86_64-latest.err | 1 + .../virtio-iommu-invalid-add

[libvirt PATCH v2 06/17] conf: Introduce virtio model for

2022-03-18 Thread Andrea Bolognani
Signed-off-by: Andrea Bolognani Reviewed-by: Ján Tomko --- docs/schemas/domaincommon.rng | 1 + src/conf/domain_conf.c | 1 + src/conf/domain_conf.h | 1 + src/qemu/qemu_command.c| 4 src/qemu/qemu_domain_address.c | 6 ++ src/qemu/qemu_validate.c | 3 +++

[libvirt PATCH v2 08/17] qemu: Validate machine type used with virtio-iommu

2022-03-18 Thread Andrea Bolognani
Signed-off-by: Andrea Bolognani Reviewed-by: Ján Tomko --- src/qemu/qemu_validate.c | 8 ...irtio-iommu-wrong-machine.x86_64-latest.err | 1 + .../virtio-iommu-wrong-machine.xml | 18 ++ tests/qemuxml2argvtest.c |

[libvirt PATCH v2 01/17] tests: Add capabilities for QEMU 7.0.0 on aarch64

2022-03-18 Thread Andrea Bolognani
The QEMU binary is built from the v7.0.0-rc0 tag. This causes the argument to -device to be generated in JSON format, same as what 1a691fe1c840 has done for x86_64. Signed-off-by: Andrea Bolognani --- .../qemu_7.0.0-virt.aarch64.xml | 193 + tests/domaincapsdata/qemu_7.0.0.aarch

[libvirt PATCH v2 05/17] qemu: Introduce QEMU_CAPS_VIRTIO_IOMMU_BOOT_BYPASS

2022-03-18 Thread Andrea Bolognani
This capability detects the availability of the boot-bypass property of the virtio-iommu-pci device. This property was only introduced in QEMU 7.0 but, since the device has been around for much longer, we end up querying its properties for several more releases. As I don't have convenient access t

[libvirt PATCH v2 03/17] qemu: Tweak some code

2022-03-18 Thread Andrea Bolognani
The altered code is functionally equivalent to the previous one, but it's already laid down in a way that will make further changes easier and less messy. Signed-off-by: Andrea Bolognani Reviewed-by: Ján Tomko --- src/qemu/qemu_command.c| 14 ++ src/qemu/qemu_domain_address.

[libvirt PATCH v2 02/17] conf: Introduce VIR_PCI_CONNECT_INTEGRATED

2022-03-18 Thread Andrea Bolognani
This new flag can be used to convince the PCI address assignment algorithm to place a device directly on the root bus. It will be used to implement support for virtio-iommu, which needs to be an integrated device in order to work correctly. Signed-off-by: Andrea Bolognani Reviewed-by: Ján Tomko

[libvirt PATCH v2 00/17] qemu: Implement virtio-iommu support

2022-03-18 Thread Andrea Bolognani
The first patch adds replies files and as such has been aggressively snipped to deal with mailing list message size limits. Grab the unabriged version with $ git fetch https://gitlab.com/abologna/libvirt.git virtio-iommu The QEMU side of the feature has now been merged, so this can finally go i

Re: [libvirt PATCH v2] nwfilter: merge updateMutex and updateLock

2022-03-18 Thread Michal Prívozník
On 3/17/22 16:30, Daniel P. Berrangé wrote: > The updateLock is a R/W lock held by anything which needs to read or > modify the rules associated with an NWFilter. > > APIs for defining/undefining NW filters rules hold a write lock on > updateLock. > > APIs for creating/deleting NW filter bindings

<    2   3   4   5   6   7   8   9   10   11   >