Re: [libvirt PATCH 2/2] ci: integration: Set 'safe.directory' when installing QEMU from git

2022-06-07 Thread Erik Skultety
On Wed, Jun 08, 2022 at 07:59:37AM +0200, Erik Skultety wrote: > Since a fix for CVE-2022-24765 was released every git command is now > checked against the context repo in which it's supposed to run > resulting in a fatal error if the repo is owned by other user than the > one running the git comma

[libvirt PATCH 2/2] ci: integration: Set 'safe.directory' when installing QEMU from git

2022-06-07 Thread Erik Skultety
Since a fix for CVE-2022-24765 was released every git command is now checked against the context repo in which it's supposed to run resulting in a fatal error if the repo is owned by other user than the one running the git command. This means that in order to be able to do 'sudo make install', we h

[libvirt PATCH 1/2] ci: integration: SELinux relabel the QEMU we installed from git

2022-06-07 Thread Erik Skultety
Signed-off-by: Erik Skultety --- ci/integration-template.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/ci/integration-template.yml b/ci/integration-template.yml index e2ccebd1f6..a04c72acbf 100644 --- a/ci/integration-template.yml +++ b/ci/integration-template.yml @@ -98,3 +98,4 @@

[libvirt PATCH 0/2] ci: Fixes to enable upstream QEMU integration tests on Fedora 36

2022-06-07 Thread Erik Skultety
- CVE-2022-24765 handling (>= git-2.35.2) - SELinux relabel the QEMU installation from source Successful pipeline: https://gitlab.com/eskultety/libvirt/-/pipelines/557712098 Erik Skultety (2): ci: integration: SELinux relabel the QEMU we installed from git ci: integration: Set 'safe.directory

Re: [PATCH v2 11/15] qemu: Wire up new virDomainSetIOThreadParams parameters

2022-06-07 Thread Peter Krempa
On Tue, Jun 07, 2022 at 17:06:50 +0200, Peter Krempa wrote: > On Tue, Jun 07, 2022 at 14:52:55 +0200, Michal Privoznik wrote: > > Introduced in previous commit, QEMU driver needs to be taught how > > to set VIR_DOMAIN_IOTHREAD_THREAD_POOL_MIN and > > VIR_DOMAIN_IOTHREAD_THREAD_POOL_MAX parameters o

Re: [PATCH v2 11/15] qemu: Wire up new virDomainSetIOThreadParams parameters

2022-06-07 Thread Peter Krempa
On Tue, Jun 07, 2022 at 14:52:55 +0200, Michal Privoznik wrote: > Introduced in previous commit, QEMU driver needs to be taught how > to set VIR_DOMAIN_IOTHREAD_THREAD_POOL_MIN and > VIR_DOMAIN_IOTHREAD_THREAD_POOL_MAX parameters on given IOThread. > Fortunately, this is fairly trivial to do and si

Re: [PATCH v2 12/15] virsh: Wire up new virDomainSetIOThreadParams parameters

2022-06-07 Thread Peter Krempa
On Tue, Jun 07, 2022 at 14:52:56 +0200, Michal Privoznik wrote: > Since virsh implements a wrapper over virDomainSetIOThreadParams() > (command iothreadset) let's wire up new typed parameters there too. > > Signed-off-by: Michal Privoznik > --- > docs/manpages/virsh.rst | 7 ++- > tools/vir

Re: [PATCH v2 11/15] qemu: Wire up new virDomainSetIOThreadParams parameters

2022-06-07 Thread Peter Krempa
On Tue, Jun 07, 2022 at 14:52:55 +0200, Michal Privoznik wrote: > Introduced in previous commit, QEMU driver needs to be taught how > to set VIR_DOMAIN_IOTHREAD_THREAD_POOL_MIN and > VIR_DOMAIN_IOTHREAD_THREAD_POOL_MAX parameters on given IOThread. > Fortunately, this is fairly trivial to do and si

Re: [PATCH v2 15/15] qemu: Generate command line for main-loop pool size

2022-06-07 Thread Peter Krempa
On Tue, Jun 07, 2022 at 14:52:59 +0200, Michal Privoznik wrote: > Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2059511 > Signed-off-by: Michal Privoznik > --- > src/qemu/qemu_command.c | 16 +--- > .../iothreads-ids-pool-sizes.x86_64-latest.args | 1

Re: [PATCH v2 14/15] qemu_validate: Check if QEMU's capable of setting main loop pool size

2022-06-07 Thread Peter Krempa
On Tue, Jun 07, 2022 at 14:52:58 +0200, Michal Privoznik wrote: > Since the main-loop and iothread classes are derived from the > same class (EventLoopBaseClass) we don't need new capability and > can use QEMU_CAPS_IOTHREAD_THREAD_POOL_MAX directly to check > whether QEMU's capable of setting worke

Re: [PATCH v2 08/15] qemu_validate: Check if QEMU's capable of setting iothread pool size

2022-06-07 Thread Peter Krempa
On Tue, Jun 07, 2022 at 14:52:52 +0200, Michal Privoznik wrote: > Now that we have a capability that reflects whether QEMU is > capable of setting iothread pool size, let's introduce a > validator check to make sure users are not trying to use this > feature with QEMU that doesn't support it. > >

Re: [PATCH v2 13/15] conf: Expose QEMU's main loop object

2022-06-07 Thread Peter Krempa
On Tue, Jun 07, 2022 at 14:52:57 +0200, Michal Privoznik wrote: > As of v7.0.0-877-g70ac26b9e5 QEMU exposes its main event loop as > an QMP object. In the very next commit (v7.0.0-878-g71ad4713cc) > it was extended for thread-pool-min and thread-pool-max > attributes. Expose them under new element

Re: [PATCH v2 10/15] include: Introduce typed params for virDomainSetIOThreadParams wrt pool size

2022-06-07 Thread Peter Krempa
On Tue, Jun 07, 2022 at 14:52:54 +0200, Michal Privoznik wrote: > Our public API offers virDomainSetIOThreadParams() function which > allows users to set various aspects of IOThreads. Introduce two > new typed parameters: VIR_DOMAIN_IOTHREAD_THREAD_POOL_MIN and > VIR_DOMAIN_IOTHREAD_THREAD_POOL_MAX

Re: [PATCH v2 09/15] qemu: Generate command line for IOThread pool size

2022-06-07 Thread Peter Krempa
On Tue, Jun 07, 2022 at 14:52:53 +0200, Michal Privoznik wrote: > Signed-off-by: Michal Privoznik > --- > src/qemu/qemu_command.c | 10 - > ...othreads-ids-pool-sizes.x86_64-latest.args | 44 +++ > tests/qemuxml2argvtest.c | 1 + > 3

Re: [PATCH v2 06/15] conf: Introduce pool_min and pool_max attributes to IOThread

2022-06-07 Thread Peter Krempa
While at it you can update the summary ^^ On Tue, Jun 07, 2022 at 14:52:50 +0200, Michal Privoznik wrote: > At least in case of QEMU an IOThread is actually a pool of > threads (see iothread_set_aio_context_params() in QEMU's code > base). As such, it can have minimal and maximal number of work

Re: [PATCH v2 23/30] storage_file: Remove unused includes

2022-06-07 Thread Ján Tomko
On a Saturday in 2022, Peng Liang wrote: Signed-off-by: Peng Liang --- src/storage_file/storage_file_backend.c | 2 -- src/storage_file/storage_file_backend_fs.c | 4 src/storage_file/storage_file_backend_gluster.c | 2 -- src/storage_file/storage_source.c | 2 -- 4 f

Re: [PATCH v2 20/30] secret: Remove unused includes

2022-06-07 Thread Ján Tomko
On a Saturday in 2022, Peng Liang wrote: Signed-off-by: Peng Liang --- src/secret/secret_driver.c | 2 -- 1 file changed, 2 deletions(-) Reviewed-by: Ján Tomko Jano signature.asc Description: PGP signature

Re: [PATCH v2 15/30] nwfilter: Remove unused includes

2022-06-07 Thread Ján Tomko
On a Saturday in 2022, Peng Liang wrote: Signed-off-by: Peng Liang --- src/nwfilter/nwfilter_dhcpsnoop.c | 2 -- src/nwfilter/nwfilter_driver.c| 5 - src/nwfilter/nwfilter_driver.h| 3 --- src/nwfilter/nwfilter_ebiptables_driver.c | 4 src/nwfilter/nwfilter_g

Re: [PATCH v2 09/30] interface: Remove unused includes

2022-06-07 Thread Ján Tomko
On a Saturday in 2022, Peng Liang wrote: Signed-off-by: Peng Liang --- src/interface/interface_backend_netcf.c | 2 -- src/interface/interface_backend_udev.c | 1 - 2 files changed, 3 deletions(-) Reviewed-by: Ján Tomko Jano signature.asc Description: PGP signature

Re: [PATCH v2 21/30] security: Remove unused includes

2022-06-07 Thread Ján Tomko
On a Saturday in 2022, Peng Liang wrote: Signed-off-by: Peng Liang --- src/security/security_dac.c | 1 - src/security/security_manager.h | 1 - src/security/security_nop.c | 2 -- src/security/security_selinux.c | 2 -- src/security/security_stack.c | 1 - src/security/security_util.c|

Re: [PATCH v2 01/30] access: Remove unused includes

2022-06-07 Thread Ján Tomko
On a Saturday in 2022, Peng Liang wrote: Signed-off-by: Peng Liang --- src/access/viraccessdriver.h | 1 - src/access/viraccessdriverpolkit.c | 4 src/access/viraccessdriverstack.c | 1 - src/access/viraccessmanager.h | 1 - src/access/viraccessperm.h | 1 - 5 files changed,

[PATCH v2 15/15] qemu: Generate command line for main-loop pool size

2022-06-07 Thread Michal Privoznik
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2059511 Signed-off-by: Michal Privoznik --- src/qemu/qemu_command.c | 16 +--- .../iothreads-ids-pool-sizes.x86_64-latest.args | 1 + 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/src/qe

[PATCH v2 13/15] conf: Expose QEMU's main loop object

2022-06-07 Thread Michal Privoznik
As of v7.0.0-877-g70ac26b9e5 QEMU exposes its main event loop as an QMP object. In the very next commit (v7.0.0-878-g71ad4713cc) it was extended for thread-pool-min and thread-pool-max attributes. Expose them under new element. Signed-off-by: Michal Privoznik --- docs/formatdomain.rst

[PATCH v2 12/15] virsh: Wire up new virDomainSetIOThreadParams parameters

2022-06-07 Thread Michal Privoznik
Since virsh implements a wrapper over virDomainSetIOThreadParams() (command iothreadset) let's wire up new typed parameters there too. Signed-off-by: Michal Privoznik --- docs/manpages/virsh.rst | 7 ++- tools/virsh-domain.c| 24 +++- 2 files changed, 29 insertions(+

[PATCH v2 14/15] qemu_validate: Check if QEMU's capable of setting main loop pool size

2022-06-07 Thread Michal Privoznik
Since the main-loop and iothread classes are derived from the same class (EventLoopBaseClass) we don't need new capability and can use QEMU_CAPS_IOTHREAD_THREAD_POOL_MAX directly to check whether QEMU's capable of setting worker pool size. Signed-off-by: Michal Privoznik --- src/qemu/qemu_valida

[PATCH v2 10/15] include: Introduce typed params for virDomainSetIOThreadParams wrt pool size

2022-06-07 Thread Michal Privoznik
Our public API offers virDomainSetIOThreadParams() function which allows users to set various aspects of IOThreads. Introduce two new typed parameters: VIR_DOMAIN_IOTHREAD_THREAD_POOL_MIN and VIR_DOMAIN_IOTHREAD_THREAD_POOL_MAX which will allow users to modify the thread-pool-min and thread-pool-ma

[PATCH v2 09/15] qemu: Generate command line for IOThread pool size

2022-06-07 Thread Michal Privoznik
Signed-off-by: Michal Privoznik --- src/qemu/qemu_command.c | 10 - ...othreads-ids-pool-sizes.x86_64-latest.args | 44 +++ tests/qemuxml2argvtest.c | 1 + 3 files changed, 53 insertions(+), 2 deletions(-) create mode 100644 tests/

[PATCH v2 11/15] qemu: Wire up new virDomainSetIOThreadParams parameters

2022-06-07 Thread Michal Privoznik
Introduced in previous commit, QEMU driver needs to be taught how to set VIR_DOMAIN_IOTHREAD_THREAD_POOL_MIN and VIR_DOMAIN_IOTHREAD_THREAD_POOL_MAX parameters on given IOThread. Fortunately, this is fairly trivial to do and since these two parameters are exposed in domain XML too the update of ina

[PATCH v2 07/15] qemu: Introduce QEMU_CAPS_IOTHREAD_THREAD_POOL_MAX

2022-06-07 Thread Michal Privoznik
This capability reflects whether QEMU allows setting thread-pool-min and thread-pool-max attributes on iothread object. Since both attributes were introduced in the same commit (v7.0.0-878-g71ad4713cc) and can't exist independently of each other we can stick with one capability covering both of the

[PATCH v2 03/15] virDomainIOThreadIDDefArrayInit: Decrease scope of @iothrid

2022-06-07 Thread Michal Privoznik
In virDomainIOThreadIDDefArrayInit() the variable @iothrid is used only inside a loop but is declared for whole function. Bring the variable into the loop so that it's obvious that the variable is not used elsewhere. Signed-off-by: Michal Privoznik Reviewed-by: Peter Krempa --- src/conf/domain_

[PATCH v2 08/15] qemu_validate: Check if QEMU's capable of setting iothread pool size

2022-06-07 Thread Michal Privoznik
Now that we have a capability that reflects whether QEMU is capable of setting iothread pool size, let's introduce a validator check to make sure users are not trying to use this feature with QEMU that doesn't support it. Signed-off-by: Michal Privoznik Reviewed-by: Peter Krempa --- src/qemu/qe

[PATCH v2 02/15] virDomainDefParseIOThreads: Use g_autoptr() for @iothrid

2022-06-07 Thread Michal Privoznik
Using g_autoptr() for @iothrid variable inside virDomainDefParseIOThreads() allows us to drop explicit call to virDomainIOThreadIDDefFree() in one case. Signed-off-by: Michal Privoznik Reviewed-by: Peter Krempa --- src/conf/domain_conf.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(

[PATCH v2 06/15] conf: Introduce pool_min and pool_max attributes to IOThread

2022-06-07 Thread Michal Privoznik
At least in case of QEMU an IOThread is actually a pool of threads (see iothread_set_aio_context_params() in QEMU's code base). As such, it can have minimal and maximal number of worker threads. Allow setting them in domain XML. Signed-off-by: Michal Privoznik --- docs/formatdomain.rst

[PATCH v2 05/15] conf: Introduce allocator for virDomainIOThreadIDDef

2022-06-07 Thread Michal Privoznik
So far, iothread configuration structure (virDomainIOThreadIDDef) is allocated by plain g_new0(). This is perfectly okay because all members of the struct default to value 0 anyway. But soon this is going to change. Therefore, replace those g_new0() with a function so that the default value can be

[PATCH v2 04/15] conf: Move iothread formatter into a separate function

2022-06-07 Thread Michal Privoznik
Formatting iothreads is currently open coded inside of virDomainDefFormatInternalSetRootName(). While this works, it makes the function needlessly long, especially if the formatting code will expand in near future. Therefore, move it into a separate function. At the same time, make virDomainDefIoth

[PATCH v2 01/15] virml: Introduce VIR_XML_PROP_NONNEGATIVE flag

2022-06-07 Thread Michal Privoznik
For easier attribute parsing we have virXMLProp*() family of functions. These accept flags through which a caller can pose some conditions onto the attribute value, for instance: VIR_XML_PROP_NONZERO when the attribute may not be zero, etc. What we are missing is VIR_XML_PROP_NONNEGATIVE when the

[PATCH v2 00/15] qemu: Allow setting EventLoopBaseProperties

2022-06-07 Thread Michal Privoznik
v2 of: https://listman.redhat.com/archives/libvir-list/2022-June/232118.html diff to v1: - switched from to - switched from long long to int for pool sizes (this means that the 02/16 patch from the original series that introduced virXMLPropLongLong() is no longer needed and thus not in this

Re: [libvirt PATCH v2 0/7] ci: Drop Fedora 34, add Fedora 36

2022-06-07 Thread Erik Skultety
On Tue, Jun 07, 2022 at 02:34:50AM -0700, Andrea Bolognani wrote: > On Thu, May 26, 2022 at 04:24:41PM +0200, Erik Skultety wrote: > > On Thu, May 26, 2022 at 04:01:45PM +0200, Andrea Bolognani wrote: > > > Test pipeline: > > > > > > https://gitlab.com/abologna/libvirt/-/pipelines/548848259 > > >

Re: [libvirt RFCv11 14/33] virfile: add new API virFileDiskCopyChannel

2022-06-07 Thread Claudio Fontana
On 6/7/22 11:19, Claudio Fontana wrote: > allow interleaved parallel write to a single file, > using a record size equal to the io buffer size (1MB). > > Signed-off-by: Claudio Fontana > --- > src/util/iohelper.c | 3 + > src/util/virfile.c | 151 +--- >

Re: [libvirt PATCH v2 0/7] ci: Drop Fedora 34, add Fedora 36

2022-06-07 Thread Andrea Bolognani
On Thu, May 26, 2022 at 04:24:41PM +0200, Erik Skultety wrote: > On Thu, May 26, 2022 at 04:01:45PM +0200, Andrea Bolognani wrote: > > Test pipeline: > > > > https://gitlab.com/abologna/libvirt/-/pipelines/548848259 > > > > Only patches 1-5 should be pushed until the issues outlined in > > Go ahe

[libvirt RFCv11 14/33] virfile: add new API virFileDiskCopyChannel

2022-06-07 Thread Claudio Fontana
allow interleaved parallel write to a single file, using a record size equal to the io buffer size (1MB). Signed-off-by: Claudio Fontana --- src/util/iohelper.c | 3 + src/util/virfile.c | 151 +--- src/util/virfile.h | 2 + 3 files changed, 106 inse

[libvirt RFCv11 16/33] qemu: saveimage: update virQEMUSaveFd struct for parallel save

2022-06-07 Thread Claudio Fontana
add nchannels to the struct and adapt the APIs slightly. Add a new API virQEMUSaveFdAddChannels to set the number of channels, which might not be known at Init time. Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c| 10 +- src/qemu/qemu_saveimage.c | 27 +

[libvirt RFCv11 18/33] qemu: capabilities: add multifd to the probed migration capabilities

2022-06-07 Thread Claudio Fontana
Signed-off-by: Claudio Fontana Reviewed-by: Ani Sinha --- src/qemu/qemu_capabilities.c | 2 ++ src/qemu/qemu_capabilities.h | 1 + tests/qemucapabilitiesdata/caps_4.0.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_4.0.0.ppc64.xml | 1 + tests/qem

[libvirt RFCv11 21/33] qemu: migration: implement qemuMigrationSrcToFilesMultiFd for save

2022-06-07 Thread Claudio Fontana
implement a function similar to qemuMigrationSrcToFile that migrates to multiple files using QEMU multifd, and use it for VIR_DOMAIN_SAVE_PARALLEL saves. Signed-off-by: Claudio Fontana --- src/qemu/qemu_migration.c | 131 +- src/qemu/qemu_migration.h | 7 ++

[libvirt RFCv11 27/33] libvirt: add new VIR_DOMAIN_SAVE_PARAM_PARALLEL_COMPRESSION

2022-06-07 Thread Claudio Fontana
Signed-off-by: Claudio Fontana --- include/libvirt/libvirt-domain.h | 11 +++ 1 file changed, 11 insertions(+) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 0b73ee27b6..0858d02e48 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt

[libvirt RFCv11 31/33] qemu: saveimage: implement multifd-compression in parallel save

2022-06-07 Thread Claudio Fontana
Signed-off-by: Claudio Fontana --- src/qemu/qemu_migration.c | 17 + src/qemu/qemu_migration.h | 2 +- src/qemu/qemu_saveimage.c | 15 --- 3 files changed, 26 insertions(+), 8 deletions(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index c214f

[libvirt RFCv11 33/33] tools: add parallel-compression parameter to virsh save command

2022-06-07 Thread Claudio Fontana
this completes the save side of the parallel compression support. Signed-off-by: Claudio Fontana --- docs/manpages/virsh.rst | 4 tools/virsh-domain.c| 12 2 files changed, 16 insertions(+) diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index 5abb03be9a..1

[libvirt RFCv11 17/33] qemu: saveimage: wire up saveimage code with the multifd helper

2022-06-07 Thread Claudio Fontana
prepare to use the multifd helper. Signed-off-by: Claudio Fontana --- src/qemu/qemu_saveimage.c | 41 ++- 1 file changed, 36 insertions(+), 5 deletions(-) diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 6418114d8a..8af9b92b35 100644 -

[libvirt RFCv11 20/33] qemu: migration_params: add APIs to set Int and Cap

2022-06-07 Thread Claudio Fontana
similarly to qemuMigrationParamsSetULL, we need to be able to set fields from qemu_saveimage. Signed-off-by: Claudio Fontana --- src/qemu/qemu_migration_params.c | 22 ++ src/qemu/qemu_migration_params.h | 9 + 2 files changed, 31 insertions(+) diff --git a/src/qemu

[libvirt RFCv11 30/33] qemu: migration: expose qemuMigrationParamsSetString

2022-06-07 Thread Claudio Fontana
change from static to external linkage, and move the function close to the other similar ones, near qemuMigrationParamsSetULL. Signed-off-by: Claudio Fontana --- src/qemu/qemu_migration_params.c | 47 +++- src/qemu/qemu_migration_params.h | 5 2 files changed, 2

[libvirt RFCv11 24/33] tools: add parallel parameter to virsh save command

2022-06-07 Thread Claudio Fontana
Signed-off-by: Claudio Fontana --- docs/manpages/virsh.rst | 11 ++- tools/virsh-domain.c| 24 ++-- 2 files changed, 32 insertions(+), 3 deletions(-) diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index 965b89c99d..9f33b0aaef 100644 --- a/docs/manp

[libvirt RFCv11 23/33] qemu: implement qemuSaveImageLoadMultiFd for restore

2022-06-07 Thread Claudio Fontana
use multifd to restore parallel images, if VIR_DOMAIN_SAVE_PARALLEL is enabled. Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c| 16 +- src/qemu/qemu_migration.c | 2 +- src/qemu/qemu_migration.h | 6 ++ src/qemu/qemu_saveimage.c | 114 +

[libvirt RFCv11 25/33] tools: add parallel parameter to virsh restore command

2022-06-07 Thread Claudio Fontana
Signed-off-by: Claudio Fontana --- docs/manpages/virsh.rst | 11 +-- tools/virsh-domain.c| 10 +- 2 files changed, 18 insertions(+), 3 deletions(-) diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index 9f33b0aaef..5abb03be9a 100644 --- a/docs/manpages/virsh.rst

[libvirt RFCv11 26/33] qemu: add migration parameter multifd-compression

2022-06-07 Thread Claudio Fontana
add it to both capabilities and migration parameters Signed-off-by: Claudio Fontana --- src/qemu/qemu_capabilities.c | 2 ++ src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_migration_params.c | 2 ++ src/qemu/qemu_migration_params.h

[libvirt RFCv11 19/33] qemu: saveimage: add multifd related fields to save format

2022-06-07 Thread Claudio Fontana
add both multifd compression and number of multifd channels Signed-off-by: Claudio Fontana --- src/qemu/qemu_saveimage.c | 17 + src/qemu/qemu_saveimage.h | 4 +++- 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage

[libvirt RFCv11 32/33] qemu: saveimage: restore compressed parallel images

2022-06-07 Thread Claudio Fontana
Signed-off-by: Claudio Fontana --- src/qemu/qemu_saveimage.c | 17 + 1 file changed, 17 insertions(+) diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index f60040ef25..d3757254a9 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -708,6 +

[libvirt RFCv11 01/33] virfile: introduce virFileDirect APIs

2022-06-07 Thread Claudio Fontana
these functions help with allocating buffers, aligning, reading and writing files opened with O_DIRECT. Signed-off-by: Claudio Fontana --- src/libvirt_private.syms | 6 ++ src/util/virfile.c | 174 +++ src/util/virfile.h | 9 ++ 3 files changed

[libvirt RFCv11 13/33] qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in restore

2022-06-07 Thread Claudio Fontana
Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 5a8d3d1309..986bbf9e58 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -5869,7 +5869,8 @@ qemuD

[libvirt RFCv11 29/33] qemu: saveimage: add stub support for multifd compression parameter

2022-06-07 Thread Claudio Fontana
Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 8839e0031d..52f3300c55 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -2874,6 +2874,7 @@ q

[libvirt RFCv11 12/33] qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in save

2022-06-07 Thread Claudio Fontana
and its companion param VIR_SAVE_PARAM_PARALLEL_CONNECTIONS Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c| 18 -- src/qemu/qemu_saveimage.c | 1 + src/qemu/qemu_saveimage.h | 1 + src/qemu/qemu_snapshot.c | 2 +- 4 files changed, 15 insertions(+), 7 deletions(

[libvirt RFCv11 03/33] qemu: saveimage: rework image read/write to be O_DIRECT friendly

2022-06-07 Thread Claudio Fontana
maintain a compatible image format (still QEMU_SAVE_VERSION 2), but change the on-disk representation to be more O_DIRECT friendly: 1) ensure that the header struct fields are packed, so we can be sure no padding will ever ruin the day in the future 2) finish the libvirt header (header + xml +

[libvirt RFCv11 15/33] multifd-helper: new helper for parallel save/restore

2022-06-07 Thread Claudio Fontana
For the save direction, this helper listens on a unix socket which QEMU connects to for multifd migration to a file. For the restore direction, this helper connects to a unix socket QEMU listens at for multifd migration from a file. The file descriptor is passed as a command line parameter, and i

[libvirt RFCv11 28/33] qemu: saveimage: add parallel compression argument to ImageCreate

2022-06-07 Thread Claudio Fontana
Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c| 11 ++- src/qemu/qemu_saveimage.c | 1 + src/qemu/qemu_saveimage.h | 1 + src/qemu/qemu_snapshot.c | 2 +- 4 files changed, 9 insertions(+), 6 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c inde

[libvirt RFCv11 22/33] qemu: add parameter to qemuMigrationDstRun to skip waiting

2022-06-07 Thread Claudio Fontana
The distinction on whether to wait for the migration completion or not was made on the async job type, but with the future addition of multifd migration from files, we need a way to avoid waiting, so we can prepare multifd migration parameters before starting the transfers. Adapt all callers. Si

[libvirt RFCv11 06/33] qemu: saveimage: introduce virQEMUSaveFd

2022-06-07 Thread Claudio Fontana
use this data type to encapsulate the pathname, file descriptor, wrapper, and need to unlink. We also carry along the oflags used to open the file, and the driver configuration. This will make management of the resources associated with an FD used for QEMU save/restore much easier, reducing the a

[libvirt RFCv11 04/33] qemu: saveimage: assume future formats will also support compression

2022-06-07 Thread Claudio Fontana
change the version check to assume even future versions of the format will continue to support compression. Signed-off-by: Claudio Fontana --- src/qemu/qemu_saveimage.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index

[libvirt RFCv11 09/33] tools: prepare doSave to use parameters

2022-06-07 Thread Claudio Fontana
Signed-off-by: Claudio Fontana --- tools/virsh-domain.c | 23 +-- 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index bac4cc0fb6..120d0ef5f0 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -4203,6 +42

[libvirt RFCv11 08/33] qemu: saveimage: convert qemuSaveImageOpen to use virQEMUSaveFd

2022-06-07 Thread Claudio Fontana
all the logic to open a fd, create a wrapper etc, is boilerplate code that is best reused, so change the Open function to take an existing already initialized virQEMUSaveFd. Adapt all callers of qemuSaveImageOpen. Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c| 101 ++

[libvirt RFCv11 05/33] virfile: virFileDiskCopy: prepare for O_DIRECT files without wrapper

2022-06-07 Thread Claudio Fontana
we will allow to use already open fds that are not empty for both read and write, as long as they are properly aligned. Adapt the truncation to take into account the initial offset. Signed-off-by: Claudio Fontana --- src/util/virfile.c | 44 +++- 1 file c

[libvirt RFCv11 02/33] virfile: use virFileDirect API in runIOCopy

2022-06-07 Thread Claudio Fontana
Signed-off-by: Claudio Fontana --- src/util/virfile.c | 28 1 file changed, 8 insertions(+), 20 deletions(-) diff --git a/src/util/virfile.c b/src/util/virfile.c index e8500704e5..770649108f 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -4776,20 +4776,1

[libvirt RFCv11 07/33] qemu: saveimage: convert qemuSaveImageCreate to use virQEMUSaveFd

2022-06-07 Thread Claudio Fontana
now that we introduced virQEMUSaveFd, use it in the creation of a new save image. Signed-off-by: Claudio Fontana --- src/qemu/qemu_saveimage.c | 54 +++ 1 file changed, 15 insertions(+), 39 deletions(-) diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_s

[libvirt RFCv11 11/33] libvirt: add new VIR_DOMAIN_SAVE_PARALLEL flag and parameter

2022-06-07 Thread Claudio Fontana
in order to enable parallel save functionality, we will need an opportune new flag and a parameter to specify the number of extra connections to use. Signed-off-by: Claudio Fontana --- include/libvirt/libvirt-domain.h | 13 + 1 file changed, 13 insertions(+) diff --git a/include/lib

[libvirt RFCv11 00/33] multifd save restore prototype

2022-06-07 Thread Claudio Fontana
This is v11 of the multifd save prototype, which focuses on saving to a single file instead of requiring multiple separate files for multifd channels. This series demonstrates a way to save and restore from a single file by using interleaved channels of a size equal to the transfer buffer size (1M

[libvirt RFCv11 10/33] tools: prepare cmdRestore to use parameters

2022-06-07 Thread Claudio Fontana
Signed-off-by: Claudio Fontana --- tools/virsh-domain.c | 32 ++-- 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 120d0ef5f0..a3f007a1d2 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -

Re: [PATCH 14/16] conf: Expose QEMU's main loop object

2022-06-07 Thread Daniel P . Berrangé
On Tue, Jun 07, 2022 at 10:06:16AM +0200, Michal Prívozník wrote: > On 6/6/22 13:40, Daniel P. Berrangé wrote: > > On Thu, Jun 02, 2022 at 01:54:39PM +0200, Peter Krempa wrote: > >> On Thu, Jun 02, 2022 at 09:18:04 +0200, Michal Privoznik wrote: > >>> As of v7.0.0-877-g70ac26b9e5 QEMU exposes its m

Re: [PATCH 14/16] conf: Expose QEMU's main loop object

2022-06-07 Thread Peter Krempa
On Tue, Jun 07, 2022 at 10:06:16 +0200, Michal Prívozník wrote: > On 6/6/22 13:40, Daniel P. Berrangé wrote: > > On Thu, Jun 02, 2022 at 01:54:39PM +0200, Peter Krempa wrote: > >> On Thu, Jun 02, 2022 at 09:18:04 +0200, Michal Privoznik wrote: > >>> As of v7.0.0-877-g70ac26b9e5 QEMU exposes its mai

Re: [PATCH 14/16] conf: Expose QEMU's main loop object

2022-06-07 Thread Michal Prívozník
On 6/6/22 13:40, Daniel P. Berrangé wrote: > On Thu, Jun 02, 2022 at 01:54:39PM +0200, Peter Krempa wrote: >> On Thu, Jun 02, 2022 at 09:18:04 +0200, Michal Privoznik wrote: >>> As of v7.0.0-877-g70ac26b9e5 QEMU exposes its main event loop as >>> an QMP object. In the very next commit (v7.0.0-878-g

Re: [PATCH] qemucapabilitiestest: Update latest caps to 'v7.0.0-1512-gca127b3fc2'

2022-06-07 Thread Michal Prívozník
On 6/6/22 17:32, Peter Krempa wrote: > Notable changes: > > - Icelake-Client cpu model family removed: > "Icelake-Client-noTSX-x86_64-cpu" > "Icelake-Client-v1-x86_64-cpu" > "Icelake-Client-v2-x86_64-cpu" > "Icelake-Client-v3-x86_64-cpu" > "Icelake-Client-x86_64-cpu" > >