RE: [libvirt][PATCH v11 1/4] qemu: provide support to query the SGX capability

> -Original Message- > From: Peter Krempa > Sent: Thursday, May 12, 2022 12:05 AM > To: Yang, Lin A > Cc: libvir-list@redhat.com; Huang, Haibin ; Ding, > Jian-feng ; Zhong, Yang > Subject: Re: [libvirt][PATCH v11 1/4] qemu: provide support to query the SGX > capability > > On Tue, Ma

[libvirt RFCv10 01/32] virfile: introduce virFileDirect APIs

these functions help with allocating buffers, aligning, reading and writing files opened with O_DIRECT. Signed-off-by: Claudio Fontana --- src/libvirt_private.syms | 8 ++ src/util/virfile.c | 249 +++ src/util/virfile.h | 11 ++ 3 files changed

[libvirt RFCv10 32/32] tools: add parallel-compression parameter to virsh save command

this completes the save side of the parallel compression support. Signed-off-by: Claudio Fontana --- docs/manpages/virsh.rst | 4 tools/virsh-domain.c| 12 2 files changed, 16 insertions(+) diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index dee748d870..5

[libvirt RFCv10 30/32] qemu: saveimage: implement multifd-compression in parallel save

Signed-off-by: Claudio Fontana --- src/qemu/qemu_migration.c | 17 + src/qemu/qemu_migration.h | 2 +- src/qemu/qemu_saveimage.c | 19 ++- 3 files changed, 28 insertions(+), 10 deletions(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index

[libvirt RFCv10 09/32] tools: prepare doSave to use parameters

Signed-off-by: Claudio Fontana --- tools/virsh-domain.c | 23 +-- 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index ba492e807e..8204d44dcd 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -4203,6 +42

[libvirt RFCv10 27/32] qemu: saveimage: add parallel compression argument to ImageCreate

Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c| 11 ++- src/qemu/qemu_saveimage.c | 1 + src/qemu/qemu_saveimage.h | 1 + src/qemu/qemu_snapshot.c | 2 +- 4 files changed, 9 insertions(+), 6 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c inde

[libvirt RFCv10 13/32] qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in restore

Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 4114d8919b..d071df1c81 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -5855,7 +5855,8 @@ qemuD

[libvirt RFCv10 19/32] qemu: migration_params: add APIs to set Int and Cap

similarly to qemuMigrationParamsSetULL, we need to be able to set fields from qemu_saveimage. Signed-off-by: Claudio Fontana --- src/qemu/qemu_migration_params.c | 22 ++ src/qemu/qemu_migration_params.h | 9 + 2 files changed, 31 insertions(+) diff --git a/src/qemu

[libvirt RFCv10 08/32] qemu: saveimage: convert qemuSaveImageOpen to use virQEMUSaveFd

all the logic to open a fd, create a wrapper etc, is boilerplate code that is best reused, so change the Open function to take an existing already initialized virQEMUSaveFd. Adapt all callers of qemuSaveImageOpen. Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c| 101 ++

[libvirt RFCv10 14/32] multifd-helper: new helper for parallel save/restore

For the save direction, this helper listens on a unix socket which QEMU connects to for multifd migration to files. For the restore direction, this helper connects to a unix socket QEMU listens at for multifd migration from files. The file descriptors are passed as command line parameters. Signe

[libvirt RFCv10 24/32] tools: add parallel parameter to virsh restore command

Signed-off-by: Claudio Fontana --- docs/manpages/virsh.rst | 12 ++-- tools/virsh-domain.c| 10 +- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index e9012b85d1..dee748d870 100644 --- a/docs/manpages/virsh.rs

[libvirt RFCv10 31/32] qemu: saveimage: restore compressed parallel images

Signed-off-by: Claudio Fontana --- src/qemu/qemu_saveimage.c | 17 + 1 file changed, 17 insertions(+) diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index 96330430e5..669e6eb97f 100644 --- a/src/qemu/qemu_saveimage.c +++ b/src/qemu/qemu_saveimage.c @@ -798,6 +

[libvirt RFCv10 25/32] qemu: add migration parameter multifd-compression

add it to both capabilities and migration parameters Signed-off-by: Claudio Fontana --- src/qemu/qemu_capabilities.c | 2 ++ src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_migration_params.c | 2 ++ src/qemu/qemu_migration_params.h

[libvirt RFCv10 23/32] tools: add parallel parameter to virsh save command

Signed-off-by: Claudio Fontana --- docs/manpages/virsh.rst | 23 +-- tools/virsh-domain.c| 24 ++-- 2 files changed, 39 insertions(+), 8 deletions(-) diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index e73e590754..e9012b85d1 100644 ---

[libvirt RFCv10 29/32] qemu: migration: expose qemuMigrationParamsSetString

change from static to external linkage, and move the function close to the other similar ones, near qemuMigrationParamsSetULL. Signed-off-by: Claudio Fontana --- src/qemu/qemu_migration_params.c | 47 +++- src/qemu/qemu_migration_params.h | 5 2 files changed, 2

[libvirt RFCv10 20/32] qemu: migration: implement qemuMigrationSrcToFilesMultiFd for save

implement a function similar to qemuMigrationSrcToFile that migrates to multiple files using QEMU multifd, and use it for VIR_DOMAIN_SAVE_PARALLEL saves. Signed-off-by: Claudio Fontana --- src/qemu/qemu_migration.c | 131 +- src/qemu/qemu_migration.h | 7 ++

[libvirt RFCv10 17/32] qemu: capabilities: add multifd to the probed migration capabilities

Signed-off-by: Claudio Fontana Reviewed-by: Ani Sinha --- src/qemu/qemu_capabilities.c | 2 ++ src/qemu/qemu_capabilities.h | 1 + tests/qemucapabilitiesdata/caps_4.0.0.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_4.0.0.ppc64.xml | 1 + tests/qem

[libvirt RFCv10 00/32] multifd save restore prototype

This is v10 of the multifd save prototype, which improves the DIRECT I/O APIs and makes changes to the on disk format backward and forward compatible, avoiding the need to alter the QEMU_SAVE_VERSION. This series demonstrates the DIRECT I/O friendly read/write code working by using the block-frien

[libvirt RFCv10 26/32] libvirt: add new VIR_DOMAIN_SAVE_PARAM_PARALLEL_COMPRESSION

Signed-off-by: Claudio Fontana --- include/libvirt/libvirt-domain.h | 11 +++ 1 file changed, 11 insertions(+) diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index 766e4d116e..c79a3c85e9 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt

[libvirt RFCv10 21/32] qemu: add parameter to qemuMigrationDstRun to skip waiting

The distinction on whether to wait for the migration completion or not was made on the async job type, but with the future addition of multifd migration from files, we need a way to avoid waiting, so we can prepare multifd migration parameters before starting the transfers. Adapt all callers. Si

[libvirt RFCv10 18/32] qemu: saveimage: add multifd related fields to save format

add both multifd compression and number of multifd channels Signed-off-by: Claudio Fontana --- src/qemu/qemu_saveimage.c | 17 + src/qemu/qemu_saveimage.h | 4 +++- 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage

[libvirt RFCv10 22/32] qemu: implement qemuSaveImageLoadMultiFd for restore

use multifd to restore parallel images, if VIR_DOMAIN_SAVE_PARALLEL is enabled. Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c| 16 - src/qemu/qemu_migration.c | 2 +- src/qemu/qemu_migration.h | 6 ++ src/qemu/qemu_saveimage.c | 119 +-

[libvirt RFCv10 10/32] tools: prepare cmdRestore to use parameters

Signed-off-by: Claudio Fontana --- tools/virsh-domain.c | 32 ++-- 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 8204d44dcd..8a3c9d53d4 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -

[libvirt RFCv10 07/32] qemu: saveimage: convert qemuSaveImageCreate to use virQEMUSaveFd

now that we introduced virQEMUSaveFd, use it in the creation of a new save image. Signed-off-by: Claudio Fontana --- src/qemu/qemu_saveimage.c | 54 +++ 1 file changed, 15 insertions(+), 39 deletions(-) diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_s

[libvirt RFCv10 11/32] libvirt: add new VIR_DOMAIN_SAVE_PARALLEL flag and parameter

in order to enable parallel save functionality, we will need an opportune new flag and a parameter to specify the number of extra connections to use. Signed-off-by: Claudio Fontana --- include/libvirt/libvirt-domain.h | 18 ++ 1 file changed, 18 insertions(+) diff --git a/includ

[libvirt RFCv10 28/32] qemu: saveimage: add stub support for multifd compression parameter

Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 6ea23ee187..faaa6d4243 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -2865,6 +2865,7 @@ q

[libvirt RFCv10 16/32] qemu: saveimage: wire up saveimage code with the multifd helper

use the multifd helper and the new virQEMUSaveFd APIs for multifd. Signed-off-by: Claudio Fontana --- src/qemu/qemu_saveimage.c | 42 --- 1 file changed, 39 insertions(+), 3 deletions(-) diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index

[libvirt RFCv10 05/32] virfile: virFileDiskCopy: prepare for O_DIRECT files without wrapper

we will allow to use already open fds that are not empty for both read and write, as long as they are properly aligned. Adapt the truncation to take into account the initial offset. Signed-off-by: Claudio Fontana --- src/util/virfile.c | 39 +-- 1 file change

[libvirt RFCv10 15/32] qemu: saveimage: add virQEMUSaveFd APIs for multifd

add APIs to Create, Close and Free MultiFD files to associate with multifd channels. Adapt virQEMUSaveFdInit to consider multifd. Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c| 10 ++-- src/qemu/qemu_saveimage.c | 117 +++--- src/qemu/qemu_saveima

[libvirt RFCv10 12/32] qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in save

and its companion param VIR_SAVE_PARAM_PARALLEL_CONNECTIONS Signed-off-by: Claudio Fontana --- src/qemu/qemu_driver.c| 18 -- src/qemu/qemu_saveimage.c | 1 + src/qemu/qemu_saveimage.h | 1 + src/qemu/qemu_snapshot.c | 2 +- 4 files changed, 15 insertions(+), 7 deletions(

[libvirt RFCv10 03/32] qemu: saveimage: rework image read/write to be O_DIRECT friendly

maintain a compatible image format (still QEMU_SAVE_VERSION 2), but change the on-disk representation to be more O_DIRECT friendly: 1) ensure that the header struct fields are packed, so we can be sure no padding will ever ruin the day in the future 2) finish the libvirt header (header + xml +

[libvirt RFCv10 04/32] qemu: saveimage: assume future formats will also support compression

change the version check to assume even future versions of the format will continue to support compression. Signed-off-by: Claudio Fontana --- src/qemu/qemu_saveimage.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_saveimage.c index

[libvirt RFCv10 06/32] qemu: saveimage: introduce virQEMUSaveFd

use this data type to encapsulate the pathname, file descriptor, wrapper, and need to unlink. This will make management of the resources associated with an FD used for QEMU save/restore much easier, reducing the amount of explicit cleanup required. Signed-off-by: Claudio Fontana --- src/qemu/qe

[libvirt RFCv10 02/32] virfile: use virFileDirect API in runIOCopy

Signed-off-by: Claudio Fontana --- src/util/virfile.c | 28 1 file changed, 8 insertions(+), 20 deletions(-) diff --git a/src/util/virfile.c b/src/util/virfile.c index 03a7cdc9bf..c529598595 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -4851,20 +4851,1

Re: [libvirt RFCv9 03/31] qemu: saveimage: rework save format and read/write to be O_DIRECT friendly

On 5/14/22 5:52 PM, Claudio Fontana wrote: > change the saveimage format to: > > 1) ensure that the header struct fields are packed, so we can be sure >no padding will ruin the day > > 2) finish the libvirt header (header + xml + cookie) with zero padding, >in order to ensure that the