[PATCH v6 1/5] docs/migration: add qatzip compression feature

2024-07-15 Thread Yichen Wang
From: Yuan Liu add Intel QATzip compression method introduction Signed-off-by: Yuan Liu Reviewed-by: Nanhai Zou Reviewed-by: Peter Xu Reviewed-by: Yichen Wang --- docs/devel/migration/features.rst | 1 + docs/devel/migration/qatzip-compression.rst | 251 2

[PATCH v6 4/5] migration: Introduce 'qatzip' compression method

2024-07-15 Thread Yichen Wang
From: Bryan Zhang Adds support for 'qatzip' as an option for the multifd compression method parameter, and implements using QAT for 'qatzip' compression and decompression. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- hw/core/qdev-properties-system.c

[PATCH v6 5/5] tests/migration: Add integration test for 'qatzip' compression method

2024-07-15 Thread Yichen Wang
From: Bryan Zhang Adds an integration test for 'qatzip'. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang Reviewed-by: Fabiano Rosas --- tests/qtest/migration-test.c | 31 +++ 1 file changed, 31 insertions(+) diff --git a/tests

[PATCH v6 3/5] migration: Add migration parameters for QATzip

2024-07-15 Thread Yichen Wang
Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- migration/migration-hmp-cmds.c | 4 migration/options.c| 34 ++ migration/options.h| 1 + qapi/migration.json| 18 ++ 4 files changed, 57 insertions

[PATCH v6 0/5] Implement QATzip compression method

2024-07-15 Thread Yichen Wang
deletions(-) create mode 100644 docs/devel/migration/qatzip-compression.rst create mode 100644 migration/multifd-qatzip.c -- Yichen Wang

[PATCH v6 2/5] meson: Introduce 'qatzip' feature to the build system

2024-07-15 Thread Yichen Wang
From: Bryan Zhang Add a 'qatzip' feature, which is automatically disabled, and which depends on the QATzip library if enabled. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- meson.build | 10 ++ meson_options.txt | 2

Re: [External] Re: [PATCH v5 4/5] migration: Introduce 'qatzip' compression method

2024-07-13 Thread Yichen Wang
On Fri, Jul 12, 2024 at 7:17 AM Fabiano Rosas wrote: > > Yichen Wang writes: > > > From: Bryan Zhang > > > > Adds support for 'qatzip' as an option for the multifd compression > > method parameter, and implements using QAT for 'qatzip' compression and >

[PATCH v5 13/13] migration/multifd: Add integration tests for multifd with Intel DSA offloading.

2024-07-11 Thread Yichen Wang
From: Hao Xiang * Add test case to start and complete multifd live migration with DSA offloading enabled. * Add test case to start and cancel multifd live migration with DSA offloading enabled. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- tests/qtest

[PATCH v5 12/13] util/dsa: Add unit test coverage for Intel DSA task submission and completion.

2024-07-11 Thread Yichen Wang
From: Hao Xiang * Test DSA start and stop path. * Test DSA configure and cleanup path. * Test DSA task submission and completion path. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- tests/unit/meson.build | 6 + tests/unit/test-dsa.c | 503

[PATCH v5 11/13] migration/multifd: Add migration option set packet size.

2024-07-11 Thread Yichen Wang
Signed-off-by: Yichen Wang --- migration/migration-hmp-cmds.c | 7 +++ migration/multifd-zlib.c | 6 -- migration/multifd-zstd.c | 6 -- migration/multifd.c| 6 -- migration/multifd.h| 3 --- migration/options.c| 38

[PATCH v5 10/13] migration/multifd: Enable DSA offloading in multifd sender path.

2024-07-11 Thread Yichen Wang
accelerator. The sender thread submits a batch of pages to DSA hardware and waits for the DSA completion thread to signal for work completion. Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- include/qemu/dsa.h| 4 +- migration/migration.c | 2 +- migration/multifd

Re: [PATCH v5 08/13] migration/multifd: Add new migration option for multifd DSA offloading.

2024-07-11 Thread Yichen Wang
On Thu, Jul 11, 2024 at 2:53 PM Yichen Wang wrote: > diff --git a/migration/options.c b/migration/options.c > index 645f55003d..f839493016 100644 > --- a/migration/options.c > +++ b/migration/options.c > @@ -29,6 +29,7 @@ > #include "ram.h" > #include "

[PATCH v5 06/13] util/dsa: Implement zero page checking in DSA task.

2024-07-11 Thread Yichen Wang
task must contain at least two individual tasks. There are times we need to submit a single task and hence a single DSA task submission is also required. Signed-off-by: Hao Xiang Signed-off-by: Bryan Zhang Signed-off-by: Yichen Wang --- include/qemu/dsa.h | 18 util/dsa.c | 247

[PATCH v5 04/13] util/dsa: Implement DSA task enqueue and dequeue.

2024-07-11 Thread Yichen Wang
From: Hao Xiang * Use a safe thread queue for DSA task enqueue/dequeue. * Implement DSA task submission. * Implement DSA batch task submission. Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- include/qemu/dsa.h | 46 ++ util/dsa.c | 222

[PATCH v5 03/13] util/dsa: Implement DSA device start and stop logic.

2024-07-11 Thread Yichen Wang
From: Hao Xiang * DSA device open and close. * DSA group contains multiple DSA devices. * DSA group configure/start/stop/clean. Signed-off-by: Hao Xiang Signed-off-by: Bryan Zhang Signed-off-by: Yichen Wang --- include/qemu/dsa.h | 86 + util/dsa.c | 303

[PATCH v5 05/13] util/dsa: Implement DSA task asynchronous completion thread model.

2024-07-11 Thread Yichen Wang
completion checking. Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- include/qemu/dsa.h | 1 + util/dsa.c | 274 - 2 files changed, 274 insertions(+), 1 deletion(-) diff --git a/include/qemu/dsa.h b/include/qemu/dsa.h index 9cc836b64c

[PATCH v5 00/13] WIP: Use Intel DSA accelerator to offload zero page checking in multifd live migration.

2024-07-11 Thread Yichen Wang
with Intel DSA offloading. Yichen Wang (1): util/dsa: Add idxd into linux header copy list. include/qemu/dsa.h | 176 + meson.build | 14 + meson_options.txt |2 + migration/migration-hmp-cmds.c | 22 +- migration/migration.c

[PATCH v5 07/13] util/dsa: Implement DSA task asynchronous submission and wait for completion.

2024-07-11 Thread Yichen Wang
Signed-off-by: Bryan Zhang Signed-off-by: Yichen Wang --- include/qemu/dsa.h | 14 + util/dsa.c | 125 +++-- 2 files changed, 135 insertions(+), 4 deletions(-) diff --git a/include/qemu/dsa.h b/include/qemu/dsa.h index 1b4baf1c80..20bb88d48c

[PATCH v5 02/13] util/dsa: Add idxd into linux header copy list.

2024-07-11 Thread Yichen Wang
Signed-off-by: Yichen Wang --- scripts/update-linux-headers.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/update-linux-headers.sh b/scripts/update-linux-headers.sh index c34ac6454e..5aba95d9cb 100755 --- a/scripts/update-linux-headers.sh +++ b/scripts/update

[PATCH v5 08/13] migration/multifd: Add new migration option for multifd DSA offloading.

2024-07-11 Thread Yichen Wang
t. Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- migration/migration-hmp-cmds.c | 15 ++- migration/options.c| 47 ++ migration/options.h| 1 + qapi/migration.json| 32 --- 4 files c

[PATCH v5 01/13] meson: Introduce new instruction set enqcmd to the build system.

2024-07-11 Thread Yichen Wang
From: Hao Xiang Enable instruction set enqcmd in build. Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- meson.build | 14 ++ meson_options.txt | 2 ++ scripts/meson-buildoptions.sh | 3 +++ 3 files changed, 19 insertions(+) diff --git

[PATCH v5 09/13] migration/multifd: Prepare to introduce DSA acceleration on the multifd path.

2024-07-11 Thread Yichen Wang
From: Hao Xiang 1. Refactor multifd_send_thread function. 2. Introduce the batch task structure in MultiFDSendParams. Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- include/qemu/dsa.h | 41 --- migration/multifd.c | 4 + migration/multifd.h | 3 + util/dsa.c | 270

Re: [External] Re: [PATCH v5 0/5] Implement QATzip compression method

2024-07-11 Thread Yichen Wang
On Thu, Jul 11, 2024 at 8:45 AM Peter Xu wrote: > > On Wed, Jul 10, 2024 at 07:52:24PM -0700, Yichen Wang wrote: > > v5: > > - Rebase changes on top of 59084feb256c617063e0dbe7e64821ae8852d7cf > > - Add documentations about migration with qatzip accerlation > > - Rem

Re: [External] Re: [PATCH v5 5/5] tests/migration: Add integration test for 'qatzip' compression method

2024-07-11 Thread Yichen Wang
On Thu, Jul 11, 2024 at 7:23 AM Peter Xu wrote: > > On Wed, Jul 10, 2024 at 07:52:29PM -0700, Yichen Wang wrote: > > From: Bryan Zhang > > > > Adds an integration test for 'qatzip'. > > > > Signed-off-by: Bryan Zhang > > Signed-off-by: Hao Xiang &g

[PATCH v5 5/5] tests/migration: Add integration test for 'qatzip' compression method

2024-07-10 Thread Yichen Wang
From: Bryan Zhang Adds an integration test for 'qatzip'. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang Reviewed-by: Fabiano Rosas Signed-off-by: Yichen Wang --- tests/qtest/migration-test.c | 35 +++ 1 file changed, 35

[PATCH v5 2/5] meson: Introduce 'qatzip' feature to the build system

2024-07-10 Thread Yichen Wang
From: Bryan Zhang Add a 'qatzip' feature, which is automatically disabled, and which depends on the QATzip library if enabled. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- meson.build | 10 ++ meson_options.txt | 2

[PATCH v5 3/5] migration: Add migration parameters for QATzip

2024-07-10 Thread Yichen Wang
Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- migration/migration-hmp-cmds.c | 4 migration/options.c| 34 ++ migration/options.h| 1 + qapi/migration.json| 18 ++ 4 files changed, 57 insertions

[PATCH v5 4/5] migration: Introduce 'qatzip' compression method

2024-07-10 Thread Yichen Wang
From: Bryan Zhang Adds support for 'qatzip' as an option for the multifd compression method parameter, and implements using QAT for 'qatzip' compression and decompression. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- hw/core/qdev-properties-system.c

[PATCH v5 0/5] Implement QATzip compression method

2024-07-10 Thread Yichen Wang
/qtest/meson.build | 4 + tests/qtest/migration-test.c| 35 ++ 15 files changed, 778 insertions(+), 3 deletions(-) create mode 100644 docs/devel/migration/qatzip-compression.rst create mode 100644 migration/multifd-qatzip.c -- Yichen Wang

[PATCH v5 1/5] docs/migration: add qatzip compression feature

2024-07-10 Thread Yichen Wang
From: Yuan Liu add Intel QATzip compression method introduction Signed-off-by: Yuan Liu Reviewed-by: Nanhai Zou Reviewed-by: Peter Xu Reviewed-by: Yichen Wang --- docs/devel/migration/features.rst | 1 + docs/devel/migration/qatzip-compression.rst | 251 2

[PATCH v4 2/4] migration: Add migration parameters for QATzip

2024-07-05 Thread Yichen Wang
Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- migration/migration-hmp-cmds.c | 8 + migration/options.c| 57 ++ migration/options.h| 2 ++ qapi/migration.json| 35 + 4 files changed, 102 insertions

[PATCH v4 4/4] tests/migration: Add integration test for 'qatzip' compression method

2024-07-05 Thread Yichen Wang
From: Bryan Zhang Adds an integration test for 'qatzip'. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang Reviewed-by: Fabiano Rosas --- tests/qtest/migration-test.c | 35 +++ 1 file changed, 35 insertions(+) diff --git a/tests

[PATCH v4 1/4] meson: Introduce 'qatzip' feature to the build system

2024-07-05 Thread Yichen Wang
From: Bryan Zhang Add a 'qatzip' feature, which is automatically disabled, and which depends on the QATzip library if enabled. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- meson.build | 10 ++ meson_options.txt | 2

[PATCH v4 3/4] migration: Introduce 'qatzip' compression method

2024-07-05 Thread Yichen Wang
From: Bryan Zhang Adds support for 'qatzip' as an option for the multifd compression method parameter, and implements using QAT for 'qatzip' compression and decompression. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- hw/core/qdev-properties-system.c

[PATCH v4 0/4] Implement using Intel QAT to offload ZLIB

2024-07-05 Thread Yichen Wang
/multifd-qatzip.c -- Yichen Wang

Re: [External] [PATCH v3 0/4] Implement using Intel QAT to offload ZLIB

2024-07-05 Thread Yichen Wang
;>>> To: Wang, Yichen >>>> Cc: Paolo Bonzini ; Daniel P. Berrangé >>>> ; Eduardo Habkost ; Marc- >> André >>>> Lureau ; Thomas Huth ; >>>> Philippe Mathieu-Daudé ; Fabiano Rosas >>>> ; Eric Blake ; Markus Armbruster >>&

[PATCH v3 4/4] tests/migration: Add integration test for 'qatzip' compression method

2024-06-27 Thread Yichen Wang
From: Bryan Zhang Adds an integration test for 'qatzip'. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang Reviewed-by: Fabiano Rosas --- tests/qtest/migration-test.c | 35 +++ 1 file changed, 35 insertions(+) diff --git a/tests

[PATCH v3 0/4] Implement using Intel QAT to offload ZLIB

2024-06-27 Thread Yichen Wang
+ tests/qtest/migration-test.c | 35 +++ 13 files changed, 551 insertions(+), 1 deletion(-) create mode 100644 migration/multifd-qatzip.c -- Yichen Wang

[PATCH v3 3/4] migration: Introduce 'qatzip' compression method

2024-06-27 Thread Yichen Wang
From: Bryan Zhang Adds support for 'qatzip' as an option for the multifd compression method parameter, and implements using QAT for 'qatzip' compression and decompression. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- hw/core/qdev-properties-system.c

[PATCH v3 1/4] meson: Introduce 'qatzip' feature to the build system

2024-06-27 Thread Yichen Wang
From: Bryan Zhang Add a 'qatzip' feature, which is automatically disabled, and which depends on the QATzip library if enabled. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- meson.build | 10 ++ meson_options.txt | 2

[PATCH v3 2/4] migration: Add migration parameters for QATzip

2024-06-27 Thread Yichen Wang
Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- migration/migration-hmp-cmds.c | 8 + migration/options.c| 57 ++ migration/options.h| 2 ++ qapi/migration.json| 35 + 4 files changed, 102 insertions

Re: [PATCH v2 2/5] migration: Add migration parameters for QATzip

2024-06-26 Thread Yichen Wang
> On Mar 28, 2024, at 12:23 AM, Liu, Yuan1 wrote: > >> -Original Message- >> From: Bryan Zhang >> Sent: Wednesday, March 27, 2024 6:42 AM >> To: qemu-devel@nongnu.org >> Cc: pet...@redhat.com; faro...@suse.de; Liu, Yuan1 ; >> berra...@redhat.com; Zou, Nanhai ; >> hao.xi...@linux.dev;