The page_size member has been removed from the MultiFDSendParams
and MultiFDRecvParams. The function multifd_ram_page_size is used to
provide the page size in the multifd compressor.
Signed-off-by: Yuan Liu
---
migration/multifd-qpl.c | 10 +-
1 file changed, 5 insertions(+), 5
add Intel QATzip compression method introduction
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
docs/devel/migration/features.rst | 1 +
docs/devel/migration/qatzip-compression.rst | 112
2 files changed, 113 insertions(+)
create mode 100644 docs/devel
fallback, always wait for work queues to become available
total time: 18381 ms
downtime: 25 ms
throughput: 13698.65 mbps
pages-per-second: 859607
If both the hardware and software paths fail, the uncompressed page is
sent directly.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
add Intel Query Processing Library (QPL) compression method
introduction
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
Reviewed-by: Fabiano Rosas
Acked-by: Peter Xu
---
docs/devel/migration/features.rst| 1 +
docs/devel/migration/qpl-compression.rst | 260
: Yuan Liu
Reviewed-by: Nanhai Zou
Reviewed-by: Peter Xu
Reviewed-by: Fabiano Rosas
---
tests/qtest/migration-test.c | 24
1 file changed, 24 insertions(+)
diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index b7e3406471..ef0c3f5e28 100644
--- a
In-Memory Analytics Accelerator(IAA) hardware for compression
and decompression acceleration.
For more live migration with IAA, please refer to the document
docs/devel/migration/qpl-compression.rst
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
Reviewed-by: Fabiano Rosas
---
meson.build
during initialization, a software job is allocated to each channel
for software path fallabck when the IAA hardware is unavailable or
the hardware job submission fails. If the IAA hardware is available,
multiple hardware jobs are allocated for batch processing.
Signed-off-by: Yuan Liu
Reviewed
Different compression methods may require different numbers of IOVs.
Based on streaming compression of zlib and zstd, all pages will be
compressed to a data block, so two IOVs are needed for packet header
and compressed data block.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
Reviewed-by
compression and decompression CPU overhead.
How to enable qpl compression during migration:
migrate_set_parameter multifd-compression qpl
There is no qpl compression level parameter added since it only supports
level one, users do not need to specify the qpl compression level.
Signed-off-by: Yuan Liu
QPL compression and decompression will use IAA hardware first.
If IAA hardware is not available, it will automatically fall
back to QPL software path, if the software job also fails,
the uncompressed page is sent directly.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/multifd
Different compression methods may require different numbers of IOVs.
Based on streaming compression of zlib and zstd, all pages will be
compressed to a data block, so two IOVs are needed for packet header
and compressed data block.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
Reviewed-by
: Yuan Liu
Reviewed-by: Nanhai Zou
Reviewed-by: Peter Xu
---
tests/qtest/migration-test.c | 24
1 file changed, 24 insertions(+)
diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index b7e3406471..ef0c3f5e28 100644
--- a/tests/qtest/migration-test.c
during initialization, a software job is allocated to each channel
for software path fallabck when the IAA hardware is unavailable or
the hardware job submission fails. If the IAA hardware is available,
multiple hardware jobs are allocated for batch processing.
Signed-off-by: Yuan Liu
Reviewed
add Intel Query Processing Library (QPL) compression method
introduction
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
docs/devel/migration/features.rst| 1 +
docs/devel/migration/qpl-compression.rst | 262 +++
2 files changed, 263 insertions(+)
create mode
In-Memory Analytics Accelerator(IAA) hardware for compression
and decompression acceleration.
For more live migration with IAA, please refer to the document
docs/devel/migration/qpl-compression.rst
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
meson.build | 8
compression and decompression CPU overhead.
How to enable qpl compression during migration:
migrate_set_parameter multifd-compression qpl
There is no qpl compression level parameter added since it only supports
level one, users do not need to specify the qpl compression level.
Signed-off-by: Yuan Liu
add Intel Query Processing Library (QPL) compression method
introduction
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
docs/devel/migration/features.rst| 1 +
docs/devel/migration/qpl-compression.rst | 262 +++
2 files changed, 263 insertions(+)
create mode
not
available, the qpl compression will fallback to the software path.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/multifd-qpl.c | 272 +++-
1 file changed, 271 insertions(+), 1 deletion(-)
diff --git a/migration/multifd-qpl.c b/migration
: Yuan Liu
Reviewed-by: Nanhai Zou
---
tests/qtest/migration-test.c | 24
1 file changed, 24 insertions(+)
diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index 5d6d8cd634..0f75ed7c49 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest
: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/multifd-qpl.c | 284 +++-
1 file changed, 280 insertions(+), 4 deletions(-)
diff --git a/migration/multifd-qpl.c b/migration/multifd-qpl.c
index 89fa51091a..9a1fddbdd0 100644
--- a/migration/multifd-qpl.c
+++ b
.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
hw/core/qdev-properties-system.c | 2 +-
migration/meson.build| 1 +
migration/multifd-qpl.c | 20
migration/multifd.h | 1 +
qapi/migration.json | 7 ++-
5 files changed
In-Memory Analytics Accelerator(IAA) hardware for compression
and decompression acceleration.
For more live migration with IAA, please refer to the document
docs/devel/migration/qpl-compression.rst
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
meson.build | 8
Different compression methods may require different numbers of IOVs.
Based on streaming compression of zlib and zstd, all pages will be
compressed to a data block, so two IOVs are needed for packet header
and compressed data block.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration
unlikely that the zero page will be migrated multiple times.
Signed-off-by: Yuan Liu
---
migration/multifd-zero-page.c | 4 +++-
migration/multifd-zlib.c | 1 +
migration/multifd-zstd.c | 1 +
migration/multifd.c | 1 +
migration/ram.c | 4
migration/ram.h
tps://lore.kernel.org/all/CAAYibXib+TWnJpV22E=adncdBmwXJRqgRjJXK7X71J=bdfa...@mail.gmail.com/
2.
https://lore.kernel.org/all/ph7pr11mb594123f7eefebfce219af100a3...@ph7pr11mb5941.namprd11.prod.outlook.com/
Yuan Liu (1):
migration/multifd: solve zero page causing multiple page faults
migratio
each qpl job is used to (de)compress a normal page and it can
be processed independently by the IAA hardware. All qpl jobs
are submitted to the hardware at once, and wait for all jobs
completion.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/multifd-qpl.c | 229
algorithm
and use Intel In-Memory Analytics Accelerator(IAA) hardware for
compression and decompression acceleration.
Please refer to the following for more information about QPL
https://intel.github.io/qpl/documentation/introduction_docs/introduction.html
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai
Different compression methods may require different numbers of IOVs.
Based on streaming compression of zlib and zstd, all pages will be
compressed to a data block, so two IOVs are needed for packet header
and compressed data block.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration
method for migration.
How to enable qpl compression during migration:
migrate_set_parameter multifd-compression qpl
The qpl only supports one compression level, there is no qpl
compression level parameter added, users do not need to specify
the qpl compression level.
Signed-off-by: Yuan Liu
compression initialization will fail.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/multifd-qpl.c | 243 +++-
1 file changed, 242 insertions(+), 1 deletion(-)
diff --git a/migration/multifd-qpl.c b/migration/multifd-qpl.c
index 056a68a060
add qpl to compression method test for multifd migration
the migration with qpl compression needs to access IAA hardware
resource, please run "check-qtest" with sudo or root permission,
otherwise migration test will fail
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
t
add Intel Query Processing Library (QPL) compression method
introduction
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
docs/devel/migration/features.rst| 1 +
docs/devel/migration/qpl-compression.rst | 231 +++
2 files changed, 232 insertions(+)
create mode
each qpl job is used to (de)compress a normal page and it can
be processed independently by the IAA hardware. All qpl jobs
are submitted to the hardware at once, and wait for all jobs
completion.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/multifd-qpl.c | 219
compression initialization will fail.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/multifd-qpl.c | 128 ++--
1 file changed, 122 insertions(+), 6 deletions(-)
diff --git a/migration/multifd-qpl.c b/migration/multifd-qpl.c
index 6b94e732ac
this data
block. For no compression, each IOV is used to send a page, so the
number of IOVs required is the same as the number of pages.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/multifd-zlib.c | 18 +-
migration/multifd-zstd.c | 18 +-
migration
add qpl to compression method test for multifd migration
the migration with qpl compression needs to access IAA hardware
resource, please run "check-qtest" with sudo or root permission,
otherwise migration test will fail
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
t
method for migration.
How to enable qpl compression during migration:
migrate_set_parameter multifd-compression qpl
The qpl only supports one compression level, there is no qpl
compression level parameter added, users do not need to specify
the qpl compression level.
Signed-off-by: Yuan Liu
algorithm
and use Intel In-Memory Analytics Accelerator(IAA) hardware for
compression and decompression acceleration.
Please refer to the following for more information about QPL
https://intel.github.io/qpl/documentation/introduction_docs/introduction.html
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai
add zlib and zstd compression levels in multifd parameter
testing and application and add compression level tests
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
Reported-by: Xiaohui Li
---
migration/options.c | 12
tests/qtest/migration-test.c | 16
2
add QPL compression method introduction
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
docs/devel/migration/features.rst| 1 +
docs/devel/migration/qpl-compression.rst | 231 +++
2 files changed, 232 insertions(+)
create mode 100644 docs/devel/migration/qpl
when starting multifd live migration, if the compression method is
enabled, compression method can be accelerated using accelerators.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
Reviewed-by: Fabiano Rosas
---
migration/multifd.c | 40 ++--
migration
acceleration function can be disabled.
Similarly, users can explicitly specify a specific accelerator name, such
as multifd-compression-accel=qpl.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
hw/core/qdev-properties-system.c| 11 ++
include/hw/qdev-properties-system.h | 4
ction in multifd.c file
Yuan Liu (4):
migration: Introduce multifd-compression-accel parameter
multifd: Implement multifd compression accelerator
configure: add qpl option
multifd: Introduce QPL compression accelerator
hw/core/qdev-properties-system.c| 11 +
include/hw/qdev-prope
during the live migration.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
meson.build | 18 ++
meson_options.txt | 2 ++
scripts/meson-buildoptions.sh | 3 +++
3 files changed, 23 insertions(+)
diff --git a/meson.build b/meson.build
index
Zlib compression algorithm. QPL can automatically choose
software or hardware acceleration based on the platform.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/meson.build | 1 +
migration/multifd-qpl.c | 323
2 files changed, 324
rator in the multifd
compression accelerator
- fixed the issue that QPL was compiled into the migration
module by default
Yuan Liu (4):
migration: Introduce multifd-compression-accel parameter
multifd: Implement multifd compression accelerator
configure: add qpl option
multifd: Intr
acceleration function can be disabled.
Similarly, users can explicitly specify a specific accelerator name, such
as multifd-compression-accel=qpl.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
hw/core/qdev-properties-system.c| 11 +++
include/hw/qdev-properties-system.h | 4
Zlib compression algorithm. QPL can automatically choose
software or hardware acceleration based on the platform.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/meson.build | 1 +
migration/multifd-qpl.c | 326
2 files changed, 327
during the live migration.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
meson.build | 7 +++
meson_options.txt | 2 ++
scripts/meson-buildoptions.sh | 3 +++
3 files changed, 12 insertions(+)
diff --git a/meson.build b/meson.build
index 259dc5f308
when starting multifd live migration, if the compression method is
enabled, compression method can be accelerated using accelerators.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/multifd.c | 38 --
migration/multifd.h | 8
2 files
Implement the functions of IAA for data compression and decompression.
The implementation uses non-blocking job submission and polling to check
the job completion status to reduce IAA's overhead in the live migration
process.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migratio
, resulting in performance gains.
The implementation of the IAA (de)compression code is based on Intel Query
Processing Library (QPL), an open-source software project designed for
IAA high-level software programming.
Best regards,
Yuan Liu
Yuan Liu (5):
configure: add qpl meson option
qapi
data (de)compression
using IAA during the live migration process.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
meson.build | 9 -
meson_options.txt | 2 ++
scripts/meson-buildoptions.sh | 3 +++
3 files changed, 13 insertions(+), 1 deletion(-)
diff
This patch defines the structure for IAA jobs related to data
compression and decompression, as well as the initialization and
deinitialization processes for IAA.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/iaa-ram-compress.c | 152
compress on. If the migration compression
capability is enabled and the IAA compression parameter is set, IAA will
be used instead of CPU for data (de)compression.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/migration-hmp-cmds.c | 8
migration/options.c| 20
Refactor legacy RAM compression functions to support both IAA
compression and CPU compression.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/migration.c| 6 +--
migration/ram-compress.c | 81
migration/ram-compress.h | 10
56 matches
Mail list logo
