Based-on: <20200213132030.57757-1-quint...@redhat.com> [v6] - rebase to latest pull request - move functions that read parameters to the parameters patechs (dave) - require zstd on travis/docker/... (please review) - rename multifd_method to multifd_compression - improve documentation
Please review [v5] - rebased on top of latest pull request - check zlib/zstd return codes in loops (denis suggestions) - check that we do the right thing in error cases (I corrupted the stream, make buffers too small, etc by hand) [v4] - create new parameters: multifd-zlib-level & multifd-zstd-level - use proper "conditionals" for qapi (thanks markus) - more than half of the patches moved to the migration PULL request that this series are based on - method type has moved to one int from a set of flags - fixed all reviews comments Please review. [v3] - rebased on top of upstream + previous multifd cancel series - split multifd code into its own file (multifd.[ch]) - split zstd/zlib compression methods (multifd-zstd/zlib.c) - use qemu module feauture to avoid ifdefs (my understanding is that zlib needs to be present, but we setup zstd only if it is not there or is disabled) - multifd-method: none|zlib|zstd As far as I can see, there is no easy way to convince qapi that zstd option could/couldn't be there depending on compliation flags. I ended just checking in migrate_parameters_check() if it is enabled and giving an error message otherwise. Questions: - I am "reusing" the compress-level parameter for both zstd and zlib, but it poses a problem: * zlib values: 1-9 (default: 6?) * zstd values: 1-19 (default: 3) So, what should I do: * create multifd-zstd-level and multifd-zlib-level (easier) * reuse compress-level, and change its maximum values depending on multifd-method * any other good option? Please, review. [v2] - rebase on top of previous arguments posted to the list - introduces zlib compression - introduces zstd compression Please help if you know anything about zstd/zlib compression. This puts compression on top of multifd. Advantages about current compression: - We copy all pages in a single packet and then compress the whole thing. - We reuse the compression stream for all the packets sent through the same channel. - We can select nocomp/zlib/zstd levels of compression. Please, review. Juan Quintela (8): multifd: Add multifd-compression parameter migration: Add support for modules multifd: Make no compression operations into its own structure multifd: Add multifd-zlib-level parameter multifd: Add zlib compression multifd support configure: Enable test and libs for zstd multifd: Add multifd-zstd-level parameter multifd: Add zstd compression multifd support .gitlab-ci.yml | 1 + .travis.yml | 1 + configure | 30 ++ hw/core/qdev-properties.c | 13 + include/hw/qdev-properties.h | 4 + include/qemu/module.h | 2 + migration/Makefile.objs | 2 + migration/migration.c | 70 ++++ migration/migration.h | 3 + migration/multifd-zlib.c | 325 +++++++++++++++++ migration/multifd-zstd.c | 339 ++++++++++++++++++ migration/multifd.c | 191 +++++++++- migration/multifd.h | 31 ++ migration/ram.c | 2 +- monitor/hmp-cmds.c | 21 ++ qapi/migration.json | 80 ++++- tests/docker/dockerfiles/centos7.docker | 3 +- .../dockerfiles/fedora-i386-cross.docker | 3 +- tests/docker/dockerfiles/fedora.docker | 3 +- tests/docker/dockerfiles/ubuntu.docker | 1 + tests/docker/dockerfiles/ubuntu1804.docker | 1 + tests/qtest/migration-test.c | 30 +- tests/vm/fedora | 5 +- tests/vm/freebsd | 3 + tests/vm/netbsd | 3 + tests/vm/openbsd | 3 + vl.c | 1 + 27 files changed, 1151 insertions(+), 20 deletions(-) create mode 100644 migration/multifd-zlib.c create mode 100644 migration/multifd-zstd.c -- 2.24.1