Re: [PATCH v6 09/10] migration/yank: Keep track of registered yank instances

2023-09-25 Thread Lukas Straub
On Mon, 25 Sep 2023 09:20:58 -0300 Fabiano Rosas wrote: > CC: Daniel for the QIOChannel discussion > > Lukas Straub writes: > > On Thu, 14 Sep 2023 10:57:47 -0400 > > Peter Xu wrote: > > > >> On Thu, Sep 14, 2023 at 10:23:38AM -0300, Fabiano Ro

Re: [PATCH v6 09/10] migration/yank: Keep track of registered yank instances

2023-09-25 Thread Lukas Straub
On Thu, 14 Sep 2023 10:57:47 -0400 Peter Xu wrote: > On Thu, Sep 14, 2023 at 10:23:38AM -0300, Fabiano Rosas wrote: > > Peter Xu writes: > > > > > On Wed, Sep 13, 2023 at 06:53:20PM -0300, Fabiano Rosas wrote: > > >> Peter Xu writes: > > >> > > >> > On Mon, Sep 11, 2023 at 02:13:19PM

Re: Tips for local testing guestfwd

2023-06-26 Thread Lukas Straub
CC'ing SLIRP and net maintainer. On Sun, 25 Jun 2023 22:58:36 -0700 Felix Wu wrote: > Hi all, > > TL,DR: I am working on QEMU ipv6 guestfwd feature and finished coding, and > would like to learn the best practice to test it. > Context: in slirp side this task is tracking by [1]. > Currently, I

[PATCH 6/8] colo: Reject colo with postcopy capability enabled

2023-06-22 Thread Lukas Straub
Signed-off-by: Lukas Straub --- migration/migration.c | 5 +++-- migration/options.c | 9 + 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index a954ff4f7d..9860f960f2 100644 --- a/migration/migration.c +++ b/migration

[PATCH 5/8] colo: Don't send ENABLE_COLO command

2023-06-22 Thread Lukas Straub
We should only migrate to qemu with newer version, and there it's not needed anymore. Signed-off-by: Lukas Straub --- migration/migration.c | 5 - migration/savevm.c | 6 -- migration/savevm.h | 1 - migration/trace-events | 1 - 4 files changed, 13 deletions(-) diff --git

[PATCH 8/8] ram: Remove useless colo special-casing

2023-06-22 Thread Lukas Straub
This is not needed, as colo primary side everything is more or less a new migration for every checkpoint. Also, we only enter colo mode after the precopy migration is finished so this if is always taken. Still add an assert just in case. Signed-off-by: Lukas Straub --- migration/ram.c | 12

[PATCH 3/8] colo: Replace migration_incoming_colo_enabled() with migrate_colo()

2023-06-22 Thread Lukas Straub
Signed-off-by: Lukas Straub --- include/migration/colo.h | 1 - migration/colo.c | 2 +- migration/migration.c| 7 +-- migration/ram.c | 2 +- 4 files changed, 3 insertions(+), 9 deletions(-) diff --git a/include/migration/colo.h b/include/migration/colo.h index

[PATCH 7/8] colo: Reject colo with block migration capability enabled

2023-06-22 Thread Lukas Straub
Signed-off-by: Lukas Straub --- migration/migration.c | 4 migration/options.c | 5 + 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 9860f960f2..270130579f 100644 --- a/migration/migration.c +++ b/migration

[PATCH 4/8] colo: Remove ENABLE_COLO loadvm command functions

2023-06-22 Thread Lukas Straub
No need for it anymore now that x-colo capability is required on incoming side. Still accept the command as noop, for compatibility with older qemu. Signed-off-by: Lukas Straub --- include/migration/colo.h | 2 -- migration/migration.c| 26 -- migration/savevm.c

[PATCH 2/8] colo: Setup ram cache in normal migration path

2023-06-22 Thread Lukas Straub
Now that x-colo capability needs to be always enabled on the incoming side we can use that to initialize the ram cache in the normal migration path. Signed-off-by: Lukas Straub --- migration/migration.c | 16 migration/savevm.c| 10 +- 2 files changed, 13 insertions

[PATCH 0/8] colo migration cleanups

2023-06-22 Thread Lukas Straub
Hello Everyone, Here are some cleanups around the colo migration code. Notably, we only support the same qemu version on both sides for now. Though in this patchset I still try to be support migration to a newer qemu version. Lukas Straub (8): colo: Only support the same qemu version on source

[PATCH 1/8] colo: Only support the same qemu version on source and destination

2023-06-22 Thread Lukas Straub
Signed-off-by: Lukas Straub --- docs/COLO-FT.txt | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/COLO-FT.txt b/docs/COLO-FT.txt index 2e760a4aee..8e64480dbd 100644 --- a/docs/COLO-FT.txt +++ b/docs/COLO-FT.txt @@ -148,6 +148,8 @@ in test procedure. Note: Here we are running both

Re: [PATCH 18/18] migration: Remove not needed file parameter for save_zero_page*

2023-06-22 Thread Lukas Straub
On Tue, 13 Jun 2023 16:57:57 +0200 Juan Quintela wrote: > Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub > --- > migration/ram.c | 13 ++--- > 1 file changed, 6 insertions(+), 7 deletions(-) > > diff --git a/migration/ram.c b/migration/ram.c > index 0

Re: [PATCH 17/18] migration: Remove not needed block parameter for save_zero_page*

2023-06-22 Thread Lukas Straub
On Tue, 13 Jun 2023 16:57:56 +0200 Juan Quintela wrote: > Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub > --- > migration/ram.c | 18 -- > 1 file changed, 8 insertions(+), 10 deletions(-) > > diff --git a/migration/ram.c b/migration/ram.c

Re: [PATCH 16/18] migration: control_save_page() can take block through pss

2023-06-22 Thread Lukas Straub
On Tue, 13 Jun 2023 16:57:55 +0200 Juan Quintela wrote: > Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub > --- > migration/ram.c | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/migration/ram.c b/migration/ram.c > index e3ff98a6

Re: [PATCH 15/18] migration: save_compress_page() can take block through pss

2023-06-22 Thread Lukas Straub
On Tue, 13 Jun 2023 16:57:54 +0200 Juan Quintela wrote: > Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub > --- > migration/ram.c | 9 + > 1 file changed, 5 insertions(+), 4 deletions(-) > > diff --git a/migration/ram.c b/migration/ram.c > index 1bd586c2

Re: [PATCH 14/18] migration: Use "i" as an for index in ram-compress.c

2023-06-22 Thread Lukas Straub
On Tue, 13 Jun 2023 16:57:53 +0200 Juan Quintela wrote: > It is used everywhere else in C. Once there, make sure that we don't > use the index outside of the for declaring the variable there. > > Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub > --- > migration/r

Re: [PATCH 13/18] migration: Simplify decompress_data_with_multi_threads()

2023-06-22 Thread Lukas Straub
On Tue, 13 Jun 2023 16:57:52 +0200 Juan Quintela wrote: > Doing a break to do another break is just confused. Just call return > when we know we want to return. > > Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub > --- > migration/ram-compress.c | 8 ++-- &g

Re: [PATCH 12/18] migration: Create compression_update_rates()

2023-06-22 Thread Lukas Straub
On Tue, 13 Jun 2023 16:57:51 +0200 Juan Quintela wrote: > So we can move more compression_counters stuff to ram-compress.c. > Create compression_counters struct to add the stuff that was on > MigrationState. > > Signed-off-by: Juan Quintela With the comment below fixed: Rev

Re: [PATCH 10/18] migration: Simplify compress_page_with_multithread()

2023-06-22 Thread Lukas Straub
On Tue, 13 Jun 2023 16:57:49 +0200 Juan Quintela wrote: > Move the goto to a while true. > > Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub > --- > migration/ram-compress.c | 48 > 1 file changed, 24 insertions(+), 24 dele

Re: [PATCH 11/18] migration: Move busy++ to migrate_with_multithread

2023-06-22 Thread Lukas Straub
On Tue, 13 Jun 2023 16:57:50 +0200 Juan Quintela wrote: > And now we can simplify save_compress_page(). > > Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub > --- > migration/ram-compress.c | 1 + > migration/ram.c | 7 +-- > 2 files changed, 2 inser

Re: [PATCH 09/18] migration: Make compress_data_with_multithreads return bool

2023-06-22 Thread Lukas Straub
On Tue, 13 Jun 2023 16:57:48 +0200 Juan Quintela wrote: > Signed-off-by: Juan Quintela Nit: Probably makes sense to rename it to try_compress_page or similar. And probably for zero_page and xbzrle (when factored out in it's own function) too. > --- > migration/ram-compress.h | 4 ++-- >

Re: [PATCH 08/18] migration: Move update_compress_threads_counts() to ram-compress.c

2023-06-22 Thread Lukas Straub
On Tue, 13 Jun 2023 16:57:47 +0200 Juan Quintela wrote: > Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub > --- > migration/ram-compress.h | 1 + > migration/ram-compress.c | 17 + > migration/ram.c | 15 --- > 3 files chan

Re: [PATCH 07/18] migration: Create ram_compressed_pages()

2023-06-22 Thread Lukas Straub
On Tue, 13 Jun 2023 16:57:46 +0200 Juan Quintela wrote: > Signed-off-by: Juan Quintela > > fix ram_compress > > Signed-off-by: Juan Quintela Commit message is off. Other than that: Reviewed-by: Lukas Straub > --- > migration/ram-compress.h | 1 + > migr

Re: [PATCH 06/18] migration: Create populate_compress()

2023-06-22 Thread Lukas Straub
On Tue, 13 Jun 2023 16:57:45 +0200 Juan Quintela wrote: > So we don't have to access compression_counters from outside > ram-compress.c. > > Signed-off-by: Juan Quintela With the comment below fixed: Reviewed-by: Lukas Straub > --- > migration/ram-compress.h |

Re: [PATCH 05/18] migration: Move compression_counters cleanup ram-compress.c

2023-06-22 Thread Lukas Straub
On Tue, 13 Jun 2023 16:57:44 +0200 Juan Quintela wrote: > Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub > --- > migration/migration.c| 4 +--- > migration/ram-compress.c | 5 + > migration/savevm.c | 1 - > 3 files changed, 6 insertions(+), 4 dele

Re: [PATCH 03/18] migration: RDMA is not compatible with anything else

2023-06-22 Thread Lukas Straub
On Tue, 13 Jun 2023 16:57:42 +0200 Juan Quintela wrote: > So give an error instead of just ignoring the other methods. > > Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub > --- > migration/migration.c | 12 > 1 file changed, 12 insertions(+) > >

Re: [PATCH 02/18] migration: Give one error if trying to set COMPRESSION and XBZRLE

2023-06-22 Thread Lukas Straub
On Tue, 13 Jun 2023 16:57:41 +0200 Juan Quintela wrote: > As we have disable to use both together in the previous patch, we can > remove this check. This commit message is off. > > Signed-off-by: Juan Quintela > --- > migration/options.c | 7 +++ > 1 file changed, 7 insertions(+) > >

Re: [PATCH 01/18] migration: Give one error if trying to set MULTIFD and XBZRLE

2023-06-22 Thread Lukas Straub
. I don't use it, but it should be easy to do by just letting xbzlre try to handle the page before multifd. There shouldn't be any incompatibilities other than that, as long as xbzlre sees the pages before multifd and the zero-page case is still handled the old way. Best Regards, Lukas Straub pgpzt4_NdWd_7.pgp Description: OpenPGP digital signature

[PATCH 2/2] savevm: Disallow devices being legacy and iterable at the same time

2023-06-22 Thread Lukas Straub
Signed-off-by: Lukas Straub --- migration/savevm.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index 5986f852b2..8d3cf7d675 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -774,6 +774,8 @@ int register_savevm_live(const char *idstr

[PATCH 1/2] vfio: Don't be a iterable and legacy device at the same time

2023-06-22 Thread Lukas Straub
Legacy savevm devices only implement save_state() and load_state(). Iterable devices shouldn't implement save_state() or else they are handled both as an iterable and legacy device in the savevm code. Signed-off-by: Lukas Straub --- Note: this patch is completely untested. hw/vfio/migration.c

[PATCH 2/2] ide: Explicitly poll for BHs on cancel

2023-06-22 Thread Lukas Straub
rim_bh_cb()), until s->bus->dma->aiocb is NULL. To stop this from being an infinite loop, assert that we made progress with every aio_poll() call (i.e., invoked some BH). Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2029980 Signed-off-by: Hanna Reitz Signed-off-by: Lukas Straub

[PATCH 1/2] ide: Fix a rare hang during block draining

2023-06-22 Thread Lukas Straub
for the BB in-flight counter to decrease. This reverts commit 7e5cdb34 "ide: Increment BB in-flight counter for TRIM BH" to fix this hang. The bug fixed by that commit will be fixed differently in the next commit. Signed-off-by: Lukas Straub --- hw/ide/core.c | 7 --- 1 file changed, 7

[PATCH v3] multifd: Add colo support

2023-06-22 Thread Lukas Straub
Like in the normal ram_load() path, put the received pages into the colo cache and mark the pages in the bitmap so that they will be flushed to the guest later. Signed-off-by: Juan Quintela Signed-off-by: Lukas Straub --- migration/meson.build| 1 + migration/multifd-colo.c | 53

Re: [PATCH] multifd: Add colo support

2023-05-11 Thread Lukas Straub
On Thu, 11 May 2023 11:52:55 +0200 Juan Quintela wrote: > Lukas Straub wrote: > > On Tue, 9 May 2023 20:15:28 +0200 > > Juan Quintela wrote: > > > >> From: Lukas Straub > >> > >> Like in the normal ram_load() path, put the received p

Re: [PATCH] multifd: Add colo support

2023-05-11 Thread Lukas Straub
On Tue, 9 May 2023 20:15:28 +0200 Juan Quintela wrote: > From: Lukas Straub > > Like in the normal ram_load() path, put the received pages into the > colo cache and mark the pages in the bitmap so that they will be > flushed to the guest later. > > Signed

[PATCH v2 6/6] multifd: Add colo support

2023-05-08 Thread Lukas Straub
Like in the normal ram_load() path, put the received pages into the colo cache and mark the pages in the bitmap so that they will be flushed to the guest later. Signed-off-by: Lukas Straub --- migration/multifd-colo.c | 30 +- 1 file changed, 29 insertions(+), 1

[PATCH v2 3/6] multifd: Introduce multifd-internal.h

2023-05-08 Thread Lukas Straub
Introduce multifd-internal.h so code that would normally go into multifd.c can go into an extra file. This way, multifd.c hopefully won't grow to 4000 lines like ram.c This will be used in the next commits to add colo support to multifd. Signed-off-by: Lukas Straub --- migration/multifd

[PATCH v2 4/6] multifd: Introduce a overridable revc_pages method

2023-05-08 Thread Lukas Straub
This allows to override the behaviour around recv_pages. Think of it like a "multifd_colo" child class of multifd. This will be used in the next commits to add colo support to multifd. Signed-off-by: Lukas Straub --- migration/meson.build| 1 + migration/multifd-colo.

[PATCH v2 1/6] ram: Add public helper to set colo bitmap

2023-05-08 Thread Lukas Straub
The overhead of the mutex in non-multifd mode is negligible, because in that case its just the single thread taking the mutex. This will be used in the next commits to add colo support to multifd. Signed-off-by: Lukas Straub --- migration/ram.c | 17 ++--- migration/ram.h | 1 + 2

[PATCH v2 2/6] ram: Let colo_flush_ram_cache take the bitmap_mutex

2023-05-08 Thread Lukas Straub
This is not required, colo_flush_ram_cache does not run concurrently with the multifd threads since the cache is only flushed after everything has been received. But it makes me more comfortable. This will be used in the next commits to add colo support to multifd. Signed-off-by: Lukas Straub

[PATCH v2 5/6] multifd: Add the ramblock to MultiFDRecvParams

2023-05-08 Thread Lukas Straub
This will be used in the next commits to add colo support to multifd. Signed-off-by: Lukas Straub --- migration/multifd.c | 11 +-- migration/multifd.h | 2 ++ 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index fb5e8859de

[PATCH v2 0/6] multifd: Add colo support

2023-05-08 Thread Lukas Straub
Hello Everyone, These patches add support for colo to multifd. -v2: - Split out addition of p->block - Add more comments Lukas Straub (6): ram: Add public helper to set colo bitmap ram: Let colo_flush_ram_cache take the bitmap_mutex multifd: Introduce multifd-internal.h mult

Re: [PULL 00/21] Migration 20230428 patches

2023-05-08 Thread Lukas Straub
On Mon, 08 May 2023 10:12:35 +0200 Juan Quintela wrote: > Lukas Straub wrote: > > On Tue, 02 May 2023 12:39:12 +0200 > > Juan Quintela wrote: > > > >> [...] > >> > >> my patches are only code movement and cleanups, so Lukas any clue? > &

Re: [PULL 00/21] Migration 20230428 patches

2023-05-07 Thread Lukas Straub
On Tue, 02 May 2023 12:39:12 +0200 Juan Quintela wrote: > [...] > > my patches are only code movement and cleanups, so Lukas any clue? > > Lukas, I am going to drop the compress code for now and pass the other > patches. In the meanwhile, I am going to try to setup some machine > where I can

[PATCH 5/5] multifd: Add colo support

2023-05-07 Thread Lukas Straub
Signed-off-by: Lukas Straub --- migration/multifd-colo.c | 30 +- migration/multifd.c | 11 +-- migration/multifd.h | 2 ++ 3 files changed, 36 insertions(+), 7 deletions(-) diff --git a/migration/multifd-colo.c b/migration/multifd-colo.c index

[PATCH 3/5] multifd: Introduce multifd-internal.h

2023-05-07 Thread Lukas Straub
Introduce multifd-internal.h so code that would normally go into multifd.c can go into an extra file. This way, multifd.c hopefully won't grow to 4000 lines like ram.c This will be used in the next commits to add colo support to multifd. Signed-off-by: Lukas Straub --- migration/multifd

[PATCH 0/5] multifd: Add colo support

2023-05-07 Thread Lukas Straub
Hello Everyone, These patches add support for colo to multifd. Lukas Straub (5): ram: Add public helper to set colo bitmap ram: Let colo_flush_ram_cache take the bitmap_mutex multifd: Introduce multifd-internal.h multifd: Introduce a overridable revc_pages method multifd: Add colo

[PATCH 1/5] ram: Add public helper to set colo bitmap

2023-05-07 Thread Lukas Straub
The overhead of the mutex in non-multifd mode is negligible, because in that case its just the single thread taking the mutex. This will be used in the next commits to add colo support to multifd. Signed-off-by: Lukas Straub --- migration/ram.c | 17 ++--- migration/ram.h | 1 + 2

[PATCH 4/5] multifd: Introduce a overridable revc_pages method

2023-05-07 Thread Lukas Straub
This allows to override the behaviour around recv_pages. Think of it like a "multifd_colo" child class of multifd. This will be used in the next commits to add colo support to multifd. Signed-off-by: Lukas Straub --- migration/meson.build| 1 + migration/multifd-colo.

[PATCH 2/5] ram: Let colo_flush_ram_cache take the bitmap_mutex

2023-05-07 Thread Lukas Straub
This will be used in the next commits to add colo support to multifd. Signed-off-by: Lukas Straub --- migration/ram.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 2d3fd2112a..f9e7aeda12 100644 --- a/migration/ram.c +++ b/migration/ram.c

Re: [PATCH v4 10/10] migration: block incoming colo when capability is disabled

2023-05-04 Thread Lukas Straub
On Fri, 5 May 2023 01:30:34 +0300 Vladimir Sementsov-Ogievskiy wrote: > On 05.05.23 01:10, Lukas Straub wrote: > > On Fri, 28 Apr 2023 22:49:28 +0300 > > Vladimir Sementsov-Ogievskiy wrote: > > > >> We generally require same set of capabilities on source and tar

Re: [PATCH v4 10/10] migration: block incoming colo when capability is disabled

2023-05-04 Thread Lukas Straub
way for COLO with multifd. Also, it allows to remove a some code, like migration_incoming_enable_colo(), qemu_savevm_send_colo_enable() etc. I will send patches for that. Or you can do it if you like. Please update the docs like below, then: Reviewed-by: Lukas Straub diff --git a/docs/COLO-FT.txt

Re: [PULL 00/21] Migration 20230428 patches

2023-04-29 Thread Lukas Straub
gration: Rename ram_counters to mig_stats > >migration: Rename RAMStats to MigrationAtomicStats > >migration/rdma: Split the zero page case from acct_update_position > >migration/rdma: Unfold last user of acct_update_position() > >migration: Drop unused parameter f

Re: [PATCH 17/17] qapi: Reformat doc comments to conform to current conventions

2023-04-28 Thread Lukas Straub
consistently use two spaces > to separate sentences. > > To check the generated documentation does not change, I compared the > generated HTML before and after this commit with "wdiff -3". Finds no > differences. Comparing with diff is not useful, as the reflown > parag

Re: [PATCH v3 4/4] configure: add --disable-colo-proxy option

2023-04-27 Thread Lukas Straub
On Thu, 27 Apr 2023 23:29:46 +0300 Vladimir Sementsov-Ogievskiy wrote: > Add option to not build filter-mirror, filter-rewriter and > colo-compare when they are not needed. > > There could be more agile configuration, for example add separate > options for each filter, but that may be done in

Re: [PATCH v3 3/4] build: move COLO under CONFIG_REPLICATION

2023-04-27 Thread Lukas Straub
colo_process_checkpoint() >abort() > > It could probably make sense to have possibility to enable COLO without > REPLICATION, but this requires deeper audit of colo & replication code, > which may be done later if needed. > > Signed-off

Re: [PATCH v3 1/4] block/meson.build: prefer positive condition for replication

2023-04-27 Thread Lukas Straub
On Thu, 27 Apr 2023 23:29:43 +0300 Vladimir Sementsov-Ogievskiy wrote: > Signed-off-by: Vladimir Sementsov-Ogievskiy > Reviewed-by: Juan Quintela > Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Lukas Straub > --- > block/meson.build | 2 +- > 1 file changed, 1 insert

Re: [PATCH 06/19] migration/rdma: Unfold last user of acct_update_position()

2023-04-27 Thread Lukas Straub
On Thu, 27 Apr 2023 18:34:36 +0200 Juan Quintela wrote: > Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub > --- > migration/ram.c | 9 - > migration/ram.h | 1 - > migration/rdma.c | 4 +++- > 3 files changed, 3 insertions(+), 11 deletions(-) > >

Re: [PATCH 05/19] migration/rdma: Split the zero page case from acct_update_position

2023-04-27 Thread Lukas Straub
On Thu, 27 Apr 2023 18:34:35 +0200 Juan Quintela wrote: > Now that we have atomic counters, we can do it on the place that we > need it, no need to do it inside ram.c. > > Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub > --- > migration/ram.c | 12

Re: [PATCH 04/19] migration: Rename RAMStats to MigrationAtomicStats

2023-04-27 Thread Lukas Straub
On Thu, 27 Apr 2023 18:34:34 +0200 Juan Quintela wrote: > It is lousely based on MigrationStats, but that name is taken, so this > is the best one that I came with. > > Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub > --- > > If you have any good suggestion f

Re: [PATCH 03/19] migration: Rename ram_counters to mig_stats

2023-04-27 Thread Lukas Straub
On Thu, 27 Apr 2023 18:34:33 +0200 Juan Quintela wrote: > migration_stats is just too long, and it is going to have more than > ram counters in the near future. > > Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub > --- > migration/migration-stats.c | 2 +- >

Re: [PATCH 02/19] migration: Move ram_stats to its own file migration-stats.[ch]

2023-04-27 Thread Lukas Straub
On Thu, 27 Apr 2023 18:34:32 +0200 Juan Quintela wrote: > There is already include/qemu/stats.h, so stats.h was a bad idea. > We want this file to not depend on anything else, we will move all the > migration counters/stats to this struct. > > Signed-off-by: Juan Quintela Rev

Re: [PATCH 01/19] multifd: We already account for this packet on the multifd thread

2023-04-27 Thread Lukas Straub
On Thu, 27 Apr 2023 18:34:31 +0200 Juan Quintela wrote: > Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub > --- > migration/multifd.c | 3 --- > 1 file changed, 3 deletions(-) > > diff --git a/migration/multifd.c b/migration/multifd.c > index 6a59c03d

Re: [PATCH v2 01/13] qtest/migration-test.c: Add postcopy tests with compress enabled

2023-04-20 Thread Lukas Straub
On Thu, 20 Apr 2023 12:20:25 +0200 Juan Quintela wrote: > Lukas Straub wrote: > > Add postcopy tests with compress enabled to ensure nothing breaks > > with the refactoring in the next commits. > > > > preempt+compress is blocked, so no test needed for that case. &

Re: [PATCH v2 4/4] configure: add --disable-colo-filters option

2023-04-20 Thread Lukas Straub
On Thu, 20 Apr 2023 09:09:48 + "Zhang, Chen" wrote: > > -Original Message- > > From: Vladimir Sementsov-Ogievskiy > > Sent: Thursday, April 20, 2023 6:53 AM > > To: qemu-devel@nongnu.org > > Cc: qemu-bl...@nongnu.org; michael.r...@amd.com; arm...@redhat.com; > > ebl...@redhat.com;

[PATCH v2 14/13] migration: Initialize and cleanup decompression in migration.c

2023-04-20 Thread Lukas Straub
This fixes compress with colo. Signed-off-by: Lukas Straub --- Oops, this one slipped trough migration/migration.c | 9 + migration/ram.c | 5 - 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index bda4789193

[PATCH v2 13/13] ram-compress.c: Make target independent

2023-04-20 Thread Lukas Straub
Make ram-compress.c target independent. Signed-off-by: Lukas Straub --- migration/meson.build| 2 +- migration/ram-compress.c | 17 ++--- 2 files changed, 11 insertions(+), 8 deletions(-) diff --git a/migration/meson.build b/migration/meson.build index 262e3c9754..16f642031c

[PATCH v2 07/13] ram.c: Remove last ram.c dependency from the core compress code

2023-04-20 Thread Lukas Straub
Make compression interfaces take send_queued_data() as an argument. Remove save_page_use_compression() from flush_compressed_data(). This removes the last ram.c dependency from the core compress code. Signed-off-by: Lukas Straub --- migration/ram.c | 27 +-- 1 file

[PATCH v2 09/13] ram.c: Move core compression code into its own file

2023-04-20 Thread Lukas Straub
No functional changes intended. Signed-off-by: Lukas Straub --- migration/meson.build| 5 +- migration/ram-compress.c | 273 +++ migration/ram-compress.h | 65 ++ migration/ram.c | 256 +--- 4 files

[PATCH v2 10/13] ram.c: Remove whitespace (squash with previous patch)

2023-04-20 Thread Lukas Straub
Signed-off-by: Lukas Straub --- migration/ram.c | 18 -- 1 file changed, 18 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 42d6a54132..7be09c18e3 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -529,24 +529,6 @@ static bool pss_overlap(PageSearchStatus

[PATCH v2 03/13] ram.c: Dont change param->block in the compress thread

2023-04-20 Thread Lukas Straub
Instead introduce a extra parameter to trigger the compress thread. Now, when the compress thread is done, we know what RAMBlock and offset it did compress. This will be used in the next commits to move save_page_header() out of compress code. Signed-off-by: Lukas Straub --- migration/ram.c

[PATCH v2 04/13] ram.c: Reset result after sending queued data

2023-04-20 Thread Lukas Straub
ommits to move save_page_header() out of compress code. Signed-off-by: Lukas Straub --- migration/ram.c | 32 ++-- 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 820b4ebaeb..5ca0f115cf 100644 --- a/migration/ram.c

[PATCH v2 01/13] qtest/migration-test.c: Add postcopy tests with compress enabled

2023-04-20 Thread Lukas Straub
Add postcopy tests with compress enabled to ensure nothing breaks with the refactoring in the next commits. preempt+compress is blocked, so no test needed for that case. Signed-off-by: Lukas Straub --- tests/qtest/migration-test.c | 83 +++- 1 file changed, 53

[PATCH v2 11/13] ram.c: Move core decompression code into its own file

2023-04-20 Thread Lukas Straub
No functional changes intended. Signed-off-by: Lukas Straub Reviewed-by: Philippe Mathieu-Daudé --- migration/ram-compress.c | 203 ++ migration/ram-compress.h | 5 + migration/ram.c | 204 --- 3 files changed

[PATCH v2 02/13] ram.c: Let the compress threads return a CompressResult enum

2023-04-20 Thread Lukas Straub
This will be used in the next commits to move save_page_header() out of compress code. Signed-off-by: Lukas Straub --- migration/ram.c | 34 ++ 1 file changed, 22 insertions(+), 12 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 79d881f735

[PATCH v2 08/13] ram.c: Introduce whitespace (squash with next patch)

2023-04-20 Thread Lukas Straub
Introduce whitespace to make it easier to reroll the series. Signed-off-by: Lukas Straub --- migration/ram.c | 12 1 file changed, 12 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index bd3773d4c4..b95c5c720d 100644 --- a/migration/ram.c +++ b/migration/ram.c

[PATCH v2 00/13] migration/ram.c: Refactor compress code

2023-04-20 Thread Lukas Straub
own file" - Add Reviewed-by: Tags Lukas Straub (13): qtest/migration-test.c: Add postcopy tests with compress enabled ram.c: Let the compress threads return a CompressResult enum ram.c: Dont change param->block in the compress thread ram.c: Reset result after sending queued data ra

[PATCH v2 12/13] ram compress: Assert that the file buffer matches the result

2023-04-20 Thread Lukas Straub
Before this series, "nothing to send" was handled by the file buffer being empty. Now it is tracked via param->result. Assert that the file buffer state matches the result. Signed-off-by: Lukas Straub --- migration/qemu-file.c| 11 +++ migration/qemu-file.h| 1 +

[PATCH v2 06/13] ram.c: Call update_compress_thread_counts from compress_send_queued_data

2023-04-20 Thread Lukas Straub
This makes the core compress code more independend from ram.c. Signed-off-by: Lukas Straub --- migration/ram.c | 18 ++ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index d248e1f062..3894d0ae79 100644 --- a/migration/ram.c

[PATCH v2 05/13] ram.c: Do not call save_page_header() from compress threads

2023-04-20 Thread Lukas Straub
. Signed-off-by: Lukas Straub --- migration/ram.c | 44 +++- 1 file changed, 35 insertions(+), 9 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 5ca0f115cf..d248e1f062 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1476,17 +1476,13

Re: [PATCH v2 0/4] COLO: improve build options

2023-04-20 Thread Lukas Straub
am.ru> Hey, This series is a good idea, and looks fine to me. Maybe you can remove the #ifdef CONFIG_REPLICATION/#ifndef CONFIG_REPLICATION from migration/colo.c too while you are at it. Regards, Lukas Straub > Vladimir Sementsov-Ogievskiy (4): > block/meson.build: prefer positive condition

Re: [PATCH v2] migration: Handle block device inactivation failures better

2023-04-20 Thread Lukas Straub
attempting inactivation, > rather than after succeeding, in order to prevent any vm_start() until > it has successfully reactivated all devices. > > See also https://bugzilla.redhat.com/show_bug.cgi?id=2058982 > > Signed-off-by: Eric Blake Looks good to me from colo side

Re: [PATCH for-8.0 v2 2/2] migration/ram.c: Fix migration with compress enabled

2023-04-11 Thread Lukas Straub
Ping... On Thu, 6 Apr 2023 15:21:55 + Lukas Straub wrote: > Ping? This should go in rc4, there is not much time left to prepare a > PULL... > > Best Regards, > Lukas Straub > > On Tue, 4 Apr 2023 14:36:03 + > Lukas Straub wrote: > > > Since ec6f3ab9

Re: [PATCH for-8.0 v2 2/2] migration/ram.c: Fix migration with compress enabled

2023-04-06 Thread Lukas Straub
Ping? This should go in rc4, there is not much time left to prepare a PULL... Best Regards, Lukas Straub On Tue, 4 Apr 2023 14:36:03 + Lukas Straub wrote: > Since ec6f3ab9, migration with compress enabled was broken, because > the compress threads use a dummy QEMUFile which jus

Re: [PATCH 00/14] migration/ram.c: Refactor compress code

2023-04-05 Thread Lukas Straub
On Sun, 2 Apr 2023 17:55:59 + Lukas Straub wrote: > This series refactors the ram compress code. > > It first removes ram.c dependencies from the core compress code, then > moves it out to its own file. Finally, on the migration destination side > the initialisation and clea

[PATCH for-8.0 v2 1/2] qtest/migration-test.c: Add tests with compress enabled

2023-04-04 Thread Lukas Straub
There has never been tests for migration with compress enabled. Add suitable tests, testing with compress-wait-thread = false too. Signed-off-by: Lukas Straub --- v2: - Split into 2 tests, one with compress-wait-thread = true and faster compress options. And one with compress-wait-thread

[PATCH for-8.0 v2 2/2] migration/ram.c: Fix migration with compress enabled

2023-04-04 Thread Lukas Straub
: ec6f3ab9f4 ("migration: Move last_sent_block into PageSearchStatus") Signed-off-by: Lukas Straub Reviewed-by: Peter Xu --- v2: - Add Fixed: and Reviewed-by: tags migration/ram.c | 24 +++- 1 file changed, 11 insertions(+), 13 deletions(-) diff --git a/migrati

Re: [PATCH RESEND 1/2] qtest/migration-test.c: Add test with compress enabled

2023-04-04 Thread Lukas Straub
On Mon, 3 Apr 2023 17:17:52 -0400 Peter Xu wrote: > On Sun, Apr 02, 2023 at 05:47:45PM +0000, Lukas Straub wrote: > > There has never been a test for migration with compress enabled. > > > > Add a suitable test, testing with compress-wait-thread = false > > t

Re: [PATCH 01/14] ram.c: Let the compress threads return a CompressResult enum

2023-04-03 Thread Lukas Straub
On Mon, 3 Apr 2023 09:25:41 +0200 Philippe Mathieu-Daudé wrote: > On 2/4/23 19:56, Lukas Straub wrote: > > This will be used in the next commits to move save_page_header() > > out of compress code. > > > > Signed-off-by: Lukas Straub > &g

[PATCH 14/14] migration: Initialize and cleanup decompression in migration.c

2023-04-02 Thread Lukas Straub
This fixes compress with colo. Signed-off-by: Lukas Straub --- migration/migration.c | 9 + migration/ram.c | 5 - 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index ae2025d9d8..cbdc10b840 100644 --- a/migration

[PATCH 10/14] ram.c: Move core decompression code into its own file

2023-04-02 Thread Lukas Straub
No functional changes intended. Signed-off-by: Lukas Straub --- migration/ram-compress.c | 203 ++ migration/ram-compress.h | 5 + migration/ram.c | 204 --- 3 files changed, 208 insertions(+), 204 deletions

[PATCH 07/14] ram.c: Introduce whitespace (squash with next patch)

2023-04-02 Thread Lukas Straub
Introduce whitespace to make it easier to reroll the series. Signed-off-by: Lukas Straub --- migration/ram.c | 12 1 file changed, 12 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 20428ccf42..475c04a18b 100644 --- a/migration/ram.c +++ b/migration/ram.c

[PATCH 08/14] ram.c: Move core compression code into its own file

2023-04-02 Thread Lukas Straub
No functional changes intended. Signed-off-by: Lukas Straub --- migration/meson.build| 5 +- migration/ram-compress.c | 273 +++ migration/ram-compress.h | 65 ++ migration/ram.c | 255 +--- 4 files

[PATCH 03/14] ram.c: Reset result after sending queued data

2023-04-02 Thread Lukas Straub
ommits to move save_page_header() out of compress code. Signed-off-by: Lukas Straub --- migration/ram.c | 32 ++-- 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 3c9fac086d..bef6292ef7 100644 --- a/migration/ram.c

[PATCH 11/14] ram compress: Assert that the file buffer matches the result

2023-04-02 Thread Lukas Straub
Before this series, "nothing to send" was handled by the file buffer being empty. Now it is tracked via param->result. Assert that the file buffer state matches the result. Signed-off-by: Lukas Straub --- migration/qemu-file.c| 11 +++ migration/qemu-file.h| 1 +

[PATCH 09/14] ram.c: Remove whitespace (squash with previous patch)

2023-04-02 Thread Lukas Straub
Signed-off-by: Lukas Straub --- migration/ram.c | 18 -- 1 file changed, 18 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 114901241e..31b9b0b9ec 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -530,24 +530,6 @@ static bool pss_overlap(PageSearchStatus

[PATCH 04/14] ram.c: Do not call save_page_header() from compress threads

2023-04-02 Thread Lukas Straub
. Signed-off-by: Lukas Straub --- migration/ram.c | 44 +++- 1 file changed, 35 insertions(+), 9 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index bef6292ef7..7ab008145b 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1476,17 +1476,13

[PATCH 06/14] ram.c: Remove last ram.c dependency from the core compress code

2023-04-02 Thread Lukas Straub
Make compression interfaces take send_queued_data() as an argument. Remove save_page_use_compression() from flush_compressed_data(). This removes the last ram.c dependency from the core compress code. Signed-off-by: Lukas Straub --- migration/ram.c | 27 +-- 1 file

[PATCH 00/14] migration/ram.c: Refactor compress code

2023-04-02 Thread Lukas Straub
COLO with compress enabled. This series is based on the following series: https://lore.kernel.org/qemu-devel/af76761aa6978071c5b8e9b872b697db465a5520.1680457631.git.lukasstra...@web.de/T/#t Lukas Straub (14): ram.c: Let the compress threads return a CompressResult enum ram.c: Dont change param

  1   2   3   4   5   6   7   >