Re: [PATCH 3/5] colo: Introduce high-level test suite

2020-06-04 Thread Lukas Straub
On Tue, 2 Jun 2020 14:19:08 +0200 Philippe Mathieu-Daudé wrote: > +Cleber/Wainer > > On 5/11/20 2:27 PM, Lukas Straub wrote: > > Add high-level test relying on the colo resource-agent to test > > all failover cases while checking guest network connectivity. > > >

Re: [PATCH] block/quorum.c: Decrease child index when del_child

2020-06-01 Thread Lukas Straub
chive/html/qemu-block/2020-05/msg00634.html > > Berto > Hi, Did you have a look at my series? There it's fixed properly: https://lore.kernel.org/qemu-devel/9df6b3723ec30cb749ceaa555d82a29a6d79496d.1589199922.git.lukasstra...@web.de/ Regards, Lukas Straub pgpOedZXmirGq.pgp Description: OpenPGP digital signature

[PATCH v4 4/4] migration: Add yank feature

2020-05-25 Thread Lukas Straub
Register yank functions on sockets to shut them down. Signed-off-by: Lukas Straub --- Makefile.objs | 1 + migration/channel.c | 12 migration/migration.c | 18 +- migration/multifd.c | 10 ++ migration/qemu-file

[PATCH v4 1/4] Introduce yank feature

2020-05-25 Thread Lukas Straub
can be queried by a 'query-yank' oob command. Signed-off-by: Lukas Straub --- qapi/misc.json | 45 + yank.c | 174 + yank.h | 67 +++ 3 files changed, 286 insertions(+) create mode 100644 yan

[PATCH v4 3/4] chardev/char-socket.c: Add yank feature

2020-05-25 Thread Lukas Straub
Register a yank function to shutdown the socket on yank. Signed-off-by: Lukas Straub --- Makefile.objs | 1 + chardev/char-socket.c | 24 2 files changed, 25 insertions(+) diff --git a/Makefile.objs b/Makefile.objs index 8e403b81f3..5582f4eda9 100644 --- a

[PATCH v4 0/4] Introduce 'yank' oob qmp command to recover from hanging qemu

2020-05-25 Thread Lukas Straub
s register callbacks which get executed with the yank command. For example the callback can shutdown() a socket. This is intended for the colo use-case, but it can be used for other things too of course. Regards, Lukas Straub v4: -fix build errors... v3: -don't touch softmmu/vl.c, use

[PATCH v4 2/4] block/nbd.c: Add yank feature

2020-05-25 Thread Lukas Straub
Register a yank function which shuts down the socket and sets s->state = NBD_CLIENT_QUIT. This is the same behaviour as if an error occured. Signed-off-by: Lukas Straub --- Makefile.objs | 1 + block/nbd.c | 101 -- 2 files changed,

[PATCH v3 4/4] migration: Add yank feature

2020-05-24 Thread Lukas Straub
Register yank functions on sockets to shut them down. Signed-off-by: Lukas Straub --- Makefile.objs | 1 + migration/channel.c | 12 migration/migration.c | 18 +- migration/multifd.c | 10 ++ migration/qemu-file

[PATCH v3 3/4] chardev/char-socket.c: Add yank feature

2020-05-24 Thread Lukas Straub
Register a yank function to shutdown the socket on yank. Signed-off-by: Lukas Straub --- Makefile.objs | 1 + chardev/char-socket.c | 24 2 files changed, 25 insertions(+) diff --git a/Makefile.objs b/Makefile.objs index 8e403b81f3..5582f4eda9 100644 --- a

[PATCH v3 1/4] Introduce yank feature

2020-05-24 Thread Lukas Straub
can be queried by a 'query-yank' oob command. Signed-off-by: Lukas Straub --- qapi/misc.json | 45 + yank.c | 174 + yank.h | 67 +++ 3 files changed, 286 insertions(+) create mode 100644 yan

[PATCH v3 2/4] block/nbd.c: Add yank feature

2020-05-24 Thread Lukas Straub
Register a yank function which shuts down the socket and sets s->state = NBD_CLIENT_QUIT. This is the same behaviour as if an error occured. Signed-off-by: Lukas Straub --- Makefile.objs | 1 + block/nbd.c | 101 -- 2 files changed,

[PATCH v3 0/4] Introduce 'yank' oob qmp command to recover from hanging qemu

2020-05-24 Thread Lukas Straub
s register callbacks which get executed with the yank command. For example the callback can shutdown() a socket. This is intended for the colo use-case, but it can be used for other things too of course. Regards, Lukas Straub v3: -don't touch softmmu/vl.c, use __contructor__ attribute

[PATCH v6 5/6] net/colo-compare.c: Check that colo-compare is active

2020-05-22 Thread Lukas Straub
iodic mode) and to use multiple colo-compare for multiple network interfaces. Signed-off-by: Lukas Straub Tested-by: Lukas Straub --- net/colo-compare.c | 35 +-- 1 file changed, 29 insertions(+), 6 deletions(-) diff --git a/net/colo-compare.c b/net/colo-comp

[PATCH v6 3/6] net/colo-compare.c: Fix deadlock in compare_chr_send

2020-05-21 Thread Lukas Straub
full. This leads to a deadlock because both event loops get blocked. Fix this by converting compare_chr_send to a coroutine and putting the packets in a send queue. Signed-off-by: Lukas Straub Reviewed-by: Zhang Chen Tested-by: Zhang Chen --- net/colo-compare.c

[PATCH v6 6/6] net/colo-compare.c: Correct ordering in complete and finalize

2020-05-21 Thread Lukas Straub
In colo_compare_complete, insert CompareState into net_compares only after everything has been initialized. In colo_compare_finalize, remove CompareState from net_compares before anything is deinitialized. Signed-off-by: Lukas Straub Reviewed-by: Zhang Chen --- net/colo-compare.c | 45

[PATCH v6 2/6] chardev/char.c: Use qemu_co_sleep_ns if in coroutine

2020-05-21 Thread Lukas Straub
This will be needed in the next patch so compare_chr_send can be converted to a coroutine. Signed-off-by: Lukas Straub Reviewed-by: Marc-André Lureau Reviewed-by: Zhang Chen Reviewed-by: Philippe Mathieu-Daudé --- chardev/char.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion

[PATCH v6 1/6] net/colo-compare.c: Create event_bh with the right AioContext

2020-05-21 Thread Lukas Straub
fix the crashes. This is safe, because the bh already takes the appropriate locks. Signed-off-by: Lukas Straub Reviewed-by: Zhang Chen Reviewed-by: Derek Su Tested-by: Derek Su --- net/colo-compare.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/net/colo-compare.c b/net

[PATCH v6 0/6] colo-compare bugfixes

2020-05-21 Thread Lukas Straub
Hello Everyone, Here are fixes for bugs that I found in my tests. I have tested this with my test suite and everything works fine. Regards, Lukas Straub Version changes: v6: -don't touch softmmu/vl.c, use __contructor__ attribute instead -better wording v5: -initialize and use notify_s

[PATCH v6 4/6] net/colo-compare.c: Only hexdump packets if tracing is enabled

2020-05-21 Thread Lukas Straub
Else the log will be flooded if there is a lot of network traffic. Signed-off-by: Lukas Straub Reviewed-by: Zhang Chen Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé --- net/colo-compare.c | 10 ++ 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a

Re: [PATCH 0/7] Latest COLO tree queued patches

2020-05-21 Thread Lukas Straub
On Thu, 21 May 2020 01:30:48 + "Zhang, Chen" wrote: > > -Original Message- > > From: Jason Wang > > Sent: Wednesday, May 20, 2020 8:23 PM > > To: Zhang, Chen ; qemu-devel@nongnu.org; Lukas > > Straub > > Cc: zhangc...@gmail.com > &

Re: [PATCH v2 4/4] migration: Add yank feature

2020-05-21 Thread Lukas Straub
On Wed, 20 May 2020 23:05:50 +0200 Lukas Straub wrote: > Register yank functions on sockets to shut them down. > > Signed-off-by: Lukas Straub Don't review this commit for now, I'll have to revamp it anyway. Regards, Lukas Straub pgpNmGVoC9Qjd.pgp Description: OpenPGP digital signature

Re: [PATCH v2 1/4] Introduce yank feature

2020-05-21 Thread Lukas Straub
On Thu, 21 May 2020 16:03:35 +0100 Stefan Hajnoczi wrote: > On Wed, May 20, 2020 at 11:05:39PM +0200, Lukas Straub wrote: > > +void yank_generic_iochannel(void *opaque) > > +{ > > +QIOChannel *ioc = QIO_CHANNEL(opaque); > > + > > +qio_channel_shutdown(ioc

[PATCH v2 4/4] migration: Add yank feature

2020-05-20 Thread Lukas Straub
Register yank functions on sockets to shut them down. Signed-off-by: Lukas Straub --- migration/migration.c | 9 + migration/qemu-file-channel.c | 6 ++ migration/socket.c| 11 +++ 3 files changed, 26 insertions(+) diff --git a/migration/migration.c b

[PATCH v2 2/4] block/nbd.c: Add yank feature

2020-05-20 Thread Lukas Straub
Register a yank function which shuts down the socket and sets s->state = NBD_CLIENT_QUIT. This is the same behaviour as if an error occured. Signed-off-by: Lukas Straub --- Makefile.objs | 1 + block/nbd.c | 101 -- 2 files changed,

[PATCH v2 3/4] chardev/char-socket.c: Add yank feature

2020-05-20 Thread Lukas Straub
Register a yank function to shutdown the socket on yank. Signed-off-by: Lukas Straub --- chardev/char-socket.c | 24 1 file changed, 24 insertions(+) diff --git a/chardev/char-socket.c b/chardev/char-socket.c index 185fe38dda..d5c6cd2153 100644 --- a/chardev/char

[PATCH v2 1/4] Introduce yank feature

2020-05-20 Thread Lukas Straub
can be queried by a 'query-yank' oob command. Signed-off-by: Lukas Straub --- qapi/misc.json | 45 + softmmu/vl.c | 2 + yank.c | 174 + yank.h | 69 4 files changed, 290 insertions

[PATCH v2 0/4] Introduce 'yank' oob qmp command to recover from hanging qemu

2020-05-20 Thread Lukas Straub
s register callbacks which get executed with the yank command. For example the callback can shutdown() a socket. This is intended for the colo use-case, but it can be used for other things too of course. Regards, Lukas Straub v2: -don't touch io/ code anymore -always register yank functio

[PATCH v2 6/6] migration/migration.c: Fix hang in ram_save_host_page

2020-05-20 Thread Lukas Straub
migration_rate_limit will erroneously ratelimit a shutdown socket, which causes the migration thread to hang in ram_save_host_page if the socket is shutdown. Fix this by explicitly testing if the socket has errors or was shutdown in migration_rate_limit. Signed-off-by: Lukas Straub

[PATCH v2 5/6] migration/colo.c: Move colo_notify_compares_event to the right place

2020-05-20 Thread Lukas Straub
If the secondary has to failover during checkpointing, it still is in the old state (i.e. different state than primary). Thus we can't expose the primary state until after the checkpoint is sent. This fixes sporadic connection reset of client connections during failover. Signed-off-by:

[PATCH v2 4/6] migration/colo.c: Relaunch failover even if there was an error

2020-05-20 Thread Lukas Straub
er on colo_incoming_sem. Fix this by relaunching failover even if there was an error. Also, to make this work properly, set vmstate_loading to false when returning during the vmstate_loading section. Signed-off-by: Lukas Straub Reviewed-by: zhanghailiang --- migration/colo.c | 17 - 1 file ch

[PATCH v2 3/6] migration/colo.c: Flush ram cache only after receiving device state

2020-05-20 Thread Lukas Straub
If we suceed in receiving ram state, but fail receiving the device state, there will be a mismatch between the two. Fix this by flushing the ram cache only after the vmstate has been received. Signed-off-by: Lukas Straub Reviewed-by: zhanghailiang --- migration/colo.c | 1 + migration/ram.c

Re: [PATCH 5/6] migration/qemu-file.c: Don't ratelimit a shutdown fd

2020-05-20 Thread Lukas Straub
On Tue, 19 May 2020 15:50:20 +0100 "Dr. David Alan Gilbert" wrote: > * Lukas Straub (lukasstra...@web.de) wrote: > > On Mon, 18 May 2020 12:55:34 +0100 > > "Dr. David Alan Gilbert" wrote: > > > > > * Zhanghailiang (zhang.zhanghaili...@huawei

[PATCH v2 0/6] colo: migration related bugfixes

2020-05-20 Thread Lukas Straub
Hello Everyone, Here are fixes for bugs that I found in my tests. Regards, Lukas Straub v2: -Don't touch qemu_file_rate_limit, solve the problem in migration_rate_limit Lukas Straub (6): migration/colo.c: Use event instead of semaphore migration/colo.c: Use cpu_synchronize_all_s

[PATCH v2 1/6] migration/colo.c: Use event instead of semaphore

2020-05-20 Thread Lukas Straub
the checkpoint event is sent to colo-compare (which releases the miscompared packets). Benchmark results (iperf3): Client-to-server tcp: without patch: ~66 Mbit/s with patch: ~61 Mbit/s Server-to-client tcp: without patch: ~702 Kbit/s with patch: ~16 Mbit/s Signed-off-by: Lukas Straub Reviewed-by

[PATCH v2 2/6] migration/colo.c: Use cpu_synchronize_all_states()

2020-05-20 Thread Lukas Straub
using cpu_synchronize_all_states(), which initializes CPUState from the current cpu registers additionally to marking the vcpus as dirty. Signed-off-by: Lukas Straub Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a

Re: [PATCH 5/6] migration/qemu-file.c: Don't ratelimit a shutdown fd

2020-05-19 Thread Lukas Straub
ork traffic anyway. > > > > > > > I'm not quite sure if this modification can take side effect on normal > > migration process or not, > > There are several places calling it. > > > > Maybe Juan and Dave can help ;) > > > > >

Re: [PATCH 3/5] block/nbd.c: Add yank feature

2020-05-15 Thread Lukas Straub
On Fri, 15 May 2020 11:26:13 +0100 Daniel P. Berrangé wrote: > On Fri, May 15, 2020 at 12:14:47PM +0200, Lukas Straub wrote: > > On Fri, 15 May 2020 11:04:13 +0100 > > Daniel P. Berrangé wrote: > > > > > On Fri, May 15, 2020 at 11:48:18AM +0200, Lukas Straub w

[PATCH v5 4/6] net/colo-compare.c: Only hexdump packets if tracing is enabled

2020-05-15 Thread Lukas Straub
Else the log will be flooded if there is a lot of network traffic. Signed-off-by: Lukas Straub Reviewed-by: Zhang Chen Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé --- net/colo-compare.c | 10 ++ 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a

[PATCH v5 5/6] net/colo-compare.c, softmmu/vl.c: Check that colo-compare is active

2020-05-15 Thread Lukas Straub
iodic mode) and to use multiple colo-compare for multiple network interfaces. Signed-off-by: Lukas Straub --- net/colo-compare.c | 35 +-- net/colo-compare.h | 1 + softmmu/vl.c | 2 ++ 3 files changed, 32 insertions(+), 6 deletions(-) diff --git a/net

[PATCH v5 6/6] net/colo-compare.c: Correct ordering in complete and finalize

2020-05-15 Thread Lukas Straub
In colo_compare_complete, insert CompareState into net_compares only after everything has been initialized. In colo_compare_finalize, remove CompareState from net_compares before anything is deinitialized. Signed-off-by: Lukas Straub Reviewed-by: Zhang Chen --- net/colo-compare.c | 45

[PATCH v5 3/6] net/colo-compare.c: Fix deadlock in compare_chr_send

2020-05-15 Thread Lukas Straub
full. This leads to a deadlock because both event loops get blocked. Fix this by converting compare_chr_send to a coroutine and putting the packets in a send queue. Signed-off-by: Lukas Straub Reviewed-by: Zhang Chen Tested-by: Zhang Chen --- net/colo-compare.c

[PATCH v5 0/6] colo-compare bugfixes

2020-05-15 Thread Lukas Straub
Hello Everyone, Here are fixes for bugs that I found in my tests. Regards, Lukas Straub Version changes: v5: -initialize and use notify_sendco only if notify_dev is set v4: -fix potential deadlock with notify_remote_frame -avoid malloc and memcpy in many cases v3: -fix checkpatch.pl error

[PATCH v5 1/6] net/colo-compare.c: Create event_bh with the right AioContext

2020-05-15 Thread Lukas Straub
fix the crashes. This is safe, because the bh already takes the appropriate locks. Signed-off-by: Lukas Straub Reviewed-by: Zhang Chen Reviewed-by: Derek Su Tested-by: Derek Su --- net/colo-compare.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/net/colo-compare.c b/net

[PATCH v5 2/6] chardev/char.c: Use qemu_co_sleep_ns if in coroutine

2020-05-15 Thread Lukas Straub
This will be needed in the next patch. Signed-off-by: Lukas Straub Reviewed-by: Marc-André Lureau Reviewed-by: Zhang Chen --- chardev/char.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/chardev/char.c b/chardev/char.c index e77564060d..5c8014199f 100644 --- a

Re: [PATCH 3/5] block/nbd.c: Add yank feature

2020-05-15 Thread Lukas Straub
On Fri, 15 May 2020 11:04:13 +0100 Daniel P. Berrangé wrote: > On Fri, May 15, 2020 at 11:48:18AM +0200, Lukas Straub wrote: > > On Tue, 12 May 2020 09:54:58 +0100 > > Daniel P. Berrangé wrote: > > > > > On Mon, May 11, 2020 at 07:05:24PM +0200, Lukas Straub w

Re: [PATCH 3/5] block/nbd.c: Add yank feature

2020-05-15 Thread Lukas Straub
On Tue, 12 May 2020 09:54:58 +0100 Daniel P. Berrangé wrote: > On Mon, May 11, 2020 at 07:05:24PM +0200, Lukas Straub wrote: > > On Mon, 11 May 2020 17:19:09 +0100 > > "Dr. David Alan Gilbert" wrote: > > > > > * Lukas Straub (lukasstra...@web.de) wrot

Re: About migration/colo issue

2020-05-15 Thread Lukas Straub
aused by Hailiang's optimize patch: Hmm, I'm on v5.0.0 (where that commit is in) and I don't have this issue in my testing. Regards, Lukas Straub > From 0393031a16735835a441b6d6e0495a1bd14adb90 Mon Sep 17 00:00:00 2001 > From: zhanghailiang > Date: Mon, 24 Feb 2020 14:

Re: [PATCH 6/6] migration/colo.c: Move colo_notify_compares_event to the right place

2020-05-14 Thread Lukas Straub
5. The client responds with ACK to the SYN-ACK packet. 6. Because it doesn't know the connection, the secondary responds with RST, connection reset. Regards, Lukas Straub > > This fixes sporadic connection reset of client connections during failover. > > > > Signed-

Re: [PATCH 2/6] migration/colo.c: Use cpu_synchronize_all_states()

2020-05-13 Thread Lukas Straub
On Wed, 13 May 2020 10:47:02 +0100 "Dr. David Alan Gilbert" wrote: > * Lukas Straub (lukasstra...@web.de) wrote: > > cpu_synchronize_all_pre_loadvm() marks all vcpus as dirty, so the > > registers are loaded from CPUState before we continue running > > the vm

Re: [PATCH 0/5] Introduce 'yank' oob qmp command to recover from hanging qemu

2020-05-13 Thread Lukas Straub
; address, migration could register a yank function to kill that external command on yank (Won't be implemented yet though). Regards, Lukas Straub pgpIjV1bZXPyk.pgp Description: OpenPGP digital signature

Re: [PATCH 0/5] Introduce 'yank' oob qmp command to recover from hanging qemu

2020-05-12 Thread Lukas Straub
On Tue, 12 May 2020 10:43:37 +0100 Daniel P. Berrangé wrote: > On Tue, May 12, 2020 at 11:32:06AM +0200, Lukas Straub wrote: > > ... > > > > Good Idea. We could name the connections (/yank callbacks) in the > > form "nbd:", "chardev:" and &qu

Re: [PATCH 0/5] Introduce 'yank' oob qmp command to recover from hanging qemu

2020-05-12 Thread Lukas Straub
ist of connection names as you suggest and silently ignore connections that don't exist. And maybe even add a 'query-yank' oob command returning a list of registered connections so the management application can do pattern matching if it wants. Comments? Regards, Lukas Str

Re: [PATCH 2/5] io/channel.c,io/channel-socket.c: Add yank feature

2020-05-11 Thread Lukas Straub
On Mon, 11 May 2020 12:51:46 +0100 Daniel P. Berrangé wrote: > On Mon, May 11, 2020 at 01:14:41PM +0200, Lukas Straub wrote: > > Add qio_channel_set_yank function to channel and to channel-socket, > > which will register a yank function. The yank function calls > > shu

Re: [PATCH 0/5] Introduce 'yank' oob qmp command to recover from hanging qemu

2020-05-11 Thread Lukas Straub
On Mon, 11 May 2020 13:17:14 +0100 Daniel P. Berrangé wrote: > On Mon, May 11, 2020 at 01:07:18PM +0100, Dr. David Alan Gilbert wrote: > > * Daniel P. Berrangé (berra...@redhat.com) wrote: > > > On Mon, May 11, 2020 at 01:14:34PM +0200, Lukas Straub wrote: > > > &

Re: [PATCH 0/5] Introduce 'yank' oob qmp command to recover from hanging qemu

2020-05-11 Thread Lukas Straub
On Mon, 11 May 2020 12:49:47 +0100 Daniel P. Berrangé wrote: > On Mon, May 11, 2020 at 01:14:34PM +0200, Lukas Straub wrote: > > Hello Everyone, > > In many cases, if qemu has a network connection (qmp, migration, chardev, > > etc.) > > to some other server and that

Re: [PATCH 3/5] block/nbd.c: Add yank feature

2020-05-11 Thread Lukas Straub
On Mon, 11 May 2020 17:19:09 +0100 "Dr. David Alan Gilbert" wrote: > * Lukas Straub (lukasstra...@web.de) wrote: > > Add yank option, pass it to the socket-channel and register a yank > > function which sets s->state = NBD_CLIENT_QUIT. This is the same > >

Re: [PATCH] tests/acceptance/boot_linux: Skip slow Aarch64 'virt' machine TCG test

2020-05-11 Thread Lukas Straub
On Mon, 11 May 2020 10:11:20 +0100 Peter Maydell wrote: > On Sat, 9 May 2020 at 14:18, Lukas Straub wrote: > > Hi, > > Why not simply add slow tag to the test. Like: > > :avocado: tags=slow > > > > The slow tests can then be skipped with > > $ make

[PATCH 3/5] colo: Introduce high-level test suite

2020-05-11 Thread Lukas Straub
Add high-level test relying on the colo resource-agent to test all failover cases while checking guest network connectivity. Signed-off-by: Lukas Straub --- scripts/colo-resource-agent/crm_master | 44 ++ scripts/colo-resource-agent/crm_resource | 12 + tests/acceptance/colo.py

[PATCH 5/5] MAINTAINERS: Add myself as maintainer for COLO resource agent

2020-05-11 Thread Lukas Straub
While I'm not going to have much time for this, I'll still try to test and review patches. Signed-off-by: Lukas Straub --- MAINTAINERS | 6 ++ 1 file changed, 6 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 8cbc1fac2b..4c623a96e1 100644 --- a/MAINTAINERS +++ b/M

[PATCH 2/5] colo: Introduce resource agent

2020-05-11 Thread Lukas Straub
Introduce a resource agent which can be used to manage qemu COLO in a pacemaker cluster. Signed-off-by: Lukas Straub --- scripts/colo-resource-agent/colo | 1429 ++ 1 file changed, 1429 insertions(+) create mode 100755 scripts/colo-resource-agent/colo diff --git a

[PATCH 4/5] configure,Makefile: Install colo resource-agent

2020-05-11 Thread Lukas Straub
Optionally install the resouce-agent so it gets picked up by pacemaker. Signed-off-by: Lukas Straub --- Makefile | 5 + configure | 10 ++ 2 files changed, 15 insertions(+) diff --git a/Makefile b/Makefile index 8a9113e666..2ebffc4465 100644 --- a/Makefile +++ b/Makefile

[PATCH 1/5] block/quorum.c: stable children names

2020-05-11 Thread Lukas Straub
If we remove the child with the highest index from the quorum, decrement s->next_child_index. This way we get stable children names as long as we only remove the last child. Signed-off-by: Lukas Straub --- block/quorum.c | 20 ++-- 1 file changed, 14 insertions(+), 6 deleti

[PATCH 0/5] colo: Introduce resource agent and test suite/CI

2020-05-11 Thread Lukas Straub
eir own CI? Regards, Lukas Straub Lukas Straub (5): block/quorum.c: stable children names colo: Introduce resource agent colo: Introduce high-level test suite configure,Makefile: Install colo resource-agent MAINTAINERS: Add myself as maintainer for COLO resource agent M

[PATCH 3/5] block/nbd.c: Add yank feature

2020-05-11 Thread Lukas Straub
Add yank option, pass it to the socket-channel and register a yank function which sets s->state = NBD_CLIENT_QUIT. This is the same behaviour as if an error occured. Signed-off-by: Lukas Straub --- Makefile.objs| 1 + block/nbd.c |

[PATCH 2/5] io/channel.c,io/channel-socket.c: Add yank feature

2020-05-11 Thread Lukas Straub
Add qio_channel_set_yank function to channel and to channel-socket, which will register a yank function. The yank function calls shutdown() on the socket. Signed-off-by: Lukas Straub --- Makefile.objs | 1 + include/io/channel-socket.h | 1 + include/io/channel.h| 12

[PATCH 6/6] migration/colo.c: Move colo_notify_compares_event to the right place

2020-05-11 Thread Lukas Straub
If the secondary has to failover during checkpointing, it still is in the old state (i.e. different state than primary). Thus we can't expose the primary state until after the checkpoint is sent. This fixes sporadic connection reset of client connections during failover. Signed-off-by:

[PATCH 5/5] migration: Add yank feature

2020-05-11 Thread Lukas Straub
Add yank option which is passed to the socket-channel. Signed-off-by: Lukas Straub --- migration/channel.c | 2 ++ migration/migration.c | 11 +++ qapi/migration.json | 17 ++--- 3 files changed, 27 insertions(+), 3 deletions(-) diff --git a/migration/channel.c b

[PATCH 5/6] migration/qemu-file.c: Don't ratelimit a shutdown fd

2020-05-11 Thread Lukas Straub
This causes the migration thread to hang if we failover during checkpoint. A shutdown fd won't cause network traffic anyway. Signed-off-by: Lukas Straub --- migration/qemu-file.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/qemu-file.c b/migration/qemu-f

[PATCH 4/6] migration/colo.c: Relaunch failover even if there was an error

2020-05-11 Thread Lukas Straub
er on colo_incoming_sem. Fix this by relaunching failover even if there was an error. Also, to make this work properly, set vmstate_loading to false when returning during the vmstate_loading section. Signed-off-by: Lukas Straub --- migration/colo.c | 17 - 1 file changed, 12 insertions(

[PATCH 4/5] chardev/char-socket.c: Add yank feature

2020-05-11 Thread Lukas Straub
Add yank option which is passed to the socket-channel. Signed-off-by: Lukas Straub --- chardev/char-socket.c | 8 chardev/char.c| 3 +++ qapi/char.json| 5 - 3 files changed, 15 insertions(+), 1 deletion(-) diff --git a/chardev/char-socket.c b/chardev/char-socket.c

[PATCH 3/6] migration/colo.c: Flush ram cache only after receiving device state

2020-05-11 Thread Lukas Straub
If we suceed in receiving ram state, but fail receiving the device state, there will be a mismatch between the two. Fix this by flushing the ram cache only after the vmstate has been received. Signed-off-by: Lukas Straub --- migration/colo.c | 1 + migration/ram.c | 5 + migration/ram.h

[PATCH 2/6] migration/colo.c: Use cpu_synchronize_all_states()

2020-05-11 Thread Lukas Straub
using cpu_synchronize_all_states(), which initializes CPUState from the current cpu registers additionally to marking the vcpus as dirty. Signed-off-by: Lukas Straub --- migration/colo.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/colo.c b/migration/colo.c index

[PATCH 0/5] Introduce 'yank' oob qmp command to recover from hanging qemu

2020-05-11 Thread Lukas Straub
s register callbacks which get executed with the yank command. For example the callback can shutdown() a socket. This is intended for the colo use-case, but it can be used for other things too of course. Regards, Lukas Straub Lukas Straub (5): Introduce yank feature io/channel.c,io/channel-soc

[PATCH 1/5] Introduce yank feature

2020-05-11 Thread Lukas Straub
The yank feature allows to recover from hanging qemu by "yanking" at various parts. Other qemu systems can register yank functions which will be called by the 'yank' out-of-band qmp command. Signed-off-by: Lukas Straub --- qapi/misc.json | 15 ++ softmmu/vl

[PATCH 1/6] migration/colo.c: Use event instead of semaphore

2020-05-11 Thread Lukas Straub
the checkpoint event is sent to colo-compare (which releases the miscompared packets). Benchmark results (iperf3): Client-to-server tcp: without patch: ~66 Mbit/s with patch: ~61 Mbit/s Server-to-client tcp: without patch: ~702 Kbit/s with patch: ~16 Mbit/s Signed-off-by: Lukas Straub

[PATCH 0/6] colo: migration related bugfixes

2020-05-11 Thread Lukas Straub
Hello Everyone, Here are fixes for bugs that I found in my tests. Regards, Lukas Straub Lukas Straub (6): migration/colo.c: Use event instead of semaphore migration/colo.c: Use cpu_synchronize_all_states() migration/colo.c: Flush ram cache only after receiving device state migration

[PATCH] block/replication.c: Avoid cancelling the job twice

2020-05-11 Thread Lukas Straub
kpoint() won't touch the job. Signed-off-by: Lukas Straub --- block/replication.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/block/replication.c b/block/replication.c index da013c2041..33f2f62a44 100644 --- a/block/replication.c +++ b/block/replication.c @@ -398,6 +398,8 @@

Re: [PATCH] tests/acceptance/boot_linux: Skip slow Aarch64 'virt' machine TCG test

2020-05-09 Thread Lukas Straub
; def test_virt_tcg(self): > """ > :avocado: tags=accel:tcg Hi, Why not simply add slow tag to the test. Like: :avocado: tags=slow The slow tests can then be skipped with $ make check-acceptance AVOCADO_TAGS='-t -slow' Regards, Lukas Straub pgp6hhth052VR.pgp Description: OpenPGP digital signature

Re: [PATCH v4 5/6] net/colo-compare.c, softmmu/vl.c: Check that colo-compare is active

2020-05-09 Thread Lukas Straub
On Fri, 8 May 2020 06:50:39 + "Zhang, Chen" wrote: > > -Original Message- > > From: Lukas Straub > > Sent: Friday, May 8, 2020 2:11 PM > > To: Zhang, Chen > > Cc: qemu-devel ; Li Zhijian > > ; Jason Wang ; Marc- > > André Lureau

Re: [PATCH v4 3/6] net/colo-compare.c: Fix deadlock in compare_chr_send

2020-05-08 Thread Lukas Straub
On Fri, 8 May 2020 06:28:45 + "Zhang, Chen" wrote: > > -Original Message- > > From: Lukas Straub > > Sent: Friday, May 8, 2020 2:08 PM > > To: Zhang, Chen > > Cc: qemu-devel ; Li Zhijian > > ; Jason Wang ; Marc- > > André Lureau

Re: [PATCH v4 5/6] net/colo-compare.c, softmmu/vl.c: Check that colo-compare is active

2020-05-07 Thread Lukas Straub
On Fri, 8 May 2020 02:26:21 + "Zhang, Chen" wrote: > > -Original Message- > > From: Lukas Straub > > Sent: Thursday, May 7, 2020 11:54 PM > > To: Zhang, Chen > > Cc: qemu-devel ; Li Zhijian > > ; Jason Wang ; Marc- > > André Lure

Re: [PATCH v4 3/6] net/colo-compare.c: Fix deadlock in compare_chr_send

2020-05-07 Thread Lukas Straub
performance drops in my tests: > > Client-to-server tcp: > > without patch: ~66 Mbit/s > > with patch: ~59 Mbit/s > > Server-to-client tcp: > > without patch: ~702 Kbit/s > > with patch: ~328 Kbit/s > > Oh, a big performance drop, is t

Re: [PATCH v4 6/6] net/colo-compare.c: Correct ordering in complete and finalize

2020-05-07 Thread Lukas Straub
On Thu, 7 May 2020 13:26:11 + "Zhang, Chen" wrote: > > -Original Message- > > From: Lukas Straub > > Sent: Monday, May 4, 2020 6:28 PM > > To: qemu-devel > > Cc: Zhang, Chen ; Li Zhijian > > ; Jason Wang ; Marc- > > André Lureau

Re: [PATCH v4 5/6] net/colo-compare.c, softmmu/vl.c: Check that colo-compare is active

2020-05-07 Thread Lukas Straub
On Thu, 7 May 2020 11:38:04 + "Zhang, Chen" wrote: > > -Original Message- > > From: Lukas Straub > > Sent: Monday, May 4, 2020 6:28 PM > > To: qemu-devel > > Cc: Zhang, Chen ; Li Zhijian > > ; Jason Wang ; Marc- > > André Lureau

Re: [PATCH v4 3/6] net/colo-compare.c: Fix deadlock in compare_chr_send

2020-05-07 Thread Lukas Straub
On Thu, 7 May 2020 11:00:26 + "Zhang, Chen" wrote: > > -Original Message- > > From: Lukas Straub > > Sent: Monday, May 4, 2020 6:28 PM > > To: qemu-devel > > Cc: Zhang, Chen ; Li Zhijian > > ; Jason Wang ; Marc- > > André Lur

Re: [Query] VM CPU scheduling

2020-05-04 Thread Lukas Straub
inux, so you can use all the tools and interfaces to manage processes for qemu as well. Keywords: top, nice, cgroups, cpuset, numactl Regards, Lukas Straub pgpJPc9LEiXQT.pgp Description: OpenPGP digital signature

[PATCH v4 4/6] net/colo-compare.c: Only hexdump packets if tracing is enabled

2020-05-04 Thread Lukas Straub
Else the log will be flooded if there is a lot of network traffic. Signed-off-by: Lukas Straub Reviewed-by: Zhang Chen --- net/colo-compare.c | 10 ++ 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/net/colo-compare.c b/net/colo-compare.c index 2a4e7f7c4e..56db3d3bfc

[PATCH v4 5/6] net/colo-compare.c, softmmu/vl.c: Check that colo-compare is active

2020-05-04 Thread Lukas Straub
iodic mode) and to use multiple colo-compare for multiple network interfaces. Signed-off-by: Lukas Straub --- net/colo-compare.c | 35 +-- net/colo-compare.h | 1 + softmmu/vl.c | 2 ++ 3 files changed, 32 insertions(+), 6 deletions(-) diff --git a/net

[PATCH v4 2/6] chardev/char.c: Use qemu_co_sleep_ns if in coroutine

2020-05-04 Thread Lukas Straub
This will be needed in the next patch. Signed-off-by: Lukas Straub Reviewed-by: Marc-André Lureau Reviewed-by: Zhang Chen --- chardev/char.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/chardev/char.c b/chardev/char.c index e77564060d..5c8014199f 100644 --- a

[PATCH v4 3/6] net/colo-compare.c: Fix deadlock in compare_chr_send

2020-05-04 Thread Lukas Straub
full. This leads to a deadlock because both event loops get blocked. Fix this by converting compare_chr_send to a coroutine and putting the packets in a send queue. Signed-off-by: Lukas Straub --- net/colo-compare.c | 187 ++--- net/colo.c | 7 ++

[PATCH v4 6/6] net/colo-compare.c: Correct ordering in complete and finalize

2020-05-04 Thread Lukas Straub
In colo_compare_complete, insert CompareState into net_compares only after everything has been initialized. In colo_compare_finalize, remove CompareState from net_compares before anything is deinitialized. Signed-off-by: Lukas Straub --- net/colo-compare.c | 45

[PATCH v4 0/6] colo-compare bugfixes

2020-05-04 Thread Lukas Straub
Hello Everyone, Here are fixes for bugs that I found in my tests. Benchmark results: Client-to-server tcp: without patch: ~63 Mbit/s with patch: ~66 Mbit/s Server-to-client tcp: without patch: ~771 Kbit/s with patch: ~702 Kbit/s Regards, Lukas Straub Version changes: v4: -fix potential

[PATCH v4 1/6] net/colo-compare.c: Create event_bh with the right AioContext

2020-05-04 Thread Lukas Straub
fix the crashes. This is safe, because the bh already takes the appropriate locks. Signed-off-by: Lukas Straub Reviewed-by: Zhang Chen Reviewed-by: Derek Su Tested-by: Derek Su --- net/colo-compare.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/net/colo-compare.c b/net

Re: [PATCH v3 3/6] net/colo-compare.c: Fix deadlock in compare_chr_send

2020-04-29 Thread Lukas Straub
On Wed, 29 Apr 2020 05:37:17 + "Zhang, Chen" wrote: > > -Original Message- > > From: Lukas Straub > > Sent: Monday, April 27, 2020 3:22 PM > > To: Zhang, Chen > > Cc: qemu-devel ; Li Zhijian > > ; Jason Wang ; Marc- > > André Lure

Re: colo: qemu 4.2.0 vs. qemu 5.0.0-rc2 performance regression

2020-04-27 Thread Lukas Straub
On Mon, 27 Apr 2020 11:34:32 +0100 "Dr. David Alan Gilbert" wrote: > * Lukas Straub (lukasstra...@web.de) wrote: > > On Sat, 11 Apr 2020 19:16:54 +0200 > > Lukas Straub wrote: > > > > > Hello Everyone, > > > I did some Benchmarking with iperf3

Re: [PATCH v3 3/6] net/colo-compare.c: Fix deadlock in compare_chr_send

2020-04-27 Thread Lukas Straub
On Mon, 27 Apr 2020 03:36:57 + "Zhang, Chen" wrote: > > -Original Message- > > From: Lukas Straub > > Sent: Monday, April 27, 2020 5:19 AM > > To: qemu-devel > > Cc: Zhang, Chen ; Li Zhijian > > ; Jason Wang ; Marc- > > André Lu

[PATCH v3 4/6] net/colo-compare.c: Only hexdump packets if tracing is enabled

2020-04-26 Thread Lukas Straub
Else the log will be flooded if there is a lot of network traffic. Signed-off-by: Lukas Straub --- net/colo-compare.c | 10 ++ 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/net/colo-compare.c b/net/colo-compare.c index ff6a740284..6634911770 100644 --- a/net/colo

[PATCH v3 6/6] net/colo-compare.c: Correct ordering in complete and finalize

2020-04-26 Thread Lukas Straub
In colo_compare_complete, insert CompareState into net_compares only after everything has been initialized. In colo_compare_finalize, remove CompareState from net_compares before anything is deinitialized. Signed-off-by: Lukas Straub --- net/colo-compare.c | 47

[PATCH v3 3/6] net/colo-compare.c: Fix deadlock in compare_chr_send

2020-04-26 Thread Lukas Straub
full. This leads to a deadlock because both event loops get blocked. Fix this by converting compare_chr_send to a coroutine and putting the packets in a send queue. Also create a new function notify_chr_send, since that should be independend. Signed-off-by: Lukas Straub --- net/colo-compare.c

[PATCH v3 5/6] net/colo-compare.c, softmmu/vl.c: Check that colo-compare is active

2020-04-26 Thread Lukas Straub
iodic mode) and to use multiple colo-compare for multiple network interfaces. Signed-off-by: Lukas Straub --- net/colo-compare.c | 35 +-- net/colo-compare.h | 1 + softmmu/vl.c | 2 ++ 3 files changed, 32 insertions(+), 6 deletions(-) diff --git a/net

<    1   2   3   4   5   6   7   >