Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo cache

2020-09-21 Thread Derek Su
Hi, Chen and Lei Using Lei's patch is OK to me. Please help to add "Signed-off-by: Derek Su " for merging it. Thank you. :) Regards Derek Zhang, Chen 於 2020年9月22日 週二 下午1:37寫道: > So, Derek, you will send new version patch? > > > > Thanks > > Zhang Chen &g

Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo cache

2020-09-21 Thread Derek Su
Hi, Lei Got it. Thanks. Regards, Derek Rao, Lei 於 2020年9月22日 週二 下午1:04寫道: > Hi, Derek and Chen > > ram_bulk_stage is false by default before Hailiang's patch. > For COLO, it does not seem to be used, so I think there is no need to > reset it to true. > > Thanks, >

Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo cache

2020-09-21 Thread Derek Su
with Lei’ patch: > > [PATCH 2/3] Reduce the time of checkpoint for COLO > > Can you discuss to merge it into one patch? > > > > Thanks > > Zhang Chen > > > > *From:* Derek Su > *Sent:* Tuesday, September 22, 2020 11:31 AM > *To:* qemu-devel >

Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo cache

2020-09-21 Thread Derek Su
Hello, all Ping... Regards, Derek Su Derek Su 於 2020年9月10日 週四 下午6:47寫道: > In secondary side, the colo_flush_ram_cache() calls > migration_bitmap_find_dirty() to finding the dirty pages and > flush them to host. But ram_state's ram_bulk_stage flag is always > enabled in se

Re: [PATCH v1 2/2] colo-compare: Record packet creation time by QEMU_CLOCK_REALTIME

2020-09-14 Thread Derek Su
as you said. > And use the QEMU_CLOCK_VIRTUAL is not correct, because we need to know > time changes during VM stop. > Thanks. Regards, Derek > Thanks > > Zhang Chen > > > > *From:* Derek Su > *Sent:* Tuesday, September 15, 2020 12:24 AM > *To:* Zhang, Chen >

Re: [PATCH v1 2/2] colo-compare: Record packet creation time by QEMU_CLOCK_REALTIME

2020-09-14 Thread Derek Su
, Chen 於 2020年9月14日 週一 下午3:42寫道: > > > > > *From:* Derek Su > *Sent:* Monday, September 14, 2020 9:00 AM > *To:* Zhang, Chen > *Cc:* jasow...@redhat.com; lizhij...@cn.fujitsu.com; qemu-devel@nongnu.org > *Subject:* Re: [PATCH v1 2/2] colo-compare: Record packet creation ti

Re: [PATCH v1 2/2] colo-compare: Record packet creation time by QEMU_CLOCK_REALTIME

2020-09-13 Thread Derek Su
Zhang, Chen 於 2020年9月14日 週一,上午4:06寫道: > > > > > > -Original Message- > > > From: Zhang, Chen > > > Sent: Monday, September 14, 2020 4:02 AM > > > To: 'Derek Su' ; qemu-devel@nongnu.org > > > Cc: lizhij...@cn.fujitsu.com; jasow...@red

Re: [PATCH v1 1/2] colo-compare: Fix incorrect data type conversion

2020-09-13 Thread Derek Su
Hi, Chen Got it, thank you :) Regards, Derek Zhang, Chen 於 2020年9月14日 週一,上午4:02寫道: > > > > > > -Original Message- > > > From: Derek Su > > > Sent: Saturday, September 12, 2020 3:05 AM > > > To: qemu-devel@nongnu.org > > > C

[PATCH v1 1/2] colo-compare: Fix incorrect data type conversion

2020-09-11 Thread Derek Su
Fix data type conversion of compare_timeout. The incorrect conversion results in a random compare_timeout value and unexpected stalls in packet comparison. Signed-off-by: Derek Su --- net/colo-compare.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/net/colo-compare.c

[PATCH v1 2/2] colo-compare: Record packet creation time by QEMU_CLOCK_REALTIME

2020-09-11 Thread Derek Su
Record packet creation time by QEMU_CLOCK_REALTIME instead of QEMU_CLOCK_HOST. The time difference between `now` and packet `creation_ms` has the possibility of an unexpected negative value and results in wrong comparison after changing the host clock. Signed-off-by: Derek Su --- net/colo

[PATCH v1 0/2] colo-compare bugfixes

2020-09-11 Thread Derek Su
Hello, The fixes are for the bugs found in colo-compare during our testing and applications. Please help to review, thanks a lot. Regards, Derek Su Derek Su (2): colo-compare: Fix incorrect data type conversion colo-compare: Record packet creation time by QEMU_CLOCK_REALTIME net/colo

[PATCH v1 0/1] COLO: only flush dirty ram pages from colo cache

2020-09-10 Thread Derek Su
to 10 ms averagely. Please help to review and give comments, thanks a lot! Derek Su (1): COLO: only flush dirty ram pages from colo cache migration/colo.c | 6 +- migration/ram.c | 10 ++ migration/ram.h | 3 +++ 3 files changed, 18 insertions(+), 1 deletion(-) -- 2.25.1

[PATCH v1 1/1] COLO: only flush dirty ram pages from colo cache

2020-09-10 Thread Derek Su
in secondary side is disabled in the preparation of COLO incoming process to avoid the whole dirty ram pages flush. Signed-off-by: Derek Su --- migration/colo.c | 6 +- migration/ram.c | 10 ++ migration/ram.h | 3 +++ 3 files changed, 18 insertions(+), 1 deletion(-) diff --git

[Bug 1894818] Re: COLO's guest VNC client hang after failover

2020-09-08 Thread Derek Su
Hi, I also tested some emulated nic devices and virtio network devices (in the attachment). The VNC client's screen cannot be recovered while using all virtio network devices and the emulated e1000e nic. Thanks. Regards, Derek ** Attachment added: "截圖 2020-09-09 上午10.39.09.png"

[Bug 1894818] Re: COLO's guest VNC client hang after failover

2020-09-08 Thread Derek Su
** Changed in: qemu Status: Invalid => New -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1894818 Title: COLO's guest VNC client hang after failover Status in QEMU: New Bug description:

[Bug 1894818] Re: COLO's guest VNC client hang after failover

2020-09-08 Thread Derek Su
Hi, Lukas After fixing the misuse of AWD. There is still a high probability of the VNC/RDP client hangs after PVM died and SVM takeover. Here are the steps. 1. Start PVM script ``` imagefolder="/mnt/nfs2/vms" qemu-system-x86_64 -enable-kvm -cpu qemu64,+kvmclock -m 4096 -smp 2 -qmp stdio \

Re: [Bug 1894818] [NEW] COLO's guest VNC client hang after failover

2020-09-08 Thread Derek Su
racker <1894...@bugs.launchpad.net> wrote: > > > You have been subscribed to a public bug by Derek Su (dereksu): > > > > Hello, > > > > After setting up COLO's primary and secondary VMs, > > I installed the vncserver and xrdp (apt install tightvncserver xrdp) > in

[Bug 1894818] Re: COLO's guest VNC client hang after failover

2020-09-08 Thread Derek Su
** Description changed: Hello, After setting up COLO's primary and secondary VMs, I installed the vncserver and xrdp (apt install tightvncserver xrdp) inside the VM. I access the VM from another PC via VNC/RDP client, and everything is OK. Then, kill the primary VM and issue the

[Bug 1894818] Re: COLO's guest VNC client hang after failover

2020-09-08 Thread Derek Su
** Description changed: Hello, After setting up COLO's primary and secondary VMs, I installed the vncserver and xrdp (apt install tightvncserver xrdp) inside the VM. I access the VM from another PC via VNC/RDP client, and everything is OK. Then, kill the primary VM and issue the

[Bug 1894818] [NEW] COLO's guest VNC client hang after failover

2020-09-08 Thread Derek Su
Public bug reported: Hello, After setting up COLO's primary and secondary VMs, I installed the vncserver and xrdp (apt install tightvncserver xrdp) inside the VM. I access the VM from another PC via VNC/RDP client, and everything is OK. Then, kill the primary VM and issue the failover

Re: [PATCH v1 0/1] COLO: migrate dirty ram pages before colo checkpoint

2020-08-15 Thread Derek Su
On Sat, Aug 15, 2020 at 9:42 AM Zhanghailiang wrote: > > > -Original Message- > > From: Derek Su [mailto:jwsu1...@gmail.com] > > Sent: Thursday, August 13, 2020 6:28 PM > > To: Lukas Straub > > Cc: Derek Su ; qemu-devel@nongnu.org; Zhanghailiang > >

Re: [PATCH v1 0/1] COLO: migrate dirty ram pages before colo checkpoint

2020-08-13 Thread Derek Su
On Fri, Jul 31, 2020 at 3:52 PM Lukas Straub wrote: > > On Sun, 21 Jun 2020 10:10:03 +0800 > Derek Su wrote: > > > This series is to reduce the guest's downtime during colo checkpoint > > by migrating dirty ram pages as many as possible before colo checkpoint. > &

Re: [Virtio-fs] virtio-fs performance

2020-08-04 Thread Derek Su
Vivek Goyal 於 2020年7月28日 週二 下午11:27寫道: > > On Tue, Jul 28, 2020 at 02:49:36PM +0100, Stefan Hajnoczi wrote: > > > I'm trying and testing the virtio-fs feature in QEMU v5.0.0. > > > My host and guest OS are both ubuntu 18.04 with kernel 5.4, and the > > > underlying storage is one single SSD. > >

Re: virtio-fs performance

2020-08-04 Thread Derek Su
Hello, Set the cache=none in virtiofsd and direct=1 in fio, here are the results and kvm-exit count in 5 seconds. --thread-pool-size=64 (default) seq read: 307 MB/s (kvm-exit count=1076463) seq write: 430 MB/s (kvm-exit count=1302493) rand 4KB read: 65.2k IOPS (kvm-exit

Re: [PATCH v1 1/1] migration/colo.c: migrate dirty ram pages before colo checkpoint

2020-07-13 Thread Derek Su
Hello, Ping... Anyone have comments about this path? To reduce the downtime during checkpoints, the patch tries to migrate memory page as many as possible just before entering COLO state. Thanks. Regards, Derek On 2020/6/21 上午10:10, Derek Su wrote: To reduce the guest's downtime during

Re: [PATCH v1] chardev/char-socket: fix double free of err after socket is disconnected

2020-06-24 Thread Derek Su
Oops! Sorry, I dont’t notice this patch before. Thanks. Derek Philippe Mathieu-Daudé 於 2020年6月24日 週三,下午6:12寫道: > On 6/24/20 12:00 PM, Derek Su wrote: > > The err is freed in check_report_connect_error() conditionally, > > calling error_free() directly may lead to a

[PATCH v1] chardev/char-socket: fix double free of err after socket is disconnected

2020-06-24 Thread Derek Su
The err is freed in check_report_connect_error() conditionally, calling error_free() directly may lead to a double-free bug. Signed-off-by: Derek Su --- chardev/char-socket.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/chardev/char-socket.c b/chardev/char-socket.c

[PATCH v1 0/1] COLO: migrate dirty ram pages before colo checkpoint

2020-06-20 Thread Derek Su
the total primary VM downtime is decreased by ~40%. Please help to review it and suggestions are welcomed. Thanks. Derek Su (1): migration/colo.c: migrate dirty ram pages before colo checkpoint migration/colo.c | 79 ++ migration/migration.c | 20

[PATCH v1 1/1] migration/colo.c: migrate dirty ram pages before colo checkpoint

2020-06-20 Thread Derek Su
-off-by: Derek Su --- migration/colo.c | 79 ++ migration/migration.c | 20 +++ migration/trace-events | 2 ++ monitor/hmp-cmds.c | 8 + qapi/migration.json| 18 -- 5 files changed, 125 insertions(+), 2 deletions(-) diff

Re: cluster_size got from backup_calculate_cluster_size()

2020-05-21 Thread Derek Su
On 2020/5/22 上午4:53, Vladimir Sementsov-Ogievskiy wrote: 21.05.2020 21:19, John Snow wrote: On 5/21/20 5:56 AM, Derek Su wrote: Hi, The cluster_size got from backup_calculate_cluster_size(), MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster_size), is 64K regardless of the target image's cluster

cluster_size got from backup_calculate_cluster_size()

2020-05-21 Thread Derek Su
Hi, The cluster_size got from backup_calculate_cluster_size(), MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster_size), is 64K regardless of the target image's cluster size. For example: If the cluster size of source and target qcow2 images are both 16K, the 64K from

Re: [PATCH v5 00/31] Add subcluster allocation to qcow2

2020-05-20 Thread Derek Su
Hi, Berto Excuse me, I'd like to test v5, but I failed to apply the series to master branch. Which commit can I use? Thanks. Regards, Derek On 2020/5/6 上午1:38, Alberto Garcia wrote: Hi, here's the new version of the patches to add subcluster allocation support to qcow2. Please refer to

Re: [PATCH v4 5/6] net/colo-compare.c, softmmu/vl.c: Check that colo-compare is active

2020-05-07 Thread Derek Su
On 2020/5/8 上午10:26, Zhang, Chen wrote: -Original Message- From: Lukas Straub Sent: Thursday, May 7, 2020 11:54 PM To: Zhang, Chen Cc: qemu-devel ; Li Zhijian ; Jason Wang ; Marc- André Lureau ; Paolo Bonzini Subject: Re: [PATCH v4 5/6] net/colo-compare.c, softmmu/vl.c: Check that

Re: [PATCH 1/3] net/colo-compare.c: Create event_bh with the right AioContext

2020-04-23 Thread Derek Su
b478a699d in qemu_thread_start (args=0x55cb498035d0) at util/qemu-thread-posix.c:519 #15 0x7f6e912376db in start_thread (arg=0x7f6da1ade700) at pthread_create.c:463 #16 0x7f6e90f6088f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 (gdb) ``` COLO works well after app

Re: [PATCH v4 00/30] Add subcluster allocation to qcow2

2020-04-20 Thread Derek Su
Hello, This work is promising and interesting. I'd like to try this new feature. Could you please export a branch because the patches cannot be applied to current master? Thanks. Regards, Derek On 2020/3/18 上午2:15, Alberto Garcia wrote: Hi, here's the new version of the patches to add

[PATCH v5 1/1] colo-compare: Fix memory leak in packet_enqueue()

2020-04-09 Thread Derek Su
The patch is to fix the "pkt" memory leak in packet_enqueue(). The allocated "pkt" needs to be freed if the colo compare primary or secondary queue is too big. Replace the error_report of full queue with a trace event. Signed-off-by: Derek Su --- net

[PATCH v5 0/1] COLO: handling of the full primary or secondary queue

2020-04-09 Thread Derek Su
andling of the full primary or secondary queue which hurt network throughput too much V4: - Remove redundant flush of packets V3: - handling of the full primary or secondary queue according to the suggestion from Zhang Chen V2: - Fix incorrect patch format Derek Su (1): colo-compare: F

Re: [PATCH v4 2/2] net/colo-compare.c: handling of the full primary or secondary queue

2020-04-09 Thread Derek Su
flood in log. May I also make "MAX_QUEUE_SIZE" be user-configurable in this series? Thanks, Derek Su Zhang, Chen 於 2020年4月9日 週四 下午2:59寫道: > > > > > -Original Message- > > From: Lukas Straub > > Sent: Thursday, April 9, 2020 3:19 A

[PATCH v4 2/2] net/colo-compare.c: handling of the full primary or secondary queue

2020-03-28 Thread Derek Su
checkpoint and flush packets. Signed-off-by: Derek Su --- net/colo-compare.c | 39 --- 1 file changed, 28 insertions(+), 11 deletions(-) diff --git a/net/colo-compare.c b/net/colo-compare.c index cdd87b2aa8..fe8779cf2d 100644 --- a/net/colo-compare.c +++ b/net/colo

[PATCH v4 0/2] COLO: handling of the full primary or secondary queue

2020-03-28 Thread Derek Su
remove all queued secondary packets (flush packets) and do checkpoint. Please help to review, thanks. V4: - Remove redundant flush of packets V3: - handling of the full primary or secondary queue according to the suggestion from Zhang Chen V2: - Fix incorrect patch format D

[PATCH v4 1/2] net/colo-compare.c: Fix memory leak in packet_enqueue()

2020-03-28 Thread Derek Su
The patch is to fix the "pkt" memory leak in packet_enqueue(). The allocated "pkt" needs to be freed if the colo compare primary or secondary queue is too big. Signed-off-by: Derek Su --- net/colo-compare.c | 23 +++ 1 file changed, 15 insertions(+),

Re: [PATCH v3 2/2] net/colo-compare.c: handling of the full primary or secondary queue

2020-03-27 Thread Derek Su
Lukas Straub 於 2020年3月28日 週六 上午2:28寫道: > > On Sat, 28 Mar 2020 02:20:21 +0800 > Derek Su wrote: > > > Lukas Straub 於 2020年3月28日 週六 上午1:46寫道: > > > > > > On Wed, 25 Mar 2020 17:43:54 +0800 > > > Derek Su wrote: > > > > > > > T

Re: [PATCH v3 2/2] net/colo-compare.c: handling of the full primary or secondary queue

2020-03-27 Thread Derek Su
Lukas Straub 於 2020年3月28日 週六 上午1:46寫道: > > On Wed, 25 Mar 2020 17:43:54 +0800 > Derek Su wrote: > > > The pervious handling of the full primary or queue is only dropping > > the packet. If there are lots of clients to the guest VM, > > the "drop"

[PATCH v3 2/2] net/colo-compare.c: handling of the full primary or secondary queue

2020-03-25 Thread Derek Su
all queued primary packets, remove all queued secondary packets and do checkpoint. Signed-off-by: Derek Su --- net/colo-compare.c | 41 ++--- 1 file changed, 30 insertions(+), 11 deletions(-) diff --git a/net/colo-compare.c b/net/colo-compare.c index cdd87b2aa8..

[PATCH v3 1/2] net/colo-compare.c: Fix memory leak in packet_enqueue()

2020-03-25 Thread Derek Su
The patch is to fix the "pkt" memory leak in packet_enqueue(). The allocated "pkt" needs to be freed if the colo compare primary or secondary queue is too big. Signed-off-by: Derek Su --- net/colo-compare.c | 23 +++ 1 file changed, 15 insertions(+),

[PATCH v3 0/2] COLO: handling of the full primary or secondary queue

2020-03-25 Thread Derek Su
ueued secondary packets and do checkpoint. Please review, thanks. V3: - handling of the full primary or secondary queue according to the suggestion Zhang Chen V2: - Fix incorrect patch format Derek Su (2): net/colo-compare.c: Fix memory leak in packet_enqueue() net/colo-compare.c: handling

Re: [PATCH v2 1/1] net/colo-compare.c: Fix memory leak in packet_enqueue()

2020-03-24 Thread Derek Su
> Cc: qemu-devel@nongnu.org; lizhij...@cn.fujitsu.com; > > > jasow...@redhat.com; dere...@qnap.com > > > Subject: Re: [PATCH v2 1/1] net/colo-compare.c: Fix memory leak in > > > packet_enqueue() > > > > > > Zhang, Chen 於 2020年3月24日 週二 上午3:24 > > >

[PATCH v2 0/1] COLO: Fix memory leak in packet_enqueue()

2020-03-22 Thread Derek Su
"iperf3 -s" in PVM (3) Run "iperfs -c -t 7200" The memory usage of qemu-system-x86_64 increases as the PVM's QMP shows "qemu-system-x86_64: colo compare secondary queue size too big, drop packet". Please review, thanks. V2: - Fix incorrect patch format Derek Su (1

[PATCH v2 1/1] net/colo-compare.c: Fix memory leak in packet_enqueue()

2020-03-22 Thread Derek Su
The patch is to fix the "pkt" memory leak in packet_enqueue(). The allocated "pkt" needs to be freed if the colo compare primary or secondary queue is too big. Signed-off-by: Derek Su --- net/colo-compare.c | 23 +++ 1 file changed, 15 insertions(+),

[PATCH 1/1] net/colo-compare.c: Fix memory leak in packet_enqueue()

2020-03-22 Thread Derek Su
The patch is to fix the "pkt" memory leak in packet_enqueue(). The allocated "pkt" needs to be freed if the colo compare primary or secondary queue is too big. Signed-off-by: Derek Su --- net/colo-compare.c | 23 +++ 1 file changed, 15 insertions(+),

[PATCH 0/1] COLO: Fix memory leak in packet_enqueue()

2020-03-22 Thread Derek Su
"iperf3 -s" in PVM (3) Run "iperfs -c -t 7200" The memory usage of qemu-system-x86_64 increases as the PVM's QMP shows "qemu-system-x86_64: colo compare secondary queue size too big,drop packet". Derek Su (1): net/colo-compare.c: Fix memory leak in pa

[PATCH 1/1] COLO: Fix memory leak in packet_enqueue()

2020-03-22 Thread Derek Su
The patch is to fix the "pkt" memory leak in packet_enqueue(). The allocated "pkt" needs to be freed if the colo compare primary or secondary queue is too big. Signed-off-by: Derek Su --- net/colo-compare.c | 23 +++ 1 file changed, 15 insertions(+),