Re: [Qemu-devel] [PATCH] docs/block-replication.txt: Add more detail about replication_do_checkpoint_all

2018-11-02 Thread Zhang Chen
Hi All, Maybe we have forgot this patch? Ping again. Related discussion: https://lists.gnu.org/archive/html/qemu-devel/2018-08/msg03352.html Thanks Zhang Chen On Fri, Sep 14, 2018 at 2:32 PM Zhang Chen wrote: > Hi All, > > No news update? > Ping... > > Thanks > Zhang C

[Qemu-devel] [PATCH V2] migration/colo.c: Fix compilation issue when disable replication

2018-10-31 Thread Zhang Chen
This compilation issue will occur when user use --disable-replication to config Qemu. Reported-by: Thomas Huth Signed-off-by: Zhang Chen --- migration/colo.c | 28 +++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/migration/colo.c b/migration/colo.c

Re: [Qemu-devel] [PATCH] migration/colo.c: Fix compilation issue when disable replication

2018-10-31 Thread Zhang Chen
On Wed, Oct 31, 2018 at 3:54 AM Dr. David Alan Gilbert wrote: > * Zhang Chen (zhangc...@gmail.com) wrote: > > This compilation issue will occur when user use --disable-replication > > to config Qemu. > > > > Reported-by: Thomas Huth > > Signed-off-by: Zhang

Re: [Qemu-devel] [PATCH] migration/colo.c: Fix compilation issue when disable replication

2018-10-31 Thread Zhang Chen
On Thu, Nov 1, 2018 at 12:47 AM Dr. David Alan Gilbert wrote: > * Dr. David Alan Gilbert (dgilb...@redhat.com) wrote: > > * Zhang Chen (zhangc...@gmail.com) wrote: > > > This compilation issue will occur when user use --disable-replication > > > to config Qemu. >

[Qemu-devel] [PATCH V2] net/filter-rewriter.c: Fix coverity static analysis issue

2018-10-30 Thread Zhang Chen
The original code just follow the TCP state diagram, but in this case, we can skip the TCPS_TIME_WAIT state to simplify the implementation. Signed-off-by: Zhang Chen --- net/filter-rewriter.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/net/filter-rewriter.c b/net

Re: [Qemu-devel] [PATCH] net/filter-rewirter.c: Fix coverity static analysis issue

2018-10-30 Thread Zhang Chen
Oh, Sorry. I will send the V2 patch. Thank Zhang Chen On Wed, Oct 31, 2018 at 12:18 AM Peter Maydell wrote: > On 30 October 2018 at 03:50, Zhang Chen wrote: > > The original code just follow the TCP state diagram, > > but in this case, we can skip the TCPS_TIME_WAIT state to s

[Qemu-devel] [PATCH] net/filter-rewirter.c: Fix coverity static analysis issue

2018-10-29 Thread Zhang Chen
The original code just follow the TCP state diagram, but in this case, we can skip the TCPS_TIME_WAIT state to simplify the implementation. Signed-off-by: Zhang Chen --- net/filter-rewriter.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/net/filter-rewriter.c b/net

Re: [Qemu-devel] [PULL V2 01/26] filter-rewriter: Add TCP state machine and fix memory leak in connection_track_table

2018-10-29 Thread Zhang Chen
On Tue, Oct 30, 2018 at 10:02 AM Jason Wang wrote: > > On 2018/10/29 下午7:01, Peter Maydell wrote: > > On 19 October 2018 at 04:22, Jason Wang wrote: > >> From: Zhang Chen > >> > >> We add almost full TCP state machine in filter-rewriter, except > >

[Qemu-devel] [PATCH] migration/colo.c: Fix compilation issue when disable replication

2018-10-24 Thread Zhang Chen
This compilation issue will occur when user use --disable-replication to config Qemu. Reported-by: Thomas Huth Signed-off-by: Zhang Chen --- migration/colo.c | 18 +- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/migration/colo.c b/migration/colo.c index

Re: [Qemu-devel] [PATCH V12 05/19] COLO: Add block replication into colo process

2018-10-24 Thread Zhang Chen
On Wed, Oct 24, 2018 at 3:21 PM Thomas Huth wrote: > On 2018-09-03 05:38, Zhang Chen wrote: > > Make sure master start block replication after slave's block > > replication started. > > > > Besides, we need to activate VM's blocks before goes into

[Qemu-devel] [PATCH] qapi: Fix COLOStatus and query-colo-status since version

2018-10-22 Thread Zhang Chen
This structure and command have missed qemu version 3.0, so fix it to since version 3.1. Signed-off-by: Zhang Chen --- qapi/migration.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/qapi/migration.json b/qapi/migration.json index 0928f4b727..38d4c41d88 100644 --- a

Re: [Qemu-devel] [PATCH] docs/block-replication.txt: Add more detail about replication_do_checkpoint_all

2018-09-13 Thread Zhang Chen
Hi All, No news update? Ping... Thanks Zhang Chen On Thu, Sep 6, 2018 at 12:12 AM Zhang Chen wrote: > Add more detail description for COLO checkpoint use case. > Suggested by Dr. David Alan Gilbert > > Signed-off-by: Zhang Chen > --- > docs/block-replication.txt | 7

[Qemu-devel] [PATCH V12-fix-V2 01/19] filter-rewriter: Add TCP state machine and fix memory leak in connection_track_table

2018-09-13 Thread Zhang Chen
Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- net/colo.c| 2 +- net/colo.h| 9 ++-- net/filter-rewriter.c | 109 ++ 3 files changed, 104 insertions(+), 16 deletions(-) diff --git a/net/colo

Re: [Qemu-devel] [PATCH V12-fix 01/19] filter-rewriter: Add TCP state machine and fix memory leak in connection_track_table

2018-09-13 Thread Zhang Chen
On Thu, Sep 13, 2018 at 10:49 PM Eric Blake wrote: > On 9/12/18 11:52 PM, Zhang Chen wrote: > > We add almost full TCP state machine in filter-rewriter, except > > TCPS_LISTEN and some simplify in VM active close FIN states. > > The reason for this simplify job is because gu

Re: [Qemu-devel] [PATCH V12 00/19] COLO: integrate colo frame with block replication and COLO proxy

2018-09-12 Thread Zhang Chen
On Thu, Sep 13, 2018 at 11:10 AM Zhang Chen wrote: > > > On Wed, Sep 12, 2018 at 3:50 PM Jason Wang wrote: > >> >> >> On 2018年09月10日 16:16, Zhang Chen wrote: >> > Hi All. >> > Have any comments? >> > Ping... >> > >> > T

[Qemu-devel] [PATCH V12-fix 01/19] filter-rewriter: Add TCP state machine and fix memory leak in connection_track_table

2018-09-12 Thread Zhang Chen
Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- net/colo.c| 2 +- net/colo.h| 9 ++-- net/filter-rewriter.c | 109 ++ 3 files changed, 104 insertions(+), 16 deletions(-) diff --git a/net/colo

Re: [Qemu-devel] [PATCH V12 01/19] filter-rewriter: Add TCP state machine and fix memory leak in connection_track_table

2018-09-12 Thread Zhang Chen
On Wed, Sep 12, 2018 at 3:36 PM Jason Wang wrote: > > > On 2018年09月03日 12:38, Zhang Chen wrote: > > We add almost full TCP state machine in filter-rewriter, except > > TCPS_LISTEN and some simplify in VM active close FIN states. > > Need to explain why it can be sim

Re: [Qemu-devel] [PATCH V12 00/19] COLO: integrate colo frame with block replication and COLO proxy

2018-09-12 Thread Zhang Chen
On Wed, Sep 12, 2018 at 3:50 PM Jason Wang wrote: > > > On 2018年09月10日 16:16, Zhang Chen wrote: > > Hi All. > > Have any comments? > > Ping... > > > > Thanks > > Zhang Chen > > I've queued them with some tweaks on the commit log. > >

Re: [Qemu-devel] [PATCH V12 00/19] COLO: integrate colo frame with block replication and COLO proxy

2018-09-10 Thread Zhang Chen
Hi All. Have any comments? Ping... Thanks Zhang Chen On Mon, Sep 3, 2018 at 12:39 PM Zhang Chen wrote: > Hi~ All~ > > COLO Frame, block replication and COLO proxy(colo-compare,filter-mirror, > filter-redirector,filter-rewriter) have been exist in qemu > for long time, it'

[Qemu-devel] [PATCH] docs/block-replication.txt: Add more detail about replication_do_checkpoint_all

2018-09-05 Thread Zhang Chen
Add more detail description for COLO checkpoint use case. Suggested by Dr. David Alan Gilbert Signed-off-by: Zhang Chen --- docs/block-replication.txt | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/block-replication.txt b/docs/block-replication.txt index

Re: [Qemu-devel] [PATCH V12 05/19] COLO: Add block replication into colo process

2018-09-03 Thread Zhang Chen
Sorry, forgot to add the reviewed by: Reviewed-by: Dr. David Alan Gilbert Thanks Zhang Chen On Mon, Sep 3, 2018 at 12:40 PM Zhang Chen wrote: > Make sure master start block replication after slave's block > replication started. > > Besides, we need to activate VM's b

[Qemu-devel] [PATCH V12 17/19] COLO: notify net filters about checkpoint/failover event

2018-09-02 Thread Zhang Chen
From: zhanghailiang Notify all net filters about the checkpoint and failover event. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 15 +++ 1 file changed, 15 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index 25d279decf..

[Qemu-devel] [PATCH V12 16/19] filter-rewriter: handle checkpoint and failover event

2018-09-02 Thread Zhang Chen
ff-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- net/colo-compare.c| 12 - net/colo.c| 8 ++ net/colo.h| 2 ++ net/filter-rewriter.c | 57 +++ 4 files changed, 73 insertions(+), 6 dele

[Qemu-devel] [PATCH V12 18/19] COLO: quick failover process by kick COLO thread

2018-09-02 Thread Zhang Chen
From: zhanghailiang COLO thread may sleep at qemu_sem_wait(&s->colo_checkpoint_sem), while failover works begin, It's better to wakeup it to quick the process. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 8 1 file changed, 8 insertions(+)

[Qemu-devel] [PATCH V12 15/19] filter: Add handle_event method for NetFilterClass

2018-09-02 Thread Zhang Chen
Filter needs to process the event of checkpoint/failover or other event passed by COLO frame. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- include/net/filter.h | 5 + net/filter.c | 17 + net/net.c| 19

[Qemu-devel] [PATCH V12 13/19] savevm: split the process of different stages for loadvm/savevm

2018-09-02 Thread Zhang Chen
) and qemu_save_device_state() public, and simplify the codes of qemu_save_device_state() by calling the wrapper qemu_savevm_state_header(). Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c

[Qemu-devel] [PATCH V12 14/19] COLO: flush host dirty ram from cache

2018-09-02 Thread Zhang Chen
From: zhanghailiang Don't need to flush all VM's ram from cache, only flush the dirty pages since last checkpoint Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- migration

[Qemu-devel] [PATCH V12 11/19] qapi/migration.json: Rename COLO unknown mode to none mode.

2018-09-02 Thread Zhang Chen
From: Zhang Chen Suggested by Markus Armbruster rename COLO unknown mode to none mode. Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Eric Blake Reviewed-by: Markus Armbruster --- migration/colo-failover.c | 2 +- migration/colo.c | 2 +- qapi/migration.json

[Qemu-devel] [PATCH V12 12/19] qapi: Add new command to query colo status

2018-09-02 Thread Zhang Chen
Libvirt or other high level software can use this command query colo status. You can test this command like that: {'execute':'query-colo-status'} Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- migration/colo.c| 21 + qapi

[Qemu-devel] [PATCH V12 19/19] docs: Add COLO status diagram to COLO-FT.txt

2018-09-02 Thread Zhang Chen
From: Zhang Chen This diagram make user better understand COLO. Suggested by Markus Armbruster. Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- docs/COLO-FT.txt | 34 ++ 1 file changed, 34 insertions(+) diff --git a/docs/COLO-FT.txt b/docs/COLO-FT.txt

[Qemu-devel] [PATCH V12 10/19] qmp event: Add COLO_EXIT event to notify users while exited COLO

2018-09-02 Thread Zhang Chen
x27;s failover verdict, it is still necessary to notify users that we exited COLO mode. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Eric Blake Reviewed-by: Markus Armbruster --- migration/colo.c| 31 +

[Qemu-devel] [PATCH V12 06/19] COLO: Remove colo_state migration struct

2018-09-02 Thread Zhang Chen
: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- include/migration/colo.h | 5 +-- migration/Makefile.objs | 2 +- migration/colo-comm.c| 76 migration/colo.c | 13 ++- migration

[Qemu-devel] [PATCH V12 09/19] COLO: Flush memory data from ram cache

2018-09-02 Thread Zhang Chen
During the time of VM's running, PVM may dirty some pages, we will transfer PVM's dirty pages to SVM and store them into SVM's RAM cache at next checkpoint time. So, the content of SVM's RAM cache will always be same with PVM's memory after checkpoint. Instead of flushing all content of PVM's RAM

[Qemu-devel] [PATCH V12 05/19] COLO: Add block replication into colo process

2018-09-02 Thread Zhang Chen
Make sure master start block replication after slave's block replication started. Besides, we need to activate VM's blocks before goes into COLO state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- migration/colo.c

[Qemu-devel] [PATCH V12 08/19] ram/COLO: Record the dirty pages that SVM received

2018-09-02 Thread Zhang Chen
the dirty pages caused by PVM and SVM, we only flush those dirty pages from RAM cache while do checkpoint. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 43 --- 1

[Qemu-devel] [PATCH V12 04/19] COLO: integrate colo compare with colo frame

2018-09-02 Thread Zhang Chen
-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 42 -- migration/migration.c | 6 ++ 2 files changed, 42 insertions(+), 6 deletions(-) diff --git a/migration

[Qemu-devel] [PATCH V12 03/19] colo-compare: use notifier to notify packets comparing result

2018-09-02 Thread Zhang Chen
It's a good idea to use notifier to notify COLO frame of inconsistent packets comparing. Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Signed-off-by: zhanghailiang --- net/colo-compare.c | 37 ++--- net/colo-compare.h | 2 ++ 2 files change

[Qemu-devel] [PATCH V12 07/19] COLO: Load dirty pages into SVM's RAM cache firstly

2018-09-02 Thread Zhang Chen
7;s state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- include/exec/ram_addr.h | 1 + migration/migration.c | 7 migration/ram.c | 83 - mig

[Qemu-devel] [PATCH V12 02/19] colo-compare: implement the process of checkpoint

2018-09-02 Thread Zhang Chen
While do checkpoint, we need to flush all the unhandled packets, By using the filter notifier mechanism, we can easily to notify every compare object to do this process, which runs inside of compare threads as a coroutine. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by

[Qemu-devel] [PATCH V12 01/19] filter-rewriter: Add TCP state machine and fix memory leak in connection_track_table

2018-09-02 Thread Zhang Chen
connection, if it is closed, its related resources will be cleared up. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- net/colo.c| 2 +- net/colo.h| 9 ++-- net/filter-rewriter.c | 105 ++---

[Qemu-devel] [PATCH V12 00/19] COLO: integrate colo frame with block replication and COLO proxy

2018-09-02 Thread Zhang Chen
h segmentation fault. V7: - Addressed Markus's comments in 11/17. - Rebased on upstream. V6: - Addressed Eric Blake's comments, use the enum to feedback in patch 11/17. - Fixed QAPI command separator problem in patch 11/17. Zhang Chen (15): filter-rewriter: Add TCP state machine a

Re: [Qemu-devel] [PATCH V11 16/20] filter: Add handle_event method for NetFilterClass

2018-08-29 Thread Zhang Chen
On Tue, Aug 28, 2018 at 3:19 PM Jason Wang wrote: > > > On 2018年08月24日 13:57, Zhang Chen wrote: > > On Wed, Aug 22, 2018 at 4:22 PM Jason Wang wrote: > > > >> On 2018年08月21日 17:25, Zhang Chen wrote: > >>> On Tue, Aug 21, 2018 at 11:30 AM Jason Wang

Re: [Qemu-devel] [PATCH V11 16/20] filter: Add handle_event method for NetFilterClass

2018-08-23 Thread Zhang Chen
On Wed, Aug 22, 2018 at 4:22 PM Jason Wang wrote: > > > On 2018年08月21日 17:25, Zhang Chen wrote: > > On Tue, Aug 21, 2018 at 11:30 AM Jason Wang wrote: > > > >> > >> On 2018年08月12日 04:59, Zhang Chen wrote: > >>> Filter needs to process the event

Re: [Qemu-devel] [PATCH V11 15/20] net/net.c: Add net client type check function for COLO

2018-08-23 Thread Zhang Chen
On Wed, Aug 22, 2018 at 4:22 PM Jason Wang wrote: > > > On 2018年08月21日 17:26, Zhang Chen wrote: > > > > > > On Tue, Aug 21, 2018 at 11:27 AM Jason Wang > <mailto:jasow...@redhat.com>> wrote: > > > > > > > > On

Re: [Qemu-devel] [PATCH V11 17/20] filter-rewriter: handle checkpoint and failover event

2018-08-21 Thread Zhang Chen
On Tue, Aug 21, 2018 at 11:40 AM Jason Wang wrote: > > > On 2018年08月12日 04:59, Zhang Chen wrote: > > After one round of checkpoint, the states between PVM and SVM > > become consistent, so it is unnecessary to adjust the sequence > > of net packets for old connectio

Re: [Qemu-devel] [PATCH V11 15/20] net/net.c: Add net client type check function for COLO

2018-08-21 Thread Zhang Chen
On Tue, Aug 21, 2018 at 11:27 AM Jason Wang wrote: > > > On 2018年08月12日 04:59, Zhang Chen wrote: > > From: Zhang Chen > > > > We add is_colo_support_client_type() to check the net client type for > > COLO-compare. Currently we just support TAP. > > Su

Re: [Qemu-devel] [PATCH V11 16/20] filter: Add handle_event method for NetFilterClass

2018-08-21 Thread Zhang Chen
On Tue, Aug 21, 2018 at 11:30 AM Jason Wang wrote: > > > On 2018年08月12日 04:59, Zhang Chen wrote: > > Filter needs to process the event of checkpoint/failover or > > other event passed by COLO frame. > > > > Signed-off-by: zhanghailiang > > --- > >

Re: [Qemu-devel] [PATCH V11 00/20] COLO: integrate colo frame with block replication and COLO proxy

2018-08-20 Thread Zhang Chen
Hi Jason and Markus, Ping... Do you have time to review this series? Thanks Zhang Chen On Sun, Aug 12, 2018 at 4:59 AM Zhang Chen wrote: > Hi~ All~ > > COLO Frame, block replication and COLO proxy(colo-compare,filter-mirror, > filter-redirector,filter-rewriter) have been exist i

[Qemu-devel] [PATCH V11 19/20] COLO: quick failover process by kick COLO thread

2018-08-11 Thread Zhang Chen
From: zhanghailiang COLO thread may sleep at qemu_sem_wait(&s->colo_checkpoint_sem), while failover works begin, It's better to wakeup it to quick the process. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 8 1 file changed, 8 insertions(+)

[Qemu-devel] [PATCH V11 20/20] docs: Add COLO status diagram to COLO-FT.txt

2018-08-11 Thread Zhang Chen
From: Zhang Chen This diagram make user better understand COLO. Suggested by Markus Armbruster. Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- docs/COLO-FT.txt | 34 ++ 1 file changed, 34 insertions(+) diff --git a/docs/COLO-FT.txt b/docs/COLO-FT.txt

[Qemu-devel] [PATCH V11 18/20] COLO: notify net filters about checkpoint/failover event

2018-08-11 Thread Zhang Chen
From: zhanghailiang Notify all net filters about the checkpoint and failover event. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 15 +++ 1 file changed, 15 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index 688d6f40b2..

[Qemu-devel] [PATCH V11 16/20] filter: Add handle_event method for NetFilterClass

2018-08-11 Thread Zhang Chen
Filter needs to process the event of checkpoint/failover or other event passed by COLO frame. Signed-off-by: zhanghailiang --- include/net/filter.h | 5 + net/filter.c | 17 + net/net.c| 28 3 files changed, 50 insertions(+)

[Qemu-devel] [PATCH V11 17/20] filter-rewriter: handle checkpoint and failover event

2018-08-11 Thread Zhang Chen
ff-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- net/colo-compare.c| 12 - net/colo.c| 8 ++ net/colo.h| 2 ++ net/filter-rewriter.c | 57 +++ 4 files changed, 73 insertions(+), 6 dele

[Qemu-devel] [PATCH V11 12/20] qapi: Add new command to query colo status

2018-08-11 Thread Zhang Chen
Libvirt or other high level software can use this command query colo status. You can test this command like that: {'execute':'query-colo-status'} Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- migration/colo.c| 21 + qapi

[Qemu-devel] [PATCH V11 14/20] COLO: flush host dirty ram from cache

2018-08-11 Thread Zhang Chen
From: zhanghailiang Don't need to flush all VM's ram from cache, only flush the dirty pages since last checkpoint Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- migration

[Qemu-devel] [PATCH V11 08/20] ram/COLO: Record the dirty pages that SVM received

2018-08-11 Thread Zhang Chen
the dirty pages caused by PVM and SVM, we only flush those dirty pages from RAM cache while do checkpoint. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 43 --- 1

[Qemu-devel] [PATCH V11 15/20] net/net.c: Add net client type check function for COLO

2018-08-11 Thread Zhang Chen
From: Zhang Chen We add is_colo_support_client_type() to check the net client type for COLO-compare. Currently we just support TAP. Suggested by Jason. Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- include/net/net.h | 1 + net/colo-compare.c | 5 + net/net.c | 14

[Qemu-devel] [PATCH V11 11/20] qapi/migration.json: Rename COLO unknown mode to none mode.

2018-08-11 Thread Zhang Chen
From: Zhang Chen Suggested by Markus Armbruster rename COLO unknown mode to none mode. Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Eric Blake Reviewed-by: Markus Armbruster --- migration/colo-failover.c | 2 +- migration/colo.c | 2 +- qapi/migration.json

[Qemu-devel] [PATCH V11 07/20] COLO: Load dirty pages into SVM's RAM cache firstly

2018-08-11 Thread Zhang Chen
7;s state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- include/exec/ram_addr.h | 1 + migration/migration.c | 6 +++ migration/ram.c | 83 - mig

[Qemu-devel] [PATCH V11 13/20] savevm: split the process of different stages for loadvm/savevm

2018-08-11 Thread Zhang Chen
) and qemu_save_device_state() public, and simplify the codes of qemu_save_device_state() by calling the wrapper qemu_savevm_state_header(). Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c

[Qemu-devel] [PATCH V11 10/20] qmp event: Add COLO_EXIT event to notify users while exited COLO

2018-08-11 Thread Zhang Chen
x27;s failover verdict, it is still necessary to notify users that we exited COLO mode. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Eric Blake Reviewed-by: Markus Armbruster --- migration/colo.c| 31 +

[Qemu-devel] [PATCH V11 09/20] COLO: Flush memory data from ram cache

2018-08-11 Thread Zhang Chen
During the time of VM's running, PVM may dirty some pages, we will transfer PVM's dirty pages to SVM and store them into SVM's RAM cache at next checkpoint time. So, the content of SVM's RAM cache will always be same with PVM's memory after checkpoint. Instead of flushing all content of PVM's RAM

[Qemu-devel] [PATCH V11 01/20] filter-rewriter: Add TCP state machine and fix memory leak in connection_track_table

2018-08-11 Thread Zhang Chen
connection, if it is closed, its related resources will be cleared up. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- net/colo.c| 2 +- net/colo.h| 9 ++-- net/filter-rewriter.c | 105 ++---

[Qemu-devel] [PATCH V11 06/20] COLO: Remove colo_state migration struct

2018-08-11 Thread Zhang Chen
: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- include/migration/colo.h | 5 +-- migration/Makefile.objs | 2 +- migration/colo-comm.c| 76 migration/colo.c | 13 ++- migration

[Qemu-devel] [PATCH V11 04/20] COLO: integrate colo compare with colo frame

2018-08-11 Thread Zhang Chen
-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 42 -- migration/migration.c | 6 ++ 2 files changed, 42 insertions(+), 6 deletions(-) diff --git a/migration

[Qemu-devel] [PATCH V11 05/20] COLO: Add block replication into colo process

2018-08-11 Thread Zhang Chen
Make sure master start block replication after slave's block replication started. Besides, we need to activate VM's blocks before goes into COLO state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- migration/colo.c

[Qemu-devel] [PATCH V11 02/20] colo-compare: implement the process of checkpoint

2018-08-11 Thread Zhang Chen
While do checkpoint, we need to flush all the unhandled packets, By using the filter notifier mechanism, we can easily to notify every compare object to do this process, which runs inside of compare threads as a coroutine. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by

[Qemu-devel] [PATCH V11 03/20] colo-compare: use notifier to notify packets comparing result

2018-08-11 Thread Zhang Chen
It's a good idea to use notifier to notify COLO frame of inconsistent packets comparing. Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Signed-off-by: zhanghailiang --- net/colo-compare.c | 37 ++--- net/colo-compare.h | 2 ++ 2 files change

[Qemu-devel] [PATCH V11 00/20] COLO: integrate colo frame with block replication and COLO proxy

2018-08-11 Thread Zhang Chen
read. - Fixed the bug that in some status COLO vm crash with segmentation fault. V7: - Addressed Markus's comments in 11/17. - Rebased on upstream. V6: - Addressed Eric Blake's comments, use the enum to feedback in patch 11/17. - Fixed QAPI command separator problem in p

Re: [Qemu-devel] [PATCH V10 08/20] ram/COLO: Record the dirty pages that SVM received

2018-08-11 Thread Zhang Chen
On Wed, Aug 8, 2018 at 2:44 AM Dr. David Alan Gilbert wrote: > * Zhang Chen (zhangc...@gmail.com) wrote: > > On Sat, Jul 28, 2018 at 2:51 AM, Dr. David Alan Gilbert < > dgilb...@redhat.com > > > wrote: > > > > > * Zhang Chen (zhangc...@gmail.com) wrote: >

Re: [Qemu-devel] [PATCH V10 05/20] COLO: Add block replication into colo process

2018-08-11 Thread Zhang Chen
On Tue, Aug 7, 2018 at 10:30 PM Dr. David Alan Gilbert wrote: > * Zhang Chen (zhangc...@gmail.com) wrote: > > Make sure master start block replication after slave's block > > replication started. > > > > Besides, we need to activate VM's blocks before goes in

Re: [Qemu-devel] [Bug 1754542] Re: colo: vm crash with segmentation fault

2018-08-07 Thread Zhang Chen
Currently, we support virtio-net, and not support all vhost-xxx. On Fri, Jul 27, 2018 at 6:41 PM WANG Chao wrote: > Yes, ide works. > > And by the way, how about other virtio devices or vhost-xxx? Are they > supported by COLO? > > Do you know the working set of devices? My preliminary test show

Re: [Qemu-devel] [PATCH V10 08/20] ram/COLO: Record the dirty pages that SVM received

2018-08-01 Thread Zhang Chen
On Sat, Jul 28, 2018 at 2:51 AM, Dr. David Alan Gilbert wrote: > * Zhang Chen (zhangc...@gmail.com) wrote: > > We record the address of the dirty pages that received, > > it will help flushing pages that cached into SVM. > > > > Here, it is a trick, we record dirty

Re: [Qemu-devel] [Bug 1754542] Re: colo: vm crash with segmentation fault

2018-07-27 Thread Zhang Chen
Hi Chao, Yes, virtio blk isn't supported by current COLO, you can try the "-drive if=ide ". Thanks Zhang Chen On Fri, Jul 27, 2018 at 12:53 PM, WANG Chao wrote: > Hi, Zhang Chen > > It seems virtio blk isn't working. > > I test coloft against htt

Re: [Qemu-devel] [PATCH V10 10/20] qmp event: Add COLO_EXIT event to notify users while exited COLO

2018-07-24 Thread Zhang Chen
On Tue, Jul 24, 2018 at 10:54 PM, Dr. David Alan Gilbert < dgilb...@redhat.com> wrote: > * Zhang Chen (zhangc...@gmail.com) wrote: > > On Tue, Jul 24, 2018 at 2:41 AM, Eric Blake wrote: > > > > > On 07/22/2018 02:33 PM, Zhang Chen wrote: > > > > >

Re: [Qemu-devel] [PATCH V10 10/20] qmp event: Add COLO_EXIT event to notify users while exited COLO

2018-07-24 Thread Zhang Chen
On Tue, Jul 24, 2018 at 2:41 AM, Eric Blake wrote: > On 07/22/2018 02:33 PM, Zhang Chen wrote: > >> From: zhanghailiang >> >> If some errors happen during VM's COLO FT stage, it's important to >> notify the users of this event. Together with 'x-co

[Qemu-devel] [PATCH V10 18/20] COLO: notify net filters about checkpoint/failover event

2018-07-22 Thread Zhang Chen
From: zhanghailiang Notify all net filters about the checkpoint and failover event. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 15 +++ 1 file changed, 15 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index 688d6f40b2..

[Qemu-devel] [PATCH V10 14/20] COLO: flush host dirty ram from cache

2018-07-22 Thread Zhang Chen
From: zhanghailiang Don't need to flush all VM's ram from cache, only flush the dirty pages since last checkpoint Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 10 ++ 1 file c

[Qemu-devel] [PATCH V10 12/20] qapi: Add new command to query colo status

2018-07-22 Thread Zhang Chen
Libvirt or other high level software can use this command query colo status. You can test this command like that: {'execute':'query-colo-status'} Signed-off-by: Zhang Chen --- migration/colo.c| 21 + qapi/migration.json | 32 +

[Qemu-devel] [PATCH V10 20/20] docs: Add COLO status diagram to COLO-FT.txt

2018-07-22 Thread Zhang Chen
From: Zhang Chen This diagram make user better understand COLO. Suggested by Markus Armbruster. Signed-off-by: Zhang Chen --- docs/COLO-FT.txt | 34 ++ 1 file changed, 34 insertions(+) diff --git a/docs/COLO-FT.txt b/docs/COLO-FT.txt index d7c7dcda8f

[Qemu-devel] [PATCH V10 08/20] ram/COLO: Record the dirty pages that SVM received

2018-07-22 Thread Zhang Chen
We record the address of the dirty pages that received, it will help flushing pages that cached into SVM. Here, it is a trick, we record dirty pages by re-using migration dirty bitmap. In the later patch, we will start the dirty log for SVM, just like migration, in this way, we can record both the

[Qemu-devel] [PATCH V10 17/20] filter-rewriter: handle checkpoint and failover event

2018-07-22 Thread Zhang Chen
ff-by: zhanghailiang Signed-off-by: Zhang Chen --- net/colo-compare.c| 12 - net/colo.c| 8 ++ net/colo.h| 2 ++ net/filter-rewriter.c | 57 +++ 4 files changed, 73 insertions(+), 6 deletions(-) diff --git a/net

[Qemu-devel] [PATCH V10 16/20] filter: Add handle_event method for NetFilterClass

2018-07-22 Thread Zhang Chen
Filter needs to process the event of checkpoint/failover or other event passed by COLO frame. Signed-off-by: zhanghailiang --- include/net/filter.h | 5 + net/filter.c | 17 + net/net.c| 28 3 files changed, 50 insertions(+)

[Qemu-devel] [PATCH V10 19/20] COLO: quick failover process by kick COLO thread

2018-07-22 Thread Zhang Chen
From: zhanghailiang COLO thread may sleep at qemu_sem_wait(&s->colo_checkpoint_sem), while failover works begin, It's better to wakeup it to quick the process. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 8 1 file changed, 8 insertions(+)

[Qemu-devel] [PATCH V10 07/20] COLO: Load dirty pages into SVM's RAM cache firstly

2018-07-22 Thread Zhang Chen
7;s state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen --- include/exec/ram_addr.h | 1 + migration/migration.c | 6 +++ migration/ram.c | 83 - migration/ram.h | 4 ++ migration/savevm.c |

[Qemu-devel] [PATCH V10 13/20] savevm: split the process of different stages for loadvm/savevm

2018-07-22 Thread Zhang Chen
) and qemu_save_device_state() public, and simplify the codes of qemu_save_device_state() by calling the wrapper qemu_savevm_state_header(). Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c

[Qemu-devel] [PATCH V10 11/20] qapi/migration.json: Rename COLO unknown mode to none mode.

2018-07-22 Thread Zhang Chen
From: Zhang Chen Suggested by Markus Armbruster rename COLO unknown mode to none mode. Signed-off-by: Zhang Chen Reviewed-by: Eric Blake Reviewed-by: Markus Armbruster --- migration/colo-failover.c | 2 +- migration/colo.c | 2 +- qapi/migration.json | 10 +- 3

[Qemu-devel] [PATCH V10 15/20] net/net.c: Add net client type check function for COLO

2018-07-22 Thread Zhang Chen
From: Zhang Chen We add is_colo_support_client_type() to check the net client type for COLO-compare. Currently we just support TAP. Suggested by Jason. Signed-off-by: Zhang Chen --- include/net/net.h | 1 + net/colo-compare.c | 5 + net/net.c | 14 ++ 3 files

[Qemu-devel] [PATCH V10 06/20] COLO: Remove colo_state migration struct

2018-07-22 Thread Zhang Chen
: zhanghailiang Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- include/migration/colo.h | 5 +-- migration/Makefile.objs | 2 +- migration/colo-comm.c| 76 migration/colo.c | 13 ++- migration/migration.c| 23

[Qemu-devel] [PATCH V10 05/20] COLO: Add block replication into colo process

2018-07-22 Thread Zhang Chen
Make sure master start block replication after slave's block replication started. Besides, we need to activate VM's blocks before goes into COLO state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen --- migration/colo.c

[Qemu-devel] [PATCH V10 09/20] COLO: Flush memory data from ram cache

2018-07-22 Thread Zhang Chen
During the time of VM's running, PVM may dirty some pages, we will transfer PVM's dirty pages to SVM and store them into SVM's RAM cache at next checkpoint time. So, the content of SVM's RAM cache will always be same with PVM's memory after checkpoint. Instead of flushing all content of PVM's RAM

[Qemu-devel] [PATCH V10 10/20] qmp event: Add COLO_EXIT event to notify users while exited COLO

2018-07-22 Thread Zhang Chen
x27;s failover verdict, it is still necessary to notify users that we exited COLO mode. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Reviewed-by: Eric Blake Reviewed-by: Markus Armbruster --- migration/colo.c| 31 ++

[Qemu-devel] [PATCH V10 02/20] colo-compare: implement the process of checkpoint

2018-07-22 Thread Zhang Chen
While do checkpoint, we need to flush all the unhandled packets, By using the filter notifier mechanism, we can easily to notify every compare object to do this process, which runs inside of compare threads as a coroutine. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen --- include

[Qemu-devel] [PATCH V10 03/20] colo-compare: use notifier to notify packets comparing result

2018-07-22 Thread Zhang Chen
It's a good idea to use notifier to notify COLO frame of inconsistent packets comparing. Signed-off-by: Zhang Chen Signed-off-by: zhanghailiang --- net/colo-compare.c | 37 ++--- net/colo-compare.h | 2 ++ 2 files changed, 28 insertions(+), 11 dele

[Qemu-devel] [PATCH V10 04/20] COLO: integrate colo compare with colo frame

2018-07-22 Thread Zhang Chen
-by: zhanghailiang Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 42 -- migration/migration.c | 6 ++ 2 files changed, 42 insertions(+), 6 deletions(-) diff --git a/migration/colo.c b/migration/colo.c index

[Qemu-devel] [PATCH V10 01/20] filter-rewriter: Add TCP state machine and fix memory leak in connection_track_table

2018-07-22 Thread Zhang Chen
connection, if it is closed, its related resources will be cleared up. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen --- net/colo.c| 2 +- net/colo.h| 9 ++-- net/filter-rewriter.c | 105 ++ 3 files changed, 100 inserti

[Qemu-devel] [PATCH V10 00/20] COLO: integrate colo frame with block replication and COLO proxy

2018-07-22 Thread Zhang Chen
in patch 11/17. - Fixed QAPI command separator problem in patch 11/17. Zhang Chen (16): filter-rewriter: Add TCP state machine and fix memory leak in connection_track_table colo-compare: implement the process of checkpoint colo-compare: use notifier to notify packets comparing r

Re: [Qemu-devel] [PATCH V9 12/20] qapi: Add new command to query colo status

2018-07-04 Thread Zhang Chen
On Tue, Jul 3, 2018 at 7:09 PM, Markus Armbruster wrote: > Zhang Chen writes: > > > Libvirt or other high level software can use this command query colo > status. > > You can test this command like that: > > {'execute':'query-colo-

Re: [Qemu-devel] [PATCH V9 11/20] qapi/migration.json: Rename COLO unknown mode to none mode.

2018-06-29 Thread Zhang Chen
On Fri, Jun 29, 2018 at 5:14 PM, Dr. David Alan Gilbert wrote: > * Eric Blake (ebl...@redhat.com) wrote: > > On 06/27/2018 03:41 PM, Zhang Chen wrote: > > > From: Zhang Chen > > > > > > Suggested by Markus Armbruster rename COLO unknown mode to none mode. &

[Qemu-devel] [PATCH V9 20/20] docs: Add COLO status diagram to COLO-FT.txt

2018-06-27 Thread Zhang Chen
From: Zhang Chen This diagram make user better understand COLO. Suggested by Markus Armbruster. Signed-off-by: Zhang Chen --- docs/COLO-FT.txt | 34 ++ 1 file changed, 34 insertions(+) diff --git a/docs/COLO-FT.txt b/docs/COLO-FT.txt index e289be2f41

<    3   4   5   6   7   8   9   10   11   12   >