[Qemu-devel] [PATCH COLO-Frame v10 23/38] qmp event: Add event notification for COLO error

2015-11-03 Thread zhanghailiang
is still necessary to notify users that we exit COLO mode. Cc: Markus Armbruster Cc: Michael Roth Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- docs/qmp-events.txt | 17 + migration/colo.c| 13 + qapi-schema.json| 16 qapi/event

[Qemu-devel] [PATCH COLO-Frame v10 06/38] migration: Integrate COLO checkpoint process into loadvm

2015-11-03 Thread zhanghailiang
_incoming_co' records the original migration incoming coroutine. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- v10: fix a bug about fd leak which is found by Dave. --- include/migration/colo.h | 7 +++ include/migration/migration.h | 7 +++ migr

[Qemu-devel] [PATCH COLO-Frame v10 02/38] migration: Introduce capability 'x-colo' to migration

2015-11-03 Thread zhanghailiang
earn if colo is supported. Cc: Juan Quintela Cc: Amit Shah Cc: Eric Blake Cc: Markus Armbruster Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei --- v10: - Rename capability 'colo' to experimental 'x-colo' (Eric's suggestion). - Rename

[Qemu-devel] [PATCH COLO-Frame v10 22/38] COLO: implement default failover treatment

2015-11-03 Thread zhanghailiang
If we detect some error in colo, we will wait for some time, hoping users also detect it. If users don't issue failover command. We will go into default failover procedure, which the PVM will takeover work while SVM is exit in default. Signed-off-by: zhanghailiang Signed-off-by: Li Zh

[Qemu-devel] [PATCH COLO-Frame v10 27/38] COLO: Process shutdown command for VM in COLO state

2015-11-03 Thread zhanghailiang
If VM is in COLO FT state, we should do some extra work before normal shutdown process. SVM will ignore the shutdown command if this command is issued directly to it, PVM will send the shutdown command to SVM if it gets this command. Cc: Paolo Bonzini Signed-off-by: zhanghailiang Signed-off-by

[Qemu-devel] [PATCH COLO-Frame v10 19/38] COLO failover: Introduce state to record failover process

2015-11-03 Thread zhanghailiang
set the value. Signed-off-by: zhanghailiang --- include/migration/failover.h | 10 ++ migration/colo-failover.c| 37 + migration/colo.c | 4 trace-events | 1 + 4 files changed, 52 insertions(+) diff --git a

[Qemu-devel] [PATCH COLO-Frame v10 28/38] COLO: Update the global runstate after going into colo state

2015-11-03 Thread zhanghailiang
If we start qemu with -S, the runstate will change from 'prelaunch' to 'running' after going into colo state. So it is necessary to update the global runstate after going into colo state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- migration/colo.c | 5

[Qemu-devel] [PATCH COLO-Frame v10 12/38] COLO: Save PVM state to secondary side when do checkpoint

2015-11-03 Thread zhanghailiang
the size of VM state, so in master, we use qsb to store VM state temporarily, get the data size by call qsb_get_length() and then migrate the data to the qsb in the secondary side. Signed-off-by: zhanghailiang Signed-off-by: Gonglei Signed-off-by: Li Zhijian --- migratio

[Qemu-devel] [PATCH COLO-Frame v10 15/38] ram/COLO: Record pages received from PVM by re-using migration dirty bitmap

2015-11-03 Thread zhanghailiang
We need to record the address of the dirty pages that received from PVM, It will help flushing pages that cached into SVM. Signed-off-by: zhanghailiang --- v10: - New patch split from v9's patch 13 - Rebase to master to use 'migration_bitmap_rcu' --- migra

[Qemu-devel] [PATCH COLO-Frame v10 07/38] migration: Rename the'file' member of MigrationState and MigrationIncomingState

2015-11-03 Thread zhanghailiang
o and post-copy, we need both directions communication, so here we rename the file member to indicate this path. Signed-off-by: zhanghailiang Cc: Dr. David Alan Gilbert --- Will be dropped if post-copy is merged. --- include/migration/migration.h | 4 ++-- migration/exec.c | 4

[Qemu-devel] [PATCH COLO-Frame v10 00/38] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service (FT)

2015-11-03 Thread zhanghailiang
read. (Dave's suggestion) - Add several netfilter related APIs to support buffer/release packets for COLO (patch 32 ~ patch 36) zhanghailiang (38): configure: Add parameter for configure to enable/disable COLO support migration: Introduce capability 'x-colo' to migration

[Qemu-devel] [PATCH COLO-Frame v10 04/38] migration: Add state records for migration incoming

2015-11-03 Thread zhanghailiang
ned-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- include/migration/migration.h | 3 +++ migration/migration.c | 43 +++ 2 files changed, 30 insertions(+), 16 deletions(-) diff --git a/include/migration/migration.h b/include/migration/migrati

[Qemu-devel] [PATCH COLO-Frame v10 25/38] COLO failover: Don't do failover during loading VM's state

2015-11-03 Thread zhanghailiang
We should not do failover work while the main thread is loading VM's state, otherwise it will destroy the consistent of VM's memory and device state. Here we add a new failover status 'RELAUNCH' which means we should relaunch the process of failover. Signed-off-by: zhanghai

[Qemu-devel] [PATCH COLO-Frame v10 11/38] QEMUSizedBuffer: Introduce two help functions for qsb

2015-11-03 Thread zhanghailiang
m the file into qsb, this is used to get VM state from socket into a buffer. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gilbert --- include/migration/qemu-file.h | 3 ++- migration/qemu-file-buf.c | 58 ++

[Qemu-devel] [PATCH COLO-Frame v10 32/38] netfilter: Add a public API to release all the buffered packets

2015-11-03 Thread zhanghailiang
For COLO or MC FT, We need a function to release all the buffered packets actively. Signed-off-by: zhanghailiang Cc: Jason Wang --- v10: new patch --- include/net/filter.h | 1 + include/net/net.h| 4 net/filter-buffer.c | 15 +++ net/net.c| 24

[Qemu-devel] [PATCH COLO-Frame v10 10/38] COLO: Add a new RunState RUN_STATE_COLO

2015-11-03 Thread zhanghailiang
Guest will enter this state when paused to save/restore VM state under colo checkpoint. Cc: Eric Blake Cc: Markus Armbruster Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Eric Blake --- qapi-schema.json | 7

[Qemu-devel] [PATCH COLO-Frame v10 09/38] COLO: Implement colo checkpoint protocol

2015-11-03 Thread zhanghailiang
rict synchronization is required, a opposite direction sync-point should be added. 3) Since sync-points are single direction, the remote side may go forward a lot when this side just receives the sync-point. 4) For now, we only support 'periodic' checkpoint, for which the Second

[Qemu-devel] [PATCH COLO-Frame v10 17/38] COLO: synchronize PVM's state to SVM periodically

2015-11-03 Thread zhanghailiang
Do checkpoint periodically, the default interval is 200ms. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- migration/colo.c | 14 ++ 1 file changed, 14 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index 0efab21..a6791f4 100644 --- a/migration/colo.c

[Qemu-devel] [PATCH COLO-Frame v10 01/38] configure: Add parameter for configure to enable/disable COLO support

2015-11-03 Thread zhanghailiang
configure --enable-colo/--disable-colo to switch COLO support on/off. COLO support is off by default. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei Reviewed-by: Dr. David Alan Gilbert --- configure | 11 +++ 1 file changed, 11 insertions(+) diff --git

Re: [Qemu-devel] [PATCH COLO-Frame v9 00/32] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service (FT)

2015-10-22 Thread zhanghailiang
Hi Dave, Thanks for feedback, i have recorded this problem, maybe we can optimize this calling in the future. Thanks, zhanghailiang On 2015/10/21 22:10, Dr. David Alan Gilbert wrote: Hi, Not something I found on this set; but on your older set, where you have the experimental 'live

[Qemu-devel] [PATCH] migration: Add state records for migration incoming

2015-10-22 Thread zhanghailiang
ate(), and make it public. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- Hi, This is picked from COLO frame series, Dave suggests me to submit it by itself. I fixed a little for the commit message. and keep Dave's reviewed-by tag. Thanks. --- include/migration/m

Re: [Qemu-devel] [PATCH COLO-Frame v9 09/32] COLO: Implement colo checkpoint protocol

2015-10-22 Thread zhanghailiang
On 2015/10/21 20:17, Eric Blake wrote: On 09/02/2015 02:22 AM, zhanghailiang wrote: We need communications protocol of user-defined to control the checkpoint process. The new checkpoint request is started by Primary VM, and the interactive process like below: Checkpoint synchronizing points

Re: [Qemu-devel] [PATCH COLO-Frame v9 10/32] COLO: Add a new RunState RUN_STATE_COLO

2015-10-22 Thread zhanghailiang
On 2015/10/21 20:18, Eric Blake wrote: On 09/02/2015 02:22 AM, zhanghailiang wrote: Guest will enter this state when paused to save/restore VM state under colo checkpoint. Cc: Eric Blake Cc: Markus Armbruster Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei

Re: [Qemu-devel] [PATCH COLO-Frame v9 08/32] COLO/migration: establish a new communication path from destination to source

2015-10-21 Thread zhanghailiang
On 2015/10/21 3:32, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: On 2015/10/19 17:54, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: Add a new member 'to_src_file' to MigrationIncomingState and a

Re: [Qemu-devel] [PATCH COLO-Frame v9 08/32] COLO/migration: establish a new communication path from destination to source

2015-10-20 Thread zhanghailiang
On 2015/10/19 17:54, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: Add a new member 'to_src_file' to MigrationIncomingState and a new member 'from_dst_file' to MigrationState. They will be used for returning messages from destinatio

Re: [Qemu-devel] [PATCH COLO-Frame v9 06/32] migration: Integrate COLO checkpoint process into loadvm

2015-10-20 Thread zhanghailiang
On 2015/10/19 17:17, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: Switch from normal migration loadvm process into COLO checkpoint process if COLO mode is enabled. We add three new members to struct MigrationIncomingState, 'have_colo_incoming_t

Re: [Qemu-devel] [PATCH COLO-Frame v9 04/32] migration: Add state records for migration incoming

2015-10-10 Thread zhanghailiang
On 2015/10/10 0:18, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: For migration destination, we also need to know its state, we will use it in COLO. Here we add a new member 'state' for MigrationIncomingState, and also use migrate_set_state()

Re: [Qemu-devel] [PATCH COLO-Frame v9 05/32] migration: Integrate COLO checkpoint process into migration

2015-10-09 Thread zhanghailiang
On 2015/10/10 0:53, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: Add a migrate state: MIGRATION_STATUS_COLO, enter this migration state after the first live migration successfully finished. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed

Re: [Qemu-devel] [PATCH COLO-Frame v9 03/32] COLO: migrate colo related info to slave

2015-10-07 Thread zhanghailiang
On 2015/10/3 2:45, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: We can know if VM in destination should go into COLO mode by refer to the info that been migrated from PVM. We skip this section if colo is not enabled (i.e. migrate_set_capability colo off

Re: [Qemu-devel] [PATCH COLO-Frame v9 02/32] migration: Introduce capability 'colo' to migration

2015-10-07 Thread zhanghailiang
On 2015/10/3 0:02, Eric Blake wrote: On 09/02/2015 02:22 AM, zhanghailiang wrote: We add helper function colo_supported() to indicate whether colo is supported or not, with which we use to control whether or not showing 'colo' string to users, they can use qmp command 

Re: [Qemu-devel] [PATCH COLO-Frame v9 00/32] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service (FT)

2015-09-15 Thread zhanghailiang
Ping again ... On 2015/9/2 16:22, zhanghailiang wrote: This is the 9th version of COLO. Please Note that, this version is very different from the previous versions. since we have decided to realize proxy in qemu, which based on slirp in qemu. We dropped all the original colo proxy related part

Re: [Qemu-devel] [PATCH] migration: Use g_new() & friends where that makes obvious sense

2015-09-14 Thread zhanghailiang
Reviewed-by: zhanghailiang On 2015/9/14 19:51, Markus Armbruster wrote: g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer, for two reasons. One, it catches multiplication overflowing size_t. Two, it returns T * rather than void *, which lets the compiler catch more

[Qemu-devel] [PATCH] piix: fix resource leak reported by Coverity

2015-09-14 Thread zhanghailiang
config_fd should be closed before return, or there will be a resource leak error. Signed-off-by: zhanghailiang --- hw/pci-host/piix.c | 11 +++ 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/hw/pci-host/piix.c b/hw/pci-host/piix.c index 1fb71c8..7b2fbf9 100644 --- a/hw

Re: [Qemu-devel] [PATCH COLO-Frame v9 00/32] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service (FT)

2015-09-08 Thread zhanghailiang
Ping... Hi Juan & Amit, Could you please help review this series ? Since it is already comes v9, i really hope to get your feedback on this series :) Thanks, zhanghailiang On 2015/9/2 16:22, zhanghailiang wrote: This is the 9th version of COLO. Please Note that, this version is

Re: [Qemu-devel] [PATCH COLO-Frame v9 00/32] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service (FT)

2015-09-02 Thread zhanghailiang
e the ramdisk for the disk. # qemu-img create -f qcow2 /mnt/ramfs/active_disk.img 10G # qemu-img create -f qcow2 /mnt/ramfs/hidden_disk.img 10G Please note, the size should be same with VM's disk. Thanks. zhanghailiang Thanks! On 09/02/2015 04:22 PM, zhanghailiang wrote: This is the 9th

[Qemu-devel] [PATCH COLO-Frame v9 32/32] COLO: Add net packets treatment into COLO

2015-09-02 Thread zhanghailiang
We will buffer all the net packets that sended by PVM, and release them during the checkpoint process. Signed-off-by: zhanghailiang Signed-off-by: Yang Hongyang --- migration/colo.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index d7a14ba

[Qemu-devel] [PATCH COLO-Frame v9 25/32] COLO: Control the checkpoint delay time by migrate-set-parameters command

2015-09-02 Thread zhanghailiang
Add checkpoint-delay parameter for migrate-set-parameters, so that we can control the checkpoint frequency when COLO is in periodic mode. Cc: Luiz Capitulino Cc: Eric Blake Cc: Markus Armbruster Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- hmp.c | 8

[Qemu-devel] [PATCH COLO-Frame v9 29/32] COLO: Separate the process of saving/loading ram and device state

2015-09-02 Thread zhanghailiang
during checkpoint. Besides, we move the colo_flush_ram_cache to the proper position after the above change. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- include/sysemu/sysemu.h | 5 ++ migration/colo.c| 43 +++ migration/ram.c | 8 --- migration

[Qemu-devel] [PATCH COLO-Frame v9 30/32] COLO: Split qemu_savevm_state_begin out of checkpoint process

2015-09-02 Thread zhanghailiang
data transferring in the later checkpoint. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- migration/colo.c | 51 +-- 1 file changed, 37 insertions(+), 14 deletions(-) diff --git a/migration/colo.c b/migration/colo.c index 22ca7e5

[Qemu-devel] [PATCH COLO-Frame v9 26/32] COLO: Implement shutdown checkpoint

2015-09-02 Thread zhanghailiang
For Secondary VM, we forbid it shutdown directly when in COLO mode, FOR Primary VM's shutdown, we should do some work to ensure the consistent action between PVM and SVM. Cc: Paolo Bonzini Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- include/sysemu/sysemu.h | 3 +++ migr

[Qemu-devel] [PATCH COLO-Frame v9 24/32] COLO failover: Don't do failover during loading VM's state

2015-09-02 Thread zhanghailiang
We should not do failover work while the main thread is loading VM's state, otherwise it will destroy the consistent of VM's memory and device state. Here we add a new failover status 'RELAUNCH' which means we should relaunch the process of failover. Signed-off-by: zhanghai

[Qemu-devel] [PATCH COLO-Frame v9 22/32] qmp event: Add event notification for COLO error

2015-09-02 Thread zhanghailiang
is still necessary to notify users that we exit COLO mode. Cc: Markus Armbruster Cc: Michael Roth Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- docs/qmp/qmp-events.txt | 17 + migration/colo.c| 17 + qapi-schema.json| 16 +++

[Qemu-devel] [PATCH COLO-Frame v9 27/32] COLO: Update the global runstate after going into colo state

2015-09-02 Thread zhanghailiang
If we start qemu with -S, the runstate will change from 'prelaunch' to 'running' after going into colo state. So it is necessary to update the global runstate after going into colo state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- migration/colo.c | 5

[Qemu-devel] [PATCH COLO-Frame v9 23/32] COLO failover: Shutdown related socket fd when do failover

2015-09-02 Thread zhanghailiang
s to wake up the blocking operation in failover BH. Besides, we should close the corresponding file descriptors after failvoer BH shutdown them, or there will be an error. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- migration/colo.c | 35 +++ 1 file cha

[Qemu-devel] [PATCH COLO-Frame v9 18/32] COLO failover: Introduce state to record failover process

2015-09-02 Thread zhanghailiang
set the value. Signed-off-by: zhanghailiang --- include/migration/failover.h | 10 ++ migration/colo-failover.c| 37 + migration/colo.c | 4 trace-events | 1 + 4 files changed, 52 insertions(+) diff --git a

[Qemu-devel] [PATCH COLO-Frame v9 28/32] savevm: Split load vm state function qemu_loadvm_state

2015-09-02 Thread zhanghailiang
qemu_loadvm_state is too long, and we can simplify it by splitting up with three helper functions. Signed-off-by: zhanghailiang --- migration/savevm.c | 165 +++-- 1 file changed, 96 insertions(+), 69 deletions(-) diff --git a/migration/savevm.c

[Qemu-devel] [PATCH COLO-Frame v9 21/32] COLO: implement default failover treatment

2015-09-02 Thread zhanghailiang
If we detect some error in colo, we will wait for some time, hoping users also detect it. If users don't issue failover command. We will go into default failover procedure, which the PVM will takeover work while SVM is exit in default. Signed-off-by: zhanghailiang Signed-off-by: Li Zh

[Qemu-devel] [PATCH COLO-Frame v9 10/32] COLO: Add a new RunState RUN_STATE_COLO

2015-09-02 Thread zhanghailiang
Guest will enter this state when paused to save/restore VM state under colo checkpoint. Cc: Eric Blake Cc: Markus Armbruster Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei Reviewed-by: Dr. David Alan Gilbert --- qapi-schema.json | 7 ++- vl.c

[Qemu-devel] [PATCH COLO-Frame v9 20/32] COLO: Implement failover work for Secondary VM

2015-09-02 Thread zhanghailiang
If users require SVM to takeover work, colo incoming thread should exit from loop while failover BH helps backing to migration incoming coroutine. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- migration/colo.c | 41 ++--- 1 file changed, 38

[Qemu-devel] [PATCH COLO-Frame v9 17/32] COLO failover: Introduce a new command to trigger a failover

2015-09-02 Thread zhanghailiang
side will exit COLO mode and take over operation. If sent to the Secondary, the secondary will run failover work, then take over server operation to become the new Primary. Cc: Luiz Capitulino Cc: Eric Blake Cc: Markus Armbruster Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Sig

[Qemu-devel] [PATCH COLO-Frame v9 31/32] COLO: Add block replication into colo process

2015-09-02 Thread zhanghailiang
From: Wen Congyang Make sure master start block replication after slave's block replication started. Signed-off-by: zhanghailiang Signed-off-by: Wen Congyang Signed-off-by: Yang Hongyang Signed-off-by: Li Zhijian --- migration/colo.c

[Qemu-devel] [PATCH COLO-Frame v9 13/32] COLO: Load PVM's dirty pages into SVM's RAM cache temporarily

2015-09-02 Thread zhanghailiang
7;s state. Signed-off-by: zhanghailiang Signed-off-by: Yang Hongyang Signed-off-by: Li Zhijian Signed-off-by: Gonglei --- include/exec/cpu-all.h | 1 + include/migration/colo.h | 3 ++ migration/colo.c | 14 ++-- migration/ram.c | 93

[Qemu-devel] [PATCH COLO-Frame v9 19/32] COLO: Implement failover work for Primary VM

2015-09-02 Thread zhanghailiang
For PVM, if there is failover request from users. The colo thread will exit the loop while the failover BH does the cleanup work and resumes VM. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- include/migration/colo.h | 4 +++ include/migration/failover.h | 1 + migration/colo

[Qemu-devel] [PATCH COLO-Frame v9 07/32] migration: Rename the'file' member of MigrationState and MigrationIncomingState

2015-09-02 Thread zhanghailiang
o and post-copy, we need both directions communication, so here we rename the file member to indicate this path. Signed-off-by: zhanghailiang Cc: Dr. David Alan Gilbert --- include/migration/migration.h | 4 ++-- migration/exec.c | 4 ++-- migration/fd.c| 4

[Qemu-devel] [PATCH COLO-Frame v9 15/32] COLO: Flush PVM's cached RAM into SVM's memory

2015-09-02 Thread zhanghailiang
all content of PVM's RAM cache into SVM's MEMORY, we do this in a more efficient way: Only flush any page that dirtied by PVM since last checkpoint. In this way, we can ensure SVM's memory same with PVM's. Besides, we must ensure flush RAM cache before load device state. Si

[Qemu-devel] [PATCH COLO-Frame v9 08/32] COLO/migration: establish a new communication path from destination to source

2015-09-02 Thread zhanghailiang
Add a new member 'to_src_file' to MigrationIncomingState and a new member 'from_dst_file' to MigrationState. They will be used for returning messages from destination to source. It will also be used by post-copy migration. Signed-off-by: zhanghailiang Signed-off-by: Li Zh

[Qemu-devel] [PATCH COLO-Frame v9 16/32] COLO: synchronize PVM's state to SVM periodically

2015-09-02 Thread zhanghailiang
Do checkpoint periodically, the default interval is 200ms. Signed-off-by: zhanghailiang Signed-off-by: Yang Hongyang --- migration/colo.c | 14 ++ 1 file changed, 14 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index b6a6d66..07e0c1b 100644 --- a/migration/colo.c

[Qemu-devel] [PATCH COLO-Frame v9 11/32] QEMUSizedBuffer: Introduce two help functions for qsb

2015-09-02 Thread zhanghailiang
m the file into qsb, this is used to get VM state from socket into a buffer. Signed-off-by: Yang Hongyang Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- include/migration/qemu-file.h | 3 ++- migration/qemu-file-buf.c | 58 ++

[Qemu-devel] [PATCH COLO-Frame v9 14/32] COLO: Load VMState into qsb before restore it

2015-09-02 Thread zhanghailiang
can ensure the data is intact. Signed-off-by: zhanghailiang Signed-off-by: Yang Hongyang Signed-off-by: Gonglei Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 49 +++-- 1 file changed, 47 insertions(+), 2 deletions(-) diff --git a

[Qemu-devel] [PATCH COLO-Frame v9 00/32] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service (FT)

2015-09-02 Thread zhanghailiang
kpoint delay time (patch 25) - Add new patch (patch 29/30) to seperate the process of saving/loading device and state during checkpoint. which will reduce the data size for sending and also reduce the qsb size used in checkpoint. Wen Congyang (1): COLO: Add block replication into colo process

[Qemu-devel] [PATCH COLO-Frame v9 06/32] migration: Integrate COLO checkpoint process into loadvm

2015-09-02 Thread zhanghailiang
_incoming_co' records the original migration incoming coroutine. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Yang Hongyang --- include/migration/colo.h | 7 +++ include/migration/migration.h | 7 +++ migration/colo-comm.c | 1

[Qemu-devel] [PATCH COLO-Frame v9 01/32] configure: Add parameter for configure to enable/disable COLO support

2015-09-02 Thread zhanghailiang
configure --enable-colo/--disable-colo to switch COLO support on/off. COLO support is off by default. Signed-off-by: zhanghailiang Signed-off-by: Yang Hongyang Signed-off-by: Gonglei --- configure | 11 +++ 1 file changed, 11 insertions(+) diff --git a/configure b/configure index

[Qemu-devel] [PATCH COLO-Frame v9 12/32] COLO: Save PVM state to secondary side when do checkpoint

2015-09-02 Thread zhanghailiang
the size of VM state, so in master, we use qsb to store VM state temporarily, get the data size by call qsb_get_length() and then migrate the data to the qsb in the secondary side. Signed-off-by: zhanghailiang Signed-off-by: Yang Hongyang Signed-off-by: Gonglei Signed-off-by: Li Zhijian

[Qemu-devel] [PATCH COLO-Frame v9 09/32] COLO: Implement colo checkpoint protocol

2015-09-02 Thread zhanghailiang
rict synchronization is required, a opposite direction sync-point should be added. 3) Since sync-points are single direction, the remote side may go forward a lot when this side just receives the sync-point. 4) For now, we only support 'periodic' checkpoint, for which the Second

[Qemu-devel] [PATCH COLO-Frame v9 04/32] migration: Add state records for migration incoming

2015-09-02 Thread zhanghailiang
ned-off-by: zhanghailiang --- include/migration/migration.h | 3 +++ migration/migration.c | 38 +++--- 2 files changed, 26 insertions(+), 15 deletions(-) diff --git a/include/migration/migration.h b/include/migration/migration.h index 05de3a1..a62068f 100644 ---

[Qemu-devel] [PATCH COLO-Frame v9 03/32] COLO: migrate colo related info to slave

2015-09-02 Thread zhanghailiang
/destination; Signed-off-by: zhanghailiang Signed-off-by: Yang Hongyang Signed-off-by: Gonglei --- include/migration/colo.h | 2 ++ migration/Makefile.objs | 1 + migration/colo-comm.c| 54 vl.c | 3 ++- 4 files changed

[Qemu-devel] [PATCH COLO-Frame v9 05/32] migration: Integrate COLO checkpoint process into migration

2015-09-02 Thread zhanghailiang
Add a migrate state: MIGRATION_STATUS_COLO, enter this migration state after the first live migration successfully finished. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei --- include/migration/colo.h | 3 +++ migration/colo.c | 58

[Qemu-devel] [PATCH COLO-Frame v9 02/32] migration: Introduce capability 'colo' to migration

2015-09-02 Thread zhanghailiang
earn if colo is supported. Cc: Juan Quintela Cc: Amit Shah Cc: Eric Blake Cc: Markus Armbruster Signed-off-by: zhanghailiang Signed-off-by: Yang Hongyang Signed-off-by: Gonglei --- include/migration/colo.h | 20 include/migration/migration.h | 1 + migration/Makefil

Re: [Qemu-devel] [PATCH COLO-Frame v8 16/34] COLO failover: Introduce a new command to trigger a failover

2015-08-31 Thread zhanghailiang
On 2015/8/29 6:06, Eric Blake wrote: On 07/29/2015 02:45 AM, zhanghailiang wrote: We leave users to use whatever heartbeat solution they want, if the heartbeat is lost, or other errors they detect, they can use command 'colo_lost_heartbeat' to tell COLO to do failover, COLO will do

Re: [Qemu-devel] [PATCH COLO-Frame v8 19/34] qmp event: Add event notification for COLO error

2015-08-31 Thread zhanghailiang
On 2015/8/31 23:07, Eric Blake wrote: On 08/31/2015 03:27 AM, zhanghailiang wrote: +Data: None. + + - "mode": COLO mode, 'primary' or 'secondary' + - "error": Error message (json-string, optional) The "Data: None" designation is inconsiste

Re: [Qemu-devel] [PATCH COLO-Frame v8 31/34] COLO: Add colo-set-checkpoint-period command

2015-08-31 Thread zhanghailiang
On 2015/8/29 6:26, Eric Blake wrote: On 07/29/2015 02:45 AM, zhanghailiang wrote: With this command, we can control the period of checkpoint, if there is no comparison of net packets. Cc: Luiz Capitulino Cc: Eric Blake Cc: Markus Armbruster Signed-off-by: zhanghailiang Signed-off-by: Li

Re: [Qemu-devel] [PATCH COLO-Frame v8 21/34] COLO: Add new command parameter 'forward_nic' 'colo_script' for net

2015-08-31 Thread zhanghailiang
Hi Eric This patch will be dropped in next version since we will realize the proxy in qemu. Thanks, zhanghailiang On 2015/8/29 6:24, Eric Blake wrote: On 07/29/2015 02:45 AM, zhanghailiang wrote: The 'forward_nic' should be assigned with network name, for exmple, 'eth2'.

Re: [Qemu-devel] [PATCH COLO-Frame v8 19/34] qmp event: Add event notification for COLO error

2015-08-31 Thread zhanghailiang
On 2015/8/29 6:13, Eric Blake wrote: On 07/29/2015 02:45 AM, zhanghailiang wrote: If some errors happen during VM's COLO FT stage, it's import to notify the users s/import/important/ this event, Togehter with 'colo_lost_heartbeat', users can intervene in COLO's s/

Re: [Qemu-devel] [PATCH COLO-Frame v8 09/34] COLO: Add a new RunState RUN_STATE_COLO

2015-08-30 Thread zhanghailiang
On 2015/8/29 5:58, Eric Blake wrote: On 07/29/2015 02:45 AM, zhanghailiang wrote: Guest will enter this state when paused to save/restore VM state under colo checkpoint. Cc: Eric Blake Cc: Markus Armbruster Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei

Re: [Qemu-devel] [PATCH COLO-Frame v8 06/34] migration: Integrate COLO checkpoint process into migration

2015-08-30 Thread zhanghailiang
On 2015/8/29 5:55, Eric Blake wrote: On 07/29/2015 02:45 AM, zhanghailiang wrote: Add a migrate state: MIGRATION_STATUS_COLO, enter this migration state after the first live migration successfully finished. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei

Re: [Qemu-devel] [PATCH COLO-Frame v8 02/34] migration: Introduce capability 'colo' to migration

2015-08-30 Thread zhanghailiang
On 2015/8/29 5:54, Eric Blake wrote: On 07/29/2015 02:45 AM, zhanghailiang wrote: We add helper function colo_supported() to indicate whether colo is supported or not, with which we use to control whether or not showing 'colo' string to users, they can use qmp command 

Re: [Qemu-devel] [PATCH COLO-Frame v8 08/34] COLO: Implement colo checkpoint protocol

2015-08-28 Thread zhanghailiang
On 2015/8/27 20:43, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: Hi Dave, On 2015/8/27 18:40, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: We need communications protocol of user-defined to control the

Re: [Qemu-devel] [PATCH COLO-Frame v8 08/34] COLO: Implement colo checkpoint protocol

2015-08-27 Thread zhanghailiang
Hi Dave, On 2015/8/27 18:40, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: We need communications protocol of user-defined to control the checkpoint process. The new checkpoint request is started by Primary VM, and the interactive process like below

Re: [Qemu-devel] [PATCH COLO-Frame v8 00/34] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service (FT)

2015-08-25 Thread zhanghailiang
On 2015/8/24 22:38, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: This is the 8th version of COLO. I'm seeing an occasional error: pcibus_reset: Assertion `bus->irq_count[i] == 0' failed. on the secondary; have you seen that? bus->

Re: [Qemu-devel] [PATCH v7 15/42] Return path: Source handling of return path

2015-08-18 Thread zhanghailiang
On 2015/8/18 18:45, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: Hi Dave, On 2015/6/16 18:26, Dr. David Alan Gilbert (git) wrote: From: "Dr. David Alan Gilbert" Open a return path, and handle messages that are received upon it. Signed-

Re: [Qemu-devel] [PATCH 4/5] migration: size_t'ify some of qemu-file

2015-08-13 Thread zhanghailiang
d fixes up all the different implementations of them. Note that I've not yet followed this deeply into bdrv_ implementations. Signed-off-by: Dr. David Alan Gilbert --- Reviewed-by: zhanghailiang include/migration/qemu-file.h | 8 migration/qemu-file-buf.c | 7 ---

Re: [Qemu-devel] [PATCH 5/5] migration: qemu-file more size_t'ifying

2015-08-13 Thread zhanghailiang
On 2015/8/13 18:51, Dr. David Alan Gilbert (git) wrote: From: "Dr. David Alan Gilbert" This time convert the external functions: qemu_get_buffer, qemu_peek_buffer qemu_put_buffer and qemu_put_buffer_async Signed-off-by: Dr. David Alan Gilbert --- Reviewed-by: zha

Re: [Qemu-devel] [PATCH 2/5] Split out end of migration code from migration_thread

2015-08-13 Thread zhanghailiang
--- Reviewed-by: zhanghailiang migration/migration.c | 75 --- trace-events | 2 ++ 2 files changed, 49 insertions(+), 28 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 662e77e..46bb410 100644

Re: [Qemu-devel] [PATCH COLO-Frame v8 00/34] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service (FT)

2015-08-12 Thread zhanghailiang
On 2015/8/5 19:24, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: This is the 8th version of COLO. Here is only COLO frame part, include: VM checkpoint, failover, proxy API, block replication API, not include block replication. The block part is treated

Re: [Qemu-devel] [PATCH v7 41/42] Disable mlock around incoming postcopy

2015-08-06 Thread zhanghailiang
ut then I thought: would the realtime case want a migration to happen at all? Then disable migration with realtime looks like saner. But that decission don't belong to this series. I added this patch because Zhanghailiang had reported trying to use it and it failing. Zhanghailiang: Do you

Re: [Qemu-devel] [PATCH COLO-Frame v8 00/34] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service (FT)

2015-08-06 Thread zhanghailiang
On 2015/8/5 19:24, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: This is the 8th version of COLO. Here is only COLO frame part, include: VM checkpoint, failover, proxy API, block replication API, not include block replication. The block part is treated

Re: [Qemu-devel] [PATCH v7 15/42] Return path: Source handling of return path

2015-08-05 Thread zhanghailiang
igrationIncomingState struct, rename its original 'file' member to 'input_file',and add a new 'output_file'. IMHO, this will make things more clear. Thanks, zhanghailiang +QemuThreadrp_thread; +bool error; +} rp_state; + doubl

Re: [Qemu-devel] [RFC/COLO: 1/3] COLO: Hybrid mode

2015-08-04 Thread zhanghailiang
Seems pretty good overall~ For the part of migration parameters command, we have discussed before and Markus promised to reconstruct this part in qemu 2.5 cycle. But for now, it is OK. Cc: Markus Armbruster On 2015/8/5 3:26, Dr. David Alan Gilbert (git) wrote: From: "Dr. David Alan Gilbert"

Re: [Qemu-devel] [POC] colo-proxy in qemu

2015-07-30 Thread zhanghailiang
On 2015/7/31 9:08, Yang Hongyang wrote: On 07/31/2015 01:53 AM, Dr. David Alan Gilbert wrote: * Yang Hongyang (yan...@cn.fujitsu.com) wrote: On 07/30/2015 09:59 PM, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: On 2015/7/30 20:30, Dr. David Alan

Re: [Qemu-devel] [POC] colo-proxy in qemu

2015-07-30 Thread zhanghailiang
On 2015/7/31 1:53, Dr. David Alan Gilbert wrote: * Yang Hongyang (yan...@cn.fujitsu.com) wrote: On 07/30/2015 09:59 PM, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: On 2015/7/30 20:30, Dr. David Alan Gilbert wrote: * Gonglei (arei.gong...@huawei.com

Re: [Qemu-devel] [POC] colo-proxy in qemu

2015-07-30 Thread zhanghailiang
On 2015/7/30 20:30, Dr. David Alan Gilbert wrote: * Gonglei (arei.gong...@huawei.com) wrote: On 2015/7/30 19:56, Dr. David Alan Gilbert wrote: * Jason Wang (jasow...@redhat.com) wrote: On 07/30/2015 04:03 PM, Dr. David Alan Gilbert wrote: * Dong, Eddie (eddie.d...@intel.com) wrote: A quest

Re: [Qemu-devel] [PATCH COLO-Frame v8 23/34] tap: Make launch_script() public

2015-07-29 Thread zhanghailiang
On 2015/7/30 11:32, Jason Wang wrote: On 07/29/2015 05:43 PM, zhanghailiang wrote: On 2015/7/29 17:24, Jason Wang wrote: On 07/29/2015 05:17 PM, zhanghailiang wrote: On 2015/7/29 16:57, Jason Wang wrote: On 07/29/2015 04:45 PM, zhanghailiang wrote: We also change the parameters of

Re: [Qemu-devel] [PATCH COLO-Frame v8 23/34] tap: Make launch_script() public

2015-07-29 Thread zhanghailiang
On 2015/7/29 17:24, Jason Wang wrote: On 07/29/2015 05:17 PM, zhanghailiang wrote: On 2015/7/29 16:57, Jason Wang wrote: On 07/29/2015 04:45 PM, zhanghailiang wrote: We also change the parameters of launch_script(). A quick question (I don't go through the codes tough). What'

Re: [Qemu-devel] [PATCH COLO-Frame v8 23/34] tap: Make launch_script() public

2015-07-29 Thread zhanghailiang
On 2015/7/29 16:57, Jason Wang wrote: On 07/29/2015 04:45 PM, zhanghailiang wrote: We also change the parameters of launch_script(). A quick question (I don't go through the codes tough). What's the plan for management(libvirt)? I believe some setup (iptables, fd creation)

[Qemu-devel] [PATCH COLO-Frame v8 34/34] COLO: Add block replication into colo process

2015-07-29 Thread zhanghailiang
Make sure master start block replication after slave's block replication started. Signed-off-by: zhanghailiang Signed-off-by: Wen Congyang Signed-off-by: Yang Hongyang Signed-off-by: Li Zhijian --- migration/colo.c | 71 +--- trace-e

[Qemu-devel] [PATCH COLO-Frame v8 31/34] COLO: Add colo-set-checkpoint-period command

2015-07-29 Thread zhanghailiang
With this command, we can control the period of checkpoint, if there is no comparison of net packets. Cc: Luiz Capitulino Cc: Eric Blake Cc: Markus Armbruster Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- hmp-commands.hx| 15 +++ hmp.c | 7

[Qemu-devel] [PATCH COLO-Frame v8 27/34] COLO NIC: Some init work related with proxy module

2015-07-29 Thread zhanghailiang
tefan Hajnoczi Cc: Jason Wang Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- configure | 22 +++- net/colo-nic.c | 160 + 2 files changed, 180 insertions(+), 2 deletions(-) diff --git a/configure b/configure index 33

[Qemu-devel] [PATCH COLO-Frame v8 02/34] migration: Introduce capability 'colo' to migration

2015-07-29 Thread zhanghailiang
earn if colo is supported. Cc: Juan Quintela Cc: Amit Shah Cc: Eric Blake Cc: Markus Armbruster Signed-off-by: zhanghailiang Signed-off-by: Yang Hongyang Signed-off-by: Gonglei --- include/migration/colo.h | 20 include/migration/migration.h | 1 + migration/Makefil

[Qemu-devel] [PATCH COLO-Frame v8 32/34] COLO NIC: Implement NIC checkpoint and failover

2015-07-29 Thread zhanghailiang
Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- include/net/colo-nic.h | 2 ++ migration/colo.c | 22 +++--- net/colo-nic.c | 23 +++ 3 files changed, 44 insertions(+), 3 deletions(-) diff --git a/include/net/colo-nic.h b/include

[Qemu-devel] [PATCH COLO-Frame v8 30/34] COLO: Improve checkpoint efficiency by do additional periodic checkpoint

2015-07-29 Thread zhanghailiang
stent). Signed-off-by: zhanghailiang Signed-off-by: Yang Hongyang --- migration/colo.c | 29 + 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/migration/colo.c b/migration/colo.c index f9f2156..f5bb668 100644 --- a/migration/colo.c +++ b/migration/c

<    3   4   5   6   7   8   9   10   11   12   >