[Qemu-devel] [PATCH COLO-Frame v6 19/31] COLO NIC: Init/remove colo nic devices when add/cleanup tap devices

2015-06-18 Thread zhanghailiang
When COLO mode, we do some init work for nic that will be used for COLO. Cc: Stefan Hajnoczi stefa...@redhat.com Cc: Jason Wang jasow...@redhat.com Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- include/net/colo-nic.h | 3

[Qemu-devel] [PATCH COLO-Frame v6 23/31] COLO NIC: Some init work related with proxy module

2015-06-18 Thread zhanghailiang
Hajnoczi stefa...@redhat.com Cc: Jason Wang jasow...@redhat.com Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- configure | 22 +++- net/colo-nic.c | 160 + 2 files

[Qemu-devel] [PATCH COLO-Frame v6 21/31] COLO NIC: Implement colo nic device interface configure()

2015-06-18 Thread zhanghailiang
Implement colo nic device interface configure() add a script to configure nic devices: ${QEMU_SCRIPT_DIR}/colo-proxy-script.sh Cc: Stefan Hajnoczi stefa...@redhat.com Cc: Jason Wang jasow...@redhat.com Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij

[Qemu-devel] [PATCH COLO-Frame v6 20/31] tap: Make launch_script() public

2015-06-18 Thread zhanghailiang
We also change the parameters of launch_script(). Cc: Stefan Hajnoczi stefa...@redhat.com Cc: Jason Wang jasow...@redhat.com Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- include/net/tap.h | 2 ++ net/tap.c | 31

[Qemu-devel] [PATCH COLO-Frame v6 16/31] qmp event: Add event notification for COLO error

2015-06-18 Thread zhanghailiang
exit COLO mode. Cc: Markus Armbruster arm...@redhat.com Cc: Michael Roth mdr...@linux.vnet.ibm.com Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- docs/qmp/qmp-events.txt | 16 migration/colo.c| 12

[Qemu-devel] [PATCH COLO-Frame v6 27/31] COLO: Add colo-set-checkpoint-period command

2015-06-18 Thread zhanghailiang
With this command, we can control the period of checkpoint, if there is no comparison of net packets. Cc: Luiz Capitulino lcapitul...@redhat.com Cc: Eric Blake ebl...@redhat.com Cc: Markus Armbruster arm...@redhat.com Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li

Re: [Qemu-devel] Adding new migration-parameters - any easier way?

2015-06-17 Thread zhanghailiang
... Thanks, zhanghailiang On 2015/6/5 17:50, Dr. David Alan Gilbert wrote: Hi, Is there any way that we could make it easier to add new migration parameters? The current way is complicated and error prone; as far as I can tell, to add a new parameter we need to: 1) qapi-schema.json

Re: [Qemu-devel] [PATCH COLO-Frame v5 25/29] COLO: Add colo-set-checkpoint-period command

2015-06-09 Thread zhanghailiang
On 2015/6/9 16:01, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: On 2015/6/6 2:45, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: With this command, we can control the period of checkpoint

Re: [Qemu-devel] [PATCH RFC] migration: Re-implement 'migrate-set-parameters' to make it easily for extension

2015-06-09 Thread zhanghailiang
On 2015/6/9 18:49, Eric Blake wrote: On 06/09/2015 01:18 AM, zhanghailiang wrote: In the subject: s/easily/easier/ 'migrate-set-parameters' was specially used for setting the compression related parameters of migration. Here we re-implement it so that it can be easily extended when we want

[Qemu-devel] [PATCH RFC] migration: Re-implement 'migrate-set-parameters' to make it easily for extension

2015-06-09 Thread zhanghailiang
'migrate-set-parameters' was specially used for setting the compression related parameters of migration. Here we re-implement it so that it can be easily extended when we want to add other parameters for migration. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com --- hmp.c

Re: [Qemu-devel] [PATCH COLO-Frame v5 11/29] COLO VMstate: Load VM state into qsb before restore it

2015-06-08 Thread zhanghailiang
On 2015/6/6 2:02, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: We should cache the device state before restore it, besides, we should call qemu_system_reset() before load VM state, which can ensure the data is intact. I think the description could

Re: [Qemu-devel] [PATCH COLO-Frame v5 25/29] COLO: Add colo-set-checkpoint-period command

2015-06-08 Thread zhanghailiang
On 2015/6/6 2:45, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: With this command, we can control the period of checkpoint, if there is no comparison of net packets. This should use the MigrationParameter stuff that's gone into qemu recently; in my

Re: [Qemu-devel] [PATCH COLO-Frame v5 00/29] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service

2015-05-29 Thread zhanghailiang
On 2015/5/29 9:29, Wen Congyang wrote: On 05/29/2015 12:24 AM, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: This is the 5th version of COLO, here is only COLO frame part, include: VM checkpoint, failover, proxy API, block replication API, not include

Re: [Qemu-devel] [PATCH v6 42/47] Postcopy; Handle userfault requests

2015-05-25 Thread zhanghailiang
On 2015/4/15 1:04, Dr. David Alan Gilbert (git) wrote: From: Dr. David Alan Gilbert dgilb...@redhat.com userfaultfd is a Linux syscall that gives an fd that receives a stream of notifications of accesses to pages registered with it and allows the program to acknowledge those stalls and tell the

Re: [Qemu-devel] [PATCH COLO-Frame v5 00/29] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service

2015-05-22 Thread zhanghailiang
On 2015/5/21 19:30, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: This is the 5th version of COLO, here is only COLO frame part, include: VM checkpoint, failover, proxy API, block replication API, not include block replication. The block part has been

[Qemu-devel] [PATCH COLO-Frame v5 05/29] migration: Integrate COLO checkpoint process into loadvm

2015-05-21 Thread zhanghailiang
Switch from normal migration loadvm process into COLO checkpoint process if COLO mode is enabled. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com Signed-off-by: Yang Hongyang yan

[Qemu-devel] [PATCH COLO-Frame v5 08/29] QEMUSizedBuffer: Introduce two help functions for qsb

2015-05-21 Thread zhanghailiang
into qsb, this is used to get VM state from socket into a buffer. Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Reviewed-by: Dr. David Alan Gilbert dgilb...@redhat.com --- include/migration/qemu-file.h | 3 ++- migration/qemu-file

[Qemu-devel] [PATCH COLO-Frame v5 15/29] COLO failover: Implement COLO master/slave failover work

2015-05-21 Thread zhanghailiang
If failover is requested, after some cleanup work, PVM or SVM will exit COLO mode, and resume to normal run. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com --- include/migration

[Qemu-devel] [PATCH COLO-Frame v5 00/29] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service

2015-05-21 Thread zhanghailiang
Eric Blake and Dr. David Alan Gilbert v1: - implement the frame of colo Wen Congyang (1): COLO: Add block replication into colo process zhanghailiang (28): configure: Add parameter for configure to enable/disable COLO support migration: Introduce capability 'colo' to migration COLO

[Qemu-devel] [PATCH COLO-Frame v5 02/29] migration: Introduce capability 'colo' to migration

2015-05-21 Thread zhanghailiang
-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com Signed-off-by: Gonglei arei.gong...@huawei.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com --- include/migration/migration-colo.h | 20 include/migration/migration.h

[Qemu-devel] [PATCH COLO-Frame v5 17/29] COLO: Add new command parameter 'colo_nicname' 'colo_script' for net

2015-05-21 Thread zhanghailiang
The 'colo_nicname' should be assigned with network name, for exmple, 'eth2'. It will be parameter of 'colo_script', 'colo_script' should be assigned with an scirpt path. We parse these parameter in tap. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij

[Qemu-devel] [PATCH COLO-Frame v5 14/29] COLO failover: Introduce a new command to trigger a failover

2015-05-21 Thread zhanghailiang
, and takeover, if to Secondary, Secondary will do failover work and at last takeover server. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com Signed-off-by: Yang Hongyang yan

[Qemu-devel] [PATCH COLO-Frame v5 28/29] COLO: Implement shutdown checkpoint

2015-05-21 Thread zhanghailiang
For SVM, we forbid it shutdown directly when in COLO mode, FOR PVM's shutdown, we should do some work to ensure the consistent action between PVM and SVM. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com Signed-off-by: Li Zhijian lizhij

[Qemu-devel] [PATCH COLO-Frame v5 29/29] COLO: Add block replication into colo process

2015-05-21 Thread zhanghailiang
From: Wen Congyang we...@cn.fujitsu.com Make sure master start block replication after slave's block replication started Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Wen Congyang we...@cn.fujitsu.com Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com Signed-off

[Qemu-devel] [PATCH COLO-Frame v5 25/29] COLO: Add colo-set-checkpoint-period command

2015-05-21 Thread zhanghailiang
With this command, we can control the period of checkpoint, if there is no comparison of net packets. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- hmp-commands.hx| 15 +++ hmp.c | 7

[Qemu-devel] [PATCH COLO-Frame v5 04/29] migration: Integrate COLO checkpoint process into migration

2015-05-21 Thread zhanghailiang
Add a migrate state: MIGRATION_STATUS_COLO, enter this migration state after the first live migration successfully finished. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Gonglei arei.gong...@huawei.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com --- include

[Qemu-devel] [PATCH COLO-Frame v5 07/29] COLO: Add a new RunState RUN_STATE_COLO

2015-05-21 Thread zhanghailiang
Guest will enter this state when paused to save/restore VM state under colo checkpoint. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Gonglei arei.gong...@huawei.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com Reviewed-by: Dr. David Alan Gilbert dgilb

[Qemu-devel] [PATCH COLO-Frame v5 22/29] COLO: Handle nfnetlink message from proxy module

2015-05-21 Thread zhanghailiang
' 'atomic_xchg'. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- net/colo-nic.c | 41 + trace-events | 1 + 2 files changed, 42 insertions(+) diff --git a/net/colo-nic.c b/net/colo-nic.c index

[Qemu-devel] [PATCH COLO-Frame v5 01/29] configure: Add parameter for configure to enable/disable COLO support

2015-05-21 Thread zhanghailiang
configure --enable-colo/--disable-colo to switch COLO support on/off. COLO support is off by default. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com Signed-off-by: Gonglei arei.gong...@huawei.com Signed-off-by: Lai Jiangshan la

[Qemu-devel] [PATCH COLO-Frame v5 18/29] COLO NIC: Init/remove colo nic devices when add/cleanup tap devices

2015-05-21 Thread zhanghailiang
When COLO mode, we do some init work for nic that will be used for COLO. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- include/net/colo-nic.h | 20 ++ net/Makefile.objs | 1 + net/colo-nic.c | 71

[Qemu-devel] [PATCH COLO-Frame v5 19/29] COLO NIC: Implement colo nic device interface configure()

2015-05-21 Thread zhanghailiang
Implement colo nic device interface configure() add a script to configure nic devices: ${QEMU_SCRIPT_DIR}/colo-proxy-script.sh Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- net/colo-nic.c | 56

[Qemu-devel] [PATCH COLO-Frame v5 26/29] COLO NIC: Implement NIC checkpoint and failover

2015-05-21 Thread zhanghailiang
Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- include/net/colo-nic.h | 2 ++ migration/colo.c | 21 ++--- net/colo-nic.c | 23 +++ 3 files changed, 43 insertions(+), 3 deletions

[Qemu-devel] [PATCH COLO-Frame v5 23/29] COLO: Do checkpoint according to the result of packets comparation

2015-05-21 Thread zhanghailiang
Only do checkpoint, when the PVM's and SVM's output net packets are inconsistent, We also limit the min time between two continuous checkpoint action, to give VM a change to run. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com

[Qemu-devel] [PATCH COLO-Frame v5 13/29] COLO RAM: Flush cached RAM into SVM's memory

2015-05-21 Thread zhanghailiang
RAM cache into SVM's MEMORY, we do this in a more efficient way: Only flush any page that dirtied by PVM or SVM since last checkpoint. In this way, we ensure SVM's memory same with PVM's. Besides, we must ensure flush RAM cache before load device state. Signed-off-by: zhanghailiang

[Qemu-devel] [PATCH COLO-Frame v5 03/29] COLO: migrate colo related info to slave

2015-05-21 Thread zhanghailiang
We can know if VM in destination should go into COLO mode by refer to the info that has been migrated from PVM. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com Signed-off-by: Gonglei

[Qemu-devel] [PATCH COLO-Frame v5 10/29] COLO RAM: Load PVM's dirty page into SVM's RAM cache temporarily

2015-05-21 Thread zhanghailiang
-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Gonglei arei.gong...@huawei.com Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- arch_init.c| 87

[Qemu-devel] [PATCH COLO-Frame v5 09/29] COLO: Save VM state to slave when do checkpoint

2015-05-21 Thread zhanghailiang
-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com Signed-off-by: Gonglei arei.gong...@huawei.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- arch_init.c | 48

[Qemu-devel] [PATCH COLO-Frame v5 20/29] COLO NIC : Implement colo nic init/destroy function

2015-05-21 Thread zhanghailiang
When in colo mode, call colo nic init/destroy function. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- include/migration/migration-colo.h | 2 +- include/net/colo-nic.h | 3 ++ migration/colo.c

[Qemu-devel] [PATCH COLO-Frame v5 12/29] arch_init: Start to trace dirty pages of SVM

2015-05-21 Thread zhanghailiang
we will use this dirty bitmap together with VM's cache RAM dirty bitmap to decide which page in cache should be flushed into VM's RAM. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com --- arch_init.c | 16 1 file changed, 16 insertions(+) diff --git a/arch_init.c b

[Qemu-devel] [PATCH COLO-Frame v5 06/29] COLO: Implement colo checkpoint protocol

2015-05-21 Thread zhanghailiang
Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com Signed-off-by: Gonglei arei.gong...@huawei.com --- migration/colo.c | 237 ++- 1 file changed, 235 insertions(+), 2 deletions(-) diff

[Qemu-devel] [PATCH COLO-Frame v5 24/29] COLO: Improve checkpoint efficiency by do additional periodic checkpoint

2015-05-21 Thread zhanghailiang
). Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com --- migration/colo.c | 29 + 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/migration/colo.c b/migration/colo.c index a7cead9..195973a 100644

[Qemu-devel] [PATCH COLO-Frame v5 11/29] COLO VMstate: Load VM state into qsb before restore it

2015-05-21 Thread zhanghailiang
, there will be inconsistent for the value of device state. This will happen when the PVM reboot or SVM run ahead of PVM in the startup process. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com Signed-off-by: Gonglei arei.gong...@huawei.com

[Qemu-devel] [PATCH COLO-Frame v5 27/29] COLO: Disable qdev hotplug when VM is in COLO mode

2015-05-21 Thread zhanghailiang
COLO do not support qdev hotplug migration, disable it. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com --- migration/colo.c | 11 +++ 1 file changed, 11 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index

[Qemu-devel] [PATCH COLO-Frame v5 21/29] COLO NIC: Some init work related with proxy module

2015-05-21 Thread zhanghailiang
-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- configure | 22 +++- net/colo-nic.c | 156 + 2 files changed, 176 insertions(+), 2 deletions(-) diff --git a/configure b/configure

[Qemu-devel] [PATCH COLO-Frame v5 16/29] COLO failover: Don't do failover during loading VM's state

2015-05-21 Thread zhanghailiang
Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com --- migration/colo.c | 10 ++ 1 file changed, 10 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index 1c8cdfe

Re: [Qemu-devel] [RFC PATCH v4 23/28] COLO: Improve checkpoint efficiency by do additional periodic checkpoint

2015-05-19 Thread zhanghailiang
On 2015/5/19 0:48, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: Besides normal checkpoint which according to the result of net packets comparing, We do additional checkpoint periodically, it will reduce the number of dirty pages when do one checkpoint

Re: [Qemu-devel] [RFC PATCH v4 09/28] COLO: Save VM state to slave when do checkpoint

2015-05-18 Thread zhanghailiang
On 2015/5/18 20:10, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: On 2015/5/15 20:09, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: We should save PVM's RAM/device to slave when needed. For VM state, we

Re: [Qemu-devel] [RFC PATCH v4 09/28] COLO: Save VM state to slave when do checkpoint

2015-05-18 Thread zhanghailiang
On 2015/5/15 20:09, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: We should save PVM's RAM/device to slave when needed. For VM state, we will cache them in slave, we use QEMUSizedBuffer to store the data, we need know the data size of VM state, so

Re: [Qemu-devel] [RFC PATCH v4 08/28] QEMUSizedBuffer: Introduce two help functions for qsb

2015-05-17 Thread zhanghailiang
On 2015/5/15 19:56, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: Introduce two new QEMUSizedBuffer APIs which will be used by COLO to buffer VM state: One is qsb_put_buffer(), which put the content of a given QEMUSizedBuffer into QEMUFile, this is used

Re: [Qemu-devel] [RFC PATCH v4 03/28] COLO: migrate colo related info to slave

2015-05-17 Thread zhanghailiang
On 2015/5/15 19:38, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: We can know if VM in destination should go into COLO mode by refer to the info that has been migrated from PVM. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off

Re: [Qemu-devel] [RFC PATCH v4 00/28] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service

2015-05-14 Thread zhanghailiang
On 2015/5/14 20:14, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: This is the 4th version of COLO, here is only COLO frame part, include: VM checkpoint, failover, proxy API, block replication API, not include block replication. The block part has been

[Qemu-devel] [PATCH] sheepdog: fix resource leak with sd_snapshot_create

2015-05-04 Thread zhanghailiang
Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com --- block/sheepdog.c | 1 + 1 file changed, 1 insertion(+) diff --git a/block/sheepdog.c b/block/sheepdog.c index 2d5f06a..bd7cbed 100644 --- a/block/sheepdog.c +++ b/block/sheepdog.c @@ -2341,6 +2341,7 @@ static int sd_snapshot_create

Re: [Qemu-devel] [RFC PATCH v4 00/28] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service

2015-04-28 Thread zhanghailiang
On 2015/4/24 16:35, Dr. David Alan Gilbert wrote: * Wen Congyang (we...@cn.fujitsu.com) wrote: On 04/22/2015 07:18 PM, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: Hi, ping ... I will get to look at this again; but not until after next week

Re: [Qemu-devel] [RFC PATCH v4 00/28] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service

2015-04-24 Thread zhanghailiang
On 2015/4/22 19:18, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: Hi, ping ... I will get to look at this again; but not until after next week. OK, thanks for your reply. :) The main blocked bugs for COLO have been solved, I've got the v3 set

Re: [Qemu-devel] [PATCH] vhost: fix log base address

2015-04-15 Thread zhanghailiang
On 2015/4/15 17:37, Michael S. Tsirkin wrote: VHOST_SET_LOG_BASE got an incorrect address, causing migration errors and potentially even memory corruption. Cc: Peter Maydell peter.mayd...@linaro.org Reported-by: Wen Congyang we...@cn.fujitsu.com Signed-off-by: Michael S. Tsirkin m...@redhat.com

Re: [Qemu-devel] [RFC PATCH v4 00/28] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service

2015-04-08 Thread zhanghailiang
into this frame patch set before it been totally reviewed. COLO is a totally new feature which is still in early stage, we hope to speed up the development, so your comments and feedback are warmly welcomed. :) Thanks, zhanghailiang On 2015/3/26 13:29, zhanghailiang wrote: This is the 4th

Re: [Qemu-devel] [Migration Bug? ] Occasionally, the content of VM's memory is inconsistent between Source and Destination of migration

2015-04-03 Thread zhanghailiang
On 2015/4/3 16:51, Jason Wang wrote: On 04/02/2015 07:52 PM, zhanghailiang wrote: On 2015/4/1 3:06, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: On 2015/3/30 15:59, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote

Re: [Qemu-devel] [Migration Bug? ] Occasionally, the content of VM's memory is inconsistent between Source and Destination of migration

2015-04-02 Thread zhanghailiang
On 2015/4/1 3:06, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: On 2015/3/30 15:59, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: On 2015/3/27 18:18, Dr. David Alan Gilbert wrote: * zhanghailiang

Re: [Qemu-devel] [Migration Bug? ] Occasionally, the content of VM's memory is inconsistent between Source and Destination of migration

2015-03-31 Thread zhanghailiang
On 2015/3/30 15:59, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: On 2015/3/27 18:18, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: On 2015/3/26 11:52, Li Zhijian wrote: On 03/26/2015 11:12 AM, Wen Congyang wrote

Re: [Qemu-devel] [Migration Bug? ] Occasionally, the content of VM's memory is inconsistent between Source and Destination of migration

2015-03-28 Thread zhanghailiang
On 2015/3/27 18:18, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: On 2015/3/26 11:52, Li Zhijian wrote: On 03/26/2015 11:12 AM, Wen Congyang wrote: On 03/25/2015 05:50 PM, Juan Quintela wrote: zhanghailiangzhang.zhanghaili...@huawei.com wrote: Hi all

Re: [Qemu-devel] [Migration Bug? ] Occasionally, the content of VM's memory is inconsistent between Source and Destination of migration

2015-03-27 Thread zhanghailiang
On 2015/3/26 11:52, Li Zhijian wrote: On 03/26/2015 11:12 AM, Wen Congyang wrote: On 03/25/2015 05:50 PM, Juan Quintela wrote: zhanghailiangzhang.zhanghaili...@huawei.com wrote: Hi all, We found that, sometimes, the content of VM's memory is inconsistent between Source side and Destination

Re: [Qemu-devel] [Migration Bug? ] Occasionally, the content of VM's memory is inconsistent between Source and Destination of migration

2015-03-27 Thread zhanghailiang
On 2015/3/27 18:51, Juan Quintela wrote: zhanghailiang zhang.zhanghaili...@huawei.com wrote: On 2015/3/26 11:52, Li Zhijian wrote: On 03/26/2015 11:12 AM, Wen Congyang wrote: On 03/25/2015 05:50 PM, Juan Quintela wrote: zhanghailiangzhang.zhanghaili...@huawei.com wrote: Hi all, We found

[Qemu-devel] [Migration Bug? ] Occasionally, the content of VM's memory is inconsistent between Source and Destination of migration

2015-03-25 Thread zhanghailiang
pages that not transferred to destination when do migration ? Or is it a bug? This issue has blocked our COLO development... :( Any help will be greatly appreciated! Thanks, zhanghailiang --- a/savevm.c +++ b/savevm.c @@ -51,6 +51,26 @@ #define ARP_PTYPE_IP 0x0800 #define ARP_OP_REQUEST_REV 0x3

Re: [Qemu-devel] [Migration Bug? ] Occasionally, the content of VM's memory is inconsistent between Source and Destination of migration

2015-03-25 Thread zhanghailiang
On 2015/3/25 17:46, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: Hi all, We found that, sometimes, the content of VM's memory is inconsistent between Source side and Destination side when we check it just after finishing migration but before VM

Re: [Qemu-devel] [Migration Bug? ] Occasionally, the content of VM's memory is inconsistent between Source and Destination of migration

2015-03-25 Thread zhanghailiang
On 2015/3/25 17:50, Juan Quintela wrote: zhanghailiang zhang.zhanghaili...@huawei.com wrote: Hi all, We found that, sometimes, the content of VM's memory is inconsistent between Source side and Destination side when we check it just after finishing migration but before VM continue to Run. We

Re: [Qemu-devel] [Migration Bug? ] Occasionally, the content of VM's memory is inconsistent between Source and Destination of migration

2015-03-25 Thread zhanghailiang
On 2015/3/25 19:36, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: On 2015/3/25 17:46, Dr. David Alan Gilbert wrote: * zhanghailiang (zhang.zhanghaili...@huawei.com) wrote: Hi all, We found that, sometimes, the content of VM's memory is inconsistent

[Qemu-devel] [RFC PATCH v4 11/28] COLO VMstate: Load VM state into qsb before restore it

2015-03-25 Thread zhanghailiang
We should cache the device state to ensure the data is intact before restore it. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com Signed-off-by: Gonglei arei.gong...@huawei.com --- migration/colo.c | 51

[Qemu-devel] [RFC PATCH v4 00/28] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service

2015-03-25 Thread zhanghailiang
Blake and Dr. David Alan Gilbert v1: - implement the frame of colo Wen Congyang (1): COLO: Add block replication into colo process zhanghailiang (27): configure: Add parameter for configure to enable/disable COLO support migration: Introduce capability 'colo' to migration COLO: migrate

[Qemu-devel] [RFC PATCH v4 04/28] migration: Integrate COLO checkpoint process into migration

2015-03-25 Thread zhanghailiang
Add a migrate state: MIGRATION_STATUS_COLO, enter this migration state after the first live migration successfully finished. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Gonglei arei.gong...@huawei.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com --- include

[Qemu-devel] [RFC PATCH v4 08/28] QEMUSizedBuffer: Introduce two help functions for qsb

2015-03-25 Thread zhanghailiang
into qsb, this is used to get VM state from socket into a buffer. Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com --- include/migration/qemu-file.h | 3 ++- migration/qemu-file-buf.c | 58

[Qemu-devel] [RFC PATCH v4 12/28] arch_init: Start to trace dirty pages of SVM

2015-03-25 Thread zhanghailiang
we will use this dirty bitmap together with VM's cache RAM dirty bitmap to decide which page in cache should be flushed into VM's RAM. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com --- arch_init.c | 15 +++ 1 file changed, 15 insertions(+) diff --git a/arch_init.c b

[Qemu-devel] [RFC PATCH v4 07/28] COLO: Add a new RunState RUN_STATE_COLO

2015-03-25 Thread zhanghailiang
Guest will enter this state when paused to save/restore VM state under colo checkpoint. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Gonglei arei.gong...@huawei.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com --- qapi-schema.json | 5 - vl.c

[Qemu-devel] [RFC PATCH v4 01/28] configure: Add parameter for configure to enable/disable COLO support

2015-03-25 Thread zhanghailiang
configure --enable-colo/--disable-colo to switch COLO support on/off. COLO support is on by default. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com Signed-off-by: Gonglei arei.gong...@huawei.com Signed-off-by: Lai Jiangshan la

[Qemu-devel] [RFC PATCH v4 23/28] COLO: Improve checkpoint efficiency by do additional periodic checkpoint

2015-03-25 Thread zhanghailiang
). Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com --- migration/colo.c | 29 + 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/migration/colo.c b/migration/colo.c index 9ef4554..da5bc5e 100644

[Qemu-devel] [RFC PATCH v4 09/28] COLO: Save VM state to slave when do checkpoint

2015-03-25 Thread zhanghailiang
-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com Signed-off-by: Gonglei arei.gong...@huawei.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- arch_init.c | 22 ++-- migration/colo.c

[Qemu-devel] [RFC PATCH v4 27/28] COLO: Implement shutdown checkpoint

2015-03-25 Thread zhanghailiang
For SVM, we forbid it shutdown directly when in COLO mode, FOR PVM's shutdown, we should do some work to ensure the consistent action between PVM and SVM. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com Signed-off-by: Li Zhijian lizhij

[Qemu-devel] [RFC PATCH v4 05/28] migration: Integrate COLO checkpoint process into loadvm

2015-03-25 Thread zhanghailiang
Switch from normal migration loadvm process into COLO checkpoint process if COLO mode is enabled. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com Signed-off-by: Yang Hongyang yan

[Qemu-devel] [RFC PATCH v4 02/28] migration: Introduce capability 'colo' to migration

2015-03-25 Thread zhanghailiang
-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com Signed-off-by: Gonglei arei.gong...@huawei.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com --- include/migration/migration-colo.h | 20 include/migration/migration.h

[Qemu-devel] [RFC PATCH v4 19/28] COLO NIC: Implement colo nic device interface configure()

2015-03-25 Thread zhanghailiang
Implement colo nic device interface configure() add a script to configure nic devices: ${QEMU_SCRIPT_DIR}/colo-proxy-script.sh Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- net/colo-nic.c | 56

[Qemu-devel] [RFC PATCH v4 18/28] COLO NIC: Init/remove colo nic devices when add/cleanup tap devices

2015-03-25 Thread zhanghailiang
When COLO mode, we will do some init work for nic that will be used for COLO. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- include/net/colo-nic.h | 20 ++ net/Makefile.objs | 1 + net/colo-nic.c | 73

[Qemu-devel] [RFC PATCH v4 25/28] COLO NIC: Implement NIC checkpoint and failover

2015-03-25 Thread zhanghailiang
Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- include/net/colo-nic.h | 3 ++- migration/colo.c | 22 +++--- net/colo-nic.c | 19 +++ 3 files changed, 40 insertions(+), 4 deletions

[Qemu-devel] [RFC PATCH v4 26/28] COLO: Disable qdev hotplug when VM is in COLO mode

2015-03-25 Thread zhanghailiang
COLO do not support qdev hotplug migration, disable it. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com --- migration/colo.c | 11 +++ 1 file changed, 11 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index

[Qemu-devel] [RFC PATCH v4 28/28] COLO: Add block replication into colo process

2015-03-25 Thread zhanghailiang
From: Wen Congyang we...@cn.fujitsu.com Make sure master start block replication after slave's block replication started Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Wen Congyang we...@cn.fujitsu.com Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com --- migration

[Qemu-devel] [RFC PATCH v4 13/28] COLO RAM: Flush cached RAM into SVM's memory

2015-03-25 Thread zhanghailiang
RAM cache into SVM's MEMORY, we do this in a more efficient way: Only flush any page that dirtied by PVM or SVM since last checkpoint. In this way, we ensure SVM's memory same with PVM's. Besides, we must ensure flush RAM cache before load device state. Signed-off-by: zhanghailiang

[Qemu-devel] [RFC PATCH v4 24/28] COLO: Add colo-set-checkpoint-period command

2015-03-25 Thread zhanghailiang
With this command, we can control the period of checkpoint, if there is no comparison of net packets. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- hmp-commands.hx| 15 +++ hmp.c | 7

[Qemu-devel] [RFC PATCH v4 22/28] COLO: Do checkpoint according to the result of net packets comparing

2015-03-25 Thread zhanghailiang
Only do checkpoint, when the VMs' output net packets are inconsistent, We also limit the min time between two continuous checkpoint action, to give VM a change to run. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- include/net

[Qemu-devel] [RFC PATCH v4 20/28] COLO NIC : Implement colo nic init/destroy function

2015-03-25 Thread zhanghailiang
When in colo mode, call colo nic init/destroy function. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- include/net/colo-nic.h | 2 ++ migration/colo.c | 17 +++ net/colo-nic.c | 80

[Qemu-devel] [RFC PATCH v4 17/28] COLO: Add new command parameter 'colo_nicname' 'colo_script' for net

2015-03-25 Thread zhanghailiang
The 'colo_nicname' should be assigned with network name, for exmple, 'eth2'. It will be parameter of 'colo_script', 'colo_script' should be assigned with an scirpt path. We parse these parameter in tap. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij

[Qemu-devel] [RFC PATCH v4 10/28] COLO RAM: Load PVM's dirty page into SVM's RAM cache temporarily

2015-03-25 Thread zhanghailiang
-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Gonglei arei.gong...@huawei.com Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- arch_init.c| 70

[Qemu-devel] [RFC PATCH v4 16/28] COLO failover: Don't do failover during loading VM's state

2015-03-25 Thread zhanghailiang
Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com --- migration/colo.c | 10 +- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/migration/colo.c b/migration/colo.c

[Qemu-devel] [RFC PATCH v4 03/28] COLO: migrate colo related info to slave

2015-03-25 Thread zhanghailiang
We can know if VM in destination should go into COLO mode by refer to the info that has been migrated from PVM. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Yang Hongyang yan...@cn.fujitsu.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com Signed-off-by: Gonglei

[Qemu-devel] [RFC PATCH v4 14/28] COLO failover: Introduce a new command to trigger a failover

2015-03-25 Thread zhanghailiang
, and takeover, if to Secondary, Secondary will do failover work and at last takeover server. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com Signed-off-by: Yang Hongyang yan

[Qemu-devel] [RFC PATCH v4 15/28] COLO failover: Implement COLO master/slave failover work

2015-03-25 Thread zhanghailiang
If failover is requested, after some cleanup work, PVM or SVM will exit COLO mode, and resume to normal run. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com Signed-off-by: Lai Jiangshan la...@cn.fujitsu.com --- include/migration

[Qemu-devel] [RFC PATCH v4 06/28] COLO: Implement colo checkpoint protocol

2015-03-25 Thread zhanghailiang
Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com Signed-off-by: Gonglei arei.gong...@huawei.com --- migration/colo.c | 237 ++- 1 file changed, 234 insertions(+), 3 deletions(-) diff

[Qemu-devel] [RFC PATCH v4 21/28] COLO NIC: Some init work related with proxy module

2015-03-25 Thread zhanghailiang
Implement communication protocol with proxy module by using netlink, and do some init work. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Li Zhijian lizhij...@cn.fujitsu.com --- net/colo-nic.c | 171 + 1 file

Re: [Qemu-devel] [PATCH v2] qga/commands-posix: Fix resource leak

2015-03-15 Thread zhanghailiang
On 2015/3/14 17:52, Shannon Zhao wrote: It's detected by coverity. Close the dirfd. Reviewed-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Shannon Zhao zhaoshengl...@huawei.com Signed-off-by: Shannon Zhao shannon.z...@linaro.org --- v1-v2: close after use [Stefan Weil

Re: [Qemu-devel] [PATCH v4 3/4] migration: Convert 'status' of MigrationInfo to use an enum type

2015-03-13 Thread zhanghailiang
On 2015/3/13 3:37, Eric Blake wrote: On 03/09/2015 12:45 AM, zhanghailiang wrote: The original 'status' is an open-coded 'str' type, convert it to use an enum type. This conversion is backwards compatible, better documented and more convenient for future extensibility. In addition, Fix a typo

[Qemu-devel] [PATCH v5 for-2.3 4/4] migration: Expose 'cancelling' status to user

2015-03-13 Thread zhanghailiang
, but that the migration is still properly cancelled. Newer libvirt will be patched to recognize the new state without the odd error message. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Reviewed-by: Eric Blake ebl...@redhat.com Cc: libvir-l...@redhat.com --- migration/migration.c | 6 +- 1

[Qemu-devel] [PATCH v5 for-2.3 0/4] Convert 'status' of MigrationInfo from open-coded 'str' to enum type

2015-03-13 Thread zhanghailiang
Hi Juan Amit, This series has been reviewed, and could you help merging them ? Thanks, zhanghailiang This series converts open-coded 'str' type to enum type for 'status'. This conversion will be more convenient for future extensibility. Actually, I will add a MIG_STATE_COLO state for COLO

[Qemu-devel] [PATCH v5 for-2.3 1/4] migration: Rename abbreviated macro MIG_STATE_* to MIGRATION_STATUS_*

2015-03-13 Thread zhanghailiang
Rename all macro MIG_STATE_* to MIGRATION_STATUS_* except MIG_STATE_ERROR, we rename it to MIGRATION_STATUS_FAILED which will match the migration status string 'failed'. Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Reviewed-by: Eric Blake ebl...@redhat.com --- migration

<    5   6   7   8   9   10   11   12   13   14   >