[Qemu-devel] [PATCH RFC v2 4/5] filter-buffer: Accept zero interval

2016-01-27 Thread zhanghailiang
We may want to accept zero interval when VM FT solutions like MC or COLO use this filter to release packets on demand. Signed-off-by: zhanghailiang Reviewed-by: Yang Hongyang --- net/filter-buffer.c | 10 -- 1 file changed, 10 deletions(-) diff --git a/net/filter-buffer.c b/net/filter

[Qemu-devel] [PATCH RFC v2 1/5] net/filter: Add a 'status' property for filter object

2016-01-27 Thread zhanghailiang
With this property, users can control if this filter is 'enable' or 'disable'. The default behavior for filter is enabled. We will skip the disabled filter when delivering packets in net layer. Signed-off-by: zhanghailiang --- v2: - Squash previous patch 3 into this patch

[Qemu-devel] [PATCH RFC v2 0/5] Netfilter: Add each netdev a default filter

2016-01-27 Thread zhanghailiang
o network' v2: - Drop the patch 'net/filter: prevent the default filter to be deleted' (Jason) - Re-implement netdev_add_filter() by re-using object_object() (Jason) - Send patch 'net/filter: Fix the output information for command 'info network' as an independen

[Qemu-devel] [PATCH v2] net/filter: Fix the output information for command 'info network'

2016-01-25 Thread zhanghailiang
e we split a helper function that could collect the output information for filter, and also remove the useless member 'info_str' from struct NetFilterState. Signed-off-by: zhanghailiang Cc: Jason Wang Cc: Eric Blake Cc: Markus Armbruster Cc: Yang Hongyang --- v2: - write the info

[Qemu-devel] [PATCH] net/filter: Fix the output information for command 'info network'

2016-01-25 Thread zhanghailiang
e we split a helper function that could colletct the output information for filter, and also remove the useless member 'info_str' from struct NetFilterState. Signed-off-by: zhanghailiang Cc: Jason Wang Cc: Eric Blake Cc: Markus Armbruster Cc: Yang Hongyang --- include/net/filter

[Qemu-devel] [PATCH RFC 6/7] net/filter: Add a default filter to each netdev

2016-01-22 Thread zhanghailiang
we hope to support hot add network during COLO state in future. Signed-off-by: zhanghailiang --- include/net/filter.h | 11 +++ net/dump.c | 2 -- net/filter.c | 15 ++- net/net.c| 18 ++ 4 files changed, 43 insertions(+), 3

[Qemu-devel] [PATCH RFC 4/7] net/filter: Introduce a helper to add a filter to the netdev

2016-01-22 Thread zhanghailiang
Signed-off-by: zhanghailiang --- include/net/filter.h | 5 + net/filter.c | 63 2 files changed, 68 insertions(+) diff --git a/include/net/filter.h b/include/net/filter.h index d797ee4..c7bd8f9 100644 --- a/include/net/filter.h

[Qemu-devel] [PATCH RFC 5/7] filter-buffer: Accept zero interval

2016-01-22 Thread zhanghailiang
We may want to accept zero interval when VM FT solutions like MC or COLO use this filter to release packets on demand. Signed-off-by: zhanghailiang Reviewed-by: Yang Hongyang --- net/filter-buffer.c | 10 -- 1 file changed, 10 deletions(-) diff --git a/net/filter-buffer.c b/net/filter

[Qemu-devel] [PATCH RFC 3/7] net/filter: Skip the disabled filter when delivering packets

2016-01-22 Thread zhanghailiang
If the filter is disabled, don't go through it. Signed-off-by: zhanghailiang --- include/net/filter.h | 5 + net/net.c| 4 2 files changed, 9 insertions(+) diff --git a/include/net/filter.h b/include/net/filter.h index 9ed5ec6..d797ee4 100644 --- a/include/net/fil

[Qemu-devel] [PATCH RFC 2/7] net/filter: Add a 'status' property for filter object

2016-01-22 Thread zhanghailiang
With this property, users can control if this filter is 'enable' or 'disable'. The default behavior for filter is enabled. Signed-off-by: zhanghailiang --- include/net/filter.h | 1 + net/filter.c | 36 2 files changed, 37 inser

[Qemu-devel] [PATCH RFC 1/7] net/filter: Fix the output information for command 'info network'

2016-01-22 Thread zhanghailiang
split a the helper function that could colletct the output information for filter, and also remove the useless member 'info_str' from struct NetFilterState. Signed-off-by: zhanghailiang --- include/net/filter.h | 3 ++- net/filter.c | 47 ++--

[Qemu-devel] [PATCH RFC 7/7] net/filter: prevent the default filter to be deleted

2016-01-22 Thread zhanghailiang
Signed-off-by: zhanghailiang --- net/filter.c | 8 1 file changed, 8 insertions(+) diff --git a/net/filter.c b/net/filter.c index a126a3b..4aafff0 100644 --- a/net/filter.c +++ b/net/filter.c @@ -323,11 +323,19 @@ static void netfilter_finalize(Object *obj) g_free(nf->netdev

[Qemu-devel] [PATCH RFC 0/7] Netfilter: Add each netdev a default filter

2016-01-22 Thread zhanghailiang
This series is a prerequisite for COLO, here we add each netdev a default buffer filter, it is disabled by default, and has no side effect for delivering packets in net layer. Besides, patch 1 fixes the ouput information of 'info network' command for filter. zhanghailiang (7): net/f

[Qemu-devel] [PATCH 5/6] qmp-commands.hx: Fix the missing options for migration parameters commands

2016-01-14 Thread zhanghailiang
at we can't set x-cpu-throttle-initial and x-cpu-throttle-increment through migrate-set-parameters qmp command. Signed-off-by: zhanghailiang --- qmp-commands.hx | 18 ++ 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/qmp-commands.hx b/qmp-commands.hx ind

[Qemu-devel] [PATCH 3/6] savevm: Split load vm state function qemu_loadvm_state

2016-01-14 Thread zhanghailiang
qemu_loadvm_state is too long, and we can simplify it by splitting up with three helper functions. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- migration/savevm.c | 156 +++-- 1 file changed, 92 insertions(+), 64 deletions

[Qemu-devel] [PATCH 4/6] migration/ram: Fix some helper functions' parameter to use PageSearchStatus

2016-01-14 Thread zhanghailiang
helper functions. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 33 +++-- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 8583da8..c36daea 100644 --- a/migration/ram.c +

[Qemu-devel] [PATCH 1/6] ram: Split host_from_stream_offset() into two helper functions

2016-01-14 Thread zhanghailiang
: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- include/exec/ram_addr.h | 8 ++-- migration/ram.c | 40 +--- 2 files changed, 31 insertions(+), 17 deletions(-) diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h index ef1489d..606e277 100644

[Qemu-devel] [PATCH 0/6] Some improvements and small fixes for migration

2016-01-14 Thread zhanghailiang
Patch 1 ~ patch 4 are picked from COLO and live memory snapshot series, They are just small improvements for migration codes and have been reviewed by Dave. Patch 5, 6 are small fixes for migration releated documention. Please review. zhanghailiang (6): ram: Split host_from_stream_offset

[Qemu-devel] [PATCH 6/6] qmp-commands.hx: Document the missing options for migration capability commands

2016-01-14 Thread zhanghailiang
Add the missing descriptions for the options of migration capability commands, and fix the example for query-migrate-capabilities command. Signed-off-by: zhanghailiang --- qmp-commands.hx | 15 ++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/qmp-commands.hx b/qmp

[Qemu-devel] [PATCH 2/6] migration: Rename the'file' member of MigrationState

2016-01-14 Thread zhanghailiang
Rename the 'file' member of MigrationState to 'to_dst_file'. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- include/migration/migration.h | 2 +- migration/exec.c | 4 +-- migration/fd.c| 4 +-- migration/migr

[Qemu-devel] [RFC 12/13] migration/ram: Fix some helper functions' parameter to use PageSearchStatus

2016-01-07 Thread zhanghailiang
helper functions. Signed-off-by: zhanghailiang --- migration/ram.c | 33 +++-- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index fc4c788..8656719 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -726,7 +726,

[Qemu-devel] [RFC 13/13] snapshot: Remove page's write-protect and copy the content during setup stage

2016-01-07 Thread zhanghailiang
rite-protect in fault thread during the setup stage. Besides, we should not try to get global lock after setup stage, or there maybe an deadlock error. Signed-off-by: zhanghailiang --- include/migration/migration.h | 4 ++-- migration/migration.c | 2 +- migration/postcopy-

[Qemu-devel] [RFC 11/13] snapshot/migration: Save VM's RAM into snapshot file

2016-01-07 Thread zhanghailiang
emove the write-protect. In this way, we can ensure, the content of the page in snapshot file is same with the time we got snapshot command. Signed-off-by: zhanghailiang --- include/migration/postcopy-ram.h | 4 migration/migration.c| 17 +++-- migration/postcopy-ram

[Qemu-devel] [RFC 08/13] snapshot: Save VM's device state into snapshot file

2016-01-07 Thread zhanghailiang
ile after finishing save VM's live state. Signed-off-by: zhanghailiang --- include/sysemu/sysemu.h | 3 +++ migration/migration.c | 14 -- migration/savevm.c | 44 3 files changed, 59 insertions(+), 2 deletions(-) diff --git a/incl

[Qemu-devel] [RFC 00/13] Live memory snapshot based on userfaultfd

2016-01-07 Thread zhanghailiang
re taking snapshot, please disable THP by: echo never > /sys/kernel/mm/transparent_hugepage/enabled TODO: - Reduce the influence for VM while taking snapshot zhanghailiang (13): postcopy/migration: Split fault related state into struct UserfaultState migration: Allow the migrate command to work on

[Qemu-devel] [RFC 02/13] migration: Allow the migrate command to work on file: urls

2016-01-07 Thread zhanghailiang
Usage: (qemu) migrate file:/path/to/vm_statefile Signed-off-by: zhanghailiang Signed-off-by: Benoit Canet --- - With this patch, we can easily test memory snapshot - Rebase on qemu 2.5 --- include/migration/migration.h | 6 +- migration/fd.c| 19

[Qemu-devel] [RFC 01/13] postcopy/migration: Split fault related state into struct UserfaultState

2016-01-07 Thread zhanghailiang
Split fault related state from MigrationIncomingState struct, and put them all into a new struct UserfaultState. We will add this state into struct MigrationState in later patch. We also fix some helper functions to use the new type. Signed-off-by: zhanghailiang --- include/migration

[Qemu-devel] [PATCH COLO-Frame v13 38/39] colo: Use default buffer-filter to buffer and release packets

2015-12-28 Thread zhanghailiang
Enable default filter to buffer packets and release the packets after a checkpoint. Signed-off-by: zhanghailiang Cc: Jason Wang Cc: Yang Hongyang --- v12: - Add a helper function to check if all netdev supports buffer packets. - Flush buffered packets when do failover. v11: - Use new helper

[Qemu-devel] [PATCH COLO-Frame v13 36/39] filter-buffer: Introduce a helper function to enable/disable default filter

2015-12-28 Thread zhanghailiang
The default buffer filter doesn't buffer packets in default, but we need to buffer packets for COLO or Micro-checkpoint, Here we add a helper function to enable/disable filter's buffer capability. Signed-off-by: zhanghailiang Cc: Jason Wang Cc: Yang Hongyang --- v12: - Rename

[Qemu-devel] [PATCH COLO-Frame v13 26/39] COLO failover: Shutdown related socket fd when do failover

2015-12-28 Thread zhanghailiang
s to wake up the blocking operation in failover BH. Besides, we should close the corresponding file descriptors after failvoer BH shutdown them, or there will be an error. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gilbert Cc: Dr. David Alan Gilbert --- v13: -

[Qemu-devel] [PATCH COLO-Frame v13 35/39] filter-buffer: Accept zero interval

2015-12-28 Thread zhanghailiang
For default buffer filter, its 'interval' value is zero, so here we should accept zero interval. Signed-off-by: zhanghailiang Reviewed-by: Yang Hongyang Cc: Jason Wang --- v12: - Add Reviewed-by tag v11: - Add comment v10: - new patch --- net/filter-buffer.c | 10 -- 1 fi

[Qemu-devel] [PATCH COLO-Frame v13 37/39] filter-buffer: Introduce a helper function to release packets

2015-12-28 Thread zhanghailiang
We need to release all the packets from VM in COLO or Micro-checkpoint, here we add a new helper function to realse the packets that buffered by default buffer-filter Signed-off-by: zhanghailiang Cc: Jason Wang Cc: Yang Hongyang --- v12: - Rename this helper function v11: - New patch

[Qemu-devel] [PATCH COLO-Frame v13 29/39] COLO: Update the global runstate after going into colo state

2015-12-28 Thread zhanghailiang
If we start qemu with -S, the runstate will change from 'prelaunch' to 'running' after going into colo state. So it is necessary to update the global runstate after going into colo state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gil

[Qemu-devel] [PATCH COLO-Frame v13 33/39] COLO: Split qemu_savevm_state_begin out of checkpoint process

2015-12-28 Thread zhanghailiang
data transferring in the later checkpoint. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gilbert --- v13: - Fix some minor issues found by Dave - Add Reviewed-by tag --- migration/colo.c | 51 --- 1 file

[Qemu-devel] [PATCH COLO-Frame v13 14/39] ram: Split host_from_stream_offset() into two helper functions

2015-12-28 Thread zhanghailiang
: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- v13: - Add Reviewed-by tag v12: - Remove the offset parameter for ram_block_from_stream() and check the validity of the related value in a new helper. (Dave's suggestion) v11: - New patch --- include/exec/ram_addr.h | 8 ++-- migration/

[Qemu-devel] [PATCH COLO-Frame v13 28/39] COLO: Process shutdown command for VM in COLO state

2015-12-28 Thread zhanghailiang
If VM is in COLO FT state, we should do some extra work before normal shutdown process. SVM will ignore the shutdown command if this command is issued directly to it, PVM will send the shutdown command to SVM if it gets this command. Cc: Paolo Bonzini Signed-off-by: zhanghailiang Signed-off-by

[Qemu-devel] [PATCH COLO-Frame v13 32/39] COLO: Separate the process of saving/loading ram and device state

2015-12-28 Thread zhanghailiang
during checkpoint. Besides, we move the colo_flush_ram_cache to the proper position after the above change. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- v13: - Re-use some existed helper functions to realize saving/loading ram and device. v11: - Remove load configuration section in

[Qemu-devel] [PATCH COLO-Frame v13 25/39] qmp event: Add COLO_EXIT event to notify users while exited from COLO

2015-12-28 Thread zhanghailiang
is still necessary to notify users that we exited COLO mode. Cc: Markus Armbruster Cc: Michael Roth Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- v13: - Remove optional 'error' string for this event. (I doubted it was usefull for users, Since users shouldn't int

[Qemu-devel] [PATCH COLO-Frame v13 11/39] COLO: Add a new RunState RUN_STATE_COLO

2015-12-28 Thread zhanghailiang
Guest will enter this state when paused to save/restore VM state under colo checkpoint. Cc: Eric Blake Cc: Markus Armbruster Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Eric Blake --- qapi-schema.json | 5

[Qemu-devel] [PATCH COLO-Frame v13 27/39] COLO failover: Don't do failover during loading VM's state

2015-12-28 Thread zhanghailiang
We should not do failover work while the main thread is loading VM's state, otherwise it will destroy the consistent of VM's memory and device state. Here we add a new failover status 'RELAUNCH' which means we should relaunch the process of failover. Signed-off-by: zhanghai

[Qemu-devel] [PATCH COLO-Frame v13 34/39] net/filter-buffer: Add default filter-buffer for each netdev

2015-12-28 Thread zhanghailiang
for the netdev. Signed-off-by: zhanghailiang Cc: Jason Wang Cc: Yang Hongyang --- v12: - Skip vhost-net when add default filter - Don't go through filter layer if the filter is disabled. v11: - New patch --- include/net/filter.h | 10 +++ net/f

[Qemu-devel] [PATCH COLO-Frame v13 15/39] COLO: Load PVM's dirty pages into SVM's RAM cache temporarily

2015-12-28 Thread zhanghailiang
7;s state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei Reviewed-by: Dr. David Alan Gilbert --- v12: - Fix minor error in error_report (Dave's comment) - Add Reviewed-by tag v11: - Rename 'host_cache' to 'colo_cache' (Dave's suggestion) v

[Qemu-devel] [PATCH COLO-Frame v13 23/39] COLO: Implement failover work for Primary VM

2015-12-28 Thread zhanghailiang
For PVM, if there is failover request from users. The colo thread will exit the loop while the failover BH does the cleanup work and resumes VM. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gilbert --- v13: - Add Reviewed-by tag v12: - Fix error report and

[Qemu-devel] [PATCH COLO-Frame v13 39/39] COLO: Add block replication into colo process

2015-12-28 Thread zhanghailiang
Make sure master start block replication after slave's block replication started. Signed-off-by: zhanghailiang Signed-off-by: Wen Congyang Signed-off-by: Li Zhijian --- migration/colo.c | 52 trace-events | 2 ++ 2 files change

[Qemu-devel] [PATCH COLO-Frame v13 06/39] migration: Integrate COLO checkpoint process into migration

2015-12-28 Thread zhanghailiang
Add a migrate state: MIGRATION_STATUS_COLO, enter this migration state after the first live migration successfully finished. We reuse migration thread, so if colo is enabled by user, migration thread will go into the process of colo. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian

[Qemu-devel] [PATCH COLO-Frame v13 02/39] migration: Introduce capability 'x-colo' to migration

2015-12-28 Thread zhanghailiang
earn if colo is supported. Cc: Juan Quintela Cc: Amit Shah Cc: Eric Blake Cc: Markus Armbruster Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei Reviewed-by: Eric Blake --- v10: - Rename capability 'colo' to experimental 'x-colo' (Eri

[Qemu-devel] [PATCH COLO-Frame v13 20/39] COLO: synchronize PVM's state to SVM periodically

2015-12-28 Thread zhanghailiang
Do checkpoint periodically, the default interval is 200ms. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gilbert --- v12: - Add Reviewed-by tag v11: - Fix wrong sleep time for checkpoint period. (Dave's review comment) --- migration/colo.c

[Qemu-devel] [PATCH COLO-Frame v13 31/39] savevm: Introduce two helper functions for save/find loadvm_handlers entry

2015-12-28 Thread zhanghailiang
emory leak. We need to check if we have the section info in loadvm_handlers list before save it. For normal migration, it is harmless. Signed-off-by: zhanghailiang --- v13: - New patch --- migration/savevm.c | 56 ++ 1 file changed, 40 insertions(+

[Qemu-devel] [PATCH COLO-Frame v13 30/39] savevm: Split load vm state function qemu_loadvm_state

2015-12-28 Thread zhanghailiang
qemu_loadvm_state is too long, and we can simplify it by splitting up with three helper functions. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert v13: - Add Reviewed-by tag --- migration/savevm.c | 156 +++-- 1 file changed, 92

[Qemu-devel] [PATCH COLO-Frame v13 22/39] COLO failover: Introduce state to record failover process

2015-12-28 Thread zhanghailiang
set the value. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- v11: - fix several typos found by Dave - Add Reviewed-by tag --- include/migration/failover.h | 10 ++ migration/colo-failover.c| 37 + migration/colo.c

[Qemu-devel] [PATCH COLO-Frame v13 24/39] COLO: Implement failover work for Secondary VM

2015-12-28 Thread zhanghailiang
If users require SVM to takeover work, colo incoming thread should exit from loop while failover BH helps backing to migration incoming coroutine. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gilbert v12: - Improve error message that suggested by Dave

[Qemu-devel] [PATCH COLO-Frame v13 09/39] COLO/migration: Create a new communication path from destination to source

2015-12-28 Thread zhanghailiang
This new communication path will be used for returning messages from destination to source. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gilbert --- v13: - Remove useless error report v12: - Add Reviewed-by tag v11: - Rebase master to use

[Qemu-devel] [PATCH COLO-Frame v13 21/39] COLO failover: Introduce a new command to trigger a failover

2015-12-28 Thread zhanghailiang
e PVM, the Primary side will exit COLO mode and take over operation. If sent to the Secondary, the secondary will run failover work, then take over server operation to become the new Primary. Cc: Luiz Capitulino Cc: Eric Blake Cc: Markus Armbruster Signed-off-by: zhanghailiang Signed-off-by:

[Qemu-devel] [PATCH COLO-Frame v13 13/39] COLO: Save PVM state to secondary side when do checkpoint

2015-12-28 Thread zhanghailiang
the size of VM state, so in master, we use qsb to store VM state temporarily, get the data size by call qsb_get_length() and then migrate the data to the qsb in the secondary side. Signed-off-by: zhanghailiang Signed-off-by: Gonglei Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gilbert

[Qemu-devel] [PATCH COLO-Frame v13 18/39] COLO: Flush PVM's cached RAM into SVM's memory

2015-12-28 Thread zhanghailiang
all content of PVM's RAM cache into SVM's MEMORY, we do this in a more efficient way: Only flush any page that dirtied by PVM since last checkpoint. In this way, we can ensure SVM's memory same with PVM's. Besides, we must ensure flush RAM cache before load device state. Si

[Qemu-devel] [PATCH COLO-Frame v13 08/39] migration: Rename the'file' member of MigrationState

2015-12-28 Thread zhanghailiang
Rename the 'file' member of MigrationState to 'to_dst_file'. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- v12: - Add Reviewed-by tag - Add the missed modification for RDMA migration. (Found by Wen Congyang) v11: - Only rename 'file' member

[Qemu-devel] [PATCH COLO-Frame v13 03/39] COLO: migrate colo related info to secondary node

2015-12-28 Thread zhanghailiang
/destination; Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei Reviewed-by: Dr. David Alan Gilbert --- v11: - Add Reviewed-by tag v10: - Use VMSTATE_BOOL instead of VMSTATE_UNIT32 for 'colo_requested' (Dave's suggestion). --- include/migrat

[Qemu-devel] [PATCH COLO-Frame v13 07/39] migration: Integrate COLO checkpoint process into loadvm

2015-12-28 Thread zhanghailiang
_incoming_co' records the original migration incoming coroutine. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gilbert --- v12: - Add Reviewed-by tag v11: - We moved the place of bdrv_invalidate_cache_all(), but done the deleting work in other patch. F

[Qemu-devel] [PATCH COLO-Frame v13 04/39] migration: Export migrate_set_state()

2015-12-28 Thread zhanghailiang
Fix the first parameter of migrate_set_state(), and export it. We will use it in later. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- v12: - Add Reviewed-by tag v11: - New patch which is split from patch 'migration: Add state records for migration incoming

[Qemu-devel] [PATCH COLO-Frame v13 16/39] ram/COLO: Record the dirty pages that SVM received

2015-12-28 Thread zhanghailiang
We record the address of the dirty pages that received, it will help flushing pages that cached into SVM. We record them by re-using migration dirty bitmap. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- v12: - Add Reviewed-by tag v11: - Split a new helper function from

[Qemu-devel] [PATCH COLO-Frame v13 17/39] COLO: Load VMState into qsb before restore it

2015-12-28 Thread zhanghailiang
load VM state, which can ensure the data is intact. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei Reviewed-by: Dr. David Alan Gilbert Cc: Dr. David Alan Gilbert --- v13: - Fix the define of colo_get_cmd_value() to use 'Error **errp' instead of return

[Qemu-devel] [PATCH COLO-Frame v13 05/39] migration: Add state records for migration incoming

2015-12-28 Thread zhanghailiang
For migration destination, we also need to know its state, we will use it in COLO. Here we add a new member 'state' for MigrationIncomingState, and also use migrate_set_state() to modify its value. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- v11: - Split

[Qemu-devel] [PATCH COLO-Frame v13 10/39] COLO: Implement colo checkpoint protocol

2015-12-28 Thread zhanghailiang
int, for which the Secondary VM is not running, later we will support 'hybrid' mode. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei Cc: Eric Blake Cc: Markus Armbruster Cc: Dr. David Alan Gilbert --- v13: - Refactor colo command related helper functions, use &

[Qemu-devel] [PATCH COLO-Frame v13 12/39] QEMUSizedBuffer: Introduce two help functions for qsb

2015-12-28 Thread zhanghailiang
m the file into qsb, this is used to get VM state from socket into a buffer. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gilbert --- v11: - size_t'ify these two help functions (Dave's suggestion) --- include/migration/qemu-file.h | 3 ++- migra

[Qemu-devel] [PATCH COLO-Frame v13 19/39] COLO: Add checkpoint-delay parameter for migrate-set-parameters

2015-12-28 Thread zhanghailiang
Add checkpoint-delay parameter for migrate-set-parameters, so that we can control the checkpoint frequency when COLO is in periodic mode. Cc: Luiz Capitulino Cc: Eric Blake Cc: Markus Armbruster Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gilbert

[Qemu-devel] [PATCH COLO-Frame v13 01/39] configure: Add parameter for configure to enable/disable COLO support

2015-12-28 Thread zhanghailiang
configure --enable-colo/--disable-colo to switch COLO support on/off. COLO support is On by default. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei Reviewed-by: Dr. David Alan Gilbert --- v11: - Turn COLO on in default (Eric's suggestion) --- configure

[Qemu-devel] [PATCH COLO-Frame v13 00/39] COarse-grain LOck-stepping(COLO) Virtual Machines for Non-stop Service (FT)

2015-12-28 Thread zhanghailiang
ault filter before going into COLO. - Reconstruct send/receive helper functions in patch 10. - Address serveral other comments from Dave v11: - Re-implement buffer/release packets based on filter-buffer according to Jason Wang's suggestion. (patch 34, patch 36 ~ patch 38) - Rebase mast

[Qemu-devel] [PATCH COLO-Frame v12 32/38] COLO: Split qemu_savevm_state_begin out of checkpoint process

2015-12-15 Thread zhanghailiang
data transferring in the later checkpoint. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- migration/colo.c | 51 +-- 1 file changed, 37 insertions(+), 14 deletions(-) diff --git a/migration/colo.c b/migration/colo.c index d253d64

[Qemu-devel] [PATCH COLO-Frame v12 14/38] ram: Split host_from_stream_offset() into two helper functions

2015-12-15 Thread zhanghailiang
: zhanghailiang --- v12: - Remove the offset parameter for ram_block_from_stream() and check the validity of the related value in a new helper. (Dave's suggestion) v11: - New patch Signed-off-by: zhanghailiang --- include/exec/ram_addr.h | 8 ++-- migration/ram.c

[Qemu-devel] [PATCH COLO-Frame v12 25/38] qmp event: Add event notification for COLO error

2015-12-15 Thread zhanghailiang
is still necessary to notify users that we exited COLO mode. Cc: Markus Armbruster Cc: Michael Roth Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- v11: - Fix several typos found by Eric Signed-off-by: zhanghailiang --- docs/qmp-events.txt | 17 + migrati

[Qemu-devel] [PATCH COLO-Frame v12 30/38] savevm: Split load vm state function qemu_loadvm_state

2015-12-15 Thread zhanghailiang
qemu_loadvm_state is too long, and we can simplify it by splitting up with three helper functions. Signed-off-by: zhanghailiang --- migration/savevm.c | 161 - 1 file changed, 97 insertions(+), 64 deletions(-) diff --git a/migration/savevm.c

[Qemu-devel] [PATCH COLO-Frame v12 31/38] COLO: Separate the process of saving/loading ram and device state

2015-12-15 Thread zhanghailiang
during checkpoint. Besides, we move the colo_flush_ram_cache to the proper position after the above change. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- v11: - Remove load configuration section in qemu_loadvm_state_begin() Signed-off-by: zhanghailiang --- include/sysemu/sysemu.h

[Qemu-devel] [PATCH COLO-Frame v12 36/38] filter-buffer: Introduce a helper function to release packets

2015-12-15 Thread zhanghailiang
We need to release all the packets from VM in COLO or Micro-checkpoint, here we add a new helper function to realse the packets that buffered by default buffer-filter Signed-off-by: zhanghailiang Cc: Jason Wang Cc: Yang Hongyang --- v12: - Rename this helper function v11: - New patch Signed

[Qemu-devel] [PATCH COLO-Frame v12 19/38] COLO: Add checkpoint-delay parameter for migrate-set-parameters

2015-12-15 Thread zhanghailiang
Add checkpoint-delay parameter for migrate-set-parameters, so that we can control the checkpoint frequency when COLO is in periodic mode. Cc: Luiz Capitulino Cc: Eric Blake Cc: Markus Armbruster Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gilbert

[Qemu-devel] [PATCH COLO-Frame v12 33/38] net/filter-buffer: Add default filter-buffer for each netdev

2015-12-15 Thread zhanghailiang
for the netdev. Signed-off-by: zhanghailiang Cc: Jason Wang Cc: Yang Hongyang --- v12: - Skip vhost-net when add default filter - Don't go through filter layer if the filter is disabled. v11: - New patch Signed-off-by: zhanghailiang --- include/net/filter.h | 10 +++ net/f

[Qemu-devel] [PATCH COLO-Frame v12 13/38] COLO: Save PVM state to secondary side when do checkpoint

2015-12-15 Thread zhanghailiang
the size of VM state, so in master, we use qsb to store VM state temporarily, get the data size by call qsb_get_length() and then migrate the data to the qsb in the secondary side. Signed-off-by: zhanghailiang Signed-off-by: Gonglei Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gilbert

[Qemu-devel] [PATCH COLO-Frame v12 26/38] COLO failover: Shutdown related socket fd when do failover

2015-12-15 Thread zhanghailiang
s to wake up the blocking operation in failover BH. Besides, we should close the corresponding file descriptors after failvoer BH shutdown them, or there will be an error. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- v12: - Shutdown both QEMUFile's fd though they may use the sam

[Qemu-devel] [PATCH COLO-Frame v12 28/38] COLO: Process shutdown command for VM in COLO state

2015-12-15 Thread zhanghailiang
If VM is in COLO FT state, we should do some extra work before normal shutdown process. SVM will ignore the shutdown command if this command is issued directly to it, PVM will send the shutdown command to SVM if it gets this command. Cc: Paolo Bonzini Signed-off-by: zhanghailiang Signed-off-by

[Qemu-devel] [PATCH COLO-Frame v12 37/38] colo: Use default buffer-filter to buffer and release packets

2015-12-15 Thread zhanghailiang
Enable default filter to buffer packets and release the packets after a checkpoint. Signed-off-by: zhanghailiang Cc: Jason Wang Cc: Yang Hongyang --- v12: - Add a helper function to check if all netdev supports buffer packets. - Flush buffered packets when do failover. v11: - Use new helper

[Qemu-devel] [PATCH COLO-Frame v12 27/38] COLO failover: Don't do failover during loading VM's state

2015-12-15 Thread zhanghailiang
We should not do failover work while the main thread is loading VM's state, otherwise it will destroy the consistent of VM's memory and device state. Here we add a new failover status 'RELAUNCH' which means we should relaunch the process of failover. Signed-off-by: zhanghai

[Qemu-devel] [PATCH COLO-Frame v12 18/38] COLO: Flush PVM's cached RAM into SVM's memory

2015-12-15 Thread zhanghailiang
all content of PVM's RAM cache into SVM's MEMORY, we do this in a more efficient way: Only flush any page that dirtied by PVM since last checkpoint. In this way, we can ensure SVM's memory same with PVM's. Besides, we must ensure flush RAM cache before load device state. Si

[Qemu-devel] [PATCH COLO-Frame v12 38/38] COLO: Add block replication into colo process

2015-12-15 Thread zhanghailiang
Make sure master start block replication after slave's block replication started. Signed-off-by: zhanghailiang Signed-off-by: Wen Congyang Signed-off-by: Li Zhijian --- migration/colo.c | 60 trace-events | 2 ++ 2 files ch

[Qemu-devel] [PATCH COLO-Frame v12 22/38] COLO failover: Introduce state to record failover process

2015-12-15 Thread zhanghailiang
set the value. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- v11: - fix several typos found by Dave - Add Reviewed-by tag Signed-off-by: zhanghailiang --- include/migration/failover.h | 10 ++ migration/colo-failover.c| 37

[Qemu-devel] [PATCH COLO-Frame v12 34/38] filter-buffer: Accept zero interval

2015-12-15 Thread zhanghailiang
For default buffer filter, its 'interval' value is zero, so here we should accept zero interval. Signed-off-by: zhanghailiang Reviewed-by: Yang Hongyang Cc: Jason Wang --- v12: - Add Reviewed-by tag v11: - Add comment v10: - new patch Signed-off-by: zhanghailiang --- net/filte

[Qemu-devel] [PATCH COLO-Frame v12 21/38] COLO failover: Introduce a new command to trigger a failover

2015-12-15 Thread zhanghailiang
e PVM, the Primary side will exit COLO mode and take over operation. If sent to the Secondary, the secondary will run failover work, then take over server operation to become the new Primary. Cc: Luiz Capitulino Cc: Eric Blake Cc: Markus Armbruster Signed-off-by: zhanghailiang Signed-off-by:

[Qemu-devel] [PATCH COLO-Frame v12 29/38] COLO: Update the global runstate after going into colo state

2015-12-15 Thread zhanghailiang
If we start qemu with -S, the runstate will change from 'prelaunch' to 'running' after going into colo state. So it is necessary to update the global runstate after going into colo state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- migration/colo.c | 5

[Qemu-devel] [PATCH COLO-Frame v12 16/38] ram/COLO: Record the dirty pages that SVM received

2015-12-15 Thread zhanghailiang
We record the address of the dirty pages that received, it will help flushing pages that cached into SVM. We record them by re-using migration dirty bitmap. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- v12: - Add Reviewed-by tag v11: - Split a new helper function from

[Qemu-devel] [PATCH COLO-Frame v12 35/38] filter-buffer: Introduce a helper function to enable/disable default filter

2015-12-15 Thread zhanghailiang
The default buffer filter doesn't buffer packets in default, but we need to buffer packets for COLO or Micro-checkpoint, Here we add a helper function to enable/disable filter's buffer capability. Signed-off-by: zhanghailiang Cc: Jason Wang Cc: Yang Hongyang --- v12: - Rename

[Qemu-devel] [PATCH COLO-Frame v12 12/38] QEMUSizedBuffer: Introduce two help functions for qsb

2015-12-15 Thread zhanghailiang
m the file into qsb, this is used to get VM state from socket into a buffer. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gilbert --- v11: - size_t'ify these two help functions (Dave's suggestion) Signed-off-by: zhanghailiang --- include/migrat

[Qemu-devel] [PATCH COLO-Frame v12 17/38] COLO: Load VMState into qsb before restore it

2015-12-15 Thread zhanghailiang
load VM state, which can ensure the data is intact. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei Reviewed-by: Dr. David Alan Gilbert Cc: Dr. David Alan Gilbert --- v12: - Use the new helper colo_get_cmd_value() instead of colo_ctl_get() Signed-off-by

[Qemu-devel] [PATCH COLO-Frame v12 08/38] migration: Rename the'file' member of MigrationState

2015-12-15 Thread zhanghailiang
Rename the 'file' member of MigrationState to 'to_dst_file'. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- v12: - Add Reviewed-by tag - Add the missed modification for RDMA migration. (Found by Wen Congyang) v11: - Only rename 'file' member

[Qemu-devel] [PATCH COLO-Frame v12 15/38] COLO: Load PVM's dirty pages into SVM's RAM cache temporarily

2015-12-15 Thread zhanghailiang
7;s state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei Reviewed-by: Dr. David Alan Gilbert --- v12: - Fix minor error in error_report (Dave's comment) - Add Reviewed-by tag v11: - Rename 'host_cache' to 'colo_cache' (Dave's suggestion) v

[Qemu-devel] [PATCH COLO-Frame v12 24/38] COLO: Implement failover work for Secondary VM

2015-12-15 Thread zhanghailiang
If users require SVM to takeover work, colo incoming thread should exit from loop while failover BH helps backing to migration incoming coroutine. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gilbert v12: - Improve error message that suggested by Dave

[Qemu-devel] [PATCH COLO-Frame v12 06/38] migration: Integrate COLO checkpoint process into migration

2015-12-15 Thread zhanghailiang
Add a migrate state: MIGRATION_STATUS_COLO, enter this migration state after the first live migration successfully finished. We reuse migration thread, so if colo is enabled by user, migration thread will go into the process of colo. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian

[Qemu-devel] [PATCH COLO-Frame v12 04/38] migration: Export migrate_set_state()

2015-12-15 Thread zhanghailiang
Fix the first parameter of migrate_set_state(), and export it. We will use it in later. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- v12: - Add Reviewed-by tag v11: - New patch which is split from patch 'migration: Add state records for migration incoming

[Qemu-devel] [PATCH COLO-Frame v12 01/38] configure: Add parameter for configure to enable/disable COLO support

2015-12-15 Thread zhanghailiang
configure --enable-colo/--disable-colo to switch COLO support on/off. COLO support is On by default. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei Reviewed-by: Dr. David Alan Gilbert --- v11: - Turn COLO on in default (Eric's suggestion) Signed-o

[Qemu-devel] [PATCH COLO-Frame v12 03/38] COLO: migrate colo related info to secondary node

2015-12-15 Thread zhanghailiang
/destination; Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei Reviewed-by: Dr. David Alan Gilbert --- v11: - Add Reviewed-by tag v10: - Use VMSTATE_BOOL instead of VMSTATE_UNIT32 for 'colo_requested' (Dave's suggestion). Signed-off-by: zhanghailian

[Qemu-devel] [PATCH COLO-Frame v12 11/38] COLO: Add a new RunState RUN_STATE_COLO

2015-12-15 Thread zhanghailiang
Guest will enter this state when paused to save/restore VM state under colo checkpoint. Cc: Eric Blake Cc: Markus Armbruster Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Eric Blake --- qapi-schema.json | 5

[Qemu-devel] [PATCH COLO-Frame v12 05/38] migration: Add state records for migration incoming

2015-12-15 Thread zhanghailiang
For migration destination, we also need to know its state, we will use it in COLO. Here we add a new member 'state' for MigrationIncomingState, and also use migrate_set_state() to modify its value. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- v11: - Split

[Qemu-devel] [PATCH COLO-Frame v12 23/38] COLO: Implement failover work for Primary VM

2015-12-15 Thread zhanghailiang
For PVM, if there is failover request from users. The colo thread will exit the loop while the failover BH does the cleanup work and resumes VM. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian --- v12: - Fix error report and remove unnecessary check in primary_vm_do_failover() (Dave&#

<    1   2   3   4   5   6   7   8   9   10   >