[Qemu-devel] [PATCH] Fix virtio migration

2016-01-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" I misunderstood the vmstate macro definition when I reworked the virtio .get/.put. The VMSTATE_STRUCT_VARRAY_KNOWN, was described as being for "a variable length array (i.e. _type *_field) but we know the length". However it actually specified operation for arrays

[Qemu-devel] [PATCH 0/3] Postcopy minor fixes

2015-11-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Hi, These are three small fixes for the postcopy code; the first two coming from Bharata's testing on Power, and the last one being a text fixup that Eric asked for. The first only affects setups with another iterable device (e.g. Power's htab and block migration

[Qemu-devel] [PATCH 3/3] migrate-start-postcopy: Improve text

2015-11-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Improve the text in both the qapi-schema and hmp help to point out you need to set the postcopy-ram capability prior to issuing migrate-start-postcopy. Signed-off-by: Dr. David Alan Gilbert --- hmp-commands.hx | 4 +++- qapi-schema.json | 3 ++- 2 files changed,

[Qemu-devel] [PATCH 1/3] Finish non-postcopiable iterative devices before package

2015-11-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Where we have iterable, but non-postcopiable devices (e.g. htab or block migration), complete them before forming the 'package' but with the CPUs stopped. This stops them filling up the package. Signed-off-by: Dr. David Alan Gilbert --- include/sysemu/sysemu.h |

[Qemu-devel] [PATCH 2/3] Postcopy: Fix TP!=HP zero case

2015-11-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Where the target page size is different from the host page we special case it, but I messed up on the zero case check. Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/ram.c b/

[Qemu-devel] [PATCH v2 1/1] migrate-start-postcopy: Improve text

2015-11-12 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Improve the text in both the qapi-schema and hmp help to point out you need to set the postcopy-ram capability prior to issuing migrate-start-postcopy. Also fix the text of the migrate_start_postcopy error that deals with capabilities. Signed-off-by: Dr. David Ala

[Qemu-devel] [PATCH 1/1] migration_init: Fix lock initialisation/make it explicit

2015-11-12 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Peter reported a lock error on MacOS after my a82d593b patch. migrate_get_current does one-time initialisation of a bunch of variables. migrate_init does reinitialisation even on a 2nd migrate after a cancel. The problem here was that I'd initialised the mutex in

[Qemu-devel] [PATCH 0/3] [For 2.5] Migration fixes

2015-11-18 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Hi, These are 3 migration fixes that fix things I messed up. Only the 1st one (set last_sent_block) fixes a problem, the other two fix coverity findings. Dave Dr. David Alan Gilbert (3): Set last_sent_block migration: Dead assignment of current_time Unneed

[Qemu-devel] [PATCH 1/3] Set last_sent_block

2015-11-18 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" In a82d593b61054b3dea43 I accidentally removed the setting of last_sent_block, put it back. Symptoms: Multithreaded compression only uses one thread. Migration is a bit less efficient since it won't use 'cont' flags. Signed-off-by: Dr. David Alan Gilbert Fix

[Qemu-devel] [PATCH 2/3] migration: Dead assignment of current_time

2015-11-18 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" I set current_time before the postcopy test but never use it; (I think this was from the original version where it was time based). Spotted by coverity, CID 1339208 Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 1 - 1 file changed, 1 deletion(

[Qemu-devel] [PATCH 3/3] Unneeded NULL check

2015-11-18 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The check is unneccesary, we read the value at the start of the thread, use it, and never change it. The value is checked to be non-NULL before thread creation. Spotted by coverity, CID 1339211 Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 2

[Qemu-devel] [PATCH] Assume madvise for (no)hugepage works

2015-11-19 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" madvise() returns EINVAL in the case of many failures, but also returns it in cases where the host kernel doesn't have THP enabled. Postcopy only really cares that THP is off before it detects faults, and turns it back on afterwards; so we're going to have to assume

[Qemu-devel] [For 2.5?? PATCH 1/1] qemu_{real_}host_page_[size|mask] change types to ram_addr_t

2015-12-02 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Anthony reported that >4GB guests on Xen with 32bit QEMU broke after my 4ed023c (Round up RAMBlock sizes). In that patch I mask sizes against qemu_host_page_size/mask which are uintptr_t, and thus 32bit on a 32bit QEMU, even though the ram space might be bigger tha

[Qemu-devel] [PATCH] Fix xbzrle vs last_sent_block update

2015-12-10 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" My fix (84e7b80a) replaced the last_sent_block update that I'd removed earlier; however it was too aggressive in the xbzrle case. save_xbzrle_page might return '0' to mean that the page didn't need sending since it was the same as the last sent version; in this cas

[Qemu-devel] [PATCH 6/6] multithread decompression: Avoid one copy

2015-12-15 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" qemu_get_buffer does a copy, we can avoid the memcpy, and we can then remove the extra buffer. Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 11 +++ 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/migration/ram.c b/migration/ra

[Qemu-devel] [PATCH 4/6] Migration: Emit event at start of pass

2015-12-15 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Emit an event each time we sync the dirty bitmap on the source; this helps libvirt use postcopy by giving it a kick when it might be a good idea to start the postcopy. Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 4 qapi/event.json | 11 +

[Qemu-devel] [PATCH 2/6] migration: Add state records for migration incoming

2015-12-15 Thread Dr. David Alan Gilbert (git)
From: zhanghailiang For migration destination, we also need to know its state, we will use it in COLO. Here we add a new member 'state' for MigrationIncomingState, and also use migrate_set_state() to modify its value. Signed-off-by: zhanghailiang dgilbert: Fixed early free of MigraitonIncomin

[Qemu-devel] [PATCH 3/6] Postcopy: Send events/change state on incoming side

2015-12-15 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" I missed the calls to send migration events on the destination side as we enter postcopy. Take care when adding them not to do it after state has been freed. Signed-off-by: Dr. David Alan Gilbert --- migration/savevm.c | 22 +++--- 1 file changed,

[Qemu-devel] [PATCH 0/6] Migration: Small fixes and improvements

2015-12-15 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This is a set of 4 small fixes and improvements I've been collecting during the 2.5 freeze; I've added in a couple of Zhanghailiang's patches that are easier to merge before rather than after. (Of which one I've added a small fix of my own, see the message) Jiri wa

[Qemu-devel] [PATCH 1/6] migration: Export migrate_set_state()

2015-12-15 Thread Dr. David Alan Gilbert (git)
From: zhanghailiang Fix the first parameter of migrate_set_state(), and export it. We will use it in later. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- include/migration/migration.h | 2 ++ migration/migration.c | 36 +--- 2 fi

[Qemu-devel] [PATCH 5/6] Use qemu_get_buffer_in_place for xbzrle data

2015-12-15 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Avoid a data copy (if we're lucky) in the xbzrle code. Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 102d1f2..994552c 100644 --- a/migrat

[Qemu-devel] [PATCH v2 1/6] migration: Export migrate_set_state()

2015-12-16 Thread Dr. David Alan Gilbert (git)
From: zhanghailiang Change the first parameter of migrate_set_state(), and export it. We will use it in a later patch to update incoming state. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert dgilbert: Updated comment as per Juan's review --- include/migration/migration.h |

[Qemu-devel] [PATCH v2 6/6] multithread decompression: Avoid one copy

2015-12-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" qemu_get_buffer does a copy, we can avoid the memcpy, and we can then remove the extra buffer. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Liang Li Reviewed-by: Juan Quintela --- migration/ram.c | 11 +++ 1 file changed, 3 insertions(+), 8 deleti

[Qemu-devel] [PATCH v2 0/6] Migration: Small fixes and improvements

2015-12-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This is a set of 4 small fixes and improvements I've been collecting during the 2.5 freeze; I've added in a couple of Zhanghailiang's patches that are easier to merge before rather than after. (Of which one I've added a small fix of my own, see the message) Jiri wa

[Qemu-devel] [PATCH v2 2/6] migration: Add state records for migration incoming

2015-12-16 Thread Dr. David Alan Gilbert (git)
From: zhanghailiang For migration destination, we also need to know its state, we will use it in COLO. Here we add a new member 'state' for MigrationIncomingState, and also use migrate_set_state() to modify its value. Signed-off-by: zhanghailiang dgilbert: Fixed early free of MigraitonIncomin

[Qemu-devel] [PATCH v2 4/6] Migration: Emit event at start of pass

2015-12-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Emit an event each time we sync the dirty bitmap on the source; this helps libvirt use postcopy by giving it a kick when it might be a good idea to start the postcopy. Signed-off-by: Dr. David Alan Gilbert --- docs/qmp-events.txt | 14 ++ migration/ra

[Qemu-devel] [PATCH v2 5/6] Use qemu_get_buffer_in_place for xbzrle data

2015-12-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Avoid a data copy (if we're lucky) in the xbzrle code. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela --- migration/ram.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 102d1f2.

[Qemu-devel] [PATCH v2 3/6] Postcopy: Send events/change state on incoming side

2015-12-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" I missed the calls to send migration events on the destination side as we enter postcopy. Take care when adding them not to do it after state has been freed. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: zhanghailiang Reviewed-by: Juan Quintela --- migrati

[Qemu-devel] [PATCH] ether/slirp: Avoid redefinition of the same constants

2016-01-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" eth.h and slirp.h both define ETH_ALEN and ETH_P_IP rtl8139.c and eth.h both define ETH_HLEN Move the related constant (ETH_P_ARP) from slirp.h to eth.h, and remove the duplicates; make slirp.h include eth.h Signed-off-by: Dr. David Alan Gilbert --- hw/net/rtl81

[Qemu-devel] [PATCH] config.status: Pass extra parameters

2016-01-12 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This allows you to do: ./config.status --the-option-you-forgot Signed-off-by: Dr. David Alan Gilbert --- configure | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure b/configure index 44ac9ab..9a9e978 100755 --- a/configure +++ b/conf

[Qemu-devel] [PATCH 1/1] HMP: Add equivalent to x-blockdev-change

2015-12-17 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" x-blockdev-change has no HMP equivalent, so add x_block_change. Example useages are: x_block_change foo -a bah to add the node bah to the parent foo x_block_change foo -d bah to delete the node bah from the parent foo Signed-off-by: Dr. David Alan Gilbert

[Qemu-devel] [PATCH 2/2] migration/virtio: Remove simple .get/.put use

2016-01-06 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The 'virtqueue_state' and 'ringsize' can be saved using VMSTATE macros rather than hand coded .get/.put Signed-off-by: Dr. David Alan Gilbert --- hw/virtio/virtio.c | 87 -- 1 file changed, 19 insertions(+), 68

[Qemu-devel] [PATCH 1/2] Add VMSTATE_STRUCT_VARRAY_KNOWN

2016-01-06 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" At the moment we have VMSTATE_STRUCT_ARRAY that requires the field is declared as an array of fixed size. We also have VMSTATE_STRUCT_VARRAY_UINT* that allows a field declared as a pointer, but requires that the length is a field member in the structure being loaded

[Qemu-devel] [PATCH 1/1] Postcopy+spice: Pass spice migration data earlier

2016-02-22 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Spice hooks the migration status changes to figure out when to transmit information to the new spice server; but the migration status in postcopy doesn't quite fit - the destination starts running before the end of the source migration. It's not a case of hanging o

[Qemu-devel] [PATCH] Postcopy: Fix sync count in info migrate

2016-02-25 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" I'd missed the sync count off in the postcopy case. Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 1 + 1 file changed, 1 insertion(+) diff --git a/migration/migration.c b/migration/migration.c index fc5e50b..34d3e5f 100644 --- a/migration/mig

[Qemu-devel] [WIP:COLO: 1] Flush colo ram in parallel

2016-02-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Flush the colo ram cache in parallel; use the same number of threads as CPU cores. On a VM with 4 cores, and 4GB RAM, I've seen a reduction from ~20ms to ~16ms using this, which is helpful but not as much as I hoped; I guess one problem might be that all the chan

[Qemu-devel] [PATCH 1/1] Fix glib_subprocess test

2015-06-24 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" A typo means that the tests dependent on glib with subprocess support are never run. Fixes: 9d41401b90fa10b335d2e739149d36437cfbf622 Signed-off-by: Dr. David Alan Gilbert --- configure | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure

[Qemu-devel] [PATCH] check_section_footers: Check the correct section_id

2015-07-02 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The section footers check was incorrectly checking the section_id in the SaveStateEntry not the LoadStateEntry. These can validly be different if the two QEMU instances have instantiated their devices in a different order. The test only cares that we're finishing

[Qemu-devel] [PATCH] Migration: Generate the completed event only when we complete

2015-10-13 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The current migration-completed event is generated a bit too early, which means that an eager libvirt that's ready to go as soon as it sees the event ends up racing with the actual end of migration. This corresponds to RH bug: https://bugzilla.redhat.com/show_bug.c

[Qemu-devel] [PATCH] gdb command: qemu aios, qemu aiohandlers

2015-10-20 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Two new gdb commands are added: qemu iohandlers that dumps the list of waiting iohandlers, this is particularly useful for trying to figure out where an incoming migration is blocked. For 'fd_coroutine_enter' cases we dump a full backtrace. qe

[Qemu-devel] [RFC/COLO: 0/3] Hybrid mode and parameterisation

2015-08-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Hi, This is an experimental adition to COLO, based off the colo-v1.5-developing branch. It's not ready for inclusion. The first patch, adds a 'hybrid mode' where the SVM is sent checkpoints from the primary but does not run, and is thus much similar to a normal

[Qemu-devel] [RFC/COLO: 2/3] Parameterise min/max/relax time

2015-08-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert --- hmp-commands.hx| 15 - hmp.c | 31 -- hmp.h | 1 - migration/colo.c | 32 ++- migration/migration.c | 59 ++

[Qemu-devel] [RFC/COLO: 3/3] COLO: Parameterise background RAM transfer limit

2015-08-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" COLO (experimentally) transfers RAM in the background when the amount to transfer reaches a limit; allow this limit to be set via the parameter mechanism. Signed-off-by: Dr. David Alan Gilbert --- hmp.c | 8 migration/colo.c | 6 ++

[Qemu-devel] [RFC/COLO: 1/3] COLO: Hybrid mode

2015-08-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Automatically switch into a passive checkpoint mode when checkpoints are repeatedly short. This saves CPU time on the SVM (since it's not running) and the network traffic and PVM CPU time for the comparison processing. Signed-off-by: Dr. David Alan Gilbert --- h

[Qemu-devel] [PATCH 0/5] Migration cleanups

2015-08-13 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This series is a set of small cleanups, some of which are from my postcopy series. Dave Dr. David Alan Gilbert (5): migration/ram.c: Use RAMBlock rather than MemoryRegion Split out end of migration code from migration_thread Init page sizes in qtest migrat

[Qemu-devel] [PATCH 2/5] Split out end of migration code from migration_thread

2015-08-13 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The code that gets run at the end of the migration process is getting large, and I'm about to add more for postcopy. Split it into a separate function. Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 75 --

[Qemu-devel] [PATCH 5/5] migration: qemu-file more size_t'ifying

2015-08-13 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This time convert the external functions: qemu_get_buffer, qemu_peek_buffer qemu_put_buffer and qemu_put_buffer_async Signed-off-by: Dr. David Alan Gilbert --- include/migration/qemu-file.h | 10 +- migration/qemu-file.c | 22 +++--

[Qemu-devel] [PATCH 1/5] migration/ram.c: Use RAMBlock rather than MemoryRegion

2015-08-13 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" RAM migration mainly works on RAMBlocks but in a few places uses data from MemoryRegions to access the same information that's already held in RAMBlocks; clean it up just to avoid the MemoryRegion use. Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 2

[Qemu-devel] [PATCH 3/5] Init page sizes in qtest

2015-08-13 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" One of my patches used a loop that was based on host page size; it dies in qtest since qtest hadn't bothered init'ing it. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Amit Shah --- qtest.c | 1 + 1 file changed, 1 insertion(+)

[Qemu-devel] [PATCH 4/5] migration: size_t'ify some of qemu-file

2015-08-13 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This is a start on using size_t more in qemu-file and friends; it fixes up QEMUFilePutBufferFunc and QEMUFileGetBufferFunc to take size_t lengths and return ssize_t return values (like read(2)) and fixes up all the different implementations of them. Note that I've

[Qemu-devel] [PATCH] Move RAMBlock and ram_list to ram_addr.h

2015-08-14 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert --- include/exec/cpu-all.h | 41 - include/exec/ram_addr.h | 40 2 files changed, 40 insertions(+), 41 deletions(-) diff --git a/include/exec/c

[Qemu-devel] [PATCH] trace-events: Add hmp completion

2015-08-14 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add completion for the trace event names in the hmp trace-event command. Signed-off-by: Dr. David Alan Gilbert --- hmp-commands.hx | 1 + hmp.h | 1 + monitor.c | 20 3 files changed, 22 insertions(+) diff --git a/hmp-comma

[Qemu-devel] [PATCH] Print error when failing to load PCI config data

2015-06-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" When loading migration fails due to a disagreement about PCI config data we don't currently get any errors explaining that was the cause of the problem or which byte in the config data was at fault. Signed-off-by: Dr. David Alan Gilbert --- hw/pci/pci.c | 4

[Qemu-devel] [PATCH] Grammar: 'as to'->'as for'

2015-06-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Fixup migrate-incoming text as requested by Eric in: http://lists.nongnu.org/archive/html/qemu-devel/2015-03/msg03362.html Signed-off-by: Dr. David Alan Gilbert --- qmp-commands.hx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qmp-commands

[Qemu-devel] [PATCH v2 00/12] Remove RDMA migration dependence on RAMBlock offset

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" RDMA migration currently relies on the source and destination RAMBlocks having the same offsets within ram_addr_t space; unfortunately that's just not true when: a) You hotplug on the source but then create the device on the command line on the destination.

[Qemu-devel] [PATCH v2 01/12] Rename RDMA structures to make destination clear

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" RDMA has two data types that are named confusingly; RDMALocalBlock (pointed to indirectly by local_ram_blocks) RDMARemoteBlock (pointed to by block in RDMAContext) RDMALocalBlocks, as the name suggests is a data strucuture that represents the RDMAable RAM Blo

[Qemu-devel] [PATCH v2 11/12] Sanity check RDMA remote data

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Perform some basic (but probably not complete) sanity checking on requests from the RDMA source. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Michael R. Hines --- migration/rdma.c | 30 ++ 1 file changed, 30 insertions(+) diff

[Qemu-devel] [PATCH v2 02/12] qemu_ram_foreach_block: pass up error value, and down the ramblock name

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" check the return value of the function it calls and error if it's non-0 Fixup qemu_rdma_init_one_block that is the only current caller, and rdma_add_block the only function it calls using it. Pass the name of the ramblock to the function; helps in debugging. Sig

[Qemu-devel] [PATCH v2 03/12] Remove unneeded memset

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Michael R. Hines --- migration/rdma.c | 1 - 1 file changed, 1 deletion(-) diff --git a/migration/rdma.c b/migration/rdma.c index 38e5f44..bc73ff8 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -2445,7

[Qemu-devel] [PATCH v2 05/12] Store block name in local blocks structure

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" In a later patch the block name will be used to match up two views of the block list. Keep a copy of the block name with the local block list. (At some point it could be argued that it would be best just to let migration see the innards of RAMBlock and avoid the n

[Qemu-devel] [PATCH v2 09/12] Rework ram block hash

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" RDMA uses a hash from block offset->RAM Block; this isn't needed on the destination, and it becomes harder to maintain after the next patch in the series that sorts the block list. Split the hash so that it's only generated on the source. Signed-off-by: Dr. David

[Qemu-devel] [PATCH v2 10/12] Sort destination RAMBlocks to be the same as the source

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Use the order of incoming RAMBlocks from the source to record an index number; that then allows us to sort the destination local RAMBlock list to match the source. Now that the RAMBlocks are known to be in the same order, this simplifies the RDMA Registration step

[Qemu-devel] [PATCH v2 06/12] Translate offsets to destination address space

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The 'offset' field in RDMACompress and 'current_addr' field in RDMARegister are commented as being offsets within a particular RAMBlock, however they appear to actually be offsets within the ram_addr_t space. The code currently assumes that the offsets on the sourc

[Qemu-devel] [PATCH v2 04/12] rdma typos

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" A couple of typo fixes. Signed-off-by: Dr. David Alan Gilbert --- migration/rdma.c | 6 +++--- trace-events | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index bc73ff8..44ed996 100644 --- a/migrat

[Qemu-devel] [PATCH v2 08/12] Allow rdma_delete_block to work without the hash

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" In the next patch we remove the hash on the destination, rdma_delete_block does two things with the hash which can be avoided: a) The caller passes the offset and rdma_delete_block looks it up in the hash; fixed by getting the caller to pass the block b) Th

[Qemu-devel] [PATCH v2 12/12] Fail more cleanly in mismatched RAM cases

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" If the number of RAMBlocks was different on the source from the destination, QEMU would hang waiting for a disconnect on the source and wouldn't release from that hang until the destination was manually killed. Mark the stream as being in error, this causes the des

[Qemu-devel] [PATCH v2 07/12] Rework ram_control_load_hook to hook during block load

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" We need the names of RAMBlocks as they're loaded for RDMA, reuse a slightly modified ram_control_load_hook: a) Pass a 'data' parameter to use for the name in the block-reg case b) Only some hook types now require the presence of a hook function. Signed-off

[Qemu-devel] [PATCH 2/2] ram_find_and_save_block: Split out the finding

2015-09-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Split out the finding of the dirty page and all the wrap detection into a separate function since it was getting a bit hairy. Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 87 - 1 file changed,

[Qemu-devel] [PATCH 0/2] Split up ram_find_and_save_block

2015-09-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Chop up ram_find_and_save_block so it's smaller again. (from comments on my postcopy patch that adds more to it). This pair is based on top of my previous 5 patch cleanup series posted in August, but rebased on current qemu master. Dave Dr. David Alan Gilbert (2

[Qemu-devel] [PATCH 1/2] Move dirty page search state into separate structure

2015-09-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Pull the sarch state for one iteration of the dirty page search into a structure. Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 54 ++ 1 file changed, 34 insertions(+), 20 deletions(-) diff --git

[Qemu-devel] [PATCH v2 0/2] Split up ram_find_and_save_block

2015-09-23 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Chop up ram_find_and_save_block so it's smaller again. (from comments on my postcopy patch that adds more to it). This pair is based on top of my previous 5 patch cleanup series posted in August, but rebased on current qemu master. Dave Dr. David Alan Gilbert (2

[Qemu-devel] [PATCH v2 2/2] ram_find_and_save_block: Split out the finding

2015-09-23 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Split out the finding of the dirty page and all the wrap detection into a separate function since it was getting a bit hairy. Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 84 - 1 file changed,

[Qemu-devel] [PATCH v2 1/2] Move dirty page search state into separate structure

2015-09-23 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Pull the search state for one iteration of the dirty page search into a structure. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Amit Shah --- migration/ram.c | 55 +++ 1 file changed, 35 insertions(+), 20

[Qemu-devel] [PATCH] RDMA: Reduce restriction on block length match

2015-07-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" My e4d633207 patch has an over zealous sanity check that checked the lengths of the RAM Blocks on source/destination were the same. This isn't true because of the 'used_length' trick for RAM blocks like the ACPI table that vary in size. Prior to that patch RDMA wo

[Qemu-devel] [PATCH] RDMA: Reduce restriction on block length match

2015-07-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" My e4d633207 patch has an over zealous sanity check that checked the lengths of the RAM Blocks on source/destination were the same. This isn't true because of the 'used_length' trick for RAM blocks like the ACPI table that vary in size. Prior to that patch RDMA wo

[Qemu-devel] [PATCH] RDMA: Fix error exits (for 2.4)

2015-07-10 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The error checks I added used 'break' after the error, but I'm in a switch inside the while loop, so they need to be 'goto out'. Spotted by coverity; entries 1311368 and 1311369 Fixes: afcddefd Signed-off-by: Dr. David Alan Gilbert --- migration/rdma.c | 8

[Qemu-devel] [PATCH] Allow mismatched virtio config-len

2014-06-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Commit 'virtio: validate config_len on load' restricted config_len loaded from the wire to match the config_len that the device had. Unfortunately, there are cases where this isn't true, the one we found it on was the wqe addition in virtio-blk. Allow mismatched c

[Qemu-devel] [PATCH v2] [PATCH] Allow mismatched virtio config-len

2014-06-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Commit 'virtio: validate config_len on load' restricted config_len loaded from the wire to match the config_len that the device had. Unfortunately, there are cases where this isn't true, the one we found it on was the wce addition in virtio-blk. Allow mismatched c

[Qemu-devel] [PATCH 00/46] Postcopy implementation

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Hi, This is the 1st cut of my version of postcopy; it is designed for use with the Linux kernel additions recently posted by Andrea Arcangeli here: http://lists.gnu.org/archive/html/qemu-devel/2014-07/msg00525.html The current status is: 1) It works - I've

[Qemu-devel] [PATCH 26/46] Postcopy page-map-incoming (PMI) structure

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The PMI holds the state of each page on the incoming side, so that we can tell if the page is missing, already received or there is a request outstanding for it. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h| 18 ++ include/migr

[Qemu-devel] [PATCH 03/46] QEMUSizedBuffer/QEMUFile

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Stefan Berger's patch to create a QEMUFile that goes to a memory buffer; from: http://lists.gnu.org/archive/html/qemu-devel/2013-03/msg05036.html Using the QEMUFile interface, this patch adds support functions for operating on in-memory sized buffers that can be w

[Qemu-devel] [PATCH 04/46] improve DPRINTF macros, add to savevm

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Improve the existing DPRINTF macros in migration.c and arch_init by: 1) Making them go to stderr rather than stdout (so you can run with -nographic and redirect your debug to a file) 2) Making them print the ms time with each debug - useful for debugging latency

[Qemu-devel] [PATCH 07/46] Return path: Open a return path on QEMUFile for sockets

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Postcopy needs a method to send messages from the destination back to the source, this is the 'return path'. Wire it up for 'socket' QEMUFile's using a dup'd fd. Signed-off-by: Dr. David Alan Gilbert --- include/migration/qemu-file.h | 8 + qemu-file.c

[Qemu-devel] [PATCH 37/46] Add assertion to check migration_dirty_pages doesn't go -ve; have seen it happen once but not sure why

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert --- arch_init.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch_init.c b/arch_init.c index c006d21..58eccc1 100644 --- a/arch_init.c +++ b/arch_init.c @@ -439,6 +439,7 @@ ram_addr_t migration_bitmap_find_and_reset_di

[Qemu-devel] [PATCH 08/46] Return path: socket_writev_buffer: Block even on non-blocking fd's

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The return path uses a non-blocking fd so as not to block waiting for the (possibly broken) destination to finish returning a message, however we still want outbound data to behave in the same way and block. Signed-off-by: Dr. David Alan Gilbert --- qemu-file.c |

[Qemu-devel] [PATCH 09/46] Migration commands

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Create QEMU_VM_COMMAND section type for sending commands from source to destination. These commands are not intended to convey guest state but to control the migration process. For use in postcopy. Signed-off-by: Dr. David Alan Gilbert --- include/migration/mig

[Qemu-devel] [PATCH 10/46] Return path: Control commands

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add two src->dest commands: * OPENRP - To request that the destination open the return path * REQACK - Request an acknowledge from the destination Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 2 ++ include/sysemu/sysemu.h

[Qemu-devel] [PATCH 46/46] Start documenting how postcopy works.

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert --- docs/migration.txt | 148 + 1 file changed, 148 insertions(+) diff --git a/docs/migration.txt b/docs/migration.txt index 0492a45..dbd5e5f 100644 --- a/docs/migration.txt

[Qemu-devel] [PATCH 44/46] postcopy: Use userfaultfd

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" userfaultfd is a Linux syscall that gives an fd that receives a stream of notifications of accesses to pages marked as MADV_USERFAULT, and allows the program to acknowledge those stalls and tell the accessing thread to carry on. Signed-off-by: Dr. David Alan Gilber

[Qemu-devel] [PATCH 23/46] MIG_STATE_POSTCOPY_ACTIVE: Add new migration state

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" 'MIG_STATE_POSTCOPY_ACTIVE' is entered after the precopy timelimit has expired and migration switches to postcopy. 'migration_postcopy_phase' is provided for other sections to know if they're in postcopy. Signed-off-by: Dr. David Alan Gilbert --- include/migrati

[Qemu-devel] [PATCH 45/46] End of migration for postcopy

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Tweak the end of migration cleanup; we don't want to close stuff down at the end of the main stream, since the postcopy is still sending pages on the other thread. Signed-off-by: Dr. David Alan Gilbert --- migration.c | 20 1 file changed, 20

[Qemu-devel] [PATCH 25/46] Postcopy: Maintain sentmap during postcopy pre phase

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Where postcopy is preceeded by a period of precopy, the destination will have received pages that may have been dirtied on the source after the page was sent. The destination must throw these pages away before starting it's CPUs. Maintain a 'sentmap' of pages that

[Qemu-devel] [PATCH 24/46] qemu_savevm_state_complete: Postcopy changes

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" When postcopy calls qemu_savevm_state_complete it's not really the end of migration, so skip: a) Finishing postcopiable iterative devices - they'll carry on b) The termination byte on the end of the stream. We then also add: qemu_savevm_state_postcopy_compl

[Qemu-devel] [PATCH 27/46] postcopy: Add incoming_init/cleanup functions

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Provide functions to be called before the start of a postcopy enabled migration (even if it's not eventually used) and at the end. During the init we must disable huge pages in the RAM that we will receive postcopy data into, since if they start off as hugepage and

[Qemu-devel] [PATCH 28/46] postcopy: Incoming initialisation

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert --- arch_init.c | 11 +++ include/migration/migration.h | 1 + migration.c | 2 ++ 3 files changed, 14 insertions(+) diff --git a/arch_init.c b/arch_init.c index 134ea7e..fd7399c

[Qemu-devel] [PATCH 29/46] postcopy: ram_enable_notify to switch on userfault

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert --- include/migration/postcopy-ram.h | 5 + postcopy-ram.c | 36 +++- 2 files changed, 40 insertions(+), 1 deletion(-) diff --git a/include/migration/postcopy-ram.h b/inc

[Qemu-devel] [PATCH 36/46] Page request: Consume pages off the post-copy queue

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" When transmitting RAM pages, consume pages that have been queued by MIG_RPCOMM_REQPAGE commands and send them ahead of normal page scanning. Note: a) After a queued page the linear walk carries on from after the unqueued page; there is a reasonable chance that th

[Qemu-devel] [PATCH 30/46] Postcopy: postcopy_start

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" postcopy_start: Perform all the initialisation associated with starting up postcopy mode from the source. Signed-off-by: Dr. David Alan Gilbert --- migration.c | 85 + 1 file changed, 85 insertions(+)

[Qemu-devel] [PATCH 31/46] Postcopy: Rework migration thread for postcopy mode

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Switch to postcopy if: 1) There's still a significant amount to transfer 2) Postcopy is enabled 3) It's taken longer than the time set by the parameter. and change the cleanup at the end of migration to match. Signed-off-by: Dr. David Alan Gilbert --- m

[Qemu-devel] [PATCH 32/46] mig fd_connect: open return path

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert --- migration.c | 12 1 file changed, 12 insertions(+) diff --git a/migration.c b/migration.c index c73fcfa..c7ba6a1 100644 --- a/migration.c +++ b/migration.c @@ -1164,6 +1164,18 @@ void migrate_fd_connect(Migra

<    1   2   3   4   5   6   7   8   9   10   >