[Qemu-devel] [PATCH 0/2] Use glib's checks for use of too-new functions

2015-05-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Glib 2.32 adds a checking mechanism to warn of use of functions that are available in glib versions newer than a particular version. This series enables that check (via a -D added during configure); with it you get warnings like: /home/dgilbert/git/qemu-world3/vl.

[Qemu-devel] [PATCH 1/2] Fix glib_subprocess test

2015-05-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" A typo means that the tests dependent on glib with subprocess support are never run. Fixes: 9d41401b90fa10b335d2e739149d36437cfbf622 Signed-off-by: Dr. David Alan Gilbert --- configure | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure

[Qemu-devel] [PATCH 2/2] Compile time checks for newer glib

2015-05-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Newer glib has support for checking that applications aren't using newer glib calls than they should be. The support for the check only went into glib 2.32 and it only has macros for version 2.26 upwards; although we only insist on 2.22 at the moment, I set the gli

[Qemu-devel] [PATCH 1/1] Add -incoming help text

2015-05-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The help/man text for -incoming defer didn't make it through the merge of the code that implemented it. Signed-off-by: Dr. David Alan Gilbert --- qemu-options.hx | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/qemu-options.hx b/qemu-opt

[Qemu-devel] [RFC/COLO: 0/3] Hybrid mode and parameterisation

2015-08-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Hi, This is an experimental adition to COLO, based off the colo-v1.5-developing branch. It's not ready for inclusion. The first patch, adds a 'hybrid mode' where the SVM is sent checkpoints from the primary but does not run, and is thus much similar to a normal

[Qemu-devel] [RFC/COLO: 2/3] Parameterise min/max/relax time

2015-08-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert --- hmp-commands.hx| 15 - hmp.c | 31 -- hmp.h | 1 - migration/colo.c | 32 ++- migration/migration.c | 59 ++

[Qemu-devel] [RFC/COLO: 3/3] COLO: Parameterise background RAM transfer limit

2015-08-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" COLO (experimentally) transfers RAM in the background when the amount to transfer reaches a limit; allow this limit to be set via the parameter mechanism. Signed-off-by: Dr. David Alan Gilbert --- hmp.c | 8 migration/colo.c | 6 ++

[Qemu-devel] [RFC/COLO: 1/3] COLO: Hybrid mode

2015-08-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Automatically switch into a passive checkpoint mode when checkpoints are repeatedly short. This saves CPU time on the SVM (since it's not running) and the network traffic and PVM CPU time for the comparison processing. Signed-off-by: Dr. David Alan Gilbert --- h

[Qemu-devel] [PATCH] Print error when failing to load PCI config data

2015-06-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" When loading migration fails due to a disagreement about PCI config data we don't currently get any errors explaining that was the cause of the problem or which byte in the config data was at fault. Signed-off-by: Dr. David Alan Gilbert --- hw/pci/pci.c | 4

[Qemu-devel] [PATCH] Grammar: 'as to'->'as for'

2015-06-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Fixup migrate-incoming text as requested by Eric in: http://lists.nongnu.org/archive/html/qemu-devel/2015-03/msg03362.html Signed-off-by: Dr. David Alan Gilbert --- qmp-commands.hx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qmp-commands

[Qemu-devel] [PATCH v2 00/12] Remove RDMA migration dependence on RAMBlock offset

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" RDMA migration currently relies on the source and destination RAMBlocks having the same offsets within ram_addr_t space; unfortunately that's just not true when: a) You hotplug on the source but then create the device on the command line on the destination.

[Qemu-devel] [PATCH v2 01/12] Rename RDMA structures to make destination clear

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" RDMA has two data types that are named confusingly; RDMALocalBlock (pointed to indirectly by local_ram_blocks) RDMARemoteBlock (pointed to by block in RDMAContext) RDMALocalBlocks, as the name suggests is a data strucuture that represents the RDMAable RAM Blo

[Qemu-devel] [PATCH v2 11/12] Sanity check RDMA remote data

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Perform some basic (but probably not complete) sanity checking on requests from the RDMA source. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Michael R. Hines --- migration/rdma.c | 30 ++ 1 file changed, 30 insertions(+) diff

[Qemu-devel] [PATCH v2 02/12] qemu_ram_foreach_block: pass up error value, and down the ramblock name

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" check the return value of the function it calls and error if it's non-0 Fixup qemu_rdma_init_one_block that is the only current caller, and rdma_add_block the only function it calls using it. Pass the name of the ramblock to the function; helps in debugging. Sig

[Qemu-devel] [PATCH v2 03/12] Remove unneeded memset

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Michael R. Hines --- migration/rdma.c | 1 - 1 file changed, 1 deletion(-) diff --git a/migration/rdma.c b/migration/rdma.c index 38e5f44..bc73ff8 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -2445,7

[Qemu-devel] [PATCH v2 05/12] Store block name in local blocks structure

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" In a later patch the block name will be used to match up two views of the block list. Keep a copy of the block name with the local block list. (At some point it could be argued that it would be best just to let migration see the innards of RAMBlock and avoid the n

[Qemu-devel] [PATCH v2 09/12] Rework ram block hash

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" RDMA uses a hash from block offset->RAM Block; this isn't needed on the destination, and it becomes harder to maintain after the next patch in the series that sorts the block list. Split the hash so that it's only generated on the source. Signed-off-by: Dr. David

[Qemu-devel] [PATCH v2 10/12] Sort destination RAMBlocks to be the same as the source

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Use the order of incoming RAMBlocks from the source to record an index number; that then allows us to sort the destination local RAMBlock list to match the source. Now that the RAMBlocks are known to be in the same order, this simplifies the RDMA Registration step

[Qemu-devel] [PATCH v2 06/12] Translate offsets to destination address space

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The 'offset' field in RDMACompress and 'current_addr' field in RDMARegister are commented as being offsets within a particular RAMBlock, however they appear to actually be offsets within the ram_addr_t space. The code currently assumes that the offsets on the sourc

[Qemu-devel] [PATCH v2 04/12] rdma typos

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" A couple of typo fixes. Signed-off-by: Dr. David Alan Gilbert --- migration/rdma.c | 6 +++--- trace-events | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index bc73ff8..44ed996 100644 --- a/migrat

[Qemu-devel] [PATCH v2 08/12] Allow rdma_delete_block to work without the hash

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" In the next patch we remove the hash on the destination, rdma_delete_block does two things with the hash which can be avoided: a) The caller passes the offset and rdma_delete_block looks it up in the hash; fixed by getting the caller to pass the block b) Th

[Qemu-devel] [PATCH v2 12/12] Fail more cleanly in mismatched RAM cases

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" If the number of RAMBlocks was different on the source from the destination, QEMU would hang waiting for a disconnect on the source and wouldn't release from that hang until the destination was manually killed. Mark the stream as being in error, this causes the des

[Qemu-devel] [PATCH v2 07/12] Rework ram_control_load_hook to hook during block load

2015-06-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" We need the names of RAMBlocks as they're loaded for RDMA, reuse a slightly modified ram_control_load_hook: a) Pass a 'data' parameter to use for the name in the block-reg case b) Only some hook types now require the presence of a hook function. Signed-off

[Qemu-devel] [PATCH] Fix older machine type compatibility on power with section footers

2015-06-12 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" I forgot to add compatibility for Power when adding section footers. Signed-off-by: Dr. David Alan Gilbert Fixes: 37fb569c0198cba58e3e --- hw/ppc/spapr.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index f174e5a..01f8da8

[Qemu-devel] [PATCH v7 03/42] Init page sizes in qtest

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" One of my patches used a loop that was based on host page size; it dies in qtest since qtest hadn't bothered init'ing it. Signed-off-by: Dr. David Alan Gilbert --- qtest.c | 1 + 1 file changed, 1 insertion(+) diff --git a/qtest.c b/qtest.c index 05cefd2..8e1034

[Qemu-devel] [PATCH v7 04/42] qemu_ram_block_from_host

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Postcopy sends RAMBlock names and offsets over the wire (since it can't rely on the order of ramaddr being the same), and it starts out with HVA fault addresses from the kernel. qemu_ram_block_from_host translates a HVA into a RAMBlock, an offset in the RAMBlock an

[Qemu-devel] [PATCH v7 00/42] Postcopy implementation

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This is the 7th cut of my version of postcopy; it is designed for use with the Linux kernel additions posted by Andrea Arcangeli here: git clone --reference linux -b userfault21 git://git.kernel.org/pub/scm/linux/kernel/git/andrea/aa.git Note this API is slightl

[Qemu-devel] [PATCH v7 02/42] Provide runtime Target page information

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The migration code generally is built target-independent, however there are a few places where knowing the target page size would avoid artificially moving stuff into migration/ram.c. Provide 'qemu_target_page_bits()' that returns TARGET_PAGE_BITS to other bits of

[Qemu-devel] [PATCH v7 05/42] Add qemu_get_buffer_less_copy to avoid copies some of the time

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" qemu_get_buffer always copies the data it reads to a users buffer, however in many cases the file buffer inside qemu_file could be given back to the caller, avoiding the copy. This isn't always possible depending on the size and alignment of the data. Thus 'qemu_g

[Qemu-devel] [PATCH v7 01/42] Start documenting how postcopy works.

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert --- docs/migration.txt | 167 + 1 file changed, 167 insertions(+) diff --git a/docs/migration.txt b/docs/migration.txt index f6df4be..b4b93d1 100644 --- a/docs/migration.txt

[Qemu-devel] [PATCH v7 09/42] Rename save_live_complete to save_live_complete_precopy

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" In postcopy we're going to need to perform the complete phase for postcopiable devices at a different point, start out by renaming all of the 'complete's to make the difference obvious. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Amit Shah --- hw/ppc/spap

[Qemu-devel] [PATCH v7 10/42] Return path: Open a return path on QEMUFile for sockets

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Postcopy needs a method to send messages from the destination back to the source, this is the 'return path'. Wire it up for 'socket' QEMUFile's. Signed-off-by: Dr. David Alan Gilbert --- include/migration/qemu-file.h | 7 + migration/qemu-file-unix.c| 6

[Qemu-devel] [PATCH v7 07/42] ram_debug_dump_bitmap: Dump a migration bitmap as text

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Useful for debugging the migration bitmap and other bitmaps of the same format (including the sentmap in postcopy). The bitmap is printed to stderr. Lines that are all the expected value are excluded so the output can be quite compact for many bitmaps. Signed-off-

[Qemu-devel] [PATCH v7 13/42] Return path: Control commands

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add two src->dest commands: * OPEN_RETURN_PATH - To request that the destination open the return path * PING - Request an acknowledge from the destination Signed-off-by: Dr. David Alan Gilbert Reviewed-by: David Gibson --- include/migration/migration.h |

[Qemu-devel] [PATCH v7 06/42] Add wrapper for setting blocking status on a QEMUFile

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add a wrapper to change the blocking status on a QEMUFile rather than having to use qemu_set_block(qemu_get_fd(f)); it seems best to avoid exposing the fd since not all QEMUFile's really have one. With this wrapper we could move the implementation down to be differ

[Qemu-devel] [PATCH v7 14/42] Return path: Send responses from destination to source

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add migrate_send_rp_message to send a message from destination to source along the return path. (It uses a mutex to let it be called from multiple threads) Add migrate_send_rp_shut to send a 'shut' message to indicate the destination is finished with the RP. Ad

[Qemu-devel] [PATCH v7 08/42] migrate_init: Call from savevm

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Suspend to file is very much like a migrate, and it makes life easier if we have the Migration state available, so initialise it in the savevm.c code for suspending. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: David Gibson --- include/migration/migration.

[Qemu-devel] [PATCH v7 22/42] migrate_start_postcopy: Command to trigger transition to postcopy

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Once postcopy is enabled (with migrate_set_capability), the migration will still start on precopy mode. To cause a transition into postcopy the: migrate_start_postcopy command must be issued. Postcopy will start sometime after this (when it's next checked in t

[Qemu-devel] [PATCH v7 16/42] Rework loadvm path for subloops

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Postcopy needs to have two migration streams loading concurrently; one from memory (with the device state) and the other from the fd with the memory transactions. Split the core of qemu_loadvm_state out so we can use it for both. Allow the inner loadvm loop to qui

[Qemu-devel] [PATCH v7 17/42] Add migration-capability boolean for postcopy-ram.

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The 'postcopy ram' capability allows postcopy migration of RAM; note that the migration starts off in precopy mode until postcopy mode is triggered (see the migrate_start_postcopy patch later in the series). Signed-off-by: Dr. David Alan Gilbert --- include/migra

[Qemu-devel] [PATCH v7 11/42] Return path: socket_writev_buffer: Block even on non-blocking fd's

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The destination sets the fd to non-blocking on incoming migrations; this also affects the return path from the destination, and thus we need to make sure we can safely write to the return path. Signed-off-by: Dr. David Alan Gilbert --- migration/qemu-file-unix.c

[Qemu-devel] [PATCH v7 23/42] MIGRATION_STATUS_POSTCOPY_ACTIVE: Add new migration state

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" 'MIGRATION_STATUS_POSTCOPY_ACTIVE' is entered after migrate_start_postcopy 'migration_postcopy_phase' is provided for other sections to know if they're in postcopy. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: David Gibson Reviewed-by: Eric Blake --- inc

[Qemu-devel] [PATCH v7 19/42] MIG_CMD_PACKAGED: Send a packaged chunk of migration stream

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" MIG_CMD_PACKAGED is a migration command that wraps a chunk of migration stream inside a package whose length can be determined purely by reading its header. The destination guarantees that the whole MIG_CMD_PACKAGED is read off the stream prior to parsing the conte

[Qemu-devel] [PATCH v7 18/42] Add wrappers and handlers for sending/receiving the postcopy-ram migration messages.

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The state of the postcopy process is managed via a series of messages; * Add wrappers and handlers for sending/receiving these messages * Add state variable that track the current state of postcopy Signed-off-by: Dr. David Alan Gilbert --- include/migration

[Qemu-devel] [PATCH v7 25/42] Postcopy: Maintain sentmap and calculate discard

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Where postcopy is preceeded by a period of precopy, the destination will have received pages that may have been dirtied on the source after the page was sent. The destination must throw these pages away before starting it's CPUs. Maintain a 'sentmap' of pages that

[Qemu-devel] [PATCH v7 12/42] Migration commands

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Create QEMU_VM_COMMAND section type for sending commands from source to destination. These commands are not intended to convey guest state but to control the migration process. For use in postcopy. Signed-off-by: Dr. David Alan Gilbert --- include/migration/mig

[Qemu-devel] [PATCH v7 24/42] Add qemu_savevm_state_complete_postcopy

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add qemu_savevm_state_complete_postcopy to complement qemu_savevm_state_complete_precopy together with a new save_live_complete_postcopy method on devices. The save_live_complete_precopy method is called on all devices during a precopy migration, and all non-postco

[Qemu-devel] [PATCH v7 26/42] postcopy: Incoming initialisation

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert Reviewed-by: David Gibson --- include/migration/migration.h| 3 + include/migration/postcopy-ram.h | 12 migration/postcopy-ram.c | 116 +++ migration/ram.c

[Qemu-devel] [PATCH v7 15/42] Return path: Source handling of return path

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Open a return path, and handle messages that are received upon it. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 8 ++ migration/migration.c | 177 +- trace-events | 1

[Qemu-devel] [PATCH v7 27/42] postcopy: ram_enable_notify to switch on userfault

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Mark the area of RAM as 'userfault' Start up a fault-thread to handle any userfaults we might receive from it (to be filled in later) Signed-off-by: Dr. David Alan Gilbert Reviewed-by: David Gibson --- include/migration/migration.h| 3 ++ include/migration/

[Qemu-devel] [PATCH v7 30/42] Page request: Add MIG_RP_MSG_REQ_PAGES reverse command

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add MIG_RP_MSG_REQ_PAGES command on Return path for the postcopy destination to request a page from the source. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 4 +++ migration/migration.c | 70 ++

[Qemu-devel] [PATCH v7 20/42] Modify save_live_pending for postcopy

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Modify save_live_pending to return separate postcopiable and non-postcopiable counts. Signed-off-by: Dr. David Alan Gilbert --- include/migration/vmstate.h | 5 +++-- include/sysemu/sysemu.h | 4 +++- migration/block.c | 7 +-- migration/migr

[Qemu-devel] [PATCH v7 32/42] Page request: Consume pages off the post-copy queue

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" When transmitting RAM pages, consume pages that have been queued by MIG_RPCOMM_REQPAGE commands and send them ahead of normal page scanning. Note: a) After a queued page the linear walk carries on from after the unqueued page; there is a reasonable chance that th

[Qemu-devel] [PATCH v7 31/42] Page request: Process incoming page request

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" On receiving MIG_RPCOMM_REQ_PAGES look up the address and queue the page. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 21 +++ migration/migration.c | 36 + migration/ram.c | 6

[Qemu-devel] [PATCH v7 21/42] postcopy: OS support test

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Provide a check to see if the OS we're running on has all the bits needed for postcopy. Creates postcopy-ram.c which will get most of the other helpers we need. Signed-off-by: Dr. David Alan Gilbert --- include/migration/postcopy-ram.h | 19 + migration/Mak

[Qemu-devel] [PATCH v7 33/42] postcopy_ram.c: place_page and helpers

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" postcopy_place_page (etc) provide a way for postcopy to place a page into guests memory atomically (using the copy ioctl on the ufd). Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h| 1 + include/migration/postcopy-ram.h | 16

[Qemu-devel] [PATCH v7 35/42] Don't sync dirty bitmaps in postcopy

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Once we're in postcopy the source processors are stopped and memory shouldn't change any more, so there's no need to look at the dirty map. There are two notes to this: 1) If we do resync and a page had changed then the page would get sent again, which the d

[Qemu-devel] [PATCH v7 37/42] Postcopy; Handle userfault requests

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" userfaultfd is a Linux syscall that gives an fd that receives a stream of notifications of accesses to pages registered with it and allows the program to acknowledge those stalls and tell the accessing thread to carry on. We convert the requests from the kernel int

[Qemu-devel] [PATCH v7 28/42] Postcopy: Postcopy startup in migration thread

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Rework the migration thread to setup and start postcopy. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 3 + migration/migration.c | 166 -- trace-events | 4 + 3 file

[Qemu-devel] [PATCH v7 29/42] Postcopy end in migration_thread

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The end of migration in postcopy is a bit different since some of the things normally done at the end of migration have already been done on the transition to postcopy. The end of migration code is getting a bit complciated now, so move out into its own function.

[Qemu-devel] [PATCH v7 38/42] Start up a postcopy/listener thread ready for incoming page data

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The loading of a device state (during postcopy) may access guest memory that's still on the source machine and thus might need a page fill; split off a separate thread that handles the incoming page data so that the original incoming migration code can finish off th

[Qemu-devel] [PATCH v7 39/42] postcopy: Wire up loadvm_postcopy_handle_ commands

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Wire up more of the handlers for the commands on the destination side, in particular loadvm_postcopy_handle_run now has enough to start the guest running. Signed-off-by: Dr. David Alan Gilbert --- migration/savevm.c | 29 - trace-event

[Qemu-devel] [PATCH v7 42/42] Inhibit ballooning during postcopy

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Postcopy detects accesses to pages that haven't been transferred yet using userfaultfd, and it causes exceptions on pages that are 'not present'. Ballooning also causes pages to be marked as 'not present' when the guest inflates the balloon. Potentially a balloon co

[Qemu-devel] [PATCH v7 34/42] Postcopy: Use helpers to map pages during migration

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" In postcopy, the destination guest is running at the same time as it's receiving pages; as we receive new pages we must put them into the guests address space atomically to avoid a running CPU accessing a partially written page. Use the helpers in postcopy-ram.c to

[Qemu-devel] [PATCH v7 40/42] End of migration for postcopy

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Tweak the end of migration cleanup; we don't want to close stuff down at the end of the main stream, since the postcopy is still sending pages on the other thread. Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 25 - tra

[Qemu-devel] [PATCH v7 36/42] Host page!=target page: Cleanup bitmaps

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Prior to the start of postcopy, ensure that everything that will be transferred later is a whole host-page in size. This is accomplished by discarding partially transferred host pages and marking any that are partially dirty as fully dirty. Signed-off-by: Dr. Davi

[Qemu-devel] [PATCH v7 41/42] Disable mlock around incoming postcopy

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Userfault doesn't work with mlock; mlock is designed to nail down pages so they don't move, userfault is designed to tell you when they're not there. munlock the pages we userfault protect before postcopy. mlock everything again at the end if mlock is enabled. Sig

[Qemu-devel] [PATCH 0/5] Migration cleanups

2015-08-13 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This series is a set of small cleanups, some of which are from my postcopy series. Dave Dr. David Alan Gilbert (5): migration/ram.c: Use RAMBlock rather than MemoryRegion Split out end of migration code from migration_thread Init page sizes in qtest migrat

[Qemu-devel] [PATCH 2/5] Split out end of migration code from migration_thread

2015-08-13 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The code that gets run at the end of the migration process is getting large, and I'm about to add more for postcopy. Split it into a separate function. Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 75 --

[Qemu-devel] [PATCH 5/5] migration: qemu-file more size_t'ifying

2015-08-13 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This time convert the external functions: qemu_get_buffer, qemu_peek_buffer qemu_put_buffer and qemu_put_buffer_async Signed-off-by: Dr. David Alan Gilbert --- include/migration/qemu-file.h | 10 +- migration/qemu-file.c | 22 +++--

[Qemu-devel] [PATCH 1/5] migration/ram.c: Use RAMBlock rather than MemoryRegion

2015-08-13 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" RAM migration mainly works on RAMBlocks but in a few places uses data from MemoryRegions to access the same information that's already held in RAMBlocks; clean it up just to avoid the MemoryRegion use. Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 2

[Qemu-devel] [PATCH 3/5] Init page sizes in qtest

2015-08-13 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" One of my patches used a loop that was based on host page size; it dies in qtest since qtest hadn't bothered init'ing it. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Amit Shah --- qtest.c | 1 + 1 file changed, 1 insertion(+)

[Qemu-devel] [PATCH 4/5] migration: size_t'ify some of qemu-file

2015-08-13 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This is a start on using size_t more in qemu-file and friends; it fixes up QEMUFilePutBufferFunc and QEMUFileGetBufferFunc to take size_t lengths and return ssize_t return values (like read(2)) and fixes up all the different implementations of them. Note that I've

[Qemu-devel] [PATCH] Move RAMBlock and ram_list to ram_addr.h

2015-08-14 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert --- include/exec/cpu-all.h | 41 - include/exec/ram_addr.h | 40 2 files changed, 40 insertions(+), 41 deletions(-) diff --git a/include/exec/c

[Qemu-devel] [PATCH] trace-events: Add hmp completion

2015-08-14 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add completion for the trace event names in the hmp trace-event command. Signed-off-by: Dr. David Alan Gilbert --- hmp-commands.hx | 1 + hmp.h | 1 + monitor.c | 20 3 files changed, 22 insertions(+) diff --git a/hmp-comma

[Qemu-devel] [PATCH] Migration compatibility for serial

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Older QEMUs dont understand the new (sub)sections that may be generated in the serial device. Limit their generation to newer machine types. Signed-off-by: Dr. David Alan Gilbert --- hw/char/serial.c | 19 +-- hw/i386/pc_piix.c|

[Qemu-devel] [PATCH] Peek dont read for vmdescription

2015-06-22 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The VMDescription section maybe after the EOF mark, the current code does a 'qemu_get_byte' and either gets the header byte identifying the description or an error (which it ignores). Doing the 'get' upsets RDMA which hangs on old machine types without the VMDescri

[Qemu-devel] [PATCH] RDMA: Missing free in (probably unreachable) error path

2015-06-22 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Coverity CID 1307773 Signed-off-by: Dr. David Alan Gilbert --- migration/rdma.c | 1 + 1 file changed, 1 insertion(+) diff --git a/migration/rdma.c b/migration/rdma.c index 53e611e..2a9e0ce 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3395,6 +3395,7

[Qemu-devel] [PATCH] gdb command: qemu iohandlers

2015-06-23 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add a gdb command to print the current set of IOHandlers and if one of them is a thread yielding for data print the backtrace. Useful for debugging why an incoming migration has stalled, e.g. {fd_read = 0x7fd4c8e40d00 , fd_write = 0x0, opaque = 0x7fd4b8

[Qemu-devel] [PATCH] Only try and read a VMDescription if it should be there

2015-06-23 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The VMDescription section maybe after the EOF mark, the current code does a 'qemu_get_byte' and either gets the header byte identifying the description or an error (which it ignores). Doing the 'get' upsets RDMA which hangs on old machine types without the VMDescri

[Qemu-devel] [PATCH 1/1] Fix glib_subprocess test

2015-06-24 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" A typo means that the tests dependent on glib with subprocess support are never run. Fixes: 9d41401b90fa10b335d2e739149d36437cfbf622 Signed-off-by: Dr. David Alan Gilbert --- configure | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure

[Qemu-devel] [PATCH] check_section_footers: Check the correct section_id

2015-07-02 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The section footers check was incorrectly checking the section_id in the SaveStateEntry not the LoadStateEntry. These can validly be different if the two QEMU instances have instantiated their devices in a different order. The test only cares that we're finishing

[Qemu-devel] [PATCH] RDMA: Reduce restriction on block length match

2015-07-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" My e4d633207 patch has an over zealous sanity check that checked the lengths of the RAM Blocks on source/destination were the same. This isn't true because of the 'used_length' trick for RAM blocks like the ACPI table that vary in size. Prior to that patch RDMA wo

[Qemu-devel] [PATCH] RDMA: Reduce restriction on block length match

2015-07-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" My e4d633207 patch has an over zealous sanity check that checked the lengths of the RAM Blocks on source/destination were the same. This isn't true because of the 'used_length' trick for RAM blocks like the ACPI table that vary in size. Prior to that patch RDMA wo

[Qemu-devel] [PATCH] RDMA: Fix error exits (for 2.4)

2015-07-10 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The error checks I added used 'break' after the error, but I'm in a switch inside the while loop, so they need to be 'goto out'. Spotted by coverity; entries 1311368 and 1311369 Fixes: afcddefd Signed-off-by: Dr. David Alan Gilbert --- migration/rdma.c | 8

[Qemu-devel] [RFC 1/7] memory: address_space_iterate

2017-11-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Iterate through an address space calling a function for each section. The iteration is done in order. Signed-off-by: Dr. David Alan Gilbert --- include/exec/memory.h | 23 +++ memory.c | 22 ++ 2 files changed

[Qemu-devel] [RFC 0/7] Rework vhost memory region updates

2017-11-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Hi, This is an experimental set that reworks the way the vhost code handles changes in physical address space layout that came from a discussion with Igor. Instead of updating and trying to merge sections of address space on each add/remove callback, we wait unti

[Qemu-devel] [RFC 7/7] vhost: Remove vhost_set_memory and children

2017-11-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Remove the old update mechanism, vhost_set_memory, and the functions it uses. Signed-off-by: Dr. David Alan Gilbert --- hw/virtio/vhost.c | 239 -- 1 file changed, 239 deletions(-) diff --git a/hw/virtio/vhost.

[Qemu-devel] [RFC 2/7] vhost: Move log_dirty check

2017-11-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Move the log_dirty check into vhost_section. Signed-off-by: Dr. David Alan Gilbert --- hw/virtio/trace-events | 3 +++ hw/virtio/vhost.c | 20 +--- 2 files changed, 16 insertions(+), 7 deletions(-) diff --git a/hw/virtio/trace-events b/hw/v

[Qemu-devel] [RFC 3/7] vhost: New memory update functions

2017-11-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" vhost_update_mem will replace the existing update mechanism. They make use of the Flatview we have now to make the update simpler. This commit just adds the basic structure. Signed-off-by: Dr. David Alan Gilbert --- hw/virtio/vhost.c | 44

[Qemu-devel] [RFC 6/7] vhost: Copy updated region data into device state

2017-11-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Copy the temporary region data we calculated into the device state. Signed-off-by: Dr. David Alan Gilbert --- hw/virtio/vhost.c | 16 +++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index ae3d57

[Qemu-devel] [RFC 5/7] vhost: Compare new and old memory lists

2017-11-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Compare the memory section list we've just built with the old one and produce an upper/lower bound of addresses changed. Signed-off-by: Dr. David Alan Gilbert --- hw/virtio/trace-events | 3 ++ hw/virtio/vhost.c | 74

[Qemu-devel] [RFC 4/7] vhost: update_mem_cb implementation

2017-11-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add the meat of update_mem_cb; this is called for each region, to add a region to our temporary list. Our temporary list is in order we look to see if this region can be merged with the current head of list. Signed-off-by: Dr. David Alan Gilbert --- hw/virtio/tr

[Qemu-devel] [RFC v2 1/8] memory: address_space_iterate

2017-12-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Iterate through an address space calling a function for each section. The iteration is done in order. Signed-off-by: Dr. David Alan Gilbert --- include/exec/memory.h | 23 +++ memory.c | 22 ++ 2 files changed

[Qemu-devel] [RFC v2 2/8] vhost: Move log_dirty check

2017-12-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Move the log_dirty check into vhost_section. Signed-off-by: Dr. David Alan Gilbert --- hw/virtio/trace-events | 3 +++ hw/virtio/vhost.c | 20 +--- 2 files changed, 16 insertions(+), 7 deletions(-) diff --git a/hw/virtio/trace-events b/hw/v

[Qemu-devel] [RFC v2 0/8] Rework vhost memory region updates

2017-12-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Hi, This is an experimental set that reworks the way the vhost code handles changes in physical address space layout that came from a discussion with Igor. It's intention is to simplify a lot of the update code, and to make it easier for the postcopy+shared code

[Qemu-devel] [RFC v2 5/8] vhost: update_mem_cb implementation

2017-12-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add the meat of update_mem_cb; this is called for each region, to add a region to our temporary list. Our temporary list is in order we look to see if this region can be merged with the current head of list. Signed-off-by: Dr. David Alan Gilbert --- hw/virtio/tr

[Qemu-devel] [RFC v2 4/8] vhost: New memory update functions

2017-12-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" vhost_update_mem will replace the existing update mechanism. They make use of the Flatview we have now to make the update simpler. This commit just adds the basic structure. Signed-off-by: Dr. David Alan Gilbert --- hw/virtio/vhost.c | 51

[Qemu-devel] [RFC v2 3/8] vhost: Simplify ring verification checks

2017-12-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" vhost_verify_ring_mappings() were used to verify that rings are still accessible and related memory hasn't been moved after flatview is updated. It were doing checks by mapping ring's GPA+len and checking that HVA hasn't changed with new memory map. To avoid maybe

[Qemu-devel] [RFC v2 6/8] vhost: Compare and copy updated region data into device state

2017-12-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Compare the temporary region data with the original, and if it's different update the original in the device state. Signed-off-by: Dr. David Alan Gilbert --- hw/virtio/trace-events | 2 ++ hw/virtio/vhost.c | 19 +-- 2 files changed, 19 inse

<    4   5   6   7   8   9   10   11   12   13   >