[Qemu-devel] [PATCH 33/46] Postcopy: Create a fault handler thread before marking the ram as userfault

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 3 +++ postcopy-ram.c| 23 +++ 2 files changed, 26 insertions(+) diff --git a/include/migration/migration.h b/include/migration/migration.h index 397f41

[Qemu-devel] [PATCH 35/46] Page request: Process incoming page request

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" On receiving MIG_RPCOMM_REQPAGES look up the address and queue the page. Signed-off-by: Dr. David Alan Gilbert --- arch_init.c | 52 +++ include/migration/migration.h | 26 ++ include/q

[Qemu-devel] [PATCH 39/46] Postcopy: Use helpers to map pages during migration

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" In postcopy, the destination guest is running at the same time as it's receiving pages; as we receive new pages we must put them into the guests address space atomically to avoid a running CPU accessing a partially written page. Use the helpers in postcopy-ram.c to

[Qemu-devel] [PATCH 16/46] Add migration-capability boolean for postcopy-ram.

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 1 + migration.c | 9 + qapi-schema.json | 6 +- 3 files changed, 15 insertions(+), 1 deletion(-) diff --git a/include/migration/migration.h b/inc

[Qemu-devel] [PATCH 17/46] Add wrappers and handlers for sending/receiving the postcopy-ram migration messages.

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add state variable showing current incoming postcopy state. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 8 ++ include/sysemu/sysemu.h | 23 savevm.c | 313 ++

[Qemu-devel] [PATCH 15/46] Rework loadvm path for subloops

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Postcopy needs to have two migration streams loading concurrently; one from memory (with the device state) and the other from the fd with the memory transactions. Split the core of qemu_loadvm_state out so we can use it for both. Allow the inner loadvm loop to qui

[Qemu-devel] [PATCH 40/46] qemu_ram_block_from_host

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Postcopy sends RAMBlock names and offsets over the wire (since it can't rely on the order of ramaddr being the same), and it starts out with HVA fault addresses from the kernel. qemu_ram_block_from_host translates a HVA into a RAMBlock, an offset in the RAMBlock, t

[Qemu-devel] [PATCH 21/46] postcopy: OS support test

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Provide a check to see if the OS we're running on has all the bits needed for postcopy. Creates postcopy-ram.c which will get most of the other helpers we need. Signed-off-by: Dr. David Alan Gilbert --- Makefile.objs| 2 +- include/migratio

[Qemu-devel] [PATCH 22/46] Migration parameters: Add qmp/hmp commands for setting/viewing

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add somewhere for the various migration parameters to be set with one command; As suggested in the thread: http://lists.gnu.org/archive/html/qemu-devel/2012-11/msg00243.html There are many existing migration parameters that are scattered over many individual comma

[Qemu-devel] [PATCH 34/46] Page request: Add MIG_RPCOMM_REQPAGES reverse command

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add MIG_RPCOMM_REQPAGES command on Return path for the postcopy destination to request a page from the source. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 3 ++ migration.c | 75

[Qemu-devel] [PATCH 11/46] Return path: Send responses from destination to source

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add migrate_send_rp_message to send a message from destination to source along the return path. (It uses a mutex to let it be called from multiple threads) Add migrate_send_rp_ack to send an 'ack' message Use it in the CMD_REQACK handler Signed-off-by: Dr. Dav

[Qemu-devel] [PATCH 19/46] migrate_init: Call from savevm

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Suspend to file is very much like a migrate, and it makes life easier if we have the Migration state available, so initialise it in the savevm.c code for suspending. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 1 + include/qemu/typed

[Qemu-devel] [PATCH 43/46] postcopy: Wire up loadvm_postcopy_ram_handle_{run, end} commands

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Wire up more of the handlers for the commands on the destination side, in particular loadvm_postcopy_ram_handle_run now has enough to start the guest running. Signed-off-by: Dr. David Alan Gilbert --- savevm.c | 47 ++-

[Qemu-devel] [PATCH 01/46] qemu_ram_foreach_block: pass up error value, and down the ramblock name

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" check the return value of the function it calls and error if it's none-0 Fixup qemu_rdma_init_one_block that is the only current caller, and __qemu_rdma_add_block the only function it calls using it. Pass the name of the ramblock to the function; helps in debuggi

[Qemu-devel] [PATCH 20/46] Allow savevm handlers to state whether they could go into postcopy

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Use that to split the qemu_savevm_state_pending counts into postcopiable and non-postcopiable amounts Signed-off-by: Dr. David Alan Gilbert --- arch_init.c | 7 +++ include/migration/vmstate.h | 2 +- include/sysemu/sysemu.h | 4 +++- m

[Qemu-devel] [PATCH 41/46] Handle userfault requests (although userfaultfd not done yet)

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 1 + postcopy-ram.c| 93 +-- 2 files changed, 90 insertions(+), 4 deletions(-) diff --git a/include/migration/migration.h b/include/

[Qemu-devel] [PATCH 38/46] postcopy_ram.c: place_page and helpers

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" postcopy_place_page (etc) provide a way for postcopy to place a page into guests memory atomically (using the new remap_anon_pages syscall). Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h| 1 + include/migration/postcopy-ram.h | 23

[Qemu-devel] [PATCH 13/46] qemu_loadvm debug

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add lots of DPRINTF debug in qemu_loadvm* Signed-off-by: Dr. David Alan Gilbert --- savevm.c | 23 +-- 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/savevm.c b/savevm.c index 16b672b..662a910 100644 --- a/savevm.c +++ b/savevm

[Qemu-devel] [PATCH 02/46] Move QEMUFile structure to qemu-file.h

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This is mostly as an easy way to get to the MigrationIncomingState that I'm hanging off the file. Signed-off-by: Dr. David Alan Gilbert buf_size = pending; len = f->ops->get_buffer(f->opaque, f->buf + pending, f->pos, -IO_BUF_SIZE - p

[Qemu-devel] [PATCH 05/46] Add qemu_get_counted_string to read a string prefixed by a count byte

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" and use it in loadvm_state. Signed-off-by: Dr. David Alan Gilbert --- include/migration/qemu-file.h | 2 ++ qemu-file.c | 15 +++ savevm.c | 18 ++ 3 files changed, 27 insertions(+), 8 deletions(

[Qemu-devel] [PATCH 14/46] ram_debug_dump_bitmap: Dump a migration bitmap as text

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Misses out lines that are all 0 so can be quite compact depending on the circumstance. Signed-off-by: Dr. David Alan Gilbert --- arch_init.c | 39 +++ include/migration/migration.h | 1 + 2 files changed, 40

[Qemu-devel] [PATCH 12/46] Return path: Source handling of return path

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Open a return path, and handle messages that are received upon it. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 10 +++ migration.c | 142 +- 2 files changed, 151 insertions(+

[Qemu-devel] [PATCH 18/46] QEMU_VM_CMD_PACKAGED: Send a packaged chunk of migration stream

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" QEMU_VM_CMD_PACKAGED is a migration command that allows a chunk of migration stream to be sent in one go, and be received by a separate instance of the loadvm loop while not interacting with the migration stream. This is used by postcopy to load device state (from

[Qemu-devel] [PATCH 06/46] Create MigrationIncomingState

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" There are currently lots of pieces of incoming migration state scattered around, and postcopy is adding more, and it seems better to try and keep it together. allocate MIS in process_incoming_migration_co Add MIS to QEMUFile Signed-off-by: Dr. David Alan Gilbert

[Qemu-devel] [PATCH 42/46] Start up a postcopy/listener thread ready for incoming page data

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The loading of a device state (during postcopy) may access guest memory that's still on the source machine and thus might need a page fill; split off a separate thread that handles the incoming page data so that the original incoming migration code can finish off th

[Qemu-devel] [PATCH] Fix older machine type compatibility on power with section footers

2015-06-12 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" I forgot to add compatibility for Power when adding section footers. Signed-off-by: Dr. David Alan Gilbert Fixes: 37fb569c0198cba58e3e --- hw/ppc/spapr.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index f174e5a..01f8da8

[Qemu-devel] [PATCH v7 03/42] Init page sizes in qtest

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" One of my patches used a loop that was based on host page size; it dies in qtest since qtest hadn't bothered init'ing it. Signed-off-by: Dr. David Alan Gilbert --- qtest.c | 1 + 1 file changed, 1 insertion(+) diff --git a/qtest.c b/qtest.c index 05cefd2..8e1034

[Qemu-devel] [PATCH v7 04/42] qemu_ram_block_from_host

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Postcopy sends RAMBlock names and offsets over the wire (since it can't rely on the order of ramaddr being the same), and it starts out with HVA fault addresses from the kernel. qemu_ram_block_from_host translates a HVA into a RAMBlock, an offset in the RAMBlock an

[Qemu-devel] [PATCH v7 00/42] Postcopy implementation

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This is the 7th cut of my version of postcopy; it is designed for use with the Linux kernel additions posted by Andrea Arcangeli here: git clone --reference linux -b userfault21 git://git.kernel.org/pub/scm/linux/kernel/git/andrea/aa.git Note this API is slightl

[Qemu-devel] [PATCH v7 02/42] Provide runtime Target page information

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The migration code generally is built target-independent, however there are a few places where knowing the target page size would avoid artificially moving stuff into migration/ram.c. Provide 'qemu_target_page_bits()' that returns TARGET_PAGE_BITS to other bits of

[Qemu-devel] [PATCH v7 05/42] Add qemu_get_buffer_less_copy to avoid copies some of the time

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" qemu_get_buffer always copies the data it reads to a users buffer, however in many cases the file buffer inside qemu_file could be given back to the caller, avoiding the copy. This isn't always possible depending on the size and alignment of the data. Thus 'qemu_g

[Qemu-devel] [PATCH v7 01/42] Start documenting how postcopy works.

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert --- docs/migration.txt | 167 + 1 file changed, 167 insertions(+) diff --git a/docs/migration.txt b/docs/migration.txt index f6df4be..b4b93d1 100644 --- a/docs/migration.txt

[Qemu-devel] [PATCH v7 09/42] Rename save_live_complete to save_live_complete_precopy

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" In postcopy we're going to need to perform the complete phase for postcopiable devices at a different point, start out by renaming all of the 'complete's to make the difference obvious. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Amit Shah --- hw/ppc/spap

[Qemu-devel] [PATCH v7 10/42] Return path: Open a return path on QEMUFile for sockets

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Postcopy needs a method to send messages from the destination back to the source, this is the 'return path'. Wire it up for 'socket' QEMUFile's. Signed-off-by: Dr. David Alan Gilbert --- include/migration/qemu-file.h | 7 + migration/qemu-file-unix.c| 6

[Qemu-devel] [PATCH v7 07/42] ram_debug_dump_bitmap: Dump a migration bitmap as text

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Useful for debugging the migration bitmap and other bitmaps of the same format (including the sentmap in postcopy). The bitmap is printed to stderr. Lines that are all the expected value are excluded so the output can be quite compact for many bitmaps. Signed-off-

[Qemu-devel] [PATCH v7 13/42] Return path: Control commands

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add two src->dest commands: * OPEN_RETURN_PATH - To request that the destination open the return path * PING - Request an acknowledge from the destination Signed-off-by: Dr. David Alan Gilbert Reviewed-by: David Gibson --- include/migration/migration.h |

[Qemu-devel] [PATCH v7 06/42] Add wrapper for setting blocking status on a QEMUFile

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add a wrapper to change the blocking status on a QEMUFile rather than having to use qemu_set_block(qemu_get_fd(f)); it seems best to avoid exposing the fd since not all QEMUFile's really have one. With this wrapper we could move the implementation down to be differ

[Qemu-devel] [PATCH v7 14/42] Return path: Send responses from destination to source

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add migrate_send_rp_message to send a message from destination to source along the return path. (It uses a mutex to let it be called from multiple threads) Add migrate_send_rp_shut to send a 'shut' message to indicate the destination is finished with the RP. Ad

[Qemu-devel] [PATCH v7 08/42] migrate_init: Call from savevm

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Suspend to file is very much like a migrate, and it makes life easier if we have the Migration state available, so initialise it in the savevm.c code for suspending. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: David Gibson --- include/migration/migration.

[Qemu-devel] [PATCH v7 22/42] migrate_start_postcopy: Command to trigger transition to postcopy

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Once postcopy is enabled (with migrate_set_capability), the migration will still start on precopy mode. To cause a transition into postcopy the: migrate_start_postcopy command must be issued. Postcopy will start sometime after this (when it's next checked in t

[Qemu-devel] [PATCH v7 16/42] Rework loadvm path for subloops

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Postcopy needs to have two migration streams loading concurrently; one from memory (with the device state) and the other from the fd with the memory transactions. Split the core of qemu_loadvm_state out so we can use it for both. Allow the inner loadvm loop to qui

[Qemu-devel] [PATCH v7 17/42] Add migration-capability boolean for postcopy-ram.

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The 'postcopy ram' capability allows postcopy migration of RAM; note that the migration starts off in precopy mode until postcopy mode is triggered (see the migrate_start_postcopy patch later in the series). Signed-off-by: Dr. David Alan Gilbert --- include/migra

[Qemu-devel] [PATCH v7 11/42] Return path: socket_writev_buffer: Block even on non-blocking fd's

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The destination sets the fd to non-blocking on incoming migrations; this also affects the return path from the destination, and thus we need to make sure we can safely write to the return path. Signed-off-by: Dr. David Alan Gilbert --- migration/qemu-file-unix.c

[Qemu-devel] [PATCH v7 23/42] MIGRATION_STATUS_POSTCOPY_ACTIVE: Add new migration state

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" 'MIGRATION_STATUS_POSTCOPY_ACTIVE' is entered after migrate_start_postcopy 'migration_postcopy_phase' is provided for other sections to know if they're in postcopy. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: David Gibson Reviewed-by: Eric Blake --- inc

[Qemu-devel] [PATCH v7 19/42] MIG_CMD_PACKAGED: Send a packaged chunk of migration stream

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" MIG_CMD_PACKAGED is a migration command that wraps a chunk of migration stream inside a package whose length can be determined purely by reading its header. The destination guarantees that the whole MIG_CMD_PACKAGED is read off the stream prior to parsing the conte

[Qemu-devel] [PATCH v7 18/42] Add wrappers and handlers for sending/receiving the postcopy-ram migration messages.

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The state of the postcopy process is managed via a series of messages; * Add wrappers and handlers for sending/receiving these messages * Add state variable that track the current state of postcopy Signed-off-by: Dr. David Alan Gilbert --- include/migration

[Qemu-devel] [PATCH v7 25/42] Postcopy: Maintain sentmap and calculate discard

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Where postcopy is preceeded by a period of precopy, the destination will have received pages that may have been dirtied on the source after the page was sent. The destination must throw these pages away before starting it's CPUs. Maintain a 'sentmap' of pages that

[Qemu-devel] [PATCH v7 12/42] Migration commands

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Create QEMU_VM_COMMAND section type for sending commands from source to destination. These commands are not intended to convey guest state but to control the migration process. For use in postcopy. Signed-off-by: Dr. David Alan Gilbert --- include/migration/mig

[Qemu-devel] [PATCH v7 24/42] Add qemu_savevm_state_complete_postcopy

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add qemu_savevm_state_complete_postcopy to complement qemu_savevm_state_complete_precopy together with a new save_live_complete_postcopy method on devices. The save_live_complete_precopy method is called on all devices during a precopy migration, and all non-postco

[Qemu-devel] [PATCH v7 26/42] postcopy: Incoming initialisation

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Signed-off-by: Dr. David Alan Gilbert Reviewed-by: David Gibson --- include/migration/migration.h| 3 + include/migration/postcopy-ram.h | 12 migration/postcopy-ram.c | 116 +++ migration/ram.c

[Qemu-devel] [PATCH v7 15/42] Return path: Source handling of return path

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Open a return path, and handle messages that are received upon it. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 8 ++ migration/migration.c | 177 +- trace-events | 1

[Qemu-devel] [PATCH v7 27/42] postcopy: ram_enable_notify to switch on userfault

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Mark the area of RAM as 'userfault' Start up a fault-thread to handle any userfaults we might receive from it (to be filled in later) Signed-off-by: Dr. David Alan Gilbert Reviewed-by: David Gibson --- include/migration/migration.h| 3 ++ include/migration/

[Qemu-devel] [PATCH v7 30/42] Page request: Add MIG_RP_MSG_REQ_PAGES reverse command

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add MIG_RP_MSG_REQ_PAGES command on Return path for the postcopy destination to request a page from the source. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 4 +++ migration/migration.c | 70 ++

[Qemu-devel] [PATCH v7 20/42] Modify save_live_pending for postcopy

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Modify save_live_pending to return separate postcopiable and non-postcopiable counts. Signed-off-by: Dr. David Alan Gilbert --- include/migration/vmstate.h | 5 +++-- include/sysemu/sysemu.h | 4 +++- migration/block.c | 7 +-- migration/migr

[Qemu-devel] [PATCH v7 32/42] Page request: Consume pages off the post-copy queue

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" When transmitting RAM pages, consume pages that have been queued by MIG_RPCOMM_REQPAGE commands and send them ahead of normal page scanning. Note: a) After a queued page the linear walk carries on from after the unqueued page; there is a reasonable chance that th

[Qemu-devel] [PATCH v7 31/42] Page request: Process incoming page request

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" On receiving MIG_RPCOMM_REQ_PAGES look up the address and queue the page. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 21 +++ migration/migration.c | 36 + migration/ram.c | 6

[Qemu-devel] [PATCH v7 21/42] postcopy: OS support test

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Provide a check to see if the OS we're running on has all the bits needed for postcopy. Creates postcopy-ram.c which will get most of the other helpers we need. Signed-off-by: Dr. David Alan Gilbert --- include/migration/postcopy-ram.h | 19 + migration/Mak

[Qemu-devel] [PATCH v7 33/42] postcopy_ram.c: place_page and helpers

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" postcopy_place_page (etc) provide a way for postcopy to place a page into guests memory atomically (using the copy ioctl on the ufd). Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h| 1 + include/migration/postcopy-ram.h | 16

[Qemu-devel] [PATCH v7 35/42] Don't sync dirty bitmaps in postcopy

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Once we're in postcopy the source processors are stopped and memory shouldn't change any more, so there's no need to look at the dirty map. There are two notes to this: 1) If we do resync and a page had changed then the page would get sent again, which the d

[Qemu-devel] [PATCH v7 37/42] Postcopy; Handle userfault requests

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" userfaultfd is a Linux syscall that gives an fd that receives a stream of notifications of accesses to pages registered with it and allows the program to acknowledge those stalls and tell the accessing thread to carry on. We convert the requests from the kernel int

[Qemu-devel] [PATCH v7 28/42] Postcopy: Postcopy startup in migration thread

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Rework the migration thread to setup and start postcopy. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 3 + migration/migration.c | 166 -- trace-events | 4 + 3 file

[Qemu-devel] [PATCH v7 29/42] Postcopy end in migration_thread

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The end of migration in postcopy is a bit different since some of the things normally done at the end of migration have already been done on the transition to postcopy. The end of migration code is getting a bit complciated now, so move out into its own function.

[Qemu-devel] [PATCH v7 38/42] Start up a postcopy/listener thread ready for incoming page data

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The loading of a device state (during postcopy) may access guest memory that's still on the source machine and thus might need a page fill; split off a separate thread that handles the incoming page data so that the original incoming migration code can finish off th

[Qemu-devel] [PATCH v7 39/42] postcopy: Wire up loadvm_postcopy_handle_ commands

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Wire up more of the handlers for the commands on the destination side, in particular loadvm_postcopy_handle_run now has enough to start the guest running. Signed-off-by: Dr. David Alan Gilbert --- migration/savevm.c | 29 - trace-event

[Qemu-devel] [PATCH v7 42/42] Inhibit ballooning during postcopy

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Postcopy detects accesses to pages that haven't been transferred yet using userfaultfd, and it causes exceptions on pages that are 'not present'. Ballooning also causes pages to be marked as 'not present' when the guest inflates the balloon. Potentially a balloon co

[Qemu-devel] [PATCH v7 34/42] Postcopy: Use helpers to map pages during migration

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" In postcopy, the destination guest is running at the same time as it's receiving pages; as we receive new pages we must put them into the guests address space atomically to avoid a running CPU accessing a partially written page. Use the helpers in postcopy-ram.c to

[Qemu-devel] [PATCH v7 40/42] End of migration for postcopy

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Tweak the end of migration cleanup; we don't want to close stuff down at the end of the main stream, since the postcopy is still sending pages on the other thread. Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 25 - tra

[Qemu-devel] [PATCH v7 36/42] Host page!=target page: Cleanup bitmaps

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Prior to the start of postcopy, ensure that everything that will be transferred later is a whole host-page in size. This is accomplished by discarding partially transferred host pages and marking any that are partially dirty as fully dirty. Signed-off-by: Dr. Davi

[Qemu-devel] [PATCH v7 41/42] Disable mlock around incoming postcopy

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Userfault doesn't work with mlock; mlock is designed to nail down pages so they don't move, userfault is designed to tell you when they're not there. munlock the pages we userfault protect before postcopy. mlock everything again at the end if mlock is enabled. Sig

[Qemu-devel] [PATCH] Migration compatibility for serial

2015-06-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Older QEMUs dont understand the new (sub)sections that may be generated in the serial device. Limit their generation to newer machine types. Signed-off-by: Dr. David Alan Gilbert --- hw/char/serial.c | 19 +-- hw/i386/pc_piix.c|

[Qemu-devel] [PATCH] Peek dont read for vmdescription

2015-06-22 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The VMDescription section maybe after the EOF mark, the current code does a 'qemu_get_byte' and either gets the header byte identifying the description or an error (which it ignores). Doing the 'get' upsets RDMA which hangs on old machine types without the VMDescri

[Qemu-devel] [PATCH] RDMA: Missing free in (probably unreachable) error path

2015-06-22 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Coverity CID 1307773 Signed-off-by: Dr. David Alan Gilbert --- migration/rdma.c | 1 + 1 file changed, 1 insertion(+) diff --git a/migration/rdma.c b/migration/rdma.c index 53e611e..2a9e0ce 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3395,6 +3395,7

[Qemu-devel] [PATCH] gdb command: qemu iohandlers

2015-06-23 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add a gdb command to print the current set of IOHandlers and if one of them is a thread yielding for data print the backtrace. Useful for debugging why an incoming migration has stalled, e.g. {fd_read = 0x7fd4c8e40d00 , fd_write = 0x0, opaque = 0x7fd4b8

[Qemu-devel] [PATCH] Only try and read a VMDescription if it should be there

2015-06-23 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The VMDescription section maybe after the EOF mark, the current code does a 'qemu_get_byte' and either gets the header byte identifying the description or an error (which it ignores). Doing the 'get' upsets RDMA which hangs on old machine types without the VMDescri

[Qemu-devel] [PATCH 1/2] postcopy: listen thread is never joined

2016-03-09 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" We don't join the listen thread, it does it's own cleanup. Mark as detached not joinable. Signed-off-by: Dr. David Alan Gilbert Reported-by: Paolo Bonzini --- migration/savevm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/savevm

[Qemu-devel] [PATCH 0/2] Postcopy fix and x- removal

2016-03-09 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Hi, This fix should go after Denis's use-after-free fix. The first one is a minor bug fix that Paolo spotted. The second removes the 'x-' experimental prefix now that things seem to be running OK, and Jiri has libvirt working with it. Dave Dr. David Alan Gilbert

[Qemu-devel] [PATCH 2/2] postcopy: Remove the x-

2016-03-09 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Postcopy seems to have survived a cycle with only a few fixes, and Jiri has the current libvirt wired up and working ( https://www.redhat.com/archives/libvir-list/2016-March/msg00080.html ) so lets remove the experimental tag. Signed-off-by: Dr. David Alan Gilbert

[Qemu-devel] [PATCH v2 1/2] postcopy: listen thread is never joined

2016-03-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" We don't join the listen thread, it does its own cleanup. Mark as detached not joinable. Signed-off-by: Dr. David Alan Gilbert Reported-by: Paolo Bonzini Reviewed-by: Eric Blake --- migration/savevm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff

[Qemu-devel] [PATCH v2 2/2] postcopy: Remove the x-

2016-03-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Postcopy seems to have survived a cycle with only a few fixes, and Jiri has the current libvirt wired up and working ( https://www.redhat.com/archives/libvir-list/2016-March/msg00080.html ) so remove the experimental tag. Signed-off-by: Dr. David Alan Gilbert Revi

[Qemu-devel] [PATCH v2 0/2] Postcopy fix and x- removal

2016-03-11 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Hi, This fix should go after Denis's use-after-free fix. The first one is a minor bug fix that Paolo spotted. The second removes the 'x-' experimental prefix now that things seem to be running OK, and Jiri has libvirt working with it. Dave v2 Update the 'since

[Qemu-devel] [PATCH] Migration: Add i82801b11 migration data

2016-04-04 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The i82801b11 bridge didn't have a vmsd and thus didn't send any migration data, including that of its parent PCIBridge object. The symptom being if the guest used any devices behind the bridge the guest crashed (mostly with various interrupt related issues). Note:

[Qemu-devel] [PATCH] Fix pflash migration

2016-04-15 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Pflash migration (e.g. q35 + EFI variable storage) fails with the assert: bdrv_co_do_pwritev: Assertion `!(bs->open_flags & 0x0800)' failed. This avoids the problem by delaying the pflash update until after the device loads complete. Tested by: Migrating Q35/EF

[Qemu-devel] [PATCH 0/4] postcopy (& 1 test) patch for 2.7

2016-04-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Hi, This is a small set of postcopy changes, the largest of which is an x86 test for postcopy. Andrea's libqtest change came about from running my test under very heavy load. The test includes a self contained migration workload that rapidly changes RAM in a pre

[Qemu-devel] [PATCH 1/4] Postcopy: Avoid 0 length discards

2016-04-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The discard code in migration/ram.c would send request for zero length discards in the case where no discards were needed. It doesn't appear to have had any bad effect. Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 4 +++- 1 file changed, 3 insertio

[Qemu-devel] [PATCH 2/4] Postcopy: Add stats on page requests

2016-04-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" On the source, add a count of page requests received from the destination. Signed-off-by: Dr. David Alan Gilbert --- hmp.c | 4 +++ include/migration/migration.h | 2 ++ migration/migration.c | 59 ++--

[Qemu-devel] [PATCH 3/4] test: Postcopy

2016-04-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This is a postcopy test (x86 only) that actually runs the guest and checks the memory contents. The test runs from an x86 boot block with the hex embedded in the test; the source for this is: ... .code16 .org 0x7c00 .file "fill.s" .text

[Qemu-devel] [PATCH 4/4] tests: fix libqtest socket timeouts

2016-04-27 Thread Dr. David Alan Gilbert (git)
From: Andrea Arcangeli I kept getting timeouts and unix socket accept failures under high load, the patch fixes it. Signed-off-by: Andrea Arcangeli --- tests/libqtest.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/libqtest.c b/tests/libqtest.c index b12a9e4..57ce29

[Qemu-devel] [PATCH v2 1/5] Postcopy: Avoid 0 length discards

2016-04-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The discard code in migration/ram.c would send request for zero length discards in the case where no discards were needed. It doesn't appear to have had any bad effect. Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 4 +++- 1 file changed, 3 insertio

[Qemu-devel] [PATCH v2 2/5] Migration: Split out ram part of qmp_query_migrate

2016-04-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The RAM section of qmp_query_migrate is reasonably complex and repeated 3 times. Split it out into a helper. Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 57 --- 1 file changed, 22 insertions(+

[Qemu-devel] [PATCH v2 0/5] postcopy (& 1 test) patch for 2.7

2016-04-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Hi, This is a small set of postcopy changes, the largest of which is an x86 test for postcopy. Andrea's libqtest change came about from running my test under very heavy load. The test includes a self contained migration workload that rapidly changes RAM in a pre

[Qemu-devel] [PATCH v2 5/5] tests: fix libqtest socket timeouts

2016-04-29 Thread Dr. David Alan Gilbert (git)
From: Andrea Arcangeli I kept getting timeouts and unix socket accept failures under high load, the patch fixes it. Signed-off-by: Andrea Arcangeli --- tests/libqtest.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/libqtest.c b/tests/libqtest.c index b12a9e4..57ce29

[Qemu-devel] [PATCH v2 3/5] Postcopy: Add stats on page requests

2016-04-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" On the source, add a count of page requests received from the destination. Signed-off-by: Dr. David Alan Gilbert --- hmp.c | 4 include/migration/migration.h | 2 ++ migration/migration.c | 2 ++ migration/ram.c

[Qemu-devel] [PATCH v2 4/5] test: Postcopy

2016-04-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This is a postcopy test (x86 only) that actually runs the guest and checks the memory contents. The test runs from an x86 boot block with the hex embedded in the test; the source for this is: ... .code16 .org 0x7c00 .file "fill.s" .text

[Qemu-devel] [PATCH v3 0/5] postcopy (& 1 test) patch for 2.7

2016-05-06 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Hi, This is a small set of postcopy changes, the largest of which is an x86 test for postcopy. Andrea's libqtest change came about from running my test under very heavy load. The test includes a self contained migration workload that rapidly changes RAM in a pre

[Qemu-devel] [PATCH v3 2/5] Migration: Split out ram part of qmp_query_migrate

2016-05-06 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The RAM section of qmp_query_migrate is reasonably complex and repeated 3 times. Split it out into a helper. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Eric Blake Reviwed-by: Denis V. Lunev --- migration/migration.c | 57 ---

[Qemu-devel] [PATCH v3 3/5] Postcopy: Add stats on page requests

2016-05-06 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" On the source, add a count of page requests received from the destination. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Eric Blake Reviewed-by: Denis V. Lunev --- hmp.c | 4 include/migration/migration.h | 2 ++ migration/migr

[Qemu-devel] [PATCH v3 4/5] test: Postcopy

2016-05-06 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This is a postcopy test (x86 only) that actually runs the guest and checks the memory contents. The test runs from an x86 boot block with the hex embedded in the test; the source for this is: ... .code16 .org 0x7c00 .file "fill.s" .text

[Qemu-devel] [PATCH v3 5/5] tests: fix libqtest socket timeouts

2016-05-06 Thread Dr. David Alan Gilbert (git)
From: Andrea Arcangeli I kept getting timeouts and unix socket accept failures under high load, the patch fixes it. Signed-off-by: Andrea Arcangeli Reviewed-by: Marcel Apfelbaum --- tests/libqtest.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/libqtest.c b/tests/l

[Qemu-devel] [PATCH v3 1/5] Postcopy: Avoid 0 length discards

2016-05-06 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The discard code in migration/ram.c would send request for zero length discards in the case where no discards were needed. It doesn't appear to have had any bad effect. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Denis V. Lunev --- migration/ram.c | 4 +++

[Qemu-devel] [PATCH] Improve avx2 configure test

2016-06-01 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" When configured with --extra-cflags=-O2 gcc optimised out the test and the readelf failed the check leaving avx2 disabled. The fix for that is to remove 'static' on 'foo'. At the same time, I thought it best to be more paranoid on checking whether both the compiler

<    2   3   4   5   6   7   8   9   10   11   >