This is the v3 patch series of postcopy migration.

The trees is available at
git://github.com/yamahata/qemu.git qemu-postcopy-oct-30-2012
git://github.com/yamahata/linux-umem.git linux-umem-oct-29-2012

Major changes v2 -> v3:
- implemented pre+post optimization
- auto detection of postcopy by incoming side
- using threads on destination instead of fork
- using blocking io instead of select + non-blocking io loop
- less memory overhead
- various improvement and code simplification
- kernel module name change umem -> uvmem to avoid name conflict.

Patches organization:
1-2: trivial fixes
3-5: prepartion for threading. cherry-picked from migration tree
6-18: refactoring existing code and preparation
19-25: implement postcopy live migration itself (essential part)
26-35: optimization/heuristic for postcopy

Usage
=====
You need load uvmem character device on the host before starting migration.
Postcopy can be used for tcg and kvm accelarator. The implementation depend
on only linux uvmem character device. But the driver dependent code is split
into a file.
I tested only host page size == guest page size case, but the implementation
allows host page size != guest page size case.

The following options are added with this patch series.
- incoming part
  use -incoming as usual. Postcopy is automatically detected.
  example:
  qemu -incoming tcp:0:4444 -monitor stdio -machine accel=kvm

- outging part
  options for migrate command
  migrate [-p [-n] [-m]] URI 
          [<precopy count> [<prefault forward> [<prefault backword>]]]

  Newly added options/arguments
  -p: indicate postcopy migration
  -n: disable background transferring pages: This is for benchmark/debugging
  -m: move background transfer of postcopy mode
  <precopy count>: The number of precopy RAM scan before postcopy.
                   default 0 (0 means no precopy)
  <prefault forward>: The number of forward pages which is sent with on-demand
  <prefault backward>: The number of backward pages which is sent with
                       on-demand

  example:
  migrate -p -n tcp:<dest ip address>:4444
  migrate -p -n -m tcp:<dest ip address>:4444 42 42 0


TODO
====
- benchmark/evaluation
- improve/optimization
  At the moment at least what I'm aware of is
  - pre+post case
    On desitnation side reading dirty bitmap would cause long latency.
    create thread for that.
- consider on FUSE/CUSE possibility

basic postcopy work flow
========================
        qemu on the destination
              |
              V
        open(/dev/uvmem)
              |
              V
        UVMEM_INIT
              |
              V
        Here we have two file descriptors to
        umem device and shmem file
              |
              |                                  umem threads
              |                                  on the destination
              |
              V    create pipe to communicate
        crete threads--------------------------------,
              |                                      |
              V                                   mmap(shmem file)
        mmap(uvmem device) for guest RAM          close(shmem file)
              |                                      |
              |                                      |
              V                                      |
        wait for ready from daemon <----pipe-----send ready message
              |                                      |
              |                                 Here the daemon takes over
        send ok------------pipe---------------> the owner of the socket
              |                                 to the source
              V                                      |
        entering post copy stage                     |
        start guest execution                        |
              |                                      |
              V                                      V
        access guest RAM                          read() to get faulted pages
              |                                      |
              V                                      V
        page fault ------------------------------>page offset is returned
        block                                        |
                                                     V
                                                  pull page from the source
                                                  write the page contents
                                                  to the shmem.
                                                     |
                                                     V
        unblock     <-----------------------------write() to tell served pages
        the fault handler returns the page           |
        page fault is resolved                       |
              |                                      V
              |                                   touch guest RAM pages
              |                                      |
              |                                      V
              |                                   release the cached page
              |                                   madvise(MADV_REMOVE)
              |
              |
              |                                   pages can be sent
              |                                   backgroundly
              |                                      |
              |                                      V
              |                                   mark page is cached
              |                                   Thus future page fault is
              |                                   avoided.
              |                                      |
              |                                      V
              |                                   touch guest RAM pages
              |                                      |
              |                                      V
              |                                   release the cached page
              |                                   madvise(MADV_REMOVE)
              |                                      |
              V                                      V

                 all the pages are pulled from the source

              |                                      |
              V                                      V
        migration completes                        exit()


Isaku Yamahata (32):
  migration.c: remove redundant line in migrate_init()
  arch_init: DPRINTF format error and typo
  osdep: add qemu_read_full() to read interrupt-safely
  savevm: export qemu_peek_buffer, qemu_peek_byte, qemu_file_skip,
    qemu_fflush
  savevm/QEMUFile: consolidate QEMUFile functions a bit
  savevm/QEMUFile: introduce qemu_fopen_fd
  savevm/QEMUFile: add read/write QEMUFile on memory buffer
  savevm, buffered_file: introduce method to drain buffer of buffered
    file
  arch_init: export RAM_SAVE_xxx flags for postcopy
  arch_init/ram_save: introduce constant for ram save version = 4
  arch_init: refactor ram_save_block() and export ram_save_block()
  arch_init/ram_save_setup: factor out bitmap alloc/free
  arch_init/ram_load: refactor ram_load
  arch_init: factor out logic to find ram block with id string
  migration: export migrate_fd_completed() and migrate_fd_cleanup()
  uvmem.h: import Linux uvmem.h and teach update-linux-headers.sh
  osdep: add QEMU_MADV_REMOVE and tirivial fix
  postcopy: introduce helper functions for postcopy
  savevm: add new section that is used by postcopy
  postcopy: implement incoming part of postcopy live migration
  postcopy outgoing: add -p option to migrate command
  postcopy: implement outgoing part of postcopy live migration
  postcopy/outgoing: add -n options to disable background transfer
  postcopy/outgoing: implement forward/backword prefault
  arch_init: factor out setting last_block, last_offset
  postcopy/outgoing: add movebg mode(-m) to migration command
  arch_init: factor out ram_load
  arch_init: export ram_save_iterate()
  postcopy: pre+post optimization incoming side
  arch_init: export migration_bitmap_sync and helper method to get
    bitmap
  postcopy/outgoing: introduce precopy_count parameter
  postcopy: pre+post optimization outgoing side

Paolo Bonzini (1):
  split MRU ram list

Umesh Deshpande (2):
  add a version number to ram_list
  protect the ramlist with a separate mutex

 Makefile.target                 |    2 +
 arch_init.c                     |  391 +++++---
 arch_init.h                     |   24 +
 buffered_file.c                 |   59 +-
 buffered_file.h                 |    1 +
 cpu-all.h                       |   16 +-
 exec.c                          |   62 +-
 hmp-commands.hx                 |   21 +-
 hmp.c                           |   12 +-
 linux-headers/linux/uvmem.h     |   41 +
 migration-exec.c                |    8 +-
 migration-fd.c                  |   23 +-
 migration-postcopy.c            | 2019 +++++++++++++++++++++++++++++++++++++++
 migration-tcp.c                 |   16 +-
 migration-unix.c                |   36 +-
 migration.c                     |   65 +-
 migration.h                     |   42 +
 osdep.c                         |   24 +
 osdep.h                         |   13 +-
 qapi-schema.json                |    6 +-
 qemu-common.h                   |    2 +
 qemu-file.h                     |   12 +-
 qmp-commands.hx                 |    4 +-
 savevm.c                        |  223 ++++-
 scripts/update-linux-headers.sh |    2 +-
 sysemu.h                        |    2 +-
 umem.c                          |  291 ++++++
 umem.h                          |   88 ++
 vl.c                            |    5 +-
 29 files changed, 3265 insertions(+), 245 deletions(-)
 create mode 100644 linux-headers/linux/uvmem.h
 create mode 100644 migration-postcopy.c
 create mode 100644 umem.c
 create mode 100644 umem.h

--
1.7.10.4

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Reply via email to