v6:
- fix documentation about new protocol feature
- add check to ensure that inflight buffer subsection has been successfully
loaded
- disable support for the new feature if in-flight or inflight migration is not
supported.
v5:
Make protocol feature flag instead of GET_VRING_BASE msg parameter,
so all changes in other devices is no longer needed.
Now back-end may set this feature for QEMU. This feature must be set
with in-flight migration parameter in vhost-user-blk.
v4:
While testing inflight migration, I notices a problem with the fact that
GET_VRING_BASE is needed during migration, so the back-end stops
dirtying pages and synchronizes `last_avail` counter with QEMU. So after
migration in-flight I/O requests will be looks like resubmited on destination
vm.
However, in new logic, we no longer need to wait for in-flight requests
to be complete at GET_VRING_BASE message. So support new parameter
`should_drain` in the GET_VRING_BASE to allow back-end stop vrings
immediately without waiting for in-flight I/O requests to complete.
Also:
- modify vhost-user rst
- refactor on vhost-user-blk.c, now `should_drain` is based on
device parameter `inflight-migration`
v3:
- use pre_load_errp instead of pre_load in vhost.c
- change vhost-user-blk property to
"skip-get-vring-base-inflight-migration"
- refactor vhost-user-blk.c, by moving vhost_user_blk_inflight_needed() higher
v2:
- rewrite migration using VMSD instead of qemufile API
- add vhost-user-blk parameter instead of migration capability
I don't know if VMSD was used cleanly in migration implementation, so
feel free for comments.
Based on Vladimir's work:
[PATCH v2 00/25] vhost-user-blk: live-backend local migration
which was based on:
- [PATCH v4 0/7] chardev: postpone connect
(which in turn is based on [PATCH 0/2] remove deprecated 'reconnect'
options)
- [PATCH v3 00/23] vhost refactoring and fixes
- [PATCH v8 14/19] migration: introduce .pre_incoming() vmsd handler
Based-on: <[email protected]>
Based-on: <[email protected]>
Based-on: <[email protected]>
Based-on: <[email protected]>
Based-on: <[email protected]>
---
Hi!
During inter-host migration, waiting for disk requests to be drained
in the vhost-user backend can incur significant downtime.
This can be avoided if QEMU migrates the inflight region in
vhost-user-blk.
Thus, during the qemu migration, with protocol feature flag the vhost-user
back-end can immediately stop vrings, so all in-flight requests will be
migrated to another host.
At first, I tried to implement migration for all vhost-user devices that
support inflight at once,
but this would require a lot of changes both in vhost-user-blk (to transfer it
to the base class) and
in the vhost-user-base base class (inflight implementation and remodeling + a
large refactor).
Therefore, for now I decided to leave this idea for later and
implement the migration of the inflight region first for vhost-user-blk.
Alexandr Moshkov (5):
vhost-user.rst: specify vhost-user back-end action on GET_VRING_BASE
vhost-user: introduce protocol feature for skip drain on
GET_VRING_BASE
vmstate: introduce VMSTATE_VBUFFER_UINT64
vhost: add vmstate for inflight region with inner buffer
vhost-user-blk: support inter-host inflight migration
docs/interop/vhost-user.rst | 56 ++++++++++++++++++-----------
hw/block/vhost-user-blk.c | 28 +++++++++++++++
hw/virtio/vhost-user.c | 5 +++
hw/virtio/vhost.c | 57 ++++++++++++++++++++++++++++++
include/hw/virtio/vhost-user-blk.h | 1 +
include/hw/virtio/vhost-user.h | 2 ++
include/hw/virtio/vhost.h | 6 ++++
include/migration/vmstate.h | 10 ++++++
8 files changed, 145 insertions(+), 20 deletions(-)
--
2.34.1
