Switchover-ack is a mechanism to synchronize between source and
destination QEMU during migration to prevent the source from
switching
over prematurely.
VFIO uses switchover-ack to ensure switchover happens only after
destination side has loaded the precopy initial bytes. This is
important
for VFIO, as otherwise downtime could be impacted and be higher.
In its current state, switchover-ack is a one-time mechanism,
meaning
that switchover is acked only once and past that another ACK
cannot be
requested again. This was sufficient until now, as VFIO precopy
initial
bytes was defined to be monotonically decreasing. Thus, when
precopy
initial bytes reached zero for all VFIO devices, a single ACK
would be
sent and its validity would hold.
However, now the new VFIO_PRECOPY_INFO_REINIT feature allows
precopy
initial bytes to be re-initialized during precopy. Specifically, it
means that initial bytes can grow after reaching zero, which would
invalidate a previously sent switchover ACK.
To solve this, make switchover-ack reusable and allow devices to
request
switchover ACKs when needed via the save_query_pending
SaveVMHandler.
Since now switchover ACK can be requested for a specific device
and in
different times, make switchover ACK per-device (instead of a
single ACK
for all devices) and let source side do the pending ACKs
accounting.
Keep the legacy switchover-ack mechanism for backward
compatibility and
turn it on by a compatibility property for older machines.
Enable the
property until VFIO implements the new switchover-ack.
Acked-by: Markus Armbruster <[email protected]>
Reviewed-by: Peter Xu <[email protected]>
Signed-off-by: Avihai Horon <[email protected]>
---
qapi/migration.json | 14 ++++----
include/migration/client-options.h | 1 +
include/migration/register.h | 2 ++
migration/migration.h | 32 ++++++++++++++++--
migration/savevm.h | 6 ++--
migration/migration.c | 37 ++++++++++++++-------
migration/options.c | 10 ++++++
migration/savevm.c | 53
+++++++++++++++++++++++-------
migration/trace-events | 5 +--
9 files changed, 123 insertions(+), 37 deletions(-)
diff --git a/qapi/migration.json b/qapi/migration.json
index 27a7970556..9b3070e494 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -508,14 +508,12 @@
# (since 7.1)
#
# @switchover-ack: If enabled, migration will not stop the
source VM
-# and complete the migration until an ACK is received from the
-# destination that it's OK to do so. Exactly when this ACK is
-# sent depends on the migrated devices that use this
feature. For
-# example, a device can use it to make sure some of its
data is
-# sent and loaded in the destination before doing switchover.
