On the postcopy destination, the postcopy-active and completed states
can be reported while the VM is still in RUN_STATE_INMIGRATE. The run
state only changes in loadvm_postcopy_handle_run_bh(), but
loadvm_postcopy_handle_run() switches to postcopy-active before it
schedules the BH, and the listen thread can report completed before the
BH has run. Precopy does not have this problem: the run state changes
before completed is reported.
We hit this in production. A VM with a persistent dirty bitmap migrates
its bitmaps in postcopy, so libvirt runs the Finish phase as soon as the
source enters postcopy-active. Our libvirt reattaches disks on the
destination in Finish, and the blockdev-add calls land while the
destination is still in RUN_STATE_INMIGRATE. The new nodes are created
inactive, the device cannot be attached to them ("Block node is
read-only"), and the guest is lost on both hosts. A bitmap-only
postcopy makes the window wide, because the listen thread has almost
nothing to load.
The series orders both destination states after the run state change
and adds a qtest check that fails on master. The last patch gives the
source's postcopy-active the same meaning. It takes Peter's idea from
the review of Juraj's RFC [1]: record the package-loaded ping and send
the pong later [2]. Here the pong is sent from the run BH, after the run
state change, rather than from loadvm_postcopy_handle_run(). It keeps
the source in postcopy-device longer, so it can be dropped without
affecting the rest.
[1]
https://lore.kernel.org/qemu-devel/[email protected]/
[2] https://lore.kernel.org/qemu-devel/[email protected]/
Many thanks to Juraj for the postcopy-device state in 7b842fe354c. It is
what gives the source a point to wait at, and this series only moves
that point to where the destination has really left
RUN_STATE_INMIGRATE.
Signed-off-by: Denis V. Lunev <[email protected]>
Cc: Peter Xu <[email protected]>
Cc: Fabiano Rosas <[email protected]>
Cc: Juraj Marcin <[email protected]>
Denis V. Lunev (4):
migration/postcopy: report COMPLETED only after the run BH has run
migration/postcopy: set destination postcopy-active in the run BH
tests/qtest/migration: check the destination run state in postcopy
tests
migration/postcopy: answer the package-loaded ping from the run BH
migration/migration.c | 3 +++
migration/migration.h | 4 ++++
migration/postcopy-ram.c | 4 ++++
migration/savevm.c | 18 +++++++++++++-----
tests/qtest/migration/framework.c | 6 ++++--
tests/qtest/migration/migration-qmp.c | 10 ++++++++--
tests/qtest/migration/migration-qmp.h | 3 ++-
tests/qtest/migration/migration-util.c | 8 ++++++++
tests/qtest/migration/migration-util.h | 2 ++
9 files changed, 48 insertions(+), 10 deletions(-)
base-commit: f7ada39edacaa5c26b30e98b94017b0b2ccbcf94
--
2.53.0
