From: Andrey Drobyshev <[email protected]> A CPR target that has taken the device ownership in post_load() doesn't always reach vhost_dev_cleanup(): a target which has loaded the state but hasn't been started yet, e.g. one started with -S, may be quit normally, with the source being resumed instead. Devices aren't unrealized on quit, so the ownership stays with the dead process. The source still holds the FD, so the kernel never releases the device, and the source can't take it back after being resumed -> SET_OWNER fails with EBUSY.
Add an exit notifier to each vhost_dev, registered for its lifetime, that releases the ownership if still held. This covers every orderly exit. A target killed with SIGKILL or crashing still leaves the device unrecoverable. Reported-by: Akihiko Odaki <[email protected]> Signed-off-by: Andrey Drobyshev <[email protected]> Acked-by: Stefano Garzarella <[email protected]> Reviewed-by: Michael S. Tsirkin <[email protected]> Reviewed-by: Akihiko Odaki <[email protected]> Signed-off-by: Michael S. Tsirkin <[email protected]> Message-ID: <[email protected]> --- include/hw/virtio/vhost.h | 2 ++ hw/virtio/vhost.c | 20 ++++++++++++++++++++ 2 files changed, 22 insertions(+) diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h index 3b9ef86bac..eb693e3289 100644 --- a/include/hw/virtio/vhost.h +++ b/include/hw/virtio/vhost.h @@ -123,6 +123,8 @@ struct vhost_dev { * when this vhost_dev goes away. Set by the device. */ bool owner_handed_over; + /* @exit_notifier: releases the ownership at exit, vhost_exit_notify() */ + Notifier exit_notifier; /* * @migration_notifier: releases the ownership when an incoming * migration fails, vhost_migration_notify() diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 7fd2883504..cbf6e82afb 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -31,6 +31,7 @@ #include "system/dma.h" #include "system/memory.h" #include "system/ramblock.h" +#include "system/system.h" #include "trace.h" /* enabled until disconnected backend stabilizes */ @@ -90,6 +91,19 @@ static int vhost_migration_notify(NotifierWithReturn *notifier, return 0; } +/* + * Release the ownership of the vhost devices still held at exit(), so that + * another process sharing the backend FDs (e.g. the CPR source) can take + * them over. + */ +static void vhost_exit_notify(Notifier *notifier, void *data) +{ + struct vhost_dev *hdev = container_of(notifier, struct vhost_dev, + exit_notifier); + + vhost_dev_try_release_owner(hdev, "exit"); +} + unsigned int vhost_get_max_memslots(void) { unsigned int max = UINT_MAX; @@ -1860,6 +1874,8 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque, hdev->log_enabled = false; hdev->started = false; memory_listener_register(&hdev->memory_listener, &address_space_memory); + hdev->exit_notifier.notify = vhost_exit_notify; + qemu_add_exit_notifier(&hdev->exit_notifier); migration_add_notifier_modes(&hdev->migration_notifier, vhost_migration_notify, BIT(MIG_MODE_CPR_TRANSFER) | @@ -1913,6 +1929,10 @@ void vhost_dev_cleanup(struct vhost_dev *hdev) memory_listener_unregister(&hdev->memory_listener); QLIST_REMOVE(hdev, entry); } + if (hdev->exit_notifier.notify) { + qemu_remove_exit_notifier(&hdev->exit_notifier); + hdev->exit_notifier.notify = NULL; + } migration_remove_notifier(&hdev->migration_notifier); migrate_del_blocker(&hdev->migration_blocker); g_free(hdev->mem); -- MST
