On 15.07.26 13:25, Andrey Drobyshev wrote:
On 7/14/26 6:03 PM, Vladimir Sementsov-Ogievskiy wrote:
On 26.06.26 19:46, Andrey Drobyshev wrote:
v2 -> v3:
Re-design device ownership hand-off (suggested by Dongli). Do not rely
on SETUP migration notifiers, as this approach forces us to reorder
generic migration code (see v2 discussion). Instead:
* .pre_save() releases ownership on the source (RESET_OWNER) once VM
is stopped;
* .realize() is adjusted to defer device ownership acquisition for an
incoming CPR;
* .post_load() actually claims the device (VHOST_SET_OWNER);
* FAILED migration event callback re-aquires the ownership on the
source after re_save released it.
v2:https://lore.kernel.org/qemu-devel/[email protected]
Hi!
Hello Vladimir!
Two notes:
1. Did you consider migrating needed FDs through main migration channel, without
use of CPR, like I do in (not yet landed) "[PATCH v19 00/15] virtio-net: live-TAP
local migration" [1]
AFAIU you're doing local same-host migration as a way to update QEMU
binary. With this approach transferring FDs through the main channel is
indeed cleaner. However in our own downstream we use enhanced in-place
memory preservation which is based entirely on cpr-exec migration mode.
So, to answer your question: yes, we considered it, and it might work
for cpr-transfer where we're also doing same-host migration. But in
cpr-exec case there's only one QEMU process involved => we can't use
UNIX socket for migration transport => no SCM_RIGHTS FDs passing => FDs
should be passed via additional transport, no way around it.
Hmm. Looking at code for fd-passing through migration channel:
static bool load_fd(QEMUFile *f, void *pv, size_t size,
const VMStateField *field, Error **errp)
{
int32_t *v = pv;
if (migrate_mode() == MIG_MODE_CPR_EXEC) {
qemu_get_sbe32s(f, v);
return true;
}
...
Seems like passing FDs through main channel (like in my TAP-series) would
work both for local migration (with two processes and UNIX socket) and for
CPR_EXEC mode. So the FD is passed as simple number in case of cpr-exec.
So, if simply pass FDs through migration state, it should work both for
CPR_EXEC and local migration.
(hmm, still, keeping in mind that TAP-series API still not negotiated,
it's simpler to start from pure CPR approach, and add local-migration
support in future)
2. I don't know how much vhost-vsock differs from vhost-net, but for vhost-net
I remember
that RESET_OWNER + SET_OWNER is effectively equal to simply recreating vhost
device on target,
because in kernel vhost device is hardly bound to the process itself, and can't
be passed to
another process. So, we can pass only "empty" FD, not the initialized vhost
device.
That's why in [1] I only pass tap-fds (and some additional state), but vhost
fds are simply
reopened on target (or passed by mgmt app).
Does vhost-vsock work in a different way? Is there real sense in passing vhost
fds here?
[1]
https://lore.kernel.org/qemu-devel/[email protected]/
I'm also not extremely keen on vhost-net internals, but as I understand
with vhost-net we have 2 FDs: TAP FD for the network endpoint and vhost
FD for the memory tables, vring addresses etc. The TAP FD is
process-independent state - and we pass it via migration channel. The
vhost FD is entirely per owning process - and we're able to close it on
source, reopen on target and bind to the passed TAP FD.
Yes, my understanding matches.
However with vhost-vsock there's only one FD. It's used both for
plumbing and the endpoint. At the very least, vhost device state holds
guest CID registration in the global vhost_vsock_hash. If we go the
"recreate/reopen the device" path - CID is removed from the hash in
vhost_vsock_dev_release(), and then we're gonna get ECONNRESET for any
incoming packets => connections are dead. We can't afford that.
OK, thanks for explanation!
--
Best regards,
Vladimir
