On Tue, Apr 28, 2026 at 10:19:30AM -0700, Jorge Moreira wrote: > On Tue, Apr 28, 2026 at 7:33 AM Stefano Garzarella <[email protected]> > wrote: > > > > On Mon, Apr 27, 2026 at 03:48:44PM -0700, Jorge Moreira wrote: > > >On Mon, Apr 27, 2026 at 3:45 PM Stefan Hajnoczi <[email protected]> > > >wrote: > > >> > > >> On Wed, Apr 22, 2026 at 12:20:52PM -0700, Jorge Moreira wrote: > > >> > On Wed, Apr 22, 2026 at 1:32 AM Stefano Garzarella > > >> > <[email protected]> wrote: > > >> > > > > >> > > On Wed, 22 Apr 2026 at 03:16, Jorge Moreira <[email protected]> > > >> > > wrote: > > >> > > > > > >> > > > On Tue, Apr 21, 2026 at 2:12 PM Stefan Hajnoczi > > >> > > > <[email protected]> wrote: > > >> > > > > > > >> > > > > On Mon, Apr 20, 2026 at 05:48:13PM -0700, Jorge Moreira wrote: > > >> > > > > > While starting the vrings on SET_VRING_KICK could solve the > > >> > > > > > state > > >> > > > > > machine issue, it still won't notify the back-end that buffers > > >> > > > > > are > > >> > > > > > ready (the driver won't do this). Non-polling back-ends depend > > >> > > > > > on this > > >> > > > > > kick, especially for queues where data flows only from the > > >> > > > > > driver to > > >> > > > > > the back-end. Most implementations likely attempt to read from > > >> > > > > > the > > >> > > > > > queue only after receiving the kick. > > >> > > > > > > >> > > > > This is an interesting question to clarify in the spec. > > >> > > > > >> > > Yep, which is in part related to what I wrote in the other reply: > > >> > > "I think the main issue to clarify is what the device should do > > >> > > when the vrings are configured, but the driver has already been > > >> > > initialized (which is usually the case after migration)." > > >> > > > > >> > > > > > > >> > > > > Stefan > > >> > > > > > >> > > > This is the question that interests me most, to be honest. I'd > > >> > > > rather > > >> > > > have the discussion about when to activate the vrings in a > > >> > > > different > > >> > > > thread and keep this one focused on whether the front-end should > > >> > > > send > > >> > > > the kick or if the back-end is expected to check if there are "new" > > >> > > > buffers in the vring after restore. > > >> > > > > > >> > > > > >> > > IMO we don't need anything from the VMM. When the device receives > > >> > > SET_VRING_KICK, it can check if the vring already contains buffers > > >> > > (and this is the part we might need to clarify) and wake-up the other > > >> > > threads (or always wake-ups them, as crosvm does IIUC, and let them > > >> > > perform this check). > > >> > > After sending the SET_VRING_KICK message to the device, the VMM has > > >> > > the exact same knowledge of the vring state as the device, therefore, > > >> > > it's still unclear to me why we need to inject that kick. > > >> > > > > >> > > Stefano > > >> > > > > >> > > > >> > Is it possible to activate a vring after it has been deactivated with > > >> > VHOST_USER_GET_VRING_BASE? If yes, does the front-end need to send the > > >> > kick file descriptor again with VHOST_USER_SET_VRING_KICK to > > >> > reactivate it? > > >> > > >> Hi Jorge and Stefano, > > >> Yes, VHOST_USER_GET_VRING_BASE -> VHOST_USER_SET_VRING_KICK occurs when > > >> a VM is paused and then resumed. > > >> > > >> You can stress test this by driving I/O using iperf (virtio-net) or fio > > >> (virtio-blk) inside the guest and sending 'stop'/'cont' commands to > > >> QEMU's monitor. > > >> > > >> Here is QEMU's code for starting (including re-starting) rings: > > >> https://gitlab.com/qemu-project/qemu/-/blob/master/hw/virtio/vhost.c?ref_type=heads#L1341 > > >> > > >> QEMU does not inject a kick. The back-end must check the rings itself. > > >> > > >> I'm not sure that all vhost-user back-ends actually check the rings. I > > >> think back-ends should do it, but we should also update the spec with an > > >> front-end implementation note recommending injecting a kick after > > >> VHOST_USER_SET_VRING_KICK completes in order to maximize compatibility > > > > Okay, but since, as we've seen, no frontend currently implements this, > > we need to make it clear that a backend shouldn't necessarily expect the > > kick injected from every frontend, but should support it in some way > > since some of them can inject it. > > > > IMHO especially new backend implementations shouldn't rely on the kick > > injection. > > > > So, to summarize: > > - the frontend should also send a kick to restart the queues > > - the backend should restart the queues after VHOST_USER_SET_VRING_KICK, > > but it might also receive a kick > > > > >> with implementations that follow the current spec wording. And at the > > >> same time I think the spec should also be changed to say that > > >> VHOST_USER_SET_VRING_KICK starts the ring and back-ends SHOULD check the > > >> vring upon processing the message. > > > > Yep, I think we are aligned. > > > > >> > > >> That seems like it would clean up the issues without introducing > > >> compatibility issues or making existing implementations non-compliant > > >> with the updated spec. > > >> > > >> What do you think? > > > > LGTM! > > > > > > > >Sounds good to me > > > > > > > @Jorge do you want to propose this change? > > You have a much better idea of where and how this should be written, > it will save us a few rounds of review if one of you makes that > change.
I can give it a shot and will CC you. Also, I wanted to apologize for not being very forthcoming in this discussion. I had an intuition about this issue but lacked the time to research and double-check the details. As a result, I didn't respond to all your points in detail. Sorry if it was frustrating. Stefan
signature.asc
Description: PGP signature
