On Mon, Apr 27, 2026 at 03:48:44PM -0700, Jorge Moreira wrote:
>On Mon, Apr 27, 2026 at 3:45 PM Stefan Hajnoczi <[email protected]> wrote:
>>
>> On Wed, Apr 22, 2026 at 12:20:52PM -0700, Jorge Moreira wrote:
>> > On Wed, Apr 22, 2026 at 1:32 AM Stefano Garzarella <[email protected]>
wrote:
>> > >
>> > > On Wed, 22 Apr 2026 at 03:16, Jorge Moreira <[email protected]> wrote:
>> > > >
>> > > > On Tue, Apr 21, 2026 at 2:12 PM Stefan Hajnoczi <[email protected]>
wrote:
>> > > > >
>> > > > > On Mon, Apr 20, 2026 at 05:48:13PM -0700, Jorge Moreira wrote:
>> > > > > > While starting the vrings on SET_VRING_KICK could solve the state
>> > > > > > machine issue, it still won't notify the back-end that buffers are
>> > > > > > ready (the driver won't do this). Non-polling back-ends depend on
this
>> > > > > > kick, especially for queues where data flows only from the driver
to
>> > > > > > the back-end. Most implementations likely attempt to read from the
>> > > > > > queue only after receiving the kick.
>> > > > >
>> > > > > This is an interesting question to clarify in the spec.
>> > >
>> > > Yep, which is in part related to what I wrote in the other reply:
>> > > "I think the main issue to clarify is what the device should do
>> > > when the vrings are configured, but the driver has already been
>> > > initialized (which is usually the case after migration)."
>> > >
>> > > > >
>> > > > > Stefan
>> > > >
>> > > > This is the question that interests me most, to be honest. I'd rather
>> > > > have the discussion about when to activate the vrings in a different
>> > > > thread and keep this one focused on whether the front-end should send
>> > > > the kick or if the back-end is expected to check if there are "new"
>> > > > buffers in the vring after restore.
>> > > >
>> > >
>> > > IMO we don't need anything from the VMM. When the device receives
>> > > SET_VRING_KICK, it can check if the vring already contains buffers
>> > > (and this is the part we might need to clarify) and wake-up the other
>> > > threads (or always wake-ups them, as crosvm does IIUC, and let them
>> > > perform this check).
>> > > After sending the SET_VRING_KICK message to the device, the VMM has
>> > > the exact same knowledge of the vring state as the device, therefore,
>> > > it's still unclear to me why we need to inject that kick.
>> > >
>> > > Stefano
>> > >
>> >
>> > Is it possible to activate a vring after it has been deactivated with
>> > VHOST_USER_GET_VRING_BASE? If yes, does the front-end need to send the
>> > kick file descriptor again with VHOST_USER_SET_VRING_KICK to
>> > reactivate it?
>>
>> Hi Jorge and Stefano,
>> Yes, VHOST_USER_GET_VRING_BASE -> VHOST_USER_SET_VRING_KICK occurs when
>> a VM is paused and then resumed.
>>
>> You can stress test this by driving I/O using iperf (virtio-net) or fio
>> (virtio-blk) inside the guest and sending 'stop'/'cont' commands to
>> QEMU's monitor.
>>
>> Here is QEMU's code for starting (including re-starting) rings:
>>
https://gitlab.com/qemu-project/qemu/-/blob/master/hw/virtio/vhost.c?ref_type=heads#L1341
>>
>> QEMU does not inject a kick. The back-end must check the rings itself.
>>
>> I'm not sure that all vhost-user back-ends actually check the rings. I
>> think back-ends should do it, but we should also update the spec with an
>> front-end implementation note recommending injecting a kick after
>> VHOST_USER_SET_VRING_KICK completes in order to maximize compatibility
Okay, but since, as we've seen, no frontend currently implements this,
we need to make it clear that a backend shouldn't necessarily expect the
kick injected from every frontend, but should support it in some way
since some of them can inject it.
IMHO especially new backend implementations shouldn't rely on the kick
injection.
So, to summarize:
- the frontend should also send a kick to restart the queues
- the backend should restart the queues after VHOST_USER_SET_VRING_KICK,
but it might also receive a kick
>> with implementations that follow the current spec wording. And at the
>> same time I think the spec should also be changed to say that
>> VHOST_USER_SET_VRING_KICK starts the ring and back-ends SHOULD check the
>> vring upon processing the message.
Yep, I think we are aligned.
>>
>> That seems like it would clean up the issues without introducing
>> compatibility issues or making existing implementations non-compliant
>> with the updated spec.
>>
>> What do you think?
LGTM!
>
>Sounds good to me
>
@Jorge do you want to propose this change?
