Re: [PATCH v3 3/3] vhost: Improve vhost_get_avail_idx() with smp_rmb()
On 4/23/24 06:46, Michael S. Tsirkin wrote: On Mon, Apr 08, 2024 at 02:15:24PM +1000, Gavin Shan wrote: On 3/30/24 19:02, Gavin Shan wrote: On 3/28/24 19:31, Michael S. Tsirkin wrote: On Thu, Mar 28, 2024 at 10:21:49AM +1000, Gavin Shan wrote: All the callers of vhost_get_avail_idx() are concerned to the memory barrier, imposed by smp_rmb() to ensure the order of the available ring entry read and avail_idx read. Improve vhost_get_avail_idx() so that smp_rmb() is executed when the avail_idx is advanced. With it, the callers needn't to worry about the memory barrier. Suggested-by: Michael S. Tsirkin Signed-off-by: Gavin Shan Previous patches are ok. This one I feel needs more work - first more code such as sanity checking should go into this function, second there's actually a difference between comparing to last_avail_idx and just comparing to the previous value of avail_idx. I will pick patches 1-2 and post a cleanup on top so you can take a look, ok? Thanks, Michael. It's fine to me. A kindly ping. If it's ok to you, could you please merge PATCH[1-2]? Our downstream 9.4 need the fixes, especially for NVidia's grace-hopper and grace-grace platforms. For PATCH[3], I also can help with the improvement if you don't have time for it. Please let me know. 1-2 are upstream go ahead and post the cleanup. Michael, a cleanup series has been sent for review. https://lore.kernel.org/virtualization/20240423032407.262329-1-gs...@redhat.com/T/#t Thanks, Gavin
Re: [PATCH v3 3/3] vhost: Improve vhost_get_avail_idx() with smp_rmb()
On Mon, Apr 08, 2024 at 02:15:24PM +1000, Gavin Shan wrote: > Hi Michael, > > On 3/30/24 19:02, Gavin Shan wrote: > > On 3/28/24 19:31, Michael S. Tsirkin wrote: > > > On Thu, Mar 28, 2024 at 10:21:49AM +1000, Gavin Shan wrote: > > > > All the callers of vhost_get_avail_idx() are concerned to the memory > > > > barrier, imposed by smp_rmb() to ensure the order of the available > > > > ring entry read and avail_idx read. > > > > > > > > Improve vhost_get_avail_idx() so that smp_rmb() is executed when > > > > the avail_idx is advanced. With it, the callers needn't to worry > > > > about the memory barrier. > > > > > > > > Suggested-by: Michael S. Tsirkin > > > > Signed-off-by: Gavin Shan > > > > > > Previous patches are ok. This one I feel needs more work - > > > first more code such as sanity checking should go into > > > this function, second there's actually a difference > > > between comparing to last_avail_idx and just comparing > > > to the previous value of avail_idx. > > > I will pick patches 1-2 and post a cleanup on top so you can > > > take a look, ok? > > > > > > > Thanks, Michael. It's fine to me. > > > > A kindly ping. > > If it's ok to you, could you please merge PATCH[1-2]? Our downstream > 9.4 need the fixes, especially for NVidia's grace-hopper and grace-grace > platforms. > > For PATCH[3], I also can help with the improvement if you don't have time > for it. Please let me know. > > Thanks, > Gavin 1-2 are upstream go ahead and post the cleanup. -- MST
Re: [PATCH v3 3/3] vhost: Improve vhost_get_avail_idx() with smp_rmb()
On Mon, Apr 08, 2024 at 02:15:24PM +1000, Gavin Shan wrote: > Hi Michael, > > On 3/30/24 19:02, Gavin Shan wrote: > > On 3/28/24 19:31, Michael S. Tsirkin wrote: > > > On Thu, Mar 28, 2024 at 10:21:49AM +1000, Gavin Shan wrote: > > > > All the callers of vhost_get_avail_idx() are concerned to the memory > > > > barrier, imposed by smp_rmb() to ensure the order of the available > > > > ring entry read and avail_idx read. > > > > > > > > Improve vhost_get_avail_idx() so that smp_rmb() is executed when > > > > the avail_idx is advanced. With it, the callers needn't to worry > > > > about the memory barrier. > > > > > > > > Suggested-by: Michael S. Tsirkin > > > > Signed-off-by: Gavin Shan > > > > > > Previous patches are ok. This one I feel needs more work - > > > first more code such as sanity checking should go into > > > this function, second there's actually a difference > > > between comparing to last_avail_idx and just comparing > > > to the previous value of avail_idx. > > > I will pick patches 1-2 and post a cleanup on top so you can > > > take a look, ok? > > > > > > > Thanks, Michael. It's fine to me. > > > > A kindly ping. > > If it's ok to you, could you please merge PATCH[1-2]? Our downstream > 9.4 need the fixes, especially for NVidia's grace-hopper and grace-grace > platforms. > > For PATCH[3], I also can help with the improvement if you don't have time > for it. Please let me know. > > Thanks, > Gavin The thing to do is basically diff with the patch I wrote :) We can also do a bit more cleanups on top of *that*, like unifying error handling. -- MST
Re: [PATCH v3 3/3] vhost: Improve vhost_get_avail_idx() with smp_rmb()
On Mon, Apr 08, 2024 at 02:15:24PM +1000, Gavin Shan wrote: > Hi Michael, > > On 3/30/24 19:02, Gavin Shan wrote: > > On 3/28/24 19:31, Michael S. Tsirkin wrote: > > > On Thu, Mar 28, 2024 at 10:21:49AM +1000, Gavin Shan wrote: > > > > All the callers of vhost_get_avail_idx() are concerned to the memory > > > > barrier, imposed by smp_rmb() to ensure the order of the available > > > > ring entry read and avail_idx read. > > > > > > > > Improve vhost_get_avail_idx() so that smp_rmb() is executed when > > > > the avail_idx is advanced. With it, the callers needn't to worry > > > > about the memory barrier. > > > > > > > > Suggested-by: Michael S. Tsirkin > > > > Signed-off-by: Gavin Shan > > > > > > Previous patches are ok. This one I feel needs more work - > > > first more code such as sanity checking should go into > > > this function, second there's actually a difference > > > between comparing to last_avail_idx and just comparing > > > to the previous value of avail_idx. > > > I will pick patches 1-2 and post a cleanup on top so you can > > > take a look, ok? > > > > > > > Thanks, Michael. It's fine to me. > > > > A kindly ping. > > If it's ok to you, could you please merge PATCH[1-2]? Our downstream > 9.4 need the fixes, especially for NVidia's grace-hopper and grace-grace > platforms. Yes - in the next rc hopefully. > For PATCH[3], I also can help with the improvement if you don't have time > for it. Please let me know. > > Thanks, > Gavin That would be great. -- MST
Re: [PATCH v3 3/3] vhost: Improve vhost_get_avail_idx() with smp_rmb()
Hi Michael, On 3/30/24 19:02, Gavin Shan wrote: On 3/28/24 19:31, Michael S. Tsirkin wrote: On Thu, Mar 28, 2024 at 10:21:49AM +1000, Gavin Shan wrote: All the callers of vhost_get_avail_idx() are concerned to the memory barrier, imposed by smp_rmb() to ensure the order of the available ring entry read and avail_idx read. Improve vhost_get_avail_idx() so that smp_rmb() is executed when the avail_idx is advanced. With it, the callers needn't to worry about the memory barrier. Suggested-by: Michael S. Tsirkin Signed-off-by: Gavin Shan Previous patches are ok. This one I feel needs more work - first more code such as sanity checking should go into this function, second there's actually a difference between comparing to last_avail_idx and just comparing to the previous value of avail_idx. I will pick patches 1-2 and post a cleanup on top so you can take a look, ok? Thanks, Michael. It's fine to me. A kindly ping. If it's ok to you, could you please merge PATCH[1-2]? Our downstream 9.4 need the fixes, especially for NVidia's grace-hopper and grace-grace platforms. For PATCH[3], I also can help with the improvement if you don't have time for it. Please let me know. Thanks, Gavin
Re: [PATCH v3 3/3] vhost: Improve vhost_get_avail_idx() with smp_rmb()
On 3/28/24 19:31, Michael S. Tsirkin wrote: On Thu, Mar 28, 2024 at 10:21:49AM +1000, Gavin Shan wrote: All the callers of vhost_get_avail_idx() are concerned to the memory barrier, imposed by smp_rmb() to ensure the order of the available ring entry read and avail_idx read. Improve vhost_get_avail_idx() so that smp_rmb() is executed when the avail_idx is advanced. With it, the callers needn't to worry about the memory barrier. Suggested-by: Michael S. Tsirkin Signed-off-by: Gavin Shan Previous patches are ok. This one I feel needs more work - first more code such as sanity checking should go into this function, second there's actually a difference between comparing to last_avail_idx and just comparing to the previous value of avail_idx. I will pick patches 1-2 and post a cleanup on top so you can take a look, ok? Thanks, Michael. It's fine to me. Thanks, Gavin
Re: [PATCH v3 3/3] vhost: Improve vhost_get_avail_idx() with smp_rmb()
On Thu, Mar 28, 2024 at 10:21:49AM +1000, Gavin Shan wrote: > All the callers of vhost_get_avail_idx() are concerned to the memory > barrier, imposed by smp_rmb() to ensure the order of the available > ring entry read and avail_idx read. > > Improve vhost_get_avail_idx() so that smp_rmb() is executed when > the avail_idx is advanced. With it, the callers needn't to worry > about the memory barrier. > > Suggested-by: Michael S. Tsirkin > Signed-off-by: Gavin Shan Previous patches are ok. This one I feel needs more work - first more code such as sanity checking should go into this function, second there's actually a difference between comparing to last_avail_idx and just comparing to the previous value of avail_idx. I will pick patches 1-2 and post a cleanup on top so you can take a look, ok? > --- > drivers/vhost/vhost.c | 75 +++ > 1 file changed, 26 insertions(+), 49 deletions(-) > > diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c > index 32686c79c41d..e6882f4f6ce2 100644 > --- a/drivers/vhost/vhost.c > +++ b/drivers/vhost/vhost.c > @@ -1290,10 +1290,28 @@ static void vhost_dev_unlock_vqs(struct vhost_dev *d) > mutex_unlock(>vqs[i]->mutex); > } > > -static inline int vhost_get_avail_idx(struct vhost_virtqueue *vq, > - __virtio16 *idx) > +static inline int vhost_get_avail_idx(struct vhost_virtqueue *vq) > { > - return vhost_get_avail(vq, *idx, >avail->idx); > + __virtio16 avail_idx; > + int r; > + > + r = vhost_get_avail(vq, avail_idx, >avail->idx); > + if (unlikely(r)) { > + vq_err(vq, "Failed to access avail idx at %p\n", > +>avail->idx); > + return r; > + } > + > + vq->avail_idx = vhost16_to_cpu(vq, avail_idx); > + if (vq->avail_idx != vq->last_avail_idx) { > + /* Ensure the available ring entry read happens > + * before the avail_idx read when the avail_idx > + * is advanced. > + */ > + smp_rmb(); > + } > + > + return 0; > } > > static inline int vhost_get_avail_head(struct vhost_virtqueue *vq, > @@ -2499,7 +2517,6 @@ int vhost_get_vq_desc(struct vhost_virtqueue *vq, > struct vring_desc desc; > unsigned int i, head, found = 0; > u16 last_avail_idx; > - __virtio16 avail_idx; > __virtio16 ring_head; > int ret, access; > > @@ -2507,12 +2524,8 @@ int vhost_get_vq_desc(struct vhost_virtqueue *vq, > last_avail_idx = vq->last_avail_idx; > > if (vq->avail_idx == vq->last_avail_idx) { > - if (unlikely(vhost_get_avail_idx(vq, _idx))) { > - vq_err(vq, "Failed to access avail idx at %p\n", > - >avail->idx); > + if (unlikely(vhost_get_avail_idx(vq))) > return -EFAULT; > - } > - vq->avail_idx = vhost16_to_cpu(vq, avail_idx); > > if (unlikely((u16)(vq->avail_idx - last_avail_idx) > vq->num)) { > vq_err(vq, "Guest moved used index from %u to %u", > @@ -2525,11 +2538,6 @@ int vhost_get_vq_desc(struct vhost_virtqueue *vq, >*/ > if (vq->avail_idx == last_avail_idx) > return vq->num; > - > - /* Only get avail ring entries after they have been > - * exposed by guest. > - */ > - smp_rmb(); > } > > /* Grab the next descriptor number they're advertising, and increment > @@ -2790,35 +2798,19 @@ EXPORT_SYMBOL_GPL(vhost_add_used_and_signal_n); > /* return true if we're sure that avaiable ring is empty */ > bool vhost_vq_avail_empty(struct vhost_dev *dev, struct vhost_virtqueue *vq) > { > - __virtio16 avail_idx; > - int r; > - > if (vq->avail_idx != vq->last_avail_idx) > return false; > > - r = vhost_get_avail_idx(vq, _idx); > - if (unlikely(r)) > - return false; > - > - vq->avail_idx = vhost16_to_cpu(vq, avail_idx); > - if (vq->avail_idx != vq->last_avail_idx) { > - /* Since we have updated avail_idx, the following > - * call to vhost_get_vq_desc() will read available > - * ring entries. Make sure that read happens after > - * the avail_idx read. > - */ > - smp_rmb(); > + if (unlikely(vhost_get_avail_idx(vq))) > return false; > - } > > - return true; > + return vq->avail_idx == vq->last_avail_idx; > } > EXPORT_SYMBOL_GPL(vhost_vq_avail_empty); > > /* OK, now we need to know about added descriptors. */ > bool vhost_enable_notify(struct vhost_dev *dev, struct vhost_virtqueue *vq) > { > - __virtio16 avail_idx; > int r; > > if (!(vq->used_flags & VRING_USED_F_NO_NOTIFY)) > @@ -2842,25 +2834,10 @@ bool vhost_enable_notify(struct vhost_dev *dev, > struct
Re: [PATCH v3 3/3] vhost: Improve vhost_get_avail_idx() with smp_rmb()
On Thu, Mar 28, 2024 at 8:22 AM Gavin Shan wrote: > > All the callers of vhost_get_avail_idx() are concerned to the memory > barrier, imposed by smp_rmb() to ensure the order of the available > ring entry read and avail_idx read. > > Improve vhost_get_avail_idx() so that smp_rmb() is executed when > the avail_idx is advanced. With it, the callers needn't to worry > about the memory barrier. > > Suggested-by: Michael S. Tsirkin > Signed-off-by: Gavin Shan Acked-by: Jason Wang Thanks