Re: [Qemu-devel] [RFC v2 8/8] virtio: guest driver reload for vhost-net
On Fri, Sep 21, 2018 at 09:30:26AM +0200, Maxime Coquelin wrote: > > > On 09/21/2018 04:33 AM, Jason Wang wrote: > > > > > > On 2018年09月21日 04:39, Maxime Coquelin wrote: > > > Hi Wei, Jason, > > > > > > On 06/19/2018 09:53 AM, Wei Xu wrote: > > > > On Wed, Jun 06, 2018 at 11:48:19AM +0800, Jason Wang wrote: > > > > > > > > > > > > > > > On 2018å¹´06月06æ—¥ 03:08, w...@redhat.com wrote: > > > > > > From: Wei Xu > > > > > > > > > > > > last_avail, avail_wrap_count, used_idx and used_wrap_count are > > > > > > needed to support vhost-net backend, all these are either 16 or > > > > > > bool variables, since state.num is 64bit wide, so here it is > > > > > > possible to put them to the 'num' without introducing a new case > > > > > > while handling ioctl. > > > > > > > > > > > > Unload/Reload test has been done successfully with a > > > > > > patch in vhost kernel. > > > > > > > > > > You need a patch to enable vhost. > > > > > > > > > > And I think you can only do it for vhost-kenrel now since vhost-user > > > > > protocol needs some extension I believe. > > > > > > > > OK. > > > > > > > > > > > > > > > > > > > > > Signed-off-by: Wei Xu > > > > > > --- > > > > > > hw/virtio/virtio.c | 42 ++ > > > > > > 1 file changed, 34 insertions(+), 8 deletions(-) > > > > > > > > > > > > diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c > > > > > > index 4543974..153f6d7 100644 > > > > > > --- a/hw/virtio/virtio.c > > > > > > +++ b/hw/virtio/virtio.c > > > > > > @@ -2862,33 +2862,59 @@ hwaddr > > > > > > virtio_queue_get_used_size(VirtIODevice *vdev, int n) > > > > > > } > > > > > > } > > > > > > -uint16_t virtio_queue_get_last_avail_idx(VirtIODevice *vdev, int n) > > > > > > +uint64_t virtio_queue_get_last_avail_idx(VirtIODevice *vdev, int n) > > > > > > { > > > > > > - return vdev->vq[n].last_avail_idx; > > > > > > + uint64_t num; > > > > > > + > > > > > > + num = vdev->vq[n].last_avail_idx; > > > > > > + if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { > > > > > > + num |= ((uint64_t)vdev->vq[n].avail_wrap_counter) << 16; > > > > > > + num |= ((uint64_t)vdev->vq[n].used_idx) << 32; > > > > > > + num |= ((uint64_t)vdev->vq[n].used_wrap_counter) << 48; > > > > > > > > > > So s.num is 32bit, I don't think this can even work. > > > > > > > > I mistakenly checked out s.num is 64bit, will add a new case in > > > > next version. > > > > > > Wouldn't be enough to just get/set avail_wrap_counter? > > > Something like this so that it fits into 32 bits: > > > if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { > > > num |= ((uint64_t)vdev->vq[n].avail_wrap_counter) << 31; > > > } > > > > > > > > > Regards, > > > Maxime > > > > Yes, actually, that's what I did for vhost kernel > > (https://github.com/jasowang/net/blob/packed-host/drivers/vhost/vhost.c#L1418) > > Oh yes, I forgot on what I based my implementation on. > Wei, Michael, do you agree with this? I will need to merge the Vhost > patch in DPDK in the coming two weeks. > > Regards, > Maxime It looks reasonable to me. > > > > Thanks
Re: [Qemu-devel] [RFC v2 8/8] virtio: guest driver reload for vhost-net
On 09/21/2018 04:33 AM, Jason Wang wrote: On 2018年09月21日 04:39, Maxime Coquelin wrote: Hi Wei, Jason, On 06/19/2018 09:53 AM, Wei Xu wrote: On Wed, Jun 06, 2018 at 11:48:19AM +0800, Jason Wang wrote: On 2018å¹´06月06æ—¥ 03:08, w...@redhat.com wrote: From: Wei Xu last_avail, avail_wrap_count, used_idx and used_wrap_count are needed to support vhost-net backend, all these are either 16 or bool variables, since state.num is 64bit wide, so here it is possible to put them to the 'num' without introducing a new case while handling ioctl. Unload/Reload test has been done successfully with a patch in vhost kernel. You need a patch to enable vhost. And I think you can only do it for vhost-kenrel now since vhost-user protocol needs some extension I believe. OK. Signed-off-by: Wei Xu --- hw/virtio/virtio.c | 42 ++ 1 file changed, 34 insertions(+), 8 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 4543974..153f6d7 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -2862,33 +2862,59 @@ hwaddr virtio_queue_get_used_size(VirtIODevice *vdev, int n) } } -uint16_t virtio_queue_get_last_avail_idx(VirtIODevice *vdev, int n) +uint64_t virtio_queue_get_last_avail_idx(VirtIODevice *vdev, int n) { - return vdev->vq[n].last_avail_idx; + uint64_t num; + + num = vdev->vq[n].last_avail_idx; + if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { + num |= ((uint64_t)vdev->vq[n].avail_wrap_counter) << 16; + num |= ((uint64_t)vdev->vq[n].used_idx) << 32; + num |= ((uint64_t)vdev->vq[n].used_wrap_counter) << 48; So s.num is 32bit, I don't think this can even work. I mistakenly checked out s.num is 64bit, will add a new case in next version. Wouldn't be enough to just get/set avail_wrap_counter? Something like this so that it fits into 32 bits: if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { num |= ((uint64_t)vdev->vq[n].avail_wrap_counter) << 31; } Regards, Maxime Yes, actually, that's what I did for vhost kernel (https://github.com/jasowang/net/blob/packed-host/drivers/vhost/vhost.c#L1418) Oh yes, I forgot on what I based my implementation on. Wei, Michael, do you agree with this? I will need to merge the Vhost patch in DPDK in the coming two weeks. Regards, Maxime Thanks
Re: [Qemu-devel] [RFC v2 8/8] virtio: guest driver reload for vhost-net
On 2018年09月21日 04:39, Maxime Coquelin wrote: Hi Wei, Jason, On 06/19/2018 09:53 AM, Wei Xu wrote: On Wed, Jun 06, 2018 at 11:48:19AM +0800, Jason Wang wrote: On 2018å¹´06月06æ—¥ 03:08, w...@redhat.com wrote: From: Wei Xu last_avail, avail_wrap_count, used_idx and used_wrap_count are needed to support vhost-net backend, all these are either 16 or bool variables, since state.num is 64bit wide, so here it is possible to put them to the 'num' without introducing a new case while handling ioctl. Unload/Reload test has been done successfully with a patch in vhost kernel. You need a patch to enable vhost. And I think you can only do it for vhost-kenrel now since vhost-user protocol needs some extension I believe. OK. Signed-off-by: Wei Xu --- hw/virtio/virtio.c | 42 ++ 1 file changed, 34 insertions(+), 8 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 4543974..153f6d7 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -2862,33 +2862,59 @@ hwaddr virtio_queue_get_used_size(VirtIODevice *vdev, int n) } } -uint16_t virtio_queue_get_last_avail_idx(VirtIODevice *vdev, int n) +uint64_t virtio_queue_get_last_avail_idx(VirtIODevice *vdev, int n) { - return vdev->vq[n].last_avail_idx; + uint64_t num; + + num = vdev->vq[n].last_avail_idx; + if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { + num |= ((uint64_t)vdev->vq[n].avail_wrap_counter) << 16; + num |= ((uint64_t)vdev->vq[n].used_idx) << 32; + num |= ((uint64_t)vdev->vq[n].used_wrap_counter) << 48; So s.num is 32bit, I don't think this can even work. I mistakenly checked out s.num is 64bit, will add a new case in next version. Wouldn't be enough to just get/set avail_wrap_counter? Something like this so that it fits into 32 bits: if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { num |= ((uint64_t)vdev->vq[n].avail_wrap_counter) << 31; } Regards, Maxime Yes, actually, that's what I did for vhost kernel (https://github.com/jasowang/net/blob/packed-host/drivers/vhost/vhost.c#L1418) Thanks
Re: [Qemu-devel] [RFC v2 8/8] virtio: guest driver reload for vhost-net
Hi Wei, Jason, On 06/19/2018 09:53 AM, Wei Xu wrote: On Wed, Jun 06, 2018 at 11:48:19AM +0800, Jason Wang wrote: On 2018年06月06日 03:08, w...@redhat.com wrote: From: Wei Xu last_avail, avail_wrap_count, used_idx and used_wrap_count are needed to support vhost-net backend, all these are either 16 or bool variables, since state.num is 64bit wide, so here it is possible to put them to the 'num' without introducing a new case while handling ioctl. Unload/Reload test has been done successfully with a patch in vhost kernel. You need a patch to enable vhost. And I think you can only do it for vhost-kenrel now since vhost-user protocol needs some extension I believe. OK. Signed-off-by: Wei Xu --- hw/virtio/virtio.c | 42 ++ 1 file changed, 34 insertions(+), 8 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 4543974..153f6d7 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -2862,33 +2862,59 @@ hwaddr virtio_queue_get_used_size(VirtIODevice *vdev, int n) } } -uint16_t virtio_queue_get_last_avail_idx(VirtIODevice *vdev, int n) +uint64_t virtio_queue_get_last_avail_idx(VirtIODevice *vdev, int n) { -return vdev->vq[n].last_avail_idx; +uint64_t num; + +num = vdev->vq[n].last_avail_idx; +if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { +num |= ((uint64_t)vdev->vq[n].avail_wrap_counter) << 16; +num |= ((uint64_t)vdev->vq[n].used_idx) << 32; +num |= ((uint64_t)vdev->vq[n].used_wrap_counter) << 48; So s.num is 32bit, I don't think this can even work. I mistakenly checked out s.num is 64bit, will add a new case in next version. Wouldn't be enough to just get/set avail_wrap_counter? Something like this so that it fits into 32 bits: if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { num |= ((uint64_t)vdev->vq[n].avail_wrap_counter) << 31; } Regards, Maxime Wei Thanks +} + +return num; } -void virtio_queue_set_last_avail_idx(VirtIODevice *vdev, int n, uint16_t idx) +void virtio_queue_set_last_avail_idx(VirtIODevice *vdev, int n, uint64_t num) { -vdev->vq[n].last_avail_idx = idx; -vdev->vq[n].shadow_avail_idx = idx; +vdev->vq[n].shadow_avail_idx = vdev->vq[n].last_avail_idx = (uint16_t)(num); + +if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { +vdev->vq[n].avail_wrap_counter = (uint16_t)(num >> 16); +vdev->vq[n].used_idx = (uint16_t)(num >> 32); +vdev->vq[n].used_wrap_counter = (uint16_t)(num >> 48); +} } void virtio_queue_restore_last_avail_idx(VirtIODevice *vdev, int n) { rcu_read_lock(); -if (vdev->vq[n].vring.desc) { +if (!vdev->vq[n].vring.desc) { +goto out; +} + +if (!virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { vdev->vq[n].last_avail_idx = vring_used_idx(>vq[n]); -vdev->vq[n].shadow_avail_idx = vdev->vq[n].last_avail_idx; } +vdev->vq[n].shadow_avail_idx = vdev->vq[n].last_avail_idx; + +out: rcu_read_unlock(); } void virtio_queue_update_used_idx(VirtIODevice *vdev, int n) { rcu_read_lock(); -if (vdev->vq[n].vring.desc) { +if (!vdev->vq[n].vring.desc) { +goto out; +} + +if (!virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { vdev->vq[n].used_idx = vring_used_idx(>vq[n]); } + +out: rcu_read_unlock(); }
Re: [Qemu-devel] [RFC v2 8/8] virtio: guest driver reload for vhost-net
On Wed, Jun 06, 2018 at 11:48:19AM +0800, Jason Wang wrote: > > > On 2018年06月06日 03:08, w...@redhat.com wrote: > >From: Wei Xu > > > >last_avail, avail_wrap_count, used_idx and used_wrap_count are > >needed to support vhost-net backend, all these are either 16 or > >bool variables, since state.num is 64bit wide, so here it is > >possible to put them to the 'num' without introducing a new case > >while handling ioctl. > > > >Unload/Reload test has been done successfully with a patch in vhost kernel. > > You need a patch to enable vhost. > > And I think you can only do it for vhost-kenrel now since vhost-user > protocol needs some extension I believe. OK. > > > > >Signed-off-by: Wei Xu > >--- > > hw/virtio/virtio.c | 42 ++ > > 1 file changed, 34 insertions(+), 8 deletions(-) > > > >diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c > >index 4543974..153f6d7 100644 > >--- a/hw/virtio/virtio.c > >+++ b/hw/virtio/virtio.c > >@@ -2862,33 +2862,59 @@ hwaddr virtio_queue_get_used_size(VirtIODevice > >*vdev, int n) > > } > > } > >-uint16_t virtio_queue_get_last_avail_idx(VirtIODevice *vdev, int n) > >+uint64_t virtio_queue_get_last_avail_idx(VirtIODevice *vdev, int n) > > { > >-return vdev->vq[n].last_avail_idx; > >+uint64_t num; > >+ > >+num = vdev->vq[n].last_avail_idx; > >+if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { > >+num |= ((uint64_t)vdev->vq[n].avail_wrap_counter) << 16; > >+num |= ((uint64_t)vdev->vq[n].used_idx) << 32; > >+num |= ((uint64_t)vdev->vq[n].used_wrap_counter) << 48; > > So s.num is 32bit, I don't think this can even work. I mistakenly checked out s.num is 64bit, will add a new case in next version. Wei > > Thanks > > >+} > >+ > >+return num; > > } > >-void virtio_queue_set_last_avail_idx(VirtIODevice *vdev, int n, uint16_t > >idx) > >+void virtio_queue_set_last_avail_idx(VirtIODevice *vdev, int n, uint64_t > >num) > > { > >-vdev->vq[n].last_avail_idx = idx; > >-vdev->vq[n].shadow_avail_idx = idx; > >+vdev->vq[n].shadow_avail_idx = vdev->vq[n].last_avail_idx = > >(uint16_t)(num); > >+ > >+if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { > >+vdev->vq[n].avail_wrap_counter = (uint16_t)(num >> 16); > >+vdev->vq[n].used_idx = (uint16_t)(num >> 32); > >+vdev->vq[n].used_wrap_counter = (uint16_t)(num >> 48); > >+} > > } > > void virtio_queue_restore_last_avail_idx(VirtIODevice *vdev, int n) > > { > > rcu_read_lock(); > >-if (vdev->vq[n].vring.desc) { > >+if (!vdev->vq[n].vring.desc) { > >+goto out; > >+} > >+ > >+if (!virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { > > vdev->vq[n].last_avail_idx = vring_used_idx(>vq[n]); > >-vdev->vq[n].shadow_avail_idx = vdev->vq[n].last_avail_idx; > > } > >+vdev->vq[n].shadow_avail_idx = vdev->vq[n].last_avail_idx; > >+ > >+out: > > rcu_read_unlock(); > > } > > void virtio_queue_update_used_idx(VirtIODevice *vdev, int n) > > { > > rcu_read_lock(); > >-if (vdev->vq[n].vring.desc) { > >+if (!vdev->vq[n].vring.desc) { > >+goto out; > >+} > >+ > >+if (!virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { > > vdev->vq[n].used_idx = vring_used_idx(>vq[n]); > > } > >+ > >+out: > > rcu_read_unlock(); > > } >
Re: [Qemu-devel] [RFC v2 8/8] virtio: guest driver reload for vhost-net
On 2018年06月06日 03:08, w...@redhat.com wrote: From: Wei Xu last_avail, avail_wrap_count, used_idx and used_wrap_count are needed to support vhost-net backend, all these are either 16 or bool variables, since state.num is 64bit wide, so here it is possible to put them to the 'num' without introducing a new case while handling ioctl. Unload/Reload test has been done successfully with a patch in vhost kernel. You need a patch to enable vhost. And I think you can only do it for vhost-kenrel now since vhost-user protocol needs some extension I believe. Signed-off-by: Wei Xu --- hw/virtio/virtio.c | 42 ++ 1 file changed, 34 insertions(+), 8 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 4543974..153f6d7 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -2862,33 +2862,59 @@ hwaddr virtio_queue_get_used_size(VirtIODevice *vdev, int n) } } -uint16_t virtio_queue_get_last_avail_idx(VirtIODevice *vdev, int n) +uint64_t virtio_queue_get_last_avail_idx(VirtIODevice *vdev, int n) { -return vdev->vq[n].last_avail_idx; +uint64_t num; + +num = vdev->vq[n].last_avail_idx; +if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { +num |= ((uint64_t)vdev->vq[n].avail_wrap_counter) << 16; +num |= ((uint64_t)vdev->vq[n].used_idx) << 32; +num |= ((uint64_t)vdev->vq[n].used_wrap_counter) << 48; So s.num is 32bit, I don't think this can even work. Thanks +} + +return num; } -void virtio_queue_set_last_avail_idx(VirtIODevice *vdev, int n, uint16_t idx) +void virtio_queue_set_last_avail_idx(VirtIODevice *vdev, int n, uint64_t num) { -vdev->vq[n].last_avail_idx = idx; -vdev->vq[n].shadow_avail_idx = idx; +vdev->vq[n].shadow_avail_idx = vdev->vq[n].last_avail_idx = (uint16_t)(num); + +if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { +vdev->vq[n].avail_wrap_counter = (uint16_t)(num >> 16); +vdev->vq[n].used_idx = (uint16_t)(num >> 32); +vdev->vq[n].used_wrap_counter = (uint16_t)(num >> 48); +} } void virtio_queue_restore_last_avail_idx(VirtIODevice *vdev, int n) { rcu_read_lock(); -if (vdev->vq[n].vring.desc) { +if (!vdev->vq[n].vring.desc) { +goto out; +} + +if (!virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { vdev->vq[n].last_avail_idx = vring_used_idx(>vq[n]); -vdev->vq[n].shadow_avail_idx = vdev->vq[n].last_avail_idx; } +vdev->vq[n].shadow_avail_idx = vdev->vq[n].last_avail_idx; + +out: rcu_read_unlock(); } void virtio_queue_update_used_idx(VirtIODevice *vdev, int n) { rcu_read_lock(); -if (vdev->vq[n].vring.desc) { +if (!vdev->vq[n].vring.desc) { +goto out; +} + +if (!virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { vdev->vq[n].used_idx = vring_used_idx(>vq[n]); } + +out: rcu_read_unlock(); }
[Qemu-devel] [RFC v2 8/8] virtio: guest driver reload for vhost-net
From: Wei Xu last_avail, avail_wrap_count, used_idx and used_wrap_count are needed to support vhost-net backend, all these are either 16 or bool variables, since state.num is 64bit wide, so here it is possible to put them to the 'num' without introducing a new case while handling ioctl. Unload/Reload test has been done successfully with a patch in vhost kernel. Signed-off-by: Wei Xu --- hw/virtio/virtio.c | 42 ++ 1 file changed, 34 insertions(+), 8 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 4543974..153f6d7 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -2862,33 +2862,59 @@ hwaddr virtio_queue_get_used_size(VirtIODevice *vdev, int n) } } -uint16_t virtio_queue_get_last_avail_idx(VirtIODevice *vdev, int n) +uint64_t virtio_queue_get_last_avail_idx(VirtIODevice *vdev, int n) { -return vdev->vq[n].last_avail_idx; +uint64_t num; + +num = vdev->vq[n].last_avail_idx; +if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { +num |= ((uint64_t)vdev->vq[n].avail_wrap_counter) << 16; +num |= ((uint64_t)vdev->vq[n].used_idx) << 32; +num |= ((uint64_t)vdev->vq[n].used_wrap_counter) << 48; +} + +return num; } -void virtio_queue_set_last_avail_idx(VirtIODevice *vdev, int n, uint16_t idx) +void virtio_queue_set_last_avail_idx(VirtIODevice *vdev, int n, uint64_t num) { -vdev->vq[n].last_avail_idx = idx; -vdev->vq[n].shadow_avail_idx = idx; +vdev->vq[n].shadow_avail_idx = vdev->vq[n].last_avail_idx = (uint16_t)(num); + +if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { +vdev->vq[n].avail_wrap_counter = (uint16_t)(num >> 16); +vdev->vq[n].used_idx = (uint16_t)(num >> 32); +vdev->vq[n].used_wrap_counter = (uint16_t)(num >> 48); +} } void virtio_queue_restore_last_avail_idx(VirtIODevice *vdev, int n) { rcu_read_lock(); -if (vdev->vq[n].vring.desc) { +if (!vdev->vq[n].vring.desc) { +goto out; +} + +if (!virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { vdev->vq[n].last_avail_idx = vring_used_idx(>vq[n]); -vdev->vq[n].shadow_avail_idx = vdev->vq[n].last_avail_idx; } +vdev->vq[n].shadow_avail_idx = vdev->vq[n].last_avail_idx; + +out: rcu_read_unlock(); } void virtio_queue_update_used_idx(VirtIODevice *vdev, int n) { rcu_read_lock(); -if (vdev->vq[n].vring.desc) { +if (!vdev->vq[n].vring.desc) { +goto out; +} + +if (!virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { vdev->vq[n].used_idx = vring_used_idx(>vq[n]); } + +out: rcu_read_unlock(); } -- 1.8.3.1