[dpdk-dev] [PATCH v2 3/3] vhost: fix vq realloc at numa_realloc

[Yuanhan Liu]

On Mon, Mar 07, 2016 at 01:49:26PM +, Loftus, Ciara wrote:
> 
> I encountered the " PANIC in rte_free():" error when using 
> RTE_LIBRTE_VHOST_NUMA too, and applying this series resolved the issue. 
> Thanks for the patches.
> 
> Tested-by: Ciara Loftus 

Thanks for testing. A new version (just about rebase) will hopefully be
sent out in one or two days.

--yliu

[dpdk-dev] [PATCH v2 3/3] vhost: fix vq realloc at numa_realloc

[Loftus, Ciara]

> 
> vq is allocated on pairs, hence we should do pair reallocation
> at numa_realloc() as well, otherwise an error like following
> occurs while do numa reallocation:
> 
> VHOST_CONFIG: reallocate vq from 0 to 1 node
> PANIC in rte_free():
> Fatal error: Invalid memory
> 
> The reason we don't catch it is because numa_realloc() will
> not take effect when RTE_LIBRTE_VHOST_NUMA is not enabled,
> which is the default case.
> 
> Fixes: e049ca6d10e0 ("vhost-user: prepare multiple queue setup")
> 
> Signed-off-by: Yuanhan Liu 
> Acked-by: Huawei Xie 
> ---
>  lib/librte_vhost/virtio-net.c | 13 +++--
>  1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/lib/librte_vhost/virtio-net.c b/lib/librte_vhost/virtio-net.c
> index 1566c93..7469312 100644
> --- a/lib/librte_vhost/virtio-net.c
> +++ b/lib/librte_vhost/virtio-net.c
> @@ -445,6 +445,13 @@ numa_realloc(struct virtio_net *dev, int index)
>   struct vhost_virtqueue *old_vq, *vq;
>   int ret;
> 
> + /*
> +  * vq is allocated on pairs, we should try to do realloc
> +  * on first queue of one queue pair only.
> +  */
> + if (index % VIRTIO_QNUM != 0)
> + return dev;
> +
>   old_dev = dev;
>   vq = old_vq = dev->virtqueue[index];
> 
> @@ -461,11 +468,12 @@ numa_realloc(struct virtio_net *dev, int index)
>   if (oldnode != newnode) {
>   RTE_LOG(INFO, VHOST_CONFIG,
>   "reallocate vq from %d to %d node\n", oldnode,
> newnode);
> - vq = rte_malloc_socket(NULL, sizeof(*vq), 0, newnode);
> + vq = rte_malloc_socket(NULL, sizeof(*vq) * VIRTIO_QNUM,
> 0,
> +newnode);
>   if (!vq)
>   return dev;
> 
> - memcpy(vq, old_vq, sizeof(*vq));
> + memcpy(vq, old_vq, sizeof(*vq) * VIRTIO_QNUM);
>   rte_free(old_vq);
>   }
> 
> @@ -491,6 +499,7 @@ numa_realloc(struct virtio_net *dev, int index)
> 
>  out:
>   dev->virtqueue[index] = vq;
> + dev->virtqueue[index + 1] = vq + 1;
>   vhost_devices[dev->device_fh] = dev;
> 
>   return dev;
> --
> 1.9.0

I encountered the " PANIC in rte_free():" error when using 
RTE_LIBRTE_VHOST_NUMA too, and applying this series resolved the issue. Thanks 
for the patches.

Tested-by: Ciara Loftus 

Thanks,
Ciara

[dpdk-dev] [PATCH v2 3/3] vhost: fix vq realloc at numa_realloc

[Yuanhan Liu]

vq is allocated on pairs, hence we should do pair reallocation
at numa_realloc() as well, otherwise an error like following
occurs while do numa reallocation:

VHOST_CONFIG: reallocate vq from 0 to 1 node
PANIC in rte_free():
Fatal error: Invalid memory

The reason we don't catch it is because numa_realloc() will
not take effect when RTE_LIBRTE_VHOST_NUMA is not enabled,
which is the default case.

Fixes: e049ca6d10e0 ("vhost-user: prepare multiple queue setup")

Signed-off-by: Yuanhan Liu 
Acked-by: Huawei Xie 
---
 lib/librte_vhost/virtio-net.c | 13 +++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/lib/librte_vhost/virtio-net.c b/lib/librte_vhost/virtio-net.c
index 1566c93..7469312 100644
--- a/lib/librte_vhost/virtio-net.c
+++ b/lib/librte_vhost/virtio-net.c
@@ -445,6 +445,13 @@ numa_realloc(struct virtio_net *dev, int index)
struct vhost_virtqueue *old_vq, *vq;
int ret;

+   /*
+* vq is allocated on pairs, we should try to do realloc
+* on first queue of one queue pair only.
+*/
+   if (index % VIRTIO_QNUM != 0)
+   return dev;
+
old_dev = dev;
vq = old_vq = dev->virtqueue[index];

@@ -461,11 +468,12 @@ numa_realloc(struct virtio_net *dev, int index)
if (oldnode != newnode) {
RTE_LOG(INFO, VHOST_CONFIG,
"reallocate vq from %d to %d node\n", oldnode, newnode);
-   vq = rte_malloc_socket(NULL, sizeof(*vq), 0, newnode);
+   vq = rte_malloc_socket(NULL, sizeof(*vq) * VIRTIO_QNUM, 0,
+  newnode);
if (!vq)
return dev;

-   memcpy(vq, old_vq, sizeof(*vq));
+   memcpy(vq, old_vq, sizeof(*vq) * VIRTIO_QNUM);
rte_free(old_vq);
}

@@ -491,6 +499,7 @@ numa_realloc(struct virtio_net *dev, int index)

 out:
dev->virtqueue[index] = vq;
+   dev->virtqueue[index + 1] = vq + 1;
vhost_devices[dev->device_fh] = dev;

return dev;
-- 
1.9.0