Hi Maxime,
> -----Original Message-----
> From: Maxime Coquelin <[email protected]>
> Sent: Thursday, June 17, 2021 11:38 PM
> To: [email protected]; [email protected]; Xia, Chenbo
> <[email protected]>
> Cc: Maxime Coquelin <[email protected]>; [email protected]
> Subject: [PATCH v4 1/7] vhost: fix missing memory table NUMA realloc
>
> When the guest allocates virtqueues on a different NUMA node
> than the one the Vhost metadata are allocated, both the Vhost
> device struct and the virtqueues struct are reallocated.
>
> However, reallocating the Vhost memory table was missing, which
> likely causes iat least one cross-NUMA accesses for every burst
> of packets.
'at least' ?
>
> This patch reallocates this table on the same NUMA node as the
> other metadata.
>
> Fixes: 552e8fd3d2b4 ("vhost: simplify memory regions handling")
> Cc: [email protected]
>
> Reported-by: David Marchand <[email protected]>
> Signed-off-by: Maxime Coquelin <[email protected]>
> ---
> lib/vhost/vhost_user.c | 15 +++++++++++++++
> 1 file changed, 15 insertions(+)
>
> diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c
> index 8f0eba6412..031e3bfa2f 100644
> --- a/lib/vhost/vhost_user.c
> +++ b/lib/vhost/vhost_user.c
> @@ -557,6 +557,9 @@ numa_realloc(struct virtio_net *dev, int index)
As we are realloc more things now, the comment above 'numa_realloc(XXX)'
should also be changed like:
Reallocate related data structure to make them on the same numa node as
the memory of vring descriptor.
Thanks,
Chenbo
> goto out;
> }
> if (oldnode != newnode) {
> + struct rte_vhost_memory *old_mem;
> + ssize_t mem_size;
> +
> VHOST_LOG_CONFIG(INFO,
> "reallocate dev from %d to %d node\n",
> oldnode, newnode);
> @@ -568,6 +571,18 @@ numa_realloc(struct virtio_net *dev, int index)
>
> memcpy(dev, old_dev, sizeof(*dev));
> rte_free(old_dev);
> +
> + mem_size = sizeof(struct rte_vhost_memory) +
> + sizeof(struct rte_vhost_mem_region) *
> dev->mem->nregions;
> + old_mem = dev->mem;
> + dev->mem = rte_malloc_socket(NULL, mem_size, 0, newnode);
> + if (!dev->mem) {
> + dev->mem = old_mem;
> + goto out;
> + }
> +
> + memcpy(dev->mem, old_mem, mem_size);
> + rte_free(old_mem);
> }
>
> out:
> --
> 2.31.1
