Re: [PATCH] vhost-vdpa: fix NULL pointer deref in _compat_vdpa_reset

2023-10-24 Thread Dragos Tatulea via Virtualization
On Mon, 2023-10-23 at 16:14 -0700, Si-Wei Liu wrote: > As subject. There's a vhost_vdpa_reset() done earlier before > vhost_dev is initialized via vhost_dev_init(), ending up with > NULL pointer dereference. Fix is to check if vqs is initialized > before checking backend features and resetting the

Re: [PATCH vhost v4 12/16] vdpa/mlx5: Improve mr update flow

2023-10-23 Thread Dragos Tatulea via Virtualization
On Fri, 2023-10-20 at 18:01 +0200, Eugenio Perez Martin wrote: > On Wed, Oct 18, 2023 at 7:21 PM Dragos Tatulea wrote: > > > > On Wed, 2023-10-18 at 20:14 +0300, Dragos Tatulea wrote: > > > The current flow for updating an mr works directly on mvdev->mr which > > > makes it cumbersome to handle m

Re: [PATCH vhost v4 12/16] vdpa/mlx5: Improve mr update flow

2023-10-18 Thread Dragos Tatulea via Virtualization
On Wed, 2023-10-18 at 20:14 +0300, Dragos Tatulea wrote: > The current flow for updating an mr works directly on mvdev->mr which > makes it cumbersome to handle multiple new mr structs. > > This patch makes the flow more straightforward by having > mlx5_vdpa_create_mr return a new mr which will up

[PATCH vhost v4 16/16] vdpa/mlx5: Update cvq iotlb mapping on ASID change

2023-10-18 Thread Dragos Tatulea via Virtualization
For the following sequence: - cvq group is in ASID 0 - .set_map(1, cvq_iotlb) - .set_group_asid(cvq_group, 1) ... the cvq mapping from ASID 0 will be used. This is not always correct behaviour. This patch adds support for the above mentioned flow by saving the iotlb on each .set_map and updating

[PATCH vhost v4 15/16] vdpa/mlx5: Make iotlb helper functions more generic

2023-10-18 Thread Dragos Tatulea via Virtualization
They will be used in a follow-up patch. For dup_iotlb, avoid the src == dst case. This is an error. Acked-by: Jason Wang Acked-by: Eugenio Pérez Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/core/mr.c | 19 +++ 1 file changed, 11 insertions(+), 8 deletions(-) diff --git

[PATCH vhost v4 14/16] vdpa/mlx5: Enable hw support for vq descriptor mapping

2023-10-18 Thread Dragos Tatulea via Virtualization
Vq descriptor mappings are supported in hardware by filling in an additional mkey which contains the descriptor mappings to the hw vq. A previous patch in this series added support for hw mkey (mr) creation for ASID 1. This patch fills in both the vq data and vq descriptor mkeys based on group AS

[PATCH vhost v4 12/16] vdpa/mlx5: Improve mr update flow

2023-10-18 Thread Dragos Tatulea via Virtualization
The current flow for updating an mr works directly on mvdev->mr which makes it cumbersome to handle multiple new mr structs. This patch makes the flow more straightforward by having mlx5_vdpa_create_mr return a new mr which will update the old mr (if any). The old mr will be deleted and unlinked f

[PATCH vhost v4 13/16] vdpa/mlx5: Introduce mr for vq descriptor

2023-10-18 Thread Dragos Tatulea via Virtualization
Introduce the vq descriptor group and mr per ASID. Until now .set_map on ASID 1 was only updating the cvq iotlb. From now on it also creates a mkey for it. The current patch doesn't use it but follow-up patches will add hardware support for mapping the vq descriptors. Acked-by: Jason Wang Acked-b

[PATCH vhost v4 11/16] vdpa/mlx5: Move mr mutex out of mr struct

2023-10-18 Thread Dragos Tatulea via Virtualization
The mutex is named like it is supposed to protect only the mkey but in reality it is a global lock for all mr resources. Shift the mutex to it's rightful location (struct mlx5_vdpa_dev) and give it a more appropriate name. Signed-off-by: Dragos Tatulea Acked-by: Eugenio Pérez Acked-by: Jason Wa

[PATCH vhost v4 10/16] vdpa/mlx5: Allow creation/deletion of any given mr struct

2023-10-18 Thread Dragos Tatulea via Virtualization
This patch adapts the mr creation/deletion code to be able to work with any given mr struct pointer. All the APIs are adapted to take an extra parameter for the mr. mlx5_vdpa_create/delete_mr doesn't need a ASID parameter anymore. The check is done in the caller instead (mlx5_set_map). This chang

[PATCH vhost v4 07/16] vdpa/mlx5: Take cvq iotlb lock during refresh

2023-10-18 Thread Dragos Tatulea via Virtualization
The reslock is taken while refresh is called but iommu_lock is more specific to this resource. So take the iommu_lock during cvq iotlb refresh. Based on Eugenio's patch [0]. [0] https://lore.kernel.org/lkml/20230112142218.725622-4-epere...@redhat.com/ Acked-by: Jason Wang Suggested-by: Eugenio

[PATCH vhost v4 06/16] vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code

2023-10-18 Thread Dragos Tatulea via Virtualization
The handling of the cvq iotlb is currently coupled with the creation and destruction of the hardware mkeys (mr). This patch moves cvq iotlb handling into its own function and shifts it to a scope that is not related to mr handling. As cvq handling is just a prune_iotlb + dup_iotlb cycle, put it al

[PATCH vhost v4 04/16] vhost-vdpa: uAPI to get dedicated descriptor group id

2023-10-18 Thread Dragos Tatulea via Virtualization
From: Si-Wei Liu With _F_DESC_ASID backend feature, the device can now support the VHOST_VDPA_GET_VRING_DESC_GROUP ioctl, and it may expose the descriptor table (including avail and used ring) in a different group than the buffers it contains. This new uAPI will fetch the group ID of the descript

[PATCH vhost v4 09/16] vdpa/mlx5: Rename mr destroy functions

2023-10-18 Thread Dragos Tatulea via Virtualization
Make mlx5_destroy_mr symmetric to mlx5_create_mr. Acked-by: Jason Wang Acked-by: Eugenio Pérez Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 4 ++-- drivers/vdpa/mlx5/core/mr.c| 6 +++--- drivers/vdpa/mlx5/net/mlx5_vnet.c | 12 ++-- 3 files changed,

[PATCH vhost v4 08/16] vdpa/mlx5: Collapse "dvq" mr add/delete functions

2023-10-18 Thread Dragos Tatulea via Virtualization
Now that the cvq code is out of mlx5_vdpa_create/destroy_mr, the "dvq" functions can be folded into their callers. Having "dvq" in the naming will no longer be accurate in the downstream patches. Acked-by: Jason Wang Acked-by: Eugenio Pérez Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/

[PATCH vhost v4 03/16] vhost-vdpa: introduce descriptor group backend feature

2023-10-18 Thread Dragos Tatulea via Virtualization
From: Si-Wei Liu Userspace knows if the device has dedicated descriptor group or not by checking this feature bit. It's only exposed if the vdpa driver backend implements the .get_vq_desc_group() operation callback. Userspace trying to negotiate this feature when it or the dependent _F_IOTLB_ASI

[PATCH vhost v4 05/16] vdpa/mlx5: Create helper function for dma mappings

2023-10-18 Thread Dragos Tatulea via Virtualization
Necessary for upcoming cvq separation from mr allocation. Acked-by: Jason Wang Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1 + drivers/vdpa/mlx5/core/mr.c| 5 + drivers/vdpa/mlx5/net/mlx5_vnet.c | 4 ++-- 3 files changed, 8 insertions(+), 2 deletions(-)

[PATCH vhost v4 02/16] vdpa: introduce dedicated descriptor group for virtqueue

2023-10-18 Thread Dragos Tatulea via Virtualization
From: Si-Wei Liu In some cases, the access to the virtqueue's descriptor area, device and driver areas (precluding indirect descriptor table in guest memory) may have to be confined to a different address space than where its buffers reside. Without loss of simplicity and generality with already

[PATCH vhost v4 01/16] vdpa/mlx5: Expose descriptor group mkey hw capability

2023-10-18 Thread Dragos Tatulea via Virtualization
Necessary for improved live migration flow. Actual support will be added in a downstream patch. Signed-off-by: Dragos Tatulea Reviewed-by: Gal Pressman --- include/linux/mlx5/mlx5_ifc.h | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/include/linux/mlx5/mlx5_ifc.h b/in

[PATCH vhost v4 00/16] vdpa: Add support for vq descriptor mappings

2023-10-18 Thread Dragos Tatulea via Virtualization
This patch series adds support for vq descriptor table mappings which are used to improve vdpa live migration downtime. The improvement comes from using smaller mappings which take less time to create and destroy in hw. The first part adds the vdpa core changes from Si-Wei [0]. The second part ad

[PATCH vhost v3 16/16] vdpa/mlx5: Update cvq iotlb mapping on ASID change

2023-10-09 Thread Dragos Tatulea via Virtualization
For the following sequence: - cvq group is in ASID 0 - .set_map(1, cvq_iotlb) - .set_group_asid(cvq_group, 1) ... the cvq mapping from ASID 0 will be used. This is not always correct behaviour. This patch adds support for the above mentioned flow by saving the iotlb on each .set_map and updating

[PATCH vhost v3 15/16] vdpa/mlx5: Make iotlb helper functions more generic

2023-10-09 Thread Dragos Tatulea via Virtualization
They will be used in a follow-up patch. For dup_iotlb, avoid the src == dst case. This is an error. Acked-by: Eugenio Pérez Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/core/mr.c | 19 +++ 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/drivers/vdpa/mlx5/c

[PATCH vhost v3 14/16] vdpa/mlx5: Enable hw support for vq descriptor mapping

2023-10-09 Thread Dragos Tatulea via Virtualization
Vq descriptor mappings are supported in hardware by filling in an additional mkey which contains the descriptor mappings to the hw vq. A previous patch in this series added support for hw mkey (mr) creation for ASID 1. This patch fills in both the vq data and vq descriptor mkeys based on group AS

[PATCH vhost v3 13/16] vdpa/mlx5: Introduce mr for vq descriptor

2023-10-09 Thread Dragos Tatulea via Virtualization
Introduce the vq descriptor group and mr per ASID. Until now .set_map on ASID 1 was only updating the cvq iotlb. From now on it also creates a mkey for it. The current patch doesn't use it but follow-up patches will add hardware support for mapping the vq descriptors. Acked-by: Eugenio Pérez Sign

[PATCH vhost v3 12/16] vdpa/mlx5: Improve mr update flow

2023-10-09 Thread Dragos Tatulea via Virtualization
The current flow for updating an mr works directly on mvdev->mr which makes it cumbersome to handle multiple new mr structs. This patch makes the flow more straightforward by having mlx5_vdpa_create_mr return a new mr which will update the old mr (if any). The old mr will be deleted and unlinked f

[PATCH vhost v3 10/16] vdpa/mlx5: Allow creation/deletion of any given mr struct

2023-10-09 Thread Dragos Tatulea via Virtualization
This patch adapts the mr creation/deletion code to be able to work with any given mr struct pointer. All the APIs are adapted to take an extra parameter for the mr. mlx5_vdpa_create/delete_mr doesn't need a ASID parameter anymore. The check is done in the caller instead (mlx5_set_map). This chang

[PATCH vhost v3 11/16] vdpa/mlx5: Move mr mutex out of mr struct

2023-10-09 Thread Dragos Tatulea via Virtualization
The mutex is named like it is supposed to protect only the mkey but in reality it is a global lock for all mr resources. Shift the mutex to it's rightful location (struct mlx5_vdpa_dev) and give it a more appropriate name. Signed-off-by: Dragos Tatulea Acked-by: Eugenio Pérez --- drivers/vdpa/

[PATCH vhost v3 09/16] vdpa/mlx5: Rename mr destroy functions

2023-10-09 Thread Dragos Tatulea via Virtualization
Make mlx5_destroy_mr symmetric to mlx5_create_mr. Acked-by: Jason Wang Acked-by: Eugenio Pérez Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 4 ++-- drivers/vdpa/mlx5/core/mr.c| 6 +++--- drivers/vdpa/mlx5/net/mlx5_vnet.c | 12 ++-- 3 files changed,

[PATCH vhost v3 08/16] vdpa/mlx5: Collapse "dvq" mr add/delete functions

2023-10-09 Thread Dragos Tatulea via Virtualization
Now that the cvq code is out of mlx5_vdpa_create/destroy_mr, the "dvq" functions can be folded into their callers. Having "dvq" in the naming will no longer be accurate in the downstream patches. Acked-by: Jason Wang Acked-by: Eugenio Pérez Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/

[PATCH vhost v3 05/16] vdpa/mlx5: Create helper function for dma mappings

2023-10-09 Thread Dragos Tatulea via Virtualization
Necessary for upcoming cvq separation from mr allocation. Acked-by: Jason Wang Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1 + drivers/vdpa/mlx5/core/mr.c| 5 + drivers/vdpa/mlx5/net/mlx5_vnet.c | 4 ++-- 3 files changed, 8 insertions(+), 2 deletions(-)

[PATCH vhost v3 06/16] vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code

2023-10-09 Thread Dragos Tatulea via Virtualization
The handling of the cvq iotlb is currently coupled with the creation and destruction of the hardware mkeys (mr). This patch moves cvq iotlb handling into its own function and shifts it to a scope that is not related to mr handling. As cvq handling is just a prune_iotlb + dup_iotlb cycle, put it al

[PATCH vhost v3 07/16] vdpa/mlx5: Take cvq iotlb lock during refresh

2023-10-09 Thread Dragos Tatulea via Virtualization
The reslock is taken while refresh is called but iommu_lock is more specific to this resource. So take the iommu_lock during cvq iotlb refresh. Based on Eugenio's patch [0]. [0] https://lore.kernel.org/lkml/20230112142218.725622-4-epere...@redhat.com/ Acked-by: Jason Wang Suggested-by: Eugenio

[PATCH vhost v3 04/16] vhost-vdpa: uAPI to get dedicated descriptor group id

2023-10-09 Thread Dragos Tatulea via Virtualization
From: Si-Wei Liu With _F_DESC_ASID backend feature, the device can now support the VHOST_VDPA_GET_VRING_DESC_GROUP ioctl, and it may expose the descriptor table (including avail and used ring) in a different group than the buffers it contains. This new uAPI will fetch the group ID of the descript

[PATCH vhost v3 02/16] vdpa: introduce dedicated descriptor group for virtqueue

2023-10-09 Thread Dragos Tatulea via Virtualization
From: Si-Wei Liu In some cases, the access to the virtqueue's descriptor area, device and driver areas (precluding indirect descriptor table in guest memory) may have to be confined to a different address space than where its buffers reside. Without loss of simplicity and generality with already

[PATCH vhost v3 03/16] vhost-vdpa: introduce descriptor group backend feature

2023-10-09 Thread Dragos Tatulea via Virtualization
From: Si-Wei Liu Userspace knows if the device has dedicated descriptor group or not by checking this feature bit. It's only exposed if the vdpa driver backend implements the .get_vq_desc_group() operation callback. Userspace trying to negotiate this feature when it or the dependent _F_IOTLB_ASI

[PATCH mlx5-vhost v3 01/16] vdpa/mlx5: Expose descriptor group mkey hw capability

2023-10-09 Thread Dragos Tatulea via Virtualization
Necessary for improved live migration flow. Actual support will be added in a downstream patch. Reviewed-by: Gal Pressman Signed-off-by: Dragos Tatulea --- include/linux/mlx5/mlx5_ifc.h | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/include/linux/mlx5/mlx5_ifc.h b/in

[PATCH vhost v3 00/16] vdpa: Add support for vq descriptor mappings

2023-10-09 Thread Dragos Tatulea via Virtualization
This patch series adds support for vq descriptor table mappings which are used to improve vdpa live migration downtime. The improvement comes from using smaller mappings which take less time to create and destroy in hw. The first part adds the vdpa core changes from Si-Wei [0]. The second part ad

Re: [PATCH 09/16] vdpa/mlx5: Allow creation/deletion of any given mr struct

2023-10-09 Thread Dragos Tatulea via Virtualization
On Mon, 2023-10-09 at 14:39 +0800, Jason Wang wrote: > On Sun, Oct 8, 2023 at 8:05 PM Dragos Tatulea wrote: > > > > On Sun, 2023-10-08 at 12:25 +0800, Jason Wang wrote: > > > On Tue, Sep 26, 2023 at 3:21 PM Dragos Tatulea > > > wrote: > > > > > > > > On Tue, 2023-09-26 at 12:44 +0800, Jason Wan

Re: [PATCH 09/16] vdpa/mlx5: Allow creation/deletion of any given mr struct

2023-10-08 Thread Dragos Tatulea via Virtualization
On Sun, 2023-10-08 at 12:25 +0800, Jason Wang wrote: > On Tue, Sep 26, 2023 at 3:21 PM Dragos Tatulea wrote: > > > > On Tue, 2023-09-26 at 12:44 +0800, Jason Wang wrote: > > > On Tue, Sep 12, 2023 at 9:02 PM Dragos Tatulea > > > wrote: > > > > > > > > This patch adapts the mr creation/deletion

Re: [PATCH vhost v2 00/16] vdpa: Add support for vq descriptor mappings

2023-10-05 Thread Dragos Tatulea via Virtualization
On Thu, 2023-10-05 at 13:31 -0400, Michael S. Tsirkin wrote: > On Thu, Sep 28, 2023 at 07:45:11PM +0300, Dragos Tatulea wrote: > > This patch series adds support for vq descriptor table mappings which > > are used to improve vdpa live migration downtime. The improvement comes > > from using smaller

Re: [PATCH vhost 16/16] vdpa/mlx5: Update cvq iotlb mapping on ASID change

2023-10-05 Thread Dragos Tatulea via Virtualization
On Thu, 2023-10-05 at 12:41 +0200, Eugenio Perez Martin wrote: > On Thu, Sep 28, 2023 at 6:50 PM Dragos Tatulea wrote: > > > > For the following sequence: > > - cvq group is in ASID 0 > > - .set_map(1, cvq_iotlb) > > - .set_group_asid(cvq_group, 1) > > > > ... the cvq mapping from ASID 0 will be

Re: [PATCH vhost 15/16] vdpa/mlx5: Make iotlb helper functions more generic

2023-10-05 Thread Dragos Tatulea via Virtualization
On Thu, 2023-10-05 at 12:14 +0200, Eugenio Perez Martin wrote: > On Thu, Sep 28, 2023 at 6:50 PM Dragos Tatulea wrote: > > > > They will be used in a followup patch. > > > > Signed-off-by: Dragos Tatulea > > --- > >  drivers/vdpa/mlx5/core/mr.c | 16 > >  1 file changed, 8 inser

Re: [PATCH vhost 14/16] vdpa/mlx5: Enable hw support for vq descriptor mapping

2023-10-05 Thread Dragos Tatulea via Virtualization
On Thu, 2023-10-05 at 11:42 +0200, Eugenio Perez Martin wrote: > On Thu, Sep 28, 2023 at 6:50 PM Dragos Tatulea wrote: > > > > Vq descriptor mappings are supported in hardware by filling in an > > additional mkey which contains the descriptor mappings to the hw vq. > > > > A previous patch in th

Re: [PATCH vhost 13/16] vdpa/mlx5: Introduce mr for vq descriptor

2023-10-05 Thread Dragos Tatulea via Virtualization
On Wed, 2023-10-04 at 20:53 +0200, Eugenio Perez Martin wrote: > On Thu, Sep 28, 2023 at 6:50 PM Dragos Tatulea wrote: > > > > Introduce the vq descriptor group and ASID 1. Until now .set_map on ASID > > s/ASID/vq group/? > Oh, indeed. > > 1 was only updating the cvq iotlb. From now on it also

Re: [PATCH vhost 10/16] vdpa/mlx5: Allow creation/deletion of any given mr struct

2023-10-03 Thread Dragos Tatulea via Virtualization
On Thu, 2023-09-28 at 19:45 +0300, Dragos Tatulea wrote: > This patch adapts the mr creation/deletion code to be able to work with > any given mr struct pointer. All the APIs are adapted to take an extra > parameter for the mr. > > mlx5_vdpa_create/delete_mr doesn't need a ASID parameter anymore.

[PATCH vhost 15/16] vdpa/mlx5: Make iotlb helper functions more generic

2023-09-28 Thread Dragos Tatulea via Virtualization
They will be used in a followup patch. Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/core/mr.c | 16 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c index 3dee6d9bed6b..a4135c16b5bf 100644 --- a/driver

[PATCH vhost 16/16] vdpa/mlx5: Update cvq iotlb mapping on ASID change

2023-09-28 Thread Dragos Tatulea via Virtualization
For the following sequence: - cvq group is in ASID 0 - .set_map(1, cvq_iotlb) - .set_group_asid(cvq_group, 1) ... the cvq mapping from ASID 0 will be used. This is not always correct behaviour. This patch adds support for the above mentioned flow by saving the iotlb on each .set_map and updating

[PATCH vhost 14/16] vdpa/mlx5: Enable hw support for vq descriptor mapping

2023-09-28 Thread Dragos Tatulea via Virtualization
Vq descriptor mappings are supported in hardware by filling in an additional mkey which contains the descriptor mappings to the hw vq. A previous patch in this series added support for hw mkey (mr) creation for ASID 1. This patch fills in both the vq data and vq descriptor mkeys based on group AS

[PATCH vhost 13/16] vdpa/mlx5: Introduce mr for vq descriptor

2023-09-28 Thread Dragos Tatulea via Virtualization
Introduce the vq descriptor group and ASID 1. Until now .set_map on ASID 1 was only updating the cvq iotlb. From now on it also creates a mkey for it. The current patch doesn't use it but follow-up patches will add hardware support for mapping the vq descriptors. Signed-off-by: Dragos Tatulea ---

[PATCH vhost 11/16] vdpa/mlx5: Move mr mutex out of mr struct

2023-09-28 Thread Dragos Tatulea via Virtualization
The mutex is named like it is supposed to protect only the mkey but in reality it is a global lock for all mr resources. Shift the mutex to it's rightful location (struct mlx5_vdpa_dev) and give it a more appropriate name. Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/core/mlx5_vdpa.h |

[PATCH vhost 08/16] vdpa/mlx5: Collapse "dvq" mr add/delete functions

2023-09-28 Thread Dragos Tatulea via Virtualization
Now that the cvq code is out of mlx5_vdpa_create/destroy_mr, the "dvq" functions can be folded into their callers. Having "dvq" in the naming will no longer be accurate in the downstream patches. Acked-by: Jason Wang Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/core/mr.c | 16 +-

[PATCH vhost 12/16] vdpa/mlx5: Improve mr update flow

2023-09-28 Thread Dragos Tatulea via Virtualization
The current flow for updating an mr works directly on mvdev->mr which makes it cumbersome to handle multiple new mr structs. This patch makes the flow more straightforward by having mlx5_vdpa_create_mr return a new mr which will update the old mr (if any). The old mr will be deleted and unlinked f

[PATCH vhost 07/16] vdpa/mlx5: Take cvq iotlb lock during refresh

2023-09-28 Thread Dragos Tatulea via Virtualization
The reslock is taken while refresh is called but iommu_lock is more specific to this resource. So take the iommu_lock during cvq iotlb refresh. Based on Eugenio's patch [0]. [0] https://lore.kernel.org/lkml/20230112142218.725622-4-epere...@redhat.com/ Acked-by: Jason Wang Suggested-by: Eugenio

[PATCH vhost 10/16] vdpa/mlx5: Allow creation/deletion of any given mr struct

2023-09-28 Thread Dragos Tatulea via Virtualization
This patch adapts the mr creation/deletion code to be able to work with any given mr struct pointer. All the APIs are adapted to take an extra parameter for the mr. mlx5_vdpa_create/delete_mr doesn't need a ASID parameter anymore. The check is done in the caller instead (mlx5_set_map). This chang

[PATCH vhost 09/16] vdpa/mlx5: Rename mr destroy functions

2023-09-28 Thread Dragos Tatulea via Virtualization
Make mlx5_destroy_mr symmetric to mlx5_create_mr. Acked-by: Jason Wang Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 4 ++-- drivers/vdpa/mlx5/core/mr.c| 6 +++--- drivers/vdpa/mlx5/net/mlx5_vnet.c | 12 ++-- 3 files changed, 11 insertions(+), 11 dele

[PATCH vhost 06/16] vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code

2023-09-28 Thread Dragos Tatulea via Virtualization
The handling of the cvq iotlb is currently coupled with the creation and destruction of the hardware mkeys (mr). This patch moves cvq iotlb handling into its own function and shifts it to a scope that is not related to mr handling. As cvq handling is just a prune_iotlb + dup_iotlb cycle, put it al

[PATCH vhost 04/16] vhost-vdpa: uAPI to get dedicated descriptor group id

2023-09-28 Thread Dragos Tatulea via Virtualization
From: Si-Wei Liu With _F_DESC_ASID backend feature, the device can now support the VHOST_VDPA_GET_VRING_DESC_GROUP ioctl, and it may expose the descriptor table (including avail and used ring) in a different group than the buffers it contains. This new uAPI will fetch the group ID of the descript

[PATCH vhost 05/16] vdpa/mlx5: Create helper function for dma mappings

2023-09-28 Thread Dragos Tatulea via Virtualization
Necessary for upcoming cvq separation from mr allocation. Acked-by: Jason Wang Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1 + drivers/vdpa/mlx5/core/mr.c| 5 + drivers/vdpa/mlx5/net/mlx5_vnet.c | 4 ++-- 3 files changed, 8 insertions(+), 2 deletions(-)

[PATCH vhost 03/16] vhost-vdpa: introduce descriptor group backend feature

2023-09-28 Thread Dragos Tatulea via Virtualization
From: Si-Wei Liu Userspace knows if the device has dedicated descriptor group or not by checking this feature bit. It's only exposed if the vdpa driver backend implements the .get_vq_desc_group() operation callback. Userspace trying to negotiate this feature when it or the dependent _F_IOTLB_ASI

[PATCH vhost 02/16] vdpa: introduce dedicated descriptor group for virtqueue

2023-09-28 Thread Dragos Tatulea via Virtualization
From: Si-Wei Liu In some cases, the access to the virtqueue's descriptor area, device and driver areas (precluding indirect descriptor table in guest memory) may have to be confined to a different address space than where its buffers reside. Without loss of simplicity and generality with already

[PATCH mlx5-next 01/16] vdpa/mlx5: Expose descriptor group mkey hw capability

2023-09-28 Thread Dragos Tatulea via Virtualization
Necessary for improved live migration flow. Actual support will be added in a downstream patch. Reviewed-by: Gal Pressman Signed-off-by: Dragos Tatulea --- include/linux/mlx5/mlx5_ifc.h | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/include/linux/mlx5/mlx5_ifc.h b/in

[PATCH vhost v2 00/16] vdpa: Add support for vq descriptor mappings

2023-09-28 Thread Dragos Tatulea via Virtualization
This patch series adds support for vq descriptor table mappings which are used to improve vdpa live migration downtime. The improvement comes from using smaller mappings which take less time to create and destroy in hw. The first part adds the vdpa core changes from Si-Wei [0]. The second part ad

[PATCH vhost v2 00/16] vdpa: Add support for vq descriptor mappings

2023-09-28 Thread Dragos Tatulea via Virtualization
This patch series adds support for vq descriptor table mappings which are used to improve vdpa live migration downtime. The improvement comes from using smaller mappings which take less time to create and destroy in hw. The first part adds the vdpa core changes from Si-Wei [0]. The second part ad

Re: [PATCH 00/16] vdpa: Add support for vq descriptor mappings

2023-09-26 Thread Dragos Tatulea via Virtualization
On Tue, 2023-09-26 at 05:22 -0700, Si-Wei Liu wrote: > > > On 9/25/2023 12:59 AM, Dragos Tatulea wrote: > > On Tue, 2023-09-12 at 16:01 +0300, Dragos Tatulea wrote: > > > This patch series adds support for vq descriptor table mappings which > > > are used to improve vdpa live migration downtime.

Re: [PATCH 09/16] vdpa/mlx5: Allow creation/deletion of any given mr struct

2023-09-26 Thread Dragos Tatulea via Virtualization
On Tue, 2023-09-26 at 12:44 +0800, Jason Wang wrote: > On Tue, Sep 12, 2023 at 9:02 PM Dragos Tatulea wrote: > > > > This patch adapts the mr creation/deletion code to be able to work with > > any given mr struct pointer. All the APIs are adapted to take an extra > > parameter for the mr. > > >

[PATCH v2] MAINTAINERS: Add myself as mlx5_vdpa driver

2023-09-25 Thread Dragos Tatulea via Virtualization
As Eli Cohen moved to other work, I'll be the contact point for mlx5_vdpa. Acked-by: Jason Wang Signed-off-by: Dragos Tatulea --- Changes since v1: - Fix alphabetical sorting. - Make naming consistent with other MELLANOX entries. --- MAINTAINERS | 6 ++ 1 file changed, 6 insertions(+) diff

Re: [PATCH 00/16] vdpa: Add support for vq descriptor mappings

2023-09-25 Thread Dragos Tatulea via Virtualization
On Tue, 2023-09-12 at 16:01 +0300, Dragos Tatulea wrote: > This patch series adds support for vq descriptor table mappings which > are used to improve vdpa live migration downtime. The improvement comes > from using smaller mappings which take less time to create and destroy > in hw. > Gentle ping

Re: [PATCH] vdpa/mlx5: Fix double release of debugfs entry

2023-09-14 Thread Dragos Tatulea via Virtualization
On Tue, 2023-08-29 at 20:40 +0300, Dragos Tatulea wrote: > The error path in setup_driver deletes the debugfs entry but doesn't > clear the pointer. During .dev_del the invalid pointer will be released > again causing a crash. > > This patch fixes the issue by always clearing the debugfs entry in

Re: [PATCH v2] vdpa/mlx5: Fix firmware error on creation of 1k VQs

2023-09-14 Thread Dragos Tatulea via Virtualization
On Thu, 2023-08-31 at 18:50 +0300, Dragos Tatulea wrote: > A firmware error is triggered when configuring a 9k MTU on the PF after > switching to switchdev mode and then using a vdpa device with larger > (1k) rings: > mlx5_cmd_out_err: CREATE_GENERAL_OBJECT(0xa00) op_mod(0xd) failed, status bad > r

[PATCH 13/16] vdpa/mlx5: Enable hw support for vq descriptor mapping

2023-09-12 Thread Dragos Tatulea via Virtualization
Vq descriptor mappings are supported in hardware by filling in an additional mkey which contains the descriptor mappings to the hw vq. A previous patch in this series added support for hw mkey (mr) creation for ASID 1. This patch fills in both the vq data and vq descriptor mkeys based on group AS

[PATCH 12/16] vdpa/mlx5: Introduce mr for vq descriptor

2023-09-12 Thread Dragos Tatulea via Virtualization
Introduce the vq descriptor group and ASID 1. Until now .set_map on ASID 1 was only updating the cvq iotlb. From now on it also creates a mkey for it. The current patch doesn't use it but follow-up patches will add hardware support for mapping the vq descriptors. Signed-off-by: Dragos Tatulea ---

[PATCH 11/16] vdpa/mlx5: Improve mr update flow

2023-09-12 Thread Dragos Tatulea via Virtualization
The current flow for updating an mr works directly on mvdev->mr which makes it cumbersome to handle multiple new mr structs. This patch makes the flow more straightforward by having mlx5_vdpa_create_mr return a new mr which will update the old mr (if any). The old mr will be deleted and unlinked f

[PATCH 14/16] vdpa/mlx5: Make iotlb helper functions more generic

2023-09-12 Thread Dragos Tatulea via Virtualization
They will be used in a followup patch. Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/core/mr.c | 16 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c index 3dee6d9bed6b..a4135c16b5bf 100644 --- a/driver

[PATCH 15/16] vdpa/mlx5: Update cvq iotlb mapping on ASID change

2023-09-12 Thread Dragos Tatulea via Virtualization
For the following sequence: - cvq group is in ASID 0 - .set_map(1, cvq_iotlb) - .set_group_asid(cvq_group, 1) ... the cvq mapping from ASID 0 will be used. This is not always correct behaviour. This patch adds support for the above mentioned flow by saving the iotlb on each .set_map and updating

[PATCH 10/16] vdpa/mlx5: Move mr mutex out of mr struct

2023-09-12 Thread Dragos Tatulea via Virtualization
The mutex is named like it is supposed to protect only the mkey but in reality it is a global lock for all mr resources. Shift the mutex to it's rightful location (struct mlx5_vdpa_dev) and give it a more appropriate name. Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/core/mlx5_vdpa.h |

[PATCH 05/16] vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code

2023-09-12 Thread Dragos Tatulea via Virtualization
The handling of the cvq iotlb is currently coupled with the creation and destruction of the hardware mkeys (mr). This patch moves cvq iotlb handling into its own function and shifts it to a scope that is not related to mr handling. As cvq handling is just a prune_iotlb + dup_iotlb cycle, put it al

[PATCH 08/16] vdpa/mlx5: Rename mr destroy functions

2023-09-12 Thread Dragos Tatulea via Virtualization
Make mlx5_destroy_mr symmetric to mlx5_create_mr. Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 4 ++-- drivers/vdpa/mlx5/core/mr.c| 6 +++--- drivers/vdpa/mlx5/net/mlx5_vnet.c | 12 ++-- 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a

[PATCH 07/16] vdpa/mlx5: Collapse "dvq" mr add/delete functions

2023-09-12 Thread Dragos Tatulea via Virtualization
Now that the cvq code is out of mlx5_vdpa_create/destroy_mr, the "dvq" functions can be folded into their callers. Having "dvq" in the naming will no longer be accurate in the downstream patches. Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/core/mr.c | 16 +--- 1 file changed

[PATCH 09/16] vdpa/mlx5: Allow creation/deletion of any given mr struct

2023-09-12 Thread Dragos Tatulea via Virtualization
This patch adapts the mr creation/deletion code to be able to work with any given mr struct pointer. All the APIs are adapted to take an extra parameter for the mr. mlx5_vdpa_create/delete_mr doesn't need a ASID parameter anymore. The check is done in the caller instead (mlx5_set_map). This chang

[PATCH 06/16] vdpa/mlx5: Take cvq iotlb lock during refresh

2023-09-12 Thread Dragos Tatulea via Virtualization
The reslock is taken while refresh is called but iommu_lock is more specific to this resource. So take the iommu_lock during cvq iotlb refresh. Based on Eugenio's patch [0]. [0] https://lore.kernel.org/lkml/20230112142218.725622-4-epere...@redhat.com/ Suggested-by: Eugenio Pérez Signed-off-by:

[PATCH 01/16] vdpa: introduce dedicated descriptor group for virtqueue

2023-09-12 Thread Dragos Tatulea via Virtualization
From: Si-Wei Liu In some cases, the access to the virtqueue's descriptor area, device and driver areas (precluding indirect descriptor table in guest memory) may have to be confined to a different address space than where its buffers reside. Without loss of simplicity and generality with already

[PATCH 02/16] vhost-vdpa: introduce descriptor group backend feature

2023-09-12 Thread Dragos Tatulea via Virtualization
From: Si-Wei Liu Userspace knows if the device has dedicated descriptor group or not by checking this feature bit. It's only exposed if the vdpa driver backend implements the .get_vq_desc_group() operation callback. Userspace trying to negotiate this feature when it or the dependent _F_IOTLB_ASI

[PATCH 04/16] vdpa/mlx5: Create helper function for dma mappings

2023-09-12 Thread Dragos Tatulea via Virtualization
Necessary for upcoming cvq separation from mr allocation. Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1 + drivers/vdpa/mlx5/core/mr.c| 5 + drivers/vdpa/mlx5/net/mlx5_vnet.c | 4 ++-- 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/drivers/

[PATCH 03/16] vhost-vdpa: uAPI to get dedicated descriptor group id

2023-09-12 Thread Dragos Tatulea via Virtualization
From: Si-Wei Liu With _F_DESC_ASID backend feature, the device can now support the VHOST_VDPA_GET_VRING_DESC_GROUP ioctl, and it may expose the descriptor table (including avail and used ring) in a different group than the buffers it contains. This new uAPI will fetch the group ID of the descript

[PATCH 00/16] vdpa: Add support for vq descriptor mappings

2023-09-12 Thread Dragos Tatulea via Virtualization
This patch series adds support for vq descriptor table mappings which are used to improve vdpa live migration downtime. The improvement comes from using smaller mappings which take less time to create and destroy in hw. The first part adds the vdpa core changes from Si-Wei [0]. The second part ad

Re: [PATCH RFC v2 0/3] vdpa: dedicated descriptor table group

2023-09-11 Thread Dragos Tatulea via Virtualization
Hi Jason, On Mon, 2023-09-11 at 14:43 +0800, Jason Wang wrote: > On Sat, Sep 9, 2023 at 4:45 PM Si-Wei Liu wrote: > > > > Following patchset introduces dedicated group for descriptor table to > > reduce live migration downtime when passthrough VQ is being switched > > to shadow VQ. This RFC v2 i

[PATCH v2] vdpa/mlx5: Fix firmware error on creation of 1k VQs

2023-08-31 Thread Dragos Tatulea via Virtualization
A firmware error is triggered when configuring a 9k MTU on the PF after switching to switchdev mode and then using a vdpa device with larger (1k) rings: mlx5_cmd_out_err: CREATE_GENERAL_OBJECT(0xa00) op_mod(0xd) failed, status bad resource(0x5), syndrome (0xf6db90), err(-22) This is due to the fa

[PATCH] vdpa/mlx5: Fix firmware error on creation of 1k VQs

2023-08-29 Thread Dragos Tatulea via Virtualization
A firmware error is triggered when configuring a 9k MTU on the PF after switching to switchdev mode and then using a vdpa device with larger (1k) rings: mlx5_cmd_out_err: CREATE_GENERAL_OBJECT(0xa00) op_mod(0xd) failed, status bad resource(0x5), syndrome (0xf6db90), err(-22) This is due to the fa

[PATCH] vdpa/mlx5: Fix double release of debugfs entry

2023-08-29 Thread Dragos Tatulea via Virtualization
The error path in setup_driver deletes the debugfs entry but doesn't clear the pointer. During .dev_del the invalid pointer will be released again causing a crash. This patch fixes the issue by always clearing the debugfs entry in mlx5_vdpa_remove_debugfs. Also, stop removing the debugfs entry in

[PATCH] MAINTAINERS: Add myself as mlx5_vdpa driver

2023-08-15 Thread Dragos Tatulea via Virtualization
As Eli Cohen moved to other work, I'll be the contact point for mlx5_vdpa. Signed-off-by: Dragos Tatulea --- MAINTAINERS | 6 ++ 1 file changed, 6 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 9a5863f1b016..c9a9259f4d37 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -13555,6 +13

Re: [PATCH RFC 2/4] vdpa/mlx5: implement .reset_map driver op

2023-08-15 Thread Dragos Tatulea via Virtualization
On Mon, 2023-08-14 at 18:43 -0700, Si-Wei Liu wrote: > This patch is based on top of the "vdpa/mlx5: Fixes > for ASID handling" series [1]. > > [1] vdpa/mlx5: Fixes for ASID handling > https://lore.kernel.org/virtualization/20230802171231.11001-1-dtatu...@nvidia.com/ > > Signed-off-by: Si-Wei Liu

Re: [PATCH 1/2] vdpa/mlx5: Fix mr->initialized semantics

2023-08-14 Thread Dragos Tatulea via Virtualization
On Wed, 2023-08-09 at 09:42 +0800, Jason Wang wrote: > On Tue, Aug 8, 2023 at 3:24 PM Dragos Tatulea wrote: > > > > On Tue, 2023-08-08 at 10:57 +0800, Jason Wang wrote: > > > On Thu, Aug 3, 2023 at 7:40 PM Dragos Tatulea wrote: > > > > > > > > On Thu, 2023-08-03 at 16:03 +0800, Jason Wang wrote

Re: [PATCH 0/2] vdpa/mlx5: Fixes for ASID handling

2023-08-10 Thread Dragos Tatulea via Virtualization
On Thu, 2023-08-10 at 04:54 -0400, Michael S. Tsirkin wrote: > On Wed, Aug 02, 2023 at 08:12:16PM +0300, Dragos Tatulea wrote: > > This patch series is based on Eugenio's fix for handling CVQs in > > a different ASID [0]. > > > > The first patch is the actual fix. > > > > The next 2 patches are f

Re: [PATCH net] virtio-net: set queues after driver_ok

2023-08-08 Thread Dragos Tatulea via Virtualization
On Tue, 2023-08-08 at 23:13 -0400, Jason Wang wrote: > Commit 25266128fe16 ("virtio-net: fix race between set queues and > probe") tries to fix the race between set queues and probe by calling > _virtnet_set_queues() before DRIVER_OK is set. This violates virtio > spec. Fixing this by setting queue

Re: [PATCH net] virtio-net: fix race between set queues and probe

2023-08-08 Thread Dragos Tatulea via Virtualization
On Tue, 2023-07-25 at 03:20 -0400, Jason Wang wrote: > A race were found where set_channels could be called after registering > but before virtnet_set_queues() in virtnet_probe(). Fixing this by > moving the virtnet_set_queues() before netdevice registering. While at > it, use _virtnet_set_queues()

Re: [PATCH 1/2] vdpa/mlx5: Fix mr->initialized semantics

2023-08-08 Thread Dragos Tatulea via Virtualization
On Tue, 2023-08-08 at 10:57 +0800, Jason Wang wrote: > On Thu, Aug 3, 2023 at 7:40 PM Dragos Tatulea wrote: > > > > On Thu, 2023-08-03 at 16:03 +0800, Jason Wang wrote: > > > On Thu, Aug 3, 2023 at 1:13 AM Dragos Tatulea wrote: > > > > > > > > The mr->initialized flag is shared between the cont

[PATCH v2] vdpa/mlx5: Fix crash on shutdown for when no ndev exists

2023-08-03 Thread Dragos Tatulea via Virtualization
The ndev was accessed on shutdown without a check if it actually exists. This triggered the crash pasted below. Instead of doing the ndev check, delete the shutdown handler altogether. The irqs will be released at the parent VF level (mlx5_core). BUG: kernel NULL pointer dereference, address: 00

Re: [PATCH] vdpa/mlx5: Fix crash on shutdown for when no ndev exists

2023-08-03 Thread Dragos Tatulea via Virtualization
On Wed, 2023-08-02 at 09:56 +0200, Dragos Tatulea wrote: > On Wed, 2023-08-02 at 10:51 +0800, Jason Wang wrote: > > On Tue, Aug 1, 2023 at 4:17 PM Dragos Tatulea wrote: > > > > > > On Tue, 2023-08-01 at 11:59 +0800, Jason Wang wrote: > > > > On Mon, Jul 31, 2023 at 5:08 PM Michael S. Tsirkin > >

Re: [PATCH 1/2] vdpa/mlx5: Fix mr->initialized semantics

2023-08-03 Thread Dragos Tatulea via Virtualization
On Thu, 2023-08-03 at 16:03 +0800, Jason Wang wrote: > On Thu, Aug 3, 2023 at 1:13 AM Dragos Tatulea wrote: > > > > The mr->initialized flag is shared between the control vq and data vq > > part of the mr init/uninit. But if the control vq and data vq get placed > > in different ASIDs, it can hap

[PATCH 2/2] vdpa/mlx5: Delete control vq iotlb in destroy_mr only when necessary

2023-08-02 Thread Dragos Tatulea via Virtualization
From: Eugenio Pérez mlx5_vdpa_destroy_mr can be called from .set_map with data ASID after the control virtqueue ASID iotlb has been populated. The control vq iotlb must not be cleared, since it will not be populated again. So call the ASID aware destroy function which makes sure that the right v

  1   2   >