Re: [RFC] vdpa/mlx5: preserve CVQ vringh index

2023-10-26 Thread Si-Wei Liu
Steve, I think this is a loose end that I myself am not sure if worth fixing, copy Eugenio for his awareness. Reason is that when CVQ is in place it always has to cope with device state saving and restoration using shadowed virtqueue for a lot of cases not just migration, and that's the reas

Re: [PATCH] vhost-vdpa: fix use-after-free in _compat_vdpa_reset

2023-10-26 Thread Si-Wei Liu
On 10/25/2023 11:55 PM, Si-Wei Liu wrote: On 10/25/2023 10:26 PM, Michael S. Tsirkin wrote: On Wed, Oct 25, 2023 at 04:13:14PM -0700, Si-Wei Liu wrote: When the vhost-vdpa device is being closed, vhost_vdpa_cleanup() doesn't clean up the vqs pointer after free. This could lead t

[PATCH v5 7/7] vdpa_sim: implement .reset_map support

2023-10-26 Thread Si-Wei Liu
tself. Additionally, implement .compat_reset to cater for older userspace, which may wish to see mapping to be cleared during reset. Signed-off-by: Si-Wei Liu Tested-by: Stefano Garzarella --- drivers/vdpa/vdpa_sim/vdpa_sim.c | 52 ++-- 1 file changed, 43 insertions(

[PATCH v5 5/7] vhost-vdpa: clean iotlb map during reset for older userspace

2023-10-26 Thread Si-Wei Liu
ompared to overloading the existing .reset with flags, .compat_reset won't cause any extra burden to the implementation of every compliant driver. Signed-off-by: Si-Wei Liu Tested-by: Dragos Tatulea Tested-by: Lei Yang --- drivers/vhost/vdpa.c | 20 drivers/virtio/vir

[PATCH v5 3/7] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-10-26 Thread Si-Wei Liu
it is safe to skip this unmap & map in the case of vhost stop & start cycle. In that sense, this feature flag is actually a signal for userspace to know that the driver bug has been solved. Not offering it indicates that userspace cannot trust the kernel will retain the maps. Signed-off-by:

[PATCH v5 6/7] vdpa/mlx5: implement .reset_map driver op

2023-10-26 Thread Si-Wei Liu
d clean up memory mappings by itself. Additionally, implement .compat_reset to cater for older userspace, which may wish to see mapping to be cleared during reset. Co-developed-by: Dragos Tatulea Signed-off-by: Dragos Tatulea Signed-off-by: Si-Wei Liu --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1

[PATCH v5 4/7] vdpa: introduce .compat_reset operation callback

2023-10-26 Thread Si-Wei Liu
ot bother to implement such op, but only those drivers that are doing or used to do non-compliant map-cleaning reset will have to. Signed-off-by: Si-Wei Liu --- include/linux/vdpa.h | 13 + 1 file changed, 13 insertions(+) diff --git a/include/linux/vdpa.h b/include/linux/vdp

[PATCH v5 1/7] vdpa: introduce .reset_map operation callback

2023-10-26 Thread Si-Wei Liu
ot a MUST for every parent that implements the .dma_map or .set_map API, because device may work with DMA ops directly by implement their own to manipulate system memory mappings, so don't have to use .reset_map to achieve a simple IOMMU device model for 1:1 passthrough mapping. Signed-off-

[PATCH v5 2/7] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-26 Thread Si-Wei Liu
evice reset should not cause mapping to go away on such IOTLB model, so persistent mapping is implied across reset. Before the userspace process using vhost-vdpa is gone, give it a chance to reset iotlb back to the initial state in vhost_vdpa_cleanup(). Signed-off-by: Si-Wei Liu Acked-by: Eugenio

[PATCH v5 0/7] vdpa: decouple reset of iotlb mapping from device reset

2023-10-26 Thread Si-Wei Liu
nd-email-si-wei@oracle.com/ --- Si-Wei Liu (7): vdpa: introduce .reset_map operation callback vhost-vdpa: reset vendor specific mapping to initial state in .release vhost-vdpa: introduce IOTLB_PERSIST backend feature bit vdpa: introduce .compat_reset operation callback vhost-vdpa: clean i

Re: [PATCH] vhost-vdpa: fix use-after-free in _compat_vdpa_reset

2023-10-25 Thread Si-Wei Liu
On 10/25/2023 10:26 PM, Michael S. Tsirkin wrote: On Wed, Oct 25, 2023 at 04:13:14PM -0700, Si-Wei Liu wrote: When the vhost-vdpa device is being closed, vhost_vdpa_cleanup() doesn't clean up the vqs pointer after free. This could lead to use-after-tree when _compat_vdpa_reset() tri

Re: [PATCH v4 0/7] vdpa: decouple reset of iotlb mapping from device reset

2023-10-25 Thread Si-Wei Liu
2:41 AM, Lei Yang wrote: On Wed, Oct 25, 2023 at 1:27 AM Si-Wei Liu wrote: Hello Si-Wei Thanks a lot for testing! Please be aware that there's a follow-up fix for a potential oops in this v4 series: The first, when I did not apply this patch [1], I will also hit this patch mentioned pr

[PATCH] vhost-vdpa: fix use-after-free in _compat_vdpa_reset

2023-10-25 Thread Si-Wei Liu
all/CAPpAL=yhdqn1aztecn3mps8o4m+bl_hvy02fdpihn7dwd91...@mail.gmail.com/ Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index 9a2343c45df0..30df5c58db73 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/

Re: [PATCH v4 0/7] vdpa: decouple reset of iotlb mapping from device reset

2023-10-24 Thread Si-Wei Liu
51 PM, Lei Yang wrote: QE tested this series v4 with regression testing on real nic, there is no new regression bug. Tested-by: Lei Yang On Tue, Oct 24, 2023 at 6:02 AM Si-Wei Liu wrote: On 10/22/2023 8:51 PM, Jason Wang wrote: Hi Si-Wei: On Sat, Oct 21, 2023 at 5:28 PM Si-Wei Liu wrote

Re: [PATCH v4 5/7] vhost-vdpa: clean iotlb map during reset for older userspace

2023-10-24 Thread Si-Wei Liu
On 10/24/2023 9:21 AM, Si-Wei Liu wrote: On 10/23/2023 10:45 PM, Jason Wang wrote: On Sat, Oct 21, 2023 at 5:28 PM Si-Wei Liu wrote: Using .compat_reset op from the previous patch, the buggy .reset behaviour can be kept as-is on older userspace apps, which don't ack the IOTLB_PE

Re: [PATCH v4 5/7] vhost-vdpa: clean iotlb map during reset for older userspace

2023-10-24 Thread Si-Wei Liu
On 10/23/2023 10:45 PM, Jason Wang wrote: On Sat, Oct 21, 2023 at 5:28 PM Si-Wei Liu wrote: Using .compat_reset op from the previous patch, the buggy .reset behaviour can be kept as-is on older userspace apps, which don't ack the IOTLB_PERSIST backend feature. As this compatibility qui

[PATCH] vhost-vdpa: fix NULL pointer deref in _compat_vdpa_reset

2023-10-23 Thread Si-Wei Liu
om/ Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index 9ce40003793b..9a2343c45df0 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -232,9 +232,11 @@ static i

Re: [PATCH v4 5/7] vhost-vdpa: clean iotlb map during reset for older userspace

2023-10-23 Thread Si-Wei Liu
59 AM, Dragos Tatulea wrote: On Sat, 2023-10-21 at 02:25 -0700, Si-Wei Liu wrote: Using .compat_reset op from the previous patch, the buggy .reset behaviour can be kept as-is on older userspace apps, which don't ack the IOTLB_PERSIST backend feature. As this compatibility quirk is limite

Re: [PATCH v4 0/7] vdpa: decouple reset of iotlb mapping from device reset

2023-10-23 Thread Si-Wei Liu
On 10/22/2023 8:51 PM, Jason Wang wrote: Hi Si-Wei: On Sat, Oct 21, 2023 at 5:28 PM Si-Wei Liu wrote: In order to reduce needlessly high setup and teardown cost of iotlb mapping during live migration, it's crucial to decouple the vhost-vdpa iotlb abstraction from the virtio device

[PATCH v4 5/7] vhost-vdpa: clean iotlb map during reset for older userspace

2023-10-21 Thread Si-Wei Liu
ompared to overloading the existing .reset with flags, .compat_reset won't cause any extra burden to the implementation of every compliant driver. Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 17 + drivers/virtio/virtio_vdpa.c | 2 +- include/linux/vdpa.h

[PATCH v4 6/7] vdpa/mlx5: implement .reset_map driver op

2023-10-21 Thread Si-Wei Liu
d clean up memory mappings by itself. Additionally, implement .compat_reset to cater for older userspace, which may wish to see mapping to be cleared during reset. Co-developed-by: Dragos Tatulea Signed-off-by: Dragos Tatulea Signed-off-by: Si-Wei Liu --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1

[PATCH v4 7/7] vdpa_sim: implement .reset_map support

2023-10-21 Thread Si-Wei Liu
tself. Additionally, implement .compat_reset to cater for older userspace, which may wish to see mapping to be cleared during reset. Signed-off-by: Si-Wei Liu Tested-by: Stefano Garzarella --- drivers/vdpa/vdpa_sim/vdpa_sim.c | 52 ++-- 1 file changed, 43 insertions(

[PATCH v4 3/7] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-10-21 Thread Si-Wei Liu
it is safe to skip this unmap & map in the case of vhost stop & start cycle. In that sense, this feature flag is actually a signal for userspace to know that the driver bug has been solved. Not offering it indicates that userspace cannot trust the kernel will retain the maps. Signed-off-by:

[PATCH v4 2/7] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-21 Thread Si-Wei Liu
evice reset should not cause mapping to go away on such IOTLB model, so persistent mapping is implied across reset. Before the userspace process using vhost-vdpa is gone, give it a chance to reset iotlb back to the initial state in vhost_vdpa_cleanup(). Signed-off-by: Si-Wei Liu Acked-by: Eugenio

[PATCH v4 1/7] vdpa: introduce .reset_map operation callback

2023-10-21 Thread Si-Wei Liu
ot a MUST for every parent that implements the .dma_map or .set_map API, because device may work with DMA ops directly by implement their own to manipulate system memory mappings, so don't have to use .reset_map to achieve a simple IOMMU device model for 1:1 passthrough mapping. Signed-off-

[PATCH v4 4/7] vdpa: introduce .compat_reset operation callback

2023-10-21 Thread Si-Wei Liu
ot bother to implement such op, but only those drivers that are doing or used to do non-compliant map-cleaning reset will have to. Signed-off-by: Si-Wei Liu --- include/linux/vdpa.h | 13 + 1 file changed, 13 insertions(+) diff --git a/include/linux/vdpa.h b/include/linux/vdp

[PATCH v4 0/7] vdpa: decouple reset of iotlb mapping from device reset

2023-10-21 Thread Si-Wei Liu
RFC v3: - fix missing return due to merge error in patch #4 RFC v2: - rebased on top of the "[PATCH RFC v2 0/3] vdpa: dedicated descriptor table group" series: https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei@oracle.com/ --- Si-Wei Liu (7):

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-19 Thread Si-Wei Liu
On 10/19/2023 9:11 PM, Jason Wang wrote: On Fri, Oct 20, 2023 at 6:28 AM Si-Wei Liu wrote: On 10/19/2023 7:39 AM, Eugenio Perez Martin wrote: On Thu, Oct 19, 2023 at 10:27 AM Jason Wang wrote: On Thu, Oct 19, 2023 at 2:47 PM Si-Wei Liu wrote: On 10/18/2023 7:53 PM, Jason Wang wrote

Re: [RFC v2 PATCH] vdpa_sim: implement .reset_map support

2023-10-19 Thread Si-Wei Liu
On 10/19/2023 2:29 AM, Stefano Garzarella wrote: On Wed, Oct 18, 2023 at 04:47:48PM -0700, Si-Wei Liu wrote: On 10/18/2023 1:05 AM, Stefano Garzarella wrote: On Tue, Oct 17, 2023 at 10:11:33PM -0700, Si-Wei Liu wrote: RFC only. Not tested on vdpa-sim-blk with user virtual address. Works

Re: [PATCH vhost v4 00/16] vdpa: Add support for vq descriptor mappings

2023-10-19 Thread Si-Wei Liu
For patches 05-16: Reviewed-by: Si-Wei Liu Tested-by: Si-Wei Liu Thanks for the fixes! On 10/18/2023 10:14 AM, Dragos Tatulea wrote: This patch series adds support for vq descriptor table mappings which are used to improve vdpa live migration downtime. The improvement comes from using

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-19 Thread Si-Wei Liu
On 10/17/2023 10:27 PM, Jason Wang wrote: If we do this without a negotiation, IOTLB will not be clear but the Qemu will try to re-program the IOTLB after reset. Which will break? 1) stick the exact old behaviour with just one line of check It's not just one line of check here, the old be

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-19 Thread Si-Wei Liu
On 10/19/2023 7:39 AM, Eugenio Perez Martin wrote: On Thu, Oct 19, 2023 at 10:27 AM Jason Wang wrote: On Thu, Oct 19, 2023 at 2:47 PM Si-Wei Liu wrote: On 10/18/2023 7:53 PM, Jason Wang wrote: On Wed, Oct 18, 2023 at 4:49 PM Si-Wei Liu wrote: On 10/18/2023 12:00 AM, Jason Wang wrote

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-18 Thread Si-Wei Liu
On 10/18/2023 7:53 PM, Jason Wang wrote: On Wed, Oct 18, 2023 at 4:49 PM Si-Wei Liu wrote: On 10/18/2023 12:00 AM, Jason Wang wrote: Unfortunately, it's a must to stick to ABI. I agree it's a mess but we don't have a better choice. Or we can fail the probe if userspace d

[PATCH v3 0/4] vdpa: decouple reset of iotlb mapping from device reset

2023-10-18 Thread Si-Wei Liu
FC v2: - rebased on top of the "[PATCH RFC v2 0/3] vdpa: dedicated descriptor table group" series: https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei@oracle.com/ --- Si-Wei Liu (5): vdpa: introduce .reset_map operation callback vhost-vdpa: reset

[PATCH v3 5/5] vdpa_sim: implement .reset_map support

2023-10-18 Thread Si-Wei Liu
Add a module paramemter, iotlb_persist, to cater for older userspace which may wish to see mapping to be cleared during reset. Signed-off-by: Si-Wei Liu Tested-by: Stefano Garzarella --- drivers/vdpa/vdpa_sim/vdpa_sim.c | 58 ++-- 1 file changed, 47 insertions(+

[PATCH v3 1/5] vdpa: introduce .reset_map operation callback

2023-10-18 Thread Si-Wei Liu
have also implemented their own DMA ops, so don't have to use .reset_map to achieve a simple IOMMU device model for 1:1 passthrough mapping, either. Signed-off-by: Si-Wei Liu Acked-by: Eugenio Pérez Acked-by: Jason Wang --- include/linux/vdpa.h | 10 ++ 1 file changed, 10 insertions(+

[PATCH v3 3/5] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-10-18 Thread Si-Wei Liu
it is safe to skip this unmap & map in the case of vhost stop & start cycle. In that sense, this feature flag is actually a signal for userspace to know that the driver bug has been solved. Not offering it indicates that userspace cannot trust the kernel will retain the maps. Signed-off-by:

[PATCH v3 2/5] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-18 Thread Si-Wei Liu
evice reset does not cause mapping go away on such IOTLB model in need of persistent mapping. Before vhost-vdpa is going away, give them a chance to reset iotlb back to the initial state in vhost_vdpa_cleanup(). Signed-off-by: Si-Wei Liu Acked-by: Eugenio Pérez --- drivers/vhost/vdpa.c

[PATCH v3 4/5] vdpa/mlx5: implement .reset_map driver op

2023-10-18 Thread Si-Wei Liu
d clean up memory mappings by itself. Add a module paramemter, persist_mapping, to cater for older userspace which may wish to see mapping to be cleared during reset. Co-developed-by: Dragos Tatulea Signed-off-by: Dragos Tatulea Signed-off-by: Si-Wei Liu --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1

Re: [RFC v2 PATCH] vdpa_sim: implement .reset_map support

2023-10-18 Thread Si-Wei Liu
On 10/18/2023 1:05 AM, Stefano Garzarella wrote: On Tue, Oct 17, 2023 at 10:11:33PM -0700, Si-Wei Liu wrote: RFC only. Not tested on vdpa-sim-blk with user virtual address. Works fine with vdpa-sim-net which uses physical address to map. This patch is based on top of [1]. [1] https

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-18 Thread Si-Wei Liu
On 10/18/2023 4:14 AM, Eugenio Perez Martin wrote: On Wed, Oct 18, 2023 at 10:44 AM Si-Wei Liu wrote: On 10/17/2023 10:27 PM, Jason Wang wrote: On Wed, Oct 18, 2023 at 12:36 PM Si-Wei Liu wrote: On 10/16/2023 7:35 PM, Jason Wang wrote: On Tue, Oct 17, 2023 at 4:30 AM Si-Wei Liu wrote

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-18 Thread Si-Wei Liu
On 10/18/2023 12:00 AM, Jason Wang wrote: Unfortunately, it's a must to stick to ABI. I agree it's a mess but we don't have a better choice. Or we can fail the probe if userspace doesn't ack this feature. Antoher idea we can just do the following in vhost_vdpa reset? config->reset() if (IOTL

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-18 Thread Si-Wei Liu
On 10/17/2023 10:27 PM, Jason Wang wrote: On Wed, Oct 18, 2023 at 12:36 PM Si-Wei Liu wrote: On 10/16/2023 7:35 PM, Jason Wang wrote: On Tue, Oct 17, 2023 at 4:30 AM Si-Wei Liu wrote: On 10/16/2023 4:28 AM, Eugenio Perez Martin wrote: On Mon, Oct 16, 2023 at 8:33 AM Jason Wang wrote

Re: [RFC PATCH] vdpa_sim: implement .reset_map support

2023-10-17 Thread Si-Wei Liu
Hi Stefano, On 10/17/2023 6:44 AM, Stefano Garzarella wrote: On Fri, Oct 13, 2023 at 10:29:26AM -0700, Si-Wei Liu wrote: Hi Stefano, On 10/13/2023 2:22 AM, Stefano Garzarella wrote: Hi Si-Wei, On Fri, Oct 13, 2023 at 01:23:40AM -0700, Si-Wei Liu wrote: RFC only. Not tested on vdpa-sim-blk

[RFC v2 PATCH] vdpa_sim: implement .reset_map support

2023-10-17 Thread Si-Wei Liu
RFC only. Not tested on vdpa-sim-blk with user virtual address. Works fine with vdpa-sim-net which uses physical address to map. This patch is based on top of [1]. [1] https://lore.kernel.org/virtualization/1696928580-7520-1-git-send-email-si-wei@oracle.com/ Signed-off-by: Si-Wei Liu

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-17 Thread Si-Wei Liu
On 10/16/2023 7:35 PM, Jason Wang wrote: On Tue, Oct 17, 2023 at 4:30 AM Si-Wei Liu wrote: On 10/16/2023 4:28 AM, Eugenio Perez Martin wrote: On Mon, Oct 16, 2023 at 8:33 AM Jason Wang wrote: On Fri, Oct 13, 2023 at 3:36 PM Si-Wei Liu wrote: On 10/12/2023 8:01 PM, Jason Wang wrote

[PATCH v2 4/4] vdpa/mlx5: implement .reset_map driver op

2023-10-16 Thread Si-Wei Liu
by itself. The cvq mapping also needs to be cleared if is in the given ASID. Co-developed-by: Dragos Tatulea Signed-off-by: Dragos Tatulea Signed-off-by: Si-Wei Liu Acked-by: Eugenio Pérez --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1 + drivers/vdpa/mlx5/core/mr.c| 17

[PATCH v2 0/4] vdpa: decouple reset of iotlb mapping from device reset

2023-10-16 Thread Si-Wei Liu
eturn due to merge error in patch #4 RFC v2: - rebased on top of the "[PATCH RFC v2 0/3] vdpa: dedicated descriptor table group" series: https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei@oracle.com/ --- Si-Wei Liu (4): vdpa: introduce .reset_map ope

[PATCH v2 1/4] vdpa: introduce .reset_map operation callback

2023-10-16 Thread Si-Wei Liu
have also implemented their own DMA ops, so don't have to use .reset_map to achieve a simple IOMMU device model for 1:1 passthrough mapping, either. Signed-off-by: Si-Wei Liu Acked-by: Eugenio Pérez Acked-by: Jason Wang --- include/linux/vdpa.h | 10 ++ 1 file changed, 10 insertions(+

[PATCH v2 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-16 Thread Si-Wei Liu
evice reset does not cause mapping go away on such IOTLB model in need of persistent mapping. Before vhost-vdpa is going away, give them a chance to reset iotlb back to the initial state in vhost_vdpa_cleanup(). Signed-off-by: Si-Wei Liu Acked-by: Eugenio Pérez --- drivers/vhost/vdpa.c

[PATCH v2 3/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-10-16 Thread Si-Wei Liu
it is safe to skip this unmap & map in the case of vhost stop & start cycle. In that sense, this feature flag is actually a signal for userspace to know that the driver bug has been solved. Not offering it indicates that userspace cannot trust the kernel will retain the maps. Signed-off-by:

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-16 Thread Si-Wei Liu
On 10/16/2023 4:28 AM, Eugenio Perez Martin wrote: On Mon, Oct 16, 2023 at 8:33 AM Jason Wang wrote: On Fri, Oct 13, 2023 at 3:36 PM Si-Wei Liu wrote: On 10/12/2023 8:01 PM, Jason Wang wrote: On Tue, Oct 10, 2023 at 5:05 PM Si-Wei Liu wrote: Devices with on-chip IOMMU or vendor

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-16 Thread Si-Wei Liu
On 10/15/2023 11:32 PM, Jason Wang wrote: On Fri, Oct 13, 2023 at 3:36 PM Si-Wei Liu wrote: On 10/12/2023 8:01 PM, Jason Wang wrote: On Tue, Oct 10, 2023 at 5:05 PM Si-Wei Liu wrote: Devices with on-chip IOMMU or vendor specific IOTLB implementation may need to restore iotlb mapping to

Re: [RFC PATCH] vdpa_sim: implement .reset_map support

2023-10-13 Thread Si-Wei Liu
Hi Stefano, On 10/13/2023 2:22 AM, Stefano Garzarella wrote: Hi Si-Wei, On Fri, Oct 13, 2023 at 01:23:40AM -0700, Si-Wei Liu wrote: RFC only. Not tested on vdpa-sim-blk with user virtual address. I can test it, but what I should stress? Great, thank you! As you see, my patch moved

[RFC PATCH] vdpa_sim: implement .reset_map support

2023-10-13 Thread Si-Wei Liu
RFC only. Not tested on vdpa-sim-blk with user virtual address. Works fine with vdpa-sim-net which uses physical address to map. This patch is based on top of [1]. [1] https://lore.kernel.org/virtualization/1696928580-7520-1-git-send-email-si-wei@oracle.com/ Signed-off-by: Si-Wei Liu

Re: [PATCH 4/4] vdpa/mlx5: implement .reset_map driver op

2023-10-13 Thread Si-Wei Liu
On 10/12/2023 8:04 PM, Jason Wang wrote: On Tue, Oct 10, 2023 at 5:05 PM Si-Wei Liu wrote: Since commit 6f5312f80183 ("vdpa/mlx5: Add support for running with virtio_vdpa"), mlx5_vdpa starts with preallocate 1:1 DMA MR at device creation time. This 1:1 DMA MR will be implicitly

Re: [PATCH 1/4] vdpa: introduce .reset_map operation callback

2023-10-13 Thread Si-Wei Liu
On 10/12/2023 7:49 PM, Jason Wang wrote: On Tue, Oct 10, 2023 at 5:05 PM Si-Wei Liu wrote: Device specific IOMMU parent driver who wishes to see mapping to be decoupled from virtio or vdpa device life cycle (device reset) can use it to restore memory mapping in the device IOMMU to the

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-13 Thread Si-Wei Liu
On 10/12/2023 8:01 PM, Jason Wang wrote: On Tue, Oct 10, 2023 at 5:05 PM Si-Wei Liu wrote: Devices with on-chip IOMMU or vendor specific IOTLB implementation may need to restore iotlb mapping to the initial or default state using the .reset_map op, as it's desirable for some parent de

Re: [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-11 Thread Si-Wei Liu
On 10/11/2023 4:21 AM, Eugenio Perez Martin wrote: On Tue, Oct 10, 2023 at 11:05 AM Si-Wei Liu wrote: Devices with on-chip IOMMU or vendor specific IOTLB implementation may need to restore iotlb mapping to the initial or default state using the .reset_map op, as it's desirable for

[PATCH 4/4] vdpa/mlx5: implement .reset_map driver op

2023-10-10 Thread Si-Wei Liu
by itself. The cvq mapping also needs to be cleared if is in the given ASID. Co-developed-by: Dragos Tatulea Signed-off-by: Dragos Tatulea Signed-off-by: Si-Wei Liu --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1 + drivers/vdpa/mlx5/core/mr.c| 17 + drivers/vdp

[PATCH 3/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-10-10 Thread Si-Wei Liu
es on-chip IOMMU to perform 1:1 passthrough translation from PA to IOVA as-is to begin with, and .reset_map is the only means to turn back iotlb to the identity mapping mode after vhost-vdpa is gone. Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 15 +++ include/

[PATCH 0/4] vdpa: decouple reset of iotlb mapping from device reset

2023-10-10 Thread Si-Wei Liu
eries: https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei@oracle.com/ --- Si-Wei Liu (4): vdpa: introduce .reset_map operation callback vhost-vdpa: reset vendor specific mapping to initial state in .release vhost-vdpa: introduce IOTLB_PERSIST backend feature

[PATCH 1/4] vdpa: introduce .reset_map operation callback

2023-10-10 Thread Si-Wei Liu
: Si-Wei Liu --- include/linux/vdpa.h | 10 ++ 1 file changed, 10 insertions(+) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index d376309..26ae6ae 100644 --- a/include/linux/vdpa.h +++ b/include/linux/vdpa.h @@ -327,6 +327,15 @@ struct vdpa_map_file

[PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release

2023-10-10 Thread Si-Wei Liu
evice reset does not cause mapping go away on such IOTLB model in need of persistent mapping. Before vhost-vdpa is going away, give them a chance to reset iotlb back to the initial state in vhost_vdpa_cleanup(). Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 16 1 file ch

Re: [PATCH 00/16] vdpa: Add support for vq descriptor mappings

2023-09-26 Thread Si-Wei Liu
mr for vq descriptor   vdpa/mlx5: Enable hw support for vq descriptor mapping   vdpa/mlx5: Make iotlb helper functions more generic   vdpa/mlx5: Update cvq iotlb mapping on ASID change   Cover letter: vdpa/mlx5: Add support for vq descriptor mappings Si-Wei Liu (3):   vdpa: introduce dedicated

Re: [PATCH 00/16] vdpa: Add support for vq descriptor mappings

2023-09-14 Thread Si-Wei Liu
pdate flow vdpa/mlx5: Introduce mr for vq descriptor vdpa/mlx5: Enable hw support for vq descriptor mapping vdpa/mlx5: Make iotlb helper functions more generic vdpa/mlx5: Update cvq iotlb mapping on ASID change Cover letter: vdpa/mlx5: Add support for vq descriptor mappings Si-

Re: [PATCH RFC v3 4/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-09-14 Thread Si-Wei Liu
On 9/12/2023 12:01 AM, Jason Wang wrote: On Tue, Sep 12, 2023 at 8:28 AM Si-Wei Liu wrote: On 9/10/2023 8:52 PM, Jason Wang wrote: On Sat, Sep 9, 2023 at 9:46 PM Si-Wei Liu wrote: Userspace needs this feature flag to distinguish if vhost-vdpa iotlb in the kernel supports persistent

Re: [PATCH RFC v3 2/4] vdpa/mlx5: implement .reset_map driver op

2023-09-14 Thread Si-Wei Liu
On 9/11/2023 11:53 PM, Jason Wang wrote: On Tue, Sep 12, 2023 at 8:02 AM Si-Wei Liu wrote: On 9/10/2023 8:48 PM, Jason Wang wrote: On Sat, Sep 9, 2023 at 9:46 PM Si-Wei Liu wrote: Today, mlx5_vdpa gets started by preallocate 1:1 DMA mapping at device creation time, while this 1:1

Re: [PATCH RFC v2 1/4] vdpa: introduce .reset_map operation callback

2023-09-14 Thread Si-Wei Liu
On 9/11/2023 11:23 PM, Jason Wang wrote: On Tue, Sep 12, 2023 at 7:31 AM Si-Wei Liu wrote: Hi Jason, On 9/10/2023 8:42 PM, Jason Wang wrote: Hi Si-Wei: On Sat, Sep 9, 2023 at 9:34 PM Si-Wei Liu wrote: On-chip IOMMU parent driver could use it to restore memory mapping to the initial

Re: [PATCH] vdpa: consume device_features parameter

2023-09-11 Thread Si-Wei Liu
Thanks David, for clarifications. Now I see the patch just got posted by Shannon (thanks!) with the correct iproute2 label in the subject line. We may expect to see this land on iproute2 repo soon? Thanks! -Siwei On 9/9/2023 1:36 PM, David Ahern wrote: On 9/8/23 12:37 PM, Si-Wei Liu wrote

Re: [PATCH RFC v3 4/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-09-11 Thread Si-Wei Liu
On 9/10/2023 8:52 PM, Jason Wang wrote: On Sat, Sep 9, 2023 at 9:46 PM Si-Wei Liu wrote: Userspace needs this feature flag to distinguish if vhost-vdpa iotlb in the kernel supports persistent IOTLB mapping across device reset. As discussed, the IOTLB persists for devices with platform IOMMU

Re: [PATCH RFC v3 2/4] vdpa/mlx5: implement .reset_map driver op

2023-09-11 Thread Si-Wei Liu
On 9/10/2023 8:48 PM, Jason Wang wrote: On Sat, Sep 9, 2023 at 9:46 PM Si-Wei Liu wrote: Today, mlx5_vdpa gets started by preallocate 1:1 DMA mapping at device creation time, while this 1:1 mapping will be implicitly destroyed when the first .set_map call is invoked. Everytime when the

Re: [PATCH RFC v2 1/4] vdpa: introduce .reset_map operation callback

2023-09-11 Thread Si-Wei Liu
Hi Jason, On 9/10/2023 8:42 PM, Jason Wang wrote: Hi Si-Wei: On Sat, Sep 9, 2023 at 9:34 PM Si-Wei Liu wrote: On-chip IOMMU parent driver could use it to restore memory mapping to the initial state. As discussed before. On-chip IOMMU is the hardware details that need to be hidden by the

[PATCH RFC v3 4/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-09-09 Thread Si-Wei Liu
has the expected .reset_map support in driver Signed-off-by: Si-Wei Liu --- RFC v2 -> v3: - fix missing return due to merge error --- drivers/vhost/vdpa.c | 16 +++- include/uapi/linux/vhost_types.h | 2 ++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --gi

[PATCH RFC v3 2/4] vdpa/mlx5: implement .reset_map driver op

2023-09-09 Thread Si-Wei Liu
mapping, then the device .reset routine can run free from having to clean up memory mappings. Signed-off-by: Si-Wei Liu --- RFC v1 -> v2: - fix error path when both CVQ and DVQ fall in same asid --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1 + drivers/vdpa/mlx5/core/mr.c|

[PATCH RFC v3 3/4] vhost-vdpa: should restore 1:1 dma mapping before detaching driver

2023-09-09 Thread Si-Wei Liu
. Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 17 + 1 file changed, 17 insertions(+) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index eabac06..71fbd559 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -131,6 +131,15 @@ static struct

[PATCH RFC v3 1/4] vdpa: introduce .reset_map operation callback

2023-09-09 Thread Si-Wei Liu
On-chip IOMMU parent driver could use it to restore memory mapping to the initial state. Signed-off-by: Si-Wei Liu --- include/linux/vdpa.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index 17a4efa..daecf55 100644 --- a/include/linux

[PATCH RFC v3 0/4] vdpa: decouple reset of iotlb mapping from device reset

2023-09-09 Thread Si-Wei Liu
[PATCH RFC v2 0/3] vdpa: dedicated descriptor table group" series: https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei@oracle.com/ --- Si-Wei Liu (4): vdpa: introduce .reset_map operation callback vdpa/mlx5: implement .reset_map driver op vhost-vdpa: s

[PATCH RFC v2 4/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-09-09 Thread Si-Wei Liu
has the expected .reset_map support in driver Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 15 ++- include/uapi/linux/vhost_types.h | 2 ++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index 71fbd559

[PATCH RFC v2 1/4] vdpa: introduce .reset_map operation callback

2023-09-09 Thread Si-Wei Liu
On-chip IOMMU parent driver could use it to restore memory mapping to the initial state. Signed-off-by: Si-Wei Liu --- include/linux/vdpa.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index 17a4efa..daecf55 100644 --- a/include/linux

[PATCH RFC v2 0/4] vdpa: decouple reset of iotlb mapping from device reset

2023-09-09 Thread Si-Wei Liu
up" series: https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei@oracle.com/ --- Si-Wei Liu (4): vdpa: introduce .reset_map operation callback vdpa/mlx5: implement .reset_map driver op vhost-vdpa: should restore 1:1 dma mapping before detaching driv

[PATCH RFC v2 2/4] vdpa/mlx5: implement .reset_map driver op

2023-09-09 Thread Si-Wei Liu
mapping, then the device .reset routine can run free from having to clean up memory mappings. Signed-off-by: Si-Wei Liu --- RFC v1 -> v2: - fix error path when both CVQ and DVQ fall in same asid --- drivers/vdpa/mlx5/core/mlx5_vdpa.h | 1 + drivers/vdpa/mlx5/core/mr.c|

[PATCH RFC v2 3/4] vhost-vdpa: should restore 1:1 dma mapping before detaching driver

2023-09-09 Thread Si-Wei Liu
. Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 17 + 1 file changed, 17 insertions(+) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index eabac06..71fbd559 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -131,6 +131,15 @@ static struct

[PATCH RFC v2 3/3] vhost-vdpa: uAPI to get dedicated descriptor group id

2023-09-09 Thread Si-Wei Liu
-off-by: Si-Wei Liu Acked-by: Eugenio Pérez --- drivers/vhost/vdpa.c | 10 ++ include/uapi/linux/vhost.h | 8 2 files changed, 18 insertions(+) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index f2e5dce..eabac06 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers

[PATCH RFC v2 0/3] vdpa: dedicated descriptor table group

2023-09-09 Thread Si-Wei Liu
|1 vq_desc_group |2 |1 --- Si-Wei Liu (3): vdpa: introduce dedicated descriptor group for virtqueue vhost-vdpa: introduce descriptor group backend feature vhost-vdpa: uAPI to get dedicated descriptor group id drivers/vhost/vdpa.c | 27

[PATCH RFC v2 2/3] vhost-vdpa: introduce descriptor group backend feature

2023-09-09 Thread Si-Wei Liu
asn't been exposed will result in an error. Signed-off-by: Si-Wei Liu Acked-by: Eugenio Pérez --- RFC v1 -> v2: - add clarifications for what areas F_DESC_ASID should cover --- drivers/vhost/vdpa.c | 17 + include/uapi/linux/vhost_types.h | 5 + 2 file

[PATCH RFC v2 1/3] vdpa: introduce dedicated descriptor group for virtqueue

2023-09-09 Thread Si-Wei Liu
to the default address space, i.e. in ASID 0. QEMU's shadow virtqueue is going to utilize dedicated descriptor group to speed up map and unmap operations, yielding tremendous downtime reduction by avoiding the full and slow remap cycle in SVQ switching. Signed-off-by: Si-Wei Liu Acked-by: Eugeni

Re: [PATCH] vdpa: consume device_features parameter

2023-09-08 Thread Si-Wei Liu
On 9/7/2023 5:07 PM, David Ahern wrote: On 9/7/23 2:41 PM, Si-Wei Liu wrote: Hi David, Why this patch doesn't get picked in the last 4 months? Maybe the subject is not clear, but this is an iproute2 patch. Would it be possible to merge at your earliest convenience? PS, adding my R-b t

Re: [PATCH] vdpa: consume device_features parameter

2023-09-07 Thread Si-Wei Liu
xdeadbeef" > > Fixes: a4442ce58ebb ("vdpa: allow provisioning device features") > Signed-off-by: Allen Hubbe > Reviewed-by: Shannon Nelson Reviewed-by: Si-Wei Liu > --- >  vdpa/vdpa.c | 2 ++ >  1 file changed, 2 insertions(+) > > diff --git a/vdpa/vdpa.c

Re: [PATCH RFC 4/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-08-28 Thread Si-Wei Liu
On 8/22/2023 1:54 AM, Jason Wang wrote: On Thu, Aug 17, 2023 at 7:44 AM Si-Wei Liu wrote: On 8/15/2023 6:48 PM, Jason Wang wrote: On Wed, Aug 16, 2023 at 6:31 AM Si-Wei Liu wrote: On 8/14/2023 7:25 PM, Jason Wang wrote: On Tue, Aug 15, 2023 at 9:45 AM Si-Wei Liu wrote: Signed-off-by

Re: [PATCH RFC 1/4] vdpa: introduce .reset_map operation callback

2023-08-21 Thread Si-Wei Liu
On 8/17/2023 8:28 AM, Eugenio Perez Martin wrote: On Thu, Aug 17, 2023 at 2:05 AM Si-Wei Liu wrote: On 8/15/2023 6:55 PM, Jason Wang wrote: On Wed, Aug 16, 2023 at 3:49 AM Si-Wei Liu wrote: On 8/14/2023 7:21 PM, Jason Wang wrote: On Tue, Aug 15, 2023 at 9:46 AM Si-Wei Liu wrote

Re: [PATCH RFC 1/4] vdpa: introduce .reset_map operation callback

2023-08-16 Thread Si-Wei Liu
On 8/15/2023 6:55 PM, Jason Wang wrote: On Wed, Aug 16, 2023 at 3:49 AM Si-Wei Liu wrote: On 8/14/2023 7:21 PM, Jason Wang wrote: On Tue, Aug 15, 2023 at 9:46 AM Si-Wei Liu wrote: Signed-off-by: Si-Wei Liu --- include/linux/vdpa.h | 7 +++ 1 file changed, 7 insertions(+) diff

Re: [PATCH RFC 4/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-08-16 Thread Si-Wei Liu
On 8/15/2023 6:48 PM, Jason Wang wrote: On Wed, Aug 16, 2023 at 6:31 AM Si-Wei Liu wrote: On 8/14/2023 7:25 PM, Jason Wang wrote: On Tue, Aug 15, 2023 at 9:45 AM Si-Wei Liu wrote: Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 16 +++- include/uapi

Re: [PATCH RFC 0/3] vdpa: dedicated descriptor table group

2023-08-15 Thread Si-Wei Liu
On 8/9/2023 8:50 PM, Jason Wang wrote: On Wed, Aug 9, 2023 at 8:56 PM Si-Wei Liu wrote: Following patchset introduces dedicated group for descriptor table to reduce live migration downtime when passthrough VQ is being switched to shadow VQ. As this RFC set is to seek early feedback on the

Re: [PATCH RFC 2/3] vhost-vdpa: introduce descriptor group backend feature

2023-08-15 Thread Si-Wei Liu
On 8/9/2023 8:49 PM, Jason Wang wrote: On Wed, Aug 9, 2023 at 8:56 PM Si-Wei Liu wrote: Userspace knows if the device has dedicated descriptor group or not by checking this feature bit. It's only exposed if the vdpa driver backend implements the .get_vq_desc_group() operation cal

Re: [PATCH RFC 0/3] vdpa: dedicated descriptor table group

2023-08-15 Thread Si-Wei Liu
On 8/9/2023 7:49 AM, Eugenio Perez Martin wrote: On Wed, Aug 9, 2023 at 2:56 PM Si-Wei Liu wrote: Following patchset introduces dedicated group for descriptor table to reduce live migration downtime when passthrough VQ is being switched to shadow VQ. As this RFC set is to seek early feedback

Re: [PATCH RFC 2/4] vdpa/mlx5: implement .reset_map driver op

2023-08-15 Thread Si-Wei Liu
On 8/15/2023 1:26 AM, Dragos Tatulea wrote: On Mon, 2023-08-14 at 18:43 -0700, Si-Wei Liu wrote: This patch is based on top of the "vdpa/mlx5: Fixes for ASID handling" series [1]. [1] vdpa/mlx5: Fixes for ASID handling https://lore.kernel.org/virtualization/20230802171231.110

Re: [PATCH RFC 3/4] vhost-vdpa: should restore 1:1 dma mapping before detaching driver

2023-08-15 Thread Si-Wei Liu
On 8/14/2023 7:32 PM, Jason Wang wrote: On Tue, Aug 15, 2023 at 9:45 AM Si-Wei Liu wrote: Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 17 + 1 file changed, 17 insertions(+) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index b43e868..62b0a01 100644

Re: [PATCH RFC 4/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit

2023-08-15 Thread Si-Wei Liu
On 8/14/2023 7:25 PM, Jason Wang wrote: On Tue, Aug 15, 2023 at 9:45 AM Si-Wei Liu wrote: Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 16 +++- include/uapi/linux/vhost_types.h | 2 ++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a

Re: [PATCH RFC 1/4] vdpa: introduce .reset_map operation callback

2023-08-15 Thread Si-Wei Liu
On 8/14/2023 7:21 PM, Jason Wang wrote: On Tue, Aug 15, 2023 at 9:46 AM Si-Wei Liu wrote: Signed-off-by: Si-Wei Liu --- include/linux/vdpa.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h index db1b0ea..3a3878d 100644 --- a/include

  1   2   3   4   5   >