Re: [PATCH 00/16] vdpa: Add support for vq descriptor mappings
On Tue, 2023-09-26 at 05:22 -0700, Si-Wei Liu wrote: > > > On 9/25/2023 12:59 AM, Dragos Tatulea wrote: > > On Tue, 2023-09-12 at 16:01 +0300, Dragos Tatulea wrote: > > > This patch series adds support for vq descriptor table mappings which > > > are used to improve vdpa live migration downtime. The improvement comes > > > from using smaller mappings which take less time to create and destroy > > > in hw. > > > > > Gentle ping. > > > > Note that I will have to send a v2. The changes in mlx5_ifc.h will need to > > be > > merged first separately into the mlx5-next branch [0] and then pulled from > > there > > when the series is applied. > This separation is unnecessary, as historically the virtio emulation > portion of the update to mlx5_ifc.h often had to go through the vhost > tree. See commits 1892a3d425bf and e13cd45d352d. Especially the > additions from this series (mainly desc group mkey) have nothing to do > with any networking or NIC driver feature. > The reason for doing that is to avoid conflicts in Linus's tree on the mlx5_ifc.h file. Thanks, Dragos > -Siwei > > > > > [0] > > https://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git/log/?h=mlx5-next > > > > Thanks, > > Dragos > > > > > The first part adds the vdpa core changes from Si-Wei [0]. > > > > > > The second part adds support in mlx5_vdpa: > > > - Refactor the mr code to be able to cleanly add descriptor mappings. > > > - Add hardware descriptor mr support. > > > - Properly update iotlb for cvq during ASID switch. > > > > > > [0] > > > https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei@oracle.com > > > > > > Dragos Tatulea (13): > > > vdpa/mlx5: Create helper function for dma mappings > > > vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code > > > vdpa/mlx5: Take cvq iotlb lock during refresh > > > vdpa/mlx5: Collapse "dvq" mr add/delete functions > > > vdpa/mlx5: Rename mr destroy functions > > > vdpa/mlx5: Allow creation/deletion of any given mr struct > > > vdpa/mlx5: Move mr mutex out of mr struct > > > vdpa/mlx5: Improve mr update flow > > > vdpa/mlx5: Introduce mr for vq descriptor > > > vdpa/mlx5: Enable hw support for vq descriptor mapping > > > vdpa/mlx5: Make iotlb helper functions more generic > > > vdpa/mlx5: Update cvq iotlb mapping on ASID change > > > Cover letter: vdpa/mlx5: Add support for vq descriptor mappings > > > > > > Si-Wei Liu (3): > > > vdpa: introduce dedicated descriptor group for virtqueue > > > vhost-vdpa: introduce descriptor group backend feature > > > vhost-vdpa: uAPI to get dedicated descriptor group id > > > > > > drivers/vdpa/mlx5/core/mlx5_vdpa.h | 31 +++-- > > > drivers/vdpa/mlx5/core/mr.c | 191 - > > > drivers/vdpa/mlx5/core/resources.c | 6 +- > > > drivers/vdpa/mlx5/net/mlx5_vnet.c | 100 ++- > > > drivers/vhost/vdpa.c | 27 > > > include/linux/mlx5/mlx5_ifc.h | 8 +- > > > include/linux/mlx5/mlx5_ifc_vdpa.h | 7 +- > > > include/linux/vdpa.h | 11 ++ > > > include/uapi/linux/vhost.h | 8 ++ > > > include/uapi/linux/vhost_types.h | 5 + > > > 10 files changed, 264 insertions(+), 130 deletions(-) > > > > ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH 00/16] vdpa: Add support for vq descriptor mappings
On 9/25/2023 12:59 AM, Dragos Tatulea wrote: On Tue, 2023-09-12 at 16:01 +0300, Dragos Tatulea wrote: This patch series adds support for vq descriptor table mappings which are used to improve vdpa live migration downtime. The improvement comes from using smaller mappings which take less time to create and destroy in hw. Gentle ping. Note that I will have to send a v2. The changes in mlx5_ifc.h will need to be merged first separately into the mlx5-next branch [0] and then pulled from there when the series is applied. This separation is unnecessary, as historically the virtio emulation portion of the update to mlx5_ifc.h often had to go through the vhost tree. See commits 1892a3d425bf and e13cd45d352d. Especially the additions from this series (mainly desc group mkey) have nothing to do with any networking or NIC driver feature. -Siwei [0] https://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git/log/?h=mlx5-next Thanks, Dragos The first part adds the vdpa core changes from Si-Wei [0]. The second part adds support in mlx5_vdpa: - Refactor the mr code to be able to cleanly add descriptor mappings. - Add hardware descriptor mr support. - Properly update iotlb for cvq during ASID switch. [0] https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei@oracle.com Dragos Tatulea (13): vdpa/mlx5: Create helper function for dma mappings vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code vdpa/mlx5: Take cvq iotlb lock during refresh vdpa/mlx5: Collapse "dvq" mr add/delete functions vdpa/mlx5: Rename mr destroy functions vdpa/mlx5: Allow creation/deletion of any given mr struct vdpa/mlx5: Move mr mutex out of mr struct vdpa/mlx5: Improve mr update flow vdpa/mlx5: Introduce mr for vq descriptor vdpa/mlx5: Enable hw support for vq descriptor mapping vdpa/mlx5: Make iotlb helper functions more generic vdpa/mlx5: Update cvq iotlb mapping on ASID change Cover letter: vdpa/mlx5: Add support for vq descriptor mappings Si-Wei Liu (3): vdpa: introduce dedicated descriptor group for virtqueue vhost-vdpa: introduce descriptor group backend feature vhost-vdpa: uAPI to get dedicated descriptor group id drivers/vdpa/mlx5/core/mlx5_vdpa.h | 31 +++-- drivers/vdpa/mlx5/core/mr.c | 191 - drivers/vdpa/mlx5/core/resources.c | 6 +- drivers/vdpa/mlx5/net/mlx5_vnet.c | 100 ++- drivers/vhost/vdpa.c | 27 include/linux/mlx5/mlx5_ifc.h | 8 +- include/linux/mlx5/mlx5_ifc_vdpa.h | 7 +- include/linux/vdpa.h | 11 ++ include/uapi/linux/vhost.h | 8 ++ include/uapi/linux/vhost_types.h | 5 + 10 files changed, 264 insertions(+), 130 deletions(-) ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH 00/16] vdpa: Add support for vq descriptor mappings
On Tue, 2023-09-12 at 16:01 +0300, Dragos Tatulea wrote: > This patch series adds support for vq descriptor table mappings which > are used to improve vdpa live migration downtime. The improvement comes > from using smaller mappings which take less time to create and destroy > in hw. > Gentle ping. Note that I will have to send a v2. The changes in mlx5_ifc.h will need to be merged first separately into the mlx5-next branch [0] and then pulled from there when the series is applied. [0] https://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git/log/?h=mlx5-next Thanks, Dragos > The first part adds the vdpa core changes from Si-Wei [0]. > > The second part adds support in mlx5_vdpa: > - Refactor the mr code to be able to cleanly add descriptor mappings. > - Add hardware descriptor mr support. > - Properly update iotlb for cvq during ASID switch. > > [0] > https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei@oracle.com > > Dragos Tatulea (13): > vdpa/mlx5: Create helper function for dma mappings > vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code > vdpa/mlx5: Take cvq iotlb lock during refresh > vdpa/mlx5: Collapse "dvq" mr add/delete functions > vdpa/mlx5: Rename mr destroy functions > vdpa/mlx5: Allow creation/deletion of any given mr struct > vdpa/mlx5: Move mr mutex out of mr struct > vdpa/mlx5: Improve mr update flow > vdpa/mlx5: Introduce mr for vq descriptor > vdpa/mlx5: Enable hw support for vq descriptor mapping > vdpa/mlx5: Make iotlb helper functions more generic > vdpa/mlx5: Update cvq iotlb mapping on ASID change > Cover letter: vdpa/mlx5: Add support for vq descriptor mappings > > Si-Wei Liu (3): > vdpa: introduce dedicated descriptor group for virtqueue > vhost-vdpa: introduce descriptor group backend feature > vhost-vdpa: uAPI to get dedicated descriptor group id > > drivers/vdpa/mlx5/core/mlx5_vdpa.h | 31 +++-- > drivers/vdpa/mlx5/core/mr.c | 191 - > drivers/vdpa/mlx5/core/resources.c | 6 +- > drivers/vdpa/mlx5/net/mlx5_vnet.c | 100 ++- > drivers/vhost/vdpa.c | 27 > include/linux/mlx5/mlx5_ifc.h | 8 +- > include/linux/mlx5/mlx5_ifc_vdpa.h | 7 +- > include/linux/vdpa.h | 11 ++ > include/uapi/linux/vhost.h | 8 ++ > include/uapi/linux/vhost_types.h | 5 + > 10 files changed, 264 insertions(+), 130 deletions(-) > ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH 00/16] vdpa: Add support for vq descriptor mappings
On 9/13/2023 9:08 AM, Eugenio Perez Martin wrote: On Wed, Sep 13, 2023 at 3:03 AM Lei Yang wrote: Hi Dragos, Eugenio and Si-Wei My name is Lei Yang, a software Quality Engineer from Red Hat. And always paying attention to improving the live migration downtime issues because there are others QE asked about this problem when I share live migration status recently. Therefore I would like to test it in my environment. Before the testing I want to know if there is an expectation of downtime range based on this series of patches? In addition, QE also can help do a regression test based on this series of patches if there is a requirement. Hi Lei, Thanks for offering the testing bandwidth! I think we can only do regression tests here, as the userland part is still not sent to qemu. Right. Regression for now, even QEMU has it, to exercise the relevant feature it would need a supporting firmware that is not yet available for now. Just stay tuned. thanks for your patience, -Siwei Regards and thanks Lei On Tue, Sep 12, 2023 at 9:04 PM Dragos Tatulea wrote: This patch series adds support for vq descriptor table mappings which are used to improve vdpa live migration downtime. The improvement comes from using smaller mappings which take less time to create and destroy in hw. The first part adds the vdpa core changes from Si-Wei [0]. The second part adds support in mlx5_vdpa: - Refactor the mr code to be able to cleanly add descriptor mappings. - Add hardware descriptor mr support. - Properly update iotlb for cvq during ASID switch. [0] https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei@oracle.com Dragos Tatulea (13): vdpa/mlx5: Create helper function for dma mappings vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code vdpa/mlx5: Take cvq iotlb lock during refresh vdpa/mlx5: Collapse "dvq" mr add/delete functions vdpa/mlx5: Rename mr destroy functions vdpa/mlx5: Allow creation/deletion of any given mr struct vdpa/mlx5: Move mr mutex out of mr struct vdpa/mlx5: Improve mr update flow vdpa/mlx5: Introduce mr for vq descriptor vdpa/mlx5: Enable hw support for vq descriptor mapping vdpa/mlx5: Make iotlb helper functions more generic vdpa/mlx5: Update cvq iotlb mapping on ASID change Cover letter: vdpa/mlx5: Add support for vq descriptor mappings Si-Wei Liu (3): vdpa: introduce dedicated descriptor group for virtqueue vhost-vdpa: introduce descriptor group backend feature vhost-vdpa: uAPI to get dedicated descriptor group id drivers/vdpa/mlx5/core/mlx5_vdpa.h | 31 +++-- drivers/vdpa/mlx5/core/mr.c| 191 - drivers/vdpa/mlx5/core/resources.c | 6 +- drivers/vdpa/mlx5/net/mlx5_vnet.c | 100 ++- drivers/vhost/vdpa.c | 27 include/linux/mlx5/mlx5_ifc.h | 8 +- include/linux/mlx5/mlx5_ifc_vdpa.h | 7 +- include/linux/vdpa.h | 11 ++ include/uapi/linux/vhost.h | 8 ++ include/uapi/linux/vhost_types.h | 5 + 10 files changed, 264 insertions(+), 130 deletions(-) -- 2.41.0 ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH 00/16] vdpa: Add support for vq descriptor mappings
This patch series adds support for vq descriptor table mappings which are used to improve vdpa live migration downtime. The improvement comes from using smaller mappings which take less time to create and destroy in hw. The first part adds the vdpa core changes from Si-Wei [0]. The second part adds support in mlx5_vdpa: - Refactor the mr code to be able to cleanly add descriptor mappings. - Add hardware descriptor mr support. - Properly update iotlb for cvq during ASID switch. [0] https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei@oracle.com Dragos Tatulea (13): vdpa/mlx5: Create helper function for dma mappings vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code vdpa/mlx5: Take cvq iotlb lock during refresh vdpa/mlx5: Collapse "dvq" mr add/delete functions vdpa/mlx5: Rename mr destroy functions vdpa/mlx5: Allow creation/deletion of any given mr struct vdpa/mlx5: Move mr mutex out of mr struct vdpa/mlx5: Improve mr update flow vdpa/mlx5: Introduce mr for vq descriptor vdpa/mlx5: Enable hw support for vq descriptor mapping vdpa/mlx5: Make iotlb helper functions more generic vdpa/mlx5: Update cvq iotlb mapping on ASID change Cover letter: vdpa/mlx5: Add support for vq descriptor mappings Si-Wei Liu (3): vdpa: introduce dedicated descriptor group for virtqueue vhost-vdpa: introduce descriptor group backend feature vhost-vdpa: uAPI to get dedicated descriptor group id drivers/vdpa/mlx5/core/mlx5_vdpa.h | 31 +++-- drivers/vdpa/mlx5/core/mr.c| 191 - drivers/vdpa/mlx5/core/resources.c | 6 +- drivers/vdpa/mlx5/net/mlx5_vnet.c | 100 ++- drivers/vhost/vdpa.c | 27 include/linux/mlx5/mlx5_ifc.h | 8 +- include/linux/mlx5/mlx5_ifc_vdpa.h | 7 +- include/linux/vdpa.h | 11 ++ include/uapi/linux/vhost.h | 8 ++ include/uapi/linux/vhost_types.h | 5 + 10 files changed, 264 insertions(+), 130 deletions(-) -- 2.41.0 ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
