Re: [PATCH 3/6] RDMA/core: remove use of dma_virt_ops

2020-11-05 Thread Jason Gunthorpe
On Thu, Nov 05, 2020 at 08:42:02AM +0100, Christoph Hellwig wrote: > diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h > index 5f8fd7976034e0..661e4a22b3cb81 100644 > +++ b/include/rdma/ib_verbs.h > @@ -3950,6 +3950,8 @@ static inline int ib_req_ncomp_notif(struct ib_cq *cq, > int wc_

Re: [PATCH 4/6] PCI/P2PDMA: Remove the DMA_VIRT_OPS hacks

2020-11-05 Thread Jason Gunthorpe
On Thu, Nov 05, 2020 at 08:42:03AM +0100, Christoph Hellwig wrote: > Now that all users of dma_virt_ops are gone we can remove the workaround > for it in the PCI peer to peer code. > > Signed-off-by: Christoph Hellwig > Reviewed-by: Logan Gunthorpe > Acked-by: Bjorn Helgaas > drivers/pci/p2pdm

Re: [PATCH 1/6] RMDA/sw: don't allow drivers using dma_virt_ops on highmem configs

2020-11-05 Thread Jason Gunthorpe
On Thu, Nov 05, 2020 at 08:42:00AM +0100, Christoph Hellwig wrote: > dma_virt_ops requires that all pages have a kernel virtual address. > Introduce a INFINIBAND_VIRT_DMA Kconfig symbol that depends on !HIGHMEM > and a large enough dma_addr_t, and make all three driver depend on the > new symbol. >

Re: [PATCH 4/6] PCI/P2PDMA: Remove the DMA_VIRT_OPS hacks

2020-11-05 Thread Jason Gunthorpe
On Thu, Nov 05, 2020 at 06:08:16PM +0100, Christoph Hellwig wrote: > On Thu, Nov 05, 2020 at 10:34:18AM -0400, Jason Gunthorpe wrote: > > The check is removed here, but I didn't see a matching check added to > > the IB side? Something like: > > > > static int rdma

Re: [PATCH 4/6] PCI/P2PDMA: Remove the DMA_VIRT_OPS hacks

2020-11-05 Thread Jason Gunthorpe
On Thu, Nov 05, 2020 at 06:29:21PM +0100, Christoph Hellwig wrote: > On Thu, Nov 05, 2020 at 01:23:57PM -0400, Jason Gunthorpe wrote: > > But that depends on the calling driver doing this properly, and we > > don't expose an API to get the PCI device of the struct ib_device

Re: [PATCH 3/6] RDMA/core: remove use of dma_virt_ops

2020-11-05 Thread Jason Gunthorpe
On Thu, Nov 05, 2020 at 08:42:02AM +0100, Christoph Hellwig wrote: > @@ -1341,7 +1322,14 @@ int ib_register_device(struct ib_device *device, const > char *name, > if (ret) > return ret; > > - setup_dma_device(device, dma_device); > + /* > + * If the caller does n

Re: [PATCH 4/6] PCI/P2PDMA: Remove the DMA_VIRT_OPS hacks

2020-11-05 Thread Jason Gunthorpe
On Thu, Nov 05, 2020 at 06:43:06PM +0100, Christoph Hellwig wrote: > On Thu, Nov 05, 2020 at 01:39:30PM -0400, Jason Gunthorpe wrote: > > Hmm. This works for real devices like mlx5, but it means the three SW > > devices will also resolve to a real PCI device that is not the

Re: [PATCH 3/6] RDMA/core: remove use of dma_virt_ops

2020-11-05 Thread Jason Gunthorpe
On Thu, Nov 05, 2020 at 01:52:53PM -0400, Jason Gunthorpe wrote: > On Thu, Nov 05, 2020 at 08:42:02AM +0100, Christoph Hellwig wrote: > > @@ -1341,7 +1322,14 @@ int ib_register_device(struct ib_device *device, > > const char *name, > > if (ret) >

Re: [RFC PATCH 14/15] PCI/P2PDMA: Introduce pci_mmap_p2pmem()

2020-11-06 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 10:00:35AM -0700, Logan Gunthorpe wrote: > Introduce pci_mmap_p2pmem() which is a helper to allocate and mmap > a hunk of p2pmem into userspace. > > Signed-off-by: Logan Gunthorpe > drivers/pci/p2pdma.c | 104 + > include/linux/pc

Re: [RFC PATCH 15/15] nvme-pci: Allow mmaping the CMB in userspace

2020-11-06 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 10:00:36AM -0700, Logan Gunthorpe wrote: > Allow userspace to obtain CMB memory by mmaping the controller's > char device. The mmap call allocates and returns a hunk of CMB memory, > (the offset is ignored) so userspace does not have control over the > address within the CMB

Re: [RFC PATCH 14/15] PCI/P2PDMA: Introduce pci_mmap_p2pmem()

2020-11-06 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 10:28:00AM -0700, Logan Gunthorpe wrote: > > > On 2020-11-06 10:22 a.m., Jason Gunthorpe wrote: > > On Fri, Nov 06, 2020 at 10:00:35AM -0700, Logan Gunthorpe wrote: > >> Introduce pci_mmap_p2pmem() which is a helper to allocate and mmap &g

Re: [RFC PATCH 14/15] PCI/P2PDMA: Introduce pci_mmap_p2pmem()

2020-11-06 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 10:53:45AM -0700, Logan Gunthorpe wrote: > > > On 2020-11-06 10:42 a.m., Jason Gunthorpe wrote: > > On Fri, Nov 06, 2020 at 10:28:00AM -0700, Logan Gunthorpe wrote: > >> > >> > >> On 2020-11-06 10:22 a.m., Jason Gunthorpe wrote

Re: [RFC PATCH 14/15] PCI/P2PDMA: Introduce pci_mmap_p2pmem()

2020-11-06 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 11:20:05AM -0700, Logan Gunthorpe wrote: > > > On 2020-11-06 11:09 a.m., Jason Gunthorpe wrote: > >> Ah, hmm, yes. I guess the pages have to be hooked and returned to the > >> genalloc through free_devmap_managed_page(). > > > > Th

Re: [RFC PATCH 14/15] PCI/P2PDMA: Introduce pci_mmap_p2pmem()

2020-11-06 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 12:44:59PM -0700, Logan Gunthorpe wrote: > > > On 2020-11-06 12:30 p.m., Jason Gunthorpe wrote: > >> I certainly can't make decisions for code that isn't currently > >> upstream. > > > > The rdma drivers are all upstream,

Re: [RFC PATCH 14/15] PCI/P2PDMA: Introduce pci_mmap_p2pmem()

2020-11-06 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 01:03:26PM -0700, Logan Gunthorpe wrote: > I don't think a function like that will work for the p2pmem use case. In > order to implement proper page freeing I expect I'll need a loop around > the allocator and vm_insert_mixed()... Something roughly like: > > for (addr = vma

REGRESSION: Re: [patch V2 00/46] x86, PCI, XEN, genirq ...: Prepare for device MSI

2020-11-12 Thread Jason Gunthorpe
On Wed, Aug 26, 2020 at 01:16:28PM +0200, Thomas Gleixner wrote: > This is the second version of providing a base to support device MSI (non > PCI based) and on top of that support for IMS (Interrupt Message Storm) > based devices in a halfways architecture independent way. Hi Thomas, Our test te

Re: remove dma_virt_ops v2

2020-11-12 Thread Jason Gunthorpe
On Thu, Nov 12, 2020 at 10:40:30AM +0100, Christoph Hellwig wrote: > ping? > > On Fri, Nov 06, 2020 at 07:19:31PM +0100, Christoph Hellwig wrote: > > Hi Jason, > > > > this series switches the RDMA core to opencode the special case of > > devices bypassing the DMA mapping in the RDMA ULPs. The v

Re: remove dma_virt_ops v2

2020-11-12 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 07:19:31PM +0100, Christoph Hellwig wrote: > Hi Jason, > > this series switches the RDMA core to opencode the special case of > devices bypassing the DMA mapping in the RDMA ULPs. The virt ops > have caused a bit of trouble due to the P2P code node working with > them due

Re: remove dma_virt_ops v2

2020-11-12 Thread Jason Gunthorpe
On Thu, Nov 12, 2020 at 06:09:56PM +0100, Christoph Hellwig wrote: > On Thu, Nov 12, 2020 at 12:59:35PM -0400, Jason Gunthorpe wrote: > > RMDA/sw: Don't allow drivers using dma_virt_ops on highmem configs > > I think this one actually is something needed in 5.10 and -stab

Re: iommu/vt-d: Cure VF irqdomain hickup

2020-11-16 Thread Jason Gunthorpe
rger effort which needs quite some other > changes to the way how x86 manages PCI and MSI domains. > > Fixes: 85a8dfc57a0b ("iommm/vt-d: Store irq domain in struct device") > Reported-by: Jason Gunthorpe > Signed-off-by: Thomas Gleixner > --- > drivers/iommu/intel/d

Re: remove dma_virt_ops v2

2020-11-17 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 07:19:31PM +0100, Christoph Hellwig wrote: > Hi Jason, > > this series switches the RDMA core to opencode the special case of > devices bypassing the DMA mapping in the RDMA ULPs. The virt ops > have caused a bit of trouble due to the P2P code node working with > them due

Re: [PATCH v3 01/20] lib/scatterlist: add flag for indicating P2PDMA segments in an SGL

2021-09-28 Thread Jason Gunthorpe
assign_page - Assign a given page to an SG entry > @@ -86,13 +103,13 @@ struct sg_append_table { > **/ > static inline void sg_assign_page(struct scatterlist *sg, struct page *page) > { > - unsigned long page_link = sg->page_link & (SG_CHAIN | SG_E

Re: [PATCH v3 03/20] PCI/P2PDMA: make pci_p2pdma_map_type() non-static

2021-09-28 Thread Jason Gunthorpe
* case of IOMMUs) should be used to program the DMA engine. > + */ > + PCI_P2PDMA_MAP_THRU_HOST_BRIDGE, > +}; Reviewed-by: Jason Gunthorpe Jason ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v3 04/20] PCI/P2PDMA: introduce helpers for dma_map_sg implementations

2021-09-28 Thread Jason Gunthorpe
On Thu, Sep 16, 2021 at 05:40:44PM -0600, Logan Gunthorpe wrote: > Add pci_p2pdma_map_segment() as a helper for simple dma_map_sg() > implementations. It takes an scatterlist segment that must point to a > pci_p2pdma struct page and will map it if the mapping requires a bus > address. > > The retu

Re: [PATCH v3 05/20] dma-mapping: allow EREMOTEIO return code for P2PDMA transfers

2021-09-28 Thread Jason Gunthorpe
--- > kernel/dma/mapping.c | 16 +--- > 1 file changed, 9 insertions(+), 7 deletions(-) Reviewed-by: Jason Gunthorpe Jason ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v3 06/20] dma-direct: support PCI P2PDMA pages in dma-direct map_sg

2021-09-28 Thread Jason Gunthorpe
On Thu, Sep 16, 2021 at 05:40:46PM -0600, Logan Gunthorpe wrote: > Add PCI P2PDMA support for dma_direct_map_sg() so that it can map > PCI P2PDMA pages directly without a hack in the callers. This allows > for heterogeneous SGLs that contain both P2PDMA and regular pages. > > A P2PDMA page may hav

Re: [PATCH v3 07/20] dma-mapping: add flags to dma_map_ops to indicate PCI P2PDMA support

2021-09-28 Thread Jason Gunthorpe
> include/linux/dma-map-ops.h | 10 ++ > include/linux/dma-mapping.h | 5 + > kernel/dma/mapping.c| 18 ++ > 3 files changed, 33 insertions(+) Reviewed-by: Jason Gunthorpe Jason ___ iommu mailing list

Re: [PATCH v3 08/20] iommu/dma: support PCI P2PDMA pages in dma-iommu map_sg

2021-09-28 Thread Jason Gunthorpe
s. > > Signed-off-by: Logan Gunthorpe > --- > drivers/iommu/dma-iommu.c | 68 +++---- > 1 file changed, 61 insertions(+), 7 deletions(-) Reviewed-by: Jason Gunthorpe Jason ___ iommu mailing list iommu@

Re: [PATCH v3 11/20] RDMA/core: introduce ib_dma_pci_p2p_dma_supported()

2021-09-28 Thread Jason Gunthorpe
arget/rdma.c | 2 +- > include/rdma/ib_verbs.h| 11 +++ > 2 files changed, 12 insertions(+), 1 deletion(-) Reviewed-by: Jason Gunthorpe > +/* > + * Check if a IB device's underlying DMA mapping supports P2PDMA transfers. > + */ > +static inline bool ib_dma_p

Re: [PATCH v3 12/20] RDMA/rw: use dma_map_sgtable()

2021-09-28 Thread Jason Gunthorpe
atch to do it. But as it is, this looks fine Reviewed-by: Jason Gunthorpe Jason ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v3 13/20] PCI/P2PDMA: remove pci_p2pdma_[un]map_sg()

2021-09-28 Thread Jason Gunthorpe
2pdma.c | 65 -- > include/linux/pci-p2pdma.h | 27 > 2 files changed, 92 deletions(-) Reviewed-by: Jason Gunthorpe Good riddance :) Jason ___ iommu mailing list iommu@lists.li

Re: [PATCH v3 14/20] mm: introduce FOLL_PCI_P2PDMA to gate getting PCI P2PDMA pages

2021-09-28 Thread Jason Gunthorpe
On Thu, Sep 16, 2021 at 05:40:54PM -0600, Logan Gunthorpe wrote: > Callers that expect PCI P2PDMA pages can now set FOLL_PCI_P2PDMA to > allow obtaining P2PDMA pages. If a caller does not set this flag > and tries to map P2PDMA pages it will fail. > > This is implemented by adding a flag and a che

Re: [PATCH v3 19/20] PCI/P2PDMA: introduce pci_mmap_p2pmem()

2021-09-28 Thread Jason Gunthorpe
On Thu, Sep 16, 2021 at 05:40:59PM -0600, Logan Gunthorpe wrote: > +int pci_mmap_p2pmem(struct pci_dev *pdev, struct vm_area_struct *vma) > +{ > + struct pci_p2pdma_map *pmap; > + struct pci_p2pdma *p2pdma; > + int ret; > + > + /* prevent private mappings from being established */ >

Re: [PATCH v3 00/20] Userspace P2PDMA with O_DIRECT NVMe devices

2021-09-28 Thread Jason Gunthorpe
On Thu, Sep 16, 2021 at 05:40:40PM -0600, Logan Gunthorpe wrote: > Hi, > > This patchset continues my work to add userspace P2PDMA access using > O_DIRECT NVMe devices. My last posting[1] just included the first 13 > patches in this series, but the early P2PDMA cleanup and map_sg error > changes f

Re: [PATCH v3 19/20] PCI/P2PDMA: introduce pci_mmap_p2pmem()

2021-09-28 Thread Jason Gunthorpe
On Thu, Sep 16, 2021 at 05:40:59PM -0600, Logan Gunthorpe wrote: > +static void pci_p2pdma_unmap_mappings(void *data) > +{ > + struct pci_dev *pdev = data; > + struct pci_p2pdma *p2pdma = rcu_dereference_protected(pdev->p2pdma, 1); > + > + p2pdma->active = false; > + synchronize_rc

Re: [PATCH v3 14/20] mm: introduce FOLL_PCI_P2PDMA to gate getting PCI P2PDMA pages

2021-09-29 Thread Jason Gunthorpe
On Wed, Sep 29, 2021 at 03:34:22PM -0600, Logan Gunthorpe wrote: > > > > On 2021-09-28 1:47 p.m., Jason Gunthorpe wrote: > > On Thu, Sep 16, 2021 at 05:40:54PM -0600, Logan Gunthorpe wrote: > >> Callers that expect PCI P2PDMA pages can now set FOLL_PCI_P2PDMA to

Re: [PATCH v3 19/20] PCI/P2PDMA: introduce pci_mmap_p2pmem()

2021-09-29 Thread Jason Gunthorpe
On Wed, Sep 29, 2021 at 03:42:00PM -0600, Logan Gunthorpe wrote: > The main reason is probably this: if we don't use VM_MIXEDMAP, then we > can't set pte_devmap(). I think that is an API limitation in the fault routines.. finish_fault() should set the pte_devmap - eg by passing the PFN_DEV|PFN_

Re: [PATCH v3 00/20] Userspace P2PDMA with O_DIRECT NVMe devices

2021-09-29 Thread Jason Gunthorpe
On Wed, Sep 29, 2021 at 03:50:02PM -0600, Logan Gunthorpe wrote: > > > On 2021-09-28 2:02 p.m., Jason Gunthorpe wrote: > > On Thu, Sep 16, 2021 at 05:40:40PM -0600, Logan Gunthorpe wrote: > >> Hi, > >> > >> This patchset continues my work to add usersp

Re: [PATCH v3 19/20] PCI/P2PDMA: introduce pci_mmap_p2pmem()

2021-09-29 Thread Jason Gunthorpe
On Wed, Sep 29, 2021 at 05:27:22PM -0600, Logan Gunthorpe wrote: > > finish_fault() should set the pte_devmap - eg by passing the > > PFN_DEV|PFN_MAP somehow through the vma->vm_page_prot to mk_pte() or > > otherwise signaling do_set_pte() that it should set those PTE bits > > when it creates the

Re: [PATCH v3 00/20] Userspace P2PDMA with O_DIRECT NVMe devices

2021-09-29 Thread Jason Gunthorpe
On Wed, Sep 29, 2021 at 05:28:38PM -0600, Logan Gunthorpe wrote: > > > On 2021-09-29 5:21 p.m., Jason Gunthorpe wrote: > > On Wed, Sep 29, 2021 at 03:50:02PM -0600, Logan Gunthorpe wrote: > >> > >> > >> On 2021-09-28 2:02 p.m., Jason Gunthorpe wrote

Re: [PATCH v3 19/20] PCI/P2PDMA: introduce pci_mmap_p2pmem()

2021-09-29 Thread Jason Gunthorpe
On Wed, Sep 29, 2021 at 05:49:36PM -0600, Logan Gunthorpe wrote: > Some of this seems out of date. Pretty sure the pages are not refcounted > with vmf_insert_mixed() and vmf_insert_mixed() is currently the only way > to use VM_MIXEDMAP mappings. Hum. vmf_insert_mixed() boils down to insert_pfn()

Re: [PATCH v3 19/20] PCI/P2PDMA: introduce pci_mmap_p2pmem()

2021-10-01 Thread Jason Gunthorpe
On Wed, Sep 29, 2021 at 09:36:52PM -0300, Jason Gunthorpe wrote: > Why would DAX want to do this in the first place?? This means the > address space zap is much more important that just speeding up > destruction, it is essential for correctness since the PTEs are not > holding refcoun

Re: [PATCH v3 19/20] PCI/P2PDMA: introduce pci_mmap_p2pmem()

2021-10-01 Thread Jason Gunthorpe
On Fri, Oct 01, 2021 at 11:01:49AM -0600, Logan Gunthorpe wrote: > In device-dax, the refcount is only used to prevent the device, and > therefore the pages, from going away on device unbind. Pages cannot be > recycled, as you say, as they are mapped linearly within the device. The > address space

Re: [PATCH v3 19/20] PCI/P2PDMA: introduce pci_mmap_p2pmem()

2021-10-01 Thread Jason Gunthorpe
On Fri, Oct 01, 2021 at 02:13:14PM -0600, Logan Gunthorpe wrote: > > > On 2021-10-01 11:45 a.m., Jason Gunthorpe wrote: > >> Before the invalidation, an active flag is cleared to ensure no new > >> mappings can be created while the unmap is proceeding. > >> u

Re: [PATCH v3 19/20] PCI/P2PDMA: introduce pci_mmap_p2pmem()

2021-10-01 Thread Jason Gunthorpe
On Fri, Oct 01, 2021 at 04:22:28PM -0600, Logan Gunthorpe wrote: > > It would close this issue, however synchronize_rcu() is very slow > > (think > 1second) in some cases and thus cannot be inserted here. > > It shouldn't be *that* slow, at least not the vast majority of the > time... it seems a

Re: [PATCH v3 19/20] PCI/P2PDMA: introduce pci_mmap_p2pmem()

2021-10-04 Thread Jason Gunthorpe
On Mon, Oct 04, 2021 at 08:58:35AM +0200, Christian König wrote: > I'm not following this discussion to closely, but try to look into it from > time to time. > > Am 01.10.21 um 19:45 schrieb Jason Gunthorpe: > > On Fri, Oct 01, 2021 at 11:01:49AM -0600, Logan Gunthorpe wrot

Re: [PATCH v3 19/20] PCI/P2PDMA: introduce pci_mmap_p2pmem()

2021-10-04 Thread Jason Gunthorpe
On Mon, Oct 04, 2021 at 03:22:22PM +0200, Christian König wrote: > That use case is completely unrelated to GUP and when this doesn't work we > have quite a problem. My read is that unmap_mapping_range() guarentees the physical TLB hardware is serialized across all CPUs upon return. It also guar

Re: [PATCH v5 02/24] mm: remove extra ZONE_DEVICE struct page refcount

2022-01-28 Thread Jason Gunthorpe
On Thu, Jan 27, 2022 at 05:25:52PM -0700, Logan Gunthorpe wrote: > From: Ralph Campbell > > ZONE_DEVICE struct pages have an extra reference count that complicates the > code for put_page() and several places in the kernel that need to check the > reference count to see that a page is not being u

Re: [PATCH v5 22/24] mm: use custom page_free for P2PDMA pages

2022-01-28 Thread Jason Gunthorpe
On Thu, Jan 27, 2022 at 05:26:12PM -0700, Logan Gunthorpe wrote: > When P2PDMA pages are passed to userspace, they will need to be > reference counted properly and returned to their genalloc after their > reference count returns to 1. This is accomplished with the existing It is reference count re

Re: [RFC v2] /dev/iommu uAPI proposal

2021-07-15 Thread Jason Gunthorpe via iommu
On Thu, Jul 15, 2021 at 06:49:54AM +, Tian, Kevin wrote: > No. You are right on this case. I don't think there is a way to > differentiate one mdev from the other if they come from the > same parent and attached by the same guest process. In this > case the fault could be reported on either m

Re: [RFC v2] /dev/iommu uAPI proposal

2021-07-15 Thread Jason Gunthorpe via iommu
On Thu, Jul 15, 2021 at 06:57:57AM -0700, Raj, Ashok wrote: > On Thu, Jul 15, 2021 at 09:48:13AM -0300, Jason Gunthorpe wrote: > > On Thu, Jul 15, 2021 at 06:49:54AM +, Tian, Kevin wrote: > > > > > No. You are right on this case. I don't think there is a way to

Re: [RFC v2] /dev/iommu uAPI proposal

2021-07-15 Thread Jason Gunthorpe via iommu
On Thu, Jul 15, 2021 at 09:21:41AM -0700, Raj, Ashok wrote: > On Thu, Jul 15, 2021 at 12:23:25PM -0300, Jason Gunthorpe wrote: > > On Thu, Jul 15, 2021 at 06:57:57AM -0700, Raj, Ashok wrote: > > > On Thu, Jul 15, 2021 at 09:48:13AM -0300, Jason Gunthorpe wrote: > > > &

Re: [RFC v2] /dev/iommu uAPI proposal

2021-07-15 Thread Jason Gunthorpe via iommu
On Thu, Jul 15, 2021 at 10:48:36AM -0700, Raj, Ashok wrote: > > > Do we have any isolation requirements here? its the same process. So if > > > the > > > page-request it sent to guest and even if you report it for mdev1, after > > > the PRQ is resolved by guest, the request from mdev2 from the sa

Re: [RFC v2] /dev/iommu uAPI proposal

2021-07-15 Thread Jason Gunthorpe via iommu
On Thu, Jul 15, 2021 at 11:05:45AM -0700, Raj, Ashok wrote: > On Thu, Jul 15, 2021 at 02:53:36PM -0300, Jason Gunthorpe wrote: > > On Thu, Jul 15, 2021 at 10:48:36AM -0700, Raj, Ashok wrote: > > > > > > > Do we have any isolation requirements here? its the same

Re: [RFC v2] /dev/iommu uAPI proposal

2021-07-16 Thread Jason Gunthorpe via iommu
On Fri, Jul 16, 2021 at 01:20:15AM +, Tian, Kevin wrote: > One thought is to have vfio device driver deal with it. In this proposal > it is the vfio device driver to define the PASID virtualization policy and > report it to userspace via VFIO_DEVICE_GET_INFO. The driver understands > the restr

Re: [RFC v2] /dev/iommu uAPI proposal

2021-07-22 Thread Jason Gunthorpe via iommu
On Wed, Jul 21, 2021 at 02:13:23AM +, Tian, Kevin wrote: > > From: Shenming Lu > > Sent: Friday, July 16, 2021 8:20 PM > > > > On 2021/7/16 9:20, Tian, Kevin wrote: > > > To summarize, for vIOMMU we can work with the spec owner to > > > define a proper interface to feedback such restriction i

Re: [RFC v2] /dev/iommu uAPI proposal

2021-07-30 Thread Jason Gunthorpe via iommu
On Mon, Jul 26, 2021 at 02:50:48PM +1000, David Gibson wrote: > That said, I'm still finding the various ways a device can attach to > an ioasid pretty confusing. Here are some thoughts on some extra > concepts that might make it easier to handle [note, I haven't thought > this all the way throug

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-04 Thread Jason Gunthorpe via iommu
On Mon, Aug 02, 2021 at 02:49:44AM +, Tian, Kevin wrote: > Can you elaborate? IMO the user only cares about the label (device cookie > plus optional vPASID) which is generated by itself when doing the attaching > call, and expects this virtual label being used in various spots > (invalidatio

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-04 Thread Jason Gunthorpe via iommu
On Tue, Aug 03, 2021 at 11:58:54AM +1000, David Gibson wrote: > > I'd rather deduce the endpoint from a collection of devices than the > > other way around... > > Which I think is confusing, and in any case doesn't cover the case of > one "device" with multiple endpoints. Well they are both confu

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-05 Thread Jason Gunthorpe via iommu
On Wed, Aug 04, 2021 at 10:59:21PM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, August 4, 2021 10:05 PM > > > > On Mon, Aug 02, 2021 at 02:49:44AM +, Tian, Kevin wrote: > > > > > Can you elaborate? IMO the user only cares ab

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-06 Thread Jason Gunthorpe via iommu
On Fri, Aug 06, 2021 at 02:45:26PM +1000, David Gibson wrote: > Well, that's kind of what I'm doing. PCI currently has the notion of > "default" address space for a RID, but there's no guarantee that other > buses (or even future PCI extensions) will. The idea is that > "endpoint" means exactly

Re: [RFC][PATCH v2 00/13] iommu/arm-smmu-v3: Add NVIDIA implementation

2021-09-02 Thread Jason Gunthorpe via iommu
On Wed, Sep 01, 2021 at 06:55:55AM +, Tian, Kevin wrote: > > From: Alex Williamson > > Sent: Wednesday, September 1, 2021 12:16 AM > > > > On Mon, 30 Aug 2021 19:59:10 -0700 > > Nicolin Chen wrote: > > > > > The SMMUv3 devices implemented in the Grace SoC support NVIDIA's > > custom > > > CM

Re: [PATCH v4 14/32] iommu: introduce iommu_domain_alloc_type and the KVM type

2022-03-14 Thread Jason Gunthorpe via iommu
On Mon, Mar 14, 2022 at 03:44:33PM -0400, Matthew Rosato wrote: > s390x will introduce an additional domain type that is used for > managing IOMMU owned by KVM. Define the type here and add an > interface for allocating a specified type vs the default type. > > Signed-off-by: Matthew Rosato > --

Re: [PATCH v4 15/32] vfio: introduce KVM-owned IOMMU type

2022-03-14 Thread Jason Gunthorpe via iommu
On Mon, Mar 14, 2022 at 03:44:34PM -0400, Matthew Rosato wrote: > diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c > index 9394aa9444c1..0bec97077d61 100644 > +++ b/drivers/vfio/vfio_iommu_type1.c > @@ -77,6 +77,7 @@ struct vfio_iommu { > bool

Re: [PATCH v4 22/32] KVM: s390: pci: routines for (dis)associating zPCI devices with a KVM

2022-03-14 Thread Jason Gunthorpe via iommu
On Mon, Mar 14, 2022 at 03:44:41PM -0400, Matthew Rosato wrote: > +int kvm_s390_pci_zpci_start(struct kvm *kvm, struct zpci_dev *zdev) > +{ > + struct vfio_device *vdev; > + struct pci_dev *pdev; > + int rc; > + > + rc = kvm_s390_pci_dev_open(zdev); > + if (rc) > + r

Re: [PATCH v4 29/32] vfio-pci/zdev: add DTSM to clp group capability

2022-03-14 Thread Jason Gunthorpe via iommu
On Mon, Mar 14, 2022 at 03:44:48PM -0400, Matthew Rosato wrote: > The DTSM, or designation type supported mask, indicates what IOAT formats > are available to the guest. For an interpreted device, userspace will not > know what format(s) the IOAT assist supports, so pass it via the > capability ch

Re: [PATCH v4 15/32] vfio: introduce KVM-owned IOMMU type

2022-03-14 Thread Jason Gunthorpe via iommu
On Mon, Mar 14, 2022 at 04:50:33PM -0600, Alex Williamson wrote: > > +/* > > + * The KVM_IOMMU type implies that the hypervisor will control the mappings > > + * rather than userspace > > + */ > > +#define VFIO_KVM_IOMMU 11 > > Then why is this hosted in the type1 code that ex

Re: [PATCH v8 00/11] Fix BUG_ON in vfio_iommu_group_notifier()

2022-03-14 Thread Jason Gunthorpe via iommu
On Tue, Mar 08, 2022 at 01:44:10PM +0800, Lu Baolu wrote: > Hi folks, > > The iommu group is the minimal isolation boundary for DMA. Devices in > a group can access each other's MMIO registers via peer to peer DMA > and also need share the same I/O address space. Joerg, are we good for the coming

Re: [PATCH v2 5/8] iommu: Add PASID support for DMA mapping API users

2022-03-15 Thread Jason Gunthorpe via iommu
On Tue, Mar 15, 2022 at 11:16:41AM +, Robin Murphy wrote: > On 2022-03-15 05:07, Jacob Pan wrote: > > DMA mapping API is the de facto standard for in-kernel DMA. It operates > > on a per device/RID basis which is not PASID-aware. > > > > Some modern devices such as Intel Data Streaming Acceler

Re: [PATCH v2 3/8] iommu/vt-d: Implement device_pasid domain attach ops

2022-03-15 Thread Jason Gunthorpe via iommu
On Mon, Mar 14, 2022 at 10:07:07PM -0700, Jacob Pan wrote: > + /* > + * Each domain could have multiple devices attached with shared or per > + * device PASIDs. At the domain level, we keep track of unique PASIDs > and > + * device user count. > + * E.g. If a domain has two

Re: [PATCH v2 5/8] iommu: Add PASID support for DMA mapping API users

2022-03-15 Thread Jason Gunthorpe via iommu
On Mon, Mar 14, 2022 at 10:07:09PM -0700, Jacob Pan wrote: > DMA mapping API is the de facto standard for in-kernel DMA. It operates > on a per device/RID basis which is not PASID-aware. > > Some modern devices such as Intel Data Streaming Accelerator, PASID is > required for certain work submissi

Re: [PATCH v4 15/32] vfio: introduce KVM-owned IOMMU type

2022-03-15 Thread Jason Gunthorpe via iommu
On Tue, Mar 15, 2022 at 09:49:01AM -0400, Matthew Rosato wrote: > The rationale for splitting steps 1 and 2 are that VFIO_SET_IOMMU doesn't > have a mechanism for specifying more than the type as an arg, no? Otherwise > yes, you could specify a kvm fd at this point and it would have some other >

Re: [PATCH v4 15/32] vfio: introduce KVM-owned IOMMU type

2022-03-15 Thread Jason Gunthorpe via iommu
On Tue, Mar 15, 2022 at 09:36:08AM -0400, Matthew Rosato wrote: > > If we do try to stick this into VFIO it should probably use the > > VFIO_TYPE1_NESTING_IOMMU instead - however, we would like to delete > > that flag entirely as it was never fully implemented, was never used, > > and isn't part of

Re: [PATCH v4 29/32] vfio-pci/zdev: add DTSM to clp group capability

2022-03-15 Thread Jason Gunthorpe via iommu
On Tue, Mar 15, 2022 at 10:39:18AM -0400, Matthew Rosato wrote: > > That is something that should be modeled as a nested iommu domain. > > > > Querying the formats and any control logic for this should be on the > > iommu side not built into VFIO. > > I agree that the DTSM is really controlled by

Re: [PATCH v2 5/8] iommu: Add PASID support for DMA mapping API users

2022-03-15 Thread Jason Gunthorpe via iommu
On Tue, Mar 15, 2022 at 09:31:35AM -0700, Jacob Pan wrote: > > IMHO it is a device mis-design of IDXD to require all DMA be PASID > > tagged. Devices should be able to do DMA on their RID when the PCI > IDXD can do DMA w/ RID, the PASID requirement is only for shared WQ where > ENQCMDS is used. E

Re: [PATCH v4 15/32] vfio: introduce KVM-owned IOMMU type

2022-03-15 Thread Jason Gunthorpe via iommu
On Tue, Mar 15, 2022 at 12:04:35PM -0400, Matthew Rosato wrote: > > You can't pin/unpin in this path, there is no real way to handle error > > and ulimit stuff here, plus it is really slow. I didn't notice any of > > this in your patches, so what do you mean by 'pin' above? > > patch 18 does some

Re: [PATCH v4 15/32] vfio: introduce KVM-owned IOMMU type

2022-03-15 Thread Jason Gunthorpe via iommu
On Tue, Mar 15, 2022 at 12:29:02PM -0400, Matthew Rosato wrote: > On 3/15/22 10:38 AM, Jason Gunthorpe wrote: > > On Tue, Mar 15, 2022 at 09:49:01AM -0400, Matthew Rosato wrote: > > > > > The rationale for splitting steps 1 and 2 are that VFIO_SET_IOMMU doesn'

Re: [PATCH v2 3/8] iommu/vt-d: Implement device_pasid domain attach ops

2022-03-15 Thread Jason Gunthorpe via iommu
On Tue, Mar 15, 2022 at 03:36:20PM -0700, Jacob Pan wrote: > Hi Jason, > > On Tue, 15 Mar 2022 11:33:22 -0300, Jason Gunthorpe wrote: > > > On Mon, Mar 14, 2022 at 10:07:07PM -0700, Jacob Pan wrote: > > > + /* > > > + * Each domain could have multiple devic

Re: [PATCH v2 5/8] iommu: Add PASID support for DMA mapping API users

2022-03-15 Thread Jason Gunthorpe via iommu
On Tue, Mar 15, 2022 at 09:38:10AM -0700, Jacob Pan wrote: > > > +int iommu_enable_pasid_dma(struct device *dev, ioasid_t *pasid) > > > +{ > > > + struct iommu_domain *dom; > > > + ioasid_t id, max; > > > + int ret; > > > + > > > + dom = iommu_get_domain_for_dev(dev); > > > + if (!dom || !dom->ops

Re: [PATCH v2 5/8] iommu: Add PASID support for DMA mapping API users

2022-03-16 Thread Jason Gunthorpe via iommu
On Wed, Mar 16, 2022 at 08:41:27AM +, Tian, Kevin wrote: > 1) When the kernel wants a more scalable way of using IDXD e.g. having > multiple CPUs simultaneously submitting works in a lockless way to a > shared work queue via a new instruction (ENQCMD) which carries > PASID. IMHO the misdesig

Re: [PATCH v2 3/8] iommu/vt-d: Implement device_pasid domain attach ops

2022-03-16 Thread Jason Gunthorpe via iommu
On Wed, Mar 16, 2022 at 01:50:04PM -0700, Jacob Pan wrote: > I guess a list of (device, pasid) tuples as you suggested could work but it > will have duplicated device entries since each device could have multiple > PASIDs. right? Is assigning the same iommu_domain to multiple PASIDs of the same d

Re: [PATCH v2 3/8] iommu/vt-d: Implement device_pasid domain attach ops

2022-03-16 Thread Jason Gunthorpe via iommu
On Wed, Mar 16, 2022 at 10:23:26PM +, Luck, Tony wrote: > Kernel users (ring0) can supply any PASID when they use > the ENQCMDS instruction. Is that what you mean when you > say "real applications"? I'm not talking about ENQCMD at all I'm saying I don't see much utility to have two PASIDs as

Re: [PATCH v2 3/8] iommu/vt-d: Implement device_pasid domain attach ops

2022-03-17 Thread Jason Gunthorpe via iommu
On Wed, Mar 16, 2022 at 05:49:59PM -0700, Jacob Pan wrote: > > I would expect real applications will try to use the same PASID for > > the same IOVA map to optimize IOTLB caching. > > > > Is there a use case for that I'm missing? > > > Yes. it would be more efficient for PASID selective domain T

Re: [PATCH v4 14/32] iommu: introduce iommu_domain_alloc_type and the KVM type

2022-03-17 Thread Jason Gunthorpe via iommu
On Thu, Mar 17, 2022 at 05:47:36AM +, Tian, Kevin wrote: > > From: Robin Murphy > > Sent: Tuesday, March 15, 2022 6:49 PM > > > > On 2022-03-14 19:44, Matthew Rosato wrote: > > > s390x will introduce an additional domain type that is used for > > > managing IOMMU owned by KVM. Define the type

Re: [PATCH v4 15/32] vfio: introduce KVM-owned IOMMU type

2022-03-18 Thread Jason Gunthorpe via iommu
On Fri, Mar 18, 2022 at 07:01:19AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Tuesday, March 15, 2022 10:55 PM > > > > The first level iommu_domain has the 'type1' map and unmap and pins > > the pages. This is the 1:1 map with the GPA

Re: [PATCH v2 3/8] iommu/vt-d: Implement device_pasid domain attach ops

2022-03-18 Thread Jason Gunthorpe via iommu
On Fri, Mar 18, 2022 at 05:47:23AM +, Tian, Kevin wrote: > may remember more detail here) and we didn't hear strong interest > from customer on it. So this is just FYI for theoretical possibility and > I'm fine with even disallowing it before those issues are resolved. I didn't mean disallow,

Re: [PATCH v2 2/8] iommu: Add attach/detach_dev_pasid domain ops

2022-03-18 Thread Jason Gunthorpe via iommu
On Fri, Mar 18, 2022 at 07:52:33PM +0800, Lu Baolu wrote: > On 2022/3/15 13:07, Jacob Pan wrote: > > From: Lu Baolu > > > > An IOMMU domain represents an address space which can be attached by > > devices that perform DMA within a domain. However, for platforms with > > PASID capability the domai

Re: [PATCH v2 2/8] iommu: Add attach/detach_dev_pasid domain ops

2022-03-18 Thread Jason Gunthorpe via iommu
On Fri, Mar 18, 2022 at 08:01:04PM +0800, Lu Baolu wrote: > On 2022/3/15 19:49, Tian, Kevin wrote: > > > From: Jean-Philippe Brucker > > > Sent: Tuesday, March 15, 2022 7:27 PM > > > > > > On Mon, Mar 14, 2022 at 10:07:06PM -0700, Jacob Pan wrote: > > > > From: Lu Baolu > > > > > > > > An IOMMU d

Re: [PATCH v4 14/32] iommu: introduce iommu_domain_alloc_type and the KVM type

2022-03-18 Thread Jason Gunthorpe via iommu
On Fri, Mar 18, 2022 at 02:23:57AM +, Tian, Kevin wrote: > Yes, that is another major part work besides the iommufd work. And > it is not compatible with KVM features which rely on the dynamic > manner of EPT. Though It is a bit questionable whether it's worthy of > doing so just for saving me

[PATCH RFC 02/12] iommufd: Overview documentation

2022-03-18 Thread Jason Gunthorpe via iommu
From: Kevin Tian Add iommufd to the documentation tree. Signed-off-by: Kevin Tian Signed-off-by: Jason Gunthorpe --- Documentation/userspace-api/index.rst | 1 + Documentation/userspace-api/iommufd.rst | 224 2 files changed, 225 insertions(+) create mode 100644

[PATCH RFC 08/12] iommufd: IOCTLs for the io_pagetable

2022-03-18 Thread Jason Gunthorpe via iommu
be connected to the IOAS Along with room in the design to add non-generic features to cater to specific HW functionality. Signed-off-by: Jason Gunthorpe --- drivers/iommu/iommufd/Makefile | 1 + drivers/iommu/iommufd/ioas.c| 248 drivers/iommu

[PATCH RFC 06/12] iommufd: Algorithms for PFN storage

2022-03-18 Thread Jason Gunthorpe via iommu
possible improvements in the IOMMU API that would greatly help performance, particularly a way for the driver to map and read the pfns lists instead of working with one driver call perpage to read, and one driver call per contiguous range to store. Signed-off-by: Jason Gunthorpe --- drivers/iom

[PATCH RFC 09/12] iommufd: Add a HW pagetable object

2022-03-18 Thread Jason Gunthorpe via iommu
es to iommu_domains and to override the automatic attachment. The future HW specific interface will allow userspace to create hw_pagetable objects using iommu_domains with IOMMU driver specific parameters. This infrastructure will allow linking those domains to IOAS's and devices. Signed-off

[PATCH RFC 05/12] iommufd: PFN handling for iopt_pages

2022-03-18 Thread Jason Gunthorpe via iommu
inned page accounting. While PFNs are always stored and accessed as full PAGE_SIZE units the iommu_domain tier can store with a sub-page offset/length to support IOMMUs with a smaller IOPTE size than PAGE_SIZE. Signed-off-by: Jason Gunthorpe --- drivers/iommu/iommufd/Makefile | 3 +- dri

[PATCH RFC 04/12] kernel/user: Allow user::locked_vm to be usable for iommufd

2022-03-18 Thread Jason Gunthorpe via iommu
nd the ulimit is not supposed to be per-process. Other places (vfio, vdpa and infiniband) have used mm->pinned_vm and/or mm->locked_vm for accounting pinned pages, but this is only per-process and inconsistent with the majority of the kernel. Signed-off-by: Jason Gunthorpe --- include/linux/

[PATCH RFC 07/12] iommufd: Data structure to provide IOVA to PFN mapping

2022-03-18 Thread Jason Gunthorpe via iommu
mmu_domains attached. Upon copy the PFNs will be read out of the xarray and mapped into the iommu_domains, avoiding any pin_user_pages() overheads. Signed-off-by: Jason Gunthorpe --- drivers/iommu/iommufd/Makefile | 1 + drivers/iommu/iommufd/io_pagetable.c| 890

[PATCH RFC 00/12] IOMMUFD Generic interface

2022-03-18 Thread Jason Gunthorpe via iommu
: k...@vger.kernel.org # iommu Cc: iommu@lists.linux-foundation.org # Collaborators Cc: "Chaitanya Kulkarni" Cc: Nicolin Chen Cc: Lu Baolu Cc: Kevin Tian Cc: Yi Liu Signed-off-by: Jason Gunthorpe Jason Gunthorpe (11): interval-tree: Add a utility to iterate over spans in an interval tre

[PATCH RFC 10/12] iommufd: Add kAPI toward external drivers

2022-03-18 Thread Jason Gunthorpe via iommu
/rw operations on that iommufd_device. Signed-off-by: Jason Gunthorpe --- drivers/iommu/iommufd/Makefile | 1 + drivers/iommu/iommufd/device.c | 274 drivers/iommu/iommufd/iommufd_private.h | 4 + drivers/iommu/iommufd/main.c| 3

[PATCH RFC 01/12] interval-tree: Add a utility to iterate over spans in an interval tree

2022-03-18 Thread Jason Gunthorpe via iommu
mufd patches have several algorithms for two of its overlapping node interval trees that are significantly simplified with this kind of iteration primitive. As it seems generally useful, put it into lib/. Signed-off-by: Jason Gunthorpe --- include/linux/interval_

[PATCH RFC 03/12] iommufd: File descriptor, context, kconfig and makefiles

2022-03-18 Thread Jason Gunthorpe via iommu
by handle' operation. Signed-off-by: Yi Liu Signed-off-by: Jason Gunthorpe --- .../userspace-api/ioctl/ioctl-number.rst | 1 + MAINTAINERS | 10 + drivers/iommu/Kconfig | 1 + drivers/iommu/Makefile|

<    1   2   3   4   5   6   7   8   9   10   >