RE: [PATCH 01/11] iommu: Add device dma ownership set/release interfaces

2021-11-17 Thread Tian, Kevin
> From: Jason Gunthorpe > Sent: Tuesday, November 16, 2021 9:46 PM > > On Tue, Nov 16, 2021 at 09:57:30AM +0800, Lu Baolu wrote: > > Hi Christoph, > > > > On 11/15/21 9:14 PM, Christoph Hellwig wrote: > > > On Mon, Nov 15, 2021 at 10:05:42AM +0800, Lu Baolu wrote: > > > > +enum iommu_dma_owner {

Re: [PATCH 01/11] iommu: Add device dma ownership set/release interfaces

2021-11-17 Thread Lu Baolu
On 11/17/21 9:35 PM, Jason Gunthorpe wrote: On Wed, Nov 17, 2021 at 01:22:19PM +0800, Lu Baolu wrote: Hi Jason, On 11/16/21 9:46 PM, Jason Gunthorpe wrote: On Tue, Nov 16, 2021 at 09:57:30AM +0800, Lu Baolu wrote: Hi Christoph, On 11/15/21 9:14 PM, Christoph Hellwig wrote: On Mon, Nov 15,

Re: [PATCH 1/2] dt-bindings: Add Arm SMMUv3 PMCG binding

2021-11-17 Thread Rob Herring
On Tue, Nov 16, 2021 at 5:52 AM Jean-Philippe Brucker wrote: > > Add binding for the Arm SMMUv3 PMU. Each node represents a PMCG, and is > placed as a sibling node of the SMMU. Although the PMCGs registers may > be within the SMMU MMIO region, they are separate devices, and there can > be

[PATCH v4 11/23] nvme-pci: convert to using dma_map_sgtable()

2021-11-17 Thread Logan Gunthorpe
The dma_map operations now support P2PDMA pages directly. So remove the calls to pci_p2pdma_[un]map_sg_attrs() and replace them with calls to dma_map_sgtable(). dma_map_sgtable() returns more complete error codes than dma_map_sg() and allows differentiating EREMOTEIO errors in case an unsupported

[PATCH v4 06/23] dma-mapping: allow EREMOTEIO return code for P2PDMA transfers

2021-11-17 Thread Logan Gunthorpe
Add EREMOTEIO error return to dma_map_sgtable() which will be used by .map_sg() implementations that detect P2PDMA pages that the underlying DMA device cannot access. Signed-off-by: Logan Gunthorpe Reviewed-by: Jason Gunthorpe --- kernel/dma/mapping.c | 4 +++- 1 file changed, 3 insertions(+),

[PATCH v4 09/23] iommu/dma: support PCI P2PDMA pages in dma-iommu map_sg

2021-11-17 Thread Logan Gunthorpe
When a PCI P2PDMA page is seen, set the IOVA length of the segment to zero so that it is not mapped into the IOVA. Then, in finalise_sg(), apply the appropriate bus address to the segment. The IOVA is not created if the scatterlist only consists of P2PDMA pages. A P2PDMA page may have three

[PATCH v4 00/23] Userspace P2PDMA with O_DIRECT NVMe devices

2021-11-17 Thread Logan Gunthorpe
Hi, This patchset continues my work to add userspace P2PDMA access using O_DIRECT NVMe devices. This posting fixes a lot of issues that were addresed in the last posting, which is here[1]. The patchset enables userspace P2PDMA by allowing userspace to mmap() allocated chunks of the CMB. The

[PATCH v4 10/23] nvme-pci: check DMA ops when indicating support for PCI P2PDMA

2021-11-17 Thread Logan Gunthorpe
Introduce a supports_pci_p2pdma() operation in nvme_ctrl_ops to replace the fixed NVME_F_PCI_P2PDMA flag such that the dma_map_ops flags can be checked for PCI P2PDMA support. Signed-off-by: Logan Gunthorpe --- drivers/nvme/host/core.c | 3 ++- drivers/nvme/host/nvme.h | 2 +-

[PATCH v4 21/23] mm: use custom page_free for P2PDMA pages

2021-11-17 Thread Logan Gunthorpe
When P2PDMA pages are passed to userspace, they will need to be reference counted properly and returned to their genalloc after their reference count returns to 1. This is accomplished with the existing DEV_PAGEMAP_OPS and the .page_free() operation. Change CONFIG_P2PDMA to select

[PATCH v4 07/23] dma-direct: support PCI P2PDMA pages in dma-direct map_sg

2021-11-17 Thread Logan Gunthorpe
Add PCI P2PDMA support for dma_direct_map_sg() so that it can map PCI P2PDMA pages directly without a hack in the callers. This allows for heterogeneous SGLs that contain both P2PDMA and regular pages. A P2PDMA page may have three possible outcomes when being mapped: 1) If the data path between

[PATCH v4 05/23] PCI/P2PDMA: Introduce helpers for dma_map_sg implementations

2021-11-17 Thread Logan Gunthorpe
Add pci_p2pdma_map_segment() as a helper for simple dma_map_sg() implementations. It takes an scatterlist segment that must point to a pci_p2pdma struct page and will map it if the mapping requires a bus address. The return value indicates whether the mapping required a bus address or whether the

[PATCH v4 20/23] block: set FOLL_PCI_P2PDMA in bio_map_user_iov()

2021-11-17 Thread Logan Gunthorpe
When a bio's queue supports PCI P2PDMA, set FOLL_PCI_P2PDMA for iov_iter_get_pages_flags(). This allows PCI P2PDMA pages to be passed from userspace and enables the NVMe passthru requests to use P2PDMA pages. Signed-off-by: Logan Gunthorpe --- block/blk-map.c | 7 ++- 1 file changed, 6

[PATCH v4 16/23] iov_iter: introduce iov_iter_get_pages_[alloc_]flags()

2021-11-17 Thread Logan Gunthorpe
Add iov_iter_get_pages_flags() and iov_iter_get_pages_alloc_flags() which take a flags argument that is passed to get_user_pages_fast(). This is so that FOLL_PCI_P2PDMA can be passed when appropriate. Signed-off-by: Logan Gunthorpe --- include/linux/uio.h | 21 +

[PATCH v4 15/23] mm: introduce FOLL_PCI_P2PDMA to gate getting PCI P2PDMA pages

2021-11-17 Thread Logan Gunthorpe
Callers that expect PCI P2PDMA pages can now set FOLL_PCI_P2PDMA to allow obtaining P2PDMA pages. If a caller does not set this flag and tries to map P2PDMA pages it will fail. This is implemented by checking failing if PCI p2pdma pages are found when FOLL_PCI_P2PDMA is set. This is only done if

[PATCH v4 22/23] PCI/P2PDMA: Introduce pci_mmap_p2pmem()

2021-11-17 Thread Logan Gunthorpe
Introduce pci_mmap_p2pmem() which is a helper to allocate and mmap a hunk of p2pmem into userspace. Pages are allocated from the genalloc in bulk and their reference count incremented. They are returned to the genalloc when the page is put. The VMA does not take a reference to the pages when

[PATCH v4 13/23] RDMA/rw: drop pci_p2pdma_[un]map_sg()

2021-11-17 Thread Logan Gunthorpe
dma_map_sg() now supports the use of P2PDMA pages so pci_p2pdma_map_sg() is no longer necessary and may be dropped. This means the rdma_rw_[un]map_sg() helpers are no longer necessary. Remove it all. Signed-off-by: Logan Gunthorpe Reviewed-by: Jason Gunthorpe --- drivers/infiniband/core/rw.c |

[PATCH v4 04/23] PCI/P2PDMA: Expose pci_p2pdma_map_type()

2021-11-17 Thread Logan Gunthorpe
pci_p2pdma_map_type() will be needed by the dma-iommu map_sg implementation because it will need to determine the mapping type ahead of actually doing the mapping to create the actual IOMMU mapping. Prototypes for this helper are added to dma-map-ops.h as they are only useful to dma map

[PATCH v4 19/23] block: set FOLL_PCI_P2PDMA in __bio_iov_iter_get_pages()

2021-11-17 Thread Logan Gunthorpe
When a bio's queue supports PCI P2PDMA, set FOLL_PCI_P2PDMA for iov_iter_get_pages_flags(). This allows PCI P2PDMA pages to be passed from userspace and enables the O_DIRECT path in iomap based filesystems and direct to block devices. Signed-off-by: Logan Gunthorpe --- block/bio.c | 8 +++-

[PATCH v4 03/23] PCI/P2PDMA: Attempt to set map_type if it has not been set

2021-11-17 Thread Logan Gunthorpe
Attempt to find the mapping type for P2PDMA pages on the first DMA map attempt if it has not been done ahead of time. Previously, the mapping type was expected to be calculated ahead of time, but if pages are to come from userspace then there's no way to ensure the path was checked ahead of time.

[PATCH v4 02/23] lib/scatterlist: add flag for indicating P2PDMA segments in an SGL

2021-11-17 Thread Logan Gunthorpe
Make use of the third free LSB in scatterlist's page_link on 64bit systems. The extra bit will be used by dma_[un]map_sg_p2pdma() to determine when a given SGL segments dma_address points to a PCI bus address. dma_unmap_sg_p2pdma() will need to perform different cleanup when a segment is marked

[PATCH v4 14/23] PCI/P2PDMA: Remove pci_p2pdma_[un]map_sg()

2021-11-17 Thread Logan Gunthorpe
This interface is superseded by support in dma_map_sg() which now supports heterogeneous scatterlists. There are no longer any users, so remove it. Signed-off-by: Logan Gunthorpe Acked-by: Bjorn Helgaas Reviewed-by: Jason Gunthorpe Reviewed-by: Max Gurtovoy --- drivers/pci/p2pdma.c |

[PATCH v4 17/23] block: add check when merging zone device pages

2021-11-17 Thread Logan Gunthorpe
Consecutive zone device pages should not be merged into the same sgl or bvec segment with other types of pages or if they belong to different pgmaps. Otherwise getting the pgmap of a given segment is not possible without scanning the entire segment. This helper returns true either if both pages

[PATCH v4 08/23] dma-mapping: add flags to dma_map_ops to indicate PCI P2PDMA support

2021-11-17 Thread Logan Gunthorpe
Add a flags member to the dma_map_ops structure with one flag to indicate support for PCI P2PDMA. Also, add a helper to check if a device supports PCI P2PDMA. Signed-off-by: Logan Gunthorpe Reviewed-by: Jason Gunthorpe --- include/linux/dma-map-ops.h | 10 ++

[PATCH v4 01/23] lib/scatterlist: cleanup macros into static inline functions

2021-11-17 Thread Logan Gunthorpe
Convert the sg_is_chain(), sg_is_last() and sg_chain_ptr() macros into static inline functions. There's no reason for these to be macros and static inline are generally preferred these days. Also introduce the SG_PAGE_LINK_MASK define so the P2PDMA work, which is adding another bit to this mask,

[PATCH v4 23/23] nvme-pci: allow mmaping the CMB in userspace

2021-11-17 Thread Logan Gunthorpe
Allow userspace to obtain CMB memory by mmaping the controller's char device. The mmap call allocates and returns a hunk of CMB memory, (the offset is ignored) so userspace does not have control over the address within the CMB. A VMA allocated in this way will only be usable by drivers that set

[PATCH v4 12/23] RDMA/core: introduce ib_dma_pci_p2p_dma_supported()

2021-11-17 Thread Logan Gunthorpe
Introduce the helper function ib_dma_pci_p2p_dma_supported() to check if a given ib_device can be used in P2PDMA transfers. This ensures the ib_device is not using virt_dma and also that the underlying dma_device supports P2PDMA. Use the new helper in nvme-rdma to replace the existing check for

[PATCH v4 18/23] lib/scatterlist: add check when merging zone device pages

2021-11-17 Thread Logan Gunthorpe
Consecutive zone device pages should not be merged into the same sgl or bvec segment with other types of pages or if they belong to different pgmaps. Otherwise getting the pgmap of a given segment is not possible without scanning the entire segment. This helper returns true either if both pages

[PATCH 4/4] iommu: dart: Support t6000 variant

2021-11-17 Thread Sven Peter via iommu
The M1 Pro/Max SoCs come with a new variant of DART which supports a larger physical address space with a slightly different PTE format. Pass through the correct paddr address space size to the io-pgtable code which will take care of the rest. Signed-off-by: Sven Peter ---

[PATCH 1/4] dt-bindings: iommu: dart: add t6000 compatible

2021-11-17 Thread Sven Peter via iommu
The M1 Max/Pro SoCs come with a new DART variant that is incompatible with the previous one. Add a new compatible for those. Signed-off-by: Sven Peter --- Documentation/devicetree/bindings/iommu/apple,dart.yaml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git

[PATCH 3/4] iommu/io-pgtable: Add DART PTE support for t6000

2021-11-17 Thread Sven Peter via iommu
The DARTs present in the M1 Pro/Max SoC support a 42bit physical address space by shifting the paddr and extending its mask inside the PTE. Signed-off-by: Sven Peter --- drivers/iommu/io-pgtable-arm.c | 30 +- include/linux/io-pgtable.h | 2 ++ 2 files changed,

[PATCH 2/4] iommu/io-pgtable: Add DART subpage protection support

2021-11-17 Thread Sven Peter via iommu
DART allows to only expose a subpage to the device. While this is an optional feature on the M1 DARTs the new ones present on the Pro/Max models require this field in every PTE. Signed-off-by: Sven Peter --- drivers/iommu/io-pgtable-arm.c | 10 ++ 1 file changed, 10 insertions(+) diff

[PATCH 0/4] iommu: M1 Pro/Max DART support

2021-11-17 Thread Sven Peter via iommu
Hi, This is a fairly brief series to add support for the DARTs present in the M1 Pro/Max. They have two differences that make them incompatible with those in the M1: - the physical addresses are shifted left by 4 bits and and have 2 more bits inside the PTE entries - the subpage

Re: [PATCH] iommu/io-pgtable-arm: Fix table descriptor paddr formatting

2021-11-17 Thread Robin Murphy
On 2021-11-17 17:12, Hector Martin wrote: Table descriptors were being installed without properly formatting the address using paddr_to_iopte, which does not match up with the iopte_deref in __arm_lpae_map. This is incorrect for the LPAE pte format, as it does not handle the high bits properly.

[PATCH] iommu/io-pgtable-arm: Fix table descriptor paddr formatting

2021-11-17 Thread Hector Martin
Table descriptors were being installed without properly formatting the address using paddr_to_iopte, which does not match up with the iopte_deref in __arm_lpae_map. This is incorrect for the LPAE pte format, as it does not handle the high bits properly. This was found on Apple T6000 DARTs, which

[PATCH v2 3/3] perf/smmuv3: Synthesize IIDR from CoreSight ID registers

2021-11-17 Thread Jean-Philippe Brucker
From: Robin Murphy The SMMU_PMCG_IIDR register was not present in older revisions of the Arm SMMUv3 spec. On Arm Ltd. implementations, the IIDR value consists of fields from several PIDR registers, allowing us to present a standardized identifier to userspace. Signed-off-by: Robin Murphy

[PATCH v2 2/3] perf/smmuv3: Add devicetree support

2021-11-17 Thread Jean-Philippe Brucker
Add device-tree support to the SMMUv3 PMCG driver. Signed-off-by: Jay Chen Signed-off-by: Jean-Philippe Brucker Signed-off-by: Robin Murphy --- drivers/perf/arm_smmuv3_pmu.c | 11 ++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/drivers/perf/arm_smmuv3_pmu.c

[PATCH v2 1/3] dt-bindings: Add Arm SMMUv3 PMCG binding

2021-11-17 Thread Jean-Philippe Brucker
Add binding for the Arm SMMUv3 PMU. Each node represents a PMCG, and is placed as a sibling node of the SMMU. Although the PMCGs registers may be within the SMMU MMIO region, they are separate devices, and there can be multiple PMCG devices for each SMMU (for example one for the TCU and one for

[PATCH v2 0/3] perf/smmuv3: Support devicetree

2021-11-17 Thread Jean-Philippe Brucker
Add devicetree binding for the SMMUv3 PMU, called Performance Monitoring Counter Group (PMCG) in the spec. Each SMMUv3 implementation can have multiple independent PMCGs, for example one for the Translation Control Unit (TCU) and one per Translation Buffer Unit (TBU). Since v1 [1]: * Fixed

Re: [PATCH 3/5] hyperv/IOMMU: Enable swiotlb bounce buffer for Isolation VM

2021-11-17 Thread Tianyu Lan
On 11/17/2021 6:01 PM, Christoph Hellwig wrote: This doesn't really have much to do with normal DMA mapping, so why does this direct through the dma ops? According to the previous discussion, dma_alloc_noncontigous() and dma_vmap_noncontiguous() may be used to handle the noncontigous memory

Re: [PATCH 01/11] iommu: Add device dma ownership set/release interfaces

2021-11-17 Thread Jason Gunthorpe via iommu
On Wed, Nov 17, 2021 at 01:22:19PM +0800, Lu Baolu wrote: > Hi Jason, > > On 11/16/21 9:46 PM, Jason Gunthorpe wrote: > > On Tue, Nov 16, 2021 at 09:57:30AM +0800, Lu Baolu wrote: > > > Hi Christoph, > > > > > > On 11/15/21 9:14 PM, Christoph Hellwig wrote: > > > > On Mon, Nov 15, 2021 at

Re: [PATCH 1/5] x86/Swiotlb: Add Swiotlb bounce buffer remap function for HV IVM

2021-11-17 Thread Tianyu Lan
Hi Christoph: Thanks for your review. On 11/17/2021 5:59 PM, Christoph Hellwig wrote: The subject is wrong, nothing x86-specific here. Please use "swiotlb: " as the prefix OK. Will update. Thanks. + * @vaddr: The vaddr of the swiotlb memory pool. The swiotlb + *

Re: [PATCH 3/5] hyperv/IOMMU: Enable swiotlb bounce buffer for Isolation VM

2021-11-17 Thread Tianyu Lan
On 11/17/2021 3:12 AM, Borislav Petkov wrote: What you should do, instead, is add an isol. VM specific hv_cc_platform_has() just like amd_cc_platform_has() and handle the cc_attrs there for your platform, like return false for CC_ATTR_GUEST_MEM_ENCRYPT and then you won't need to add that hv_*

Re: [RFC 1/2] arm64: Add support for system cache memory type

2021-11-17 Thread Christoph Hellwig
On Tue, Nov 16, 2021 at 03:15:59PM -0800, Georgi Djakov wrote: > include/linux/dma-map-ops.h | 8 Your forgot to CC the maintainer. Also don't try to ever hide DMA core changes in arch specific patches ever again. ___ iommu mailing list

Re: [PATCH 3/5] hyperv/IOMMU: Enable swiotlb bounce buffer for Isolation VM

2021-11-17 Thread Christoph Hellwig
This doesn't really have much to do with normal DMA mapping, so why does this direct through the dma ops? ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH 1/5] x86/Swiotlb: Add Swiotlb bounce buffer remap function for HV IVM

2021-11-17 Thread Christoph Hellwig
The subject is wrong, nothing x86-specific here. Please use "swiotlb: " as the prefix > + * @vaddr: The vaddr of the swiotlb memory pool. The swiotlb > + * memory pool may be remapped in the memory encrypted case and > store Please avoid the overly long line. > + /* > + *