Re: [PATCH v6 00/21] Userspace P2PDMA with O_DIRECT NVMe devices

2022-05-16 Thread Chaitanya Kulkarni via iommu
On 4/7/22 08:46, Logan Gunthorpe wrote: > Hi, > > This patchset continues my work to add userspace P2PDMA access using > O_DIRECT NVMe devices. This posting contains some minor fixes and a > rebase onto v5.18-rc1 which contains cleanup from Christoph around > free_zone_device_page() that helps to

Re: Error when running fio against nvme-of rdma target (mlx5 driver)

2022-02-09 Thread Chaitanya Kulkarni via iommu
On 2/8/22 6:50 PM, Martin Oliveira wrote: > Hello, > > We have been hitting an error when running IO over our nvme-of setup, using > the mlx5 driver and we are wondering if anyone has seen anything similar/has > any suggestions. > > Both initiator and target are AMD EPYC 7502 machines connected

Re: [PATCH v5 01/24] ext4/xfs: add page refcount helper

2022-01-28 Thread Chaitanya Kulkarni via iommu
On 1/27/22 4:25 PM, Logan Gunthorpe wrote: > External email: Use caution opening links or attachments > > > From: Ralph Campbell > > There are several places where ZONE_DEVICE struct pages assume a reference > count == 1 means the page is idle and free. Instead of open coding this, > add a help

Re: [PATCH v5 12/24] nvme-pci: convert to using dma_map_sgtable()

2022-01-28 Thread Chaitanya Kulkarni via iommu
On 1/27/22 4:26 PM, Logan Gunthorpe wrote: > External email: Use caution opening links or attachments > > > The dma_map operations now support P2PDMA pages directly. So remove > the calls to pci_p2pdma_[un]map_sg_attrs() and replace them with calls > to dma_map_sgtable(). > > dma_map_sgtable() r

Re: [PATCH v4 11/23] nvme-pci: convert to using dma_map_sgtable()

2021-12-13 Thread Chaitanya Kulkarni via iommu
> static blk_status_t nvme_pci_setup_sgls(struct nvme_dev *dev, > - struct request *req, struct nvme_rw_command *cmd, int entries) > + struct request *req, struct nvme_rw_command *cmd) > { > struct nvme_iod *iod = blk_mq_rq_to_pdu(req); > struct dma_pool *p

Re: [PATCH v4 10/23] nvme-pci: check DMA ops when indicating support for PCI P2PDMA

2021-12-13 Thread Chaitanya Kulkarni via iommu
On 11/17/21 1:53 PM, Logan Gunthorpe wrote: > Introduce a supports_pci_p2pdma() operation in nvme_ctrl_ops to > replace the fixed NVME_F_PCI_P2PDMA flag such that the dma_map_ops > flags can be checked for PCI P2PDMA support. > > Signed-off-by: Logan Gunthorpe > --- Looks good. Reviewed-by: Cha

Re: [PATCH v4 04/23] PCI/P2PDMA: Expose pci_p2pdma_map_type()

2021-12-13 Thread Chaitanya Kulkarni via iommu
On 11/17/21 1:53 PM, Logan Gunthorpe wrote: > pci_p2pdma_map_type() will be needed by the dma-iommu map_sg > implementation because it will need to determine the mapping type > ahead of actually doing the mapping to create the actual IOMMU mapping. > > Prototypes for this helper are added to dma-m

Re: [PATCH v4 03/23] PCI/P2PDMA: Attempt to set map_type if it has not been set

2021-12-13 Thread Chaitanya Kulkarni via iommu
On 11/17/21 1:53 PM, Logan Gunthorpe wrote: > Attempt to find the mapping type for P2PDMA pages on the first > DMA map attempt if it has not been done ahead of time. > > Previously, the mapping type was expected to be calculated ahead of > time, but if pages are to come from userspace then there's

Re: [PATCH v4 02/23] lib/scatterlist: add flag for indicating P2PDMA segments in an SGL

2021-12-13 Thread Chaitanya Kulkarni via iommu
On 11/17/21 1:53 PM, Logan Gunthorpe wrote: > Make use of the third free LSB in scatterlist's page_link on 64bit systems. > > The extra bit will be used by dma_[un]map_sg_p2pdma() to determine when a > given SGL segments dma_address points to a PCI bus address. > dma_unmap_sg_p2pdma() will need to

Re: [PATCH v4 01/23] lib/scatterlist: cleanup macros into static inline functions

2021-12-13 Thread Chaitanya Kulkarni via iommu
On 11/17/21 1:53 PM, Logan Gunthorpe wrote: > Convert the sg_is_chain(), sg_is_last() and sg_chain_ptr() macros > into static inline functions. There's no reason for these to be macros > and static inline are generally preferred these days. > > Also introduce the SG_PAGE_LINK_MASK define so the P2

Re: [PATCH v3 09/20] nvme-pci: check DMA ops when indicating support for PCI P2PDMA

2021-09-30 Thread Chaitanya Kulkarni via iommu
>> >> Is this new ops only needed for the PCIe transport ? or do you have >> following patches to use this op for the other transports ? > > No, I don't think this will make sense for transports that are not based > on PCI devices. > >> If it is only needed for the PCIe then we need to find a w

Re: [PATCH v3 09/20] nvme-pci: check DMA ops when indicating support for PCI P2PDMA

2021-09-30 Thread Chaitanya Kulkarni via iommu
Logan, > diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c > index 7efb31b87f37..916750a54f60 100644 > --- a/drivers/nvme/host/core.c > +++ b/drivers/nvme/host/core.c > @@ -3771,7 +3771,8 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, > unsigned nsid, > blk_q

Re: [PATCH v3 01/20] lib/scatterlist: add flag for indicating P2PDMA segments in an SGL

2021-09-30 Thread Chaitanya Kulkarni via iommu
Logan, > +/* > + * bit 2 is the third free bit in the page_link on 64bit systems which > + * is used by dma_unmap_sg() to determine if the dma_address is a PCI > + * bus address when doing P2PDMA. > + * Note: CONFIG_PCI_P2PDMA depends on CONFIG_64BIT because of this. > + */ > + > +#ifdef CONFIG_PC

Re: [PATCH v3 01/20] lib/scatterlist: add flag for indicating P2PDMA segments in an SGL

2021-09-30 Thread Chaitanya Kulkarni via iommu
> +/** > + * sg_unmark_pci_p2pdma - Unmark the scatterlist entry for PCI p2pdma > + * @sg: SG entryScatterlist > + * > + * Description: > + * Clears the PCI P2PDMA mark > + **/ nit:- Probably want to add '.' above. > +static inline void sg_dma_unmark_pci_p2pdma(struct scatterlist