Re: [PATCH v2 08/10] nvme-pci: Add support for P2P memory in requests
On Thu, 2018-03-01 at 15:58 +, Stephen Bates wrote: > > Any plans adding the capability to nvme-rdma? Should be > > straight-forward... In theory, the use-case would be rdma backend > > fabric behind. Shouldn't be hard to test either... > > Nice idea Sagi. Yes we have been starting to look at that. Though again we > would probably want to impose the "attached to the same PCIe switch" rule > which might be less common to satisfy in initiator systems. > > Down the road I would also like to discuss the best way to use this P2P > framework to facilitate copies between NVMe namespaces (on both PCIe and > fabric attached namespaces) without having to expose the CMB up to user > space. Wasn't something like that done in the SCSI world at some point > Martin? Are you perhaps referring to the following patch series: "Copy Offload" (https://www.spinics.net/lists/linux-scsi/msg74680.html / https://lwn.net/Articles/592094/)? I will contact Martin off-list and in case he wouldn't have the time myself to revive that patch series then I will free up some time to work on this. Bart. ___ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm
Re: [PATCH v2 08/10] nvme-pci: Add support for P2P memory in requests
> Any plans adding the capability to nvme-rdma? Should be > straight-forward... In theory, the use-case would be rdma backend > fabric behind. Shouldn't be hard to test either... Nice idea Sagi. Yes we have been starting to look at that. Though again we would probably want to impose the "attached to the same PCIe switch" rule which might be less common to satisfy in initiator systems. Down the road I would also like to discuss the best way to use this P2P framework to facilitate copies between NVMe namespaces (on both PCIe and fabric attached namespaces) without having to expose the CMB up to user space. Wasn't something like that done in the SCSI world at some point Martin? Stephen ___ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm
Re: [PATCH v2 08/10] nvme-pci: Add support for P2P memory in requests
For P2P requests we must use the pci_p2pmem_[un]map_sg() functions instead of the dma_map_sg functions. With that, we can then indicate PCI_P2P support in the request queue. For this, we create an NVME_F_PCI_P2P flag which tells the core to set QUEUE_FLAG_PCI_P2P in the request queue. This looks fine to me, Reviewed-by: Sagi GrimbergAny plans adding the capability to nvme-rdma? Should be straight-forward... In theory, the use-case would be rdma backend fabric behind. Shouldn't be hard to test either... ___ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm