On 05/03/18 03:28 PM, Bjorn Helgaas wrote:
If you put the #ifdef right here, then it's easier to read because we
can see that "oh, this is a special and uncommon case that I can
probably ignore".
Makes sense. I'll do that.
Thanks,
Logan
On 05/03/18 05:49 PM, Oliver wrote:
It's in arch/powerpc/kernel/io.c as _memcpy_toio() and it has two full barriers!
Awesome!
Our io.h indicates that our iomem accessors are designed to provide x86ish
strong ordering of accesses to MMIO space. The git log indicates
arch/powerpc/kernel/io.c ha
k and feedback from Christoph
Hellwig.
Signed-off-by: Christoph Hellwig
Signed-off-by: Logan Gunthorpe
---
drivers/pci/Kconfig| 16 ++
drivers/pci/Makefile | 1 +
drivers/pci/p2pdma.c | 679 +
include/linux/memremap.h
Add a sysfs group to display statistics about P2P memory that is
registered in each PCI device.
Attributes in the group display the total amount of P2P memory, the
amount available and whether it is published or not.
Signed-off-by: Logan Gunthorpe
---
Documentation/ABI/testing/sysfs-bus-pci
-off-by: Logan Gunthorpe
Reviewed-by: Sagi Grimberg
---
drivers/nvme/host/core.c | 4
drivers/nvme/host/nvme.h | 1 +
drivers/nvme/host/pci.c | 19 +++
3 files changed, 20 insertions(+), 4 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index
all IO transfers. And if
a port is using P2P memory, adding new namespaces that are not supported
by that memory will fail.
Logan Gunthorpe (11):
PCI/P2PDMA: Support peer-to-peer memory
PCI/P2PDMA: Add sysfs group to display p2pmem stats
PCI/P2PDMA: Add PCI p2pmem dma mappings to adjust the
Introduce a quirk to use CMB-like memory on older devices that have
an exposed BAR but do not advertise support for using CMBLOC and
CMBSIZE.
We'd like to use some of these older cards to test P2P memory.
Signed-off-by: Logan Gunthorpe
Reviewed-by: Sagi Grimberg
---
drivers/nvme/host/n
converted to restructured text
at this time.
Signed-off-by: Logan Gunthorpe
Cc: Jonathan Corbet
---
Documentation/PCI/index.rst | 14
Documentation/PCI/p2pdma.rst | 164 +++
Documentation/index.rst | 3 +-
3 files changed, 180 insertions(+), 1
The DMA address used when mapping PCI P2P memory must be the PCI bus
address. Thus, introduce pci_p2pmem_[un]map_sg() to map the correct
addresses when using P2P memory.
For this, we assume that an SGL passed to these functions contain all
P2P memory or no P2P memory.
Signed-off-by: Logan
Register the CMB buffer as p2pmem and use the appropriate allocation
functions to create and destroy the IO SQ.
If the CMB supports WDS and RDS, publish it for use as P2P memory
by other devices.
Signed-off-by: Logan Gunthorpe
---
drivers/nvme/host/pci.c | 75
-off-by: Logan Gunthorpe
---
drivers/pci/Kconfig| 9 +
drivers/pci/p2pdma.c | 44
drivers/pci/pci.c | 6 ++
include/linux/pci-p2pdma.h | 5 +
4 files changed, 64 insertions(+)
diff --git a/drivers/pci/Kconfig b
I_P2P
flag set.
Signed-off-by: Logan Gunthorpe
Reviewed-by: Sagi Grimberg
---
block/blk-core.c | 3 +++
include/linux/blk_types.h | 18 +-
include/linux/blkdev.h| 3 +++
3 files changed, 23 insertions(+), 1 deletion(-)
diff --git a/block/blk-core.c b/block/blk-c
is P2P the entire SGL should be P2P.
Signed-off-by: Logan Gunthorpe
---
drivers/infiniband/core/rw.c | 13 +++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/core/rw.c b/drivers/infiniband/core/rw.c
index c8963e91f92a..f495e8a7f8ac 100644
--- a
f-by: Christoph Hellwig
Signed-off-by: Logan Gunthorpe
---
drivers/nvme/target/configfs.c | 67 ++
drivers/nvme/target/core.c | 106 -
drivers/nvme/target/io-cmd.c | 3 ++
drivers/nvme/target/nvmet.h| 12 +
driv
On 3/12/2018 1:41 PM, Jonathan Corbet wrote:
This all seems good, but...could we consider moving this documentation to
driver-api/PCI as it's converted to RST? That would keep it together with
similar materials and bring a bit more coherence to Documentation/ as a
whole.
Yup, I'll change this
On 12/03/18 09:28 PM, Sinan Kaya wrote:
On 3/12/2018 3:35 PM, Logan Gunthorpe wrote:
Regarding the switch business, It is amazing how much trouble you went into
limit this functionality into very specific hardware.
I thought that we reached to an agreement that code would not impose
any
On 12/03/18 09:28 PM, Sinan Kaya wrote:
Maybe, dev parameter should also be struct pci_dev so that you can get rid of
all to_pci_dev() calls in this code including find_parent_pci_dev() function.
No, this was mentioned in v2. find_parent_pci_dev is necessary because
the calling drivers aren'
On 13/03/18 11:49 AM, Sinan Kaya wrote:
And there's also the ACS problem which means if you want to use P2P on the root
ports you'll have to disable ACS on the entire system. (Or preferably, the
IOMMU groups need to get more sophisticated to allow for dynamic changes).
Do you think you can
On 13/03/18 01:10 PM, Sinan Kaya wrote:
> I was thinking of this for the pci_p2pdma_add_client() case for the
> parent pointer.
>
> +struct pci_p2pdma_client {
> + struct list_head list;
> + struct pci_dev *client;
> + struct pci_dev *provider;
> +};
Yeah, that structure only exists
On 13/03/18 01:53 PM, Sinan Kaya wrote:
> I agree disabling globally would be bad. Somebody can always say I have
> ten switches on my system. I want to do peer-to-peer on one switch only. Now,
> this change weakened security for the other switches that I had no intention
> with doing P2P.
>
> I
On 13/03/18 03:22 PM, Sinan Kaya wrote:
> It sounds like you have very tight hardware expectations for this to work
> at this moment. You also don't want to generalize this code for others and
> address the shortcomings.
No, that's the way the community has pushed this work. Our original work
wa
On 13/03/18 04:29 PM, Sinan Kaya wrote:
> If hardware doesn't support it, blacklisting should have been the right
> path and I still think that you should remove all switch business from the
> code.
> I did not hear enough justification for having a switch requirement
> for P2P.
I disagree.
>
On 13/03/18 05:08 PM, Bjorn Helgaas wrote:
> On Tue, Mar 13, 2018 at 10:31:55PM +, Stephen Bates wrote:
> If it *is* necessary because Root Ports and devices below them behave
> differently than in conventional PCI, I think you should include a
> reference to the relevant section of the spec
On 13/03/18 05:19 PM, Sinan Kaya wrote:
> It is still a switch it can move packets but, maybe it can move data at
> 100kbps speed.
As Stephen pointed out, it's a requirement of the PCIe spec that a
switch supports P2P. If you want to sell a switch that does P2P with bad
performance then that's
On 13/03/18 08:56 PM, Bjorn Helgaas wrote:
> I assume you want to exclude Root Ports because of multi-function
> devices and the "route to self" error. I was hoping for a reference
> to that so I could learn more about it.
I haven't been able to find where in the spec it forbids route to self.
On 14/03/18 06:16 AM, David Laight wrote:
> That surprises me (unless I missed something last time I read the spec).
> While P2P writes are relatively easy to handle, reads and any other TLP that
> require acks are a completely different proposition.
> There are no additional fields that can be s
On 14/03/18 12:51 PM, Bjorn Helgaas wrote:
> You are focused on PCIe systems, and in those systems, most topologies
> do have an upstream switch, which means two upstream bridges. I'm
> trying to remove that assumption because I don't think there's a
> requirement for it in the spec. Enforcing
On 14/03/18 01:28 PM, Dan Williams wrote:
> P2P over PCI/PCI-X is quite common in devices like raid controllers.
> It would be useful if those configurations were not left behind so
> that Linux could feasibly deploy offload code to a controller in the
> PCI domain.
Thanks for the note. Neat. In
On 21/03/18 03:27 AM, Christoph Hellwig wrote:
>> + const char *page, size_t count)
>> +{
>> +struct nvmet_port *port = to_nvmet_port(item);
>> +struct device *dev;
>> +struct pci_dev *p2p_dev = NULL;
>> +bool use_p2pmem;
>> +
>> +switch (page[0])
On 23/03/18 03:50 PM, Bjorn Helgaas wrote:
> Popping way up the stack, my original point was that I'm trying to
> remove restrictions on what devices can participate in peer-to-peer
> DMA. I think it's fairly clear that in conventional PCI, any devices
> in the same PCI hierarchy, i.e., below th
On 24/03/18 09:28 AM, Stephen Bates wrote:
> 1. There is no requirement for a single function to support internal DMAs but
> in the case of NVMe we do have a protocol specific way for a NVMe function to
> indicate it supports via the CMB BAR. Other protocols may also have such
> methods but I
On 26/03/18 08:01 AM, Bjorn Helgaas wrote:
> On Mon, Mar 26, 2018 at 12:11:38PM +0100, Jonathan Cameron wrote:
>> On Tue, 13 Mar 2018 10:43:55 -0600
>> Logan Gunthorpe wrote:
>>> It turns out that root ports that support P2P are far less common than
>>> anyon
On 26/03/18 10:41 AM, Jason Gunthorpe wrote:
> On Mon, Mar 26, 2018 at 12:11:38PM +0100, Jonathan Cameron wrote:
>> On Tue, 13 Mar 2018 10:43:55 -0600
>> Logan Gunthorpe wrote:
>>
>>> On 12/03/18 09:28 PM, Sinan Kaya wrote:
>>>> On 3/12/2018 3:35 PM,
On 26/03/18 01:35 PM, Jason Gunthorpe wrote:
> I think this is another case of the HW can do it but the SW support is
> missing. IOMMU configuration and maybe firmware too, for instance.
Nope, not sure how you can make this leap. We've been specifically told
that peer-to-peer PCIe DMA is not sup
On 27/03/18 02:47 AM, Jonathan Cameron wrote:
> I'll see if I can get our PCI SIG people to follow this through and see if
> it is just an omission or as Bjorn suggested, there is some reason we
> aren't thinking of that makes it hard.
That would be great! Thanks!
Logan
MA to io memory, memory that is unmappable, or memory that
doesn't have struct page backings.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
include/linux/pfn_t.h | 37 -
include/linux/scatterlist.h | 31 ++
r other's
opinions.
This series is based on v4.12-rc2 and a git tree is available here:
https://github.com/sbates130272/linux-p2pmem.git io_pfn_t
Thanks for your time,
Logan
[1] https://lkml.org/lkml/2017/4/25/738
Logan Gunthorpe (16):
dmaengine: ste_dma40, imx-dma: Cleanup scatterlist
Add a default ioremap function which was not provided in all
circumstances. (Only when CONFIG_PCI and CONFIG_TILEGX was set).
I have designs to use them in scatterlist.c where they'd likely never
be called with this architecture, but it is needed to compile.
Signed-off-by: Logan Gunt
ut it does need
to compile.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
arch/um/include/asm/io.h | 17 +
1 file changed, 17 insertions(+)
create mode 100644 arch/um/include/asm/io.h
diff --git a/arch/um/include/asm/io.h b/arch/um/include/asm/io.h
new file m
This is a layering violation so we replace it with calls to
sg_page. This is a prep patch for replacing page_link and this
is one of the very few uses outside of scatterlist.h.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
drivers/staging/ccree/ssi_buffer_mgr.c | 17
Going straight from pfn_t to physical address is cheaper and
avoids the potential BUG_ON in bvec_page for unmappable memory.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
include/linux/bio.h | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/include
affect is that, on 32 bit systems, the sgl entry will be
32 bits larger seeing pfn_t is always 64 bits and the unsigned long it
replaced was 32 bits. However, it should still fit the same
SG_CHUNK_SIZE entries into a single page so this probably isn't a huge
issue.
Signed-off-by: Logan Gunt
Introduce two accessor functions for bv_page: bvec_page to return the
page and bvec_set_page. A follow on patch will mechanically convert all the
individual uses within the kernel.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
include/linux/bvec.h | 10 ++
1 file
d-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
include/linux/dma-mapping.h | 9 +++--
include/linux/scatterlist.h | 16
2 files changed, 23 insertions(+), 2 deletions(-)
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 4f3eece..5e
We introduce bio_add_pfn which is the same as bio_add_page but expects a
pfn_t instead of a page pointer. bio_add_page is then converted to be
an inline call to bio_add_pfn.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
block/bio.c | 16
include
This is a layering violation so we replace the uses with calls to
sg_page(). This is a prep patch for replacing page_link and this
is one of the very few uses outside of scatterlist.h.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
samples/kfifo/dma-example.c | 8
1
actually a struct.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
block/blk-integrity.c | 4 ++--
block/blk-merge.c | 6 +++---
include/linux/bvec.h | 13 +
3 files changed, 14 insertions(+), 9 deletions(-)
diff --git a/block/blk-integrity.c b/block/blk
len;
expression offset;
@@
-sg_set_page(sg, bvec_page(&bv), len, offset);
+sg_set_pfn(sg, bv.bv_pfn, len, offset);
@@
expression sg;
expression bv;
expression len;
expression offset;
@@
-sg_set_page(sg, bvec_page(bv), len, offset);
+sg_set_pfn(sg, bv->bv_pfn, len, offset);
Signed-off-by
18593/
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
drivers/nvme/host/pci.c | 3 ++-
drivers/nvme/target/Kconfig | 12
drivers/nvme/target/io-cmd.c | 2 +-
drivers/nvme/target/rdma.c | 29 +
4 files changed, 40 insertions(
Two dma engine drivers directly accesses page_link assuming knowledge
that should be contained only in scatterlist.h.
We replace these with calls to sg_chain and sg_assign_page.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
drivers/dma/imx-dma.c | 7 ++-
drivers/dma
support IO memory by simply calling
the appropriate memcpy when required.
Signed-off-by: Logan Gunthorpe
Signed-off-by: Stephen Bates
---
include/linux/scatterlist.h | 3 +++
lib/scatterlist.c | 35 +--
2 files changed, 32 insertions(+), 6 deletions(-)
diff
each major number in the correct
order, regardless of where they are stored in the hash table.
In order to do this, we introduce BLKDEV_MAJOR_MAX as an artificial
limit (chosen to be 512). It will then print all devices in major
order number from 0 to the maximum.
Signed-off-by: Logan Gunthorpe
Introduce a quirk to use CMB-like memory on older devices that have
an exposed BAR but do not advertise support for using CMBLOC and
CMBSIZE.
We'd like to use some of these older cards to test P2P memory.
Signed-off-by: Logan Gunthorpe
Reviewed-by: Sagi Grimberg
---
drivers/nvme/host/n
Add a new directory in the driver API guide for PCI specific
documentation.
This is in preparation for adding a new PCI P2P DMA driver writers
guide which will go in this directory.
Signed-off-by: Logan Gunthorpe
Cc: Jonathan Corbet
Cc: Mauro Carvalho Chehab
Cc: Greg Kroah-Hartman
Cc: Vinod
SDs (Intel, Seagate, Samsung) and p2pdma devices (Eideticom,
Microsemi, Chelsio and Everspin) using switches from both Microsemi
and Broadcomm.
Logan Gunthorpe (14):
PCI/P2PDMA: Support peer-to-peer memory
PCI/P2PDMA: Add sysfs group to display p2pmem stats
PCI/P2PDMA: Add PCI p2pmem dma mappi
, devm_memremap_pages() allocates regular memory without
side effects that's accessible without the iomem accessors.
Signed-off-by: Logan Gunthorpe
---
drivers/nvme/host/pci.c | 75 +++--
1 file changed, 41 insertions(+), 34 deletions(-)
diff --git a/dr
is P2P the entire SGL should be P2P.
Signed-off-by: Logan Gunthorpe
Reviewed-by: Christoph Hellwig
---
drivers/infiniband/core/rw.c | 13 +++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/core/rw.c b/drivers/infiniband/core/rw.c
index c8963e91f92a
called once.
Signed-off-by: Logan Gunthorpe
Cc: Christoph Hellwig
Cc: Sagi Grimberg
---
drivers/nvme/target/rdma.c | 20
1 file changed, 12 insertions(+), 8 deletions(-)
diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
index 52e0c5d579a7..f7a3459d618f
he initial code]
Signed-off-by: Christoph Hellwig
Signed-off-by: Logan Gunthorpe
---
drivers/nvme/target/configfs.c | 67 ++
drivers/nvme/target/core.c | 127 -
drivers/nvme/target/io-cmd.c | 3 +
drivers/nvme/target/nvmet.
transactions.
Signed-off-by: Logan Gunthorpe
---
drivers/pci/Kconfig| 9 +
drivers/pci/p2pdma.c | 45 ++---
drivers/pci/pci.c | 6 ++
include/linux/pci-p2pdma.h | 5 +
4 files changed, 50 insertions(+), 15 deletions
converted to restructured text
at this time.
Signed-off-by: Logan Gunthorpe
Cc: Jonathan Corbet
---
Documentation/PCI/index.rst | 14 +++
Documentation/driver-api/pci/index.rst | 1 +
Documentation/driver-api/pci/p2pdma.rst | 166
Documentation
Add a sysfs group to display statistics about P2P memory that is
registered in each PCI device.
Attributes in the group display the total amount of P2P memory, the
amount available and whether it is published or not.
Signed-off-by: Logan Gunthorpe
---
Documentation/ABI/testing/sysfs-bus-pci
I_P2P
flag set.
Signed-off-by: Logan Gunthorpe
Reviewed-by: Sagi Grimberg
Reviewed-by: Christoph Hellwig
---
block/blk-core.c | 3 +++
include/linux/blk_types.h | 18 +-
include/linux/blkdev.h| 3 +++
3 files changed, 23 insertions(+), 1 deletion(-)
diff --git a/
-off-by: Logan Gunthorpe
Reviewed-by: Sagi Grimberg
Reviewed-by: Christoph Hellwig
---
drivers/nvme/host/core.c | 4
drivers/nvme/host/nvme.h | 1 +
drivers/nvme/host/pci.c | 19 +++
3 files changed, 20 insertions(+), 4 deletions(-)
diff --git a/drivers/nvme/host/core.c b
drivers. The presently
unused 'sq' argument in the alloc function will be necessary to
decide whether to use peer-to-peer memory and obtain the correct
provider to allocate the memory.
Signed-off-by: Logan Gunthorpe
Cc: Christoph Hellwig
Cc: Sagi Grimberg
---
drivers/nvme/target/co
The DMA address used when mapping PCI P2P memory must be the PCI bus
address. Thus, introduce pci_p2pmem_[un]map_sg() to map the correct
addresses when using P2P memory.
For this, we assume that an SGL passed to these functions contain all
P2P memory or no P2P memory.
Signed-off-by: Logan
ience. The PCI-SIG may be exploring adding a new
capability bit to advertise whether this is possible for future
hardware.
This commit includes significant rework and feedback from Christoph
Hellwig.
Signed-off-by: Christoph Hellwig
Signed-off-by: Logan Gunthorpe
---
drivers/pci/Kconfig
Hi Christian,
On 5/2/2018 5:51 AM, Christian König wrote:
it would be rather nice to have if you could separate out the functions
to detect if peer2peer is possible between two devices.
This would essentially be pci_p2pdma_distance() in the existing
patchset. It returns the sum of the distanc
On 03/05/18 03:05 AM, Christian König wrote:
> Ok, I'm still missing the big picture here. First question is what is
> the P2PDMA provider?
Well there's some pretty good documentation in the patchset for this,
but in short, a provider is a device that provides some kind of P2P
resource (ie. BAR
On 03/05/18 11:29 AM, Christian König wrote:
> Ok, that is the point where I'm stuck. Why do we need that in one
> function call in the PCIe subsystem?
>
> The problem at least with GPUs is that we seriously don't have that
> information here, cause the PCI subsystem might not be aware of all
On 04/05/18 08:27 AM, Christian König wrote:
> Are you sure that this is more convenient? At least on first glance it
> feels overly complicated.
>
> I mean what's the difference between the two approaches?
>
> sum = pci_p2pdma_distance(target, [A, B, C, target]);
>
> and
>
> sum =
Thanks for the review. I'll apply all of these for the changes for next
version of the set.
>> +/*
>> + * If a device is behind a switch, we try to find the upstream bridge
>> + * port of the switch. This requires two calls to pci_upstream_bridge():
>> + * one for the upstream port on the switch, o
> How do you envison merging this? There's a big chunk in drivers/pci, but
> really no opportunity for conflicts there, and there's significant stuff in
> block and nvme that I don't really want to merge.
>
> If Alex is OK with the ACS situation, I can ack the PCI parts and you could
> merge it
On 08/05/18 01:17 AM, Christian König wrote:
> AMD APUs mandatory need the ACS flag set for the GPU integrated in the
> CPU when IOMMU is enabled or otherwise you will break SVM.
Well, given that the current set only disables ACS bits on bridges
(previous versions were only on switches) this sh
On 08/05/18 10:50 AM, Christian König wrote:
> E.g. transactions are initially send to the root complex for
> translation, that's for sure. But at least for AMD GPUs the root complex
> answers with the translated address which is then cached in the device.
>
> So further transactions for the s
On 08/05/18 10:57 AM, Alex Williamson wrote:
> AIUI from previously questioning this, the change is hidden behind a
> build-time config option and only custom kernels or distros optimized
> for this sort of support would enable that build option. I'm more than
> a little dubious though that we'r
On 08/05/18 01:34 PM, Alex Williamson wrote:
> They are not so unrelated, see the ACS Direct Translated P2P
> capability, which in fact must be implemented by switch downstream
> ports implementing ACS and works specifically with ATS. This appears to
> be the way the PCI SIG would intend for P2P
On 08/05/18 02:13 PM, Alex Williamson wrote:
> Well, I'm a bit confused, this patch series is specifically disabling
> ACS on switches, but per the spec downstream switch ports implementing
> ACS MUST implement direct translated P2P. So it seems the only
> potential gap here is the endpoint, whi
On 08/05/18 02:43 PM, Alex Williamson wrote:
> Yes, GPUs seem to be leading the pack in implementing ATS. So now the
> dumb question, why not simply turn off the IOMMU and thus ACS? The
> argument of using the IOMMU for security is rather diminished if we're
> specifically enabling devices to p
On 08/05/18 04:03 PM, Alex Williamson wrote:
> If IOMMU grouping implies device assignment (because nobody else uses
> it to the same extent as device assignment) then the build-time option
> falls to pieces, we need a single kernel that can do both. I think we
> need to get more clever about al
On 08/05/18 05:00 PM, Dan Williams wrote:
>> I'd advise caution with a user supplied BDF approach, we have no
>> guaranteed persistence for a device's PCI address. Adding a device
>> might renumber the buses, replacing a device with one that consumes
>> more/less bus numbers can renumber the bus
On 08/05/18 05:11 PM, Alex Williamson wrote:
> On to the implementation details... I already mentioned the BDF issue
> in my other reply. If we had a way to persistently identify a device,
> would we specify the downstream points at which we want to disable ACS
> or the endpoints that we want to
On 09/05/18 07:40 AM, Christian König wrote:
> The key takeaway is that when any device has ATS enabled you can't
> disable ACS without breaking it (even if you unplug and replug it).
I don't follow how you came to this conclusion...
The ACS bits we'd be turning off are the ones that force TLP
On 10/05/18 08:16 AM, Stephen Bates wrote:
> Hi Christian
>
>> Why would a switch not identify that as a peer address? We use the PASID
>>together with ATS to identify the address space which a transaction
>>should use.
>
> I think you are conflating two types of TLPs here. If the de
On 10/05/18 11:11 AM, Stephen Bates wrote:
>> Not to me. In the p2pdma code we specifically program DMA engines with
>> the PCI bus address.
>
> Ah yes of course. Brain fart on my part. We are not programming the P2PDMA
> initiator with an IOVA but with the PCI bus address...
>
>> So regardl
On 10/05/18 12:41 PM, Stephen Bates wrote:
> Hi Jerome
>
>>Note on GPU we do would not rely on ATS for peer to peer. Some part
>>of the GPU (DMA engines) do not necessarily support ATS. Yet those
>>are the part likely to be use in peer to peer.
>
> OK this is good to know. I agree
On 5/11/2018 2:52 AM, Christian König wrote:
This only works when the IOVA and the PCI bus addresses never overlap.
I'm not sure how the IOVA allocation works but I don't think we
guarantee that on Linux.
I find this hard to believe. There's always the possibility that some
part of the system
On 5/11/2018 4:24 PM, Stephen Bates wrote:
All
Alex (or anyone else) can you point to where IOVA addresses are generated?
A case of RTFM perhaps (though a pointer to the code would still be
appreciated).
https://www.kernel.org/doc/Documentation/Intel-IOMMU.txt
Some exceptions to IOVA
--
Thanks for the review Randy! I'll make the changes for the next time we
post the series.
On 22/05/18 03:24 PM, Randy Dunlap wrote:
>> +The first task an orchestrator driver must do is compile a list of
>> +all client drivers that will be involved in a given transaction. For
>> +example, the NVMe T
ot be supported by memremap()
and therefore will not be support PCI P2P and have no support for CMB.
Signed-off-by: Logan Gunthorpe
---
drivers/nvme/host/pci.c | 80 +++--
1 file changed, 45 insertions(+), 35 deletions(-)
diff --git a/drivers/nvme/host/pci.c b/drivers
: Logan Gunthorpe
Reviewed-by: Sagi Grimberg
Reviewed-by: Christoph Hellwig
---
drivers/nvme/host/core.c | 4
drivers/nvme/host/nvme.h | 1 +
drivers/nvme/host/pci.c | 17 +
3 files changed, 18 insertions(+), 4 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers
a new capability bit to advertise whether this is possible for
future hardware.
This commit includes significant rework and feedback from Christoph
Hellwig.
Signed-off-by: Christoph Hellwig
Signed-off-by: Logan Gunthorpe
Acked-by: Bjorn Helgaas # PCI pieces
---
drivers/pci/Kconfig
attributes which take a boolean or a PCI device.
Any boolean as accepted by strtobool() turn P2P on or off (such as 'y', 'n',
'1', '0', etc). Specifying a full PCI device name/BDF will select the
specific device.
Signed-off-by: Logan Gunthorpe
Acked-by:
NVMe
SSDs (Intel, Seagate, Samsung) and p2pdma devices (Eideticom,
Microsemi, Chelsio and Everspin) using switches from both Microsemi
and Broadcomm.
--
Logan Gunthorpe (13):
PCI/P2PDMA: Support peer-to-peer memory
PCI/P2PDMA: Add sysfs group to display p2pmem stats
PCI/P2PDMA: Add PCI p2pme
Introduce a quirk to use CMB-like memory on older devices that have
an exposed BAR but do not advertise support for using CMBLOC and
CMBSIZE.
We'd like to use some of these older cards to test P2P memory.
Signed-off-by: Logan Gunthorpe
Reviewed-by: Sagi Grimberg
---
drivers/nvme/host/n
Add a new directory in the driver API guide for PCI specific
documentation.
This is in preparation for adding a new PCI P2P DMA driver writers
guide which will go in this directory.
Signed-off-by: Logan Gunthorpe
Cc: Jonathan Corbet
Cc: Mauro Carvalho Chehab
Cc: Greg Kroah-Hartman
Cc: Vinod
converted to restructured text
at this time.
Signed-off-by: Logan Gunthorpe
Acked-by: Bjorn Helgaas
Cc: Jonathan Corbet
---
Documentation/driver-api/pci/index.rst | 1 +
Documentation/driver-api/pci/p2pdma.rst | 170
2 files changed, 171 insertions(+)
create mode
he initial code]
Signed-off-by: Christoph Hellwig
Signed-off-by: Logan Gunthorpe
---
drivers/nvme/target/configfs.c| 36
drivers/nvme/target/core.c| 138 +-
drivers/nvme/target/io-cmd-bdev.c | 3 +
drivers/nvme/target/nvmet.h | 13 +++
dr
Add a sysfs group to display statistics about P2P memory that is
registered in each PCI device.
Attributes in the group display the total amount of P2P memory, the
amount available and whether it is published or not.
Signed-off-by: Logan Gunthorpe
Acked-by: Bjorn Helgaas
---
Documentation/ABI
QUEUE_FLAG_PCI_P2P is introduced meaning a driver's request queue
supports targeting P2P memory. This will be used by P2P providers and
orchestrators (in subsequent patches) to ensure block devices can
support P2P memory before submitting P2P backed pages to submit_bio().
Signed-off-by:
101 - 200 of 371 matches
Mail list logo
