Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-04-25 Thread Logan Gunthorpe
On 25/04/17 05:58 AM, Marta Rybczynska wrote: > I would add one issue that doesn't seem to be addressed: in my experience > P2P doesn't work when IOMMU activated. It works best with deactivation at > the BIOS level, even the kernel options are not enough in some cases. Well this would likely be

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-04-25 Thread Logan Gunthorpe
On 25/04/17 05:58 AM, Marta Rybczynska wrote: > I would add one issue that doesn't seem to be addressed: in my experience > P2P doesn't work when IOMMU activated. It works best with deactivation at > the BIOS level, even the kernel options are not enough in some cases. Well this would likely be

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-04-25 Thread Marta Rybczynska
> On 02/04/17 03:03 PM, Sinan Kaya wrote: >> Push the decision all the way to the user. Let them decide whether they >> want this feature to work on a root port connected port or under the >> switch. > > Yes, I prefer this too. If other folks agree with that I'd be very happy > to go back to user

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-04-25 Thread Marta Rybczynska
> On 02/04/17 03:03 PM, Sinan Kaya wrote: >> Push the decision all the way to the user. Let them decide whether they >> want this feature to work on a root port connected port or under the >> switch. > > Yes, I prefer this too. If other folks agree with that I'd be very happy > to go back to user

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-04-02 Thread Logan Gunthorpe
On 02/04/17 03:03 PM, Sinan Kaya wrote: > Push the decision all the way to the user. Let them decide whether they > want this feature to work on a root port connected port or under the > switch. Yes, I prefer this too. If other folks agree with that I'd be very happy to go back to user chooses.

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-04-02 Thread Logan Gunthorpe
On 02/04/17 03:03 PM, Sinan Kaya wrote: > Push the decision all the way to the user. Let them decide whether they > want this feature to work on a root port connected port or under the > switch. Yes, I prefer this too. If other folks agree with that I'd be very happy to go back to user chooses.

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-04-02 Thread Sinan Kaya
On 4/2/2017 1:21 PM, Logan Gunthorpe wrote: >> This is when the BIOS date helps so that you don't break existing systems. > I'm not that worried about this code breaking existing systems. There > are significant trade-offs with using p2pmem (ie. you are quite likely > sacrificing performance for

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-04-02 Thread Sinan Kaya
On 4/2/2017 1:21 PM, Logan Gunthorpe wrote: >> This is when the BIOS date helps so that you don't break existing systems. > I'm not that worried about this code breaking existing systems. There > are significant trade-offs with using p2pmem (ie. you are quite likely > sacrificing performance for

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-04-02 Thread Logan Gunthorpe
On 01/04/17 08:26 PM, Sinan Kaya wrote: > I recommended a combination of blacklist + p2p capability + BIOS date. > Not just BIOS date. BIOS date by itself is useless. Well this proposal doesn't work for me at all. None of my hardware has the p2p ACS capability and my BIOS date is in 2013 and

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-04-02 Thread Logan Gunthorpe
On 01/04/17 08:26 PM, Sinan Kaya wrote: > I recommended a combination of blacklist + p2p capability + BIOS date. > Not just BIOS date. BIOS date by itself is useless. Well this proposal doesn't work for me at all. None of my hardware has the p2p ACS capability and my BIOS date is in 2013 and

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-04-01 Thread Sinan Kaya
Hi Logan, I added Alex and Bjorn above. On 4/1/2017 6:16 PM, Logan Gunthorpe wrote: > Hey, > > On 31/03/17 08:17 PM, ok...@codeaurora.org wrote: >> See drivers/pci and drivers/acpi directory. > > The best I could find was the date of the firmware/bios. I really don't > think that makes sense

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-04-01 Thread Sinan Kaya
Hi Logan, I added Alex and Bjorn above. On 4/1/2017 6:16 PM, Logan Gunthorpe wrote: > Hey, > > On 31/03/17 08:17 PM, ok...@codeaurora.org wrote: >> See drivers/pci and drivers/acpi directory. > > The best I could find was the date of the firmware/bios. I really don't > think that makes sense

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-04-01 Thread Logan Gunthorpe
Hey, On 31/03/17 08:17 PM, ok...@codeaurora.org wrote: > See drivers/pci and drivers/acpi directory. The best I could find was the date of the firmware/bios. I really don't think that makes sense to tie the two together. And really the more that I think about it trying to do a date cutoff for

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-04-01 Thread Logan Gunthorpe
Hey, On 31/03/17 08:17 PM, ok...@codeaurora.org wrote: > See drivers/pci and drivers/acpi directory. The best I could find was the date of the firmware/bios. I really don't think that makes sense to tie the two together. And really the more that I think about it trying to do a date cutoff for

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-03-31 Thread okaya
On 2017-03-31 21:57, Logan Gunthorpe wrote: On 31/03/17 05:51 PM, Sinan Kaya wrote: You can put a restriction with DMI/SMBIOS such that all devices from 2016 work else they belong to blacklist. How do you get a manufacturing date for a given device within the kernel? Is this actually

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-03-31 Thread okaya
On 2017-03-31 21:57, Logan Gunthorpe wrote: On 31/03/17 05:51 PM, Sinan Kaya wrote: You can put a restriction with DMI/SMBIOS such that all devices from 2016 work else they belong to blacklist. How do you get a manufacturing date for a given device within the kernel? Is this actually

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-03-31 Thread Logan Gunthorpe
On 31/03/17 05:51 PM, Sinan Kaya wrote: > You can put a restriction with DMI/SMBIOS such that all devices from 2016 > work else they belong to blacklist. How do you get a manufacturing date for a given device within the kernel? Is this actually something generically available? Logan

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-03-31 Thread Logan Gunthorpe
On 31/03/17 05:51 PM, Sinan Kaya wrote: > You can put a restriction with DMI/SMBIOS such that all devices from 2016 > work else they belong to blacklist. How do you get a manufacturing date for a given device within the kernel? Is this actually something generically available? Logan

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-03-31 Thread Sinan Kaya
On 3/31/2017 6:42 PM, Logan Gunthorpe wrote: > > > On 31/03/17 03:38 PM, Sinan Kaya wrote: >> On 3/31/2017 5:23 PM, Logan Gunthorpe wrote: >>> What exactly would you white/black list? It can't be the NIC or the >>> disk. If it's going to be a white/black list on the switch or root port >>> then

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-03-31 Thread Sinan Kaya
On 3/31/2017 6:42 PM, Logan Gunthorpe wrote: > > > On 31/03/17 03:38 PM, Sinan Kaya wrote: >> On 3/31/2017 5:23 PM, Logan Gunthorpe wrote: >>> What exactly would you white/black list? It can't be the NIC or the >>> disk. If it's going to be a white/black list on the switch or root port >>> then

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-03-31 Thread Logan Gunthorpe
On 31/03/17 03:38 PM, Sinan Kaya wrote: > On 3/31/2017 5:23 PM, Logan Gunthorpe wrote: >> What exactly would you white/black list? It can't be the NIC or the >> disk. If it's going to be a white/black list on the switch or root port >> then you'd need essentially the same code to ensure they are

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-03-31 Thread Logan Gunthorpe
On 31/03/17 03:38 PM, Sinan Kaya wrote: > On 3/31/2017 5:23 PM, Logan Gunthorpe wrote: >> What exactly would you white/black list? It can't be the NIC or the >> disk. If it's going to be a white/black list on the switch or root port >> then you'd need essentially the same code to ensure they are

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-03-31 Thread Sinan Kaya
On 3/31/2017 5:23 PM, Logan Gunthorpe wrote: > What exactly would you white/black list? It can't be the NIC or the > disk. If it's going to be a white/black list on the switch or root port > then you'd need essentially the same code to ensure they are all behind > the same switch or root port.

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-03-31 Thread Sinan Kaya
On 3/31/2017 5:23 PM, Logan Gunthorpe wrote: > What exactly would you white/black list? It can't be the NIC or the > disk. If it's going to be a white/black list on the switch or root port > then you'd need essentially the same code to ensure they are all behind > the same switch or root port.

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-03-31 Thread Logan Gunthorpe
On 31/03/17 12:49 PM, Sinan Kaya wrote: > Don't you need to clean up the p->pool here. See Patch 7 in the series. >> +put_device(>dev); >> +} >> +EXPORT_SYMBOL(p2pmem_unregister); >> + > > I don't like the ugliness around the switch port to be honest. > > Going to whitelist/blacklist

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-03-31 Thread Logan Gunthorpe
On 31/03/17 12:49 PM, Sinan Kaya wrote: > Don't you need to clean up the p->pool here. See Patch 7 in the series. >> +put_device(>dev); >> +} >> +EXPORT_SYMBOL(p2pmem_unregister); >> + > > I don't like the ugliness around the switch port to be honest. > > Going to whitelist/blacklist

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-03-31 Thread Sinan Kaya
Hi Logan, > +/** > + * p2pmem_unregister() - unregister a p2pmem device > + * @p: the device to unregister > + * > + * The device will remain until all users are done with it > + */ > +void p2pmem_unregister(struct p2pmem_dev *p) > +{ > + if (!p) > + return; > + > +

Re: [RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-03-31 Thread Sinan Kaya
Hi Logan, > +/** > + * p2pmem_unregister() - unregister a p2pmem device > + * @p: the device to unregister > + * > + * The device will remain until all users are done with it > + */ > +void p2pmem_unregister(struct p2pmem_dev *p) > +{ > + if (!p) > + return; > + > +

[RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-03-30 Thread Logan Gunthorpe
A p2pmem device is simply a PCI card with a BAR space that points to regular memory. This may be an independent PCI card or part of another completely unrelated device (like an IB card or a NVMe card). The p2pmem device is designed such that other drivers may register p2pmem memory for use by the

[RFC 1/8] Introduce Peer-to-Peer memory (p2pmem) device

2017-03-30 Thread Logan Gunthorpe
A p2pmem device is simply a PCI card with a BAR space that points to regular memory. This may be an independent PCI card or part of another completely unrelated device (like an IB card or a NVMe card). The p2pmem device is designed such that other drivers may register p2pmem memory for use by the