Re: [PATCH 04/12] pci-p2p: Clear ACS P2P flags for all client devices

2018-01-05 Thread Alex Williamson
On Fri, 5 Jan 2018 10:10:51 -0700 Logan Gunthorpe wrote: > On 04/01/18 08:33 PM, Alex Williamson wrote: > > That's exactly what IOMMU groups represent, the smallest set of devices > > which have DMA isolation from other devices. By poking this hole, the > > IOMMU group is invalid. We cannot tur

Re: [PATCH 04/12] pci-p2p: Clear ACS P2P flags for all client devices

2018-01-05 Thread Logan Gunthorpe
On 04/01/18 08:33 PM, Alex Williamson wrote: That's exactly what IOMMU groups represent, the smallest set of devices which have DMA isolation from other devices. By poking this hole, the IOMMU group is invalid. We cannot turn off ACS only for a specific device, in order to enable p2p it needs

Re: [PATCH 04/12] pci-p2p: Clear ACS P2P flags for all client devices

2018-01-05 Thread Alex Williamson
On Fri, 5 Jan 2018 01:47:01 -0500 Jerome Glisse wrote: > On Thu, Jan 04, 2018 at 08:33:00PM -0700, Alex Williamson wrote: > > On Thu, 4 Jan 2018 17:00:47 -0700 > > Logan Gunthorpe wrote: > > > > > On 04/01/18 03:35 PM, Alex Williamson wrote: > > > > Yep, flipping these ACS bits invalidates

Re: [PATCH 04/12] pci-p2p: Clear ACS P2P flags for all client devices

2018-01-04 Thread Jerome Glisse
On Thu, Jan 04, 2018 at 08:33:00PM -0700, Alex Williamson wrote: > On Thu, 4 Jan 2018 17:00:47 -0700 > Logan Gunthorpe wrote: > > > On 04/01/18 03:35 PM, Alex Williamson wrote: > > > Yep, flipping these ACS bits invalidates any IOMMU groups that depend > > > on the isolation of that downstream po

Re: [PATCH 04/12] pci-p2p: Clear ACS P2P flags for all client devices

2018-01-04 Thread Alex Williamson
On Thu, 4 Jan 2018 17:00:47 -0700 Logan Gunthorpe wrote: > On 04/01/18 03:35 PM, Alex Williamson wrote: > > Yep, flipping these ACS bits invalidates any IOMMU groups that depend > > on the isolation of that downstream port and I suspect also any peers > > within the same PCI slot of that port and

Re: [PATCH 04/12] pci-p2p: Clear ACS P2P flags for all client devices

2018-01-04 Thread Logan Gunthorpe
On 04/01/18 05:00 PM, Logan Gunthorpe wrote: > > > On 04/01/18 03:35 PM, Alex Williamson wrote: >> Yep, flipping these ACS bits invalidates any IOMMU groups that depend >> on the isolation of that downstream port and I suspect also any peers >> within the same PCI slot of that port and their do

Re: [PATCH 04/12] pci-p2p: Clear ACS P2P flags for all client devices

2018-01-04 Thread Logan Gunthorpe
On 04/01/18 03:35 PM, Alex Williamson wrote: Yep, flipping these ACS bits invalidates any IOMMU groups that depend on the isolation of that downstream port and I suspect also any peers within the same PCI slot of that port and their downstream devices. The entire sub-hierarchy grouping needs t

Re: [PATCH 04/12] pci-p2p: Clear ACS P2P flags for all client devices

2018-01-04 Thread Alex Williamson
On Thu, 4 Jan 2018 15:57:21 -0600 Bjorn Helgaas wrote: > [+cc Alex] > > On Thu, Jan 04, 2018 at 12:01:29PM -0700, Logan Gunthorpe wrote: > > When the ACS P2P flags are set in the downstream port of the switch, > > any P2P TLPs will be sent back to the root complex. The whole point of > > the P2P

Re: [PATCH 04/12] pci-p2p: Clear ACS P2P flags for all client devices

2018-01-04 Thread Bjorn Helgaas
[+cc Alex] On Thu, Jan 04, 2018 at 12:01:29PM -0700, Logan Gunthorpe wrote: > When the ACS P2P flags are set in the downstream port of the switch, > any P2P TLPs will be sent back to the root complex. The whole point of > the P2P work is to have TLPs avoid the RC seeing it may not support > P2P tr

[PATCH 04/12] pci-p2p: Clear ACS P2P flags for all client devices

2018-01-04 Thread Logan Gunthorpe
When the ACS P2P flags are set in the downstream port of the switch, any P2P TLPs will be sent back to the root complex. The whole point of the P2P work is to have TLPs avoid the RC seeing it may not support P2P transactions or, at best, it will perform poorly. So we clear these flags for all devic