Re: [PATCH kernel RFC 2/2] vfio-pci-nvlink2: Implement interconnect isolation

2019-04-04 Thread David Gibson
On Fri, Mar 22, 2019 at 05:10:25PM -0600, Alex Williamson wrote: > On Fri, 22 Mar 2019 14:08:38 +1100 > David Gibson wrote: > > > On Thu, Mar 21, 2019 at 12:19:34PM -0600, Alex Williamson wrote: > > > On Thu, 21 Mar 2019 10:56:00 +1100 > > > David Gibson wrote: > > > > > > > On Wed, Mar 20,

Re: [PATCH kernel RFC 2/2] vfio-pci-nvlink2: Implement interconnect isolation

2019-03-22 Thread Alex Williamson
On Fri, 22 Mar 2019 14:08:38 +1100 David Gibson wrote: > On Thu, Mar 21, 2019 at 12:19:34PM -0600, Alex Williamson wrote: > > On Thu, 21 Mar 2019 10:56:00 +1100 > > David Gibson wrote: > > > > > On Wed, Mar 20, 2019 at 01:09:08PM -0600, Alex Williamson wrote: > > > > On Wed, 20 Mar 2019

Re: [PATCH kernel RFC 2/2] vfio-pci-nvlink2: Implement interconnect isolation

2019-03-21 Thread David Gibson
On Thu, Mar 21, 2019 at 12:19:34PM -0600, Alex Williamson wrote: > On Thu, 21 Mar 2019 10:56:00 +1100 > David Gibson wrote: > > > On Wed, Mar 20, 2019 at 01:09:08PM -0600, Alex Williamson wrote: > > > On Wed, 20 Mar 2019 15:38:24 +1100 > > > David Gibson wrote: > > > > > > > On Tue, Mar 19,

Re: [PATCH kernel RFC 2/2] vfio-pci-nvlink2: Implement interconnect isolation

2019-03-21 Thread Alex Williamson
On Thu, 21 Mar 2019 10:56:00 +1100 David Gibson wrote: > On Wed, Mar 20, 2019 at 01:09:08PM -0600, Alex Williamson wrote: > > On Wed, 20 Mar 2019 15:38:24 +1100 > > David Gibson wrote: > > > > > On Tue, Mar 19, 2019 at 10:36:19AM -0600, Alex Williamson wrote: > > > > On Fri, 15 Mar 2019

Re: [PATCH kernel RFC 2/2] vfio-pci-nvlink2: Implement interconnect isolation

2019-03-20 Thread David Gibson
On Wed, Mar 20, 2019 at 01:09:08PM -0600, Alex Williamson wrote: > On Wed, 20 Mar 2019 15:38:24 +1100 > David Gibson wrote: > > > On Tue, Mar 19, 2019 at 10:36:19AM -0600, Alex Williamson wrote: > > > On Fri, 15 Mar 2019 19:18:35 +1100 > > > Alexey Kardashevskiy wrote: > > > > > > > The

Re: [PATCH kernel RFC 2/2] vfio-pci-nvlink2: Implement interconnect isolation

2019-03-20 Thread Alex Williamson
On Wed, 20 Mar 2019 15:38:24 +1100 David Gibson wrote: > On Tue, Mar 19, 2019 at 10:36:19AM -0600, Alex Williamson wrote: > > On Fri, 15 Mar 2019 19:18:35 +1100 > > Alexey Kardashevskiy wrote: > > > > > The NVIDIA V100 SXM2 GPUs are connected to the CPU via PCIe links and > > > (on POWER9)

Re: [PATCH kernel RFC 2/2] vfio-pci-nvlink2: Implement interconnect isolation

2019-03-20 Thread David Gibson
On Tue, Mar 19, 2019 at 10:36:19AM -0600, Alex Williamson wrote: > On Fri, 15 Mar 2019 19:18:35 +1100 > Alexey Kardashevskiy wrote: > > > The NVIDIA V100 SXM2 GPUs are connected to the CPU via PCIe links and > > (on POWER9) NVLinks. In addition to that, GPUs themselves have direct > > peer to

Re: [PATCH kernel RFC 2/2] vfio-pci-nvlink2: Implement interconnect isolation

2019-03-19 Thread Alexey Kardashevskiy
On 20/03/2019 03:36, Alex Williamson wrote: > On Fri, 15 Mar 2019 19:18:35 +1100 > Alexey Kardashevskiy wrote: > >> The NVIDIA V100 SXM2 GPUs are connected to the CPU via PCIe links and >> (on POWER9) NVLinks. In addition to that, GPUs themselves have direct >> peer to peer NVLinks in groups

Re: [PATCH kernel RFC 2/2] vfio-pci-nvlink2: Implement interconnect isolation

2019-03-19 Thread Alex Williamson
On Fri, 15 Mar 2019 19:18:35 +1100 Alexey Kardashevskiy wrote: > The NVIDIA V100 SXM2 GPUs are connected to the CPU via PCIe links and > (on POWER9) NVLinks. In addition to that, GPUs themselves have direct > peer to peer NVLinks in groups of 2 to 4 GPUs. At the moment the POWERNV > platform

[PATCH kernel RFC 2/2] vfio-pci-nvlink2: Implement interconnect isolation

2019-03-15 Thread Alexey Kardashevskiy
The NVIDIA V100 SXM2 GPUs are connected to the CPU via PCIe links and (on POWER9) NVLinks. In addition to that, GPUs themselves have direct peer to peer NVLinks in groups of 2 to 4 GPUs. At the moment the POWERNV platform puts all interconnected GPUs to the same IOMMU group. However the user may