Re: Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-20 Thread Leo Yan
On Tue, Mar 19, 2019 at 09:33:58AM +0800, Leo Yan wrote: [...] > So far, it still cannot work well if I only set "sky2.disable_msi=1" > for host kernel command line, with this config it runs with below flow > and which cannot forward interrupt properly from host to guest: > > host

Re: Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-18 Thread Leo Yan
Hi Robin, On Mon, Mar 18, 2019 at 12:25:33PM +, Robin Murphy wrote: [...] > > diff --git a/arm/include/arm-common/kvm-arch.h > > b/arm/include/arm-common/kvm-arch.h > > index b9d486d..43f78b1 100644 > > --- a/arm/include/arm-common/kvm-arch.h > > +++ b/arm/include/arm-common/kvm-arch.h > >

Re: Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-18 Thread Robin Murphy
On 16/03/2019 04:56, Leo Yan wrote: Hi Robin, On Fri, Mar 15, 2019 at 12:54:10PM +, Robin Murphy wrote: Hi Leo, Sorry for the delay - I'm on holiday this week, but since I've made the mistake of glancing at my inbox I should probably save you from wasting any more time... Sorry for distu

Re: Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-15 Thread Leo Yan
Hi Robin, On Fri, Mar 15, 2019 at 12:54:10PM +, Robin Murphy wrote: > Hi Leo, > > Sorry for the delay - I'm on holiday this week, but since I've made the > mistake of glancing at my inbox I should probably save you from wasting any > more time... Sorry for disturbing you in holiday and appre

Re: Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-15 Thread Robin Murphy
Hi Leo, Sorry for the delay - I'm on holiday this week, but since I've made the mistake of glancing at my inbox I should probably save you from wasting any more time... On 2019-03-15 11:03 am, Auger Eric wrote: Hi Leo, + Jean-Philippe On 3/15/19 10:37 AM, Leo Yan wrote: Hi Eric, Robin, O

Re: Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-15 Thread Auger Eric
Hi Leo, + Jean-Philippe On 3/15/19 10:37 AM, Leo Yan wrote: > Hi Eric, Robin, > > On Wed, Mar 13, 2019 at 11:24:25AM +0100, Auger Eric wrote: > > [...] > >>> If the NIC supports MSIs they logically are used. This can be easily >>> checked on host by issuing "cat /proc/interrupts | grep vfio".

Re: Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-15 Thread Leo Yan
Hi Eric, Robin, On Wed, Mar 13, 2019 at 11:24:25AM +0100, Auger Eric wrote: [...] > > If the NIC supports MSIs they logically are used. This can be easily > > checked on host by issuing "cat /proc/interrupts | grep vfio". Can you > > check whether the guest received any interrupt? I remember tha

Re: Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-13 Thread Leo Yan
On Wed, Mar 13, 2019 at 11:24:25AM +0100, Auger Eric wrote: [...] > >> I am stucking at the network card cannot receive interrupts in guest > >> OS. So took time to look into the code and added some printed info to > >> help me to understand the detailed flow, below are two main questions > >> I

Re: Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-13 Thread Leo Yan
Hi Eric, On Wed, Mar 13, 2019 at 11:01:33AM +0100, Auger Eric wrote: [...] > > I want to confirm, if this is the recommended mode for > > passthrough PCI-e device to use msi both in host OS and geust OS? > > Or it's will be fine for host OS using msi and guest OS using > > INTx mode? >

Re: Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-13 Thread Auger Eric
Hi, On 3/13/19 11:01 AM, Auger Eric wrote: > Hi Leo, > > + Robin > > On 3/13/19 9:00 AM, Leo Yan wrote: >> Hi Eric & all, >> >> On Mon, Mar 11, 2019 at 10:35:01PM +0800, Leo Yan wrote: >> >> [...] >> >>> So now I made some progress and can see the networking card is >>> pass-through to guest OS,

Re: Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-13 Thread Auger Eric
Hi Leo, On 3/13/19 11:01 AM, Leo Yan wrote: > On Wed, Mar 13, 2019 at 04:00:48PM +0800, Leo Yan wrote: > > [...] > >> - The second question is for GICv2m. If I understand correctly, when >> passthrough PCI-e device to guest OS, in the guest OS we should >> create below data path for PCI-e d

Re: Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-13 Thread Auger Eric
Hi Leo, + Robin On 3/13/19 9:00 AM, Leo Yan wrote: > Hi Eric & all, > > On Mon, Mar 11, 2019 at 10:35:01PM +0800, Leo Yan wrote: > > [...] > >> So now I made some progress and can see the networking card is >> pass-through to guest OS, though the networking card reports errors >> now. Below i

Re: Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-13 Thread Leo Yan
On Wed, Mar 13, 2019 at 04:00:48PM +0800, Leo Yan wrote: [...] > - The second question is for GICv2m. If I understand correctly, when > passthrough PCI-e device to guest OS, in the guest OS we should > create below data path for PCI-e devices: >

Re: Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-13 Thread Leo Yan
Hi Eric & all, On Mon, Mar 11, 2019 at 10:35:01PM +0800, Leo Yan wrote: [...] > So now I made some progress and can see the networking card is > pass-through to guest OS, though the networking card reports errors > now. Below is detailed steps and info: > > - Bind devices in the same IOMMU gro

Re: Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-11 Thread Leo Yan
Hi Eric, [ + Mark Rutland ] On Mon, Mar 11, 2019 at 10:47:22AM +0100, Auger Eric wrote: [...] > >>> P.s. I also checked the sysfs node and found it doesn't contain node > >>> 'iommu_group': > >>> > >>> # ls /sys/bus/pci/devices/\:08\:00.0/iommu_group > >>> ls: cannot access '/sys/bus/pci/de

Re: Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-11 Thread Auger Eric
Hi Leo, On 3/11/19 10:39 AM, Leo Yan wrote: > Hi Auger, > > On Mon, Mar 11, 2019 at 09:23:20AM +0100, Auger Eric wrote: > > [...] > >>> P.s. I also checked the sysfs node and found it doesn't contain node >>> 'iommu_group': >>> >>> # ls /sys/bus/pci/devices/\:08\:00.0/iommu_group >>> ls: ca

Re: Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-11 Thread Leo Yan
Hi Auger, On Mon, Mar 11, 2019 at 09:23:20AM +0100, Auger Eric wrote: [...] > > P.s. I also checked the sysfs node and found it doesn't contain node > > 'iommu_group': > > > > # ls /sys/bus/pci/devices/\:08\:00.0/iommu_group > > ls: cannot access '/sys/bus/pci/devices/:08:00.0/iommu_gro

Re: Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-11 Thread Auger Eric
Hi Leo, On 3/11/19 7:42 AM, Leo Yan wrote: > Hi all, > > I am trying to enable PCI-e device pass-through mode with KVM, since > Juno-r2 board has PCI-e bus so I firstly try to use vfio to > passthrough the network card on PCI-e bus. > > According to Juno-r2 board TRM [1], there has a CoreLink MM

Re: Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-10 Thread Leo Yan
On Mon, Mar 11, 2019 at 02:42:48PM +0800, Leo Yan wrote: > Hi all, > > I am trying to enable PCI-e device pass-through mode with KVM, since > Juno-r2 board has PCI-e bus so I firstly try to use vfio to > passthrough the network card on PCI-e bus. Sorry for spamming, just want to add info for Linu

Question: KVM: Failed to bind vfio with PCI-e / SMMU on Juno-r2

2019-03-10 Thread Leo Yan
Hi all, I am trying to enable PCI-e device pass-through mode with KVM, since Juno-r2 board has PCI-e bus so I firstly try to use vfio to passthrough the network card on PCI-e bus. According to Juno-r2 board TRM [1], there has a CoreLink MMU-401 (SMMU) between PCI-e devices and CCI bus; IIUC, PCI-