Re: [ovirt-users] VM support for Infiniband

[Kevin Hung]

On 9/30/2016 10:10 AM, Kevin Hung wrote:
oVirt sees the HCA as SR-IOV capable and it sees the VFs, so I believe 
I have set up SR-IOV correctly. I was able to do a generic passthrough 
of the HCA, but I was not able to pass through the VFs to more than 
one VM probably because they are all in the same IOMMU group. I first 
tried following the oVirt 3.6 Deep Dive video on SR-IOV [1] but was 
stuck because of the bridging problem.

[1] https://www.youtube.com/watch?v=A-MROZ8D06Y


I was able to pass through the VFs after fixing the IOMMU issue. I had 
to read up on IOMMU groups and discovered that the PCIe root ports on 
Xeon E3 processors do not support ACS [1], which is needed for the VFs 
to be in different IOMMU groups. I was able to plug the HCA into a 
different PCIe slot which was connected to the PCH instead, allowing 
VFIO to see the VFs in different IOMMU groups.


The way SR-IOV works for Ethernet network adapters in oVirt seems to be 
really nice as the scheduler takes care of the pass-through. Is this 
possible for Infiniband HCAs or is that a technical/architectural 
limitation?


[1] http://vfio.blogspot.com.ar/2014/08/iommu-groups-inside-and-out.html

Thanks,
Kevin Hung
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] VM support for Infiniband

[Kevin Hung]

On 9/30/2016 9:28 AM, Michal Skrivanek wrote:

On 30 Sep 2016, at 03:56, Kevin Hung  wrote:

Hello,

I have been trying (unsuccessfully) to add my Infiniband HCAs on my VM hosts to 
oVirt 4.0.4. I am not using IBoIP, just plain Infiniband. I would like to be 
able to pass through the card (or better yet the VF from SR-IOV) to the guest 
VM.

SR-IOV should work (or general PCI passthrough), it’s then transparent to 
oVirt. Did you not succeed with setting up SR-IOV or generic passthrough of the 
HCA?
oVirt sees the HCA as SR-IOV capable and it sees the VFs, so I believe I 
have set up SR-IOV correctly. I was able to do a generic passthrough of 
the HCA, but I was not able to pass through the VFs to more than one VM 
probably because they are all in the same IOMMU group. I first tried 
following the oVirt 3.6 Deep Dive video on SR-IOV [1] but was stuck 
because of the bridging problem.





 From what I can figure out, oVirt is trying to bridge the interface but I 
don't think that will work as shown here [1]. Does oVirt currently have support 
for passing through Infiniband to VMs? If not, are there any plans to support 
it?

Well, at least for generic passthrough the PCI device would be detached from 
the host so should not be recognized/touched by oVirt networking code at all.

Thanks,
michal


[1] http://lists.ovirt.org/pipermail/users/2016-July/040898.html

Thanks,
Kevin Hung
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users



[1] https://www.youtube.com/watch?v=A-MROZ8D06Y
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] VM support for Infiniband

[Michal Skrivanek]

> On 30 Sep 2016, at 03:56, Kevin Hung  wrote:
> 
> Hello,
> 
> I have been trying (unsuccessfully) to add my Infiniband HCAs on my VM hosts 
> to oVirt 4.0.4. I am not using IBoIP, just plain Infiniband. I would like to 
> be able to pass through the card (or better yet the VF from SR-IOV) to the 
> guest VM.

SR-IOV should work (or general PCI passthrough), it’s then transparent to 
oVirt. Did you not succeed with setting up SR-IOV or generic passthrough of the 
HCA?

> 
> From what I can figure out, oVirt is trying to bridge the interface but I 
> don't think that will work as shown here [1]. Does oVirt currently have 
> support for passing through Infiniband to VMs? If not, are there any plans to 
> support it?

Well, at least for generic passthrough the PCI device would be detached from 
the host so should not be recognized/touched by oVirt networking code at all.

Thanks,
michal

> 
> [1] http://lists.ovirt.org/pipermail/users/2016-July/040898.html
> 
> Thanks,
> Kevin Hung
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
> 
> 

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users