On 08/21/2013 03:32 AM, René Koch (ovido) wrote:
On Tue, 2013-08-20 at 00:24 +0200, René Koch wrote:
-----Original message-----
From:Dan Kenigsberg <dan...@redhat.com>
Sent: Monday 19th August 2013 23:48
To: René Koch <r.k...@ovido.at>; Itzik Brown <itz...@mellanox.com>
Cc: ovirt-users <users@ovirt.org>
Subject: Re: [Users] oVirt and Infiniband
On Tue, Aug 13, 2013 at 03:48:14PM +0200, René Koch (ovido) wrote:
Hi,
Does anyone have oVirt (or a plain KVM hypervisor) running on Infiniband
where the virtual machines are bridged to an Infiniband interface (with
EoIB kernel module)?
As far as I recall the EoIB module is not yet in upstream kernel. Could
you give more details on your setup (distro, kernel, module)?
Do you get connectivity between the hosts? Maybe Itzik and his
colleagues could help us here.
Thanks for your answer.
That's right - EoIB isn't in the kernel.
I installed module from Mellanox OFED package and even there it's marked as
unstable...
OS is CentOS 6.4.
At the moment I've also an open support case with Mellanox trying to solve the
connectivity issue.
So I have support from official site, but was thinking maybe someone has
already experience with such a setup and also had some issues/tipps for me...
My problem is some sort of a bridge or maybe MAC translation issue (unsure
cause of a lot of unanswered ARP requests in tcpdump and bridge is working on
Ethernet interfaces).
I have bridge ovirtmgmt on eth2 interface (eth2 is a (virtual) ethernet
interface bound to the Infiniband-interface) with an IP address configured on
it.
Hosts can reach each other, so ethernet to infiniband translation seems to work.
But when I create a vm in oVirt this vm can only communicate with IPs/vms on
the bridge.
This mean I can ping the IP of the host and other vms on this bridge, but no
host behind the bridge (like e.g. other hosts or vms on other hosts)...
Mellanox support team found the issue:
Daemon openibd requires write access to libvirt which is restricted by
oVirt per default.
When changing auth_unix_rw to "none" networking of the vms is working fine
over the inifinband network.
So my question is now:
I think there's a good reason why write access to libvirt is restricted.
In my particular setup no one will do a virsh start/stop/whatever so
from a user point I can live with an open libvirt.
But are there any troubles I can run into from oVirt side with
auth_unix_rw="none" beside users doing evil virsh stuff?
Today I tested it and ran into a first issue which I didn't investigate
so far (will do this week):
- Started vm from oVirt on node
- Changed auth_unix_rw to "none"
- Restarted libvirt on node
- VM was running according to "virsh list"
- VM was stopped in oVirt
- Started vm in oVirt on another node
- vm was running twice
Will have a look at the logs what append exactly this vm...
isn't the right thing is to configure openibd to use same credentials
vdsm is using (or with their own credentials)?
Regards,
René
I'm having issues in such a setup where vms can't communicate over this
bridge...
Regards,
René
_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
