On Tue, 2013-08-20 at 00:24 +0200, René Koch wrote: > > -----Original message----- > > From:Dan Kenigsberg <dan...@redhat.com> > > Sent: Monday 19th August 2013 23:48 > > To: René Koch <r.k...@ovido.at>; Itzik Brown <itz...@mellanox.com> > > Cc: ovirt-users <users@ovirt.org> > > Subject: Re: [Users] oVirt and Infiniband > > > > On Tue, Aug 13, 2013 at 03:48:14PM +0200, René Koch (ovido) wrote: > > > Hi, > > > > > > Does anyone have oVirt (or a plain KVM hypervisor) running on Infiniband > > > where the virtual machines are bridged to an Infiniband interface (with > > > EoIB kernel module)? > > > > As far as I recall the EoIB module is not yet in upstream kernel. Could > > you give more details on your setup (distro, kernel, module)? > > Do you get connectivity between the hosts? Maybe Itzik and his > > colleagues could help us here. > > > Thanks for your answer. > > That's right - EoIB isn't in the kernel. > I installed module from Mellanox OFED package and even there it's marked as > unstable... > OS is CentOS 6.4. > > At the moment I've also an open support case with Mellanox trying to solve > the connectivity issue. > So I have support from official site, but was thinking maybe someone has > already experience with such a setup and also had some issues/tipps for me... > > My problem is some sort of a bridge or maybe MAC translation issue (unsure > cause of a lot of unanswered ARP requests in tcpdump and bridge is working on > Ethernet interfaces). > I have bridge ovirtmgmt on eth2 interface (eth2 is a (virtual) ethernet > interface bound to the Infiniband-interface) with an IP address configured on > it. > Hosts can reach each other, so ethernet to infiniband translation seems to > work. > But when I create a vm in oVirt this vm can only communicate with IPs/vms on > the bridge. > This mean I can ping the IP of the host and other vms on this bridge, but no > host behind the bridge (like e.g. other hosts or vms on other hosts)... >
Mellanox support team found the issue: Daemon openibd requires write access to libvirt which is restricted by oVirt per default. When changing auth_unix_rw to "none" networking of the vms is working fine over the inifinband network. So my question is now: I think there's a good reason why write access to libvirt is restricted. In my particular setup no one will do a virsh start/stop/whatever so from a user point I can live with an open libvirt. But are there any troubles I can run into from oVirt side with auth_unix_rw="none" beside users doing evil virsh stuff? Today I tested it and ran into a first issue which I didn't investigate so far (will do this week): - Started vm from oVirt on node - Changed auth_unix_rw to "none" - Restarted libvirt on node - VM was running according to "virsh list" - VM was stopped in oVirt - Started vm in oVirt on another node - vm was running twice Will have a look at the logs what append exactly this vm... > > Regards, > René > > > > > > > > > > I'm having issues in such a setup where vms can't communicate over this > > > bridge... > > > > > > > > > Regards, > > > René > > > > > > > > > > > > _______________________________________________ > > > Users mailing list > > > Users@ovirt.org > > > http://lists.ovirt.org/mailman/listinfo/users > > > _______________________________________________ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users