On Tue, Dec 11, 2012 at 04:21:51PM -0500, Antoni Segura Puimedon wrote: > > > ----- Original Message ----- > > From: "Dan Kenigsberg" <dan...@redhat.com> > > To: "Antoni Segura Puimedon" <asegu...@redhat.com>, "Igor Lvovsky" > > <ilvov...@redhat.com> > > Cc: "Mark Wu" <wu...@linux.vnet.ibm.com>, vdsm-devel@lists.fedorahosted.org > > Sent: Tuesday, December 11, 2012 7:44:05 PM > > Subject: Re: [vdsm] Fwd: Bonding, bridges and ifcfg > > > > On Tue, Dec 11, 2012 at 11:42:49AM -0500, Antoni Segura Puimedon > > wrote: > > > > > > > > > ----- Original Message ----- > > > > From: "Mark Wu" <wu...@linux.vnet.ibm.com> > > > > To: "Antoni Segura Puimedon" <asegu...@redhat.com> > > > > Cc: vdsm-devel@lists.fedorahosted.org > > > > Sent: Tuesday, December 11, 2012 6:42:45 AM > > > > Subject: Re: [vdsm] Fwd: Bonding, bridges and ifcfg > > > > > > > > On 12/10/2012 08:24 PM, Antoni Segura Puimedon wrote: > > > > > Hello everybody, > > > > > > > > > > We found some unexpected behavior with bonds and we'd like to > > > > > discuss it. > > > > > Please, read the forwarded messages. > > > > > > > > > > Best, > > > > > > > > > > Toni > > > > > > > > > > ----- Forwarded Message ----- > > > > >> From: "Dan Kenigsberg" <dan...@redhat.com> > > > > >> To: "Antoni Segura Puimedon" <asegu...@redhat.com> > > > > >> Cc: "Livnat Peer" <lp...@redhat.com>, "Igor Lvovsky" > > > > >> <ilvov...@redhat.com> > > > > >> Sent: Monday, December 10, 2012 1:03:48 PM > > > > >> Subject: Re: Bonding, ifcfg and luck > > > > >> > > > > >> On Mon, Dec 10, 2012 at 06:47:58AM -0500, Antoni Segura > > > > >> Puimedon > > > > >> wrote: > > > > >>> Hi all, > > > > >>> > > > > >>> I discussed this briefly with Livnat over the phone and > > > > >>> mentioned > > > > >>> it to Dan. > > > > >>> The issue that we have is that, if I understand correctly our > > > > >>> current > > > > >>> configNetwork, it could very well be that it works by means > > > > >>> of > > > > >>> good > > > > >>> design with > > > > >>> a side-dish of luck. > > > > >>> > > > > >>> I'll explain myself: > > > > >>> By design, as documented in > > > > >>> http://www.kernel.org/doc/Documentation/networking/bonding.txt: > > > > >>> "All slaves of bond0 have the same MAC address (HWaddr) as > > > > >>> bond0 > > > > >>> for all modes > > > > >>> except TLB and ALB that require a unique MAC address for each > > > > >>> slave." > > > > >>> > > > > >>> Thus, all operations on the slave interfaces after they are > > > > >>> added > > > > >>> to the bond > > > > >>> (except on TLB and ALB modes) that rely on ifcfg will fail > > > > >>> with a > > > > >>> message like: > > > > >>> "Device eth3 has different MAC address than expected, > > > > >>> ignoring.", > > > > >>> and no > > > > >>> ifup/ifdown will be performed. > > > > >>> > > > > >>> Currently, we were not noticing this, because we were > > > > >>> ignoring > > > > >>> completely > > > > >>> errors in ifdown and ifup, but > > > > >>> http://gerrit.ovirt.org/#/c/8415/ > > > > >>> shed light on > > > > >>> the matter. As you can see in the following example (bonding > > > > >>> mode > > > > >>> 4) the > > > > >>> behavior is just as documented: > > > > >>> > > > > >>> [root@rhel64 ~]# cat /sys/class/net/eth*/address > > > > >>> 52:54:00:a2:b4:50 > > > > >>> 52:54:00:3f:9b:28 > > > > >>> 52:54:00:51:50:49 > > > > >>> 52:54:00:ac:32:1b <----------------- > > > > >>> [root@rhel64 ~]# echo "+eth2" > > > > > >>> /sys/class/net/bond0/bonding/slaves > > > > >>> [root@rhel64 ~]# echo "+eth3" > > > > > >>> /sys/class/net/bond0/bonding/slaves > > > > >>> [root@rhel64 ~]# cat /sys/class/net/eth*/address > > > > >>> 52:54:00:a2:b4:50 > > > > >>> 52:54:00:3f:9b:28 > > > > >>> 52:54:00:51:50:49 > > > > >>> 52:54:00:51:50:49 <----------------- > > > > >>> [root@rhel64 ~]# echo "-eth3" > > > > > >>> /sys/class/net/bond0/bonding/slaves > > > > >>> [root@rhel64 ~]# cat /sys/class/net/eth*/address > > > > >>> 52:54:00:a2:b4:50 > > > > >>> 52:54:00:3f:9b:28 > > > > >>> 52:54:00:51:50:49 > > > > >>> 52:54:00:ac:32:1b <----------------- > > > > >>> > > > > >>> Obviously, this means that, for example, when we add a bridge > > > > >>> on > > > > >>> top of a bond, > > > > >>> the ifdown, ifup of the bond slaves will be completely > > > > >>> fruitless > > > > >>> (although > > > > >>> luckily that doesn't prevent them from working). > > > > >> > > > > >> Sorry, thi is not obvious to me. > > > > >> When we change something in a nic, we first take it down > > > > >> (which > > > > >> break > > > > >> it > > > > >> away from the bond), change it, and then take it up again (and > > > > >> back > > > > >> to > > > > >> the bond). > > > > >> > > > > >> I did not understand which flow of configuration leads us to > > > > >> the > > > > >> "unexpected mac" error. I hope that we can circumvent it. > > > > I get the same question. The warning message should be only seen > > > > when > > > > you run ifup on bonding or its slave, which is already up, > > > > otherwise > > > > the slave nic's mac address should hold its own permanent mac > > > > address. > > > > If the bonding is down before, you shouldn't see this message > > > > because the nic is not enslaved. > > > > > > It's easy enough to reproduce. To the current HEAD do: > > > > > > vdsClient 0 addNetwork bridge=kaboom bonding=bond0 nics=eth1,eth2 > > > bridged=True > > > > > > Go to /var/log/vdsm/vdsm.log and you'll see that: > > > http://pastebin.com/09eRNzv6 > > > > > > On line 33 you'll see that the ifup of eth2 is FAILED. > > > > Ok, I think this is a bug in the sequence of our ifup'ing. Reviewing > > ifup-eth I see that in its "if bonding device" stanza there's > > > > if [ "$ISALIAS" = no ] && is_bonding_device ${DEVICE} ; then > > ... > > for device in $(LANG=C egrep -l > > "^[[:space:]]*MASTER=\"?${DEVICE}\"?[[:space:]]*$" > > /etc/sysconfig/network-scripts/ifcfg-*) ; do > > is_ignored_file "$device" && continue > > /sbin/ifup ${device##*/} > > ... > > > > meaning that if we prepare ifcfg-eth* files on time, the interfaces > > would be > > brought up by means of `ifup bond??`; and it seems that there is no > > need > > for a specific `ifup eth0` at all. > > Yes this is the same conclusion I arrived when I looked at it earlier, > Basically on bonding case we can just not do the ifups of the underlying > devices.
s/we can/we should/ . And it would also save us some wasted time. _______________________________________________ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
