-----Original Message----- From: Tantilov, Emil S [mailto:[email protected]] Sent: Friday, May 31, 2013 2:52 PM To: J.B. Broccard; [email protected] Subject: RE: [E1000-devel] Bonding + ixgbe breaks with jumbo frames if the MTU is not set on bond0 before adding slaves
>-----Original Message----- >From: J.B. Broccard [mailto:[email protected]] >Sent: Friday, May 31, 2013 2:26 PM >To: Tantilov, Emil S; [email protected] >Subject: RE: [E1000-devel] Bonding + ixgbe breaks with jumbo frames if >the MTU is not set on bond0 before adding slaves > > > >-----Original Message----- >From: Tantilov, Emil S [mailto:[email protected]] >Sent: Friday, May 31, 2013 2:09 PM >To: JB Broccard; [email protected] >Subject: RE: [E1000-devel] Bonding + ixgbe breaks with jumbo frames if >the MTU is not set on bond0 before adding slaves > >>-----Original Message----- >>From: JB Broccard [mailto:[email protected]] >>Sent: Friday, May 31, 2013 1:47 PM >>To: [email protected] >>Subject: Re: [E1000-devel] Bonding + ixgbe breaks with jumbo frames if >>the MTU is not set on bond0 before adding slaves >> >> >> >> >> >>Ko, Stephen S <stephen.s.ko <at> intel.com> writes: >> >>> >>> >>> > -----Original Message----- >>> > From: bmack [mailto:bomack08 <at> gmail.com] >>> > Sent: Tuesday, October 02, 2012 11:22 AM >>> > To: e1000-devel <at> lists.sourceforge.net >>> > Subject: Re: [E1000-devel] Bonding + ixgbe breaks with jumbo >>> > frames if >>the >>> > MTU is not set on bond0 before adding slaves >>> > >>> > >>> > >>> > Nathan March <nathan <at> gt.net> writes: >>> > >>> > > >>> > > Hi All, >>> > > >>> > > I think I've found a bug in the ixgbe driver when using bonding >>> > > + jumbo frames. Adding slaves to the bond device and setting mtu >>> > > 9000 after enslaving, results in one of the slaves dropping >>> > > traffic. The strange thing is putting bond0 into promiscuous >>> > > mode (by running >>> > > tcpdump) will solve the problem (until you close tcpdump). >>> > > >>> > >>> > Hi, >>> > >>> > Seeing what looks to be the same issue with bonded interface and >>> > jumbo frames on ixgbe driver for 82599 chip. Can someone please >>> > point me to >>the >>> > fix for this and also the link to get history of fixes for ixgbe? >>> > >>> > Thanks! >>> > >>> > >>> > >>> > ------------------------------------------------------------------ >>> > - >>> > ---- >>- >>------ >>> > Don't let slow site performance ruin your business. Deploy New >>> > Relic >>APM >>> > Deploy New Relic app performance management and know exactly what >>> > is happening inside your Ruby, Python, PHP, Java, and .NET app Try >>> > New >>Relic >>> > at no cost today and get our sweet Data Nerd shirt too! >>> > http://p.sf.net/sfu/newrelic-dev2dev >>> > _______________________________________________ >>> > E1000-devel mailing list >>> > E1000-devel <at> lists.sourceforge.net >>> > https://lists.sourceforge.net/lists/listinfo/e1000-devel >>> > To learn more about Intel® Ethernet, visit >>> > http://communities.intel.com/community/wired >>> >>> Hi, >>> >>> Fix is contained in 3.10.17: >>http://sourceforge.net/projects/e1000/files/ixgbe%20stable/3.10.17/ >>> >>> Thanks, >>> S >>> >>> -------------------------------------------------------------------- >>> - >>> ---- >>- >>---- >>> Don't let slow site performance ruin your business. Deploy New Relic >>> APM Deploy New Relic app performance management and know exactly >>> what is happening inside your Ruby, Python, PHP, Java, and .NET app >>> Try New Relic at no cost today and get our sweet Data Nerd shirt too! >>> http://p.sf.net/sfu/newrelic-dev2dev >>> _______________________________________________ >>> E1000-devel mailing list >>> E1000-devel <at> lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/e1000-devel >>> To learn more about Intel® Ethernet, visit >>http://communities.intel.com/community/wired >>> >>> >> >>Hi All, >> >>On our servers, we are using the Intel chip 82599 and the same issues >>happen even with the latest IXGBE driver available which is 3.15.1. > >If I remember correctly the issue in this thread had to do with the >interfaces failing to pass traffic (unless in promisc). Are you not >able to pass traffic after you set the MTU? > >> >>This is what I do to reproduce the issue. >> >>I'm running OVM Server 3.0.3, kernel is 2.6.32.21-45xen >> >>eth2 and eth3 makes up the bonded interface bond0. Each interface are >>connected to a different NEM/switch. eth2 is supposed to be the active >>interface and eth3 the backup slave of my bond0, see ifcfg-bond0 file: >>BONDING_OPTS="mode=1 miimon=100 use_carrier=0 primary=eth2 >>primary_reselect=2" >> >>When I boot, the "Currently active slave" is eth2 as expected. >>As soon as I change the MTU size (ifconfig bond0 mtu 9000), the bond0 >>interface fails over to eth3. See messages: > >When you set the MTU the interface is reset which is causing the link >to go down and the bonding driver to switch to the slave interface. If >your issue is related to the switching of the active interface then >perhaps you should change your config to set the MTU before bringing up the >bonding interface. > >Thanks, >Emil > >------------------------------------------- > >Hi Emil, > >Thanks for the insights. >When I set the MTU to 9000 in the ifcfg-bond0 file, the behavior is the >same: i.e. the eth3 becomes active instead of eth2 (which is wrong in >my case), see dmesg sequence: >[ 45.163896] bonding: bond0: setting mode to active-backup (1). >[ 45.164045] bonding: bond0: Setting MII monitoring interval to 100. >[ 45.164186] bonding: bond0: Setting use_carrier to 0. >[ 45.164323] bonding: bond0: Unable to set eth2 as primary slave as it is >not a slave. >[ 45.165061] bonding: bond0: setting primary_reselect to failure (2). >[ 45.266019] bonding: bond0: Adding slave eth2. >[ 45.424272] bonding: bond0: enslaving eth2 as a backup interface with a >down link. >[ 45.430611] ixgbe 0000:b0:00.0: eth2: NIC Link is Up 10 Gbps, Flow >Control: RX/TX >[ 45.466570] bonding: bond0: link status definitely up for interface >eth2. >[ 45.466573] bonding: bond0: making interface eth2 the new active one. >[ 45.466977] bonding: bond0: first active interface up! >[ 45.522509] bonding: bond0: Adding slave eth3. >[ 45.704223] bonding: bond0: enslaving eth3 as a backup interface with a >down link. >[ 45.706726] bonding: bond0: Setting eth2 as primary slave. >[ 45.708377] ixgbe 0000:b0:00.0: eth2: Setting MTU > 1500 will disable >legacy VFs >[ 45.708382] ixgbe 0000:b0:00.0: eth2: changing MTU from 1500 to 9000 >[ 45.710628] ixgbe 0000:b0:00.1: eth3: NIC Link is Up 10 Gbps, Flow >Control: RX/TX >[ 47.951705] ixgbe 0000:b0:00.1: eth3: Setting MTU > 1500 will disable >legacy VFs >[ 47.951709] ixgbe 0000:b0:00.1: eth3: changing MTU from 1500 to 9000 >[ 50.259697] ixgbe 0000:b0:00.1: eth3: NIC Link is Up 10 Gbps, Flow >Control: RX/TX >[ 50.259846] ixgbe 0000:b0:00.0: eth2: NIC Link is Up 10 Gbps, Flow >Control: RX/TX >[ 50.262100] bonding: bond0: link status definitely down for interface >eth2, disabling it >[ 50.262363] bonding: bond0: now running without any active interface ! >[ 50.262365] bonding: bond0: link status definitely up for interface >eth3. >[ 50.262367] bonding: bond0: making interface eth3 the new active one. >[ 50.262887] bonding: bond0: first active interface up! > >How can I keep using the Linux configuration files (modprobe.conf and >ifcfg >files) and make sure that I set the MTU to 9K before enslaving? > >Are you suggesting that I need to create a script that would coordinate >this sequence (i.e. set the MTU first then enslave)? I think that configuring the MTU in ifcfg-ethX should be sufficient. Thanks, Emil > >Thanks, >JB Emil, Thanks again for considering my issue. So, I tried to set MTU=9000 in ifcfg-eth2 and 3, tried to restart the network (unload/reload ixgbe and bonding modules), even tried to reboot the system ... the system seems to ignore the parameter, eth2/3 as well as bond0 are still at 1500. I currently have primary_reselect set to 2 and we don't want to change this to 0 (automatic failback). Any other idea? Thanks, JB ------------------------------------------------------------------------------ Get 100% visibility into Java/.NET code with AppDynamics Lite It's a free troubleshooting tool designed for production Get down to code-level detail for bottlenecks, with <2% overhead. Download for free and get started troubleshooting in minutes. http://p.sf.net/sfu/appdyn_d2d_ap2 _______________________________________________ E1000-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
