-----Original Message----- From: Tantilov, Emil S [mailto:[email protected]] Sent: Friday, May 31, 2013 2:09 PM To: JB Broccard; [email protected] Subject: RE: [E1000-devel] Bonding + ixgbe breaks with jumbo frames if the MTU is not set on bond0 before adding slaves
>-----Original Message----- >From: JB Broccard [mailto:[email protected]] >Sent: Friday, May 31, 2013 1:47 PM >To: [email protected] >Subject: Re: [E1000-devel] Bonding + ixgbe breaks with jumbo frames if >the MTU is not set on bond0 before adding slaves > > > > > >Ko, Stephen S <stephen.s.ko <at> intel.com> writes: > >> >> >> > -----Original Message----- >> > From: bmack [mailto:bomack08 <at> gmail.com] >> > Sent: Tuesday, October 02, 2012 11:22 AM >> > To: e1000-devel <at> lists.sourceforge.net >> > Subject: Re: [E1000-devel] Bonding + ixgbe breaks with jumbo frames >> > if >the >> > MTU is not set on bond0 before adding slaves >> > >> > >> > >> > Nathan March <nathan <at> gt.net> writes: >> > >> > > >> > > Hi All, >> > > >> > > I think I've found a bug in the ixgbe driver when using bonding + >> > > jumbo frames. Adding slaves to the bond device and setting mtu >> > > 9000 after enslaving, results in one of the slaves dropping >> > > traffic. The strange thing is putting bond0 into promiscuous mode >> > > (by running >> > > tcpdump) will solve the problem (until you close tcpdump). >> > > >> > >> > Hi, >> > >> > Seeing what looks to be the same issue with bonded interface and >> > jumbo frames on ixgbe driver for 82599 chip. Can someone please >> > point me to >the >> > fix for this and also the link to get history of fixes for ixgbe? >> > >> > Thanks! >> > >> > >> > >> > ------------------------------------------------------------------- >> > ---- >- >------ >> > Don't let slow site performance ruin your business. Deploy New >> > Relic >APM >> > Deploy New Relic app performance management and know exactly what >> > is happening inside your Ruby, Python, PHP, Java, and .NET app Try >> > New >Relic >> > at no cost today and get our sweet Data Nerd shirt too! >> > http://p.sf.net/sfu/newrelic-dev2dev >> > _______________________________________________ >> > E1000-devel mailing list >> > E1000-devel <at> lists.sourceforge.net >> > https://lists.sourceforge.net/lists/listinfo/e1000-devel >> > To learn more about Intel® Ethernet, visit >> > http://communities.intel.com/community/wired >> >> Hi, >> >> Fix is contained in 3.10.17: >http://sourceforge.net/projects/e1000/files/ixgbe%20stable/3.10.17/ >> >> Thanks, >> S >> >> --------------------------------------------------------------------- >> ---- >- >---- >> Don't let slow site performance ruin your business. Deploy New Relic >> APM Deploy New Relic app performance management and know exactly what >> is happening inside your Ruby, Python, PHP, Java, and .NET app Try >> New Relic at no cost today and get our sweet Data Nerd shirt too! >> http://p.sf.net/sfu/newrelic-dev2dev >> _______________________________________________ >> E1000-devel mailing list >> E1000-devel <at> lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/e1000-devel >> To learn more about Intel® Ethernet, visit >http://communities.intel.com/community/wired >> >> > >Hi All, > >On our servers, we are using the Intel chip 82599 and the same issues >happen even with the latest IXGBE driver available which is 3.15.1. If I remember correctly the issue in this thread had to do with the interfaces failing to pass traffic (unless in promisc). Are you not able to pass traffic after you set the MTU? > >This is what I do to reproduce the issue. > >I'm running OVM Server 3.0.3, kernel is 2.6.32.21-45xen > >eth2 and eth3 makes up the bonded interface bond0. Each interface are >connected to a different NEM/switch. eth2 is supposed to be the active >interface and eth3 the backup slave of my bond0, see ifcfg-bond0 file: >BONDING_OPTS="mode=1 miimon=100 use_carrier=0 primary=eth2 >primary_reselect=2" > >When I boot, the "Currently active slave" is eth2 as expected. >As soon as I change the MTU size (ifconfig bond0 mtu 9000), the bond0 >interface fails over to eth3. See messages: When you set the MTU the interface is reset which is causing the link to go down and the bonding driver to switch to the slave interface. If your issue is related to the switching of the active interface then perhaps you should change your config to set the MTU before bringing up the bonding interface. Thanks, Emil ------------------------------------------- Hi Emil, Thanks for the insights. When I set the MTU to 9000 in the ifcfg-bond0 file, the behavior is the same: i.e. the eth3 becomes active instead of eth2 (which is wrong in my case), see dmesg sequence: [ 45.163896] bonding: bond0: setting mode to active-backup (1). [ 45.164045] bonding: bond0: Setting MII monitoring interval to 100. [ 45.164186] bonding: bond0: Setting use_carrier to 0. [ 45.164323] bonding: bond0: Unable to set eth2 as primary slave as it is not a slave. [ 45.165061] bonding: bond0: setting primary_reselect to failure (2). [ 45.266019] bonding: bond0: Adding slave eth2. [ 45.424272] bonding: bond0: enslaving eth2 as a backup interface with a down link. [ 45.430611] ixgbe 0000:b0:00.0: eth2: NIC Link is Up 10 Gbps, Flow Control: RX/TX [ 45.466570] bonding: bond0: link status definitely up for interface eth2. [ 45.466573] bonding: bond0: making interface eth2 the new active one. [ 45.466977] bonding: bond0: first active interface up! [ 45.522509] bonding: bond0: Adding slave eth3. [ 45.704223] bonding: bond0: enslaving eth3 as a backup interface with a down link. [ 45.706726] bonding: bond0: Setting eth2 as primary slave. [ 45.708377] ixgbe 0000:b0:00.0: eth2: Setting MTU > 1500 will disable legacy VFs [ 45.708382] ixgbe 0000:b0:00.0: eth2: changing MTU from 1500 to 9000 [ 45.710628] ixgbe 0000:b0:00.1: eth3: NIC Link is Up 10 Gbps, Flow Control: RX/TX [ 47.951705] ixgbe 0000:b0:00.1: eth3: Setting MTU > 1500 will disable legacy VFs [ 47.951709] ixgbe 0000:b0:00.1: eth3: changing MTU from 1500 to 9000 [ 50.259697] ixgbe 0000:b0:00.1: eth3: NIC Link is Up 10 Gbps, Flow Control: RX/TX [ 50.259846] ixgbe 0000:b0:00.0: eth2: NIC Link is Up 10 Gbps, Flow Control: RX/TX [ 50.262100] bonding: bond0: link status definitely down for interface eth2, disabling it [ 50.262363] bonding: bond0: now running without any active interface ! [ 50.262365] bonding: bond0: link status definitely up for interface eth3. [ 50.262367] bonding: bond0: making interface eth3 the new active one. [ 50.262887] bonding: bond0: first active interface up! How can I keep using the Linux configuration files (modprobe.conf and ifcfg files) and make sure that I set the MTU to 9K before enslaving? Are you suggesting that I need to create a script that would coordinate this sequence (i.e. set the MTU first then enslave)? Thanks, JB ------------------------------------------------------------------------------ Get 100% visibility into Java/.NET code with AppDynamics Lite It's a free troubleshooting tool designed for production Get down to code-level detail for bottlenecks, with <2% overhead. Download for free and get started troubleshooting in minutes. http://p.sf.net/sfu/appdyn_d2d_ap2 _______________________________________________ E1000-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
