So this same setup using the VLANs doesn't work on the 82546 NICs? Or has this not been tried?
Cheers, John ----------------------------------------------------------- "...that your people will judge you on what you can build, not what you destroy.", B. Obama, 2009 >-----Original Message----- >From: Support Team [mailto:[email protected]] >Sent: Friday, March 27, 2009 11:32 AM >To: Ronciak, John; 'Jay Vosburgh' >Cc: [email protected] >Subject: RE: [E1000-devel] Bug report E1000 driver bonding in >802.3ad mode can not go beyond 1GB/s throughput > >It is 82573 NIC built-in on the motherboard working with E1000E driver. >The 82573 board is powered by core 2 dual 1.6GHz. The 82546 >based MT server >adapter >is powered by Dual Xeon 3.6GHz. There are two VLANs on the bonded >interface, one VLAN >has the clients that generate the load, the other VLAN on the bonded >interfaces >talks to the servers. > >All the clients and servers and switches are same during the testing. > >-----Original Message----- >From: Ronciak, John [mailto:[email protected]] >Sent: Friday, March 27, 2009 11:15 AM >To: [email protected]; 'Jay Vosburgh' >Cc: [email protected] >Subject: [work] RE: [E1000-devel] Bug report E1000 driver >bonding in 802.3ad >mode can not go beyond 1GB/s throughput > >OK one last question. When you say you are using the e1000e >driver what >NICs (devices) are being used? > >I'll ask around to see if any one here has done anything with bonding >lately. This could some how be releated to you trying to route packets >through a bonded interface. > >Cheers, >John >----------------------------------------------------------- >"...that your people will judge you on what you can build, not what you >destroy.", B. Obama, 2009 > > >>-----Original Message----- >>From: Support Team [mailto:[email protected]] >>Sent: Friday, March 27, 2009 10:19 AM >>To: Ronciak, John; 'Jay Vosburgh' >>Cc: [email protected] >>Subject: RE: [E1000-devel] Bug report E1000 driver bonding in >>802.3ad mode can not go beyond 1GB/s throughput >> >>John, >> >>8257x device work fine on a less powerful HW system with same >>network cable, >>switch, clients and servers. >> >>"HW in between" is the Hw you wanted me to described. It is a >>motherboard >>with Intel NIC running in bond >>mode to act as a router. Both 82546 and 8257x are tested on >this "HW in >>between". 8257x based system >>is less powerful CPU and less memory but using E1000E driver. >It scales >>well: with 1GB load, it will >>pass 1GB, with 2GB load, it will pass 2GB. 82546 NIC based >>Intel Pro/1000 >>MT server adapter based HW >>has more powerful CPU and more memory. When we apply 1GB load, >>it will pass >>1GB. When we apply 2GB/s load, >>it still only pass 1GB/s. >> >>Both UUTs have exactly same kernel configuration, except one >>loaded with >>E1000E driver and the other loaded >>with E1000 driver. All NICs in bonding are full duplex >>reported by ethtool. >> >>Thanks, >>Wayne >> >>-----Original Message----- >>From: Ronciak, John [mailto:[email protected]] >>Sent: Friday, March 27, 2009 9:55 AM >>To: [email protected]; 'Jay Vosburgh' >>Cc: [email protected] >>Subject: [work] RE: [E1000-devel] Bug report E1000 driver >>bonding in 802.3ad >>mode can not go beyond 1GB/s throughput >> >>So you are saying that the 8257x devices work fine on the same >>systems that >>the 82546 ones don't work in? Running the same kernel, HW, >>wiring, etc. >>I'm just trying to get a picture of what is going on. >> >>Also, can you please explain this comment: >>>Without this HW in between, we can get 4GB/s throughput from clients >>>directly to servers through the >>>switch. >>What HW in between? I can't seem to picture how you have >>things set up. >> >>Cheers, >>John >>----------------------------------------------------------- >>"...that your people will judge you on what you can build, >not what you >>destroy.", B. Obama, 2009 >> >> >>>-----Original Message----- >>>From: Support Team [mailto:[email protected]] >>>Sent: Friday, March 27, 2009 9:27 AM >>>To: Ronciak, John; 'Jay Vosburgh' >>>Cc: [email protected] >>>Subject: RE: [E1000-devel] Bug report E1000 driver bonding in >>>802.3ad mode can not go beyond 1GB/s throughput >>> >>>John, >>> >>>The E1000 driver(V8.0.6) is the latest one from Intel web >>>site, which moved >>>out all the E1000E support to a >>>separate E1000E driver. The OS is Linux 2.4 with latest >>>kernel build from >>>kernel.org. >>>HW is dual Xeon processor with PCI-X buses on the north bridge. >>>Clients are four Load Runner machines and servers are four >>web servers. >>>Going through a Netgear switch. >>>Without this HW in between, we can get 4GB/s throughput from clients >>>directly to servers through the >>>switch. >>> >>>With the E1000 driver bonding together we expect to see the >>>traffic scale >>>up. But somehow hit a limit >>>at 1GB/s. Well your new E1000E driver works nicely on the >>>8257x chps on the >>>same test environment. >>> >>>Thanks for your help, >>>Wayne >>> >>>-----Original Message----- >>>From: Ronciak, John [mailto:[email protected]] >>>Sent: Friday, March 27, 2009 9:07 AM >>>To: [email protected]; 'Jay Vosburgh' >>>Cc: [email protected] >>>Subject: [work] RE: [E1000-devel] Bug report E1000 driver >>>bonding in 802.3ad >>>mode can not go beyond 1GB/s throughput >>> >>>We are working on the support issue. I lead the team >>>responsible for the >>>e1000 driver. Since this is old HW and has been out in the >>>field for a very >>>long time I have some questions for you. >>>What is the OS and driver version being used? >>>What kind of system (HW-wise) is being used? >>>What is the networking setup like? (like clients, switches, etc.) >>> >>>So without bonding but using multiple interfaces, can you >run iperf or >>>netperf on each interface each with multiple streams so that >>>each interface >>>is running to a bandwidth of 1 gigabit? I have a hard time >>>believing that >>>this can't be done as this is a very simple test to do and has >>>been working >>>for years without problem. >>> >>>Running one adapter to 1 gigabit should not be taking 25% of >>>the CPU unless >>>this is on some very old HW. So something looks wrong right there. >>> >>> >>>Cheers, >>>John >>>----------------------------------------------------------- >>>"...that your people will judge you on what you can build, >>not what you >>>destroy.", B. Obama, 2009 >>> >>> >>>>-----Original Message----- >>>>From: Support Team [mailto:[email protected]] >>>>Sent: Friday, March 27, 2009 8:52 AM >>>>To: Ronciak, John; 'Jay Vosburgh' >>>>Cc: [email protected] >>>>Subject: RE: [E1000-devel] Bug report E1000 driver bonding in >>>>802.3ad mode can not go beyond 1GB/s throughput >>>> >>>>John, >>>> >>>>Did you get my email about the person's name in Intel? I >>>>think finding his >>>>name is not that >>>>important. Finding the problem and solution is more important. >>>> >>>>RD >>>> >>>>-----Original Message----- >>>>From: Ronciak, John [mailto:[email protected]] >>>>Sent: Friday, March 27, 2009 8:42 AM >>>>To: [email protected]; 'Jay Vosburgh' >>>>Cc: [email protected] >>>>Subject: [work] RE: [E1000-devel] Bug report E1000 driver >>>>bonding in 802.3ad >>>>mode can not go beyond 1GB/s throughput >>>> >>>>>Intel engineers told us that if not in 802.3ad mode, the >>>>>throughput will be >>>>>limited to 1GB/s. >>>>>But we are setting up the 802.3ad mode on these 82546 chips. >>>> >>>>Who at Intel are you talking to? >>>> >>>>Cheers, >>>>John >>>>----------------------------------------------------------- >>>>"...that your people will judge you on what you can build, >>>not what you >>>>destroy.", B. Obama, 2009 >>>> >>>> >>>>>-----Original Message----- >>>>>From: Support Team [mailto:[email protected]] >>>>>Sent: Friday, March 27, 2009 8:32 AM >>>>>To: 'Jay Vosburgh'; [email protected] >>>>>Cc: [email protected] >>>>>Subject: Re: [E1000-devel] Bug report E1000 driver bonding in >>>>>802.3ad mode can not go beyond 1GB/s throughput >>>>> >>>>>Hi Jay, >>>>> >>>>>Our date stream to the bond interfaces are sending requests >>>>>and data replyes >>>>>at 4GB/s. >>>>>The xmit_hash_policy is L3+4. >>>>> >>>>>However, we noticed that if we just apply 1GB/s load, the CPU >>>>>usage is about >>>>>25%. Adding >>>>>more load will increase the CPU usage all the way to 100%, but the >>>>>throughput would not >>>>>go up at all. >>>>> >>>>>Intel engineers told us that if not in 802.3ad mode, the >>>>>throughput will be >>>>>limited to 1GB/s. >>>>>But we are setting up the 802.3ad mode on these 82546 chips. >>>>> >>>>>What else do you think can cause this limit? Just for make >>>>>this clear, we >>>>>use the >>>>>E1000E driver with 8257x chip can get throughput scale well, >>>with same >>>>>kernel configuration >>>>>and testing environment. >>>>> >>>>>Thanks! >>>>>Wayne >>>>> >>>>>-----Original Message----- >>>>>From: Jay Vosburgh [mailto:[email protected]] >>>>>Sent: Friday, March 27, 2009 7:02 AM >>>>>To: [email protected] >>>>>Cc: [email protected] >>>>>Subject: [work] Re: [E1000-devel] Bug report E1000 driver >>>>>bonding in 802.3ad >>>>>mode can not go beyond 1GB/s throughput >>>>> >>>>>Support Team <[email protected]> wrote: >>>>> >>>>>>Hello, >>>>>> >>>>>>Intel [email protected] recommended us to >>>>>open a ticket >>>>>>with you that >>>>>>your e1000 driver for 82546 chips has throughput limit. >>With 802.3ad >>>>>>bonding, the total >>>>>>throughput of 8 NIC is still 1GB/s, same as single NIC. >>>>> >>>>> How are you testing throughput? If you're only running a single >>>>>stream test, you'll only see the throughput of one adapter. >>>>This is by >>>>>design, the 802.3ad standard requires that a given >>>"conversation" (TCP >>>>>connection, stream of UDP packets to/from the same ports, >>>etc) be sent >>>>>across the same slave adapter. This is done to prevent >>reordering of >>>>>packets within the conversation. >>>>> >>>>> If you're running multiple streams, then you may want to set the >>>>>xmit_hash_policy option to layer3+4 or layer2+3. The layer3+4 >>>>>hash will >>>>>place multiple streams between the same two peer systems >on multiple >>>>>slaves (with a small risk of packet reordering if IP fragments are >>>>>generated); the layer2+3 won't, but will place all traffic >>>for a given >>>>>peer on the same slave (but balances better than the default layer2 >>>>>hash). >>>>> >>>>> The hashes are described in detail in the bonding.txt >>>>>documentation supplied with the kernel source. >>>>> >>>>> -J >>>>> >>>>>--- >>>>> -Jay Vosburgh, IBM Linux Technology Center, [email protected] >>>>> >>>>> >>>>> >>>>>--------------------------------------------------------------- >>>>>--------------- >>>>>_______________________________________________ >>>>>E1000-devel mailing list >>>>>[email protected] >>>>>https://lists.sourceforge.net/lists/listinfo/e1000-devel >>>>> >>>> >>>> >>> >>> >> >> > > ------------------------------------------------------------------------------ _______________________________________________ E1000-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/e1000-devel
