Hi Jay, Our date stream to the bond interfaces are sending requests and data replyes at 4GB/s. The xmit_hash_policy is L3+4.
However, we noticed that if we just apply 1GB/s load, the CPU usage is about 25%. Adding more load will increase the CPU usage all the way to 100%, but the throughput would not go up at all. Intel engineers told us that if not in 802.3ad mode, the throughput will be limited to 1GB/s. But we are setting up the 802.3ad mode on these 82546 chips. What else do you think can cause this limit? Just for make this clear, we use the E1000E driver with 8257x chip can get throughput scale well, with same kernel configuration and testing environment. Thanks! Wayne -----Original Message----- From: Jay Vosburgh [mailto:[email protected]] Sent: Friday, March 27, 2009 7:02 AM To: [email protected] Cc: [email protected] Subject: [work] Re: [E1000-devel] Bug report E1000 driver bonding in 802.3ad mode can not go beyond 1GB/s throughput Support Team <[email protected]> wrote: >Hello, > >Intel [email protected] recommended us to open a ticket >with you that >your e1000 driver for 82546 chips has throughput limit. With 802.3ad >bonding, the total >throughput of 8 NIC is still 1GB/s, same as single NIC. How are you testing throughput? If you're only running a single stream test, you'll only see the throughput of one adapter. This is by design, the 802.3ad standard requires that a given "conversation" (TCP connection, stream of UDP packets to/from the same ports, etc) be sent across the same slave adapter. This is done to prevent reordering of packets within the conversation. If you're running multiple streams, then you may want to set the xmit_hash_policy option to layer3+4 or layer2+3. The layer3+4 hash will place multiple streams between the same two peer systems on multiple slaves (with a small risk of packet reordering if IP fragments are generated); the layer2+3 won't, but will place all traffic for a given peer on the same slave (but balances better than the default layer2 hash). The hashes are described in detail in the bonding.txt documentation supplied with the kernel source. -J --- -Jay Vosburgh, IBM Linux Technology Center, [email protected] ------------------------------------------------------------------------------ _______________________________________________ E1000-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/e1000-devel
