[E1000-devel] Bitrate throughput for Gigabit capped around 300Mbps
Hello everyone, I see a throughput cap of around 300 Mbps on my Linux router with two Intel Gigabit NICs. Packets are dropped inside the kernel. What could be the bottleneck? Configuration: -- I have a PC with an onboard NVIDIA GbE NIC, an Intel 82540OEM NIC and an Intel 82545EM NIC in PCI slots. The onboard NVIDIA NIC is disabled. None of the NICs share an interrupt. I am using ver 6.3.9 of the e1000 driver (I tried version 7.0.36 as well). The e1000 is NAPI enabled. Linux kernel version 2.6.16.13. Processor is AMD Opteron @ 2GHz. IP Forwarding is enabled. Execution: For performance analysis, I send packets from 64 bytes to 1470 bytes long through my router. At 64 bytes, I see packet forwarding rate of 360Kpps (about 184Mbps). At 256 bytes, I see a packet forwarding rate of 160Kpps (around 327Mbps). At 1470 bytes, I see a packet forwarding rate of 23Kpps (around 270Mbps). For higher rates, packets are being dropped inside the kernel (as per the ifconfig statistics). This seems bizarre since, with one Intel NIC and the onboard NVIDIA NIC (when its enabled), I get 230Kpps for 256 byte packets and 79Kpps at 1470bytes (about 920Mbps). I don't think this is a card issued since I swapped another card as well and got the same behaviour. Any insight will be very helpful. Thank you. Azeem Khan Mumbai. _ How fun is this? IMing with Windows Live Messenger just got better. http://www.microsoft.com/india/windows/windowslive/messenger.aspx-- Stay on top of everything new and different, both inside and around Java (TM) technology - register by April 22, and save $200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco. 300 plus technical and hands-on sessions. Register today. Use priority code J9JMT32. http://p.sf.net/sfu/p___ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel
Re: [E1000-devel] Bitrate throughput for Gigabit capped around 300Mbps
Hi Azeem! > I have a PC with an onboard NVIDIA GbE NIC, an Intel 82540OEM NIC and an > Intel 82545EM NIC in PCI slots. If it's in a PCI slot, could you please have a look in dmesg at the point where the e1000 driver module loads and could you check which kind of PCI interface the driver uses? Should look like this: > Intel(R) PRO/1000 Network Driver - version 7.6.15.5-NAPI > Copyright (c) 1999-2008 Intel Corporation. > PCI: Setting latency timer of device 01:00.0 to 64 > e1000: 01:00.0: e1000_probe: (PCI Express:2.5Gb/s:Width x4) 00:10:f3:0e:c5:d4 If it's only operating in PCI 33Mhz, 32 bit mode you'll never get more throughput than 300 - 400 mbit/s Frank -- Stay on top of everything new and different, both inside and around Java (TM) technology - register by April 22, and save $200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco. 300 plus technical and hands-on sessions. Register today. Use priority code J9JMT32. http://p.sf.net/sfu/p ___ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel
Re: [E1000-devel] Bitrate throughput for Gigabit capped around 300Mbps
Hello Frank, Thank you for replying. Yes indeed, the Intel NIC is operating at 33MHz and 32 bit mode. From lspci -v I had mistakenly concluded that the NIC was operating at 66MHz. It turns out that the reason I had very high throughput earlier with the onboard NIC is that the onboard NIC is connected directly to the southbridge chipset and therefore does not compete with the slot based NICs for bandwidth. As a result it is almost like having two separate PCI buses on a single system. I had not known this fact. Only a perusal of some chipset / IO hub technical documentation brought this to my attention. I confirmed this by testing with another PC's onboard Gigabit NIC and an add-on GbE card. So, with two Intel 82540EM Gigabit NICs on the PCI bus and some tweaking of params, I never cross 410Mbps of routing throughput (about 820Mbps of PCI I/O throughput). Guess the bad things people say about the PCI bus is true :-) I will start searching for PCIe cards next time I want to build a Gigabit router. Thank you again. Azeem Khan Mumbai. > From: f.da...@phion.com > To: azee...@hotmail.com; e1000-devel@lists.sourceforge.net > Date: Mon, 20 Apr 2009 08:54:14 +0200 > Subject: RE: [E1000-devel] Bitrate throughput for Gigabit capped around > 300Mbps > > Hi Azeem! > > > I have a PC with an onboard NVIDIA GbE NIC, an Intel 82540OEM NIC and an > > Intel 82545EM NIC in PCI slots. > > If it's in a PCI slot, could you please have a look in dmesg at the point > where the e1000 driver module loads and could you check which kind of PCI > interface the driver uses? > > Should look like this: > > > Intel(R) PRO/1000 Network Driver - version 7.6.15.5-NAPI > > Copyright (c) 1999-2008 Intel Corporation. > > PCI: Setting latency timer of device 01:00.0 to 64 > > e1000: 01:00.0: e1000_probe: (PCI Express:2.5Gb/s:Width x4) > > 00:10:f3:0e:c5:d4 > > If it's only operating in PCI 33Mhz, 32 bit mode you'll never get more > throughput than 300 - 400 mbit/s > > Frank _ So many new options, so little time. Windows Live Messenger. http://www.microsoft.com/india/windows/windowslive/messenger.aspx-- Stay on top of everything new and different, both inside and around Java (TM) technology - register by April 22, and save $200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco. 300 plus technical and hands-on sessions. Register today. Use priority code J9JMT32. http://p.sf.net/sfu/p___ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel
