I'm curious what others are doing to achieve high connection rates -say 10Kconnections/ second.
We're serving objects averaging around 100KB, so 10K/sec is a fully utilized 1G ethernet card. I'd like to safely hit 7-800 Mb/sec, but interrupts are just eating the machine alive. Before adjusting the ethernet irq to allow interrupts to be delivered to either core instead of just cpu1, I was hitting a limit right around 480Mb/sec, cpu1 was taxed out servicing both hardware and software interrupts. I adjusted the ethernet card's IRQ to remove it's affinity to cpu1 and now the limit is around 560Mb/sec before the machine starts dropping packets. I did this against the advice that this could cause cpu cache misses. machine is: Intel(R) Core(TM)2 Duo CPU E7200 @ 2.53GHz os: fedora 10 2.6.27.38-170.2.113.fc10.x86_64 #1 SMP card: Intel Corporation 82573L Gigabit Ethernet Controller I've had some ideas on cutting down interrupts: - jumbo frames behind haproxy (inside my network) - LRO enabled cards (not even sure what this is yet) I'm not even exactly sure which cards support either of these features yet. Also, an msi-x card sound like it might reduce interrupts, but I'm uncertain....might be trying these soonest. Here's some net kernel settings. sysctl -A | grep net > http://pastebin.com/m26e88d16 I also have everything out of /proc/net/nestat graphed for the last few weeks if anybody wants to see. Is this the best I can expect out of the card, the machine and the kernel? Are there any amount of tuning that can alleviate this?
