Gents, I'm having quite an aweful problem that I need a bit of help with.
I have an HPDL360 G3 ( http://h18000.www1.hp.com/products/quickspecs/11504_na/11504_na.HTML ) which acts as a NAT (via PF) for several (600+) class C's amongst 24+ machines sitting behind it. It's running FPSense (FreeBSD 8.1-RELEASE-p4). The important guts are: 2 x 2.8 GHz Cpus 2 BGE interfaces on a PCI-X bus. During peak times this machine is only able to handle between 500Mbps - 600Mbps before running out of cpu capacity. (300Mbps(ish) on the LAN, 300Mbps(ish) on the WAN) It's due to the high number of interrupts. I was speaking with a networking engineer here and he mentioned that I should look at "Interrupt Coalescing" to increase throughput. The only information I found online regarding this was a post from 2 years ago here: http://lists.freebsd.org/pipermail/freebsd-net/2009-June/022227.html The tunables mentioned in the above post aren't present in my system, so I imagine this never made it into the bge driver. Assuming this to be the case, I started looking at DEVICE_POLLING as a solution. I did try implementing device polling, but the results were worse than I expected. netisr was using 100% of a single cpu while the other cpu remained mostly idle. Not knowing exactly what netisr is, I reverted the changes. This leads me to this list. Given the scenario above, I'm nearly certain I need to use device polling instead of the standard interrupt driven setup. The two sysctl's that I've come across thus far that I think are what I need are: net.isr.maxthreads hern.hz I would assume setting net.isr.maxthreads to 2 given my dual core machine is advisable, but I'm not 100% sure. What are the caveats in setting this higher? Given the output of `sysctl -d net.isr.maxthreads` I would expect anything higher than the number of cores to be detrimental. Is this correct? kern.hz I'm more unsure of. I understand what the sysctl is, but I'm not sure how to come up with a reasonable number. Generally speaking, and in your experience, would a setting of 2000 achive close to the theoritical meximum of the cards? Is there an upper limit that I would be worried about? Random Question: - is device polling really the answer? I am missing something in the bge driver that I've overlooked? - what tunables directly effect processing high volumes of packets. Network Interfaces: ################################################################################## bge0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE> ether 00:0b:cd:ca:1d:1a inet 209.18.70.211 netmask 0xffffff00 broadcast 209.18.70.255 inet6 fe80::20b:cdff:feca:1d1a%bge0 prefixlen 64 scopeid 0x1 nd6 options=3<PERFORMNUD,ACCEPT_RTADV> media: Ethernet autoselect (1000baseT <full-duplex>) status: active bge1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE> ether 00:0b:cd:ca:1a:74 inet 172.16.0.3 netmask 0xfffc0000 broadcast 172.19.255.255 inet6 fe80::20b:cdff:feca:1a74%bge1 prefixlen 64 scopeid 0x2 nd6 options=3<PERFORMNUD,ACCEPT_RTADV> media: Ethernet autoselect (1000baseT <full-duplex>) status: active ################################################################################## I appreciate the help in advance. Thanks, Paul ________________________________ This message may contain confidential or privileged information. If you are not the intended recipient, please advise us immediately and delete this message. See http://www.datapipe.com/about-us-legal-email-disclaimer.htm for further information on confidentiality and the risks of non-secure electronic communication. If you cannot access these links, please notify us by reply message and we will send the contents to you. _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
