Ryan McBride wrote: > On Thu, Mar 10, 2011 at 12:18:32PM +0000, Tom Murphy wrote: >> I had a pair of Dell PowerEdge R200s that have both em(4) and bge(4)s >> in them, however, it's the em(4) doing the heavy lifting. Roughly 30-40 >> megabits/s sustained and doing anywhere between 3000-4000 packets/s. >> >> On OpenBSD 4.4, it happily forwards packets along. I upgraded one of >> the firewalls to 4.8 and switched CARP over to it (yes, I know the >> redundancy is broken anyway now.) and it couldn't seem to handle the >> traffic. Any inbound connections would stall and I have no idea why. > >I assume that you don't have the 'defer' option set on your pfsync >interface (it would be broken until you upgrade both firewalls)
Correct. The defer option is off by default and when I looked at pfsync0 on the 4.8 box it said: pfsync: syncdev: bge1 maxupd: 128 defer: off >> There were no net.inet.ip.ifq.drops, but I noticed 10 livelocks when >> running systat mbufs (on em0). > > >I think in 4.8 systat mbufs is showing the total number of livelocks >ever, and 10 is a tiny number. On a system nearing it's limit you could >expect the livelocks counter to get hit a few times a second, but if >it's getting hit 50 times per second you're way over capacity. Yeah I only had 10 after about 3-4 hours and the number did not increase. >Note you can also look at 'sysctl kern.netlivelocks' which is a little >less ambiguous, and shows the total number of livelocks since boot. Thanks! I will bear that in mind. >> Could MCLGETI be hindering performance? > >I'm doing a lot of testing in this area these days on a broad range of >hardware, and I have yet to find a case where MCLGETI does not improve >a system's ability to handle load. If anything MCLGETI needs to be more >aggressive, and we're looking at ways to do that. I notice the machines are mostly idle.. between 90-95%. They also use very little memory (top reports 15-18M of memory used). The 4.8 box only has 1 gig of RAM, whereas the 4.4 box has 2 gig. It doesn't seem to make much of a difference in this case. Whichever firewall is active can handle upwards to about 62000 states during peak times. Would it be worth just shutting down pfsync(4) on both machines to test performance? I wouldn't want pfsync getting in the way since pfsync is broken anyway. It would be one more variable to remove from the equation. Tom
