Re: pf performance?

2013-04-26 Thread Erich Weiler
If this machine is dedicated for pf then setting sysctl net.isr.direct=0 might also improve performance, by forcing all packets to go through a single netisr thread (assuming that net.isr.maxthreads is 1). Note that this will apply to traffic that does not go through pf, so if this machine were d

Re: pf performance?

2013-04-26 Thread Erich Weiler
In other words, until I see like 100% system usage in one core, I would have room to grow? Probably not. Mutexes in FreeBSD use "adaptive spinning". This means that when a thread is unable to acquire a mutex, if the owner of the mutex is still running on another CPU core then the threa

Re: pf performance?

2013-04-26 Thread Erich Weiler
But the work pf does would show up in 'system' on top right? So if I see all my CPUs tied up 100% in 'interrupts' and very little 'system', would it be a reasonable assumption to think that if I got more CPU cores to handle the interrupts that eventually I would see 'system' load increase as the

Re: pf performance?

2013-04-26 Thread Erich Weiler
The pf isn't a process, so you can't see it in top. pf has some helper threads however, but packet processing isn't performed by any of them. But the work pf does would show up in 'system' on top right? So if I see all my CPUs tied up 100% in 'interrupts' and very little 'system', would it be

Re: pf performance?

2013-04-25 Thread Erich Weiler
How many pf rules do you have?. And, as I asked in my previous post, do you create states on both sides of the firewall? One interface has 12 rules and other other interface has one rule. We do create states on both sides. ___ freebsd-net@freebsd.or

Re: pf performance?

2013-04-25 Thread Erich Weiler
... please ask the pfsense guys to either migrate to -9, or backport the -head pf (with the locking fixes!) to -8 for that. Otherwise you're very likely going to be wasting time on something you can't really push that much harder. I can ask for that (and will soon, likely), but to play with my

Re: pf performance?

2013-04-25 Thread Erich Weiler
As far as I understand, processing of packets by pf takes place in receiving network card's interrupt handler even up to sending the packet via another network card (at least in my case, when using route-to targets, which make routing inside pf). That's interesting. So even though pf is giant l

pf performance?

2013-04-23 Thread Erich Weiler
Hello all, I have a question here about how FreeBSD (8.1-RELEASE-p13 specifically) behaves when acting as a firewall. I understand the pf process is "giant locked" to a single CPU core when inspecting packets inbound and outbound. I was wondering, how does that manifest when I look at "top

Re: Arg. TCP slow start killing me.

2011-11-13 Thread Erich Weiler
Have you considered empty ACK prioritization? I implemented this a year ago on a pair of production edge routers and noticed significant improvement on throughput. I have production code examples if you require them, but this link should be more than enough to get you started: Fascinating. pfSe

Re: Arg. TCP slow start killing me.

2011-11-13 Thread Erich Weiler
Actually, another question might be: How can I prove that my firewall really is dropping packets in transit, as it forwards them on? Is there some sysctl oid that would show dropped packets, so some netstat counter I can look at? On 11/13/11 4:22 PM, Erich Weiler wrote: Yeah, skimming fail

Re: Arg. TCP slow start killing me.

2011-11-13 Thread Erich Weiler
Yeah, skimming fail, I didn't realize the machine was not the termination point of your connections. I do have patches back ported that would likely get the modular congestion control working on 8.1, but neither my suggestions nor the implementation of Cubic will help much as mentioned. Given t

Re: Arg. TCP slow start killing me.

2011-11-13 Thread Erich Weiler
I suspect my firewall *is* the cause of the packet loss, unfortunately. We're sending multiple streams in from multiple sources and destinations, but the aggregate bandwidth coming into the firewall is consistent no matter how many sources and destinations we have. It maxes at about 2Gb/s. That's

Re: Arg. TCP slow start killing me.

2011-11-13 Thread Erich Weiler
If your firewall isn't the cause of the packet losses, then you don't really have much control -- the TCP source(s) _will_ detect the packet losses, either due to duplicate ACKs coming back from the destination or timeout waiting for ACK from destination. I suspect my firewall *is* the cause o

Re: Arg. TCP slow start killing me.

2011-11-13 Thread Erich Weiler
Thanks Jason! Is it possible to upgrade to 8.2-STABLE? Cubic has shown some really great improvement in my latent paths, a steady 10% overall increase in same cases. Alas no, my OS is a pre-baked install for pfSense, and if I tried to upgrade it, it would likely break some of the functionali

Arg. TCP slow start killing me.

2011-11-13 Thread Erich Weiler
So, I have a FreeBSD 8.1 box that I'm using as a firewall (pfSense 2.0 really, which uses 8.1 as a base), and I'm filtering packets inbound and I'm seeing a typical sawtooth pattern where I get high bandwidth, then a packet drops somewhere, and the TCP connections back off a *lot*, then slowly

net.isr.maxthreads tunable?

2011-11-12 Thread Erich Weiler
Greetings! I was looking at this page on BSD firewalling at: https://calomel.org/network_performance.html and got pretty far in it for tuning some pf stuff but am having a couple tuning issues... I'm using FreeBSD 8.1 amd64. I was able to set most the the loader.conf stuff, but "net.isr.ma