On Sun, Jan 08, 2012 at 01:50:54PM -0600, Graham Allan wrote:
> On 1/7/2012 4:48 AM, Stuart Henderson wrote:
> >In gmane.os.openbsd.misc, you wrote:
> >>I'm trying to troubleshoot some performance issues for high speed data
> >>transfers across a long network path with a fairly high bandwidth delay.
> >
> >As a test to help pinpoint things, can you try passing the traffic
> >near the top of your ruleset with 'pass quick..flags any no state'?
> >Or with PF ddisabled if that's possible?
>
> Their iperf server was down for a while, but I was able to do a
> quick test with pf disabled today, with no change in behavior.
>
> However I did realize something which might be interesting.
>
> In my tcp iperf tests to the distant location thus far, I had been
> specifying a huge tcp buffer ("-w 30M", based on the approx BDP). I
> think doing this disables much of the tcp tuning in the linux
> kernel, so perhaps it's a bad idea... retesting without that, I get
> some quite different results:
>
> original, with 30M buffer:
> through router: 10/80 Mbps (in/out)
> bypassing router: 400/750 Mbps (in/out)
>
> permitting linux tcp tuning:
> through router: 300/80 Mbps (in/out)
> bypassing router: 300/500 Mbps (in/out)
>
> some of the baseline figures have changed, either due to other
> congestion on the link, or the lack of explicit buffer size, but
> this makes the performance in one direction match with/without the
> router.
>
> I don't know what parameters they are using to run the iperf server
> at the other end, but if they are also specifying the buffer size
> manually, that could account for the other direction being off (I'll
> find out).
>
> If this really is the case, the question becomes, why does the
> OpenBSD router care about this?
>
Large buffer sizes may cause bursty traffic. So it is possible that
these bursts cause packet drops and retransmits.
Packet drops on OpenBSD can be seen on the ip input queue
(sysctl net.inet.ip.ifq.drops) and on the individual interfaces
(netstat -i / -I<if>). The best is to monitor the various counters to
figure out which one is growing the fastest.
PS: about CWM and the other values in the systat output.
CWM stands for current watermark it is between the LWM (low) and HWM
(high). The livelock mitigation between your box and -current (I think
even 5.0) was changed to handle bursty traffic a bit better. So maybe an
update may give you better results.
--
:wq Claudio
