Re: Linux Router: TCP slow, UDP fast

2009-02-15 Thread Chris
Thanks, Karl, Allen and Nickola. I failed-over to another router last night and briefly had full expected throughput but this morning despite dropping providers and moving between routers again for trial and error I still see _outbound_ TCP at about the same 300 - 600kbps per session. I

Re: Linux Router: TCP slow, UDP fast

2009-02-15 Thread Mathias Wolkert
I would turn off ethernet flow control. Maybe you already have. It can be really mean on tcp's own flow control if the switch has an issue of some kind (load). /Tias 15 feb 2009 kl. 10.24 skrev Chris ch...@ghostbusters.co.uk: Thanks, Karl, Allen and Nickola. I failed-over to another

Re: Linux Router: TCP slow, UDP fast

2009-02-15 Thread Chris
Thanks for the suggestions everyone. I've got to the bottom of the problem now (I'm sure there will be a collective sigh of relief from the list because of the noise this thread generated :-)). I installed two brand new, low spec, 3Com switches one at the 'front' of the network and one 'behind'

Re: Linux Router: TCP slow, UDP fast

2009-02-15 Thread Geert Bosch
On Feb 15, 2009, at 04:24, Chris wrote: Any last ideas appreciated before causing headaches removing switches would be appreciated. The TCP offloading should be suspect. Any current PC hardware should be able to deal with huge amounts of traffic without any offloading. Start with turning

Linux Router: TCP slow, UDP fast

2009-02-14 Thread Chris
Hi All, I'm losing the will to live with this networking headache ! Please feel free to point me at a Linux list if NANOG isn't suitable. I'm at a loss where else to ask. I've diagnosed some traffic oddities and after lots of head-scratching, reading and trial and error I can say with certainty

Re: Linux Router: TCP slow, UDP fast

2009-02-14 Thread Lee
Try enabling window scaling echo 1 /proc/sys/net/ipv4/tcp_window_scaling or, if you really want it disabled, configure a larger minimum window size net.ipv4.tcp_rmem = 64240 87380 16777216 HTH, Lee On 2/14/09, Chris ch...@ghostbusters.co.uk wrote: Hi All, I'm losing the will to live

Re: Linux Router: TCP slow, UDP fast

2009-02-14 Thread Florian Weimer
I'm losing the will to live with this networking headache ! Please feel free to point me at a Linux list if NANOG isn't suitable. I'm at a loss where else to ask. The linux-net might be more appropriate indeed. With and without shaping and over different bandwidth providers using the e1000

Re: Linux Router: TCP slow, UDP fast

2009-02-14 Thread Mikael Abrahamsson
On Sat, 14 Feb 2009, Lee wrote: Try enabling window scaling echo 1 /proc/sys/net/ipv4/tcp_window_scaling or, if you really want it disabled, configure a larger minimum window size net.ipv4.tcp_rmem = 64240 87380 16777216 Without window scaling, you're limited to 64k window size anyway.

Re: Linux Router: TCP slow, UDP fast

2009-02-14 Thread Chris
Thanks loads for the quick replies. I'll try and respond individually. Lee I recently disabled tcp_window_scaling and it didn't solve the problem. I don't know enough about it. Should I enable it again ? Settings differing from defaults are copied in my first post. Mike Strangely I'm not seeing

Re: Linux Router: TCP slow, UDP fast

2009-02-14 Thread Nickola Kolev
Hello, Chris, So, as it seems you have problem with TCP, and not UDP, maybe this is something with regard to TCP segmentation offloading. It could be a total shot in the dark, but can you see what ethtool -k devname says? Then you can have a look at 'man ethtool' and turn on/off the appropriate

Re: Linux Router: TCP slow, UDP fast

2009-02-14 Thread Chris
Thanks, Nickola. What's your opinion on these settings ? Do you recommend switching off tcp segmentation offload ? Offload parameters for eth0: rx-checksumming: on tx-checksumming: on scatter-gather: on tcp segmentation offload: on udp fragmentation offload: off generic segmentation offload: on

Re: Linux Router: TCP slow, UDP fast

2009-02-14 Thread Lee
On 2/14/09, Chris ch...@ghostbusters.co.uk wrote: Thanks loads for the quick replies. I'll try and respond individually. Lee I recently disabled tcp_window_scaling and it didn't solve the problem. I don't know enough about it. Should I enable it again ? Settings differing from defaults are

Re: Linux Router: TCP slow, UDP fast

2009-02-14 Thread Chris
Thanks very much, Lee. My head's whirring. Am I right in thinking by turning on scaling (which I just did) then the window size is automatically set ? I'll do some more reading. I'm looking at TSO too as above, mentioned by Nickola. I'll maybe risk changing it with ethtool during a quiet network

Re: Linux Router: TCP slow, UDP fast

2009-02-14 Thread Lee
On 2/14/09, Chris ch...@ghostbusters.co.uk wrote: Thanks very much, Lee. My head's whirring. Am I right in thinking by turning on scaling (which I just did) then the window size is automatically set ? No. Scaling just allows you to have a window size larger than 64KB. These might help

Re: Linux Router: TCP slow, UDP fast

2009-02-14 Thread Chris
Hi Mikael, I just realised that I didn't respond to your post. The RTTs vary massively because the router is forwarding from websites on the LAN to visitors worldwide. Is that what you meant ? Disabling TSO didn't work unfortunately. Thanks again, Chris

Re: Linux Router: TCP slow, UDP fast

2009-02-14 Thread Florian Weimer
I'm looking at TSO too as above, mentioned by Nickola. I'll maybe risk changing it with ethtool during a quiet network moment. Turning off offloading might be something to try indeed. Regarding the negotation issue, can you look at the other end of the link and check what it's saying? Looking

Re: Linux Router: TCP slow, UDP fast

2009-02-14 Thread Mikael Abrahamsson
On Sat, 14 Feb 2009, Chris wrote: The RTTs vary massively because the router is forwarding from websites on the LAN to visitors worldwide. Is that what you meant ? And your TCP speed when doing testing is always 300-600 kilobyte/s regardless of RTT between the boxes with which you're

Re: Linux Router: TCP slow, UDP fast

2009-02-14 Thread Chris
Thanks for all the answers. I'm currently going down the path of looking at IPtables' conntrack slowing the forwarding rate. If I can't find any more docs then I'll boot the router with a kernel without any IPtables built-in and see if that's it. As Lee said rollback ! That's the last change to

Re: Linux Router: TCP slow, UDP fast

2009-02-14 Thread Allen Smith
If this router is not doing some kind of proxying, tuning tcp related kernel bits will not impact long fat pipe or long fat network issues. The place that needs to be tuned for larger window sizes/scaling is the web server. http://www.psc.edu/networking/projects/tcptune/#Linux or search for