Re: Bandwidth issues in the Sprint network
On Thu, Apr 17, 2008 at 1:00 PM, Brian Raaen [EMAIL PROTECTED] wrote: Some people wanted to know what I found the problem to be. I have discovered. the problem for a fact is the TCP window size on uploads. I have a Linux box that I changed the Window sizes to match and I still get 32k on a upload window and 64k on a download window. With a ping time of 50ms I have a max theoretical throughput of 5.2Mbps Which is about what I was getting. The formula to calculate this is the following. (((Ts/Tw)*Rtd)/1000)+((Ts*8)/(Lr*1000))) Where the following are Ts = Transfer size in Bytes Tw = Tcp Window size in Bytes Rtd = Round trip Delay in milliseconds Lr = Line rate in bps At this point I am still trying to locate the offending device that is changing the window size. After I determine for sure whether the problem is with my router, the sprint network, or another upstream system I will let everybody know what I find. -- Brian Raaen Network Engineer [EMAIL PROTECTED] On Monday 07 April 2008, Brian Raaen wrote: I am currently having problems get upload bandwidth on a Sprint circuit. I am using a full OC3 circuit. I am doing fine on downloading data, but uploading data I can only get about 5Mbps with ftp or a speedtest. I have tested against multiple networks and this has stayed the same. Monitoring Cacti graphs and the router I do get about 30Mbps total traffic outbound, but individual (flows/ip?) test always seem limited. I would like to know if anyone else sees anything similar, or where I can get help. The assistance I have gotten from Sprint up to this point is that they find no problems. Due to the consistency of 5Mbps I am suspecting rate limiting, but wanted to know if I was overlooking something else. -- Brian Raaen Network Engineer [EMAIL PROTECTED] Thanks for reporting back to curious minds. Mike Gonnason
Re: Does TCP Need an Overhaul? (internetevolution, via slashdot)
On Tue, Apr 8, 2008 at 3:33 PM, Greg Skinner [EMAIL PROTECTED] wrote: On Wed, Apr 09, 2008 at 01:10:53AM +0200, Marcin Cieslak wrote: The problem is that fairness was probably never a design goal of TCP, even with Van Jacobson's congestion avoidance patch. Bob Briscoe is a member of the IETF Transport Working Group (TSVWG). This subject got some publicity and politics involved, but please see some real discussion on the TSVWG list, with my favorite answer highlighted: This issue also got some publicity and politics on the IRTF end2end list. For example, start at http://www.postel.org/pipermail/end2end-interest/2007-August/006925.html . --gregbo Thank you both (Marcin and Greg) for the links, they have made for some great reading tonight. It would appear that I have introduced a slight tangent from the goal of this topic. The main discussion here was regarding an overhaul of TCP, whereas my Briscoe suggestion is more of an architectural overhaul which leads to many other changes (infrastructural, economical and protocol) in the network. Briscoe's somewhat undiplomatic introduction of his idea seems to have elicited an initially negative response, however I happy to see that others have seen his ideas to have merit and hence test his reasoning. I surmise that the big question now is, do we go with a small step (enhanced congestion detection) or a jump (total reworking of network policing architecture). I am glad to say that whatever is decided, it will most likely be implemented far faster than IPv6. As we will not have a specific feature to buy us time from congestion, unlike what NAT did for IPv4. :) -Mike Gonnason
Re: Bandwidth issues in the Sprint network
On Tue, Apr 8, 2008 at 9:19 AM, Brian Raaen [EMAIL PROTECTED] wrote: I have been using the Java based versions of the speed test. At this point I have had some Sprint people get in contact with me so I will see what they find. Thank you for all your help to everyone. -- Brian Raaen Network Engineer [EMAIL PROTECTED] On Monday 07 April 2008, you wrote: I am currently having problems get upload bandwidth on a Sprint circuit. I am using a full OC3 circuit. I am doing fine on downloading data, but uploading data I can only get about 5Mbps with ftp or a speedtest. I have tested against multiple networks and this has stayed the same. Monitoring Cacti graphs and the router I do get about 30Mbps total traffic outbound, but individual (flows/ip?) test always seem limited. I would like to know if anyone else sees anything similar, or where I can get help. The assistance I have gotten from Sprint up to this point is that they find no problems. Due to the consistency of 5Mbps I am suspecting rate limiting, but wanted to know if I was overlooking something else. -- Brian Raaen Network Engineer [EMAIL PROTECTED] Most of the speed test sites on the Internet basically issue a HTTP GET request to a server and time the download. For upload they utilize a HTTP POST via a CGI script and time that. The main issue I have with these speed tests is that they only use a single TCP session for data transfer, which is fine if you have a large or self adjusting TCP window size and a relatively low latency link. However for high capacity links, it is unlikely (but possible) that you are planning to use a single TCP session and consume all the available capacity. Realistically you will have a few dozen server/applications/users and produce hundreds/thousands of TCP sessions which will fully utilize the link. For our PtP customers that have concerns regarding capacity, I generally they suggest setup iperf at both ends and run a few tests with multiple TCP sessions so they can independently verify. Hopefully Sprint will take your concerns to heart and assist you with testing. -Mike Gonnason
Re: Bandwidth issues in the Sprint network
A quick search comes up with Scientific Linux, but I cannot provide any claims to suitability. I have never even heard of it before, but it is provided as a LiveCD. http://linux.web.psi.ch/livecd/software.html -Mike Gonnason On Wed, Apr 9, 2008 at 6:28 AM, Frank Bulk [EMAIL PROTECTED] wrote: Does anyone know of bootable Linux CD with iperf on it? Frank -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Mike Gonnason Sent: Wednesday, April 09, 2008 9:05 AM To: nanog@merit.edu Subject: Re: Bandwidth issues in the Sprint network On Tue, Apr 8, 2008 at 9:19 AM, Brian Raaen [EMAIL PROTECTED] wrote: I have been using the Java based versions of the speed test. At this point I have had some Sprint people get in contact with me so I will see what they find. Thank you for all your help to everyone. -- Brian Raaen Network Engineer [EMAIL PROTECTED] On Monday 07 April 2008, you wrote: I am currently having problems get upload bandwidth on a Sprint circuit. I am using a full OC3 circuit. I am doing fine on downloading data, but uploading data I can only get about 5Mbps with ftp or a speedtest. I have tested against multiple networks and this has stayed the same. Monitoring Cacti graphs and the router I do get about 30Mbps total traffic outbound, but individual (flows/ip?) test always seem limited. I would like to know if anyone else sees anything similar, or where I can get help. The assistance I have gotten from Sprint up to this point is that they find no problems. Due to the consistency of 5Mbps I am suspecting rate limiting, but wanted to know if I was overlooking something else. -- Brian Raaen Network Engineer [EMAIL PROTECTED] Most of the speed test sites on the Internet basically issue a HTTP GET request to a server and time the download. For upload they utilize a HTTP POST via a CGI script and time that. The main issue I have with these speed tests is that they only use a single TCP session for data transfer, which is fine if you have a large or self adjusting TCP window size and a relatively low latency link. However for high capacity links, it is unlikely (but possible) that you are planning to use a single TCP session and consume all the available capacity. Realistically you will have a few dozen server/applications/users and produce hundreds/thousands of TCP sessions which will fully utilize the link. For our PtP customers that have concerns regarding capacity, I generally they suggest setup iperf at both ends and run a few tests with multiple TCP sessions so they can independently verify. Hopefully Sprint will take your concerns to heart and assist you with testing. -Mike Gonnason
Re: Does TCP Need an Overhaul? (internetevolution, via slashdot)
On Mon, Apr 7, 2008 at 8:43 AM, Iljitsch van Beijnum [EMAIL PROTECTED] wrote: On 7 apr 2008, at 16:20, Kevin Day wrote: As a quick example on two FreeBSD 7.0 boxes attached directly over GigE, with New Reno, fast retransmit/recovery, and 256K window sizes, with an intermediary router simulating packet loss. A single HTTP TCP session going from a server to client. Ok, assuming a 1460 MSS that leaves the RTT as the unknown. SACK enabled, 0% packet loss: 780Mbps SACK disabled, 0% packet loss: 780Mbps Is that all? Try with jumboframes. SACK enabled, 0.005% packet loss: 734Mbps SACK disabled, 0.005% packet loss: 144Mbps (19.6% the speed of having SACK enabled) 144 Mbps and 0.5 packet loss probability would result in a ~ 110 ms RTT so obviously something isn't right with that case. 734 would be an RTT of around 2 ms, which sounds fairly reasonable. I'd be interested to see what's really going on here, I suspect that the packet loss isn't sufficiently random so multiple segments are lost from a single window. Or maybe disabling SACK also disables fast retransmit? I'll be happy to look at a tcpdump for the 144 Mbps case. It would be very nice if more network-friendly protocols were in use, but with download optimizers for Windows that cranks the TCP window sizes way up, the general move to solving latency by opening more sockets, and P2P doing whatever it can to evade ISP detection - it's probably a bit late. Don't forget that the user is only partially in control, the data also has to come from somewhere. Service operators have little incentive to break the network. And users would probably actually like it if their p2p was less aggressive, that way you can keep it running when you do other stuff without jumping through traffic limiting hoops. This might have been mentioned earlier in the thread, but has anyone read the paper by Bob Briscoe titled Flow Rate Fairness:Dismantling a Religion? http://www.cs.ucl.ac.uk/staff/bbriscoe/projects/2020comms/refb/draft-briscoe-tsvarea-fair-02.pdf The paper essentially describes the fault in TCP congestion avoidance and how P2P applications leverage that flaw to consume as much bandwidth as possible. He also proposes that we redefine the mechanism we use to determine fair resource consumption. His example is individual flow rate fairness (traditional TCP congestion avoidance) vs cost fairness (a combination of congestion cost and flow rate associated with a specific entity). He also compares his cost fairness methodology to existing proposed TCP variants, which Hank previously mentioned. i.e. XCP, WFQ, ... Any thoughts regarding this? -Mike Gonnason
