Re: Online gaming and file downloads - latency hell! (fwd)
Hi .. as suggested, posting this discussion to ipfw@ too .. thanks, Ian -- Forwarded message -- Date: Mon, 21 Jun 2010 13:00:14 +0200 From: Luigi Rizzo To: Ian Smith Subject: Re: Online gaming and file downloads - latency hell! (fwd) On Mon, Jun 21, 2010 at 10:25 AM, Ian Smith wrote: > Hi Luigi, > > thought you might be interested, seeing you and ipfw+dummynet rate a > mention .. maybe I'm ill-informed about the gaming latency aspect? > > cheers, Ian for responsive flows (e.g. tcp), it is actually effective to shape incoming traffic for the flows that you want slow down. In fact, even though it is true that packets have already traversed the bottleneck, you are slowing down their arrival to the receiving TCP, which has the side effect of slowing down outgoing acks and thus slowing down the source. Queues will build up at the remote end while the system settles, but after that (and if you make sure that the total rate is lower than link capacity) it will be the sources that do not generate traffic that saturates the link. Pretty trivial to implement, too: you just need a set of rules that match the incoming traffic that you want to slow down, pass it to one pipe, and let the other traffic go through without filtering. cheers luigi > > -- Forwarded message -- > Date: Mon, 21 Jun 2010 15:50:05 +1000 (EST) > From: Ian Smith > To: Morgan Wesstr?m > Cc: freebsd-questions@freebsd.org, Modulok > Subject: Re: Online gaming and file downloads - latency hell! > > In freebsd-questions Digest, Vol 315, Issue 11, Message: 9 > On Fri, 18 Jun 2010 12:11:48 +0200 > Morgan Wesstr?m wrote: > > On 2010-06-16 02:51, Modulok wrote: > > > Yo, > > > > > > I have a FreeBSD box acting as a router between me and the Internet. > > > Whenever someone on the local network downloads something, the other > > > connections have a really high latency. A second or more. For people > > > who like to download large files and play online games, it's not good. > > > > > > I tried traffic shaping with PF, which works - almost: I tried the > > > home example in the PF book, but it doesn't work out so well. I can > > > throttle users with no trouble, but even so that doesn't seem to help > > > the latency issue unless I choke the 'big file download' users almost > > > completely off. It's like nothing helps. I tried a priority based > > > queue where all traffic on the gaming ports was placed in front of all > > > other traffic, and while I saw a very mild improvement, latency was > > > still really pitiful. > > > > > > Is there anything else I can do? Anyone have a similar setup and wish > > > to share config files? Are there some sysctl's that would help this > > > out or something. I'm almost ready to just buy a 'gaming' *gag* router > > > which implements their own brand of QoS, but don't want to sink to > > > that level if I don't have to. > > > > > > Help! > > > -Modulok- > > > > Traffic shaping on your side when downloading unfortunately doesn't help > > you. The data has already been transferred across your cable or DSL > > connection by then and reordering any packets on your side will not > > change the latency. Traffic shaping your download has to be performed at > > your upstream router which you probably don't control. PF can help you > > traffic shape your outgoing traffic. I have used it for this for the > > past 6 years to help me maintain a low and stable ping while I play > > online, even if I upload simultaneously. I've read about people trying > > to throttle outgoing ACKs to slow down their download but that still > > wouldn't rearrange any incoming data packets so I don't see how that > > would help. I haven't tried it myself though but neither have I read > > about anyone successfully accomplishing this. > > > > Regards > > Morgan > > A short story: > > About 15 months ago, before becoming aware that Luigi and colleagues had > been busy porting ipfw and dummynet to Linux, I was asked to implement a > shaping solution for a very limited (512/512kbps) ADSL connection for a > community radio station using a Linux firewall-in-a-box called IPCop as > router, whose shaping was based on Bert Hubert's WonderShaper script, > using Linux' tc module to prioritise and shape only outbound traffic. > > Having used ipfw+dummynet successfully for some years to shape traffic > for a local voluntary organisation 'Community Technology Centre'
Re: Online gaming and file downloads - latency hell!
On Mon, 21 Jun 2010, Olivier Nicole wrote: > Hi, > > > I've read about people trying > > to throttle outgoing ACKs to slow down their download but that still > > wouldn't rearrange any incoming data packets so I don't see how that > > would help. I haven't tried it myself though but neither have I read > > about anyone successfully accomplishing this. > > TCP uses a window: the maximum number of packects that you can receive > before you send an ACK. As long as ACK come flowing, the window size > increases. > > Limit the ACK, you limit/reduce the size of the window, so you > limit/reduce the incoming trafic. Indeed. If you've an in-house router queueing through traffic against some bandwidth limit imposed on an inside clients' download, dropping any excess TCP packets arriving on top of a full queue or pipe (eg with ipfw/dummynet), there'll be a few packets requiring retransmission to continue the transfer, now and again, without need to throttle ACKs; in fact we're expediting ACKs uphill, after streaming, ssh and ICMP. I've been surprised by how few packets get dropped and so resent, way less than 1%, even when pulling large files from fast providers through a slow link (512/512 ADSL as mentioned) then further limited to clients. Which are mostly 'doze, a mac or two, a couple of linux boxes; all seem to use SACK but I haven't looked into negotiated window sizes. I don't know TCP in any depth but watch with awe as people enhance and tune the stack; all I can say is 'it seems to mostly work pretty well here' .. How UDP-based services cope with dropped packets is another matter; perhaps that's a big issue for some games that may need expediting? > I beleive there could even be some nasty rewritting that would > artifically change the window size so the TCP stream is slowed down. Quite a job, intervening and rewriting packets, and maintaining state on whole streams; I gather TCP is resistant to Man in the Middle attack .. Anyway, a lot harder than configuring a few dummynet pipes and queues :) cheers, Ian ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Online gaming and file downloads - latency hell!
On 2010-06-21 07:50, Ian Smith wrote: > In freebsd-questions Digest, Vol 315, Issue 11, Message: 9 > On Fri, 18 Jun 2010 12:11:48 +0200 > Morgan Wesstr?m wrote: > > On 2010-06-16 02:51, Modulok wrote: > > > Yo, > > > > > > I have a FreeBSD box acting as a router between me and the Internet. > > > Whenever someone on the local network downloads something, the other > > > connections have a really high latency. A second or more. For people > > > who like to download large files and play online games, it's not good. > > > > > > I tried traffic shaping with PF, which works - almost: I tried the > > > home example in the PF book, but it doesn't work out so well. I can > > > throttle users with no trouble, but even so that doesn't seem to help > > > the latency issue unless I choke the 'big file download' users almost > > > completely off. It's like nothing helps. I tried a priority based > > > queue where all traffic on the gaming ports was placed in front of all > > > other traffic, and while I saw a very mild improvement, latency was > > > still really pitiful. > > > > > > Is there anything else I can do? Anyone have a similar setup and wish > > > to share config files? Are there some sysctl's that would help this > > > out or something. I'm almost ready to just buy a 'gaming' *gag* router > > > which implements their own brand of QoS, but don't want to sink to > > > that level if I don't have to. > > > > > > Help! > > > -Modulok- > > > > Traffic shaping on your side when downloading unfortunately doesn't help > > you. The data has already been transferred across your cable or DSL > > connection by then and reordering any packets on your side will not > > change the latency. Traffic shaping your download has to be performed at > > your upstream router which you probably don't control. PF can help you > > traffic shape your outgoing traffic. I have used it for this for the > > past 6 years to help me maintain a low and stable ping while I play > > online, even if I upload simultaneously. I've read about people trying > > to throttle outgoing ACKs to slow down their download but that still > > wouldn't rearrange any incoming data packets so I don't see how that > > would help. I haven't tried it myself though but neither have I read > > about anyone successfully accomplishing this. > > > > Regards > > Morgan > > A short story: > > About 15 months ago, before becoming aware that Luigi and colleagues had > been busy porting ipfw and dummynet to Linux, I was asked to implement a > shaping solution for a very limited (512/512kbps) ADSL connection for a > community radio station using a Linux firewall-in-a-box called IPCop as > router, whose shaping was based on Bert Hubert's WonderShaper script, > using Linux' tc module to prioritise and shape only outbound traffic. > > Having used ipfw+dummynet successfully for some years to shape traffic > for a local voluntary organisation 'Community Technology Centre', I was > staggered to find that all of the collective Linux wisdom on the subject > chanted that same mantra .. that you can't prioritise download traffic, > as the ISP will have 'gigantic queues' of TCP traffic that you can't > control, and that prioritising ACKs, QoS and ICMP traffic and such is > the best you can do. By this philosophy, tc only implements limiting > total bandwidth of inbound traffic, shaping outbound by QoS and classes. > > To disprove this pervasive myth I had to implement inbound shaping by > using tc to control the _outbound_ traffic to the _inside_ interface, > where all sorts of random clients are doing big downloads, yootoobing > and such plus some big uploads, while guaranteeing that the station's > outbound audio stream had fully half the outbound-to-net bandwidth free > without undue pressure and that remote ssh sessions etc remained snappy. > > This involves queuing inbound (mostly TCP) traffic on the local router, > dropping any excess, which works most effectively to maintain a hard > limit to downloads (at around 85% of 512kbps) while keeping the outbound > (to-net) channel lightly loaded after streaming, ACKs, and uploads. > > I don't know how pf works (or can be made to work) in this regard, nor > can I speculate about gaming latency particularly, but hope to find out > soon by either replacing the old IPCop box with pfSense, or trying ipfw > and dummynet on Linux .. I know, but they're still reluctant to shop > other than Linux, and the idea of implementing a FreeBSD-derived > firewall and shaping solution on Linux has a good deal of appeal .. > > HTH (or at least, doesn't hurt :) > > cheers, Ian PF can handle the bandwidth limiting on the internal interface, no problem there. However, bandwidth limiting is only one part of traffic shaping. The other part is packet reordering. Even if you limit incoming bandwidth, your ISP may still have hundreds of packets of P2P traffic in it's queue to send
Re: Online gaming and file downloads - latency hell!
Hi, > I've read about people trying > to throttle outgoing ACKs to slow down their download but that still > wouldn't rearrange any incoming data packets so I don't see how that > would help. I haven't tried it myself though but neither have I read > about anyone successfully accomplishing this. TCP uses a window: the maximum number of packects that you can receive before you send an ACK. As long as ACK come flowing, the window size increases. Limit the ACK, you limit/reduce the size of the window, so you limit/reduce the incoming trafic. I beleive there could even be some nasty rewritting that would artifically change the window size so the TCP stream is slowed down. Bests, olivier ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Online gaming and file downloads - latency hell!
In freebsd-questions Digest, Vol 315, Issue 11, Message: 9 On Fri, 18 Jun 2010 12:11:48 +0200 Morgan Wesstr?m wrote: > On 2010-06-16 02:51, Modulok wrote: > > Yo, > > > > I have a FreeBSD box acting as a router between me and the Internet. > > Whenever someone on the local network downloads something, the other > > connections have a really high latency. A second or more. For people > > who like to download large files and play online games, it's not good. > > > > I tried traffic shaping with PF, which works - almost: I tried the > > home example in the PF book, but it doesn't work out so well. I can > > throttle users with no trouble, but even so that doesn't seem to help > > the latency issue unless I choke the 'big file download' users almost > > completely off. It's like nothing helps. I tried a priority based > > queue where all traffic on the gaming ports was placed in front of all > > other traffic, and while I saw a very mild improvement, latency was > > still really pitiful. > > > > Is there anything else I can do? Anyone have a similar setup and wish > > to share config files? Are there some sysctl's that would help this > > out or something. I'm almost ready to just buy a 'gaming' *gag* router > > which implements their own brand of QoS, but don't want to sink to > > that level if I don't have to. > > > > Help! > > -Modulok- > > Traffic shaping on your side when downloading unfortunately doesn't help > you. The data has already been transferred across your cable or DSL > connection by then and reordering any packets on your side will not > change the latency. Traffic shaping your download has to be performed at > your upstream router which you probably don't control. PF can help you > traffic shape your outgoing traffic. I have used it for this for the > past 6 years to help me maintain a low and stable ping while I play > online, even if I upload simultaneously. I've read about people trying > to throttle outgoing ACKs to slow down their download but that still > wouldn't rearrange any incoming data packets so I don't see how that > would help. I haven't tried it myself though but neither have I read > about anyone successfully accomplishing this. > > Regards > Morgan A short story: About 15 months ago, before becoming aware that Luigi and colleagues had been busy porting ipfw and dummynet to Linux, I was asked to implement a shaping solution for a very limited (512/512kbps) ADSL connection for a community radio station using a Linux firewall-in-a-box called IPCop as router, whose shaping was based on Bert Hubert's WonderShaper script, using Linux' tc module to prioritise and shape only outbound traffic. Having used ipfw+dummynet successfully for some years to shape traffic for a local voluntary organisation 'Community Technology Centre', I was staggered to find that all of the collective Linux wisdom on the subject chanted that same mantra .. that you can't prioritise download traffic, as the ISP will have 'gigantic queues' of TCP traffic that you can't control, and that prioritising ACKs, QoS and ICMP traffic and such is the best you can do. By this philosophy, tc only implements limiting total bandwidth of inbound traffic, shaping outbound by QoS and classes. To disprove this pervasive myth I had to implement inbound shaping by using tc to control the _outbound_ traffic to the _inside_ interface, where all sorts of random clients are doing big downloads, yootoobing and such plus some big uploads, while guaranteeing that the station's outbound audio stream had fully half the outbound-to-net bandwidth free without undue pressure and that remote ssh sessions etc remained snappy. This involves queuing inbound (mostly TCP) traffic on the local router, dropping any excess, which works most effectively to maintain a hard limit to downloads (at around 85% of 512kbps) while keeping the outbound (to-net) channel lightly loaded after streaming, ACKs, and uploads. I don't know how pf works (or can be made to work) in this regard, nor can I speculate about gaming latency particularly, but hope to find out soon by either replacing the old IPCop box with pfSense, or trying ipfw and dummynet on Linux .. I know, but they're still reluctant to shop other than Linux, and the idea of implementing a FreeBSD-derived firewall and shaping solution on Linux has a good deal of appeal .. HTH (or at least, doesn't hurt :) cheers, Ian ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Online gaming and file downloads - latency hell!
On Fri, 18 Jun 2010 12:11:48 +0200 Morgan Wesström wrote: > On 2010-06-16 02:51, Modulok wrote: > > Yo, > > > > I have a FreeBSD box acting as a router between me and the Internet. > > Whenever someone on the local network downloads something, the other > > connections have a really high latency. A second or more. For people > > who like to download large files and play online games, it's not > > good. > Traffic shaping on your side when downloading unfortunately doesn't > help you. The data has already been transferred across your cable or > DSL connection by then and reordering any packets on your side will > not change the latency. Traffic shaping your download has to be > performed at your upstream router which you probably don't control. If the downloads are ordinary http and ftp, rather than P2P, you can use squid to throttle at the TCP level. It needs to be built with delay pools. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Online gaming and file downloads - latency hell!
On 2010-06-16 02:51, Modulok wrote: > Yo, > > I have a FreeBSD box acting as a router between me and the Internet. > Whenever someone on the local network downloads something, the other > connections have a really high latency. A second or more. For people > who like to download large files and play online games, it's not good. > > I tried traffic shaping with PF, which works - almost: I tried the > home example in the PF book, but it doesn't work out so well. I can > throttle users with no trouble, but even so that doesn't seem to help > the latency issue unless I choke the 'big file download' users almost > completely off. It's like nothing helps. I tried a priority based > queue where all traffic on the gaming ports was placed in front of all > other traffic, and while I saw a very mild improvement, latency was > still really pitiful. > > Is there anything else I can do? Anyone have a similar setup and wish > to share config files? Are there some sysctl's that would help this > out or something. I'm almost ready to just buy a 'gaming' *gag* router > which implements their own brand of QoS, but don't want to sink to > that level if I don't have to. > > Help! > -Modulok- Traffic shaping on your side when downloading unfortunately doesn't help you. The data has already been transferred across your cable or DSL connection by then and reordering any packets on your side will not change the latency. Traffic shaping your download has to be performed at your upstream router which you probably don't control. PF can help you traffic shape your outgoing traffic. I have used it for this for the past 6 years to help me maintain a low and stable ping while I play online, even if I upload simultaneously. I've read about people trying to throttle outgoing ACKs to slow down their download but that still wouldn't rearrange any incoming data packets so I don't see how that would help. I haven't tried it myself though but neither have I read about anyone successfully accomplishing this. Regards Morgan ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Online gaming and file downloads - latency hell!
Yo, I have a FreeBSD box acting as a router between me and the Internet. Whenever someone on the local network downloads something, the other connections have a really high latency. A second or more. For people who like to download large files and play online games, it's not good. I tried traffic shaping with PF, which works - almost: I tried the home example in the PF book, but it doesn't work out so well. I can throttle users with no trouble, but even so that doesn't seem to help the latency issue unless I choke the 'big file download' users almost completely off. It's like nothing helps. I tried a priority based queue where all traffic on the gaming ports was placed in front of all other traffic, and while I saw a very mild improvement, latency was still really pitiful. Is there anything else I can do? Anyone have a similar setup and wish to share config files? Are there some sysctl's that would help this out or something. I'm almost ready to just buy a 'gaming' *gag* router which implements their own brand of QoS, but don't want to sink to that level if I don't have to. Help! -Modulok- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
