[gentoo-user] traffic shaping
Hi, i would like to shape the traffic of my DSL-connection, but somehow i never really understood the machanisms that linux offers. All the scripts i wrote were simply worthless somehow, because they didn't really improve anything. Is there any application or script that is easy to configure and does all the necessary things to shape my DSL traffic? Thanks Sven -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] traffic shaping
On 03 April 2006 03:06, Sven Köhler wrote: > Hi, > > i would like to shape the traffic of my DSL-connection, but somehow i > never really understood the machanisms that linux offers. All the > scripts i wrote were simply worthless somehow, because they didn't > really improve anything. > > Is there any application or script that is easy to configure and does > all the necessary things to shape my DSL traffic? No, because traffic shaping is such a broad field that no simply script can cover it. On the other hand, it isn't a mystery either. One can understand it. Try Bert Hubert's "Linux Advanced Routing & Traffic Control HOWTO". I don't have the URL handy right. Just google for it. Uwe -- Why do consumers keep buying products they will live to curse? -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] traffic shaping
On Sunday 02 April 2006 21:06, Sven Köhler <[EMAIL PROTECTED]> wrote about '[gentoo-user] traffic shaping': > Is there any application or script that is easy to configure and does > all the necessary things to shape my DSL traffic? Not at this time. However, it's actually fairly easy to throw something together with HTB that works fairly well. If you are only matching on protocol (TCP or UDP) and (source or destination) (IP or port) a custom iproute2 script will not be that hard. tc qdisc created your root HTB(s); tc class creates your child/sibling HTBs [classes under the same qdisc can swap bandwidth]; tc filter decides what classes receive packets. Hit the TLDP Advanced Routing HOWTO and just hit the parts about HTB and tc filter. Then, go to the HTB site and read the user documentation there. Then come back and read the tc filter stuff again -- expect to learn a little to a lot about how a packet looks be it IP, TCP, or UDP. Write some scripts and experiment, you won't get it perfect the first time but after a little bit of work you'll find your browsing experience much better. Your filtering policies won't match mine, so it's hard to give a single script. I find it works best to load up your connection then start experimenting with a network monitor in another window -- you'll be able to see changes quickly and (at least with HTB) you won't drop packets just because you were changing settings. When it comes down to it, if you are killing your upstream /something/ is going to suffer, tc just lets you put the pain where you want it; if you regularly saturate your upstream, buy fatter pipes. TC WILL NOT HELP AN OVERLOADED DOWNSTREAM since it has no control over what packets are sent to you. If your requirements are fairly simple, send it my way and I'll try to write out a script for you; I know I need more practice. -- "If there's one thing we've established over the years, it's that the vast majority of our users don't have the slightest clue what's best for them in terms of package stability." -- Gentoo Developer Ciaran McCreesh pgphbrh6s5wQc.pgp Description: PGP signature
Re: [gentoo-user] traffic shaping
On Monday 03 April 2006 04:06, Sven Köhler wrote: > Hi, > > i would like to shape the traffic of my DSL-connection, but somehow i > never really understood the machanisms that linux offers. All the > scripts i wrote were simply worthless somehow, because they didn't > really improve anything. > > Is there any application or script that is easy to configure and does > all the necessary things to shape my DSL traffic? > > > Thanks > Sven From what I've heard, this script should be pretty easy to use: http://monkey.org/~marius/pages/?page=trickle -- Rick van Hattem Rick.van.Hattem(at)Fawo.nl pgp50TxnkXjbM.pgp Description: PGP signature
Re: [gentoo-user] traffic shaping
Hi, On Mon, 2006-04-03 at 02:47 -0500, Boyd Stephen Smith Jr. wrote: > On Sunday 02 April 2006 21:06, Sven Köhler <[EMAIL PROTECTED]> wrote > about '[gentoo-user] traffic shaping': > > Is there any application or script that is easy to configure and does > > all the necessary things to shape my DSL traffic? > ...SKIP... Check "trickle" (comes from OpenBSD) latest version is 1.06. Bad thing is that currently it's masked for removal as being unmaintained since 2003, plus there's an open Bug, doesn't compile (126597). As i got interested managed to make it compile (check the Bug). Be warned i still haven't tested it, but by man pages desc. its' usage is quite/very simple (both as a daemon or run on command line). Can shape both incoming and up-going connections. If it's working will hate to see it go. HTH.Rumen smime.p7s Description: S/MIME cryptographic signature
[gentoo-user] Traffic shaping - downstream data
Hi all I'm looking for some help setting up traffic shaping on my internet connection. I have a bit of an odd setup in that I run a remote VPN server that all of my traffic is pushed through and out on to the internet. As I understand generally it isn't possible to shape incoming traffic but as I have control of the VPN server which pushes the traffic to me I wondered if it was possible to implement something on that side? No traffic other than the VPN tunnel goes out of my home connection. I'm trying to do this because I have a service running on one of my home machines that requires around 5kbps constantly with low latency (<200ms), but as my home connection is 750kbps it gets saturated very quickly causing huge spikes in latency. Does anyone have any ideas as to how I could achieve this? Generally any pointers at all would be greatly appreciated. Thanks for your time Oliver
[gentoo-user] traffic shaping and p2p
I've a small home network, actuall consisting of two gentoo boxes, where one box acts as router, firewall, svn server and desktop for my sister (i know this isn't an optimal setup) and the other one is my workstation. Now, when i start a p2p app on my workstation the latency of my internet connection suffers greatly, allthogh i've 384 kbit/s up and 3072 kbit/s down. I know that there are some approaches to solve this kind of problem by categorizing packets and assign different priorities to them, as explained at http://gentoo-wiki.com/HOWTO_Packet_Shaping. However, my knowledge of iptables and networking is very limited and i just want a simple and clean solution as i don't plan to trick myself by switching my p2p apps to non standard ports or manipulating the packet size ... To cut a long story short: I want high latency for ssh, browsing, subversion while offering p2p services a maximum of bandwidth in a small homenetwork containing only 2 boxes. Any suggestions ? Thanks, Matthias -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] Traffic shaping - downstream data
On Mon, June 11, 2012 5:27 pm, Datty wrote: > Hi all > > I'm looking for some help setting up traffic shaping on my internet > connection. I have a bit of an odd setup in that I run a remote VPN server > that all of my traffic is pushed through and out on to the internet. As I > understand generally it isn't possible to shape incoming traffic but as I > have control of the VPN server which pushes the traffic to me I wondered > if > it was possible to implement something on that side? No traffic other than > the VPN tunnel goes out of my home connection. > > I'm trying to do this because I have a service running on one of my home > machines that requires around 5kbps constantly with low latency (<200ms), > but as my home connection is 750kbps it gets saturated very quickly > causing > huge spikes in latency. Does anyone have any ideas as to how I could > achieve this? Generally any pointers at all would be greatly appreciated. If VPN is the only traffic to/from your home, eg. using your internet connection and you control the VPN-server on the other side, you could limit the "upstream" of the remote server to your home. > Thanks for your time > > Oliver > -- Joost
Re: [gentoo-user] Traffic shaping - downstream data
On Tue, Jun 12, 2012 at 9:58 AM, J. Roeleveld wrote: > On Mon, June 11, 2012 5:27 pm, Datty wrote: > > Hi all > > > > I'm looking for some help setting up traffic shaping on my internet > > connection. I have a bit of an odd setup in that I run a remote VPN > server > > that all of my traffic is pushed through and out on to the internet. As I > > understand generally it isn't possible to shape incoming traffic but as I > > have control of the VPN server which pushes the traffic to me I wondered > > if > > it was possible to implement something on that side? No traffic other > than > > the VPN tunnel goes out of my home connection. > > > > I'm trying to do this because I have a service running on one of my home > > machines that requires around 5kbps constantly with low latency (<200ms), > > but as my home connection is 750kbps it gets saturated very quickly > > causing > > huge spikes in latency. Does anyone have any ideas as to how I could > > achieve this? Generally any pointers at all would be greatly appreciated. > > If VPN is the only traffic to/from your home, eg. using your internet > connection and you control the VPN-server on the other side, you could > limit the "upstream" of the remote server to your home. > > > Thanks for your time > > > > Oliver > > > > > -- > Joost > > > Thanks that makes total sense. I was looking at it backwards, not thinking that I could apply the same upstream limit to my VPN server. A bit of background/my aims - The vpn interface is 100mbps, I want everybody but me on the VPN to be able to use up to that speed, but for traffic sent to 192.168.50.0/24 to be limited to 750kbps, with 700kbps of that for normal traffic and 50kbps for my tcp traffic from port . Based on that do the following rules make sense? tc qdisc add dev tap0 root handle 1: htb default 12 -- Set the interface to be handle 1 and default traffic to be in class 1:12 tc class add dev tap0 parent 1: classid 1:1 htb rate 100mbps ceil 100mbps -- Set 100mbps to be available to all classes overall tc class add dev tap0 parent 1:1 classid 1:12 htb rate 100mbps ceil 100mbps -- Set 100mbps to be available to all people on the vpn tc class add dev tap0 parent 1:1 classid 1:15 htb rate 750kbps ceil 750kbps -- To be applied to all traffic from my home network tc class add dev tap0 parent 1:15 classid 1:16 htb rate 700kbps ceil 700kbps -- To be applied to all traffic other than special on home network tc class add dev tap0 parent 1:15 classid 1:17 htb rate 50kbps ceil 50kbps -- To be applied to special traffic on home network tc qdisc add dev $modemif parent 1:15 handle 20: sfq perturb 10 -- I understand this to prevent high bandwidth traffic in a class from filling up the whole of the class bandwidth and allow fair sharing. Is this right/needed? tc qdisc add dev $modemif parent 1:12 handle 20: sfq perturb 10 iptables -t mangle -A POSTROUTING -o tap0 -d 192.168.50.0/24 -p tcp --sport -j CLASSIFY --set-class 1:17 iptables -t mangle -A POSTROUTING -o tap0 -d 192.168.50.4/24 -j CLASSIFY --set-class 1:16 iptables -t mangle -A POSTROUTING -o tap0 -j CLASSIFY --set-class 1:12 Thanks again for your help Oliver
Re: [gentoo-user] Traffic shaping - downstream data
More detail later...but make sure your vpn link is not TCP. UDP, fine, IP-IP, fine, but not TCP. TCP transport for a VPN tunnel leads to ugly traffic problems. On Jun 12, 2012 8:59 AM, "Datty" wrote: > > On Tue, Jun 12, 2012 at 9:58 AM, J. Roeleveld wrote: > >> On Mon, June 11, 2012 5:27 pm, Datty wrote: >> > Hi all >> > >> > I'm looking for some help setting up traffic shaping on my internet >> > connection. I have a bit of an odd setup in that I run a remote VPN >> server >> > that all of my traffic is pushed through and out on to the internet. As >> I >> > understand generally it isn't possible to shape incoming traffic but as >> I >> > have control of the VPN server which pushes the traffic to me I wondered >> > if >> > it was possible to implement something on that side? No traffic other >> than >> > the VPN tunnel goes out of my home connection. >> > >> > I'm trying to do this because I have a service running on one of my home >> > machines that requires around 5kbps constantly with low latency >> (<200ms), >> > but as my home connection is 750kbps it gets saturated very quickly >> > causing >> > huge spikes in latency. Does anyone have any ideas as to how I could >> > achieve this? Generally any pointers at all would be greatly >> appreciated. >> >> If VPN is the only traffic to/from your home, eg. using your internet >> connection and you control the VPN-server on the other side, you could >> limit the "upstream" of the remote server to your home. >> >> > Thanks for your time >> > >> > Oliver >> > >> >> >> -- >> Joost >> >> >> Thanks that makes total sense. I was looking at it backwards, not > thinking that I could apply the same upstream limit to my VPN server. > A bit of background/my aims - The vpn interface is 100mbps, I want > everybody but me on the VPN to be able to use up to that speed, but for > traffic sent to 192.168.50.0/24 to be limited to 750kbps, with 700kbps of > that for normal traffic and 50kbps for my tcp traffic from port . > > Based on that do the following rules make sense? > > tc qdisc add dev tap0 root handle 1: htb default 12 -- Set the interface > to be handle 1 and default traffic to be in class 1:12 > tc class add dev tap0 parent 1: classid 1:1 htb rate 100mbps ceil 100mbps > -- Set 100mbps to be available to all classes overall > tc class add dev tap0 parent 1:1 classid 1:12 htb rate 100mbps ceil > 100mbps -- Set 100mbps to be available to all people on the vpn > tc class add dev tap0 parent 1:1 classid 1:15 htb rate 750kbps ceil > 750kbps -- To be applied to all traffic from my home network > tc class add dev tap0 parent 1:15 classid 1:16 htb rate 700kbps ceil > 700kbps -- To be applied to all traffic other than special on home network > tc class add dev tap0 parent 1:15 classid 1:17 htb rate 50kbps ceil 50kbps > -- To be applied to special traffic on home network > tc qdisc add dev $modemif parent 1:15 handle 20: sfq perturb 10 -- I > understand this to prevent high bandwidth traffic in a class from filling > up the whole of the class bandwidth and allow fair sharing. Is this > right/needed? > tc qdisc add dev $modemif parent 1:12 handle 20: sfq perturb 10 > > iptables -t mangle -A POSTROUTING -o tap0 -d 192.168.50.0/24 -p tcp > --sport -j CLASSIFY --set-class 1:17 > iptables -t mangle -A POSTROUTING -o tap0 -d 192.168.50.4/24 -j CLASSIFY > --set-class 1:16 > iptables -t mangle -A POSTROUTING -o tap0 -j CLASSIFY --set-class 1:12 > > > Thanks again for your help > > Oliver >
Re: [gentoo-user] Traffic shaping - downstream data
On Tue, Jun 12, 2012 at 2:21 PM, Michael Mol wrote: > More detail later...but make sure your vpn link is not TCP. UDP, fine, > IP-IP, fine, but not TCP. TCP transport for a VPN tunnel leads to ugly > traffic problems. > On Jun 12, 2012 8:59 AM, "Datty" wrote: > >> >> On Tue, Jun 12, 2012 at 9:58 AM, J. Roeleveld wrote: >> >>> On Mon, June 11, 2012 5:27 pm, Datty wrote: >>> > Hi all >>> > >>> > I'm looking for some help setting up traffic shaping on my internet >>> > connection. I have a bit of an odd setup in that I run a remote VPN >>> server >>> > that all of my traffic is pushed through and out on to the internet. >>> As I >>> > understand generally it isn't possible to shape incoming traffic but >>> as I >>> > have control of the VPN server which pushes the traffic to me I >>> wondered >>> > if >>> > it was possible to implement something on that side? No traffic other >>> than >>> > the VPN tunnel goes out of my home connection. >>> > >>> > I'm trying to do this because I have a service running on one of my >>> home >>> > machines that requires around 5kbps constantly with low latency >>> (<200ms), >>> > but as my home connection is 750kbps it gets saturated very quickly >>> > causing >>> > huge spikes in latency. Does anyone have any ideas as to how I could >>> > achieve this? Generally any pointers at all would be greatly >>> appreciated. >>> >>> If VPN is the only traffic to/from your home, eg. using your internet >>> connection and you control the VPN-server on the other side, you could >>> limit the "upstream" of the remote server to your home. >>> >>> > Thanks for your time >>> > >>> > Oliver >>> > >>> >>> >>> -- >>> Joost >>> >>> >>> Thanks that makes total sense. I was looking at it backwards, not >> thinking that I could apply the same upstream limit to my VPN server. >> A bit of background/my aims - The vpn interface is 100mbps, I want >> everybody but me on the VPN to be able to use up to that speed, but for >> traffic sent to 192.168.50.0/24 to be limited to 750kbps, with 700kbps >> of that for normal traffic and 50kbps for my tcp traffic from port . >> >> Based on that do the following rules make sense? >> >> tc qdisc add dev tap0 root handle 1: htb default 12 -- Set the interface >> to be handle 1 and default traffic to be in class 1:12 >> tc class add dev tap0 parent 1: classid 1:1 htb rate 100mbps ceil 100mbps >> -- Set 100mbps to be available to all classes overall >> tc class add dev tap0 parent 1:1 classid 1:12 htb rate 100mbps ceil >> 100mbps -- Set 100mbps to be available to all people on the vpn >> tc class add dev tap0 parent 1:1 classid 1:15 htb rate 750kbps ceil >> 750kbps -- To be applied to all traffic from my home network >> tc class add dev tap0 parent 1:15 classid 1:16 htb rate 700kbps ceil >> 700kbps -- To be applied to all traffic other than special on home network >> tc class add dev tap0 parent 1:15 classid 1:17 htb rate 50kbps ceil >> 50kbps -- To be applied to special traffic on home network >> tc qdisc add dev $modemif parent 1:15 handle 20: sfq perturb 10 -- I >> understand this to prevent high bandwidth traffic in a class from filling >> up the whole of the class bandwidth and allow fair sharing. Is this >> right/needed? >> tc qdisc add dev $modemif parent 1:12 handle 20: sfq perturb 10 >> >> iptables -t mangle -A POSTROUTING -o tap0 -d 192.168.50.0/24 -p tcp >> --sport -j CLASSIFY --set-class 1:17 >> iptables -t mangle -A POSTROUTING -o tap0 -d 192.168.50.4/24 -j CLASSIFY >> --set-class 1:16 >> iptables -t mangle -A POSTROUTING -o tap0 -j CLASSIFY --set-class 1:12 >> >> >> Thanks again for your help >> >> Oliver >> > Ah it is TCP at the moment. Not something I could change too easily either. Is it possible to work around or is it not worth fighting with?
Re: [gentoo-user] Traffic shaping - downstream data
On Tue, Jun 12, 2012 at 9:37 AM, Datty wrote: > On Tue, Jun 12, 2012 at 2:21 PM, Michael Mol wrote: >> On Jun 12, 2012 8:59 AM, "Datty" wrote: [snip] >> More detail later...but make sure your vpn link is not TCP. UDP, fine, >> IP-IP, fine, but not TCP. TCP transport for a VPN tunnel leads to ugly >> traffic problems. > Ah it is TCP at the moment. Not something I could change too easily either. > Is it possible to work around or is it not worth fighting with? If all of these cases are true: * You only have TCP traffic going over that VPN * You don't have any latency-sensitive traffic going over that VPN (no VOIP, no interactive terminal sessions and you won't pull your hair out over 10s or more round-trips slowing down page loads) * You don't have large bulk data transfers going over that VPN (my best example of personal experience here was trying to locally sync my work-related IMAP mailbox) ...then it's not worth fighting with. It's very unlikely you fall in that camp. The problem of TCP VPN transport is a confluence of three issues: 1) You're likely to have packet loss underneath that transport due to things like congestion...the TCP transport will hide this from tunneled traffic and retransmit itself. 2) In TCP, Nagle's Algorithm handles flow throttling, but it depends on detecting packet loss to limit how many packets it pushes. 3) Your VPN endpoint will very probably buffer a very large amount of data for sending if its TCP transport link is acting slow. Here's what happens: 1) Your TCP app on your computer opens a connection with a remote host. This connection is encapsulated inside your TCP OpenVPN tunnel. 2) Your app's TCP connection starts exchanging data. For as long as it's not losing any packets, it figures it can send more and more data without waiting for a response; this is Nagle's Algorithm managing your TCP sliding window, and it's why TCP can scale from dial-up speeds up to 10g ethernet. 3) Your VPN link's TCP connection experiences packet loss. Maybe it's because of a congested router between you and the remote side of the VPN, or maybe it's because someone's ADSL connection is pushing more than its measly 768Kb/s upstream speed allows for. Or maybe it's noise on the copper causing packet loss on the ADSL link. Or maybe it's a frame collision on the PPoE link. ...time passes... 4) Your VPN link's TCP stack notes the packet loss and retransmits the lost packet until the packet gets through. 5) The connection traffic from step (2) is completely unaware that the VPN's TCP connection is fielding packet loss issues for it, and Nagle's Algorithm figures, 'hey, this is a high-bandwith link! Let's shove more data!' 6) OpenVPN link is now receving data it can't stuff into the pipe right this second, so it buffers it for a moment, and then sends it when its turn has come. Still, no data is lost. ...time passes... 7) Steps 4-6 repeat themselves, causing your original connection to become more and more confident about the bandwidth of the pipe. ...time passes... 8) The connection from step 2 is now so confident of the connection speed of the pipe, it's pushing data to OpenVPN faster than OpenVPN could conceivably push out, even if there were no packet loss issues. You've now got a cycle of just steps 5 and 6. Presumably, you'd eventually hit OpenVPN's buffer limit. I don't know what that is, and I don't think it's tuneable. The one time I personally saw, measured and helped diagnose this, I was getting up to a *fifteen minute* round-trip ping time over the VPN, even though the round-trip time for ping outside the VPN between the VPN endpoints was only about 100ms. Watching that round-trip time climb was surreal until I figured out what was happening. Switching the VPN transport to UDP allowed the tunneled connections' TCP stacks to properly gauge and react to available throughput. Even SIP started working over that VPN link. -- :wq
Re: [gentoo-user] Traffic shaping - downstream data
On Tue, Jun 12, 2012 at 11:06 AM, Michael Mol wrote: > On Tue, Jun 12, 2012 at 9:37 AM, Datty wrote: >> On Tue, Jun 12, 2012 at 2:21 PM, Michael Mol wrote: >>> On Jun 12, 2012 8:59 AM, "Datty" wrote: > > [snip] > >>> More detail later...but make sure your vpn link is not TCP. UDP, fine, >>> IP-IP, fine, but not TCP. TCP transport for a VPN tunnel leads to ugly >>> traffic problems. > >> Ah it is TCP at the moment. Not something I could change too easily either. >> Is it possible to work around or is it not worth fighting with? > > If all of these cases are true: > > * You only have TCP traffic going over that VPN > * You don't have any latency-sensitive traffic going over that VPN (no > VOIP, no interactive terminal sessions and you won't pull your hair > out over 10s or more round-trips slowing down page loads) > * You don't have large bulk data transfers going over that VPN (my > best example of personal experience here was trying to locally sync my > work-related IMAP mailbox) > > ...then it's not worth fighting with. I could stand to be more precise and concise: If you're going to use a TCP transport for VPN: * You need to not mix TCP and UDP traffic * You need to not have latency-sensitive traffic. In practice, you'll almost always have some UDP traffic; that's how DNS generally operates. And even where DNS uses TCP, it's still latency-sensitive. So I can be even more concise: If you're going to use a TCP transport for VPN, you must avoid having TCP traffic over that VPN link. -- :wq
Re: [gentoo-user] Traffic shaping - downstream data
On Tue, Jun 12, 2012 at 5:05 PM, Michael Mol wrote: > On Tue, Jun 12, 2012 at 11:06 AM, Michael Mol wrote: > > On Tue, Jun 12, 2012 at 9:37 AM, Datty wrote: > >> On Tue, Jun 12, 2012 at 2:21 PM, Michael Mol wrote: > >>> On Jun 12, 2012 8:59 AM, "Datty" wrote: > > > > [snip] > > > >>> More detail later...but make sure your vpn link is not TCP. UDP, fine, > >>> IP-IP, fine, but not TCP. TCP transport for a VPN tunnel leads to ugly > >>> traffic problems. > > > >> Ah it is TCP at the moment. Not something I could change too easily > either. > >> Is it possible to work around or is it not worth fighting with? > > > > If all of these cases are true: > > > > * You only have TCP traffic going over that VPN > > * You don't have any latency-sensitive traffic going over that VPN (no > > VOIP, no interactive terminal sessions and you won't pull your hair > > out over 10s or more round-trips slowing down page loads) > > * You don't have large bulk data transfers going over that VPN (my > > best example of personal experience here was trying to locally sync my > > work-related IMAP mailbox) > > > > ...then it's not worth fighting with. > > I could stand to be more precise and concise: > If you're going to use a TCP transport for VPN: > * You need to not mix TCP and UDP traffic > * You need to not have latency-sensitive traffic. > > In practice, you'll almost always have some UDP traffic; that's how > DNS generally operates. And even where DNS uses TCP, it's still > latency-sensitive. > > So I can be even more concise: > If you're going to use a TCP transport for VPN, you must avoid having > TCP traffic over that VPN link. > > -- > :wq > Thank you for that very thorough explanation, I had no idea there was a problem with using TCP, I figured the error correction would help it be more stable than just throwing data at it and hoping it got there. Somehow I've avoided the majority of the issues you've mentioned up to now, but then again generally my connection is very slow so maybe I'm just not feeling the effects. My ping however is around 40ms higher over the VPN link so I'm guessing that may be a sign. I'll set up a second vpn tunnel using UDP to test it out, my resistance to changing the main one is purely down to the fact that I have around 30 clients, probably half of which would reach for antiseptic if I mentioned TCP and I don't fancy having to drive 200+ miles to each of them to change it for them. I'll give it a shot tomorrow and report back on how it gets on. Regarding the tc rules I mentioned, do they look alright? I'm not 100% on how it all goes together still and would appreciate a prod in the right direction. Thanks again Oliver
Re: [gentoo-user] Traffic shaping - downstream data
On Tue, Jun 12, 2012 at 4:43 PM, Datty wrote: > > > On Tue, Jun 12, 2012 at 5:05 PM, Michael Mol wrote: >> >> On Tue, Jun 12, 2012 at 11:06 AM, Michael Mol wrote: >> > On Tue, Jun 12, 2012 at 9:37 AM, Datty wrote: >> >> On Tue, Jun 12, 2012 at 2:21 PM, Michael Mol wrote: >> >>> On Jun 12, 2012 8:59 AM, "Datty" wrote: >> > >> > [snip] >> > >> >>> More detail later...but make sure your vpn link is not TCP. UDP, fine, >> >>> IP-IP, fine, but not TCP. TCP transport for a VPN tunnel leads to ugly >> >>> traffic problems. >> > >> >> Ah it is TCP at the moment. Not something I could change too easily >> >> either. >> >> Is it possible to work around or is it not worth fighting with? >> > >> > If all of these cases are true: >> > >> > * You only have TCP traffic going over that VPN >> > * You don't have any latency-sensitive traffic going over that VPN (no >> > VOIP, no interactive terminal sessions and you won't pull your hair >> > out over 10s or more round-trips slowing down page loads) >> > * You don't have large bulk data transfers going over that VPN (my >> > best example of personal experience here was trying to locally sync my >> > work-related IMAP mailbox) >> > >> > ...then it's not worth fighting with. >> >> I could stand to be more precise and concise: >> If you're going to use a TCP transport for VPN: >> * You need to not mix TCP and UDP traffic >> * You need to not have latency-sensitive traffic. >> >> In practice, you'll almost always have some UDP traffic; that's how >> DNS generally operates. And even where DNS uses TCP, it's still >> latency-sensitive. >> >> So I can be even more concise: >> If you're going to use a TCP transport for VPN, you must avoid having >> TCP traffic over that VPN link. >> >> -- >> :wq > > > Thank you for that very thorough explanation, I had no idea there was a > problem with using TCP, I figured the error correction would help it be more > stable than just throwing data at it and hoping it got there. Somehow I've > avoided the majority of the issues you've mentioned up to now, but then > again generally my connection is very slow so maybe I'm just not feeling the > effects. My ping however is around 40ms higher over the VPN link so I'm > guessing that may be a sign. > > I'll set up a second vpn tunnel using UDP to test it out, my resistance to > changing the main one is purely down to the fact that I have around 30 > clients, probably half of which would reach for antiseptic if I mentioned > TCP and I don't fancy having to drive 200+ miles to each of them to change > it for them. > > I'll give it a shot tomorrow and report back on how it gets on. Regarding > the tc rules I mentioned, do they look alright? I'm not 100% on how it all > goes together still and would appreciate a prod in the right direction. > > Thanks again I'd suggest setting up that second VPN link for parallel use, get all the clients up on that one (can you remote admin?), and then take down the old one once the TCP link is no longer actively used. You should be able to do it pretty seamlessly. Regarding the tc rules...no idea off the top of my head. I think when I first saw them I had more questions about topology, but I don't have time to grok it again today. -- :wq
Re: [gentoo-user] traffic shaping and p2p
On Wed, 2005-12-14 at 20:02 +0100, Matthias Langer wrote: > I've a small home network, actuall consisting of two gentoo boxes, where > one box acts as router, firewall, svn server and desktop for my sister > (i know this isn't an optimal setup) and the other one is my > workstation. > > Now, when i start a p2p app on my workstation the latency of my internet > connection suffers greatly, allthogh i've > 384 kbit/s up and 3072 kbit/s down. I know that there are some > approaches to solve this kind of problem by categorizing packets and > assign different priorities to them, as explained at > http://gentoo-wiki.com/HOWTO_Packet_Shaping. However, my knowledge of > iptables and networking is very limited and i just want a simple and > clean solution as i don't plan to trick myself by switching my p2p apps > to non standard ports or manipulating the packet size ... > > To cut a long story short: I want high latency for ssh, browsing, (what i mean is in fact low latency :-) > subversion while offering p2p services a maximum of bandwidth in a small > homenetwork containing only 2 boxes. > > Any suggestions ? > > Thanks, > Matthias > -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] traffic shaping and p2p
Matthias Langer wrote: Now, when i start a p2p app on my workstation the latency of my internet connection suffers greatly, allthogh i've 384 kbit/s up and 3072 kbit/s down. I know that there are some approaches to solve this kind of problem by categorizing packets and assign different priorities to them, as explained at http://gentoo-wiki.com/HOWTO_Packet_Shaping. However, my knowledge of iptables and networking is very limited and i just want a simple and clean solution as i don't plan to trick myself by switching my p2p apps to non standard ports or manipulating the packet size ... I've used that HOWTO (and contributed bits to it), and its great. But why can't you just limit your P2P application's upload speed? I mean, the program should have some controls that let you do that, right? I know every sane bittorrent app has this. -- [Name ] :: [Matan I. Peled] [Location ] :: [Israel] [Public Key] :: [0xD6F42CA5] [Keyserver ] :: [keyserver.kjsl.com] encrypted/signed plain text preferred -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] traffic shaping and p2p
On Thu, 2005-12-15 at 09:53 +0200, Matan Peled wrote: > Matthias Langer wrote: > > Now, when i start a p2p app on my workstation the latency of my internet > > connection suffers greatly, allthogh i've > > 384 kbit/s up and 3072 kbit/s down. I know that there are some > > approaches to solve this kind of problem by categorizing packets and > > assign different priorities to them, as explained at > > http://gentoo-wiki.com/HOWTO_Packet_Shaping. However, my knowledge of > > iptables and networking is very limited and i just want a simple and > > clean solution as i don't plan to trick myself by switching my p2p apps > > to non standard ports or manipulating the packet size ... > > I've used that HOWTO (and contributed bits to it), and its great. > > But why can't you just limit your P2P application's upload speed? I mean, the > program should have some controls that let you do that, right? I know every > sane > bittorrent app has this. > Well, i use azureus - and of course i know that upload-speed can be limited - which is maybe in fact the best solution to my problem. However, what i have in mind is somehow similar to cpu-resources and process-priority. If i start at process with nice level 15, it will get all available cpu-resources without slowing down the other apps. As far as i understand, this is not the same as limiting the process to, say 80% of cpu power. Now, what i want is the same for p2p apps - give them as much bandwidth they can reasonably get but don't let them slow down firefox, ssh etc. Because i want this setup just for my homenetwork, it would perfectly suffice if packages get their priorities by examining port-numbers. And because i want to at least partially understand what i'm doing i would prefer a simple and clean setup. I know that in principle the neccessairy steps to do what i wannt can be found in the 'Packet Shaping HOWTO'. But i wanted to hear experiences and opinions of others first before starting messing around with my router. By the way, there are many different packet shedulers in the kernel - and the HOWTO only explains the HTP-scheduler. What about the other schedulers - can they be usefull for my purposes too - and if yes, how can they be configured and used ? Matthias -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] traffic shaping and p2p
Matthias Langer schreef: > Now, what i want is the same for p2p apps - give them > as much bandwidth they can reasonably get but don't let them slow down > firefox, ssh etc. In the case of Azureus specifically, your problem is actually not with Azureus, but with Java (that's what's slowing down, and further what is likely to be slowing down Firefox as well if it's running. Certainly I find that running both Firefox and Azureus together is the fast road to The System of Molasses). You might consider aliasing Java to run at a "good" niceness (in ~/.bashrc) alias java="nice -n 15 java" so that when Azureus starts the (many, many) Java processes that it uses, they will be niced to something you can live with. What effect this will have on Firefox, I cannot say, however. Just an idea, hope it helps, Holly -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] traffic shaping and p2p
On Thu, 2005-12-15 at 20:15 +0100, Holly Bostick wrote: > Matthias Langer schreef: > > > Now, what i want is the same for p2p apps - give them > > as much bandwidth they can reasonably get but don't let them slow down > > firefox, ssh etc. > > In the case of Azureus specifically, your problem is actually not with > Azureus, but with Java (that's what's slowing down, and further what is > likely to be slowing down Firefox as well if it's running. Certainly I > find that running both Firefox and Azureus together is the fast road to > The System of Molasses). Hmm, i can't confirm this, bacause as long as azureus is not down/uploading heavily browsing is not really affected. But this may differ from java-vm to vm. I use sun-jdk-1.5.05 because i do same java programming stuff ... > > You might consider aliasing Java to run at a "good" niceness > > (in ~/.bashrc) > > alias java="nice -n 15 java" > > so that when Azureus starts the (many, many) Java processes that it > uses, they will be niced to something you can live with. > > What effect this will have on Firefox, I cannot say, however. > > Just an idea, hope it helps, > > Holly > But thanks for your answer nevertheless, Matthias -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] traffic shaping and p2p
On Dec 15, 2005, at 5:05 pm, Matthias Langer wrote: Well, i use azureus - and of course i know that upload-speed can be limited - which is maybe in fact the best solution to my problem. ... for p2p apps - give them as much bandwidth they can reasonably get but don't let them slow down firefox, ssh etc. Because i want this setup just for my homenetwork, it would perfectly suffice if packages get their priorities by examining port-numbers. And because i want to at least partially understand what i'm doing i would prefer a simple and clean setup. I haven't used it yet, but my understanding of traffic-shaping is that it's exactly what you want. I believe that other quality-of-service mechanisms may require applications to be QoS aware (setting a QoS bit in the packet header). You're absolutely right in that reducing the bandwidth of the p2p app isn't the ideal way to achieve what you want - I find latency in browsing & surfing with BitTorrent consuming only 60% - 70% of my upload - it doesn't help that other peers are continually making requests of you. If you lower the bandwidth consumption in Azureous then you have to remember to up it again when you go to bed - traffic shaping WILL allow you to permanently maximise your p2p bandwidth, with the ROUTER reducing it only when your priority services send packets. I know that in principle the neccessairy steps to do what i wannt can be found in the 'Packet Shaping HOWTO'. By the way, there are many different packet shedulers in the kernel - and the HOWTO only explains the HTP-scheduler. What about the other schedulers - can they be usefull for my purposes too - and if yes, how can they be configured and used ? No idea. I hope you'll give us feedback when you've discovered more. Stroller. -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] traffic shaping and p2p
On Dec 15, 2005, at 7:15 pm, Holly Bostick wrote: Matthias Langer schreef: Now, what i want is the same for p2p apps - give them as much bandwidth they can reasonably get but don't let them slow down firefox, ssh etc. In the case of Azureus specifically, your problem is actually not with Azureus, but with Java ... You might consider aliasing Java to run at a "good" niceness Hi Holly, Matthias seems to have confused the issue with his "bandwidth niceness" analogy. I believe that his problem are with saturation of his broadband connection, in which case he'd get the same problem even if Azureus was running on a different PC from his web-browser. Matthias wants to give p2p maximum bandwidth & have the router sort it out so that he gets no latency on other connections - this is what traffic-shaping does. Stroller. -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] traffic shaping and p2p
>From the sounds of things it looks like this is a problem with network latency, not any issue with his computer. One of the first things that should be done to help would be adjust the settings in Azureus for your specific up/dl speeds. This is a pretty good guide to get started on that. http://azureus.aelitis.com/wiki/index.php/Good_settings The other thing I would recommend doing is using the plugin "Auto Speed". This plugin will automatically adjust your dl & ul speeds according to the network latency. Note, If your router blocks or rejects ICMP then you will not be able to use the plugin.
Re: [gentoo-user] traffic shaping and p2p
On Fri, 2005-12-16 at 13:08 +, Stroller wrote: > On Dec 15, 2005, at 5:05 pm, Matthias Langer wrote: > > > > Well, i use azureus - and of course i know that upload-speed can be > > limited - which is maybe in fact the best solution to my problem. > > ... for p2p apps - give them > > as much bandwidth they can reasonably get but don't let them slow down > > firefox, ssh etc. Because i want this setup just for my homenetwork, it > > would perfectly suffice if packages get their priorities by examining > > port-numbers. And because i want to at least partially understand what > > i'm doing i would prefer a simple and clean setup. > > I haven't used it yet, but my understanding of traffic-shaping is that > it's exactly what you want. I believe that other quality-of-service > mechanisms may require applications to be QoS aware (setting a QoS bit > in the packet header). > > You're absolutely right in that reducing the bandwidth of the p2p app > isn't the ideal way to achieve what you want - I find latency in > browsing & surfing with BitTorrent consuming only 60% - 70% of my > upload - it doesn't help that other peers are continually making > requests of you. If you lower the bandwidth consumption in Azureous > then you have to remember to up it again when you go to bed - traffic > shaping WILL allow you to permanently maximise your p2p bandwidth, with > the ROUTER reducing it only when your priority services send packets. > > > I know that in > > principle the neccessairy steps to do what i wannt can be found in the > > 'Packet Shaping HOWTO'. By the way, > > there are many different packet shedulers in the kernel - and the HOWTO > > only explains the HTP-scheduler. What about the other schedulers - can > > they be usefull for my purposes too - and if yes, how can they be > > configured and used ? > > No idea. I hope you'll give us feedback when you've discovered more. Ok i found out that in fact the HFSC scheduler should be the one which does exactly what i like because it handles bandwidth and latency seperatley. Here is my current setup, which seems not to be ideal - ssh is still slow when my upload is high: # create the following tree # 1: # 1:1 # 1:10 1:20 1:301:40 # where 1:10 is for ssh, 1:20 for svn, 1:30 for surfing and 1:40 for unmatched traffic # creates the root qdisc tc qdisc add dev eth0 root handle 1: hfsc default 40 # node 1:1 tc class add dev eth0 parent 1: classid 1:1 hfsc sc rate 441kbit ul rate 441kbit # node 1:10 (ssh) - guaranty 1500b in 20ms with an overarall rate of 88kbit tc class add dev eth0 parent 1:1 classid 1:10 hfsc sc umax 800b dmax 20ms rate 88kbit # node 1:20 (svn) - guaranty 1500b in 30ms with an overall rate of of 147kbit tc class add dev eth0 parent 1:1 classid 1:20 hfsc sc umax 800b dmax 30ms rate 147kbit # node 1:30 (firefox) - garanty 2b in 100ms with an overall rate of 120kbit tc class add dev eth0 parent 1:1 classid 1:30 hfsc sc umax 2b dmax 100ms rate 120kbit #node 1:40 (unmatched) tc class add dev eth0 parent 1:1 classid 1:40 hfsc sc rate 96kbit # now that we have our qdiscs we need filters for them # ssh tc filter add dev eth0 protocol ip parent 1: prio 1 u32 match ip dport 22 0x flowid 1:10 # svn tc filter add dev eth0 protocol ip parent 1: prio 2 u32 match ip dport 3690 0x flowid 1:20 # firefox tc filter add dev eth0 protocol ip parent 1: prio 3 u32 match ip dport 80 0x flowid 1:30 Note that i use the u32 filter (must be enabled in the kernel) and not iptables. By the way, there is a very interesting article about traffic control with qdiscs and different schedulers, in particular HFSC, written by the author of HFSC himself in the german 'Linux Magazin' 02/2005. I'll tell you about further experiences with HFSC - comments and suggestions are welcome. Matthias -- gentoo-user@gentoo.org mailing list
