Re: [LARTC] HTB, strange capacity distribution
On Tue, Feb 21, 2006 at 02:21:36PM +0100, Boris Gereg wrote: thanks Andreas, I reconfigured HTB to get your suggested hierarhy: One thing I forgot in my graph: Make sure that the rates always add up, i.e. the sum of the child class rates should equal the parent class rate. It's unlikely to be the cause of your problem, but it's important to get this right nevertheless. Any other things to test, please? Just to see wether we are going into the right direction at all, could you run the following experiment: - Lower the rate and ceil of class 1:2 to 8096kbit. - Lower the rate and ceil of class 1:2000 to 7072kbit. - Lower the rate and ceil of class 1:3000 to 1024kbit. - For class 1:3010, set rate to 64kbit, ceil to 256kbit. - For class 1:3020, set rate to 128kbit, ceil to 768kbit. - For class 1:3040, set rate to 704kbit, ceil to 1024kbit. - For class 1:5040, set rate to 128kbit, ceil to 1024kbit. (You can adjust the rates for these classes as you like, just make sure that the sum is 1024kbit) If in this setup the shaping works as expected - WWW should get 704kbit at all times, P2P only slightly more than 128kbit while WWW downloads are active - then the limiting and distribution of HTB most likely works, and it's just too high rates or r2q/quantum that make it go bad. In this case, you'd have to measure realistic throughput rates of your network (even a 100mbit LAN may not be able to guarantee 10kbit at all times) and of your internet connection (may not be able to serve 2048kbit at all times). For downstream shaping to work, you have to be the bottleneck. If you get the same problem in this setup (P2P taking all the bandwidth away from WWW), then the problem is most likely something different, and we have to look at it from a different angle. Regards Andreas Klauer ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re[2]: [LARTC] HTB, strange capacity distribution
Hello Andreas, AK Just to see wether we are going into the right direction at all, could AK you run the following experiment: AK - Lower the rate and ceil of class 1:2 to 8096kbit. AK - Lower the rate and ceil of class 1:2000 to 7072kbit. AK - Lower the rate and ceil of class 1:3000 to 1024kbit. AK - For class 1:3010, set rate to 64kbit, ceil to 256kbit. AK - For class 1:3020, set rate to 128kbit, ceil to 768kbit. AK - For class 1:3040, set rate to 704kbit, ceil to 1024kbit. AK - For class 1:5040, set rate to 128kbit, ceil to 1024kbit. I did what you suggested and the results are as expected! You can see this picture to verify: http://elusion.sk/visual_inet_7.png At 4:25 I started HTTP download. P2P class immediately droped down to it's RATE, WWW class got it's RATE. At 4:33 I stopped HTTP download, P2P class got rest of capacity. AK ... it's just too high rates or r2q/quantum that make it go bad. In this AK case, you'd have to measure realistic throughput rates of your network AK (even a 100mbit LAN may not be able to guarantee 10kbit at all times) AK and of your internet connection (may not be able to serve 2048kbit at AK all times). For downstream shaping to work, you have to be the bottleneck. There are messages in syslog like this: kernel: HTB: quantum of class 10002 is big. Consider r2q change. kernel: HTB: quantum of class 12000 is big. Consider r2q change. kernel: HTB: quantum of class 13010 is small. Consider r2q change. Please, are there some hints for setting r2q or quantum parameters? http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm does not give extensive information on this. Or, is it just the matter of testing and searching for optimal parameters? Anyway, thanks a lot Andreas. This was the most important break for me. Rest is just tuning. Best Regards B. Gereg ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] HTB, strange capacity distribution
On Thu, Feb 23, 2006 at 05:00:12AM +0100, Boris Gereg wrote: I did what you suggested and the results are as expected! You can see this picture to verify: http://elusion.sk/visual_inet_7.png At 4:25 I started HTTP download. P2P class immediately droped down to it's RATE, WWW class got it's RATE. At 4:33 I stopped HTTP download, P2P class got rest of capacity. Alright. I suggest you do some measuring, to find out your real rates, and set HTB rates to be slightly lower so that you are the bottleneck. Most likely you'll have to experiment a little until you find the best setting for your setup. Please, are there some hints for setting r2q or quantum parameters? Actually, I specify the quantum directly, with 'quantum $MTU' for every class. I don't know wether that's a good thing or a bad thing, but it worked very well for me, and seems to work well for others... at least nobody reported a problem to me so far that could be traced to be caused by this quantum setting. It should not be smaller than your MTU, and not too big. With a huge difference in rates (100Mbit vs 64kbit) there is no r2q that will fit all classes. So there is no other way as to set quantum directly at least for some classes (and I set it for all...). Regards Andreas Klauer ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] HTB, strange capacity distribution
Hello, thanks Andreas, I reconfigured HTB to get your suggested hierarhy: AK 1: HTB Qdisc AK | AK \--- 1:2 HTB root class (10Kbit:10kbit) AK | AK \--- 1:2000 HTB leaf class (5Kbit:10Kbit) #local AK | AK \--- 1:3000 HTB parent class (2048Kbit:2048Kbit) AK | AK \--- 1:3010 HTB leaf class (64000bit:256000bit) #ctrl AK \--- 1:3020 HTB leaf class (128000bit:768000bit) #other AK \--- 1:3030 HTB leaf class (512000bit:2048Kbit) #www AK \--- 1:5040 HTB leaf class (386000bit:386000bit) #p2p tc -d class show dev eth0 class htb 1:2 root rate 10Kbit ceil 10Kbit burst 51587b/8 mpu 0b overhead 0b cburst 51587b/8 mpu 0b overhead 0b level 7 # local: class htb 1:2000 parent 1:2 leaf 2000: prio 0 quantum 20 rate 5Kbit ceil 10Kbit burst 26593b/8 mpu 0b overhead 0b cburst 51587b/8 mpu 0b overhead 0b level 0 # root for internet trafic class htb 1:3000 parent 1:2 rate 2048Kbit ceil 2048Kbit burst 2623b/8 mpu 0b overhead 0b cburst 2623b/8 mpu 0b overhead 0b level 6 # ctrl class class htb 1:3010 parent 1:3000 leaf 3010: prio 1 quantum 1000 rate 64000bit ceil 256000bit burst 1631b/8 mpu 0b overhead 0b cburst 1727b/8 mpu 0b overhead 0b level 0 # other class class htb 1:3020 parent 1:3000 leaf 3020: prio 2 quantum 1600 rate 128000bit ceil 768000bit burst 1663b/8 mpu 0b overhead 0b cburst 1983b/8 mpu 0b overhead 0b level 0 # www class class htb 1:3030 parent 1:3000 leaf 3030: prio 3 quantum 12800 rate 1024Kbit ceil 2048Kbit burst 2111b/8 mpu 0b overhead 0b cburst 2623b/8 mpu 0b overhead 0b level 0 # p2p class class htb 1:5040 parent 1:3000 leaf 5040: prio 4 quantum 1600 rate 128000bit ceil 256000bit burst 1663b/8 mpu 0b overhead 0b cburst 1727b/8 mpu 0b overhead 0b level 0 Sorry to say, results are not as expected. I made new measurements, please see following picture (based on HTB config above): http://elusion.sk/visual_inet_6.png Point A - until now, p2 clas had CEIL 386 kbit. Now, I reset p2p class CEIL to 2048 kbit. As you can see, p2p rised as expected Point B - www class RATE-CEIL is 512-2048 kbit, p2p class RATE-CEIL is 128-2048 kbit. PROBLEM: why did not www get more capacity? Point C - www class RATE-CEIL is 1536-2048 kbit, p2p class RATE-CEIL is 128-2048 kbit. PROBLEM: RATE of www is 1536, but class did not get this capacity; p2p should fall to it's RATE 256 kbit Point D - www class RATE-CEIL is 1024-2048 kbit, p2p class RATE-CEIL is 128-2048 kbit - no change Point E - www class RATE-CEIL is 1024-2048 kbit, p2p class RATE-CEIL is 128-256 kbit - as expected: p2p is bounded to 256 kbit limit, so the rest of the capacity was used by www class I have expected that in points B, C and D, p2p class falls down to it's specified rate and www gets most of the capacity. But this is not happening - p2p will not fall down Any other things to test, please? Best regards, B. Gereg mailto:[EMAIL PROTECTED] -Originalna sprava- Od: Andreas Klauer [mailto:[EMAIL PROTECTED] Poslane: Tuesday, February 21, 2006, 8:52:49 AM Komu: [EMAIL PROTECTED] Predmet: [LARTC] HTB, strange capacity distribution AK On Tue, Feb 21, 2006 at 12:49:59AM +0100, Boris Gereg wrote: (first of all, please, how to reply to some article in LARTC via mail to post it into right thread?) AK Using 'reply all', or 'reply list' if your mail software offers it. AK If all else fails, just hit 'reply' and add the mailing list to CC. So, I am definitely shaping outgoing traffic (upstream) AK Yes, outgoing traffic from router to your network, which actually AK contains the downstream traffic from the internet. Right? tc -d class show dev eth0 class htb 1:2 root rate 10Kbit ceil 10Kbit burst 51587b/8 mpu 0b overhead 0b cburst 51587b/8 mpu 0b overhead 0b level 7 class htb 1:2000 parent 1:2 leaf 2000: prio 0 quantum 20 rate 5Kbit ceil 10Kbit burst 26593b/8 mpu 0b overhead 0b cburst 51587b/8 mpu 0b overhead 0b level 0 class htb 1:3010 parent 1:2 leaf 3010: prio 1 quantum 1000 rate 64000bit ceil 256000bit burst 1631b/8 mpu 0b overhead 0b cburst 1727b/8 mpu 0b overhead 0b level 0 class htb 1:3020 parent 1:2 leaf 3020: prio 2 quantum 1600 rate 128000bit ceil 768000bit burst 1663b/8 mpu 0b overhead 0b cburst 1983b/8 mpu 0b overhead 0b level 0 class htb 1:3030 parent 1:2 leaf 3030: prio 3 quantum 6400 rate 512000bit ceil 2048Kbit burst 1855b/8 mpu 0b overhead 0b cburst 2623b/8 mpu 0b overhead 0b level 0 class htb 1:5040 parent 1:2 leaf 5040: prio 4 quantum 4825 rate 386000bit ceil 386000bit burst 1792b/8 mpu 0b overhead 0b cburst 1792b/8 mpu 0b overhead 0b level 0 AK It's as I suspected, your current HTB tree looks like this: AK 1: HTB Qdisc AK | AK \--- 1:2 HTB root class (10Kbit:10kbit) AK | AK \--- 1:2000 HTB leaf class (5Kbit:10Kbit) AK \--- 1:3010 HTB leaf class (64000bit:256000bit) AK \--- 1:3020 HTB leaf class (128000bit:768000bit) AK \--- 1
[LARTC] HTB, strange capacity distribution
Hello, after spending several hours reading archives, I decided to write new post. I successfully set up packet classification, made some basic HTB setup, made some simple graphical representation from HTB statistics data... BUT, I cannot figure out how to refine HTB to get this behaviour: I need that class p2p should be the last one to get some link capacity. If I set both RATE and CEIL for p2p to some constant value, everything is OK. When I set CEIL for p2p e.g. to 2048 kbps (total link capacity), p2p starts to get most of the link capacity and does not want to give capacity to other classes with higher priorities! I made a screen to help explain my problem. Please, see this picture: http://elusion.sk/visual_inet_hory.png Point A - p2p class (red line) has both RATE and CEIL set to fixed value. I started download from web to fill www class. You can see, www got remaining link capacity, p2p without a change - all as expected. Point B - I reset CEIL of p2p class to 2048 kbit (to use rest of the unused capacity by other classes with higher priorites, up to link capacity) Point C - I started the same download from web as in Point A and this is the PROBLEM. You can see, that p2p lowered a bit, and www class got some capacity. What I expect is, that www class get most of the link capacity and p2p class will fall to it's RATE, because http class has higher priority than p2p Point D - CEIL of p2p reset to fixed value Point E - I tried the same download from web just to be sure it works. This is my HTB config (using latest htb-init script): root: RATE=100Mbit ( local traffic: RATE=50Mbit CEIL=100Mbit PRIO=0 ) ctrl class: RATE=64kbit CEIL=256kbit PRIO=1 other class: RATE=128kbit CEIL=768kbit PRIO=2 www class: RATE=512kbit CEIL=2048kbit PRIO=3 p2p class: RATE=386kbit CEIL=386kbit (or 2048, for Points B and C in figure above) PRIO=4 All want to set up is: If class www with prio 3 gets fully utilized (I start 10 http downloads at the same time), I expect p2p class with prio 4 to fall down to it's RATE 386 kbit - to free capacity for www class In another words: I expect www to overrule p2p (p2p still getting it's RATE) Can someone help to explain this? Maybe I did not get purpose of PRIO setting, not sure. Help, please Best Regards, B. Gereg ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] HTB, strange capacity distribution
On Mon, Feb 20, 2006 at 10:59:33PM +0100, Boris Gereg wrote: I made a screen to help explain my problem. Please, see this picture: http://elusion.sk/visual_inet_hory.png Nice graph. I assume this is on downstream, and you rely on HTB to drop packets for you. You may have read this in the archives already - it's much harder to shape downstream than upstream, because you can't really influence what the other side is sending you. So no matter what you do it's probably hard to get near-optimal results. This is my HTB config (using latest htb-init script): I must admit I'm not familiar with htb-init. What are the parent-child relationships here? I'm missing the internet parent class that groups all the other traffics (except local) together. Does htb-init generate that on it's own somehow? If not, chances are your HTB tree is just exceeding your line capacity in general, as all classes are allowed to borrow without limit, rendering the prio setting uneffective, leading to random results. Could you post the output of 'tc -d qdisc/class show dev $DEVICE'? Regards, Andreas Klauer ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] HTB, strange capacity distribution
On Tue, Feb 21, 2006 at 12:49:59AM +0100, Boris Gereg wrote: (first of all, please, how to reply to some article in LARTC via mail to post it into right thread?) Using 'reply all', or 'reply list' if your mail software offers it. If all else fails, just hit 'reply' and add the mailing list to CC. So, I am definitely shaping outgoing traffic (upstream) Yes, outgoing traffic from router to your network, which actually contains the downstream traffic from the internet. Right? tc -d class show dev eth0 class htb 1:2 root rate 10Kbit ceil 10Kbit burst 51587b/8 mpu 0b overhead 0b cburst 51587b/8 mpu 0b overhead 0b level 7 class htb 1:2000 parent 1:2 leaf 2000: prio 0 quantum 20 rate 5Kbit ceil 10Kbit burst 26593b/8 mpu 0b overhead 0b cburst 51587b/8 mpu 0b overhead 0b level 0 class htb 1:3010 parent 1:2 leaf 3010: prio 1 quantum 1000 rate 64000bit ceil 256000bit burst 1631b/8 mpu 0b overhead 0b cburst 1727b/8 mpu 0b overhead 0b level 0 class htb 1:3020 parent 1:2 leaf 3020: prio 2 quantum 1600 rate 128000bit ceil 768000bit burst 1663b/8 mpu 0b overhead 0b cburst 1983b/8 mpu 0b overhead 0b level 0 class htb 1:3030 parent 1:2 leaf 3030: prio 3 quantum 6400 rate 512000bit ceil 2048Kbit burst 1855b/8 mpu 0b overhead 0b cburst 2623b/8 mpu 0b overhead 0b level 0 class htb 1:5040 parent 1:2 leaf 5040: prio 4 quantum 4825 rate 386000bit ceil 386000bit burst 1792b/8 mpu 0b overhead 0b cburst 1792b/8 mpu 0b overhead 0b level 0 It's as I suspected, your current HTB tree looks like this: 1: HTB Qdisc | \--- 1:2 HTB root class (10Kbit:10kbit) | \--- 1:2000 HTB leaf class (5Kbit:10Kbit) \--- 1:3010 HTB leaf class (64000bit:256000bit) \--- 1:3020 HTB leaf class (128000bit:768000bit) \--- 1:3030 HTB leaf class (512000bit:2048Kbit) \--- 1:5040 HTB leaf class (386000bit:386000bit) HTB classes borrow from their parent; in this setup, the parent class offers a whopping 10Kbit for that purpose. Unless the 1:2000 class has got a very high priority and is maxing out the line all the time, there is no limit to borrowing at all, because the other classes will never reach the 10Kbit from their parent. So the classes above are actually not limited by their rate, but by their ceil; the only class that will respect its rate in this setup is 1:5040, because it's got the same rate and ceil. Assuming that 1:5040 was your P2P class, if you set the ceil of this class to 2048Kbit, it will (try to) use 2048Kbit at all times, because the parent (thinks it) is able to offer it. You need a class that knows of your total internet bandwidth somewhere. Assuming that it is 2048Kbit, your tree should maybe look more like this: 1: HTB Qdisc | \--- 1:2 HTB root class (10Kbit:10kbit) | \--- 1:2000 HTB leaf class (5Kbit:10Kbit) | \--- 1:3000 HTB parent class (2048Kbit:2048Kbit) | \--- 1:3010 HTB leaf class (64000bit:256000bit) \--- 1:3020 HTB leaf class (128000bit:768000bit) \--- 1:3030 HTB leaf class (512000bit:2048Kbit) \--- 1:5040 HTB leaf class (386000bit:386000bit) In this setup, the 2048Kbit class is the limiting factor for the leaf classes, except for the 1:2000 class, which should be used for local LAN traffic only. HTH Andreas Klauer ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
