[LARTC] HTB - not borrowing, not exceeding rate
Hello! I have a quite complicated setup. In my network on each interface there is bandwidth limitation for each user. Booth outgoing (on interface itself) and incoming (attached IMQ) traffic. There is main HTB class which limits bandwidth for whole interface and HTB subclasses for each user. Filtering is done with hashing filters. This setup was working correctly. But now in the network I have a DC hub (p2p) which allows user to exchange files. The problem is that I want in-network p2p connections to be faster than Internet bandwidth (and not eating their Internet bandwidth) and also I want in-network connections to have lower priority in borrowing the link bandwidth (I am using radio links between network segments and having good Internet access is priority over local traffic). Here is current setup: tc qdisc del root dev eth2.24 2/dev/null tc qdisc add root dev eth2.24 handle 1: htb default 1 # main rate limitation for whole connection (802.11a radio link) tc class add dev eth2.24 parent 1: classid 1:1 htb rate 15000kbit ceil 15000kbit burst 10kbit # class for internet connections - this one can use nearly whole link tc class add dev eth2.24 parent 1:1 classid 1:6667 htb rate 12000kbit ceil 13500kbit burst 100kbit # class for in-network p2p connections - this one has lower guaranteed rate tc class add dev eth2.24 parent 1:1 classid 1: htb rate 3000kbit ceil 14000kbit burst 10kbit tc qdisc add dev eth2.24 parent 1: handle : sfq perturb 5 quantum 1500b # users (htb+sfq for each): tc class add dev eth2.24 parent 1:6667 classid 1:2 htb rate 1kbit ceil 256kbit quantum 2000 burst 100kbit tc qdisc add dev eth2.24 parent 1:2 handle 2: sfq perturb 5 quantum 1500b tc class add dev eth2.24 parent 1:6667 classid 1:3 htb rate 1kbit ceil 256kbit quantum 2000 burst 100kbit tc qdisc add dev eth2.24 parent 1:3 handle 3: sfq perturb 5 quantum 1500b tc class add dev eth2.24 parent 1:6667 classid 1:1006 htb rate 1kbit ceil 384kbit quantum 2000 burst 100kbit tc qdisc add dev eth2.24 parent 1:1006 handle 1006: sfq perturb 5 quantum 1500b (some hashing filters - which I think are working properly, so not shown here) # traffic going to these networks goes to hashing filters tc filter add dev eth2.24 protocol ip parent 1:0 u32 match ip dst 192.168.1.0/24 hashkey mask 0x00ff at 16 link 2: $TC filter add dev eth2.24 protocol ip parent 1:0 u32 match ip dst 192.168.3.0/24 hashkey mask 0x00ff at 16 link 3: # now in-network p2p traffic. if it comes from other LANs then direct it to # 1: tc filter add dev eth2.24 protocol ip parent 1:0 u32 match ip src 192.168.5.0/24 flowid 1: tc filter add dev eth2.24 protocol ip parent 1:0 u32 match ip src 192.168.4.0/24 flowid 1: Filters look OK. AFAIK the last filter is the most important so even trafiic to 192.168.1.0/24 but comming from 192.168.5.0/24 will go to 1: instead of hashing filters and user bandwidth HTB. The hashing filtes were working on previous setup and now users still have their proper bandwidth from the Internet. Here is the graph representing the traffic: http://tuxpowered.net/lan_p2p/lan_eth1_rx_dzien.png the colors are: #ff00ff - htb 1: (LAN p2p) #00ff00 - htb 1:6667 (Internet) #00 - real interface traffic (tc -s li show ethXX) And now description of the problem: Class 1: never has more traffic than 'rate'. AFAIK it should be having 'rate' as minimum guaranted rate and going up to 'ceil' if there is free bandwidth. (directing LAN traffic to 1:1 works OK - look at the graph at about 14:45 - 14:55) Bandwidth sharing setup works for me on WAN interfaces, but there I have only 1 level of classes tree. Here are 2 levels and it is not working. What I want: .-. | 1:1 rate = ceil = link bandwidth| `-' | | .. .-. | 1: p2p traffic | | 1:6667 Internet traffic | | low priority in| | high prioriy in | | badwidth sharing: | | bandwidth sharing: | | ceil =~ from 1:1 | | ceil =~ from 1:1| | rate = small | | rate = (from1:1)-1: | `' `-' | +-- 1:2 user rate=1kbit ceil=256kbit +-- 1:2 user rate=1kbit ceil=256kbit . +-- 1:1203 user rate=1kbit ceil=384kbit -- | pozdrawiam / greetings | powered by Trustix, Gentoo and FreeBSD | | Kajetan Staszkiewicz | JID: [EMAIL PROTECTED] | |Vegeta | IMQ devnames: http://tuxpowered.net| `^' ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Sharing a DSL between 40 subnets with htb
gypsy wrote: I recommend that you look here. It may not be what you want, but it certainly is worth checking out even if it turns out not to be your answer: http://www.shurdix.org/ I'm not going to change the entire OS just to get the traffic shaping set up and the traffic shaper in shuredix does shaping pr. ip (which is not what I want) . However, shurdix does use the imq to do ingres shaping (aka policing) and that's a neat trick that had somehow escaped my attention, so thanks for the hint. Someone really ought to start a LARTC cookbook wiki to go with the LARTC howto. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Sharing a DSL between 40 subnets with htb
Flemming Frandsen wrote: I have a network with around 40 /24 subnets that shares a common DSL, this cries out for shaping so here I am trying to make it work as my first tc project. I have managed to cargocult some snippets from this list and tried to come up with a config, but there are a few things that I'd really like some input on: 1) Are the NAT'ed addresses available in the PREROUTING table of eth0? eth0 doesn't have a prerouting table everything coming in from anywhere hits prerouting. If eth0 is WAN then packets coming in will still have real ip addresses in PREROUTING. 2) If not then can I have the iptable --set-mark stuff in the tables for one interface and use the mark in tc on another interface? Yes tables are not device specific. 3) Is it possible to filter on the routing table in stead of the --set-mark? so all traffic going to a certain router gets filtered into the same htb? You could use tc filters on ip/dst mac etc #Have the bucket that traffic gets dropped into #be determined by the firewall mark #btw: --set-mark 0xbabeface maps to class id babe:face tc filter add dev eth0 parent 1: protocol ip prio 1 handle 1 fw Don't think you need handle 1 here. #Hook up the classification table to the interface iptables -t mangle -A PREROUTING -o eth0 -j to-dsl Out dev isn't known in prerouting. Andy. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] unexpected drop
Roberto Belletti wrote: Hello everybody, I have a Linux router with an ethernet adapter and a ADSL device. My router has a GRE tunnel connected to a remote IP address. Using tc I have configured 3 different classes: 1. 290kbit rate (used for Voip Services) 2. 180kbit rate (used for GRE tunnel) 3. 80kbit rate (for generic data traffic) For each class, using iperf, I send some data from a PC (connected attraverso eth0 device) to a remote destination. The remote destination is the GRE tunnel terminator. The data traffic test is different for each class: 1. 240kbit, 60byte packet size, UDP protocol 2. 200kbit, 250byte packet size, UDP protocol 3. 80kbit, 235byte packet size, UDP protocol In this way I got a lot of dropped packets on each class, while my expected results was a many drop packages only on the second class. iperf may be using data rates htb uses ip level packet sizes (I guess on atm0 - it's ip len +14 on eth). I don't know cbq but notice the rates don't add up. Andy. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Wondershaper and DSCP
Keith Mitchell wrote: Did anyone ever answer this one? THIS is what I am trying to do: [LARTC] cbq+sfq and DSCP marking I haven't used dscp but it looks like you need to add cbq below dsmark and then filter with tcindex see http://lartc.org/howto/lartc.adv-qdisc.dsmark.html Andy. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] HTB traffic shaping odd effects
Denis Ovsienko wrote: tc qdisc add dev eth0 root handle 1: htb default 10 tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit burst 24k Does the following help? tc qdisc add dev eth0 root handle 1: htb default 1 tc class add dev eth0 parent 1: classid 1:1 htb rate 81mbit burst 24k That seems a bit backward, limiting the total available bandwidth to enforce child class limits. Or is that how htb works, that the sum of the child classes must add up to no less than the parent class? In any case we have 100mbit available so I don't want to set it any lower than that, if possible. -- Regards, Oliver Hookins ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] HTB traffic shaping odd effects
Oliver Hookins wrote: Hi, I'm trying to perform some (what I consider) basic traffic shaping on our network utilising HTB. I have mostly reused the example on the lartc.org site: tc qdisc add dev eth0 root handle 1: htb default 10 tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit burst 24k tc class add dev eth0 parent 1:1 classid 1:10 htb rate 50mbit ceil 80mbit burst 19k burst 19k will limit you unless your HZ=1000 tc class add dev eth0 parent 1:1 classid 1:20 htb rate 1mbit ceil 1mbit tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10 tc qdisc add dev eth0 parent 1:20 handle 20: sfq perturb 10 tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip src X.X.X.X flowid 1:20 So we have a total of 100mbit to be used, the default class 1:10 gets 50mbit and a ceiling of 80mbit whereas my test host X.X.X.X gets only 1mbit in any situation. Some rate limiting is definitely happening, but I am finding the outbound traffic is limited to 2mbit instead of 1mbit. If I change the rate (to say 10mbit) the outbound traffic gets up to again twice the rate (in this case 20mbit). Not sure can you show output of tc -s -d class ls dev eth0 while it's running at 2x speed. Any thoughts? I have had a look at the tc statistics but it doesn't appear as I would expect it to. Class 1:10 shows a lot of dropped packets but it is only averaging around 30mbit constantly. On the other hand class 1:20 doesn't show any dropped packets. Similarly there are no packets marked as overlimit for any class. Apart from the 30mbit that would be normal. Andy. I occasionally see the tokens for 1:20 go negative... everything is quite strange. Any help would be appreciated. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] HTB traffic shaping odd effects
Andy Furniss wrote: Oliver Hookins wrote: Hi, I'm trying to perform some (what I consider) basic traffic shaping on our network utilising HTB. I have mostly reused the example on the lartc.org site: tc qdisc add dev eth0 root handle 1: htb default 10 tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit burst 24k tc class add dev eth0 parent 1:1 classid 1:10 htb rate 50mbit ceil 80mbit burst 19k burst 19k will limit you unless your HZ=1000 Our HZ is 512. tc class add dev eth0 parent 1:1 classid 1:20 htb rate 1mbit ceil 1mbit tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10 tc qdisc add dev eth0 parent 1:20 handle 20: sfq perturb 10 tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip src X.X.X.X flowid 1:20 So we have a total of 100mbit to be used, the default class 1:10 gets 50mbit and a ceiling of 80mbit whereas my test host X.X.X.X gets only 1mbit in any situation. Some rate limiting is definitely happening, but I am finding the outbound traffic is limited to 2mbit instead of 1mbit. If I change the rate (to say 10mbit) the outbound traffic gets up to again twice the rate (in this case 20mbit). Not sure can you show output of tc -s -d class ls dev eth0 while it's running at 2x speed. class htb 1:1 root rate 100Mbit ceil 100Mbit burst 24Kb/8 mpu 0b cburst 132644b/8 mpu 0b level 7 Sent 741267447736 bytes 735629628 pkts (dropped 0, overlimits 0) rate 4316188bps 22688pps lended: 489885589 borrowed: 0 giants: 0 tokens: -78 ctokens: 6677 class htb 1:10 parent 1:1 leaf 10: prio 0 quantum 20 rate 50Mbit ceil 80Mbit burst 19Kb/8 mpu 0b cburst 106440b/8 mpu 0b level 0 Sent 740240463634 bytes 734522966 pkts (dropped 429910165, overlimits 0) rate 4233957bps 22358pps lended: 244988819 borrowed: 489885528 giants: 0 tokens: -293 ctokens: 6636 class htb 1:20 parent 1:1 leaf 20: prio 0 quantum 13107 rate 1Mbit ceil 1Mbit burst 2Kb/8 mpu 0b cburst 2Kb/8 mpu 0b level 0 Sent 755354078 bytes 755048 pkts (dropped 0, overlimits 0) rate 84198bps 321pps backlog 4p lended: 755220 borrowed: 61 giants: 0 tokens: -21517 ctokens: -21517 -- Regards, Oliver Hookins ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
