I did the same tests that I outlined earlier, but this time by setting hysteresis to 0. The config for the core router is included at the bottom. The graphs for the delay of the voip stream and the traffic going through the core router can be found at the following addresses.
http://140.203.56.30/~jlynch/htb/core_router_hysteresis.png http://140.203.56.30/~jlynch/htb/voip_stream_24761_hysteresis.png The max delay of the stream has dropped to 1.8ms. Again the jitter seems to be around 1ms. There seems to be a pattern going whereby the delay reaches about 1.6ms then drops back to 0.4 ms, jumps back to 1.6ms and then back to 0.4ms repeatedly and then it rises from 0.5ms gradually and repeats this behaviour. Is there any explanation to this pattern ? Would it have anything go to do with burst being 1ms ? When the ceil is specified as being 90mbit, is this at IP level ? What does this correspond to when a Mbit = 1,000,000 bits. Im a bit confused with the way tc interprets this rate. If the ceil is based at IP level then the max ceil is going to be a value between 54 Mbit and 97 Mbit (not the tc values) for a 100 Mbit interface depending on the size of the packets passing through, right ? Minimum Ethernet frame 148,809 * (46 * 8) = 148,809 * 368 = 54,761,712 Mbps Maximum Ethernet frame 8,127 * (1500 * 8) = 8,127 * 12,000 = 97,524,000 Mbps About the red settings, I dont understand properly how to configure the settings. I was using the configuration that came with the examples. Jonathan On Thu, 2005-07-28 at 22:49 +0100, Andy Furniss wrote: > Jonathan Lynch wrote: > > Andy, thanks again for your help. Yes, HZ is still 1000 in 2.6.12. I > > tried your suggestions are here are the results. > > > > ASCII diagram > > > > (network A) --> (eth1) core router (eth0) --> (network C) > > (eth2) > > ^ > > | > > | > > (network B) > > > > Looking at the following graphics > > > > http://140.203.56.30/~jlynch/htb/core_router.png > > http://140.203.56.30/~jlynch/htb/voip_stream_23691.png > > > > voip_stream_23691.png is a graph of the delay of the voice stream > > travelling from network A to network C in test 2. Notice from the core > > router graph that there is only voip traffic passing through the core > > router until time 07:55 and the delay in voip stream is 0.25 ms until > > then. After this time tcp traffic is introduced saturating the outgoing > > interface of the core router (eth0). The delay increases to a maximum of > > 2.75 ms , which is a considerable improvement on 30ms when I was using > > the ceil value of 100mbit. But there is a lot of jitter. > > I suppose you could hope for a bit less jitter 12k burst is about 1ms at > 100mbit. > > There is a tweak you can do for htb which may help - in > net/sched/sch_htb.c there is a #define HYSTERESIS 1 - changing it to 0 > and recompiling kernel/the module makes things more accurate. > > > > > > > With the ceil at 90Mbit, the outgoing bit rate of eth0 has gone from > > 98mbit to approx 90Mbit as can be seen from the core router graph for > > eth0 bytes out. Note that with the tcp traffic is all http downloads, so > > most Ethernet frames will be of maximum size, 1518 bytes, so 98mbits is > > the maximum throughput possible on a 100mbit card, taking into account > > the overheads of ethernet such as the interframe gap, preamble and start > > frame delimiter. > > > > Im not sure how to configure some of the htb parameters. The following > > is my understanding of them and a few questions I have as well. > > > > How exactly does the HZ value have a bearing on the ceil value ? How can > > I calculate a maximum for the ceil value ? > > It's more to do with burst/cburst than ceil. > > > > > 12kb is the minimum burst size for a 100 mbit NIC with a timer > > resolution of 1ms (1000hz) and tc calculates the smallest possible burst > > when it is not specified, right ?. > > It seems not, I think hysteresis may be involved again here (but then > one of my tcs is hacked about a bit). > > You can see what htb is using as defaults by doing tc -s -d class ls .. > > If I do that on similar kernels one with hysteresis 0 and one with 1 I > see quite different values. > > I chose 12k as big enough for the 90mbit test 12000*8*1000=96mbit at ip > level and it seemed like a nice multiple of 1500mtu :-) > > > > > > cburst is the number of bytes that can be burst as fast as the interface > > can transmit them. It is smaller than burst can is ideally one packet > > size, right ? > > Ideally 1 packet but not achievable with htb at lan speed and hz 1000, > also AIUI the way htb does drr means with mixed packet sizes things > aren't packet perfect even at low rates. > > Saying that I use htb at low rates and can apparently get packet perfect > with my traffic mix. > > I think hfsc can do it perfectly on both counts. > > > > > quantum determines the ratio at which the classes share their parents > > bandwidth. Each class is given quantum number of bytes before serving > > the next class, right ? > > Yea setting 1500 probably makes no difference for this test. > > > > > Is there any way I can limit the jitter of the VoIP traffic passing > > through the htb class ? > > Try the hysteresis and/or setting the rate for interactive way higher > than it's traffic rate. > > I did a quick test to see how things were for me at 100mbit. Because my > other pcs are slow I needed to use two as receivers for netperf. > > I noticed something I didn't expect with red or the settings you use - > one of the pcs is slower and has less memory thus smaller tcp socket > size. Using 4 streams two to each unshaped they get about the same, > though with txqueuelen = 1000 there are no drops (with window scalng off > there is a difference). With red and wscale on, the red really favoured > the fast pc - I tried a 40k bfifo so that I got drops, expecting to see > the same, but it was still far more even than the red. > > I couldn't really simulate the voip traffic in theory I should be able > to use ping with -i < 1 sec, but using the latest inetutils you get a > flood ping if you do that. I reported this about 18 months ago and it's > supposedly fixed in the cvs (though I don't know if fixed means it just > says invalid argument rather than actually does what's asked, because I > have failed to build it so far). > > So if anyone reading this has a i386 ping that -i 0.5 works on, please > mail me the binary :-) > > Andy. Main dsmark & classifier tc qdisc add dev $1 handle 1:0 root dsmark indices 64 set_tc_index tc filter add dev $1 parent 1:0 protocol ip prio 1 tcindex mask 0xfc shift 2 Main htb qdisc & class tc qdisc add dev $1 parent 1:0 handle 2:0 htb tc class add dev $1 parent 2:0 classid 2:1 htb rate 90Mbit ceil 90Mbit burst 12k cburst 12k EF Class (2:10) tc class add dev $1 parent 2:1 classid 2:10 htb rate 5Mbit ceil 90Mbit burst 12k cburst 12k prio 1 tc qdisc add dev $1 parent 2:10 pfifo limit 5 tc filter add dev $1 parent 2:0 protocol ip prio 1 handle 0x2e tcindex classid 2:10 pass_on BE Class (2:20) tc class add dev $1 parent 2:1 classid 2:20 htb rate 10Mbit ceil 90Mbit burst 12k cburst 12k prio 2 tc qdisc add dev $1 parent 2:20 red limit 60KB min 15KB max 45KB burst 20 avpkt 1000 bandwidth 100Mbit probability 0.4 tc filter add dev $1 parent 2:0 protocol ip prio 2 handle 0 tcindex mask 0 classid 2:20 pass_on _______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
