Re: [LARTC] HTB speed
Simon Byrnand wrote: At 12:46 26/03/2004, Simon Byrnand wrote: At 18:17 25/03/2004, Andrew Hall wrote: You need to recompile the kernel after altering this value in linux/include/net/pkt_sched.h. Also remember that if using SFQ on leaf qdisc, then the queue length may cause delay problems if it's too long (default is 128). Changing this to 32 for rates below 100kb/s, I have found to help things considerably. This change needs to be done in linux/net/sched/sch_sfq.c. which also needs a kernel recompilation. Hmm, When I use sfq with cbq at speeds like 256Kbit there is no problem at all, runs very sweetly, but with HTB and sfq, it is very jerky and poor. I'll try the change in pkt_sched.h first and see how I go... Ok, I tried the change in pkt_sched.h and didn't notice any difference whatsoever. Any other ideas ? cbq is still fine but htb for the same speed is very jerky and the speed fluctuates around 60-80% of the wanted speed, while cbq gives a steady 99% of the wanted speed... You can make HTB more accurate by setting HTB_HYSTERESIS to 0 in net/sched/sch_htb.c. To save time - if you built HTB as a module, you can probably (well it worked for me) get away with editing htb.c and do make SUBDIRS=net/sched modules and replacing /lib/modules/[kversion]/kernel/net/sched/htb.o with the new htb.o from your source tree. If you are doing it live stop shaping and check with lsmod that modprobe -r gets rid (do it again if it's still there) of the old htb.o and reload shaping scripts. Make sure the quantum is your mtu, set 0 burst for bulk classes and don't set perturb too low on sfq (I use 20 as it causes packet reordering). Are you shaping egress or ingress and how are you measuring speed? Andy. ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] HTB speed
Strange, must be somthing else going on. I had the same 40 to 50 percent too slow, which was completely fixed bij using PSCHED_CPU in pkt_sched.h. Using kernel 2.4.24, I measure the speed with the iptables byte counters. On Fri, 26 Mar 2004 16:27:42 +1200 Simon Byrnand [EMAIL PROTECTED] wrote: At 12:46 26/03/2004, Simon Byrnand wrote: At 18:17 25/03/2004, Andrew Hall wrote: You need to recompile the kernel after altering this value in linux/include/net/pkt_sched.h. Also remember that if using SFQ on leaf qdisc, then the queue length may cause delay problems if it's too long (default is 128). Changing this to 32 for rates below 100kb/s, I have found to help things considerably. This change needs to be done in linux/net/sched/sch_sfq.c. which also needs a kernel recompilation. Hmm, When I use sfq with cbq at speeds like 256Kbit there is no problem at all, runs very sweetly, but with HTB and sfq, it is very jerky and poor. I'll try the change in pkt_sched.h first and see how I go... Ok, I tried the change in pkt_sched.h and didn't notice any difference whatsoever. Any other ideas ? cbq is still fine but htb for the same speed is very jerky and the speed fluctuates around 60-80% of the wanted speed, while cbq gives a steady 99% of the wanted speed... Regards, Simon ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] HTB speed
Would the problem with queue lengths also exist if CONFIG_HZ=1000 in .config? (it certainly improves latency) I've configured rates from 10kbit/s...2Mbit/s on one machine, so changing the kernel code for low rates would probably effect the high rates too. Or, in other words, what is exactly the problem with an SFQ length of 128 for low rates? Cheers, Jeroen. On Thu, 25 Mar 2004 17:17:44 +1100 Andrew Hall [EMAIL PROTECTED] wrote: You need to recompile the kernel after altering this value in linux/include/net/pkt_sched.h. Also remember that if using SFQ on leaf qdisc, then the queue length may cause delay problems if it's too long (default is 128). Changing this to 32 for rates below 100kb/s, I have found to help things considerably. This change needs to be done in linux/net/sched/sch_sfq.c. which also needs a kernel recompilation. - Original Message - From: Simon Byrnand [EMAIL PROTECTED] To: Jeroen Vriesman [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Thursday, March 25, 2004 4:36 PM Subject: Re: [LARTC] HTB speed At 11:14 14/03/2004, Jeroen Vriesman wrote: Hi, just putting the answer to my own question here, for those who have the same problem, and read the mailing list archive. The timing of the P4 based on jiffies is hopeless, it's different for every processor, and can be a wrong by a factor 3. If the tsc (time stamp counter) is used, the htb works fine, the error in speed is now only about 1 %. It's set by: in pkt_sched.h: #define PSCHED_CLOCK_SOURCE PSCHED_CPU that's all, I wonder why it's not default to do this, or maybe it's an idea to make the packet scheduler detect the presence of tsc when the module is loaded. Hi, Which pkt_sched.h are you refering to ? /usr/src/linux/include/linux/pkt_sched.h or /usr/src/linux/include/net/pkt_sched.h ? And after changing it what did you do ? Recompile the kernel ? Or recompile tc ? I too see the same problems with htb (very poor accuracy of speed, significantly too slow, also very jerky) while cbq is very accurate and smooth. (But lacks some features I need) Regards, Simon ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
RE: [LARTC] HTB speed
Thanks, Andrew, it's intresting...
- (default is 128). Changing this to 32 for rates below 100kb/s, I
- have found
you mean that try change linux/net/sched/sch_sfq.c this: ?
#define SFQ_DEPTH 128
#define SFQ_HASH_DIVISOR1024
for this.
#define SFQ_DEPTH 32
#define SFQ_HASH_DIVISOR1024
I 've 2 questions about it
1) what is I use various rates (from 8k to 256k)
2) how I must proceed in Esfq ??
(in order to work with those htb classes)
Usage: ... esfq [ perturb SECS ] [ quantum BYTES ] [ depth FLOWS ]
[ divisor HASHBITS ] [ limit PKTS ] [ hash HASHTYPE]
Where:
HASHTYPE := { classic | src | dst }
Thankyou!
andres
-
- You need to recompile the kernel after altering this value in
- linux/include/net/pkt_sched.h. Also remember that if using SFQ on leaf
- qdisc, then the queue length may cause delay problems if it's too long
- (default is 128). Changing this to 32 for rates below 100kb/s, I
- have found
- to help things considerably. This change needs to be done in
- linux/net/sched/sch_sfq.c. which also needs a kernel recompilation.
-
- - Original Message -
- From: Simon Byrnand [EMAIL PROTECTED]
- To: Jeroen Vriesman [EMAIL PROTECTED]; [EMAIL PROTECTED]
- Sent: Thursday, March 25, 2004 4:36 PM
- Subject: Re: [LARTC] HTB speed
-
-
- At 11:14 14/03/2004, Jeroen Vriesman wrote:
- Hi,
-
- just putting the answer to my own question here, for those
- who have the
- same problem, and read the mailing list archive.
-
- The timing of the P4 based on jiffies is hopeless, it's
- different for
- every processor, and can be a wrong by a factor 3.
-
- If the tsc (time stamp counter) is used, the htb works fine,
- the error in
- speed is now only about 1 %.
-
- It's set by:
-
- in pkt_sched.h:
-
- #define PSCHED_CLOCK_SOURCE PSCHED_CPU
-
- that's all, I wonder why it's not default to do this, or maybe it's an
- idea to make the packet scheduler detect the presence of tsc when the
- module is loaded.
-
- Hi,
-
- Which pkt_sched.h are you refering to ?
- /usr/src/linux/include/linux/pkt_sched.h or
- /usr/src/linux/include/net/pkt_sched.h ?
-
- And after changing it what did you do ? Recompile the kernel ?
-
- Or recompile tc ?
-
- I too see the same problems with htb (very poor accuracy of speed,
- significantly too slow, also very jerky) while cbq is very accurate and
- smooth. (But lacks some features I need)
-
- Regards,
- Simon
-
- ___
- LARTC mailing list / [EMAIL PROTECTED]
- http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
-
- ___
- LARTC mailing list / [EMAIL PROTECTED]
- http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
-
___
LARTC mailing list / [EMAIL PROTECTED]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] HTB speed
At 11:14 14/03/2004, Jeroen Vriesman wrote: Hi, just putting the answer to my own question here, for those who have the same problem, and read the mailing list archive. The timing of the P4 based on jiffies is hopeless, it's different for every processor, and can be a wrong by a factor 3. If the tsc (time stamp counter) is used, the htb works fine, the error in speed is now only about 1 %. It's set by: in pkt_sched.h: #define PSCHED_CLOCK_SOURCE PSCHED_CPU that's all, I wonder why it's not default to do this, or maybe it's an idea to make the packet scheduler detect the presence of tsc when the module is loaded. Hi, Which pkt_sched.h are you refering to ? /usr/src/linux/include/linux/pkt_sched.h or /usr/src/linux/include/net/pkt_sched.h ? And after changing it what did you do ? Recompile the kernel ? Or recompile tc ? I too see the same problems with htb (very poor accuracy of speed, significantly too slow, also very jerky) while cbq is very accurate and smooth. (But lacks some features I need) Regards, Simon ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] HTB speed
You need to recompile the kernel after altering this value in linux/include/net/pkt_sched.h. Also remember that if using SFQ on leaf qdisc, then the queue length may cause delay problems if it's too long (default is 128). Changing this to 32 for rates below 100kb/s, I have found to help things considerably. This change needs to be done in linux/net/sched/sch_sfq.c. which also needs a kernel recompilation. - Original Message - From: Simon Byrnand [EMAIL PROTECTED] To: Jeroen Vriesman [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Thursday, March 25, 2004 4:36 PM Subject: Re: [LARTC] HTB speed At 11:14 14/03/2004, Jeroen Vriesman wrote: Hi, just putting the answer to my own question here, for those who have the same problem, and read the mailing list archive. The timing of the P4 based on jiffies is hopeless, it's different for every processor, and can be a wrong by a factor 3. If the tsc (time stamp counter) is used, the htb works fine, the error in speed is now only about 1 %. It's set by: in pkt_sched.h: #define PSCHED_CLOCK_SOURCE PSCHED_CPU that's all, I wonder why it's not default to do this, or maybe it's an idea to make the packet scheduler detect the presence of tsc when the module is loaded. Hi, Which pkt_sched.h are you refering to ? /usr/src/linux/include/linux/pkt_sched.h or /usr/src/linux/include/net/pkt_sched.h ? And after changing it what did you do ? Recompile the kernel ? Or recompile tc ? I too see the same problems with htb (very poor accuracy of speed, significantly too slow, also very jerky) while cbq is very accurate and smooth. (But lacks some features I need) Regards, Simon ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] HTB speed
in pkt_sched.h: #define PSCHED_CLOCK_SOURCE PSCHED_CPU that's all, I wonder why it's not default to do this, or maybe it's an idea to make the packet scheduler detect the presence of tsc when the module is loaded. Hi, I think not all processors accept this #define PSCHED_CLOCK_SOURCE PSCHED_CPU. At least, I couldn't do it in my router. I am not using HTB but TBF, but I think the timing problems are similar. As to tune TBF with higher precision, I had to manually change Hz value from 100 to 1000, in param.h, and actually I am getting better results. My question is, does anybody tried to do this in an production environment? For instance, in a router that integrates traffic control, NAT, firewall, services gateway, etc., can this change in HZ value harm in any way the rest of the system? Thanks in advance, Joana Urbano
Re: [LARTC] HTB speed
Hi, Indeed, changing the timer to 1000Hz is possible, it turned out that I have a machine here running with a 1000Hz timer ticker. (I've installed a realtime kernel on it for audio recording). About your previous question, I've noticed that the system with 1000Hz ticker (which has been running for a few months) is UNSTABLE. might be more to it than just the 1000Hz, but I did notice that some realtime applications running in userspace where able to crash the system completely, happened 2 times in 3 months. Cheers, Jeroen. On Mon, 15 Mar 2004 11:37:16 + Maria Joana Urbano [EMAIL PROTECTED] wrote: in pkt_sched.h: #define PSCHED_CLOCK_SOURCE PSCHED_CPU that's all, I wonder why it's not default to do this, or maybe it's an idea to make the packet scheduler detect the presence of tsc when the module is loaded. Hi, I think not all processors accept this #define PSCHED_CLOCK_SOURCE PSCHED_CPU. At least, I couldn't do it in my router. I am not using HTB but TBF, but I think the timing problems are similar. As to tune TBF with higher precision, I had to manually change Hz value from 100 to 1000, in param.h, and actually I am getting better results. My question is, does anybody tried to do this in an production environment? For instance, in a router that integrates traffic control, NAT, firewall, services gateway, etc., can this change in HZ value harm in any way the rest of the system? Thanks in advance, Joana Urbano ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: {Spam?} Re: [LARTC] HTB speed
Indeed, changing the timer to 1000Hz is possible, it turned out that I have a machine here running with a 1000Hz timer ticker. (I've installed a realtime kernel on it for audio recording). About your previous question, I've noticed that the system with 1000Hz ticker (which has been running for a few months) is UNSTABLE. might be more to it than just the 1000Hz, but I did notice that some realtime applications running in userspace where able to crash the system completely, happened 2 times in 3 months. Hi Jeroen and all, Does anybody else related the same problems of unstabilitty changing the timer to 1000Hz? Thanks in advance, Joana
Re: {Spam?} Re: [LARTC] HTB speed
Hi Jeroen, Thanx for your fast answer. to use the TSC, the processor has to have a tsc, you can see that in /proc/cpuinfo, for as far as I know, every P4 has it (but I'm not sure), it's in the flags of cpuinfo: On my P4-1800: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm The loop calibrated jiffies stuff dates back to 386, it's used everywhere in the kernel timing, and it's very inaccurate, I don't know if 2.6 still uses it everwhere, what kind of processor do you have? I am using and old test machine with an AMD-K7 processor. I think that's the reason I cannot use TSC, it only works in Pentium processors. I've never tried to change the Hz value, does changing it in param.h really changes the frequency of the clock ticks? If so, why is the default only 100Hz these days? doesn't make sense to me. I have no ideia why it is still at 100Hz. However, I took this clue from [1]. I do use the PSCHED_CPU in a production environement at the moment, works fine. OK, that's good news :-) Regards, Joana [1] K. Wagner, Short Evaluation of Linux's Token-Bucket-Filter (TBF) Queuing Discipline
Re: [LARTC] HTB speed
Hi, to use the TSC, the processor has to have a tsc, you can see that in /proc/cpuinfo, for as far as I know, every P4 has it (but I'm not sure), it's in the flags of cpuinfo: On my P4-1800: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm The loop calibrated jiffies stuff dates back to 386, it's used everywhere in the kernel timing, and it's very inaccurate, I don't know if 2.6 still uses it everwhere, what kind of processor do you have? I've never tried to change the Hz value, does changing it in param.h really changes the frequency of the clock ticks? If so, why is the default only 100Hz these days? doesn't make sense to me. I do use the PSCHED_CPU in a production environement at the moment, works fine. Cheers, Jeroen. On Mon, 15 Mar 2004 11:37:16 + Maria Joana Urbano [EMAIL PROTECTED] wrote: in pkt_sched.h: #define PSCHED_CLOCK_SOURCE PSCHED_CPU that's all, I wonder why it's not default to do this, or maybe it's an idea to make the packet scheduler detect the presence of tsc when the module is loaded. Hi, I think not all processors accept this #define PSCHED_CLOCK_SOURCE PSCHED_CPU. At least, I couldn't do it in my router. I am not using HTB but TBF, but I think the timing problems are similar. As to tune TBF with higher precision, I had to manually change Hz value from 100 to 1000, in param.h, and actually I am getting better results. My question is, does anybody tried to do this in an production environment? For instance, in a router that integrates traffic control, NAT, firewall, services gateway, etc., can this change in HZ value harm in any way the rest of the system? Thanks in advance, Joana Urbano ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: {Spam?} Re: [LARTC] HTB speed
List, I just logged in to a machine with a modern AMD cpu, it also has a TSC. [EMAIL PROTECTED]:~ cat /proc/cpuinfo processor : 0 vendor_id : AuthenticAMD cpu family : 6 model : 6 model name : AMD Athlon(TM) XP 2000+ stepping: 2 cpu MHz : 1668.736 cache size : 256 KB fdiv_bug: no hlt_bug : no f00f_bug: no coma_bug: no fpu : yes fpu_exception : yes cpuid level : 1 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr sse syscall mmxext 3dnowext 3dnow bogomips: 3329.22/^\ | So, I think you have to pull some managers' tie and demand new hardware :) Jeroen. On Mon, 15 Mar 2004 12:11:05 + Maria Joana Urbano [EMAIL PROTECTED] wrote: Hi Jeroen, Thanx for your fast answer. to use the TSC, the processor has to have a tsc, you can see that in /proc/cpuinfo, for as far as I know, every P4 has it (but I'm not sure), it's in the flags of cpuinfo: On my P4-1800: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm The loop calibrated jiffies stuff dates back to 386, it's used everywhere in the kernel timing, and it's very inaccurate, I don't know if 2.6 still uses it everwhere, what kind of processor do you have? I am using and old test machine with an AMD-K7 processor. I think that's the reason I cannot use TSC, it only works in Pentium processors. I've never tried to change the Hz value, does changing it in param.h really changes the frequency of the clock ticks? If so, why is the default only 100Hz these days? doesn't make sense to me. I have no ideia why it is still at 100Hz. However, I took this clue from [1]. I do use the PSCHED_CPU in a production environement at the moment, works fine. OK, that's good news :-) Regards, Joana [1] K. Wagner, Short Evaluation of Linux's Token-Bucket-Filter (TBF) Queuing Discipline ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] HTB speed
Hi, just putting the answer to my own question here, for those who have the same problem, and read the mailing list archive. The timing of the P4 based on jiffies is hopeless, it's different for every processor, and can be a wrong by a factor 3. If the tsc (time stamp counter) is used, the htb works fine, the error in speed is now only about 1 %. It's set by: in pkt_sched.h: #define PSCHED_CLOCK_SOURCE PSCHED_CPU that's all, I wonder why it's not default to do this, or maybe it's an idea to make the packet scheduler detect the presence of tsc when the module is loaded. Cheers, Jeroen. On Fri, 12 Mar 2004 10:58:40 +0100 Jeroen Vriesman [EMAIL PROTECTED] wrote: Dear all, I'm running htb on 2.4.24, following setup: running on a bridge, ebtables patch, firewall marks with iptables, and an SFQ for every HTB. Everything works fine, so I don't go into details, there is just one thing which suprises me: When I configure HTB to shape to about 900 kbit/s, the resulting speed is about 400 kbit/s. For higher speeds it's more accurate (but stil too slow), for lower speeds it gets worse. Does anyone here have any idea why? I've just added a correction factor to my scripts, so it's kind of allright, but I would like to know if others experience the same thing. Jeroen. ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
