Re: Routing latency
Devin Butterfield wrote: On Monday 19 March 2001 4:36, Will Andrews wrote: On Mon, Mar 19, 2001 at 07:46:53PM -0500, Dennis wrote: I never got an answer (as usual) from bill paul when I made the suggestions, and noone seemed interested in getting it fixed. He seems to get insulted when I infer that he did something wrong. It's like they say: "money talks". Similarly, "patches talk". Suggestions don't really do that. I'm not defending Dennis here, but this statement infers that nothing gets done unless maintainers are a) paid or b) someone else does the work for them. I certainly hope this is not the case. It is not the case, but it is the case when you want it done RIGHT NOW. You're not allowed to make DEMANDS of a volunteer, as Dennis has done a number of times in the past. There is a lot of history sweeping around this discussion -- feel free to search the archives if you are interested. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC [EMAIL PROTECTED] http://softweyr.com/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Routing latency
On Monday 19 March 2001 4:36, Will Andrews wrote: On Mon, Mar 19, 2001 at 07:46:53PM -0500, Dennis wrote: I never got an answer (as usual) from bill paul when I made the suggestions, and noone seemed interested in getting it fixed. He seems to get insulted when I infer that he did something wrong. It's like they say: "money talks". Similarly, "patches talk". Suggestions don't really do that. I'm not defending Dennis here, but this statement infers that nothing gets done unless maintainers are a) paid or b) someone else does the work for them. I certainly hope this is not the case. -- Regards, Devin. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Routing latency
At 02:43 AM 03/20/2001, you wrote: I'm using the de driver. Alas, the NICs seems quite old. They are 21140's. I've only got one 21143. I think there is a 3COM 3c905b in the lab too. Would it be better to use the 21143 + 3com than two 21140s? definitely : in my packet blaster, I get an order of magnitude less packet drops with a 3c905 than with a dc NIC (which is on a multi-port NIC : the PCI-PCI bridge may be a hindrance there) not my experience -- with the 21143 i can blast 140kpacket/s and receive them with no problems. For sure the "de" driver might have its own problems, but i think a lot of packet drops also depend on the card not being properly set for full duplex (which can cause collisions and lots of drops). You should initially test mono-directional in a controlled environment to avoid "collisions" to compare the true efficiency of the driver. dennis To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
RE: Routing latency
At 02:04 AM 03/20/2001, Mrten Wikstrm wrote: [snip] triggers every second and steals too much cpu. So my question is, how can I decrease this routing delay? Were you loading the interface, or just passing nominal streams? What pps did you pass through the box? Most likely the "delays" are only seen when the machine is close to capacity (the slow CPU you are using doesnt help). I sent 2 packets/s, three UDP streams with 60, 200 and 1000 bytes sized packets respectively. I also tried just one stream with 60 bytes packets and the same behaviour occured. 20k pps is probably beyond the capacity of a 200Mhz PPRO machine to forward on an ongoing basis (ie if other processes are running at the machine). the way the machine behaves over capacity is not as important as its abiltiy to continue running. How it works during normal operations its what is important. Dennis To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
RE: Routing latency
[snip] For sure the "de" driver might have its own problems, but i think a lot of packet drops also depend on the card not being properly set for full duplex (which can cause collisions and lots of drops). You should initially test mono-directional in a controlled environment to avoid "collisions" to compare the true efficiency of the driver. Yes, that is what I have tested. One card just receiving and one card just outputting. Thats why I thought it would be good to have two identical cards. But the question is, will there be a significant improvement by using 3c905 + 21143 instead of 21140 + 21140? /Mrten To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Routing latency
Hello, the FreeBSD TCP/IP stack uses the "system tick timer" for some delay (maybe only for TCP). you may want to use a HZ=1000 option (see the LINT config file) in a recompiled kernel and see if things go better. (moreover, the dc(4) driver which is used for your NIC has some interesting performance improvements in the forthcoming 4.3-Release) TfH Mrten Wikstrm wrote: I've performed a routing test between a FreeBSD box and a Linux box. I measured the latency and the result was not what I had expected. Both systems had the peak at 100 us (microseconds), but whereas the Linux box had _no_ packet over 200 us, the FreeBSD box delayed some packets up to 2 ms! Looking at the time series, it seems that the packets are delayed at regular intervals, about every second. My guess is that some timer interrupt triggers every second and steals too much cpu. So my question is, how can I decrease this routing delay? Test info: I used two identical boxes, each equipped with a Pentium Pro 200Mhz and 64Mb mem. RedHat 7.0 with 2.4 kernel in one and FreeBSD 4.2-RELEASE in the other. I used two DEC 100Mbit ethernet cards (21140 I think). I measured the latency with a SmartBits instrument. Fastforwarding was disabled. Three UDP streams was sent from the SmartBits to one of the ethernet cards in the box, which routed the streams to the other interface, which in turn was connected back to the SmartBits. I had not made any changes to the standard kernel configuration. No other processes was running in the background, apart from those necessary to perform the test. The ARP table was set statically, so no ARP traffic would disturb. I would at least want to know what is causing the extra delays. /Mrten To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message -- Thierry Herbelot To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Routing latency
(moreover, the dc(4) driver which is used for your NIC has some interesting performance improvements in the forthcoming 4.3-Release) like what ? cheers luigi To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Routing latency
At 02:32 PM 03/19/2001, Thierry Herbelot wrote: Hello, the FreeBSD TCP/IP stack uses the "system tick timer" for some delay (maybe only for TCP). you may want to use a HZ=1000 option (see the LINT config file) in a recompiled kernel and see if things go better. (moreover, the dc(4) driver which is used for your NIC has some interesting performance improvements in the forthcoming 4.3-Release) TfH Cool. Is the 21143 now started in store-and-forward mode and has the mandatory watchdog timeout been fixed? Im getting tired of hacking it every release. DB To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Routing latency
At 09:22 AM 03/19/2001, Mrten Wikstrm wrote: I've performed a routing test between a FreeBSD box and a Linux box. I measured the latency and the result was not what I had expected. Both systems had the peak at 100 us (microseconds), but whereas the Linux box had _no_ packet over 200 us, the FreeBSD box delayed some packets up to 2 ms! Looking at the time series, it seems that the packets are delayed at regular intervals, about every second. My guess is that some timer interrupt triggers every second and steals too much cpu. So my question is, how can I decrease this routing delay? Were you loading the interface, or just passing nominal streams? What pps did you pass through the box? Most likely the "delays" are only seen when the machine is close to capacity (the slow CPU you are using doesnt help). Latency under load and general latency are very different. Differing methods of handling backup conditions may have different goals; the proper goal is overall stability and NOT packet efficiency. It doesnt matter how fast a man runs if he doesnt finish the race. The problem with LINUX is that it works to a point and then chokes, while freebsd works up to higher thresholds. You cant evaluate a subsystem with one somewhat bogus test, without looking at the system as a whole. If you are using the dc driver, make certain it is operating in store-and-forward mode, the default configuration starts in a mode that only works on 10mb/s connections. dennis To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Routing latency
On Mon, Mar 19, 2001 at 07:14:54PM -0500, Dennis wrote: Cool. Is the 21143 now started in store-and-forward mode and has the mandatory watchdog timeout been fixed? Im getting tired of hacking it every release. Submit a PR to fix the problem? -- wca PGP signature
Re: Routing latency
At 07:20 PM 03/19/2001, Will Andrews wrote: On Mon, Mar 19, 2001 at 07:14:54PM -0500, Dennis wrote: Cool. Is the 21143 now started in store-and-forward mode and has the mandatory watchdog timeout been fixed? Im getting tired of hacking it every release. Submit a PR to fix the problem? I never got an answer (as usual) from bill paul when I made the suggestions, and noone seemed interested in getting it fixed. He seems to get insulted when I infer that he did something wrong. Dennis To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Routing latency
On Mon, Mar 19, 2001 at 07:46:53PM -0500, Dennis wrote: I never got an answer (as usual) from bill paul when I made the suggestions, and noone seemed interested in getting it fixed. He seems to get insulted when I infer that he did something wrong. It's like they say: "money talks". Similarly, "patches talk". Suggestions don't really do that. -- wca PGP signature
Re: Routing latency
On Mon, Mar 19, 2001 at 06:11:55PM -0800, Devin Butterfield wrote: I'm not defending Dennis here, but this statement infers that nothing gets done unless maintainers are a) paid or b) someone else does the work for them. I certainly hope this is not the case. No, it is not. My statement is an attack on his complaint that he has to "re hack a fix in every release", a problem he could solve by submitting his patches. It is pointless to complain about having to do something if you don't consider simple avenues to solve the problem. -- wca PGP signature
Re: Routing latency
Dennis wrote: [SNIP] If you are using the dc driver, make certain it is operating in store-and-forward mode, the default configuration starts in a mode that only works on 10mb/s connections. patches ? dennis -- Thierry Herbelot To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
RE: Routing latency
[snip] triggers every second and steals too much cpu. So my question is, how can I decrease this routing delay? Were you loading the interface, or just passing nominal streams? What pps did you pass through the box? Most likely the "delays" are only seen when the machine is close to capacity (the slow CPU you are using doesnt help). I sent 2 packets/s, three UDP streams with 60, 200 and 1000 bytes sized packets respectively. I also tried just one stream with 60 bytes packets and the same behaviour occured. Latency under load and general latency are very different. Differing methods of handling backup conditions may have different goals; the proper goal is overall stability and NOT packet efficiency. It doesnt matter how fast a man runs if he doesnt finish the race. The problem with LINUX is that it works to a point and then chokes, while freebsd works up to higher thresholds. You cant evaluate a subsystem with one somewhat bogus test, without looking at the system as a whole. Yep, that is exactly what my test showed when I tested the packet throughput capacity. Linux choked at 27000 pps and then the output rate _decreased_ with higher input rate, whereas the FreeBSD box started to drop packets at 19000 packets/s but the throughput did still increase up untill approximately 4 pps. (output rate). The input rate was then 7 pps. If you are using the dc driver, make certain it is operating in store-and-forward mode, the default configuration starts in a mode that only works on 10mb/s connections. I'm using the de driver. Alas, the NICs seems quite old. They are 21140's. I've only got one 21143. I think there is a 3COM 3c905b in the lab too. Would it be better to use the 21143 + 3com than two 21140s? /Mrten To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Routing latency
Mrten Wikstrm wrote: [SNIP] I'm using the de driver. Alas, the NICs seems quite old. They are 21140's. I've only got one 21143. I think there is a 3COM 3c905b in the lab too. Would it be better to use the 21143 + 3com than two 21140s? definitely : in my packet blaster, I get an order of magnitude less packet drops with a 3c905 than with a dc NIC (which is on a multi-port NIC : the PCI-PCI bridge may be a hindrance there) TfH /Mrten -- Thierry Herbelot To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Routing latency
I'm using the de driver. Alas, the NICs seems quite old. They are 21140's. I've only got one 21143. I think there is a 3COM 3c905b in the lab too. Would it be better to use the 21143 + 3com than two 21140s? definitely : in my packet blaster, I get an order of magnitude less packet drops with a 3c905 than with a dc NIC (which is on a multi-port NIC : the PCI-PCI bridge may be a hindrance there) not my experience -- with the 21143 i can blast 140kpacket/s and receive them with no problems. For sure the "de" driver might have its own problems, but i think a lot of packet drops also depend on the card not being properly set for full duplex (which can cause collisions and lots of drops). cheers luigi TfH /M_rten -- Thierry Herbelot To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
