RE: [LARTC] Linux router performance
Damjan wrote: On an AMD Athlon64 3200+ (2 GHz) I was able to saturate 2 PCI-Express gigabit cards (but that was with 1500 byte packets). Never tried more although the box has 6 interfaces capable of gigabit, 4 of them attached via PCI-Express. But that's _only_ 8 packets/s isn't it. Hm. How do you arrive at that result? I get twice the numbers. nic a: 1 gbit in - nic b: 1 gbit out nic b: 1 gbit in - nic a: 1 gbit out total 2 gbit 2 gbit /(1500*8 bit/frame) ~ 160k packets/s Please note that I did not test with smaller frame sizes, so 1Mp/s may be possible (I'll test that if I have some spare time). I've done some benchmarks on a Sunfire x2100 with 2 port PCI Express ethernet cards. It switches 800KPPS for 64B packets. Regards Mohan ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Linux router performance
Fermín Galán Márquez skrev: Hi, I wonder about the performance of a Linux box used as router (I guest I'm not the first :). Althought I know it mainly depends on the hardware, I'm trying to find some references on the topic or comparations with other routing solutions (FreeBSD box used as router, Cisco, etc). For example, http://facweb.cti.depaul.edu/jyu/Publications/Yu-Linux-TSM2004.pdf (althought is related with Linux-briding more than with Linux-routing) shows in Figure 14 that with an AMD Duron 1.3GHz 512M RAM a throughput of 90 Mbps can be achieved. Anybody knows any other similar analysis, please? Best regards, Fermín Galán Márquez CTTC - Centre Tecnològic de Telecomunicacions de Catalunya Parc Mediterrani de la Tecnologia, Av. del Canal Olímpic s/n, 08860 Castelldefels, Spain Room 1.02 Tel : +34 93 645 29 12 Fax : +34 93 645 29 01 Email address: [EMAIL PROTECTED] ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc This was seen on the mailing list a couple of years ago, doesnt say much but it shows what could be done. On Mon, 02 Dec 2002 22:30:10 +0100 Anton Tinchev [EMAIL PROTECTED] wrote: Hi, first i wonna thank you for the great work. I have few slack boxes with several 3com cards that acts as routers. Some of them has 50+ vlans, 100 000+ routing entries, full BGP (zebra) with 10+ peers and routes 50-70 mb/s traffic. Everithing is rock solid, few months uptimes. Sounds pretty impressive, really. I admire such setups. I wona to upgrade some of my cards and need advice what to use. On 100+mb/s interrups killing my boxes - 20 000+/s (yes, coalescing, i know:)) What to use? tigon2 or tigon3 for gigabit? (3c985 or 3c996) None of them! Or at least not tigon3! I've tried to use one (3c996-T), and I experienced strange system lockups. The board is a dual Tyan Tiger MP with couple of Athlon MP 1600+. It was just hanging from time to time with completely no output of any kind. Just rock solid lockup. :/ Anyway, I changed to a good old 3c905C and now I don't have any problems. Well, I'm serving at half of your rate, but anyway. So, I would suggest using HP equipment. At least I've heard that it works quote well. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Linux router performance (fwd)
I think that Robert Olssons post never made it through the filters... -- Forwarded message -- Date: Fri, 2 Jun 2006 12:32:53 +0200 From: Robert Olsson [EMAIL PROTECTED] To: Jesper Dangaard Brouer [EMAIL PROTECTED] Cc: Andreas John [EMAIL PROTECTED], Robert Olsson [EMAIL PROTECTED], lartc@mailman.ds9a.nl Subject: Re: [LARTC] Linux router performance Jesper Dangaard Brouer writes: Hi I'm sure that Robert can provide us with some interesting numbers. I have just tested routing performance on a AMD opteron 270 (dual core), here I can route 400 kpps (tg3 netcards on PCI-X). I use the kernel module pktgen to generate the packets (64 bytes in size). 400 kpps is decent but it all depends on your setup what you're testing. Single flow? Number of packets in environment with hi-number of flows. ( Forces lookup of dst cache, route lookup and garbage collection) is the most challenging Also how to handle filters eventually stateful information. For single flow tests most things end up in L2-cache and we most limited to latency. Bus latency, Memory latency etc. Large packets bus and memory bandwidth. We've seen 500 kpps in some of our production routers for BGP and about 500 filters. Dual Opteron 2.6 GHz. But you need to have a setup routing so it can make best use of your HW. Cheers. --ro ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
RE: [LARTC] Linux router performance
xOn an AMD Athlon64 3200+ (2 GHz) I was able to saturate 2 PCI-Express gigabit cards (but that was with 1500 byte packets). Never tried more although the box has 6 interfaces capable of gigabit, 4 of them attached via PCI-Express. What NIC's are you using? Are they multiport or do you have several pci-express single port cards? Andy Registered Office: J.O. Sims Ltd, Pudding Lane, Pinchbeck, Spalding, Lincs. PE11 3TJ Company reg No: 2084187 Vat reg No: GB 437 4621 47 Tel: +44 (0) 1775 842100 Fax: +44 (0) 1775 842101 Web: www.josims.com Email:[EMAIL PROTECTED] The information contained in this e-mail is confidential and is intended for the addressee only. The contents of this e-mail must not be disclosed or copied without the sender's consent. If you are not the intended recipient of the message, please notify the sender immediately, and delete the message. The statements and opinions expressed in this message are those of the author and do not necessarily reflect those of the company. No commitment may be inferred from the contents unless explicitly stated. The company does not take any responsibility for the personal views of the author. This message has been scanned for viruses before sending, but the company does not accept any responsibility for infection and recommends that you scan any attachments.JOSEDV001TAG ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Linux router performance
On Wed, 2006-05-31 at 22:27 +0200, Andreas John wrote: Hi, Maybe: Khan, Sohel; Waheed, Abdul (2003): High Performance Routing on PCshttp://www.ccse.kfupm.edu.sa/~sohel/networking/references/Routing.pdf A rule of thumb: - with current COTS hardware and (standard) PCI Bus, you can reach the maximum of the PCI bus bandwidth. That's 1 GB/s, e.h. two NICs with 500 Meg/s each ( one in and one out ) - with PCI-X and in the future PCI-express you'll for sure be able to reach more performance. I didnt find a sponsor for a test-lab yet :) - in DoS secnarios it may get worse :/ I heavily depends on driver type (polling and NAPI preferred). ofcouse prefered. Does it exsist a list of driver/nic combos that are know to support NAPI on linux on stock kernels ? -- Ronny Aasen [EMAIL PROTECTED] ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Linux router performance
Hi I'm sure that Robert can provide us with some interesting numbers. I have just tested routing performance on a AMD opteron 270 (dual core), here I can route 400 kpps (tg3 netcards on PCI-X). I use the kernel module pktgen to generate the packets (64 bytes in size). Cheers, Jesper Brouer -- --- MSc. Master of Computer Science Dept. of Computer Science, University of Copenhagen Author of http://www.adsl-optimizer.dk --- On Wed, 31 May 2006, Andreas John wrote: Hi, Maybe: Khan, Sohel; Waheed, Abdul (2003): High Performance Routing on PCshttp://www.ccse.kfupm.edu.sa/~sohel/networking/references/Routing.pdf A rule of thumb: - with current COTS hardware and (standard) PCI Bus, you can reach the maximum of the PCI bus bandwidth. That's 1 GB/s, e.h. two NICs with 500 Meg/s each ( one in and one out ) - with PCI-X and in the future PCI-express you'll for sure be able to reach more performance. I didnt find a sponsor for a test-lab yet :) - in DoS secnarios it may get worse :/ I heavily depends on driver type (polling and NAPI preferred). The problem with the performace is _always_ the number of interrupts, nothing else is a bottleneck (well, we didn't talk about thousands of iptables rules yet, but you ask for a 'maximum'). - The question you have to ask in high-performance scenarios is not MBit/s but MPPS (megapackets per seconds). FreeBSD and Linux broke the 1 MPPS barrier some time ago (on dual xeons). rgds, Andreas Fermín Galán Márquez wrote: Hi, I wonder about the performance of a Linux box used as router (I guest I'm not the first :). Althought I know it mainly depends on the hardware, I'm trying to find some references on the topic or comparations with other routing solutions (FreeBSD box used as router, Cisco, etc). For example, http://facweb.cti.depaul.edu/jyu/Publications/Yu-Linux-TSM2004.pdf (althought is related with Linux-briding more than with Linux-routing) shows in Figure 14 that with an AMD Duron 1.3GHz 512M RAM a throughput of 90 Mbps can be achieved. Anybody knows any other similar analysis, please? Best regards, Fermín Galán Márquez CTTC - Centre Tecnològic de Telecomunicacions de Catalunya Parc Mediterrani de la Tecnologia, Av. del Canal Olímpic s/n, 08860 Castelldefels, Spain Room 1.02 Tel : +34 93 645 29 12 Fax : +34 93 645 29 01 Email address: [EMAIL PROTECTED] ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Linux router performance
Hi, Maybe: Khan, Sohel; Waheed, Abdul (2003): High Performance Routing on PCshttp://www.ccse.kfupm.edu.sa/~sohel/networking/references/Routing.pdf A rule of thumb: - with current COTS hardware and (standard) PCI Bus, you can reach the maximum of the PCI bus bandwidth. That's 1 GB/s, e.h. two NICs with 500 Meg/s each ( one in and one out ) - with PCI-X and in the future PCI-express you'll for sure be able to reach more performance. I didnt find a sponsor for a test-lab yet :) - in DoS secnarios it may get worse :/ I heavily depends on driver type (polling and NAPI preferred). The problem with the performace is _always_ the number of interrupts, nothing else is a bottleneck (well, we didn't talk about thousands of iptables rules yet, but you ask for a 'maximum'). - The question you have to ask in high-performance scenarios is not MBit/s but MPPS (megapackets per seconds). FreeBSD and Linux broke the 1 MPPS barrier some time ago (on dual xeons). rgds, Andreas Fermín Galán Márquez wrote: Hi, I wonder about the performance of a Linux box used as router (I guest I'm not the first :). Althought I know it mainly depends on the hardware, I'm trying to find some references on the topic or comparations with other routing solutions (FreeBSD box used as router, Cisco, etc). For example, http://facweb.cti.depaul.edu/jyu/Publications/Yu-Linux-TSM2004.pdf (althought is related with Linux-briding more than with Linux-routing) shows in Figure 14 that with an AMD Duron 1.3GHz 512M RAM a throughput of 90 Mbps can be achieved. Anybody knows any other similar analysis, please? Best regards, Fermín Galán Márquez CTTC - Centre Tecnològic de Telecomunicacions de Catalunya Parc Mediterrani de la Tecnologia, Av. del Canal Olímpic s/n, 08860 Castelldefels, Spain Room 1.02 Tel : +34 93 645 29 12 Fax : +34 93 645 29 01 Email address: [EMAIL PROTECTED] ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Linux router performance
Fermín Galán Márquez wrote: Hi, I wonder about the performance of a Linux box used as router (I guest I'm not the first :). Althought I know it mainly depends on the hardware, I'm trying to find some references on the topic or comparations with other routing solutions (FreeBSD box used as router, Cisco, etc). For example, http://facweb.cti.depaul.edu/jyu/Publications/Yu-Linux-TSM2004.pdf (althought is related with Linux-briding more than with Linux-routing) shows in Figure 14 that with an AMD Duron 1.3GHz 512M RAM a throughput of 90 Mbps can be achieved. On an AMD Athlon64 3200+ (2 GHz) I was able to saturate 2 PCI-Express gigabit cards (but that was with 1500 byte packets). Never tried more although the box has 6 interfaces capable of gigabit, 4 of them attached via PCI-Express. Regards, Carl-Daniel -- http://www.hailfinger.org/ ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Linux router performance
I wonder about the performance of a Linux box used as router (I guest I'm not the first :). Althought I know it mainly depends on the hardware, I'm trying to find some references on the topic or comparations with other routing solutions (FreeBSD box used as router, Cisco, etc). For example, http://facweb.cti.depaul.edu/jyu/Publications/Yu-Linux-TSM2004.pdf (althought is related with Linux-briding more than with Linux-routing) shows in Figure 14 that with an AMD Duron 1.3GHz 512M RAM a throughput of 90 Mbps can be achieved. On an AMD Athlon64 3200+ (2 GHz) I was able to saturate 2 PCI-Express gigabit cards (but that was with 1500 byte packets). Never tried more although the box has 6 interfaces capable of gigabit, 4 of them attached via PCI-Express. But that's _only_ 8 packets/s isn't it. -- damjan | дамјан This is my jabber ID -- [EMAIL PROTECTED] -- not my mail address, it's a Jabber ID --^ :) ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Linux router performance
On Thu, Jun 01, 2006 at 02:44:57AM +0200, Carl-Daniel Hailfinger wrote: Damjan wrote: I wonder about the performance of a Linux box used as router (I guest I'm not the first :). Althought I know it mainly depends on the hardware, I'm trying to find some references on the topic or comparations with other routing solutions (FreeBSD box used as router, Cisco, etc). For example, http://facweb.cti.depaul.edu/jyu/Publications/Yu-Linux-TSM2004.pdf (althought is related with Linux-briding more than with Linux-routing) shows in Figure 14 that with an AMD Duron 1.3GHz 512M RAM a throughput of 90 Mbps can be achieved. On an AMD Athlon64 3200+ (2 GHz) I was able to saturate 2 PCI-Express gigabit cards (but that was with 1500 byte packets). Never tried more although the box has 6 interfaces capable of gigabit, 4 of them attached via PCI-Express. But that's _only_ 8 packets/s isn't it. Hm. How do you arrive at that result? I get twice the numbers. nic a: 1 gbit in - nic b: 1 gbit out nic b: 1 gbit in - nic a: 1 gbit out total 2 gbit 2 gbit /(1500*8 bit/frame) ~ 160k packets/s Please note that I did not test with smaller frame sizes, so 1Mp/s may be possible (I'll test that if I have some spare time). what if you test inbound and outbound at the same time - the cards should be capable of full duplex ? Regards, Carl-Daniel -- http://www.hailfinger.org/ ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc signature.asc Description: Digital signature ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Linux router performance
Alexander Samad wrote: On Thu, Jun 01, 2006 at 02:44:57AM +0200, Carl-Daniel Hailfinger wrote: Damjan wrote: I wonder about the performance of a Linux box used as router (I guest I'm not the first :). Althought I know it mainly depends on the hardware, I'm trying to find some references on the topic or comparations with other routing solutions (FreeBSD box used as router, Cisco, etc). For example, http://facweb.cti.depaul.edu/jyu/Publications/Yu-Linux-TSM2004.pdf (althought is related with Linux-briding more than with Linux-routing) shows in Figure 14 that with an AMD Duron 1.3GHz 512M RAM a throughput of 90 Mbps can be achieved. On an AMD Athlon64 3200+ (2 GHz) I was able to saturate 2 PCI-Express gigabit cards (but that was with 1500 byte packets). Never tried more although the box has 6 interfaces capable of gigabit, 4 of them attached via PCI-Express. But that's _only_ 8 packets/s isn't it. Hm. How do you arrive at that result? I get twice the numbers. nic a: 1 gbit in - nic b: 1 gbit out nic b: 1 gbit in - nic a: 1 gbit out total 2 gbit 2 gbit /(1500*8 bit/frame) ~ 160k packets/s Please note that I did not test with smaller frame sizes, so 1Mp/s may be possible (I'll test that if I have some spare time). what if you test inbound and outbound at the same time - the cards should be capable of full duplex ? I tested 1 gbit in and 1 gbit out per nic at the same time. That's how I arrived at my results. Regards, Carl-Daniel -- http://www.hailfinger.org/ ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Linux router performance
On Thu, Jun 01, 2006 at 04:03:29AM +0200, Carl-Daniel Hailfinger wrote: Alexander Samad wrote: On Thu, Jun 01, 2006 at 02:44:57AM +0200, Carl-Daniel Hailfinger wrote: Damjan wrote: I wonder about the performance of a Linux box used as router (I guest I'm not the first :). Althought I know it mainly depends on the hardware, I'm trying to find some references on the topic or comparations with other routing solutions (FreeBSD box used as router, Cisco, etc). For example, http://facweb.cti.depaul.edu/jyu/Publications/Yu-Linux-TSM2004.pdf (althought is related with Linux-briding more than with Linux-routing) shows in Figure 14 that with an AMD Duron 1.3GHz 512M RAM a throughput of 90 Mbps can be achieved. On an AMD Athlon64 3200+ (2 GHz) I was able to saturate 2 PCI-Express gigabit cards (but that was with 1500 byte packets). Never tried more although the box has 6 interfaces capable of gigabit, 4 of them attached via PCI-Express. But that's _only_ 8 packets/s isn't it. Hm. How do you arrive at that result? I get twice the numbers. nic a: 1 gbit in - nic b: 1 gbit out nic b: 1 gbit in - nic a: 1 gbit out total 2 gbit 2 gbit /(1500*8 bit/frame) ~ 160k packets/s Please note that I did not test with smaller frame sizes, so 1Mp/s may be possible (I'll test that if I have some spare time). what if you test inbound and outbound at the same time - the cards should be capable of full duplex ? I tested 1 gbit in and 1 gbit out per nic at the same time. That's how I arrived at my results. sorry I might be being very dense on this, but 2 nics 1G in and out shouldn't that be 4gbit / (1500*8 bit/frame) ~ 320k packets/s My presumption is that the nic can send and recieve at the same time Regards, Carl-Daniel -- http://www.hailfinger.org/ ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc signature.asc Description: Digital signature ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Linux router performance
Alexander Samad wrote: On Thu, Jun 01, 2006 at 04:03:29AM +0200, Carl-Daniel Hailfinger wrote: Alexander Samad wrote: On Thu, Jun 01, 2006 at 02:44:57AM +0200, Carl-Daniel Hailfinger wrote: Damjan wrote: I wonder about the performance of a Linux box used as router (I guest I'm not the first :). Althought I know it mainly depends on the hardware, I'm trying to find some references on the topic or comparations with other routing solutions (FreeBSD box used as router, Cisco, etc). For example, http://facweb.cti.depaul.edu/jyu/Publications/Yu-Linux-TSM2004.pdf (althought is related with Linux-briding more than with Linux-routing) shows in Figure 14 that with an AMD Duron 1.3GHz 512M RAM a throughput of 90 Mbps can be achieved. On an AMD Athlon64 3200+ (2 GHz) I was able to saturate 2 PCI-Express gigabit cards (but that was with 1500 byte packets). Never tried more although the box has 6 interfaces capable of gigabit, 4 of them attached via PCI-Express. But that's _only_ 8 packets/s isn't it. Hm. How do you arrive at that result? I get twice the numbers. nic a: 1 gbit in - nic b: 1 gbit out nic b: 1 gbit in - nic a: 1 gbit out total 2 gbit 2 gbit /(1500*8 bit/frame) ~ 160k packets/s Please note that I did not test with smaller frame sizes, so 1Mp/s may be possible (I'll test that if I have some spare time). what if you test inbound and outbound at the same time - the cards should be capable of full duplex ? I tested 1 gbit in and 1 gbit out per nic at the same time. That's how I arrived at my results. sorry I might be being very dense on this, but 2 nics 1G in and out shouldn't that be 4gbit / (1500*8 bit/frame) ~ 320k packets/s No, because you can count each packet passing through the router only once. If the machine works as a router, each entering packet also has to leave, so if the router has 2 interfaces A+B, you can have 1 Gbit from A to B and 1 Gbit from B to A. Your calculation would be correct if the machine is a server and generates and consumes all traffic locally. My presumption is that the nic can send and recieve at the same time Yes. Regards, Carl-Daniel -- http://www.hailfinger.org/ ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
