Re: Preferred Gigabit interfaces for -CURRENT
On Saturday 08 February 2003 17:22, Terry Lambert wrote: > Wes Peters wrote: > > On Friday 07 February 2003 01:25, David Gilbert wrote: > > > I believe that someone here recomended Tigon III based cards ... but I > > > was recently looking through 5.0-RELEASE's hardware notes and couldn't > > > find any mention of Tigon III. > > > > The follow-on to the Tigon II is the Broadcom BCM570x supported by > > the bge(4) driver in FreeBSD. This is not what you want. They're > > certainly cheap to test with, though; the Netgear GA302T sells for > > under $40 at a few online retailers. > > I personally really like the Tigon III. It doesn't have the > alignment issues that some of the cards do, so you get to avoid > the m_pullup() (and the copy that happenes with it, in tcp_input()), > since it can scatter/gather to an unaligned address. > > It's also the first card I'm aware of that does the full range of > checksum offloading, without slowing the card down, which (finally!) > lets you offload some of the network processing to the card (i.e. > it does IP, TCP, and UDP). > > The card itself does interrupt coelescing in hardware, and you can > adjust both the trigger and buffer thresholds from the driver. > > Using 64bit 66MHz slots, it's possible to keep two interfaces > completely loaded, while retaining sufficient CPU and bus > bandwidth to actually do other work (though, in general, you will > want to tune your stack, and replace the mbuf allocator). > > About the only complaint I really have about it is that, unlike > the Tigon II, now that Broadcomm got their grubby little hands on > it, unlike Alteon, they are refusing to make the firmware sources > available so people can do useful work in the context of the > firmware. Yeah, a prototype Xylan GigE switch blade was done with Tigon-II's and we did a bit of hacking in the firmware. They were pretty cool; we used the usual Xylan SPARC processors on the card only for bus and chassis management and did most of the cool packet stuff in the Tigon itself. > Actually, there are some really brilliant things you can do, if > you can replace the firmware, that can take you up to theoretical > max packets a second very easily and quickly. We were able to get > in the neighborhood of 31,000 connections per second with the Tigon > III, alll other things being equal, even before FreeBSD added the > SYN cache and SYN cookie code. > > Is there a particular reason you don't like the card, or at least > prefer the other card more? Our testing, which mostly comprised throwing a pair of cards into a system, turning on bridging and blasting it with a SmartBits, showed the Intels to be faster with less CPU load. The Intel cards were 2x the price, but still well within our rather permissive budget. When you're putting 4, 8, or even 16 GB DDR RAM into a box the cost of a pair of network cards isn't significant. ;^) If I were buying a card myself, I'd likely go with the NetGear because it's cheap and it works, but you well know I'm a cheapskate. Not having access to the doco suxxors. -- Where am I, and what am I doing in this handbasket? Wes Peters [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Preferred Gigabit interfaces for -CURRENT
Wes Peters wrote: > On Friday 07 February 2003 01:25, David Gilbert wrote: > > I believe that someone here recomended Tigon III based cards ... but I > > was recently looking through 5.0-RELEASE's hardware notes and couldn't > > find any mention of Tigon III. > > The follow-on to the Tigon II is the Broadcom BCM570x supported by > the bge(4) driver in FreeBSD. This is not what you want. They're > certainly cheap to test with, though; the Netgear GA302T sells for > under $40 at a few online retailers. I personally really like the Tigon III. It doesn't have the alignment issues that some of the cards do, so you get to avoid the m_pullup() (and the copy that happenes with it, in tcp_input()), since it can scatter/gather to an unaligned address. It's also the first card I'm aware of that does the full range of checksum offloading, without slowing the card down, which (finally!) lets you offload some of the network processing to the card (i.e. it does IP, TCP, and UDP). The card itself does interrupt coelescing in hardware, and you can adjust both the trigger and buffer thresholds from the driver. Using 64bit 66MHz slots, it's possible to keep two interfaces completely loaded, while retaining sufficient CPU and bus bandwidth to actually do other work (though, in general, you will want to tune your stack, and replace the mbuf allocator). About the only complaint I really have about it is that, unlike the Tigon II, now that Broadcomm got their grubby little hands on it, unlike Alteon, they are refusing to make the firmware sources available so people can do useful work in the context of the firmware. Actually, there are some really brilliant things you can do, if you can replace the firmware, that can take you up to theoretical max packets a second very easily and quickly. We were able to get in the neighborhood of 31,000 connections per second with the Tigon III, alll other things being equal, even before FreeBSD added the SYN cache and SYN cookie code. Is there a particular reason you don't like the card, or at least prefer the other card more? -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Preferred Gigabit interfaces for -CURRENT
Likely part of the performance issue was due to the Chipset of the motherboard. Your typical 32bit 33MHz PCI bus is going to be marginal for routing GigE traffic, just due to bus bandwidth limitations, but it'll handle multiple 100BaseTX cards just fine. While a higher-end setup like a Serverworks chipset, with a 64bit, 66MHz bus will handle the traffic better. OF course, if you really want good routing performance on a*BSD platform, you should be looking at a Juniper M20 or M40. Adam - Original Message - From: "Wes Peters" <[EMAIL PROTECTED]> To: "David Gilbert" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]> Sent: Friday, February 07, 2003 7:49 PM Subject: Re: Preferred Gigabit interfaces for -CURRENT > On Friday 07 February 2003 01:25, David Gilbert wrote: > > We're about to make the switch from 100M interfaces to GigE interfaces > > for our transit routers ... which are FreeBSD-5.0 based SMP (Athlon) > > boxes. Our current favorite card is the intel i82559-based fxp > > cards. They handle the load best on our testing of 100M cards. > > Remember that our load is large and small packets and that hardware > > checksums are not a win (although hardware vlans are). > > > > So... I need to know what GigE chipsets I should test. I recently > > tested Intel GigE cards ... with dismal results... less than half the > > packets-per-second on the (otherwise) same hardware. Small packets > > (as in DOS attacks) are a real concern here. > > Wow, this wasn't my experience at all. At my previous employer we > used Intel EEPro 1000 Server cards with the em(4) driver on FreeBSD > 4.5 with nary a hitch and excellent performance. This was on > ServerWorks chipset motherboards with P-III and P4 processors. > > > I believe that someone here recomended Tigon III based cards ... but I > > was recently looking through 5.0-RELEASE's hardware notes and couldn't > > find any mention of Tigon III. > > The follow-on to the Tigon II is the Broadcom BCM570x supported by > the bge(4) driver in FreeBSD. This is not what you want. They're > certainly cheap to test with, though; the Netgear GA302T sells for > under $40 at a few online retailers. > > -- > > Where am I, and what am I doing in this handbasket? > > Wes Peters [EMAIL PROTECTED] > > > To Unsubscribe: send mail to [EMAIL PROTECTED] > with "unsubscribe freebsd-current" in the body of the message > To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Preferred Gigabit interfaces for -CURRENT
On Friday 07 February 2003 01:25, David Gilbert wrote: > We're about to make the switch from 100M interfaces to GigE interfaces > for our transit routers ... which are FreeBSD-5.0 based SMP (Athlon) > boxes. Our current favorite card is the intel i82559-based fxp > cards. They handle the load best on our testing of 100M cards. > Remember that our load is large and small packets and that hardware > checksums are not a win (although hardware vlans are). > > So... I need to know what GigE chipsets I should test. I recently > tested Intel GigE cards ... with dismal results... less than half the > packets-per-second on the (otherwise) same hardware. Small packets > (as in DOS attacks) are a real concern here. Wow, this wasn't my experience at all. At my previous employer we used Intel EEPro 1000 Server cards with the em(4) driver on FreeBSD 4.5 with nary a hitch and excellent performance. This was on ServerWorks chipset motherboards with P-III and P4 processors. > I believe that someone here recomended Tigon III based cards ... but I > was recently looking through 5.0-RELEASE's hardware notes and couldn't > find any mention of Tigon III. The follow-on to the Tigon II is the Broadcom BCM570x supported by the bge(4) driver in FreeBSD. This is not what you want. They're certainly cheap to test with, though; the Netgear GA302T sells for under $40 at a few online retailers. -- Where am I, and what am I doing in this handbasket? Wes Peters [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Preferred Gigabit interfaces for -CURRENT
David Gilbert <[EMAIL PROTECTED]> wrote: > We're about to make the switch from 100M interfaces to GigE interfaces > for our transit routers ... which are FreeBSD-5.0 based SMP (Athlon) > boxes. Our current favorite card is the intel i82559-based fxp > cards. They handle the load best on our testing of 100M cards. > Remember that our load is large and small packets and that hardware > checksums are not a win (although hardware vlans are). As far as I know, all of the GigE cards do hardware checksums anyway. Those which support VLAN de/muxing in hardware are listed in the vlan(4) manpage and in the respective interface manpages (bge, em, gx etc.) -- I think most of them do, if not all. > So... I need to know what GigE chipsets I should test. I recently > tested Intel GigE cards ... with dismal results... less than half the > packets-per-second on the (otherwise) same hardware. Small packets > (as in DOS attacks) are a real concern here. > > I believe that someone here recomended Tigon III based cards ... but I > was recently looking through 5.0-RELEASE's hardware notes and couldn't > find any mention of Tigon III. That's the Broadcom BCM570x chipset (bge driver). It's only inofficially called "Tigon III" because it's based on the Tigon I/II from Alteon. Having said that -- We do have a bunch of Compaq DL360-G2 machines in production, each of which has two of those bge interfaces. We make heavy use of VLANs, and the performance is very good. However, we're running 4.7 (because they are production servers), and our packet profile is probably very different from yours (most of the traffic is NFS, HTTP, SQL queries and similar things). Regards Oliver PS: "a bunch" == about 20 of them, more to come. -- Oliver Fromme, secnetix GmbH & Co KG, Oettingenstr. 2, 80538 München Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. "All that we see or seem is just a dream within a dream" (E. A. Poe) To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
