We are about to discover the joys of upstream BGP routing :-P The current plan is to use a pair of OpenBSD+bgpd hosts as the routers.
Each host will require 4x10gig ports (SFP+). One of those links (to AWS) will be close to saturated, along with the downlink to our switches. The other two will only need to carry ~1Gb/s of traffic. We are pretty much a Supermicro shop, and I'm wondering if anyone out there is running a similar setup on SM hardware. My main concern is finding NICs that will let us squeeze every last drop of bandwidth on the 10gig links. I did run some brief ttcp tests on a pair of SM 1Us (don't have the model number handy, maybe 5018-FTN4s?) with add-in Intel cards (550s?) and was able to get 700 MBytes/s of throughput. This would have been circa the 6.7 or 6.8 releases. I'm hoping to get >70% of the theoretical bandwidth out of the new hardware, and my gut says it's the NIC that's constraining us. So, I'd be interested in hearing from anyone running a similar setup, or who has benchmarked any of the current crop of 10gig NICs and has good/bad things to say about specific models. Thanks, --lyndon
