Re: Advice requested for OpenBSD vs. Linux/OpenBGP vs. Quagga router deployment.
Hi Marc, > We are a software development firm that currently delivers our install ISOs > via Sourceforge. We need to start serving them ourselves for marketing > reasons and are therefore increasing our bandwidth and getting a 2nd ISP in > our datacenter. Both ISPs will be delivering 100mbit/sec links. We don't > expect to increase that for the next year or so and expect average traffic to > be about 40-60mbit/sec. > > We are planning to run two OpenBSD based firewalls (with CARP and pf) running > OpenBGP in order to connect to the two ISPs. > > I saw from previous email that Quagga was recommended as opposed to OpenBGP. > Any further comments on that? Also, any comments on the choice of OpenBSD > vs. Linux? I would suggset checking out Vyatta Linux as a possible Linux solution. It's designed to be configured as a routing/firewall platform. One caveat, I have never used it but it seems to be mentioned in this list from time to time. Now for my rant. I attempted a setup as you describe using two servers using pf, carp, and openbgp. I also had VLANs configured (each VLAN interface had it's own CARP interface). I tried both load-balanced and failover mode but the results weren't desirable. The routers were connected to a switch which connects the servers and the ISP connection. There was only one drop from the ISP but each router had it's own /30 and BGP session on it's own VLAN. The remaining servers were also VLANned appropriately. Each VLAN interface on the router that connects to the servers would also have an accompanying CARP interface. There were a myriad of problems when attempting my setup. These are some that I distinctly recall. * In load-balancing mode I would unplug a router. The other router would register as a CARP master but didn't forward the remaining traffic. * In failover mode when unplugging a router the other router would forward traffic for certain VLANs and wouldn't register as master for the others. In hindsight I should've reached out to the openbsd community for assistance. It's possible I was running into bugs in the CARP code or I was simply doing it all wrong. However I was under a time crunch and this was merely a favour for a friend in need. I didn't want to further disrupt the network by testing so I ended up going with a single router setup (still openbsd though). I haven't revisited the daul router setup since everything has been working fine and dandy with one router. Regardless of what OS choice you make be sure to thoroughly test your network setup and make sure it works as planned. Lastly don't hesitate to ask the appropriate people for help. You may have discovered oddities that noone else has. Good luck, Naveen
Re: Advice requested for OpenBSD vs. Linux/OpenBGP vs. Quagga router deployment.
On Thu, Dec 18, 2008 at 8:55 AM, Beat Vontobel wrote: > Hi Marc, > >> I saw from previous email that Quagga was recommended as opposed to >> OpenBGP. Any further comments on that? Also, any comments on the choice >> of OpenBSD vs. Linux? >> >> I don't want to start a religious war :-) Just curious about what most >> folks are doing and what their experiences have been. For the past couple of years we've had good success running Quagga border router/firewall boxes on Debian booting from Sony Microvault 1GB USB. Standard Debian install with some minor mods to make it USB friendly (noatime, a few /dev/shm links.) Once you've got used to apt, it's hard to accept anything else. I know lots of people prefer a stripped down system, but if you're running the same basic services (BGP, SSH) I don't see the difference. Disclaimer: we only take default from our upstreams, so can't comment on Quagga and full routes. Tim:>
Re: Advice requested for OpenBSD vs. Linux/OpenBGP vs. Quagga router deployment.
Hi Marc, I saw from previous email that Quagga was recommended as opposed to OpenBGP. Any further comments on that? Also, any comments on the choice of OpenBSD vs. Linux? I don't want to start a religious war :-) Just curious about what most folks are doing and what their experiences have been. We run a similar setup since about a year. I also don't want to start a "religious war" (being a happy user of both Linux and OpenBSD, for different purposes), but in this scenario my decision was quick and clear: I went for OpenBSD with OpenBGPD, consistent with my experience throughout the last few years, that for the basic, "hidden" (from end user perspective) network services (routing, firewalling, DHCP, DNS…) OpenBSD never let me down and saved me a _lot_ of time and hassle as an admin (doing this stuff with Linux before). And admin time is often more valuable than that of one or two CPU cycles… (and as long as I get the throughput I demand plus a large enough margin I really don't care about those). My basic rule of thumb now is (and I'm just pragmatic, not religious): If I can get away with the base installation of OpenBSD for a service, I really give it the first try. So for OpenBGPD. It was also the documentation, the clean design and the usability (okay, that's really personal taste, but I really got to love the OpenBSD config file style) that helped with that decision. And from my perspective, it really was the right one: The setup just works, right from the beginning. Flawless. With both Junipers and Ciscos as neighbors. We are planning to run two OpenBSD based firewalls (with CARP and pf) running OpenBGP in order to connect to the two ISPs. Just one thing independent of the OpenBSD vs. Linux question: Depending on the complexity of your setup and maybe also for a cleaner design and possibly additional layers of security, I'd recommend to think about separating the "pure" firewalls from the BGP stuff. I do have three OpenBGPD boxes towards the Internet as our BGP peers plus two redundant pairs of OpenBSD carp/pf boxes towards different internal networks and DMZs. Between the OpenBGPD and the carp/pf boxes is our "backbone". I experimented with a setup as you describe it (many different BGP/ router/firewalling roles combined on one pair of OpenBSD boxes) first, but soon realized that (while perfectly okay for a simple setup) as soon as you get more and more specialized requirements, things tend to get unneccessarily complicated and you're probably better of with dedicated boxes (if not for performance reasons, then still for the design). Best regards, Beat Vontobel -- Beat Vontobel, CTO, MeteoNews AG Siewerdtstr. 105, CH-8050 Zurich, Switzerland E-Mail: b.vonto...@meteonews.ch IT Department: +41 (0)43 288 40 54 Main phone: +41 (0)43 288 40 50
Re: Advice requested for OpenBSD vs. Linux/OpenBGP vs. Quagga router deployment.
On Wed, Dec 17, 2008 at 9:37 AM, Marc Runkel wrote:
[snip]
> Greetings all,
>
> We are a software development firm that currently delivers our install ISOs
> via Sourceforge.
> We need to start serving them ourselves for marketing reasons and are
> therefore increasing
> our bandwidth and getting a 2nd ISP in our datacenter. Both ISPs will be
> delivering
> 100mbit/sec links. We don't expect to increase that for the next year or so
> and expect
> average traffic to be about 40-60mbit/sec.
>
> We are planning to run two OpenBSD based firewalls (with CARP and pf) running
> OpenBGP
> in order to connect to the two ISPs.
>
> I saw from previous email that Quagga was recommended as opposed to OpenBGP.
> Any
> further comments on that? Also, any comments on the choice of OpenBSD vs.
> Linux?
IMO, the performance and utility of OpenBSD as a routing/networking
platform is unmatched by any other open source platform. OpenBGPD
(recent 4-byte ASN issues notwithstanding) has been very stable for us
in production (running roughly equivalent traffic levels to what
you're discussing), and the best part is that you get stateful
transparent failover with CARP, filtering/redirection with pf, load
balancing all the way up through layer7 with relayd, and a host of
other excellent tools for the network engineer's toolkit, all
included, and all integrated. Then of course there's the wider issues
of OpenBSD's track record on security and networking in comparison
with the other OSS platforms, the smaller pool of folks to draw on who
are experienced in running and tuning OpenBSD (although any reasonably
competent UNIX admin should be able to adapt to it in a few days,
given the generally clean layout and high degree of internal
consistency).
advoc...@openbsd.org is down the hall, so I'll stop there. :)
As Adrian said, there are other platforms with better SMP
implementations ... but my experience has been that for small and
mid-size sites, CPU utilization on a reasonably modern x86-based
router is the least of one's worries.
--
darkun...@{gmail.com,darkuncle.net} || 0x5537F527
http://darkuncle.net/pubkey.asc for public key
Re: Advice requested for OpenBSD vs. Linux/OpenBGP vs. Quagga router deployment.
OpenBSD SMP support is quite limited. NetBSD SMP is quite limited. FreeBSD and Linux seem to be running better. :) Adrian On Wed, Dec 17, 2008, Marc Runkel wrote: > Greetings all, > > We are a software development firm that currently delivers our install ISOs > via Sourceforge. We need to start serving them ourselves for marketing > reasons and are therefore increasing our bandwidth and getting a 2nd ISP in > our datacenter. Both ISPs will be delivering 100mbit/sec links. We don't > expect to increase that for the next year or so and expect average traffic to > be about 40-60mbit/sec. > > We are planning to run two OpenBSD based firewalls (with CARP and pf) running > OpenBGP in order to connect to the two ISPs. > > I saw from previous email that Quagga was recommended as opposed to OpenBGP. > Any further comments on that? Also, any comments on the choice of OpenBSD > vs. Linux? > > I don't want to start a religious war :-) Just curious about what most folks > are doing and what their experiences have been. > > Thanks in advance, > > Marc Runkel > Technical Operations Manager > Untangle, Inc. -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support - - $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -
