Re: Load Balancing Multiple DS3s (outgoing) on a 7500
Drew, Something that was just released that you might be interested in if you haven't already found an alternate solution. http://www.cisco.com/en/US/products/sw/iosswrel/ps5207/products_feature_guide09186a0080221544.html It's a new feature in 12.3(8)T. Rodney On Fri, Mar 12, 2004 at 10:39:16PM -0500, Drew Weaver wrote: > Does anyone know of an article, or documentation regarding load > balancing the traffic on 3 or more FastEthernet interfaces on the outgoing > direction? Right now we're running BGP internally, and the routes that are > being chosen based upon the final BGP decision step or what I like to call > the 'IP address tie breaker' which is not always optimal. We have a cisco > 7500 that is connected to 4 other Cisco 7500s which each have 45Mbps ds3s to > the Internet, we would like to load balance the outgoing traffic across all > 4 of these 7500s, can anyone shine any advice my way? I noticed that there > are instructions on Cisco's site regarding doing LB on 12000s. > > > > Anyways thanks in advance ;-) > > > > -Drew > > >
Re: Load Balancing Multiple DS3s (outgoing) on a 7500
On Sat, Mar 13, 2004 at 04:19:40PM -0800, Will Yardley wrote: > > On Fri, Mar 12, 2004 at 11:37:25PM -0500, Joe Abley wrote: > > On 12 Mar 2004, at 23:24, joe mcguckin wrote: > > > > > I suspect that each FE goes to a different AS... > > > > In that case, sample/count outbound traffic volumes by > > (prefix/AS/AS_PATH/something), sort the resulting list, and develop an > > import policy based on the top N entries which shares the traffic by > > tweaking some other attribute to avoid the last-resort tie-break. > > The tool "ehnt" is pretty useful for generating a "top" style list of > ASes in order of the amount of traffic you're sending their way. > > By the way, w/r/t to the tiebreaker stuff, note that (on Cisco devices) > if you don't have "bgp bestpath compare-routerid" set, the route that > was received first will be preferred. This minimizes route-flap, but can > cause weird shifts in your traffic patterns when one bgp session or > another goes down (credit goes to Mark Nagel for figuring out this one > for me). > Also, check out: http://www.nanog.org/mtg-0206/feamster.html for some general guidelines, pitfalls, etc. (The paper linked from the presentation recently appeared in ACM Sigcomm CCR.) -Nick
Re: Load Balancing Multiple DS3s (outgoing) on a 7500
Hi Drew - We have 6 backbones distributed across two 7507s and we messed around with a lot of different ways to make this happen. MEDs, Weights, manual BGP configurations every time one of the connections would get overloaded (even at 2am), you name it - we tried it, and in the end we determined that we needed something that could keep an eye on everything and do it automatically within guidelines I had set. In the end, we headed the route of performance-based routing optimization hardware. After testing many different vendors, we choose the RouteScience PathControl box to make my life (as well as the life of my lead backbone engineer) much, much simpler. About a month or two ago, there was quite a discussion on route-optimization hardware on the list including a lot of different ideas. If you do a search on the list for RouteScience or route optimization, you should hit the core of the discussion around the different platforms. If you need more info, feel free to contact me off-list. On Fri, 12 Mar 2004 22:39:16 -0500 Drew Weaver <[EMAIL PROTECTED]> wrote: > Does anyone know of an article, or documentation regarding load > balancing the traffic on 3 or more FastEthernet interfaces on the outgoing > direction? Right now we're running BGP internally, and the routes that are > being chosen based upon the final BGP decision step or what I like to call > the 'IP address tie breaker' which is not always optimal. We have a cisco > 7500 that is connected to 4 other Cisco 7500s which each have 45Mbps ds3s to > the Internet, we would like to load balance the outgoing traffic across all > 4 of these 7500s, can anyone shine any advice my way? I noticed that there > are instructions on Cisco's site regarding doing LB on 12000s. > > > > Anyways thanks in advance ;-) > > > > -Drew > > > ** Richard J. Sears Vice President American Digital Network [EMAIL PROTECTED] http://www.adnc.com 858.576.4272 - Phone 858.427.2401 - Fax I fly because it releases my mind from the tyranny of petty things . . "Work like you don't need the money, love like you've never been hurt and dance like you do when nobody's watching."
Re: Load Balancing Multiple DS3s (outgoing) on a 7500
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Joe Abley wrote: | | | On 12 Mar 2004, at 23:24, joe mcguckin wrote: | |> Patrick, |> |> I suspect that each FE goes to a different AS... | | | In that case, sample/count outbound traffic volumes by | (prefix/AS/AS_PATH/something), sort the resulting list, and develop an | import policy based on the top N entries which shares the traffic by | tweaking some other attribute to avoid the last-resort tie-break. | | Or bypass the measurement part, and make wild guesses about where your | traffic is going, and apply an import policy based on that. Either way, | lather, rinse, repeat. | | There might be something relevant in the slot I did in this tutorial in | Richmond Hill: | | http://www.nanog.org/mtg-0206/te.html | And products from folks like Proficient Networks and Routescience can automate the process for you. - -- = bep -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.2 (MingW32) iD8DBQFAVgGrE1XcgMgrtyYRAo3xAJ4qwszZ/lXxMeMJ5jF2OD9LDaMR/QCeOjz+ a8Mzb383mIOoEE2J0IzVq+I= =4QaS -END PGP SIGNATURE-
Re: Load Balancing Multiple DS3s (outgoing) on a 7500
On Mar 13, 2004, at 4:57 PM, Stephen J. Wilcox wrote: He'll be okie. It's just a little difficult for BGP to "load balance" outbound bits when the bulk of the Internet these days is 2 AS hops away from each of four upstreams. Not impossible, but it doesn't happen by default either. I used to do this ages ago, I did it by setting MEDs on the incoming BGP prefixes, in my route-maps I arbitrarily gave some nets (/8s or smaller) lower med on one feed and higher on the others to influence path selection. I shy away from anything but the gentlest of tweaks so I personally wouldnt mess with as-path, localpref, weight etc Yeah, probably a good idea not to use Weights, but not sure about AS_PATH. Nothing wrong with a prepend here or there, IMHO. :) But also nothing wrong with setting the MEDs if you like. Just be fore to have "always compare MED" on, or MEDs between multiple providers are not useful (which you obviously had set or this wouldn't work). I kinda like setting the origin code. No one pays attention to it, but it is in the selection criteria. that way you can use MEDs from the same provider and still influence routes between providers. -- TTFN, patrick
Re: Load Balancing Multiple DS3s (outgoing) on a 7500
On Fri, Mar 12, 2004 at 11:37:25PM -0500, Joe Abley wrote: > On 12 Mar 2004, at 23:24, joe mcguckin wrote: > > > I suspect that each FE goes to a different AS... > > In that case, sample/count outbound traffic volumes by > (prefix/AS/AS_PATH/something), sort the resulting list, and develop an > import policy based on the top N entries which shares the traffic by > tweaking some other attribute to avoid the last-resort tie-break. The tool "ehnt" is pretty useful for generating a "top" style list of ASes in order of the amount of traffic you're sending their way. By the way, w/r/t to the tiebreaker stuff, note that (on Cisco devices) if you don't have "bgp bestpath compare-routerid" set, the route that was received first will be preferred. This minimizes route-flap, but can cause weird shifts in your traffic patterns when one bgp session or another goes down (credit goes to Mark Nagel for figuring out this one for me). -- "Since when is skepticism un-American? Dissent's not treason but they talk like it's the same..." (Sleater-Kinney - "Combat Rock")
Re: Load Balancing Multiple DS3s (outgoing) on a 7500
> He'll be okie. It's just a little difficult for BGP to "load balance" > outbound bits when the bulk of the Internet these days is 2 AS hops > away from each of four upstreams. Not impossible, but it doesn't > happen by default either. I used to do this ages ago, I did it by setting MEDs on the incoming BGP prefixes, in my route-maps I arbitrarily gave some nets (/8s or smaller) lower med on one feed and higher on the others to influence path selection. I shy away from anything but the gentlest of tweaks so I personally wouldnt mess with as-path, localpref, weight etc Steve
RE: Load Balancing Multiple DS3s (outgoing) on a 7500
> Patrick W.Gilmore wrote: > It's just a little difficult for BGP to "load balance" > outbound bits when the bulk of the Internet these days > is 2 AS hops away from each of four upstreams. Not > impossible, but it doesn't happen by default either. Indeed. If the following conditions are met: a) all four upstreams are transit providers (and therefore even a default to any would be fair game) b) the goal is to distribute more evenly (in terms of bandwidth) the egress traffic between the four upstreams (in other terms, the egress traffic tends to peg one of the upstreams (which are being paid to carry the traffic) for no clear reason) IMHO there is nothing wrong with some WAEG route-map to police egress traffic to a different upstream that the one the BGP process would have chose at the 5th tie-breaker, all 4 including the AS-PATH being equal. My $0.02 plus tax. Michel.
Re: Load Balancing Multiple DS3s (outgoing) on a 7500
On 12 Mar 2004, at 23:24, joe mcguckin wrote: Patrick, I suspect that each FE goes to a different AS... In that case, sample/count outbound traffic volumes by (prefix/AS/AS_PATH/something), sort the resulting list, and develop an import policy based on the top N entries which shares the traffic by tweaking some other attribute to avoid the last-resort tie-break. Or bypass the measurement part, and make wild guesses about where your traffic is going, and apply an import policy based on that. Either way, lather, rinse, repeat. There might be something relevant in the slot I did in this tutorial in Richmond Hill: http://www.nanog.org/mtg-0206/te.html Joe
Re: Load Balancing Multiple DS3s (outgoing) on a 7500
On Mar 12, 2004, at 11:24 PM, joe mcguckin wrote: I suspect that each FE goes to a different AS... Yeppers. We've been corresponding privately, and you got it right (unlike me). He'll be okie. It's just a little difficult for BGP to "load balance" outbound bits when the bulk of the Internet these days is 2 AS hops away from each of four upstreams. Not impossible, but it doesn't happen by default either. -- TTFN, patrick
Re: Load Balancing Multiple DS3s (outgoing) on a 7500
Patrick, I suspect that each FE goes to a different AS... On 3/12/04 7:27 PM, "Patrick W.Gilmore" <[EMAIL PROTECTED]> wrote: > > On Mar 12, 2004, at 10:39 PM, Drew Weaver wrote: > >> Does anyone know of an article, or documentation regarding >> load balancing the traffic on 3 or more FastEthernet interfaces on the >> outgoing direction? Right now we're running BGP internally, and the >> routes that are being chosen based upon the final BGP decision step or >> what I like to call the 'IP address tie breaker' which is not always >> optimal. We have a cisco 7500 that is connected to 4 other Cisco 7500s >> which each have 45Mbps ds3s to the Internet, we would like to load >> balance the outgoing traffic across all 4 of these 7500s, can anyone >> shine any advice my way? I noticed that there are instructions on >> Cisco's site regarding doing LB on 12000s. > > Load balancing with BGP is the same on any cisco router. > > Are you doing BGP with the routers on the other side of those DS3s? If > you are, you will need their help in load balancing properly. Get them > to allow you peering with a loopback interface and use equal cost > static routes to do the load balancing to that loopback interface.
Re: Load Balancing Multiple DS3s (outgoing) on a 7500
On Mar 12, 2004, at 10:39 PM, Drew Weaver wrote: Does anyone know of an article, or documentation regarding load balancing the traffic on 3 or more FastEthernet interfaces on the outgoing direction? Right now we're running BGP internally, and the routes that are being chosen based upon the final BGP decision step or what I like to call the 'IP address tie breaker' which is not always optimal. We have a cisco 7500 that is connected to 4 other Cisco 7500s which each have 45Mbps ds3s to the Internet, we would like to load balance the outgoing traffic across all 4 of these 7500s, can anyone shine any advice my way? I noticed that there are instructions on Cisco's site regarding doing LB on 12000s. Load balancing with BGP is the same on any cisco router. Are you doing BGP with the routers on the other side of those DS3s? If you are, you will need their help in load balancing properly. Get them to allow you peering with a loopback interface and use equal cost static routes to do the load balancing to that loopback interface. -- TTFN, patrick
Load Balancing Multiple DS3s (outgoing) on a 7500
Does anyone know of an article, or documentation regarding load balancing the traffic on 3 or more FastEthernet interfaces on the outgoing direction? Right now we're running BGP internally, and the routes that are being chosen based upon the final BGP decision step or what I like to call the 'IP address tie breaker' which is not always optimal. We have a cisco 7500 that is connected to 4 other Cisco 7500s which each have 45Mbps ds3s to the Internet, we would like to load balance the outgoing traffic across all 4 of these 7500s, can anyone shine any advice my way? I noticed that there are instructions on Cisco's site regarding doing LB on 12000s. Anyways thanks in advance ;-) -Drew
