I will definitely look into some of the resources you mentioned. Again, thanks to all who responded.
~-----Original Message----- ~From: Peter van Oene [mailto:[EMAIL PROTECTED]] ~Sent: Thursday, July 11, 2002 10:16 AM ~To: Lupi, Guy; [EMAIL PROTECTED] ~Subject: RE: Route Reflection with Multiple POPs [7:48509] ~ ~ ~Not really :) Howard's is one of the few SP oriented books on ~the market ~though admittedly I haven't picked it up yet (very broke :) ~ ~The ISP resources at NANOG are helpful and contain some ~valuable links and ~one can glean some valuable information by looking at some of the ~historical design oriented presentations that many have given over the ~years. Cisco also has a document (and recently a book based on the ~document) from Phillip Smith which deals with ISP design tips (called ~something like IOS Essentials: Design tips every ISP should know -- ~something similar to that) Some quick tips would include: ~ ~IGPs: ~- install only link/loopback addresses (ie no redistribution ~into the IGP ~of static/connected/or otherwise) ~- tune metrics for optimal SPF(usually to link delay - ie 70ms ~= 70) - see ~latest nanog for netflow/lots of math based method which seems cool! :) ~- pre allocate contiguous IP space to non-backbone areas such that ~summarization _could_ take place ~- keep in mind summarization when multiple exits from an area ~exist can ~cause sub-optimal routing in normal states and blackholes in ~some failure ~states ~- if you _really_ need to put externals in your IGP (say you ~have a bunch ~of dial routers and marketing gave some customers static IP's that are ~radius assigned (shame on you marketing), use a separate ~instance of the ~IGP or use NSSA with no type 7/5 conversion or full external ~filtering at ~L1L2 border in ISIS and advertise a BGP aggregate from the ~related ABR's. ~- authentication can't hurt ~- hard code RIDs ~ ~ ~BGP ~- peer on lookbacks ~- hard code RIDs ~- use recognizable /32 space for loopbacks for easy router ~identification ~- use bgp-deterministic-med (to reduce some potential for med oriented ~oscillation) (lots of ietf chat on this on IDR group as howard ~points out ~from time to time) ~- build resilient RR clusters (ie two RR servers per group of clients) ~- one cluster ID per RR server (subject to some memory vs ~potential for bad ~routing debate) (also cisco default) ~- use communities and do community based advertisements ~- wrt to advertising to transit providers/peers, use an ~explicit accept / ~default reject style policy for safety (ie when you mess it ~up, you don't ~send anything vs messing it up and advertising transit) ~- authenticate IBGP and possibly EBGP where appropriate ~- use next-hop-self vs passive interface (helps hook mpls into ~forwarding ~plane if you decide to deploy it and keeps IGP smaller) ~- don't dynamically redistribute anything in bgp - ie only ~redist statics ~routes for aggregation ~- try not to leak too much junk to your transit providers (ie ~aggregate ~where you can without too much TE loss) ~- filter customer prefix advertisements - IRR based would be ~ideal, but ~manual is cool too ~- local pref customer routes over peer routes over transit routes ~- either use meds or zero them in and out (better to know what ~is going to ~happen than learn :) ~ ~Lots of more stuff out there likely to cover, but thats my 10 ~minute off ~the top of my head while my soup cooks :) Likely lots there ~to debate as ~well! Building networks generally involves a lot of personal design ~philosophy which makes for healthy debate. ~ ~Pete ~ ~ ~At 09:20 AM 7/11/2002 -0400, you wrote: ~>This is where my thinking went wrong then, Phil also said ~basically the same ~>thing. It seems that I have a basic misunderstanding of how larger ~>providers carry their routing information. Where would I be ~able to get ~>information about proper addressing structures and routing protocol ~>implementation in large/service provider networks? Are there ~any books that ~>go into detail on these particular topics, take you step by ~step from a ~>customer through to the core? ~> ~>-----Original Message----- ~>From: Peter van Oene ~>To: Lupi, Guy; [EMAIL PROTECTED] ~>Sent: 7/11/2002 8:18 AM ~>Subject: RE: Route Reflection with Multiple POPs [7:48509] ~> ~>Hi Guy, ~> ~>As others have pointed out, it is not necessary, nor usually ~desirable, ~>to ~>run separate IGPs in your AS assuming you use your IGP properly. A ~>properly utilized ISP IGP should carry only link and loopback ~addresses ~>and ~>thus be reasonably small. Use of hierarchy can also help ~larger domains ~> ~>scale, though modern IGP implementations on reasonably powered routers ~>should be able to handle rather large single areas. ~> ~>BGP never runs entirely on its own as one always requires ~some method of ~> ~>resolving BGP Next-Hops to forwarding next hops. Static ~routes can work ~> ~>here to connect otherwise disconnected IGP domains, or, separate IGP ~>domains can be used for this purpose as well. ~> ~>At 06:19 PM 7/10/2002 -0400, Lupi, Guy wrote: ~> >I know that you can run confederations and reflectors, and seperate ~>levels ~> >of reflection, which Cisco refers to as "nested reflection". Now my ~> >question is, how would you set up your bgp peering? Due to ~financial ~> >constraints I would imagine that the best thing to do would ~be to have ~>one ~> >circuit from POP router 1 to core router 1, and another circuit from ~>POP ~> >router 2 to core router 2. Assuming that you are only ~running BGP in ~>the ~> >core, and that the clients have to have a session with each ~reflector, ~>how ~> >would you communicate the loopback addresses of all the ~routers to each ~> >other? Are static routes used in this situation? Thanks ~to all of the ~> >people who responded by the way, I appreciate the ~direction. Ever feel ~>like ~> >you know so much, only to read a book and find out that you know so ~>little ~> >:)? ~> > ~> >*-----Original Message----- ~> >*From: Peter van Oene [mailto:[EMAIL PROTECTED]] ~> >*Sent: Wednesday, July 10, 2002 3:00 PM ~> >*To: [EMAIL PROTECTED] ~> >*Subject: Re: Route Reflection with Multiple POPs [7:48509] ~> >* ~> >* ~> >*Couple thoughts. First off, one can confederate and reflect ~> >*at the same ~> >*time (ie rr clusters inside sub-as's). Also, keep in mind ~that these ~> >*techniques deal with control, not forwarding and thus it is ~> >*possible, and ~> >*somewhat common, to have a RR hierarchy that does not map ~> >*directly to the ~> >*physical topology. I tend to see an arbitrary set of core ~> >*routers (from ~> >*2-20ish) that form the central IBGP mesh with the second ~level of the ~> >*hierarchy being formed by lead major pop routers, or ~dedicated route ~> >*servers (ie one armed routers that simply reflect routes). I ~> >*personally ~> >*try to keep the topology as flat as possible (ie less than 4 ~> >*levels in the ~> >*hierarchy) but I don't have any particular technical driver ~> >*for that other ~> >*than to minimize complexity and aid troubleshooting. ~> >* ~> >*Of note, this assumes you are reflecting full routes ~throughout your ~> >*backbone. I've seen providers try and reflect partial routes ~> >*to areas of ~> >*their backbone to allow for smaller routers where RR ~> >*topologies can lead to ~> >*blackholes. In general, if you are reflecting all routes, you ~> >*shouldn't run ~> >*into issues here. ~> >* ~> >*I ~> >* ~> >* ~> >* ~> >*At 04:42 PM 7/10/2002 +0000, Lupi, Guy wrote: ~> >*>Let me preface this by saying that I am trying to learn more ~> >*about large ~> >*>scale BGP design and operation. This question is on route ~> >*reflectors when ~> >*>you have multiple POPs in seperate IGP domains. If you ~> >*currently have one ~> >*>POP and are going to move to 2 within the same AS, you can ~> >*either run full ~> >*>mesh (doesn't scale), reflectors, or confederations. ~> >*Assuming you don't ~> >*>currently have a central core that the POPs connect back to, ~> >*how well does ~> >*>reflection scale? I was reading Building Service ~Provider Networks ~> >*>[Berkowitz], and it states that iBGP doesn't scale well once ~> >*you go above ~> >*>15-20 sessions per router. It also states that most ISPs run ~> >*reflectors ~> >*>instead of confederations, but I believe that statement is ~> >*being made under ~> >*>the assumption that the ISP will have a central core to which ~> >*the POPs will ~> >*>connect. This would indicate to me that assuming you don't ~> >*have a central ~> >*>core, one could only connect 6 or 7 POPs (dual reflectors for ~> >*redundancy) ~> >*>together using reflection before you would have to either ~> >*create a central ~> >*>core to reduce the amount of iBGP sessions, or turn to ~>confederations. ~> >*>Perhaps the best way to accomplish this would be to establish ~> >*a "core" in ~> >*>one of the POPs and run reflection from there, which is also ~> >*presented as a ~> >*>solution in the book? Any opinions? I have made an ~attempt at ASCII ~> >*>drawing below, to me the central core solution makes more sense. ~> >*> ~> >*> First Scenario, no central core ~> >*> ~> >*> POP 1 POP 2 ~> >*> Cluster Cluster ~> >*> o---------------o ~> >*> Full Mesh Between Reflectors ~> >*> o---------------o ~> >*> ~> >*> Second Scenario, central core establised at one of the POPs ~> >*> ~> >*> POP 1 POP 2 ~> >*> Core Reflectors ~> >*> Clients o----- -----o Clients ~> >*> o----- \ / -----o ~> >*> o o ~> >*> POP 3 / \ POP 4 ~> >*> Clients o----- -----o Clients ~> >*> o----- -----o ~> >*> ~> >*> ~> >*>Guy H. Lupi ~> >*>CCIE No. 9275 ~> >* ~> >* ~> >* ~> >* ~[EMAIL PROTECTED] ~ ~ Message Posted at: http://www.groupstudy.com/form/read.php?f=7&i=48590&t=48509 -------------------------------------------------- FAQ, list archives, and subscription info: http://www.groupstudy.com/list/cisco.html Report misconduct and Nondisclosure violations to [EMAIL PROTECTED]
