Re: Scalability issues in the Internet routing system
Susasn, Using the compression (cooking) per router can provide one level of abstraction [reduction of prefix space] at router. So cooking down your Large number of routes to a minimum set of routes can provide some leverage against the prefix growth. By cooking down the prefixes you unfortunately lose topology information which might be a bad thing, and at the same moment disrespect the site's wish to how it would like to be routed. Another bad thing, if you think of companies/sites paying for the entire network in the long run. Apart from that, IMHO cooking down the prefixes only buys time, but does not solve the problem. More people will multihome, and with the current mechanisms and routing cloud, they have to do it by injecting prefixes. I'm not sure whether this hasn't long become an architectural question and should be moved to the (new) IETF arch list. Opinions? Yours, Elmi. PS: Btw, anyone can give me a hint on where to discuss new ideas for e.g. routing schemes (and finding out whether it's an old idea)? -- Begehe nur nicht den Fehler, Meinung durch Sachverstand zu substituieren. (PLemken, [EMAIL PROTECTED]) --[ ELMI-RIPE ]---
Re: And Now for Something Completely Different (was Re: IPv6 news)
Obviously if the RIRs contacted the folks responsible for a given block and were provided justification for its continued allocation, then it should not be reclaimed. On the other hand, folks sitting on several class Bs and not using them could have their blocks reclaimed trivially; ditto for companies that no longer exist. The last is certainly doable without much risk of controversy. This is exactly what the Internic did many years ago. I, like many other people, had registered a .com domain name at no cost. Then suddenly one day, the Internic said that I had to pay an annual subscription fee for this domain name. Like many others, I paid my fees. There were a few complaints about this but by and large people accepted the idea that you had to MAINTAIN a business relationship with the domain name registry in order to continue using a domain name. For some reason, this concept of MAINTAINING a business relationship with the registry, has not caught on in the Regional IP Registry arena. Of course, a large number of IP address users do pay annual membership subscription fees to ARIN (or other RIRs) but not all. And the RIRs seem unwilling to withdraw services (in-addr.arpa) from those who do not MAINTAIN a business relationship. However, one of the articles referred to recently in this thread (I forget which) showed that even if we reclaimed all of the address space that is currently unannounced (in use or not), we'd buy ourselves a trivially short extension of the IPv4 address space exhaustion date. Considering the cost of performing the task, doing so seems rather pointless; our time would be better spent getting IPv6 deployed and either reengineering the routing system or switching to geo addresses. Probably this article from the Cisco IP Journal which has been mentioned a few times in the past week. http://www.cisco.com/en/US/about/ac123/ac147/archived_issues/ipj_8-3/ipv4.html From the viewpoint of avoiding an addressing crisis and avoiding a v6 transition crisis, your advice is sound. However, from the viewpoint of having a sensibly functioning RIR system, I think we still need to deal with two issues. One is that the holders of IP address allocations should be required to maintain a business relationship with the RIR or lose the right to use those IP addresses. The other is that the RIRs need to fix the whole debacle that is whois and routing registries. There should be no need for 3rd party bogon lists. The RIR's should publish an authoritative registry, rooted in IANA, that covers the entire IPv4 and IPv6 address spaces. An operator faced with receiving a new BGP announcement should be able to query such a registry and find out whether or not the address block is allocated, who it is allocated to, and whether that party intends to announce that exact block size from that exact AS number. --Michael Dillon
Re: IPv6 news
Again, phone numbers and their portability can and should not be compared with the IP address portability issues. They're very different animals. That's your elephant. My elephant looks different. Phone numbers and IP addresses are exactly the same. They are numbers used to identify the location to which I want to connect. They are allocated from a numbering plan with a fixed number of digits/bits. The numbering plans are running out of space. One solution being used in both telephony and IP networks, is to carve out smaller allocations (CIDR/pooling) to make the number plan last longer. Sometimes the fact that phone networks are like pink elephants, not grey, is important. Other times The grey elephant keepers watch the behavior of pink elephants to learn about elephants in general. I never suggested that one could mindlessly apply techniques from the telephony world in the IP network world. But we can understand our problems better if we compare them to the telephony world, the ATM world, and maybe even the world of Advanced Data Path Routing Techniques for Three Tier KVM Networks? http://www.tron.com/i032.html There are also lessons to be learned outside the world of telecoms networks by studying the distribution of market towns in ancient Mesopotamia or the crystalline lattice of the skeletons of diatoms and dinoflagellates. --Michael Dillon
Re: And Now for Something Completely Different (was Re: IPv6 news)
On Tue, 2005-10-18 at 15:52 -0700, David Conrad wrote: Hmm. Are the aliens who took the _real_ IETF and replaced it with what's there now going to give it back? :-) Sure they'll hand it back ... when there is no more money to be made from IETF-related technology and politicians no longer feel it's interesting ;) Otoh, the IETF is a function of its partitipants. Businesses today have such fear of competitors and interllectual-propery-issues that they hardly can cooperate on anything. Thus, the number of technology reasearchers available to partitipate in public forums is just a fraction of what it was 20 or 30 years ago. If enough people find IETF unworkable, wouldn't they just form an alternative forum that would be to the IETF what the IETF has been to ITU? //Per
Re: Scalability issues in the Internet routing system
Elmar K. Bins wrote: Susasn, Using the compression (cooking) per router can provide one level of abstraction [reduction of prefix space] at router. So cooking down your Large number of routes to a minimum set of routes can provide some leverage against the prefix growth. By cooking down the prefixes you unfortunately lose topology information which might be a bad thing, and at the same moment disrespect the site's wish to how it would like to be routed. Another bad thing, if you think of companies/sites paying for the entire network in the long run. Cooking prefixes was only meant to be done within the router between the control plane and the (hardware) FIB or forwarding engine. This ain't prefix aggregation within the BGP system. Apart from that, IMHO cooking down the prefixes only buys time, but does not solve the problem. More people will multihome, and with the current mechanisms and routing cloud, they have to do it by injecting prefixes. And this won't change in future. I'm not sure whether this hasn't long become an architectural question and should be moved to the (new) IETF arch list. Opinions? Yours, Elmi. PS: Btw, anyone can give me a hint on where to discuss new ideas for e.g. routing schemes (and finding out whether it's an old idea)? With pretty high certainy one can say that it is an old idea with some minor twist or wording change. -- Andre
Re: Scalability issues in the Internet routing system
[EMAIL PROTECTED] (Andre Oppermann) wrote: Apart from that, IMHO cooking down the prefixes only buys time, but does not solve the problem. More people will multihome, and with the current mechanisms and routing cloud, they have to do it by injecting prefixes. And this won't change in future. I'm not sure whether this hasn't long become an architectural question and should be moved to the (new) IETF arch list. Opinions? Yours, Elmi. PS: Btw, anyone can give me a hint on where to discuss new ideas for e.g. routing schemes (and finding out whether it's an old idea)? With pretty high certainy one can say that it is an old idea with some minor twist or wording change. I was thinking of something different from Susan's approach, hence my question. Cooking up stuff to save memory and processing in the router isn't it, IMHO, but I have ideas in my head which I would like to share. But then, I wouldn't want to spoil everybody's time if it's old. Elmar. -- Begehe nur nicht den Fehler, Meinung durch Sachverstand zu substituieren. (PLemken, [EMAIL PROTECTED]) --[ ELMI-RIPE ]---
Re: Scalability issues in the Internet routing system
Tony Li wrote: Andre, capacity = prefix * path * churnfactor / second capacity = prefixes * packets / second I think it is safe, even with projected AS and IP uptake, to assume Moore's law can cope with this. This one is much harder to cope with as the number of prefixes and the link speeds are rising. Thus the problem is multiplicative to quadratic. You'll note that the number of prefixes is key to both of your equations. If the number of prefixes exceeds Moore's law, then it will be very difficult to get either of your equations to remain under Moore's law on the left hand side. That's the whole point of the discussion. Let me rephrase my statement so we aren't talking past each other. The control plane (BGP) scales pretty much linearly (as Richard has observed too) with the number of prefixes. It is unlikely that the growth in prefixes and prefix churn manages to exceed the increase in readily available control plane CPU power. For example a little VIA C3-800MHz can easily handle 10 current full feeds running OpenBDPd (for which I have done the internal data structures design). Guess what a $500 AMD Opteron or Intel P4 can handle. In addition BGP lends itself relatively well to scaling on SMP. So upcoming dual- or multicore CPU's help to keep at least pace with prefix growth. Conclusion: There is not much risk on the control plane running BGP even with high prefix growth. On the other hand the forwarding plane doesn't have the same scaling properties. It faces not one but two raising factors. The number of prefixes (after cooking) and the number of lookups per second (equal pps) as defined by link speed. Here a 10-fold increase in prefixes and a 10-fold increase in lookups/second may well exceed the advances in chip design and manufactoring capabilities. A 10-fold increase in prefixes means you have search 10 times as many prefixes (however optimized that is) and a 10-fold increase in link speed means you have only 1/10 the time for search you had before. There are many optimization thinkable to solve each of these. Some scale better in terms of price/performance, others dont. My last remark in the original mail meant that the scaling properties of longest-match in hardware are less good than for perfect matching. My personal conclusion is that we may have to move the DFZ routing to some sort of fixed sized (32bit for example) identifier on which the forwarding plane can do perfect matching. This is not unlike the rationale behind MPLS. However here we need something that administratively and politically works inter-AS like prefix+BGP today. Maybe the new 32bit AS number may serve as such a perfect match routing identifier. That'd make up to 4 billion possible entries in the DFZ routing system. Or about 16k at todays size of the DFZ. One AS == one routing policy. -- Andre
Re: Scalability issues in the Internet routing system
On Wed, 2005-10-19 at 09:31 +0200, Elmar K. Bins wrote: Susasn, Using the compression (cooking) per router can provide one level of abstraction [reduction of prefix space] at router. So cooking down your Large number of routes to a minimum set of routes can provide some leverage against the prefix growth. By cooking down the prefixes you unfortunately lose topology information which might be a bad thing, and at the same moment disrespect the site's wish to how it would like to be routed. Another bad thing, if you think of companies/sites paying for the entire network in the long run. Don't expect an interest in your topology to reach much beyond your peers. This isn't about prefix-aggregation, but about dropping information from a router's memory that isn likely to be used. With many peers you may have a large number of route entries for a given prefix from which your router chooses one to be used and possibly re-distributed to others. Ex: All that happens if you choose to only keep the 2 best alternatives out of 5 or more is loss of redundancy in case both the best routes are withdrawn. The algorithms used to select the best path(s) remain unchanged. Apart from that, IMHO cooking down the prefixes only buys time, but does not solve the problem. More people will multihome, and with the current mechanisms and routing cloud, they have to do it by injecting prefixes. I'm not sure whether this hasn't long become an architectural question and should be moved to the (new) IETF arch list. Opinions? Agree ... unless Yours, Elmi. PS: Btw, anyone can give me a hint on where to discuss new ideas for e.g. routing schemes (and finding out whether it's an old idea)? I'm aware of the new architecture list, but maybe the IRTF Routing Research Group (http://psg.com/~avri/irtf/rrg-page.html) is an even more appropriate place. From their charter: The Routing Research Group (RRG) is a group chartered under the Internet Research Task Force to explore routing problems that are important to the development of the Internet but are not yet mature enough for engineering work within the IETF. Of partial relevance to the recent discussion you'll find documents there like a recent compilation of requirements for future inter-domain-routing: http://www.watersprings.org/pub/id/draft-irtf-routing-reqs-03.txt //Per
Re: IPv6 news
On Wed, 19 Oct 2005, [EMAIL PROTECTED] wrote: Again, phone numbers and their portability can and should not be compared with the IP address portability issues. They're very different animals. That's your elephant. My elephant looks different. Survey says... BZT. Read about SS7 LNP implementation before speaking, please. They are very different creatures. Something that resembles telephony LNP will not scale to the quantity of micro-streams currently used by WWW applications. The reason it works (FSVO works) for telephony is because, unlike TCP streams, telephone circuits are comparatively large streams with much longer keepalive times. -- -- Todd Vierling [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
RE: Scalability issues in the Internet routing system
Elmer: You are not cooking for the routing table (hence loss of information). You are cooking for the forwarding entries in the line cards.. As to architectural discussion, I believe the IRTF is publishing work from On NG architecture. I'll be glad to send you pointers. 2 panels of routing experts suggested algorithms and changes. That will tell you some of the things that were thought of. Sue -Original Message- From: Elmar K. Bins [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 19, 2005 3:31 AM To: Susan Hares Cc: [EMAIL PROTECTED] Subject: Re: Scalability issues in the Internet routing system Susasn, Using the compression (cooking) per router can provide one level of abstraction [reduction of prefix space] at router. So cooking down your Large number of routes to a minimum set of routes can provide some leverage against the prefix growth. By cooking down the prefixes you unfortunately lose topology information which might be a bad thing, and at the same moment disrespect the site's wish to how it would like to be routed. Another bad thing, if you think of companies/sites paying for the entire network in the long run. Apart from that, IMHO cooking down the prefixes only buys time, but does not solve the problem. More people will multihome, and with the current mechanisms and routing cloud, they have to do it by injecting prefixes. I'm not sure whether this hasn't long become an architectural question and should be moved to the (new) IETF arch list. Opinions? Yours, Elmi. PS: Btw, anyone can give me a hint on where to discuss new ideas for e.g. routing schemes (and finding out whether it's an old idea)? -- Begehe nur nicht den Fehler, Meinung durch Sachverstand zu substituieren. (PLemken, [EMAIL PROTECTED]) --[ ELMI-RIPE ]---
Re: IPv6 news
Survey says... BZT. Yaur argument is fallacious. Read about SS7 LNP implementation before speaking, please. I never said anything about SS7 implementation of LNP. They are very different creatures. Something that resembles telephony LNP will not scale to the quantity of micro-streams currently used by WWW applications. The reason it works (FSVO works) for telephony is because, unlike TCP streams, telephone circuits are comparatively large streams with much longer keepalive times. This is a strawman argument. I certainly agree with what you have said about TCP streams versus calls on the PSTN but that has nothing whatsoever to do with what I was talking about. Why is it that whenever people suggest that the IP networking world can learn from the experience of the telephony world, some people assume that the proposal is to imitate the telephony world in every detail? The fact is that both worlds are completely different in the details. But these different details lead the telephony world to make different technology choices and then gain real world operational experience with those choices. As the IP world evolves and changes (remember this started with a discussion of IPv6) it is possible that some of the hard-won experience from the telephony world can be applied in the IP world. No doubt it will be necessary to implement things differently in the IP world because of the details. But it is crazy to reject the hard won experience of the telephony world wholesale just because you worship at the temple of IP. In any case, the telephony world owns and runs the Internet today. Bellhead and nethead arguments belong in the past. Today's bellheads are running IP networks and VoIP along with all the PDH, SDH, X.25, SS7, ATM, Frame Relay etc. --Michael Dillon
multi homing pressure
Firms must defend against ISP clashes, warns Gartner Commercial row between ISPs shows vulnerability of single sourcing says analyst http://www.computerweekly.com/Articles/Article.aspx?liArticleID=212391 Looks like it's about to enter the corporate rule book Gartner said every location that requires mission-critical internet connectivity, including externally hosted websites, should be multi-homed and end tier 1 hosting brandon
Re: IPv6 news
On Oct 19, 2005, at 9:39 AM, [EMAIL PROTECTED] wrote: Why is it that whenever people suggest that the IP networking world can learn from the experience of the telephony world, some people assume that the proposal is to imitate the telephony world in every detail? Seems to me to be a species of the broader attitude among some that any analogy or reference to anything outside of the IP world is inherently flawed. On this view, there is no such thing as a good analogy or comparison. All are equally bad and misleading. Full stop. Perhaps an ungenerous observation, but this is exactly the sort of attitude that one would expect of narrow experts in the field who have no knowledge whether and how it might resemble or relate to anything else... Private editorial ends. TV
Re: multi homing pressure
On Wed, Oct 19, 2005 at 03:48:09PM +0100, Brandon Butterworth wrote: Firms must defend against ISP clashes, warns Gartner Commercial row between ISPs shows vulnerability of single sourcing says analyst http://www.computerweekly.com/Articles/Article.aspx?liArticleID=212391 Looks like it's about to enter the corporate rule book Gartner said every location that requires mission-critical internet connectivity, including externally hosted websites, should be multi-homed it will be interesting to see if this has acutal impact on ASN allocation rates globally. - jared -- Jared Mauch | pgp key available via finger from [EMAIL PROTECTED] clue++; | http://puck.nether.net/~jared/ My statements are only mine.
Re: multi homing pressure
On Wed, 19 Oct 2005, Brandon Butterworth wrote: Gartner said every location that requires mission-critical internet connectivity, including externally hosted websites, should be multi-homed 200k routes, here we come! -- -- Todd Vierling [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: multi homing pressure
On Wed, 19 Oct 2005, Todd Vierling wrote: On Wed, 19 Oct 2005, Brandon Butterworth wrote: Gartner said every location that requires mission-critical internet connectivity, including externally hosted websites, should be multi-homed 200k routes, here we come! it is just good common sense though, eh? Also, there have been some pressures through the SOX and other compliance activities to push dual carrier things in the recent past.
Re: multi homing pressure
In a message written on Wed, Oct 19, 2005 at 11:31:32AM -0400, Jared Mauch wrote: it will be interesting to see if this has acutal impact on ASN allocation rates globally. I have done no analysis, but I do believe this is having an effect on the number of prefixes announced by many of the players involved. Looking at the top 10-20 peers over here, all of them show prefixes announced by the peers to be growing faster than the global prefix table. The only way that makes sense is if existing prefixes are being announced through additional providers. It would be interesting to see those more into BGP routing analysis to look at that (possible) trend. It's probably causing a shift in how BGP processing occures on both a device and a network level (more redundant paths) which could have implications for future gear. -- Leo Bicknell - [EMAIL PROTECTED] - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/ Read TMBG List - [EMAIL PROTECTED], www.tmbg.org pgpNw9b16ZBtr.pgp Description: PGP signature
Re: multi homing pressure
[EMAIL PROTECTED] (Todd Vierling) wrote: Tier-2s should be given much more credit than they typically are in write-ups like this. When a customer is single homed to a tier-2 that has multiple tier-1 upstreams, and uses a delegated netblock from the tier-2's aggregations, that means one less ASN and one or more less routes in the global table. That's the operators' view, but not the customer's. The customer wants redundancy. So we should try to find a way to tell them Hey, it's mostly Tier-1's (or wannabes) that play such games, stick to a trustworthy Tier-2. And, hey, btw., connect redundantly to them, so you have line failure resiliency and also a competent partner that cares for everything else. Only seeing the operators' view will amount to nothing in the customer's will to run along with the Tier-2. Eventually, it breaks down to trust. And customers learn that the big players are not always trustworthy. Oh, and customers do not always remember names. Yours, Elmar. -- Begehe nur nicht den Fehler, Meinung durch Sachverstand zu substituieren. (PLemken, [EMAIL PROTECTED]) --[ ELMI-RIPE ]---
Re: multi homing pressure
On Oct 19, 2005, at 11:54 AM, Todd Vierling wrote: Gartner said every location that requires mission-critical internet connectivity, including externally hosted websites, should be multi-homed 200k routes, here we come! it is just good common sense though, eh? Well, not necessarily. Tier-2s should be given much more credit than they typically are in write-ups like this. When a customer is single homed to a tier-2 that has multiple tier-1 upstreams, and uses a delegated netblock from the tier-2's aggregations, that means one less ASN and one or more less routes in the global table. It's a Good Thing(tm). For you. For the customer with an Internet mission critical app, being tied to a Tier 2 has it's own set of problems, which might actually be worse than being tied to a Tier 1. The Internet is a business tool. If providers do not meet business requirements, providers will not be supported. Period. If 200K routes, or more ASNs, or small customers multihoming, or stuff like that scares you, find another line of work. (Hint-hint to the v6 fanatics.) -- TTFN, patrick
Re: multi homing pressure
On Wed, 19 Oct 2005, Todd Vierling wrote: On Wed, 19 Oct 2005, Christopher L. Morrow wrote: Gartner said every location that requires mission-critical internet connectivity, including externally hosted websites, should be multi-homed 200k routes, here we come! it is just good common sense though, eh? Well, not necessarily. Sorry, Gartner said every location that requires mission-critical internet connectivity, including externally hosted websites, should be multi-homed is common sense. If you have something 'mission critical' to your business you had better have more than one link out... It'd make great sense to make sure that the links in question atleast didn't end up on the same router at the far end, and while you are at it, get them to different providers (hopefully) in different telco-hotels. Tier-2s should be given much more credit than they typically are in write-ups like this. When a customer is single homed to a tier-2 that has multiple tier-1 upstreams, and uses a delegated netblock from the tier-2's aggregations, that means one less ASN and one or more less routes in the global table. I'm not such a believer in the tier-n classifications, single homing a critical resource is just dumb, regardless of the 'tier' you stick it in. gartner, as gartner normally does, is just stating the obvious and making money doing it.
Re: multi homing pressure
[EMAIL PROTECTED] (Patrick W. Gilmore) wrote: For the customer with an Internet mission critical app, being tied to a Tier 2 has it's own set of problems, which might actually be worse than being tied to a Tier 1. Please elaborate. Elmar.
Re: multi homing pressure
On Wed, 19 Oct 2005, Elmar K. Bins wrote: Tier-2s should be given much more credit than they typically are in write-ups like this. When a customer is single homed to a tier-2 that has multiple tier-1 upstreams, and uses a delegated netblock from the tier-2's aggregations, that means one less ASN and one or more less routes in the global table. That's the operators' view, but not the customer's. The customer wants redundancy. That's why SLAs exist. So we should try to find a way to tell them Hey, it's mostly Tier-1's (or wannabes) that play such games, stick to a trustworthy Tier-2. And, hey, btw., connect redundantly to them, so you have line failure resiliency and also a competent partner that cares for everything else. Something like that, but not quite. Whenever one of these reports, which boil down to everyone must multi-home!, appears, it typically has a stark lack of information on alternatives to *direct* multi-homing. Many customers would rather not multihome directly, and prefer set it and forget it connectivity. It's much easier to maintain a multi-pipe connection that consists of one static default route than a pipe to multiple carriers. The former requires simple physical pipe management, which can be left alone for 99% of the time. The latter requires BGP feed, an ASN, and typically much more than 1% of an employee's time to keep running smoothly. Obtaining single-homed connectivity from a Tier-2 mostly outsources network support, and small to medium size businesses tend to like that. It's not the only leaf end solution to the problem, but it's a viable one (and can be less costly to the rest of the world in turn). -- -- Todd Vierling [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: multi homing pressure
On Wed, 2005-10-19 at 12:03 -0400, Patrick W. Gilmore wrote: For the customer with an Internet mission critical app, being tied to a Tier 2 has it's own set of problems, which might actually be worse than being tied to a Tier 1. I think this is largely dependant on the specific topology and redundancy in the Tier-2's network and the way they provide multiple uplinks. When done well, with uplinks spread over separate physical locations, well thought out IP adressing and de-centralised exits from the Tier-2's network out to multiple Tier-n's, there's usually a benefit to multi-homed connections to a Tier-2 rather than a Tier-1, with minimum capacity and pricing being the most important ones. -- --- Erik Haagsman Network Architect We Dare BV Tel: +31(0)10-7507008 Fax: +31(0)10-7507005 http://www.we-dare.nl
Re: multi homing pressure
On Oct 19, 2005, at 12:08 PM, Elmar K. Bins wrote: [EMAIL PROTECTED] (Patrick W. Gilmore) wrote: For the customer with an Internet mission critical app, being tied to a Tier 2 has it's own set of problems, which might actually be worse than being tied to a Tier 1. Please elaborate. I probably used poor word choice. The Tier of a provider is just marketing unless you are talking about the networks in the DFZ (SFI club, or whatever). When I made that statement, I was thinking more about the marketing hype, meaning a tier two not only has transit, but is a smaller, probably regional provider. Naturally, a tier two might very well be a huge company with network assets on multiple continents and 10s of Gbps (or more) of traffic. The problems with a small provider might include: * Business viability * Global reach * Capacity * Redundant architecture * Etc., etc., etc. Depends on the customer which of these are important. The guy with 10 Mbps of traffic all in Nashville, TN doesn't care about global reach or capacity, but might be VERY interested in redundant architecture business viability. A mom-n-pop shop might not be the right choice for him. The guy with 12 datacenters in 6 states - or countries - might have different ideas of what is important. Maybe he's OK with one site going offline one day a year if he can save $10K on transit costs, so a mom-n-pop shop would be fine. Personally, I think if your application really is mission critical, then you _must_ be multi-homed with your own space and own ASN. Anything less and you've tied your business to a vendor. I might like some networks, but I don't want my corporate life death to be tied to any of them when I can ensure my independence for what is probably a small sum of money in the grand scheme of things. The question of which providers to use as upstreams is a business decision based on the items above, cost, and lots of other stuff. Sorry if I was unclear before. -- TTFn, patrick
Re: multi homing pressure
[EMAIL PROTECTED] (Patrick W. Gilmore) wrote: The problems with a small provider might include: * Business viability * Global reach * Capacity * Redundant architecture * Etc., etc., etc. Thanks - understood ;-) I see, btw, a lot of Tier-3 (or -4, -5) providers that have easily survived their Tier-2 Transits going down the river. If customers can be convinced that the tier thing has nothing to do with business viability and/or stability, and that size does not always matter, we might get them on the right track. As long as they believe the marketing-speak, that Tier-1 ISPs are god (no double-o here) and Tier-2's are still quite good, but everything else is crap for real business, well... Always remember: For every customer, their stuff _is_ mission critical. So everyone will take the multihoming road if they can afford it. We can make it more expensive, or we can offer other solutions. Yours, Elmar (formerly with a Tier-17, quite stable though) -- Begehe nur nicht den Fehler, Meinung durch Sachverstand zu substituieren. (PLemken, [EMAIL PROTECTED]) --[ ELMI-RIPE ]---
Re: multi homing pressure
At 12:20 -0400 10/19/05, Todd Vierling wrote: That's why SLAs exist. Do SLA's say if you are out of the water for 30 minutes we will also cover your lost business revenue? There are some times with service guarantees just are not enough (e.g., manned space flight support). -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Edward Lewis+1-571-434-5468 NeuStar True story: Only a routing expert would fly London-Minneapolis-Dallas-Minneapolis to get home from a conference. (Cities changed to protect his identity.)
Re: multi homing pressure
On Oct 19, 2005, at 12:34 PM, Edward Lewis wrote: At 12:20 -0400 10/19/05, Todd Vierling wrote: That's why SLAs exist. Do SLA's say if you are out of the water for 30 minutes we will also cover your lost business revenue? There are some times with service guarantees just are not enough (e.g., manned space flight support). There is no such thing as an SLA on the Internet that is worth what you lose when the line goes down. Part of the problem of paying so little for a mission critical application. And part of the reason multi-homing is so important for a mission critical application. -- TTFN, patrick
Re: multi homing pressure
On Wed, 19 Oct 2005, Todd Vierling wrote: That's the operators' view, but not the customer's. The customer wants redundancy. That's why SLAs exist. SLAs exist to provide a means of allowing a vendor to 'feel your pain' when you experience some type of a service outage. They generally do not exist to act as a cost recovery mechanism for customers who lose revenue when mission critical app XYZ can't access 'the network', people coming in from 'the network' cannot access it. Being able to deduct some percentage of your monthly spend with your carrier often doesn't balance well against a network outage that affects the mission critical app that brings in a substantial percentage of the company's revenue. Each organization's tolerance for outages (read: revenue impact) must be weighed against the costs of multihoming and making the investments in infrastructure to improve relibility. Something like that, but not quite. Whenever one of these reports, which boil down to everyone must multi-home!, appears, it typically has a stark lack of information on alternatives to *direct* multi-homing. Hence Chris's earlier post about the multitude of think tanks which exist to state the obvious and make a buck while doing it :-) Many customers would rather not multihome directly, and prefer set it and forget it connectivity. It's much easier to maintain a multi-pipe connection that consists of one static default route than a pipe to multiple carriers. The former requires simple physical pipe management, which can be left alone for 99% of the time. The latter requires BGP feed, an ASN, and typically much more than 1% of an employee's time to keep running smoothly. I disagree with some of this. I've set BGP up for customers before on many occasions. Many were quite happy with a primary-backup mode of connectivity, which can be accommodated by getting an ASN, configuring BGP on your router(s) and with your upstreams, announcing your route(s) and accepting a default route from those upstreams. My experience has been that these types of setups are also pretty much 'fire and forget' as far as the customer is concerned - *once they're up and running*. If customer XYZ doesn't have the expertise in-house to set it up, they will often bring in a consultant to do it. I do agree that the process of applying for an ASN and so forth can take some time, but it's typically a one-time process. Customers who want to actually attempt to tune traffic to fit the size of their pipes are the ones who require much more time in the maintenance of their BGP environment, and often have higher capex and opex to go with it (bigger router to handle full BGP feeds, $router_vendor support contracts for same, etc). Obtaining single-homed connectivity from a Tier-2 mostly outsources network support, and small to medium size businesses tend to like that. It's not the only leaf end solution to the problem, but it's a viable one (and can be less costly to the rest of the world in turn). That depends greatly on the business need that drives the decision in the first place. Plus, some organizations are finding that locating critical service outside of their borders either voliates their security policies, or can hamper compliance with outside security mandates (GLB, SOX, HIPAA, etc). Maintaining compliance + improving reliability can motivate organizations to multihome. jms
Re: multi homing pressure
TV Date: Wed, 19 Oct 2005 12:20:25 -0400 (EDT) TV From: Todd Vierling TV That's why SLAs exist. I thought SLAs existed to comfort nontechnical people into signing contracts, then denying credits via careful weasel words when the time comes for the customer to collect. Or maybe I'm just cynical. TV Many customers would rather not multihome directly, and prefer set it and TV forget it connectivity. It's much easier to maintain a multi-pipe TV connection that consists of one static default route than a pipe to multiple TV carriers. The former requires simple physical pipe management, which can be TV left alone for 99% of the time. The latter requires BGP feed, an ASN, and TV typically much more than 1% of an employee's time to keep running smoothly. Single carrier + multiple POPs != difficult. Even a lowly 2500 can be loaded up with a carrier-assigned private ASN and fed a couple routes. (Maybe it's a little more complex when one needs equal-cost multipath, but it's still hardly rocket science.) TV Obtaining single-homed connectivity from a Tier-2 mostly outsources TV network support, and small to medium size businesses tend to like that. See above. TV It's not the only leaf end solution to the problem, but it's a viable one TV (and can be less costly to the rest of the world in turn). Eddy -- Everquick Internet - http://www.everquick.net/ A division of Brotsman Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita DO NOT send mail to the following addresses: [EMAIL PROTECTED] -*- [EMAIL PROTECTED] -*- [EMAIL PROTECTED] Sending mail to spambait addresses is a great way to get blocked. Ditto for broken OOO autoresponders and foolish AV software backscatter.
Re: multi homing pressure
For the customer with an Internet mission critical app, being tied to a Tier 2 has it's own set of problems, which might actually be worse than being tied to a Tier 1. The key word is might. In fact, I would posit that a Tier 2 with multiply redundant transit to all of the Tier 1s could theoretically have better connectivity than an actual Tier 1. The Tier 2 transit provides flexibility that the transit-free Tier 1s do not have. Just my opinion. Anyway, it has been my experience that most (but not all) of the customers that want to multihome are _really_ wanting either: A. geographic/router redundancy. or B. easy renumbering. Geographic redundancy can be done within a single AS and IP block. They just don't know to ask it that way. (And easy renumbering will eventually be solved with v6. Eventually.) The demand for multi-homing might not be as great as suspected. John
Re: Verizon outage in Southern California?
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
I wonder what ever happened to redundancy? I guess 5 9s (dunno what the
going number is) got blown out of the water for them.
regards,
/virendra
David Lesher wrote:
Speaking on Deep Background, the Press Secretary whispered:
I'm not completely familiar with the telco jargon.
Does Tandem mean the same as a local central office, where
POTS lines terminate at the switch? Long Beach has a population
of 470,000. The C/Os I know of are:
A Central Office switch talks to subscribers aka end-users.
On its backside, it talks to other CO's and tandems. Time
was, that was also VF copper pairs, but it's long since all
DS1 and up.
A tandem is a switch that talks not to subs, but only to CO's. In
days of old, when a {dialup} call went to the other side of town,
chances are it went you-yourCO-downtown tandem-joesCO-joe. {copper
all the way...}.
A tandem was always housed in large CO building, but might have
been ATT's vice the operationg company, etc...
But ESS's and classless switching and massive expansion of the
plant really muddled the picture. An ESS could be both a CO switch
[for multiple prefixes and even multiple NPA's..] AND act like a
tandem.. And oh, the actual line cards can be remoted 100 miles
away in a horz. phonebooth box alongside the road in Smallville
with DS1's/OC coming back.
My guess is a DACS, a cross-connect point that is an software-driven
patch panel, lost its marbles. [engineering term of art.]
A DACS could have dozen-MANY dozen DS1/DS3/OC-n going hither
and yon. Some will be leased circuits. Others will be the CO trunks
going from one switch to another. It may/may not have muxes internal,
so that what arrives on a DS1 leaves in a OC96..
I note it went down at 2:20 AM. That SCREAMS software
upgrade/cutover. What's to bet GEE, no...VZEEE, was doing just
that and there was a major ohshit.
Sean noted a long while back that somehow, DACS crashes always
seem to take hours to recover. Maybe the backups are on Kansas
City standard tapes, I donno.. but this sounds like that..
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (MingW32)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFDVoJXpbZvCIJx1bcRAstJAJ0dnrQL1P2QJyxNU3r0T/X8g9fukQCgnm/N
yW5EvW7gI3gfjY7XSozyMds=
=ocNd
-END PGP SIGNATURE-
Re: multi homing pressure
At 01:05 PM 10/19/2005, John Dupuy wrote: For the customer with an Internet mission critical app, being tied to a Tier 2 has it's own set of problems, which might actually be worse than being tied to a Tier 1. The key word is might. In fact, I would posit that a Tier 2 with multiply redundant transit to all of the Tier 1s could theoretically have better connectivity than an actual Tier 1. The Tier 2 transit provides flexibility that the transit-free Tier 1s do not have. Just my opinion. Anyway, it has been my experience that most (but not all) of the customers that want to multihome are _really_ wanting either: A. geographic/router redundancy. or B. easy renumbering. Geographic redundancy can be done within a single AS and IP block. They just don't know to ask it that way. (And easy renumbering will eventually be solved with v6. Eventually.) It has been my experience that most needing to multihome wish to do so to avoid failures within an ISP, failures with a circuit to the ISP, and failures with routers. I should think that with the recent L3/Cogent issue, it should be QUITE clear that multihoming requires linking to two separate backbones, or two separate regionals that buy transit from multiple backbones. Vagaries in backbone providers is high on the list, IMO, and rules out the multihome to a single provider approach. The demand for multi-homing might not be as great as suspected. John
Re: multi homing pressure
At 11:59 AM 19/10/2005, Elmar K. Bins wrote: [EMAIL PROTECTED] (Todd Vierling) wrote: Tier-2s should be given much more credit than they typically are in write-ups like this. When a customer is single homed to a tier-2 that has multiple tier-1 upstreams, and uses a delegated netblock from the tier-2's aggregations, that means one less ASN and one or more less routes in the global table. That's the operators' view, but not the customer's. The customer wants redundancy. The customer wants reliability and BGP is not necessarily the way for them to do it. Telling a typical corporate IT department with generalized IT skills (read no large Internetworking experience) to now become BGP masters will only open up a news ways to disrupt their network connectivity. There are better ways to do it as you describe below. So we should try to find a way to tell them Hey, it's mostly Tier-1's (or wannabes) that play such games, stick to a trustworthy Tier-2. And, hey, btw., connect redundantly to them, so you have line failure resiliency and also a competent partner that cares for everything else. Agreed! ---Mike
Re: multi homing pressure
[EMAIL PROTECTED] (Mike Tancsa) wrote: The customer wants redundancy. The customer wants reliability That's what you know and what I know. The customer has already jumped one step ahead from reliability to multiple providers, just like he does with parcel services etc. There are better ways to do it as you describe below. Yup, but the customer needs to be made aware of them. Elmi. -- Begehe nur nicht den Fehler, Meinung durch Sachverstand zu substituieren. (PLemken, [EMAIL PROTECTED]) --[ ELMI-RIPE ]---
Re: multi homing pressure
On Oct 19, 2005, at 12:20 PM, Todd Vierling wrote: Many customers would rather not multihome directly, and prefer set it and forget it connectivity. It's much easier to maintain a multi-pipe connection that consists of one static default route than a pipe to multiple carriers. The former requires simple physical pipe management, which can be left alone for 99% of the time. The latter requires BGP feed, an ASN, and typically much more than 1% of an employee's time to keep running smoothly. Hrm, people keep saying that BGP is hard and takes time. As well as my end-user-facing network responsibilities, I also have corporate network responsibilities here. All of our corporate hub locations are multi-homed (or soon will be)... and I honestly can't remember the last time I made any changes (besides IOS upgrades) to BGP configs for the 2 hubs in the US. (We're moving physical locations in the international hubs and taking new providers, so I'm discounting those changes as you'd have similar changes in a single homed statically routed move). If you don't have multihoming requirements other than availability then it really can be fire and forget.
Re: multi homing pressure
John Payne wrote: Hrm, people keep saying that BGP is hard and takes time. As well as my end-user-facing network responsibilities, I also have corporate network responsibilities here. All of our corporate hub locations are multi-homed (or soon will be)... and I honestly can't remember the last time I made any changes (besides IOS upgrades) to BGP configs for the 2 hubs in the US. (We're moving physical locations in the international hubs and taking new providers, so I'm discounting those changes as you'd have similar changes in a single homed statically routed move). If you don't have multihoming requirements other than availability then it really can be fire and forget. Except for those pesky bogon filters which corporations seem to like to fire and forget. -- Mark Radabaugh Amplex [EMAIL PROTECTED] 419.837.5015
Re: multi homing pressure
On Wed, 19 Oct 2005, John Payne wrote: Hrm, people keep saying that BGP is hard and takes time. As well as my end-user-facing network responsibilities, I also have corporate network responsibilities here. All of our corporate hub locations are multi-homed (or soon will be)... and I honestly can't remember the last time I made any changes It's not changes that make BGP maintenance consume time; rather, it's tracking down connectivity issues when problems do arise. If the upstream is responsible for multihoming instead, they also are responsible for keeping the knowledge resources to do that problem-hunt. It's another side effect of the choice to outsource the multihoming responsibility, and is one of the factors to consider when choosing a redundancy approach. -- -- Todd Vierling [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED]
Re: multi homing pressure
On Oct 19, 2005, at 3:31 PM, Mark Radabaugh wrote: John Payne wrote: [...] If you don't have multihoming requirements other than availability then it really can be fire and forget. Except for those pesky bogon filters which corporations seem to like to fire and forget. Perhaps that's something you should outsource to your provider? Plus, you missed the part: If you don't have multihoming requirements other than availability. Bogon filters do _not_ enhance availability. (And please don't argue about things like attacks from unallocated space - the disconnectivity caused by un- updated bogon filters is much higher than the increase from things like this.) -- TTFN, patrick
RE: multi homing pressure
Or you can get automated bogon feeds from our good friends at cymru.. http://www.cymru.com/BGP/bogon-rs.html Peter Kranz [EMAIL PROTECTED] -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Patrick W. Gilmore Sent: Wednesday, October 19, 2005 12:49 PM To: nanog@merit.edu Cc: Patrick W. Gilmore Subject: Re: multi homing pressure On Oct 19, 2005, at 3:31 PM, Mark Radabaugh wrote: John Payne wrote: [...] If you don't have multihoming requirements other than availability then it really can be fire and forget. Except for those pesky bogon filters which corporations seem to like to fire and forget. Perhaps that's something you should outsource to your provider? Plus, you missed the part: If you don't have multihoming requirements other than availability. Bogon filters do _not_ enhance availability. (And please don't argue about things like attacks from unallocated space - the disconnectivity caused by un- updated bogon filters is much higher than the increase from things like this.) -- TTFN, patrick
Re: multi homing pressure
Well, not necessarily. Tier-2s should be given much more credit than they typically are in write-ups like this. When a customer is single homed to a tier-2 that has multiple tier-1 upstreams, and uses a delegated netblock from the tier-2's aggregations, that means one less ASN and one or more less routes in the global table. It's a Good Thing(tm). Not for the single-homed customer when the Tier-2 service is interrupted. Owen -- If it wasn't crypto-signed, it probably didn't come from me. pgpCsdSZNt0H5.pgp Description: PGP signature
Re: Verizon outage in Southern California?
5 9s can be measured all sorts of ways...
Network wide, it probably isn't even a blip. Even in terms of all of
California service its probably not much more than a blip.
Vicky Rode wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
I wonder what ever happened to redundancy? I guess 5 9s (dunno what the
going number is) got blown out of the water for them.
regards,
/virendra
David Lesher wrote:
Speaking on Deep Background, the Press Secretary whispered:
I'm not completely familiar with the telco jargon.
Does Tandem mean the same as a local central office, where
POTS lines terminate at the switch? Long Beach has a population
of 470,000. The C/Os I know of are:
A Central Office switch talks to subscribers aka end-users.
On its backside, it talks to other CO's and tandems. Time
was, that was also VF copper pairs, but it's long since all
DS1 and up.
A tandem is a switch that talks not to subs, but only to CO's. In
days of old, when a {dialup} call went to the other side of town,
chances are it went you-yourCO-downtown tandem-joesCO-joe. {copper
all the way...}.
A tandem was always housed in large CO building, but might have
been ATT's vice the operationg company, etc...
But ESS's and classless switching and massive expansion of the
plant really muddled the picture. An ESS could be both a CO switch
[for multiple prefixes and even multiple NPA's..] AND act like a
tandem.. And oh, the actual line cards can be remoted 100 miles
away in a horz. phonebooth box alongside the road in Smallville
with DS1's/OC coming back.
My guess is a DACS, a cross-connect point that is an software-driven
patch panel, lost its marbles. [engineering term of art.]
A DACS could have dozen-MANY dozen DS1/DS3/OC-n going hither
and yon. Some will be leased circuits. Others will be the CO trunks
going from one switch to another. It may/may not have muxes internal,
so that what arrives on a DS1 leaves in a OC96..
I note it went down at 2:20 AM. That SCREAMS software
upgrade/cutover. What's to bet GEE, no...VZEEE, was doing just
that and there was a major ohshit.
Sean noted a long while back that somehow, DACS crashes always
seem to take hours to recover. Maybe the backups are on Kansas
City standard tapes, I donno.. but this sounds like that..
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (MingW32)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFDVoJXpbZvCIJx1bcRAstJAJ0dnrQL1P2QJyxNU3r0T/X8g9fukQCgnm/N
yW5EvW7gI3gfjY7XSozyMds=
=ocNd
-END PGP SIGNATURE-
Re: multi homing pressure
That's the operators' view, but not the customer's. The customer wants redundancy. That's why SLAs exist. No... SLAs exist to extract some compensation when the level of service doesn't meet the need. In a mission critical situation, SLAs are pretty worthless. The primary benefit of an SLA is that it (hopefully) provides incentive to the provider to avoid screwing up the service. It doesn't directly do anything to directly improve the service or restore service from an outage. Many customers would rather not multihome directly, and prefer set it and forget it connectivity. It's much easier to maintain a multi-pipe connection that consists of one static default route than a pipe to multiple carriers. The former requires simple physical pipe management, which can be left alone for 99% of the time. The latter requires BGP feed, an ASN, and typically much more than 1% of an employee's time to keep running smoothly. I've done simple ASN/BGP based multihoming for a number of businesses, and, it can be done on a mostly set-and-forget basis. If you have your upstreams supply 0.0.0.0/0 via BGP and no other routes, and, you advertise your networks, believe it or not, that's a pretty stable configuration. If your upstreams are reasonably reliable, that works pretty well. If not, and, you care about knowing what your upstreams can't reach at the moment, then, you need a full feed and life becomes slightly more complicated. Obtaining single-homed connectivity from a Tier-2 mostly outsources network support, and small to medium size businesses tend to like that. It's not the only leaf end solution to the problem, but it's a viable one (and can be less costly to the rest of the world in turn). It's also not a complete solution to the problem. Sure, there are a class of customers whose needs that meets. However, because this meets some needs does not mean that it is legitimate to pretend that other needs do not exist. While we're at this, I'll debunk a few common myths: Myth: Renumbering pain is proportional to network size. Fact: Renumbering pain is proportional to the number of devices which need to be touched over which you do not have administrative control. A /16 which is entirely under my control and which is not present in ACL and other entries outside of my control is much easier to renumber than a /24 which contains a bunch of VPN terminators and serves 10,000s of customers who all have my addresses in their VPN boxes and ACLs on their firewalls. Myth: The need to multihome is proportional to the size of the organization. Fact: Some large organizations have only a few critical needs for the network and those needs are easily colocated in other facilities. For the rest of their use, being single-homed or multi-piped to a single provider is quite adequate. Some small organizations, OTOH, cannot function if their access to the network is impaired. For these organizations, multihoming is not only important, it's life and death. Myth: PI space is not needed in IPv6 because we fixed the need. Fact: PI space in IPv6 scares people because of the potential for unconstrained routing table growth. What is needed is to fundamentally change the way we do routing. IPv6 stopped well short of this goal, and, as such, provides little benefit beyond the original TUBA proposal, having decided that all of the real problems that needed to be solved were too hard. IPv6 does nothing to eliminate the need for PI space and ASNs at end sites that need to be truly multihomed. Shim6 is an attempt at providing some level of workaround to this deficiency, but, for full functionality, it requires significant implementation on all affected end-nodes and some of the routing infrastructure. For now, it's just a pipe dream. In the long-run, it's a very complex hack to replace what could be a relatively simple correction. Owen -- If it wasn't crypto-signed, it probably didn't come from me. pgp0EBw8MYSNS.pgp Description: PGP signature
Re: multi homing pressure
Always remember: For every customer, their stuff _is_ mission critical. So everyone will take the multihoming road if they can afford it. We can make it more expensive, or we can offer other solutions. Why should we do either? Why not fix the way we do routing so that it's OK for everyone to multihome? The fundamental problem is that IP addresses are serving more than one purpose, and, as a result, we have a bunch of unnecessary baggage on the routing system. Today, an IP address serves as an End System Identifier _AND_ as a Routing Location Identifier. This is sort of divided in the Network/Host portions, but, the problem is that it isn't really divided. The Host portion of the address is only part of the End System Identifier, and, the network portion is also necessary in order to uniquely identify that Host. There's the rub. Imagine instead, a world where Routing Location Identifiers are not coupled to End System Identifiers and Interdomain routing (AS-AS routing) occurred based on Routing Location Identifier, and only Intra-AS routing depended on the End System Identifier. For example: Host A connected to ISP X then ISP Y to ISP Z which provides service to Host B. Today, A, X, Y, Z all need to know how to reach B. If we separated the RLI from the ESI, then, the fact that B is reached via Z only has to be available information that can be looked up by A, and, X and Y only need to know how to get to Z. Only Z needs to know how to reach B. This allows the amount of data kept by each point along the way to be much smaller. We already have a separate RLI in IP, but, we fail to use it this way because we are missing: The way for A (or X) to look up the Host-RLI data. Routers and routing protocols that think in terms of RLI reachability instead of Host group (prefix) reachability. Todays RLI is called an ASN. Imagine if you could lookup the Origin-AS(es) for a given prefix through a query (similar to DNS, some protocol to be developed) and then, instead of sending to B, you send a packet addressed as: DST RLI: Z DST HOST: B SRC HOST: A Now, until it reaches ISP Z, nobody needs to look at anything but DST RLI Z to make a forwarding decision. Ideally, I think this should be implemented so that A sends to default and the first DFZ router does the lookup and DST RLI insertion, but, it could be done at the source host. I realize this requires code modification and protocol modification to make it work, but, doesn't this solve the routing table size problem and allow universal multihoming? A multihomed site that was not in the DFZ could simply return multiple RLI records and the inserting router could choose what it thought was the best one. Owen -- If it wasn't crypto-signed, it probably didn't come from me. pgp1zecGiCPvQ.pgp Description: PGP signature
Re: non-provider aggregation, was: IPv6 news
On 17-okt-2005, at 14:18, Jeroen Massar wrote: Another alternative is to force-align allocation and topology in some way /other/ than by Providers (geographical allocation in whatever hierarchy, IX allocation, whatever), such that networks were easily aggregatable. Lots of objections though (the providers and geography don't align one though is ultimately slightly bogus, because with non-provider-aligned allocation policies in place it would be in providers interests to align their peering to match the allocation policy). The current assumption is that all aggregation happens on ISP. Replacing that with the assumption that all aggregation will happen on geography isn't all that useful. The important thing here is that you can aggregate on pretty much anything: hair color, router vendor, market capitalization, you name it. In the end, you always aggregate on the way the addresses are given out, which may or may not be meaningful. Aggregating on provider is the most powerful because the aggregate leads you fairly directly to the place where you need to go as long as the destination is single homed. But suppose at some point we end up with a routing table consisting of 10 million PI blocks from multihomers and some unimportant stuff that disappears in the error margin (i.e., those 5000 IPv6 /20s for huge ISPs). Also suppose that it's possible to build a reasonably cost effective router that handles 1M routes, but this router technology doesn't scale to the next order of magnitude. The simple solution is to build a big router that actually consists of 11 small ones: 10 sub-routers that each hold one tenth of the global routing table, and an 11th sub-router that distributes packets to the sub-router that holds the right part of the global routing table. So sub-router 1 has the part of the global IPv6 routing table that falls within 2000::/6, sub-router 2 has 2400::/6, sub-router 3 2800::/6 and so on. So we're aggregating here, but not really on something. This has the unpleasant side effect that we now have to spend 11 times more money to keep a 10 times larger routing table. Alternatively, we can trade hardware costs for bandwidth, by having 10 routers that are present in the network anyway each handle part of the global routing table. So a router in Boston would handle everything under 2000::/6, a router in Chicago 2400::/6, one in Seattle 2800::/6 and so on. Obviously this isn't great if you're in Boston and your address is 2800::1, but it doesn't require additional hardware. This scheme can be optimized by aligning addressing and geography to a reasonable degree. So if you're in Boston, you'd get 2000::1 rather than 2800::1. But that doesn't magically shrink the routing table to one route per city. In the case of Boston, it's likely that the source and destination ISPs for a certain packet don't interconnect within the city itself. So someone sitting in New York probably won't see much difference: he or she still has to carry all the routes for multihomers in Boston. Some of these will point to her own customers in Boston, some to peers in New York, others to peers in DC, and so on. However, as distance increases the difference between this packet needs to go to a customer in Boston, this packet needs to go to a peer in New York and this packet needs to go to a peer in DC becomes meaningless, so it's possible to replace a large number of individual routes by a single city or region aggregate. So even without magic interconnection dust, aggregation based on geographical addressing can have benefits. However, it has several limitations. An important one is that early exit routing is replaced by late exit routing. Also, when someone multihomes by connecting to ISPs in Miami and Tokyo you don't get to aggregate. But worst case, you just don't get to aggregate, either because people multihome in weird ways, for traffic engineering reasons or because of lack of interconnection (however as interconnects become really sparse the savings go up again) so you're no worse off than today. But if and when the routing tables explode and routers can't keep up, having geographical addressing in place for multihoming allows for a plan B that we don't have today. I think we need a researcher to sit down and figure out exactly what this would look like in a sample city and a sample national provider. There has been quite some research on it, there where ideas, there was even talk of a vendor going to implement it, but it never happened. It won't work because of cash reasons (read: telco/transit don't want it) I'm not familiar with that... Do you have a reference? For your 'city data' check: http://unstats.un.org/unsd/demographic/default.htm or for pre-processed files: http://arneill-py.sacramento.ca.us/ipv6mh/ under Geographical data. Note that this page hasn't been updated in more than
Re: multi homing pressure
[EMAIL PROTECTED] (Jared Mauch) writes: it will be interesting to see if this has acutal impact on ASN allocation rates globally. i don't think so. multihoming without bgp isn't as hard as qualifying for PI space. i think we'll finally see enterprise-sized multihoming NAT/proxy products. -- Paul Vixie
Re: multi homing pressure
[EMAIL PROTECTED] (Todd Vierling) writes: The customer wants redundancy. That's why SLAs exist. no. sla's exist because actuarial tables and lawyers and accountants exist. -- Paul Vixie
Two things more important than NANOG....
Sorry for the interruption, but I couldn't let these two anniversaries pass without bringing them to your attention. http://fergdawg.blogspot.com/2005/10/in-memoriam-abha-ahuja.html [and] http://fergdawg.blogspot.com/2005/10/belatedly-october-16-1998-rip-jon.html I'll crawl back under my rock now... Cheers, - ferg -- Fergie, a.k.a. Paul Ferguson Engineering Architecture for the Internet [EMAIL PROTECTED] or [EMAIL PROTECTED] ferg's tech blog: http://fergdawg.blogspot.com/
[Be liberal in what you accept, and conservative in what you send.]
lest we forget... - Forwarded message - Bill, Could you forward to NANOG for me? Thanks, [snip] Jon, I'm sorry I forgot until just today... http://www.postel.org/postel.html - End forwarded message -
Re: non-provider aggregation, was: IPv6 news
Hi, On Wed, 19 Oct 2005, Iljitsch van Beijnum wrote: On 17-okt-2005, at 14:18, Jeroen Massar wrote: Another alternative is to force-align allocation and topology in some way /other/ than by Providers (geographical allocation in whatever hierarchy, IX allocation, whatever), such that networks were easily aggregatable. Lots of objections though (the providers and geography don't align one though is ultimately slightly bogus, because with non-provider-aligned allocation policies in place it would be in providers interests to align their peering to match the allocation policy). Iljitsch, fix your attributions, that's my text. (Jeroen might not appreciate you attributing my incoherent mumblings to him). The current assumption is that all aggregation happens on ISP. Replacing that with the assumption that all aggregation will happen on geography isn't all that useful. That's a bold assertion. You'll have to show why because the fact is that that is how other networks achieve portability (after which multi-homing is easy). Fact is I can change my fixed phone provider and my mobile phone provider, but I can't change my ISP without some pain (and I'm a /tiny/ site ;) ). The important thing here is that you can aggregate on pretty much anything: hair color, router vendor, market capitalization, you name it. Hmm, no ;). In the end, you always aggregate on the way the addresses are given out, which may or may not be meaningful. No, you have to aggregate on topology. Aggregating on provider is the most powerful because the aggregate leads you fairly directly to the place where you need to go as long as the destination is single homed. Sure. But it means you're tied to the provider (for that address at least). interconnect within the city itself. So someone sitting in New York probably won't see much difference: he or she still has to carry all the routes for multihomers in Boston. Some of these will point to her own customers in Boston, some to peers in New York, others to peers in DC, and so on. But at least, to the rest of the world, all the multihomers in Boston and New York have reduced down to just 2 routes. That's a significant step forward. (And eventually those ISPs back-hauling lots of very specific Boston customer prefixes to New York will figure out they should just peer in Boston and confine the very specific Boston routes there). limitations. An important one is that early exit routing is replaced by late exit routing. Can you expand on this? Also, when someone multihomes by connecting to ISPs in Miami and Tokyo you don't get to aggregate. Or, that entity just gets two prefixes, one for its Miami site allocated from the Miami area prefix and one for its Tokyo site allocated from the Tokyo area prefix. Really large networks with their own internal-transit across multiple areas for whom this would not work can just get a global prefix. But those kinds of networks are rare, a fraction of multi-homers. So it's still a step forward. really sparse the savings go up again) so you're no worse off than today. You're better off, because small/medium sites can be aggregated with all the other small/medium sites in their $AREA. The really large trans-$AREA networks are rare. Let's be honest, the reasons that make $AREA-allocated addresses and aggregation difficult are /not/ technical. ;) (Paul Jakma wrote something to the effect that I am involved with shim6 so that says something about other options. It doesn't, as far as I'm concerned. But shim6 is a worthy pursuit in its own right.) I said possibly is telling ;). But apologies for any presumption ;). regards, -- Paul Jakma [EMAIL PROTECTED] [EMAIL PROTECTED] Key ID: 64A2FF6A Fortune: fortune: not found
origin as numbers to ignore in an analysis
if one is looking at origin-as in routing annoucements in route views, there are some asns that should be ignored, e.g., . is there a good list of these somewhere. randy
Re: multi homing pressure
On Wed, Oct 19, 2005 at 10:19:28PM +, Paul Vixie scribed: [EMAIL PROTECTED] (Jared Mauch) writes: it will be interesting to see if this has acutal impact on ASN allocation rates globally. i don't think so. multihoming without bgp isn't as hard as qualifying for PI space. i think we'll finally see enterprise-sized multihoming NAT/proxy products. If you can run Squid, you can multihome your web connections today. It's a little bit awkward to configure, but then again, so is Squid. People are welcome to poke at, fold, spindle, or mutilate: http://nms.lcs.mit.edu/ron/ronweb/#code (Part of my thesis work, Monet is a modification to Squid that causes it to try to open N TCP connections to a Web server that it wants to talk to. It uses the first SYN ACK to return, and closes the other connections to be a nice neighbor. It's shockingly effective at improving availability to Web sites that are themselves multihomed or otherwise good. Warning: Often still leads to annoyance if you find yourself able to browse the web but not do anything else. We do have a NAT version of this that works with arbitrary protocols. If people are interested, I'll try to convince my former student to dig up the code and make it a bit prettier.) -Dave
Re: multi homing pressure
On Wed, 19 Oct 2005, David G. Andersen wrote: If you can run Squid, you can multihome your web connections today. It's a little bit awkward to configure, but then again, so is Squid. People are welcome to poke at, fold, spindle, or mutilate: http://nms.lcs.mit.edu/ron/ronweb/#code (Part of my thesis work, Hehe, google for vixie ifdefault. regards, -- Paul Jakma [EMAIL PROTECTED] [EMAIL PROTECTED] Key ID: 64A2FF6A Fortune: Why do they call it baby-SITTING when all you do is run after them?
Re: multi homing pressure
On Thu, Oct 20, 2005 at 03:18:35AM +0100, Paul Jakma scribed: On Wed, 19 Oct 2005, David G. Andersen wrote: If you can run Squid, you can multihome your web connections today. It's a little bit awkward to configure, but then again, so is Squid. People are welcome to poke at, fold, spindle, or mutilate: http://nms.lcs.mit.edu/ron/ronweb/#code (Part of my thesis work, Hehe, google for vixie ifdefault. Right. Vix was talking about the inbound path - I'm talking about the outbound path. Complimentary solutions to the same problem. -Dave
Re: Scalability issues in the Internet routing system
PS: Btw, anyone can give me a hint on where to discuss new ideas for e.g. routing schemes (and finding out whether it's an old idea)? You might start with the routing-discussion mailing list: http://www.rtg.ietf.org/ Please expect that your idea has been discussed before. We're an old bunch. ;-) Tony
Re: Scalability issues in the Internet routing system
Daniel, I think it is safe, even with projected AS and IP uptake, to assume Moore's law can cope with this. Moore will keep up reasonably with both the CPU needed to keep BGP perking, and with memory requirements for the RIB, as well as other non-data-path functions of routers. That's only true if the rate of prefix addition can be constrained to be below the rate dictated by Moore's law, which is the entire point of the discussion. There is nothing today that acts as a pressure against bloat except portions of the net periodically blowing up as their hardware capacity is exceeded. Several items regarding FIB lookup: 1) The design of the FIB need not be the same as the RIB. There is plenty of room for creativity in router design in this space. Specifically, the FIB could be dramatically reduced in size via aggregation. The number of egress points (real or virtual) and/or policies within a router is likely FAR smaller than the total number of routes. It's unclear if any significant effort has been put into this. In fact, there has been. In a previous life, we actually had some FIB pre-processing that did a great deal of aggregation prior to pushing the FIB down into hardware. We found that it was workable, but consumed extensive CPU resources to keep up with the churn. Thus, this turns out to be a tool to push the problem from the FIB back up to the CPU. Previous comments still apply, and this just increases the CPU burn rate. 2) Nothing says the design of the FIB lookup hardware has to be longest match. Other designs are quite possible. Longest match is fundamental in the workings of all of the classless protocols today. Changing this means changing almost every protocol. 3) Don't discount novel uses of commodity components. There are fast CPU chips available today that may be appropriate to embed on line cards with a bit of firmware, and may be a lot more cost effective and sufficiently fast compared to custom ASICs of a few years ago. The definition of what's hardware and what's software on line cards need not be entirely defined by whether the design is executed entirely by a hardware engineer or a software engineer. This has been tried as well. Tony
Re: multi homing pressure
On Wed, 19 Oct 2005, Owen DeLong wrote: I've done simple ASN/BGP based multihoming for a number of businesses, and, it can be done on a mostly set-and-forget basis. If you have your upstreams supply 0.0.0.0/0 via BGP and no other routes, and, you advertise your networks, believe it or not, that's a pretty stable configuration. If your upstreams are reasonably reliable, that works pretty well. If not, and, you care about knowing what your upstreams can't reach at the moment, then, you need a full feed and life becomes slightly more complicated. There's really nothing more complicated about taking 2 (or more) full views, other than keeping an eye on available memory. The CW/PSI incident a few years ago and the more recent Cogent/Level3 incident are perfect examples of why taking two 0/0's really doesn't cut it if you want reliable connectivity to the whole internet. Cisco burned a lot people by building routers with needlessly limited RAM capacities (planned obsolescence?). Because of that, one customer wouldn't buy another cisco, and instead went Imagestream. They have 3 full views and no worries now. They were so happy with that Imagestream, they ended up buying a bunch more for internal WAN needs. Another customer I dealt with recently was fairly typical of the small multihomer I'd guess. They were multihomed to two Tier1 providers and wanted to replace one of them with us. Their BGP had been done either by a consultant or former employee and was definitely set and forgot on autopilot. Their router (cisco 3640) kept dying and they'd just power cycle it as needed. When I got in to take a look, I found it was taking full views and had pretty much no RAM left...and it was announcing all their space deaggregated as /24s for no reason. They weren't willing to shell out the $ for a bigger router, so I ended up configuring them for full routes from us and customer routes from their other (a Tier1) provider (and fixing their advertisements). Other than expansion (more network statements), running out of RAM again, or changing providers, I doubt their BGP config will need to be touched in the forseeable future. -- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net| _ http://www.lewis.org/~jlewis/pgp for PGP public key_
Re: IPv6 daydreams
--- David Conrad [EMAIL PROTECTED] wrote: On Oct 17, 2005, at 10:39 PM, Paul Jakma wrote: Wrong issue. What I'm unhappy about is not the size of the address - you'll notice that I didn't say make the whole address space smaller. What I'm unhappy about is the exceedingly sparse allocation policies You can allocate to 100% density on the network identifier if you want, right down to /64. I believe the complaint isn't about what _can be_ done, rather what _is being_ done. Yes and yes. I am certainly complaining about what *is* being done. See below for my bigger issue. The host identifier simply is indivisible, and just happens to be 64bit. I've always wondered why they made a single address field if the IPv6 architects really wanted a hard separation between the host identifier and the network identifer. Making the address a contiguous set of bits seems to imply that the components of the address can be variable length. Now we're cooking with gas: what we've learned from MAC addresses is that it's really nice to have a world-unique address which only has local significance. The /64 host identifier is a misnomer: there are folks who use /127s and /126s for point-to-point links, and there are all sorts of variable length masks in use today. The whole reason for a /64 to be associated with a host is to have enough room to encode MAC addresses. I ask again - why exactly do we want to do this? Layer-2 works just fine as a locally-significant host identifier, and keeping that out of layer-3 keeps everything considerably simpler. -David Barak- -Fully RFC 1925 Compliant- __ Yahoo! Music Unlimited Access over 1 million songs. Try it free. http://music.yahoo.com/unlimited/
Re: multi homing pressure
--On October 19, 2005 11:17:02 PM -0400 Jon Lewis [EMAIL PROTECTED] wrote: On Wed, 19 Oct 2005, Owen DeLong wrote: I've done simple ASN/BGP based multihoming for a number of businesses, and, it can be done on a mostly set-and-forget basis. If you have your upstreams supply 0.0.0.0/0 via BGP and no other routes, and, you advertise your networks, believe it or not, that's a pretty stable configuration. If your upstreams are reasonably reliable, that works pretty well. If not, and, you care about knowing what your upstreams can't reach at the moment, then, you need a full feed and life becomes slightly more complicated. There's really nothing more complicated about taking 2 (or more) full views, other than keeping an eye on available memory. The CW/PSI incident a few years ago and the more recent Cogent/Level3 incident are perfect examples of why taking two 0/0's really doesn't cut it if you want reliable connectivity to the whole internet. Yes and no. Most people that will spend the $$ for routers with enough memory to handle multiple full feeds are also looking to get a certain amount of TE capability out of the deal, and, at that point, babysitting the TE becomes more than 0.01 FTE (closer to 0.30 in my experience). Cisco burned a lot people by building routers with needlessly limited RAM capacities (planned obsolescence?). Because of that, one customer wouldn't buy another cisco, and instead went Imagestream. They have 3 full views and no worries now. They were so happy with that Imagestream, they ended up buying a bunch more for internal WAN needs. That's an interesting way to look at it. I think that at the time those routers were designed (I'm assuming you are talking AGS+ here), there was no concept of why anyone would ever need that much memory, and, designing a board to accommodate it would have seriously increased the size and price of the router. If you're talking about more recent, then, it's a marketing decision to not facilitate full tables on low-end routers lest they start eating into their high-end router business. Another customer I dealt with recently was fairly typical of the small multihomer I'd guess. They were multihomed to two Tier1 providers and wanted to replace one of them with us. Their BGP had been done either by a consultant or former employee and was definitely set and forgot on autopilot. Their router (cisco 3640) kept dying and they'd just power Lol... Yep, that happens. cycle it as needed. When I got in to take a look, I found it was taking full views and had pretty much no RAM left...and it was announcing all their space deaggregated as /24s for no reason. They weren't willing to shell out the $ for a bigger router, so I ended up configuring them for full routes from us and customer routes from their other (a Tier1) provider (and fixing their advertisements). Other than expansion (more network statements), running out of RAM again, or changing providers, I doubt their BGP config will need to be touched in the forseeable future. That could be true, but, how long do you really think the RAM will last? Owen pgp2aiBtPjkh7.pgp Description: PGP signature
Re: multi homing pressure
On Wed, 19 Oct 2005, Owen DeLong wrote: Yes and no. Most people that will spend the $$ for routers with enough memory to handle multiple full feeds are also looking to get a certain amount of TE capability out of the deal, and, at that point, babysitting the TE becomes more than 0.01 FTE (closer to 0.30 in my experience). Some may. The one I'm talking about with the Imagestreams really doesn't. They've overprovisioned the heck out of their network after the CW/PSI thing and really have no need for TE. In fact, no attempt at all has been made to influence their traffic. Just a simple let BGP take care of it config. That's an interesting way to look at it. I think that at the time those routers were designed (I'm assuming you are talking AGS+ here), there I wasn't thinking that far back. I'm talking about the 3640 and 2610/2611/2620/2621s. For many end users, these routers would be just fine for multihoming with a few T1s, if they had the RAM capacity for several full views. At the time the above customer was multihoming, their only real options with cisco were the 3660 and 7200 series, which were overkill (and overpriced if you want new gear from say Tech Data). cisco finally has come out with replacements for those little routers with much bigger RAM capacities...but they're a little late. That could be true, but, how long do you really think the RAM will last? I suppose it won't. I just checked up on them. Seems they must have canceled their other provider (I hope so anyway...its been down at least a week), and with just 1 full view from us, they have 2.3mb free. I guess its time to get them on the phone and see about either shutting off BGP or just sending them 0/0. Another 3640 bites the dust. -- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net| _ http://www.lewis.org/~jlewis/pgp for PGP public key_
/24 multihoming issue
I'm having trouble announcing a single /24 from an ASN. ASN is multi-homed to ISP-A and ISP-B, prepending on ISP-B side. ASN in question has one and only one /24 which originally was from ISP-B /17 block. Some ISP only sees path from ISP-A and some from ISP-B and very few sees both paths. Apparently, when we are testing failover, it failed. (can't get to most of the internet, can't VPN in from outside, can't send mails etc BGP paht/route disappear from some of looking glasses) After the test, I registered /24 and ASN with RADB and things get slightly better, meaning a few more ISP sees both but majority of them still seeing single ISP path. I've contacted both ISPs and they both claimed they are announcing our /24 to the rest of the world, without manipulation. What am I missing here? Thanks, - Kyaw __ Yahoo! Mail - PC Magazine Editors' Choice 2005 http://mail.yahoo.com
Re: /24 multihoming issue
On Wed, 19 Oct 2005, Kyaw Khine wrote: I've contacted both ISPs and they both claimed they are announcing our /24 to the rest of the world, without manipulation. What am I missing here? some providers (for whatever reason, not really relevant to this conversation) do filter at boundaries NOT /24 :( Some filter /20 or /22 or other odd-ball boundaries. -Chris
Re: /24 multihoming issue
try a peek at route views and, if you want help debugging, folk will want to know the prefix and the asn randy
Re: /24 multihoming issue
routeviews is seeing both paths. 64.9.17.0/24 AS 33105 ISP-A = 701 :) ISP-B = 19094 Thanks ... --- Randy Bush [EMAIL PROTECTED] wrote: try a peek at route views and, if you want help debugging, folk will want to know the prefix and the asn randy __ Yahoo! Mail - PC Magazine Editors' Choice 2005 http://mail.yahoo.com
Re: /24 multihoming issue
On Wed, 19 Oct 2005, Kyaw Khine wrote: routeviews is seeing both paths. 64.9.17.0/24 AS 33105 ISP-A = 701 :) ISP-B = 19094 You might talk to 701 about why for instance, all I see is your prepended path via 19094 through 3356, 6461, 4323, and 19962. Maybe 701 is only propogating your route to customers? -- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net| _ http://www.lewis.org/~jlewis/pgp for PGP public key_
Re: /24 multihoming issue
I've heard and seen those filters a few years ago. In this particular case, I've seen a bunch of other /24s (from remote ASs) on the looking glasses. Looks like filters are base on other criteria on top of prefix length and I wonder which criterion this /24 falls into. --- Christopher L. Morrow [EMAIL PROTECTED] wrote: On Wed, 19 Oct 2005, Kyaw Khine wrote: I've contacted both ISPs and they both claimed they are announcing our /24 to the rest of the world, without manipulation. What am I missing here? some providers (for whatever reason, not really relevant to this conversation) do filter at boundaries NOT /24 :( Some filter /20 or /22 or other odd-ball boundaries. -Chris __ Start your day with Yahoo! - Make it your home page! http://www.yahoo.com/r/hs
Re: /24 multihoming issue
On Thu, 20 Oct 2005, Jon Lewis wrote: On Wed, 19 Oct 2005, Kyaw Khine wrote: routeviews is seeing both paths. 64.9.17.0/24 AS 33105 ISP-A = 701 :) ISP-B = 19094 You might talk to 701 about why for instance, all I see is your prepended path via 19094 through 3356, 6461, 4323, and 19962. Maybe 701 is only propogating your route to customers? shouldn't be the case, it's not looking like it's tagged anything 'special'. :) the other folks might see telecove as a better path (assuming telecove/19094 is also multihomed to these other asn's you have above)
Re: /24 multihoming issue
There are a few ISP they are seeing path from 701. But very few compare to prepended path via 19094. e.g route-server.colt.net (2914 701 33105) route-views.bmcag.net (1239 701 33105) --- Jon Lewis [EMAIL PROTECTED] wrote: On Wed, 19 Oct 2005, Kyaw Khine wrote: routeviews is seeing both paths. 64.9.17.0/24 AS 33105 ISP-A = 701 :) ISP-B = 19094 You might talk to 701 about why for instance, all I see is your prepended path via 19094 through 3356, 6461, 4323, and 19962. Maybe 701 is only propogating your route to customers? -- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net| _ http://www.lewis.org/~jlewis/pgp for PGP public key_ __ Yahoo! Music Unlimited Access over 1 million songs. Try it free. http://music.yahoo.com/unlimited/
