Re: And Now for Something Completely Different (was Re: IPv6 news)
Randy; we are living on Earth with small size (only 6,000 km in radius), so we will never see unlimited grouth in multihomed networks. It is not a problem. We are not building Internet for the whole universe. Good old Moore can deal with our planet very well. I repeated many times - IPv6 idea of changing multihome approach is VERY BAD and will not survise for more that 1 - 2 years. (if IPv6 survive at all, which I have many doubts about). - Original Message - From: Randy Bush [EMAIL PROTECTED] To: Daniel Golding [EMAIL PROTECTED] Cc: Tony Li [EMAIL PROTECTED]; Fred Baker [EMAIL PROTECTED]; Per Heldal [EMAIL PROTECTED]; nanog@merit.edu Sent: Monday, October 17, 2005 2:16 PM Subject: Re: And Now for Something Completely Different (was Re: IPv6 news) There is a fundamental difference between a one-time reduction in the table and a fundamental dissipation of the forces that cause it to bloat in the first place. Simply reducing the table as a one-off only buys you linearly more time. Eliminating the drivers for bloat buys you technology generations. If we're going to put the world thru the pain of change, it seems that we should do our best to ensure that it never, ever has to happen again. That's the goal here? To ensure we'll never have another protocol transition? I hope you realize what a flawed statement that is. tony probably did not think about it because that's not what he said at all. he was speaking of routing table bloat, not transitions. and he was spot on. randy
Re: And Now for Something Completely Different (was Re: IPv6 news)
Obviously if the RIRs contacted the folks responsible for a given block and were provided justification for its continued allocation, then it should not be reclaimed. On the other hand, folks sitting on several class Bs and not using them could have their blocks reclaimed trivially; ditto for companies that no longer exist. The last is certainly doable without much risk of controversy. This is exactly what the Internic did many years ago. I, like many other people, had registered a .com domain name at no cost. Then suddenly one day, the Internic said that I had to pay an annual subscription fee for this domain name. Like many others, I paid my fees. There were a few complaints about this but by and large people accepted the idea that you had to MAINTAIN a business relationship with the domain name registry in order to continue using a domain name. For some reason, this concept of MAINTAINING a business relationship with the registry, has not caught on in the Regional IP Registry arena. Of course, a large number of IP address users do pay annual membership subscription fees to ARIN (or other RIRs) but not all. And the RIRs seem unwilling to withdraw services (in-addr.arpa) from those who do not MAINTAIN a business relationship. However, one of the articles referred to recently in this thread (I forget which) showed that even if we reclaimed all of the address space that is currently unannounced (in use or not), we'd buy ourselves a trivially short extension of the IPv4 address space exhaustion date. Considering the cost of performing the task, doing so seems rather pointless; our time would be better spent getting IPv6 deployed and either reengineering the routing system or switching to geo addresses. Probably this article from the Cisco IP Journal which has been mentioned a few times in the past week. http://www.cisco.com/en/US/about/ac123/ac147/archived_issues/ipj_8-3/ipv4.html From the viewpoint of avoiding an addressing crisis and avoiding a v6 transition crisis, your advice is sound. However, from the viewpoint of having a sensibly functioning RIR system, I think we still need to deal with two issues. One is that the holders of IP address allocations should be required to maintain a business relationship with the RIR or lose the right to use those IP addresses. The other is that the RIRs need to fix the whole debacle that is whois and routing registries. There should be no need for 3rd party bogon lists. The RIR's should publish an authoritative registry, rooted in IANA, that covers the entire IPv4 and IPv6 address spaces. An operator faced with receiving a new BGP announcement should be able to query such a registry and find out whether or not the address block is allocated, who it is allocated to, and whether that party intends to announce that exact block size from that exact AS number. --Michael Dillon
Re: And Now for Something Completely Different (was Re: IPv6 news)
On Tue, 2005-10-18 at 15:52 -0700, David Conrad wrote: Hmm. Are the aliens who took the _real_ IETF and replaced it with what's there now going to give it back? :-) Sure they'll hand it back ... when there is no more money to be made from IETF-related technology and politicians no longer feel it's interesting ;) Otoh, the IETF is a function of its partitipants. Businesses today have such fear of competitors and interllectual-propery-issues that they hardly can cooperate on anything. Thus, the number of technology reasearchers available to partitipate in public forums is just a fraction of what it was 20 or 30 years ago. If enough people find IETF unworkable, wouldn't they just form an alternative forum that would be to the IETF what the IETF has been to ITU? //Per
Re: And Now for Something Completely Different (was Re: IPv6 news)
True enough, but unfortunately, it's not done in a way that we can make use of the identifier in the routing subsystem or in the transport protocols. The transport protocols, well they generally act on behalf of something which can do the lookup and supply transport with right address, as long the DNS server does not require who-where indirection ;). The transport protocols unfortunately need the identifier in the packet to demux connections. Tony
Re: And Now for Something Completely Different (was Re: IPv6 news)
check out The Landmark Hierarchy: A New Hierarchy for Routing in Very Large Networks; Paul Tsuchiya; 1989. great stuff... i have a hardcopy. is it online yet? Just google for landmark routing and you will find lots of papers and presentations that deal with the topic. If OSPF area boundaries were more fluid, rather like the period covered by a moving average... Of course, this might not be so nice if it was done across AS boundaries. --Michael Dillon
Re: And Now for Something Completely Different (was Re: IPv6 news)
# True enough, but unfortunately, it's not done in a way that we can make # use of the identifier in the routing subsystem or in the transport # protocols. # # The transport protocols, well they generally act on behalf of something # which can do the lookup and supply transport with right address, as long # the DNS server does not require who-where indirection ;). # # The transport protocols unfortunately need the identifier in the packet to # demux connections. the idea of a transport protocol comes from the OSI Reference Model which might not be the best conceptual fabric for re-thinking Internet routing. we know it's a distributed system and we know that various waypoints will or will not have state, but i don't think we know that there will always be a layer that does what the transport protocol does in the OSIRM. i mention this because padlipsky's mantra about maps and territories came into my head just now as i was listening to folks talk about what the transport protocol had to have or had to provide. there's only a transport protocol if we decide to keep thinking in ISORM terms. and with that, i do indeed wonder if this has stopped being operational and if so, whether nanog wants to overlap THIS much with the irtf? refs: http://www.amazon.com/exec/obidos/tg/detail/-/0132681110/103-3252601-1266225
Re: And Now for Something Completely Different (was Re: IPv6 news)
this because padlipsky's mantra about maps and territories came into my head S.I. Hayakawa - Language and Thought in Action The symbol is not the thing the thing symbolized; The map is not the territory: The word is not the thing. Nevertheless, Padlipsky is a good thing to read. Here is the book review from the Cisco IP Journal with a taste of the book. http://www.cisco.com/en/US/about/ac123/ac147/ac174/ac179/about_cisco_ipj_archive_article09186a00800e55d2.html --Michael Dillon
Re: And Now for Something Completely Different (was Re: IPv6 news)
Tony, On Oct 16, 2005, at 1:15 AM, Tony Li wrote: A real locator/identifier separation requires a rewrite. Not necessarily. If you transition at the edge, what happens within the site matters only to the site and what matters to the core only matters to the core. No stacks, either core or edge, need to be rewritten. Any system that provided site-wide source address control was going to require a rewrite. Not necessarily (depending on what you mean by the ambiguous term address). A lot depends on the actual requirements for source locator or identifier control. David, I should point out that if only a small number of folks care about multihoming, then only a small number of folks need to change their stacks. I thought all clients would have to be modified if they wanted to take full advantage of a shim6 enabled multi-homed server? And even in your solution, there would need to be some changes to the end host if you want to support exit point selection, or carry alternate locators in the transport. One of the problems that I have seen in the IETF is calling desires requirements. How important is exit point selection? Are there ways of implementing exit point selection without changing the IP stack? How critical is it that alternate locators be carried in the transport? Does the lack of that functionality make the protocol unusable? What _are_ the actual requirements (not the Goals)? From my perspective, the really, really critical flaw in both IPv4 and IPv6 is the lack of _transparent_ renumberability. Multi-homing is also a flaw, but less critical and I believe it can be addressed with the right solution to renumberability. A few folks worry about multi- homing. Everybody worries about end site renumbering. It's just a mess. I think that we all can agree that a real locator/identifier split is the correct architectural direction, but that's simply not politically tractable. Right. And since it couldn't be done the right way in the protocol, we make the protocol much more complicated and force a reset to address functionality that relatively few folks need. I'm suggesting not mucking with the packet format anymore. It might be ugly, but it can be made to work until somebody comes up with IPv7. Instead, since the locator/identifier split wasn't done in the protocol, do the split in _operation_. Make the edge/core boundary real. Both edge and core could be addressed without hierarchy, but the mapping between the edge and core would change such that the edge would never be seen in the DFZ. Within the core, nothing new or different need be done (well, except for deploying IPv6 and running the core/edge translators). Within the edge, nothing new need be done either. Yes, it is a hack. But I suspect it would address the real requirements (or, at least my pet requirement :-)). Rgds, -drc
Re: And Now for Something Completely Different (was Re: IPv6 news)
[EMAIL PROTECTED] (David Conrad) wrote: I'm suggesting not mucking with the packet format anymore. It might be ugly, but it can be made to work until somebody comes up with IPv7. Instead, since the locator/identifier split wasn't done in the protocol, do the split in _operation_. It has been done a long time ago, IMHO. I wonder whether I am the only one seeing this, but we already have a (albeit routing-) locator (ASN) and an identifier (IP address), that are pretty much distinct and where the routing locator is not used inside the local network, but only outside. There's your edge/core boundary. Every multi-homer will be needing their own ASN, so that's what clutters up your routing tables. It's economy there. Btw, a lot of ASNs advertise one network only. People surely think multihoming is important to them (and I cannot blame them for that). Hierarchical routing is one possible solution, with a lot of drawbacks and problems. Forget about geographic hierarchies; there's always people who do not peer. Visibility radius limitation is another (I cannot believe the idea is new, I only don't know what it's called). Cheers, Elmi. -- Begehe nur nicht den Fehler, Meinung durch Sachverstand zu substituieren. (PLemken, [EMAIL PROTECTED]) --[ ELMI-RIPE ]---
RE: And Now for Something Completely Different (was Re: IPv6 news)
Nanog, I've been thinking a bunch about this IPv6 multihoming issue. It seems that the method of hierarchical summarization will keep the global tables small for all single-homed end user blocks. But the multihomed ones will be the problem. The possible solution I've been thinking about is 'adjacency blocks', for lack of a better term. In theory, the first customer to home to two different ISPs causes a new large address block to be advertised upstream by these two ISPs. Further customers homing to these two ISPs get an allocation out of this same block. The two ISPs will only announce the large block. Of course there are issues involving failure and scalability... Failure would involve an ISP losing contact with end customer, but still announcing the aggregate upstream. This can be solved by requiring that two ISPs must have a direct peering agreement, before they can accept dual-homed customers. Or a possible method (maybe using communities?) where ISP B will announce the customer's actual block (the small one) to it's upstreams, if notified by ISP A that it's not reachable by them. When ISP A resumes contact with end customer, ISP B retracts the smaller prefix. Scalability is an obvious issue, as the possible number of these 'adjacency blocks' would be N * (N-1), where N is the number of ISPs in the world. Obviously pretty large. But I feel the number of ISPs that people would actually dual home to (due to reputation, regional existence, etc) is a few orders of magnitude smaller. ~100,000 prefixes (each can be an ASN, I suppose) should cover all needs, doing some simple math. The downside is that end customers are going to lose the ability to prefer traffic from one ISP versus another for inbound traffic. That alone might be a show-stopper, not sure how important it is. Since IPv6 is a whole new ballgame, maybe it's ok to change the rules... Looking for any thoughts about it. I'm sure there's things I haven't considered, but the people I've bounced it off of haven't seen any obvious problems. Flame-retardant clothes on, just in case though. Chuck Every multi-homer will be needing their own ASN, so that's what clutters up your routing tables. It's economy there. Btw, a lot of ASNs advertise one network only. People surely think multihoming is important to them (and I cannot blame them for that). Hierarchical routing is one possible solution, with a lot of drawbacks and problems. Forget about geographic hierarchies; there's always people who do not peer. Visibility radius limitation is another (I cannot believe the idea is new, I only don't know what it's called).
Re: And Now for Something Completely Different (was Re: IPv6 news)
the principal issue I see with your proposal is that it is DUAL homing vs MULTI homing. To make it viable, I think you have to say something like two or more ISPs must participate in a multilateral peering arrangement that shares the address pool among them. The location of the actual peering is immaterial - doing one for Santa Barbara County in California might actually mean peering at One Wilshire Way in LA, for example. However, the peering arrangement would have to be open to the ISPs that serve the community; otherwise, it would be exposed to anti-trust litigation (if Cox and Verizon, the Cable Modem and DSL providers in Santa Barbara, did this, but it was not open to smaller ISPs in the community, the latter might complain that it had the effect of locking out competition). But yes, communities of a rational size and density could get an address block, the relevant ISPs could all advertise it into the backbone, and the ISPs could determine among themselves how to deliver traffic to the homes, which I should expect would mean that they would deliver directly if they could and otherwise hand off to another ISP, and that handoff would require an appropriate routing exchange. Can you say lots of long prefixes within a limited domain? They would want to configure the home's address block on its interior interface and route to it through their own networks. Note that NAT breaks this... this requires end to end connectivity. I should expect that they would not literally expect the homes to run BGP (heaven forfend); I could imagine the last mile being the last bastion of RIP - the home sends a IP update upstream for its interior address, and the ISP sends a default route plus routes to its own data centers down. The biggest issue here might be the effect on cost models in routing. Today, hot potato routing makes a some relationships relatively cheap while other relationships are more expensive; this reverses those. Today, if a datagram is handed to me that will be delivered in your network, I hand it to you at my earliest opportunity and you deliver it. In this model, I can't tell who will deliver it until I get relatively close to the community. Hence, I will carry the packet to that exchange point, and only then hand it to you. Funny. I described this in an internet draft nearly a decade ago, and got dumped on because of this issue - something about living in an ivory tower and playing with people's livelihoods, as I recall. I'll see if I can resurrect that, if you like. On Oct 18, 2005, at 10:40 AM, Church, Chuck wrote: Nanog, I've been thinking a bunch about this IPv6 multihoming issue. It seems that the method of hierarchical summarization will keep the global tables small for all single-homed end user blocks. But the multihomed ones will be the problem. The possible solution I've been thinking about is 'adjacency blocks', for lack of a better term. In theory, the first customer to home to two different ISPs causes a new large address block to be advertised upstream by these two ISPs. Further customers homing to these two ISPs get an allocation out of this same block. The two ISPs will only announce the large block. Of course there are issues involving failure and scalability... Failure would involve an ISP losing contact with end customer, but still announcing the aggregate upstream. This can be solved by requiring that two ISPs must have a direct peering agreement, before they can accept dual-homed customers. Or a possible method (maybe using communities?) where ISP B will announce the customer's actual block (the small one) to it's upstreams, if notified by ISP A that it's not reachable by them. When ISP A resumes contact with end customer, ISP B retracts the smaller prefix. Scalability is an obvious issue, as the possible number of these 'adjacency blocks' would be N * (N-1), where N is the number of ISPs in the world. Obviously pretty large. But I feel the number of ISPs that people would actually dual home to (due to reputation, regional existence, etc) is a few orders of magnitude smaller. ~100,000 prefixes (each can be an ASN, I suppose) should cover all needs, doing some simple math. The downside is that end customers are going to lose the ability to prefer traffic from one ISP versus another for inbound traffic. That alone might be a show-stopper, not sure how important it is. Since IPv6 is a whole new ballgame, maybe it's ok to change the rules... Looking for any thoughts about it. I'm sure there's things I haven't considered, but the people I've bounced it off of haven't seen any obvious problems. Flame-retardant clothes on, just in case though. Chuck Every multi-homer will be needing their own ASN, so that's what clutters up your routing tables. It's economy there. Btw, a lot of ASNs advertise one network
Re: And Now for Something Completely Different (was Re: IPv6 news)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Yo Fred! On Tue, 18 Oct 2005, Fred Baker wrote: But yes, communities of a rational size and density could get an address block, the relevant ISPs could all advertise it into the backbone, and the ISPs could determine among themselves how to deliver traffic to the homes, That assumes they can agree on how to get traffic to/from the world and the local IX. One of our local ISPs goes the cheap way and uses an overloaded (and therefore cost effective) link to a cheap tier 2. Another pays a premium price to have a lightly loaded link for it's customers. They will never agree on their business model, not should they have to. By forcing local ISPs to use the same routing prefix you force them to share the same routing strategy to the outside world. For semi-isolated communities this is a big issue. RGDS GARY - --- Gary E. Miller Rellim 20340 Empire Blvd, Suite E-3, Bend, OR 97701 [EMAIL PROTECTED] Tel:+1(541)382-8588 Fax: +1(541)382-8676 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFDVV4g8KZibdeR3qURAuhjAKCuvsd/ZmXebyyTNkfdQ3tBbQvdmACg1OnL RE0lRoxSElVzNaZFpdYcObA= =b5O1 -END PGP SIGNATURE-
Re: And Now for Something Completely Different (was Re: IPv6 news)
Thus spake [EMAIL PROTECTED] E.g prevously announced address-blocks that has disappeared from the global routing-table for more than X months should go back to the RIR-pool (X=6). In RFC 2050 section 3 a) the organization has no intention of connecting to the Internet-either now or in the future-but it still requires a globally unique IP address. The organization should consider using reserved addresses from RFC1918. If it is determined this is not possible, they can be issued unique (if not Internet routable) IP addresses. Seems to me that the Internet routing table contents past and present are irrelevant. Note also that the so-called Internet routing table contents vary depending on where you look at it. Obviously if the RIRs contacted the folks responsible for a given block and were provided justification for its continued allocation, then it should not be reclaimed. On the other hand, folks sitting on several class Bs and not using them could have their blocks reclaimed trivially; ditto for companies that no longer exist. The last is certainly doable without much risk of controversy. However, one of the articles referred to recently in this thread (I forget which) showed that even if we reclaimed all of the address space that is currently unannounced (in use or not), we'd buy ourselves a trivially short extension of the IPv4 address space exhaustion date. Considering the cost of performing the task, doing so seems rather pointless; our time would be better spent getting IPv6 deployed and either reengineering the routing system or switching to geo addresses. S Stephen SprunkStupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them. --Aaron Sorkin
Re: And Now for Something Completely Different (was Re: IPv6 news)
David, A real locator/identifier separation requires a rewrite. Not necessarily. If you transition at the edge, what happens within the site matters only to the site and what matters to the core only matters to the core. No stacks, either core or edge, need to be rewritten. Transitioning at the edge implies to me that the hosts need to know about different semantics for the IPv6 header. That, in turn, implies that there is different code for the hosts. Alternately, if there is no new code anywhere, it's clear that you must necessarily have the same semantics and must not have made a change. Any system that provided site-wide source address control was going to require a rewrite. Not necessarily (depending on what you mean by the ambiguous term address). A lot depends on the actual requirements for source locator or identifier control. Again, source address selection is going to require something different than what we have today. The host might have to interact with some centralized policy server, execute a selection algorithm, or consult an oracle. Whatever that might be, there is new code involved. David, I should point out that if only a small number of folks care about multihoming, then only a small number of folks need to change their stacks. I thought all clients would have to be modified if they wanted to take full advantage of a shim6 enabled multi-homed server? The keyword there is full. Unmodified clients can still interact with a multi-homed server in a legacy manner. And even in your solution, there would need to be some changes to the end host if you want to support exit point selection, or carry alternate locators in the transport. One of the problems that I have seen in the IETF is calling desires requirements. How important is exit point selection? Are there ways of implementing exit point selection without changing the IP stack? How critical is it that alternate locators be carried in the transport? Does the lack of that functionality make the protocol unusable? What _are_ the actual requirements (not the Goals)? From my perspective, the really, really critical flaw in both IPv4 and IPv6 is the lack of _transparent_ renumberability. Multi-homing is also a flaw, but less critical and I believe it can be addressed with the right solution to renumberability. A few folks worry about multi-homing. Everybody worries about end site renumbering. As with any political process, the stated requirements are a function of perspective. The stated requirements may or may not have anything to do with reality, realizability, practicality, or architectural elegance. It's just a mess. I think that we all can agree that a real locator/identifier split is the correct architectural direction, but that's simply not politically tractable. Right. And since it couldn't be done the right way in the protocol, we make the protocol much more complicated and force a reset to address functionality that relatively few folks need. It could have been done the right way in the protocol, but it wasn't. Yes, the result is that the subsequent 'work around' solution is much more complicated than it could have been. Again, between multihoming and mobility, the ubiquity and necessity of Internet access, and the reliability of the last mile, this is not going to remain a rare or even minority issue. Regards, Tony
Re: And Now for Something Completely Different (was Re: IPv6 news)
Daniel, But wasn't that the rationale for originally putting the kitchen sink into IPv6, rather than fixing the address length issue? The stated rationale was to fix the address length issue. I think we missed a lot of opportunities. Amen. We're 10 years on, and talking about whether there will need to be more than one massive pain of migration, because the kitchen sink didn't take into account multihoming. More generally because we were unwilling to make changes to the routing architecture. Now we're talking about a solution that appear to be an even worse Rube Goldberg than token ring source-route bridging. No one has proposed anything that is as bad as the exponential traffic explosion caused by explorers. Moore will likely have to continue to produce the solution. What happens if he can't? Silicon technology *is* topping out. What happens to v6 if every single household and business on the planet decides to multihome? Tony
Re: And Now for Something Completely Different (was Re: IPv6 news)
Tony, On Oct 18, 2005, at 1:56 PM, Tony Li wrote: Not necessarily. If you transition at the edge, what happens within the site matters only to the site and what matters to the core only matters to the core. No stacks, either core or edge, need to be rewritten. Transitioning at the edge implies to me that the hosts need to know about different semantics for the IPv6 header. That, in turn, implies that there is different code for the hosts. Alternately, if there is no new code anywhere, it's clear that you must necessarily have the same semantics and must not have made a change. No. The only code change that must occur is at the core/edge transition device _at both ends_. Let me try explaining this by example: Assume you have: - ISP A providing service to site X. - ISP B providing service to site Y. - ISP A has locator prefix A000::0 - ISP B has locator prefix B000::0 - Site X has identifier prefix 1000::0 - Site Y has identifier prefix 2000::0 - Host 1000::1 wants to send a packet to host 2000::2 Then: 1. Packet leaves host 1000::1 destined for 2000:2 and ends up at the site edge router for ISP A. 2. The site edge router for ISP A sees destination 2000:2 and looks up the locator in some globally distributed database using the identifier prefix 2000::0, getting back locator prefix B000::0. 3. The site edge router for ISP A rewrites the destination address with the locator prefix, i.e., B000::2. 4. The site edge router for ISP A forwards the packet to the next (core) hop for destination B000::2. 5. The site edge router for ISP B receives the packet destined for B000::2 6. The site edge router for ISP B rewrites the destination prefix with the identifier prefix, i.e., 2000::2 7. The site edge router for ISP B forwards the packet to the destination. You want multi-homing? Site Y has two ISPs, each having their own locator prefix, e.g., ISP B (B000::0) and ISP C (C000::0). The lookup at step 2 returns two locators and the site edge router for ISP A can choose which path to take (perhaps with advice from the administrator of Site Y encoded in the response from the lookup, e.g., a preference or priority value). Transparent renumbering is obvious. Mobility might be possible with a little work and the old site edge router forwarding to the new site edge router for the duration of the cached response from the lookup. No code changes within the site or within the core would be necessary. Of course, the tricky bit is in looking up the locator in the globally distributed database and caching the response (which presumably would be necessary because the lookup will take a long time, relatively speaking). When a new 'conversation' between two hosts start, the initial packet will obviously have increased latency, but subsequent packets could rely on cached information. Again, I realize this is a hack, but I suspect it is a hack that impacts fewer points than something like shim6. Again, source address selection is going to require something different than what we have today. The host might have to interact with some centralized policy server, execute a selection algorithm, or consult an oracle. Whatever that might be, there is new code involved. Well, yes, if source address selection is important. My point was that there didn't have to be new code in the IP stack. As with any political process, the stated requirements are a function of perspective. The stated requirements may or may not have anything to do with reality, realizability, practicality, or architectural elegance. Hmm. Are the aliens who took the _real_ IETF and replaced it with what's there now going to give it back? :-) It could have been done the right way in the protocol, but it wasn't. Yes, the result is that the subsequent 'work around' solution is much more complicated than it could have been. I grant additional complexity is necessary. However, additional complexity in every system seems like a bad idea to me. Again, between multihoming and mobility, the ubiquity and necessity of Internet access, and the reliability of the last mile, this is not going to remain a rare or even minority issue. I very much agree. Rgds, -drc
Re: And Now for Something Completely Different (was Re: IPv6 news)
man, 17,.10.2005 kl. 07.17 +0200, skrev Mikael Abrahamsson: Both MPLS and any tunneled VPN over IP means the core won't have to know about all those prefixes (think aggregation of addresses regionally in the IP case and outer label in the MPLS case). Hope you don't imply NAT and private addresses like it is usually associated with VPN in the IPv4 world ;) So if you're building a 100G capable platform that'll do IPv6 and MPLS, how much difference would it be if you only had to support 16000 labels and 16000 IPv6 prefixes, rather than 2 million? Then of course I guess the argument can be made to put everything on MPLS to avoid the core knowing about anything but outer labels. flameMPLS on its own won't solve anything. Although MPLS has its uses, it smells too much like another desperate attempt from the telco-heads in the ITU crowd to make a packet-switched network look and behave like a circuit-switched network./flame What this discussion boils down to is that a long term solution has to remove the size of the routing-table as a limiting factor in internet routing. Something must eliminate the need for every node in the default-free transit-network to know how to reach every allocated address-block at all times. Allocation policies, operational agreements on filtering, BCPs etc can only slow the growth of the routing-table. Growth can't be eliminated. In the future network you'll have routers that may know a lot about their local region of the network but have to rely on nodes that are several hops (even AS-hops) away to pass the packets to more remote destinations. These trust-relationships have to be built and maintained automatically (may involve packet tagging / tunnelling etc), similar to current route-cache mechanisms, but will require a whole new set of routing protocols. Despite lots of research there's no such solution today or anytime soon. Just think of the added complexity. How do you build trust with remote nodes given the problems you see in trusting your direct peers in the BGP world today? How can routing loops be prevented in such a network? All we know is that if there is no such solution, at some point in time the network will fragment due to its size and complexity. In the meantime we have to manage with what we've got, and treat v6 just like we've done with v4 - multihoming and all. We know we'll run out of v4 addresses at some point, and that v6 is the only realistic alternative. Without improved routing protocols, all we can do is to pray that the development of routing hardware in terms of memory and processing capability outpaces the growth of the routing table. Initiatives like shim6 that changes the behaviour of leaf-nodes are only a supplement and won't replace the need for true multi-homing for end-sites. Here we have to adapt to business needs, and businesses have made it pretty clear that it is unacceptable to them to be tied to any single provider. Besides, shim6 doesn't eliminate the need for a mechanism to locate any globally unique address. What if there's suddenly 10M LIR's, or otherwise a trend towards a market with very small providers each handling only a small number of customers? Who gets to decide who may peer with whom, or decide which providers will be denied the ability to build redundant connectivity with multiple upstreams? //Per
Re: And Now for Something Completely Different (was Re: IPv6 news)
On Mon, 17 Oct 2005, Per Heldal wrote: man, 17,.10.2005 kl. 07.17 +0200, skrev Mikael Abrahamsson: Both MPLS and any tunneled VPN over IP means the core won't have to know about all those prefixes (think aggregation of addresses regionally in the IP case and outer label in the MPLS case). Hope you don't imply NAT and private addresses like it is usually associated with VPN in the IPv4 world ;) No, no NAT and RFC1918 implied, even though it might be part of it. Then of course I guess the argument can be made to put everything on MPLS to avoid the core knowing about anything but outer labels. flameMPLS on its own won't solve anything. Although MPLS has its uses, it smells too much like another desperate attempt from the telco-heads in the ITU crowd to make a packet-switched network look and behave like a circuit-switched network./flame Why? The initial argument for MPLS was that it would solve the core problem and put intelligence at the edge. You would have a core that only needed to know about hundreds of nodes instead of 100.000:nds of nodes. Growth can't be eliminated. In the future network you'll have routers that may know a lot about their local region of the network but have to rely on nodes that are several hops (even AS-hops) away to pass the packets to more remote destinations. These trust-relationships have to Yes, that is what's being proposed. Know your internal nodes, announce single big prefix externally. With ISPs only having a single prefix and no single customer prefixes, routing table can be kept low. Redundancy can be solved with for instance shim6. alternative. Without improved routing protocols, all we can do is to pray that the development of routing hardware in terms of memory and processing capability outpaces the growth of the routing table. We have done this for 15 years or so, what good has it brought us? Yes, TCAM size etc has been fairly good in keeping up with routing table size, but at quite high cost. Initiatives like shim6 that changes the behaviour of leaf-nodes are only a supplement and won't replace the need for true multi-homing for end-sites. Here we have to adapt to business needs, and businesses have Why? What problem does multihoming with single prefix solve that a fully working shim6 doesn't? What is the argument that the internet needs to know about a lot of end-users, instead of the end-user knowing that each end user might have n number of IP addresses and that there are n^2 combinations to send packets? Convergence time in the real world today is in the minutes, with shim6 it would for the end user be much quicker to route around the problem. Shouldn't be any problem to have failover in the subsecond timeframe, even thought that might need some kind of hello mechanism that is suboptimal because it sends traffic not carrying any data. single provider. Besides, shim6 doesn't eliminate the need for a mechanism to locate any globally unique address. What if there's I thought DNS solved that? suddenly 10M LIR's, or otherwise a trend towards a market with very small providers each handling only a small number of customers? Who gets to decide who may peer with whom, or decide which providers will be denied the ability to build redundant connectivity with multiple upstreams? It costs money to maintain a LIR which limits the number of LIRs economically viable in the world. -- Mikael Abrahamssonemail: [EMAIL PROTECTED]
Re: And Now for Something Completely Different (was Re: IPv6 news)
On Sun, 16 Oct 2005, Tony Li wrote: This is completely orthogonal to a real identifier/locator split, which would divide what we know of as the 'address' into two separate spaces, one which says where the node is, topologically, and one which says who the node is. Hmm, no idea whether it's a good idea or not, but from POV of scaling while it might make 'where' scaleable, you still have to find a way to tie who to where. Some might say we already have this split though, DNS. Tony regards, -- Paul Jakma [EMAIL PROTECTED] [EMAIL PROTECTED] Key ID: 64A2FF6A Fortune: Robotic tape changer mistook operator's tie for a backup tape.
Re: And Now for Something Completely Different (was Re: IPv6 news)
On Mon, 17 Oct 2005, Per Heldal wrote: Well, let's try to turn the problem on its head and see if thats clearer; Imagine an internet where only your closest neighbors know you exist. The rest of the internet knows nothing about you, except there are mechanisms that let them track you down when necessary. That is very different from today's full-routing-table. Yes, it's true that it's different, but is it better? It does not provide 100% provider-indepence to begin with. Depending on who you ask that alone is a show-stopper. Well, the reason for people wanting to stick to their own IP adresses are administrative and technical. If we solve that then hopefully, it wont be such a big hassle to renumber to go to another provider. Also, if everybody got their equal size subnet delegation from each ISP then it shouldnt be that much of a problem to run two networks side-by-side by using the subnet part of the delegation equal to both networks, but keep the prefix separate. If you switch providers you change the prefix part. This means we need new mechanisms to handle this, but I feel that's better than doing the routing mistake again. The internet shouldn't need to know anything about individual users to begin with, provided there are mechanisms avilable track them down. By that I mean that algorithms to locate end-nodes may include mechanisms to interrogate a large number of nodes to find the desired location as opposed to looking it up in a locally stored database (routing-table). So what is it you're proposing? I understand what shim6 tries to do (since it basically keeps most of todays mechanisms) but I do not understand your proposal. Could you please elaborate? I thought DNS only provided a name for an address ;) How does DNS tell us that e.g. 193.10.6.6 is part of a subnet belonging to AS2838 and how to get there? Should end users really care for that level of routing information? Also, your proposal seems to indicate that we need something that sounds like a proxy server that actually do know more about the internet and who needs to keep state, this doesn't sound scalable? -- Mikael Abrahamssonemail: [EMAIL PROTECTED]
Re: And Now for Something Completely Different (was Re: IPv6 news)
is that anything like using, in Cisco terms, a fast-switching cache vs a FIB? On Oct 17, 2005, at 6:47 AM, Mikael Abrahamsson wrote: Well, let's try to turn the problem on its head and see if thats clearer; Imagine an internet where only your closest neighbors know you exist. The rest of the internet knows nothing about you, except there are mechanisms that let them track you down when necessary. That is very different from today's full-routing-table.
Re: And Now for Something Completely Different (was Re: IPv6 news)
man, 17,.10.2005 kl. 07.25 -0700, skrev Fred Baker: is that anything like using, in Cisco terms, a fast-switching cache vs a FIB? I'll bite as I wrote the paragraph you're quoting; Actually, hanging on to the old concepts may be more confusing than trying to look at it in completely new ways. Imagine a situation with no access to any means of direct communication (phone etc). You've got a message to deliver to some person, and have no idea where to find that person. Chances are there's a group of people nearby you can ask. They may know how to find the one you're looking for. If not they may know others they can ask on your behalf. Several iterations later the person is located and you've established a path through which you can pass the information you wanted. Translated into cisco terms this mean that the FIB is just a partial routing database, enough to start the search and otherwise handle communications in the neighborhood (no more than X router-hops, maybe AS-hops away). When the destination is located you keep that information for a while in case there are more packets going to the same place, similar to what you do with traditional route-cache. On Oct 17, 2005, at 6:47 AM, Mikael Abrahamsson wrote: Well, let's try to turn the problem on its head and see if thats clearer; Imagine an internet where only your closest neighbors know you exist. The rest of the internet knows nothing about you, except there are mechanisms that let them track you down when necessary. That is very different from today's full-routing-table.
Re: And Now for Something Completely Different (was Re: IPv6 news)
Paul, This is completely orthogonal to a real identifier/locator split, which would divide what we know of as the 'address' into two separate spaces, one which says where the node is, topologically, and one which says who the node is. Hmm, no idea whether it's a good idea or not, but from POV of scaling while it might make 'where' scaleable, you still have to find a way to tie who to where. True. Even better, you get to change this binding (mobility) or have multiple bindings (multihoming). Some might say we already have this split though, DNS. True enough, but unfortunately, it's not done in a way that we can make use of the identifier in the routing subsystem or in the transport protocols. Tony
Re: And Now for Something Completely Different (was Re: IPv6 news)
That reminds me of anycasting or routing issues. Hackers did use this technique to make use of ip addresses not really allocated. There would be no need for IPv6 if this was more widespread. How about claiming to be f.root-servers.net and setting up our own root :) Regards, Peter and Karin Dambier is that anything like using, in Cisco terms, a fast-switching cache vs a FIB? On Oct 17, 2005, at 6:47 AM, Mikael Abrahamsson wrote: Well, let's try to turn the problem on its head and see if thats clearer; Imagine an internet where only your closest neighbors know you exist. The rest of the internet knows nothing about you, except there are mechanisms that let them track you down when necessary. That is very different from today's full-routing-table.
Re: And Now for Something Completely Different (was Re: IPv6 news)
OK. What you just described is akin to an enterprise network with a default route. It's also akin to the way DNS works. The big question becomes not only who knows what I need to know, but how do I know that they actually know it?. For example, let's postulate that the concept is that each ISP advertises some sort of routing service that will install routes on demand, but requires that someone initiate a request for the route, and requires either the target system or the edge router in that domain that is closest to the target respond with a route. Simplistically, perhaps I am trying to route from my edge network (A) towards your edge network (B), and we are both customers of some ISP (C). The host A' that is trying to get to your host B' initiates a request. Lets presume that this goes to some name in domain A that lists all border routers, or some multicast group that they are members of. Presumably every border router does this, but for present discussion the border router in A connecting to router C' in C asks all of his peers (POPs?) for the route, and some other router C asks B's border router. B's border router has the route, and so replies; C replies to C', C' to A's border router, and that router to A'. A' can now send a message. Presumably, if someone else now ask C about the route, either C' or C, or if the route was multicast to all of C's edge routers then any router in C would be able to respond directly. This becomes more interesting if C is in fact a succession of peer ISPs or ISPs that purchase transit from other ISPs. It also becomes very interesting if some router D' is misconfigured to advertise itself as B. It's not dissimilar to ant routing. For that, there is a variety of literature; Google is your friend. In manet and sensor networks, it works pretty well, especially in the sense that once it finds a route it keeps using it while it continues working even if other routes are changing around it, and it can use local repair to deal with local changes. At least as the researchers have described it, it doesn't do policy very well, and in networks that tend to be stable (such as wired networks) its load and convergence properties can be improved on. I'll let you read there. On Oct 17, 2005, at 9:20 AM, Per Heldal wrote: man, 17,.10.2005 kl. 07.25 -0700, skrev Fred Baker: is that anything like using, in Cisco terms, a fast-switching cache vs a FIB? I'll bite as I wrote the paragraph you're quoting; Actually, hanging on to the old concepts may be more confusing than trying to look at it in completely new ways. Imagine a situation with no access to any means of direct communication (phone etc). You've got a message to deliver to some person, and have no idea where to find that person. Chances are there's a group of people nearby you can ask. They may know how to find the one you're looking for. If not they may know others they can ask on your behalf. Several iterations later the person is located and you've established a path through which you can pass the information you wanted. Translated into cisco terms this mean that the FIB is just a partial routing database, enough to start the search and otherwise handle communications in the neighborhood (no more than X router-hops, maybe AS-hops away). When the destination is located you keep that information for a while in case there are more packets going to the same place, similar to what you do with traditional route-cache. On Oct 17, 2005, at 6:47 AM, Mikael Abrahamsson wrote: Well, let's try to turn the problem on its head and see if thats clearer; Imagine an internet where only your closest neighbors know you exist. The rest of the internet knows nothing about you, except there are mechanisms that let them track you down when necessary. That is very different from today's full-routing-table.
Re: And Now for Something Completely Different (was Re: IPv6 news)
man, 17,.10.2005 kl. 15.47 +, skrev Mikael Abrahamsson: On Mon, 17 Oct 2005, Per Heldal wrote: Well, let's try to turn the problem on its head and see if thats clearer; Imagine an internet where only your closest neighbors know you exist. The rest of the internet knows nothing about you, except there are mechanisms that let them track you down when necessary. That is very different from today's full-routing-table. Yes, it's true that it's different, but is it better? This thread, as well as most messages on this mailinglist in the last 2 days says so. Everyone uses all their energy trying to work within the limits of the current scheme. Common sense says it would be to eliminate the problem. What happens to policies if there's no limit to the size of the routing-table? It does not provide 100% provider-indepence to begin with. Depending on who you ask that alone is a show-stopper. Well, the reason for people wanting to stick to their own IP adresses are administrative and technical. If we solve that then hopefully, it wont be such a big hassle to renumber to go to another provider. I'm not so sure it will be that easy to get the flexibility you want. How do you for example enforce rules of flexibilty on *all* dns-providers. Also, if everybody got their equal size subnet delegation from each ISP then it shouldnt be that much of a problem to run two networks side-by-side by using the subnet part of the delegation equal to both networks, but keep the prefix separate. If you switch providers you change the prefix part. This means we need new mechanisms to handle this, but I feel that's better than doing the routing mistake again. True, but it creates unnecessary complexity for end-systems. It still doesn't help for scaleability on the next level up. The internet shouldn't need to know anything about individual users to begin with, provided there are mechanisms avilable track them down. By that I mean that algorithms to locate end-nodes may include mechanisms to interrogate a large number of nodes to find the desired location as opposed to looking it up in a locally stored database (routing-table). So what is it you're proposing? I understand what shim6 tries to do (since it basically keeps most of todays mechanisms) but I do not understand your proposal. Could you please elaborate? What I've got can't be called a proposal. There's no solution to propose. I just think that network complexity should be handled in the network and not by exporting the problem to connected clients. BGP and its related path-selection algorithms have served us well for many years, but there's a need for alternatives and somebody have to get involved. I thought DNS only provided a name for an address ;) How does DNS tell us that e.g. 193.10.6.6 is part of a subnet belonging to AS2838 and how to get there? Should end users really care for that level of routing information? I never said so. Their equipment, their upstream, or the upstream's upstream may need to know to get there though. Also, your proposal seems to indicate that we need something that sounds like a proxy server that actually do know more about the internet and who needs to keep state, this doesn't sound scalable? There's no proxy server involved unless you count forwarding of route location requests between inter-domain routers as proxy. If so, all intra-domain routers would be proxies. Data transport along an established forwarding path would not change. This mailinglist isn't really the place to discuss future concepts and further discussion should move to the IETF Inter-Domain-Routing working-group or other suitable forum. //Per
Re: And Now for Something Completely Different (was Re: IPv6 news)
man, 17,.10.2005 kl. 19.16 +0200, skrev Peter Dambier: That reminds me of anycasting or routing issues. Hackers did use this technique to make use of ip addresses not really allocated. There would be no need for IPv6 if this was more widespread. How about claiming to be f.root-servers.net and setting up our own root :) Yeah, you'd love that, wouldn't you? ;) Trust that security considerations would be an important part of the design of any replacement for current routing protocols. //Per
Re: And Now for Something Completely Different (was Re: IPv6 news)
Imagine a situation with no access to any means of direct communication (phone etc). You've got a message to deliver to some person, and have no idea where to find that person. Chances are there's a group of people nearby you can ask. They may know how to find the one you're looking for. If not they may know others they can ask on your behalf. Several iterations later the person is located and you've established a path through which you can pass the information you wanted. Translated into cisco terms this mean that the FIB is just a partial routing database, enough to start the search and otherwise handle communications in the neighborhood (no more than X router-hops, maybe AS-hops away). When the destination is located you keep that information for a while in case there are more packets going to the same place, similar to what you do with traditional route-cache. check out The Landmark Hierarchy: A New Hierarchy for Routing in Very Large Networks; Paul Tsuchiya; 1989. randy
Re: And Now for Something Completely Different (was Re: IPv6 news)
On Mon, Oct 17, 2005 at 09:03:45AM -1000, Randy Bush wrote: Imagine a situation with no access to any means of direct communication (phone etc). You've got a message to deliver to some person, and have no idea where to find that person. Chances are there's a group of people nearby you can ask. They may know how to find the one you're looking for. If not they may know others they can ask on your behalf. Several iterations later the person is located and you've established a path through which you can pass the information you wanted. Translated into cisco terms this mean that the FIB is just a partial routing database, enough to start the search and otherwise handle communications in the neighborhood (no more than X router-hops, maybe AS-hops away). When the destination is located you keep that information for a while in case there are more packets going to the same place, similar to what you do with traditional route-cache. check out The Landmark Hierarchy: A New Hierarchy for Routing in Very Large Networks; Paul Tsuchiya; 1989. great stuff... i have a hardcopy. is it online yet? --bill (checking citesear...) randy
Re: And Now for Something Completely Different (was Re: IPv6 news)
mon, 17,.10.2005 kl. 11.29 -0700, Fred Baker: OK. What you just described is akin to an enterprise network with a default route. It's also akin to the way DNS works. No default, just one or more *potential* routes. Your input is appreciated, and yes I'm very much aware that many people who maintain solutions that assume full/total control of the entire routing-table will be screaming bloody murder if that is going to change. Further details about future inter-domain-routing concepts belong in other fora (e.g. ietf's inter-domain-routing wg). The long-term operational impact is that the current inter-domain-routing concepts (BGP etc) don't scale indefinitely and will have to be changed some time in the future. Thus expect the size of the routing-table to be eliminated from the list of limiting factors, or that the bar is considerably raised. --- Note that I'm not saying that nothing should be done to preserve and optimise the use of the resources that are available today just because there will be something better available in a distant future. I'm in favor of the most restrictive allocation policies in place today. The development of the internet has for a long time been based on finding better ways to use available resources (CIDR anyone). To me a natural next-step in that process is for RIR's to start reclaiming unused v4 address-blocks, or at least start collect data to document that space is not being used (if they're not already doing so). E.g prevously announced address-blocks that has disappeared from the global routing-table for more than X months should go back to the RIR-pool (X=6). //Per
Re: And Now for Something Completely Different (was Re: IPv6 news)
check out The Landmark Hierarchy: A New Hierarchy for Routing in Very Large Networks; Paul Tsuchiya; 1989. great stuff... i have a hardcopy. is it online yet? dunno if i would say great. but certainly good. http://portal.acm.org/citation.cfm?id=52329 randy
Re: And Now for Something Completely Different (was Re: IPv6 news)
--bill (checking citesear...) does that only yield rare papers :-) and citeseer does not have the paper, only a few cites to it randy
Re: And Now for Something Completely Different (was Re: IPv6 news)
That is an assumption that I haven't found it necessary to make. I have concluded that there is no real debate about whether the Internet will have to change to something that gives us the ability to directly address (e.g. not behind a NAT, which imposes some interesting requirements at the application layer and gateways of the sort which were what the Internet came about to not need) a whole lot more things than it does today. The debate is about how and when. when seems pretty solidly in the 3-10 year timeframe, exactly where in that timeframe being a point of some discussion, and how comes down to a choice of IPv6 or something else. Fleming's IPv8 was a non-stupid idea that has a fundamental flaw in that it re-interprets parts of the IPv4 header as domain identifiers - it effectively extends the IP address by 16 bits, which is good, but does so in a way that is not backward compatible. If we could make those 16 bits be AS numbers (and ignoring for the moment the fact that we seem to need larger AS numbers), the matter follows pretty quickly. If one is going to change the header, though, giving up fragmentation as a feature sees a little tough; one may as well change the header and manage to keep the capability. One also needs to change some other protocols, such as routing AS numbers and including them in DNS records as part of the address. From my perspective, we are having enough good experience with IPv6 that we should simply choose that approach; there isn't a real good reason to find a different one. Yes, that means there is a long coexistence period yada yada yada. This is also true of any other fundamental network layer protocol change. The RIRs have been trying pretty hard to make IPv6 allocations be one prefix per ISP, with truly large edge networks being treated as functionally equivalent to an ISP (PI addressing without admitting it is being done). Make the bald assertion that this is equal to one prefix per AS (they're not the same statement at all, but the number of currently assigned AS numbers exceeds the number of prefixes under discussion, so in my mind it makes a reasonable thumb-in-the-wind- guesstimate), that is a reduction of the routing table size by an order of magnitude. If we are able to reduce the routing table size by an order of magnitude, I don't see that we have a requirement to fundamentally change the routing technology to support it. We may *want* to (and yes, I would like to, for various reasons), but that is a different assertion. On Oct 17, 2005, at 12:42 PM, Per Heldal wrote: mon, 17,.10.2005 kl. 11.29 -0700, Fred Baker: OK. What you just described is akin to an enterprise network with a default route. It's also akin to the way DNS works. No default, just one or more *potential* routes. Your input is appreciated, and yes I'm very much aware that many people who maintain solutions that assume full/total control of the entire routing-table will be screaming bloody murder if that is going to change. Further details about future inter-domain-routing concepts belong in other fora (e.g. ietf's inter-domain-routing wg). The long-term operational impact is that the current inter-domain- routing concepts (BGP etc) don't scale indefinitely and will have to be changed some time in the future. Thus expect the size of the routing-table to be eliminated from the list of limiting factors, or that the bar is considerably raised. --- Note that I'm not saying that nothing should be done to preserve and optimise the use of the resources that are available today just because there will be something better available in a distant future. I'm in favor of the most restrictive allocation policies in place today. The development of the internet has for a long time been based on finding better ways to use available resources (CIDR anyone). To me a natural next-step in that process is for RIR's to start reclaiming unused v4 address-blocks, or at least start collect data to document that space is not being used (if they're not already doing so). E.g prevously announced address-blocks that has disappeared from the global routing-table for more than X months should go back to the RIR-pool (X=6). //Per
Re: And Now for Something Completely Different (was Re: IPv6 news)
Fred, If we are able to reduce the routing table size by an order of magnitude, I don't see that we have a requirement to fundamentally change the routing technology to support it. We may *want* to (and yes, I would like to, for various reasons), but that is a different assertion. There is a fundamental difference between a one-time reduction in the table and a fundamental dissipation of the forces that cause it to bloat in the first place. Simply reducing the table as a one-off only buys you linearly more time. Eliminating the drivers for bloat buys you technology generations. If we're going to put the world thru the pain of change, it seems that we should do our best to ensure that it never, ever has to happen again. Regards, Tony
Re: And Now for Something Completely Different (was Re: IPv6 news)
works for me - I did say I'd like to change the routing protocol - but I think the routing protocol can be changed asynchronously, and will have to. On Oct 17, 2005, at 1:51 PM, Tony Li wrote: Fred, If we are able to reduce the routing table size by an order of magnitude, I don't see that we have a requirement to fundamentally change the routing technology to support it. We may *want* to (and yes, I would like to, for various reasons), but that is a different assertion. There is a fundamental difference between a one-time reduction in the table and a fundamental dissipation of the forces that cause it to bloat in the first place. Simply reducing the table as a one- off only buys you linearly more time. Eliminating the drivers for bloat buys you technology generations. If we're going to put the world thru the pain of change, it seems that we should do our best to ensure that it never, ever has to happen again. Regards, Tony
Re: And Now for Something Completely Different (was Re: IPv6 news)
It doesn't look like were talking about the same thing. A. Address conservation and aggregation (IPv4 and IPv6) is very important to get the most out of what we've got. Read; limit the combined routing-table to a manageable size whatever that may be. B. There seems to be widespread fear that the global routing-table will grow to a non-manageable size sooner or later regardless of the efforts under A. So the limit has to be removed. What you address below mostly belong under A (and I mostly agree), whereas I so far have focused on B. On Mon, 2005-10-17 at 13:12 -0700, Fred Baker wrote: That is an assumption that I haven't found it necessary to make. I have concluded that there is no real debate about whether the Internet will have to change to something that gives us the ability to directly address (e.g. not behind a NAT, which imposes some interesting requirements at the application layer and gateways of the sort which were what the Internet came about to not need) a whole lot more things than it does today. The debate is about how and when. when seems pretty solidly in the 3-10 year timeframe, exactly where in that timeframe being a point of some discussion, and how comes down to a choice of IPv6 or something else. Sure, something has to replace IPv4 sooner or later and IPv6 is almost certainly the thing. Personally I belive the most trustworthy projections points towards 2010 as the time we'll run out of addresses in V4 with an additional 2-3 years if we can manage to reclaim up to 90% of those previously allocated addressblocks which are not used or not announced to the public internet. Fleming's IPv8 was a non-stupid idea that has a fundamental flaw in that it re-interprets parts of the IPv4 header as domain identifiers - it effectively extends the IP address by 16 bits, which is good, but does so in a way that is not backward compatible. If we could make those 16 bits be AS numbers (and ignoring for the moment the fact that we seem to need larger AS numbers), the matter follows pretty quickly. If one is going to change the header, though, giving up fragmentation as a feature sees a little tough; one may as well change the header and manage to keep the capability. One also needs to change some other protocols, such as routing AS numbers and including them in DNS records as part of the address. For the record: You brought up IPv8. Nothing of what I've mentioned requires any change to transport protocols wether implemented on top of IPv4 or 6. From my perspective, we are having enough good experience with IPv6 that we should simply choose that approach; there isn't a real good reason to find a different one. Yes, that means there is a long coexistence period yada yada yada. This is also true of any other fundamental network layer protocol change. The RIRs have been trying pretty hard to make IPv6 allocations be one prefix per ISP, with truly large edge networks being treated as functionally equivalent to an ISP (PI addressing without admitting it is being done). Make the bald assertion that this is equal to one prefix per AS (they're not the same statement at all, but the number of currently assigned AS numbers exceeds the number of prefixes under discussion, so in my mind it makes a reasonable thumb-in-the-wind- guesstimate), that is a reduction of the routing table size by an order of magnitude. I wouldn't even characterise that as being bald. Initial allocations of more than one prefix per AS should not be allowed. Further; initial allocations should differentiate between network of various sizes into separate address-blocks to simplify and promote strict prefix-filtering policies. Large networks may make arrangements with their neighbors to honor more specifics, but that shouldn't mean that the rest of the world should accept those. If we are able to reduce the routing table size by an order of magnitude, I don't see that we have a requirement to fundamentally change the routing technology to support it. We may *want* to (and yes, I would like to, for various reasons), but that is a different assertion. Predictions disagree. //Per
Re: And Now for Something Completely Different (was Re: IPv6 news)
works for me - I did say I'd like to change the routing protocol - but I think the routing protocol can be changed asynchronously, and will have to. and that is what the other v6 ivory tower crew said a decade ago. which is why we have the disaster we have now. randy
Re: And Now for Something Completely Different (was Re: IPv6 news)
There is a fundamental difference between a one-time reduction in the table and a fundamental dissipation of the forces that cause it to bloat in the first place. Simply reducing the table as a one-off only buys you linearly more time. Eliminating the drivers for bloat buys you technology generations. If we're going to put the world thru the pain of change, it seems that we should do our best to ensure that it never, ever has to happen again. That's the goal here? To ensure we'll never have another protocol transition? I hope you realize what a flawed statement that is. tony probably did not think about it because that's not what he said at all. he was speaking of routing table bloat, not transitions. and he was spot on. randy
Re: And Now for Something Completely Different (was Re: IPv6 news)
At 04:51 PM 10/17/2005, Tony Li wrote: Fred, If we are able to reduce the routing table size by an order of magnitude, I don't see that we have a requirement to fundamentally change the routing technology to support it. We may *want* to (and yes, I would like to, for various reasons), but that is a different assertion. There is a fundamental difference between a one-time reduction in the table and a fundamental dissipation of the forces that cause it to bloat in the first place. Simply reducing the table as a one-off only buys you linearly more time. Eliminating the drivers for bloat buys you technology generations. If we're going to put the world thru the pain of change, it seems that we should do our best to ensure that it never, ever has to happen again. But wasn't that the rationale for originally putting the kitchen sink into IPv6, rather than fixing the address length issue? I think we missed a lot of opportunities. Extended addressing may well have been possible to integrate in the mid 1990's ahead of much of the massive Internet expansion. Too late. We're 10 years on, and talking about whether there will need to be more than one massive pain of migration, because the kitchen sink didn't take into account multihoming. Now we're talking about a solution that appear to be an even worse Rube Goldberg than token ring source-route bridging. Moore will likely have to continue to produce the solution.
Re: And Now for Something Completely Different (was Re: IPv6 news)
Daniel, If we're going to put the world thru the pain of change, it seems that we should do our best to ensure that it never, ever has to happen again. That's the goal here? To ensure we'll never have another protocol transition? I hope you realize what a flawed statement that is. We can't see into the future. However, assuming that IPv6 is the Last Protocol seems a bit short sighted. If we get 20 years out of IPv6, that will be just peachy. I see that as a worthy goal and no, I don't see that as flawed. While we certainly cannot guarantee that v6 will be the last protocol, we should certainly be designing for it to be the best that we can possibly make it. Just how many times do you think that we will replace all implementations? This change is simply fundamental to the way the Internet works. There is almost as much pain associated with this change as if we were to change the electric outlet voltage in every single country to a mutually incompatible standard. Can you imagine power companies making that change and then telling consumers to expect another such change in 20 years? To not even *attempt* to avoid future all-systems changes is nothing short of negligent, IMHO. Of course, if we can't get PI address space for enterprises and real multihoming, there won't be any real IPv6 deployment. Lots of (possibly illegitimate) IPv4 trading and NAT, but not IPv6. These aren't nice to haves. Even if it shortens the life of IPv6, that is an acceptable trade-off. IPv6 is not the Last Protocol. If you do get PI space for multihoming, then by definition, it cannot be the last protocol. In fact, it will have cemented v6's lifetime as just 10 more years. Tony
Re: And Now for Something Completely Different (was Re: IPv6 news)
we agree that at least initially every prefix allocated should belong to a different AS (eg, no AS gets more than one); the fly in that is whether there is an ISP somewhere that is so truly large that it needs two super-sized blocks. I don't know if such exists, but one hopes it is very much the exception. The question is does every AS get a prefix. Under current rules, most AS's assigned to edge networks to support multihoming will not get a prefix. I personally think that's probably the wrong answer (eg, you and I seem to agree on PI space for networks that would warrant an AS number does to size, connectivity, and use of BGP to manage their borders), but it is the current answer. On Oct 17, 2005, at 2:06 PM, Per Heldal wrote: The RIRs have been trying pretty hard to make IPv6 allocations be one prefix per ISP, with truly large edge networks being treated as functionally equivalent to an ISP (PI addressing without admitting it is being done). Make the bald assertion that this is equal to one prefix per AS (they're not the same statement at all, but the number of currently assigned AS numbers exceeds the number of prefixes under discussion, so in my mind it makes a reasonable thumb-in-the-wind- guesstimate), that is a reduction of the routing table size by an order of magnitude. I wouldn't even characterise that as being bald. Initial allocations of more than one prefix per AS should not be allowed. Further; initial allocations should differentiate between network of various sizes into separate address-blocks to simplify and promote strict prefix-filtering policies. Large networks may make arrangements with their neighbors to honor more specifics, but that shouldn't mean that the rest of the world should accept those.
Re: And Now for Something Completely Different (was Re: IPv6 news)
On Oct 17, 2005, at 2:24 PM, Tony Li wrote: To not even *attempt* to avoid future all-systems changes is nothing short of negligent, IMHO. On Oct 17, 2005, at 2:17 PM, Randy Bush wrote: and that is what the other v6 ivory tower crew said a decade ago. which is why we have the disaster we have now. and there I would agree, on both points. now, the proposal put forward lo these many moons ago to avoid any possibility of a routing change was, as I recall, Nimrod, and the Nimrod architecture called for variable length addresses in the network layer protocol and the use of a flow label (as in IPv6 flow label) as a short-form address in some senses akin to a virtual circuit ID. There has been a lot of work on that in rrg among other places, but the word from those who would deploy it has been uniformly think in terms of an incremental upgrade to BGP and maybe MPLS will work as a virtual circuit ID if we really need one. As you no doubt recall all too well, the variable length address was in fact agreed on at one point, but that failed for political reasons. Something about OSI. The 16 byte length of an IPv6 address derived from that as well - it didn't allow one to represent an NSAP in IPv6, which was an objective. So the routing problem was looked at, and making a fundamental routing change was rejected by both the operational community and the routing folks. No, IPv6 doesn't fix (or even change) the routing of the system, and that problem will fester until it becomes important enough to change. But lets not blame that on the ivory tower folks, at least not wholly. We were all involved.
Re: And Now for Something Completely Different (was Re: IPv6 news)
On Mon, 17 Oct 2005 14:24:08 -0700 Tony Li [EMAIL PROTECTED] wrote: Dear Tony et al.; This is beginning to sound like an IETF or IRTF mail list, and, lo!, I get an email today from Leslie Daigle : A new mailing list has been created to provide a forum for general discussion of Internet architectural issues: [EMAIL PROTECTED] https://www1.ietf.org/mailman/listinfo/architecture-discuss The architecture-discuss list serves as a technical discussion forum for all members of the IETF community that are interested in larger architectural issues. It is meant to be an open discussion forum for all long and/or wide range architectural concerns related to the Internet Architecture. In particular, it may be used to discuss and bring forth different points of view on controversial architectural questions. Discussions that drill down and dwell on specifics of individual protocols or technologies are to be discouraged, redirected to appropriate other fora, or re-cast to discussions of broader implications for Internet architecture. Maybe this conversation should move there. Maybe a lot of operators should join that list. Probably couldn't hurt. Regards Marshall Eubanks Daniel, If we're going to put the world thru the pain of change, it seems that we should do our best to ensure that it never, ever has to happen again. That's the goal here? To ensure we'll never have another protocol transition? I hope you realize what a flawed statement that is. We can't see into the future. However, assuming that IPv6 is the Last Protocol seems a bit short sighted. If we get 20 years out of IPv6, that will be just peachy. I see that as a worthy goal and no, I don't see that as flawed. While we certainly cannot guarantee that v6 will be the last protocol, we should certainly be designing for it to be the best that we can possibly make it. Just how many times do you think that we will replace all implementations? This change is simply fundamental to the way the Internet works. There is almost as much pain associated with this change as if we were to change the electric outlet voltage in every single country to a mutually incompatible standard. Can you imagine power companies making that change and then telling consumers to expect another such change in 20 years? To not even *attempt* to avoid future all-systems changes is nothing short of negligent, IMHO. Of course, if we can't get PI address space for enterprises and real multihoming, there won't be any real IPv6 deployment. Lots of (possibly illegitimate) IPv4 trading and NAT, but not IPv6. These aren't nice to haves. Even if it shortens the life of IPv6, that is an acceptable trade-off. IPv6 is not the Last Protocol. If you do get PI space for multihoming, then by definition, it cannot be the last protocol. In fact, it will have cemented v6's lifetime as just 10 more years. Tony
Re: And Now for Something Completely Different (was Re: IPv6 news)
- Original Message - From: Fred Baker [EMAIL PROTECTED] To: Per Heldal [EMAIL PROTECTED] Cc: nanog@merit.edu Sent: Monday, October 17, 2005 15:12 Subject: Re: And Now for Something Completely Different (was Re: IPv6 news) That is an assumption that I haven't found it necessary to make. I have concluded that there is no real debate about whether the Internet will have to change to something that gives us the ability to directly address (e.g. not behind a NAT, which imposes some interesting requirements at the application layer and gateways of the sort which were what the Internet came about to not need) a whole lot more things than it does today. The debate is about how and when. when seems pretty solidly in the 3-10 year timeframe, exactly where in that timeframe being a point of some discussion, and how comes down to a choice of IPv6 or something else. Fleming's IPv8 was a non-stupid idea that has a fundamental flaw in that it re-interprets parts of the IPv4 header as domain identifiers - it effectively extends the IP address by 16 bits, which is good, but does so in a way that is not backward compatible. If we could make those 16 bits be AS numbers (and ignoring for the moment the fact that we seem to need larger AS numbers), the matter follows pretty quickly. If one is going to change the header, though, giving up fragmentation as a feature sees a little tough; one may as well change the header and manage to keep the capability. One also needs to change some other protocols, such as routing AS numbers and including them in DNS records as part of the address. From my perspective, we are having enough good experience with IPv6 that we should simply choose that approach; there isn't a real good reason to find a different one. Yes, that means there is a long coexistence period yada yada yada. This is also true of any other fundamental network layer protocol change. The RIRs have been trying pretty hard to make IPv6 allocations be one prefix per ISP, with truly large edge networks being treated as functionally equivalent to an ISP (PI addressing without admitting it is being done). Make the bald assertion that this is equal to one prefix per AS (they're not the same statement at all, but the number of currently assigned AS numbers exceeds the number of prefixes under discussion, so in my mind it makes a reasonable thumb-in-the-wind- guesstimate), that is a reduction of the routing table size by an order of magnitude. If we are able to reduce the routing table size by an order of magnitude, I don't see that we have a requirement to fundamentally change the routing technology to support it. We may *want* to (and yes, I would like to, for various reasons), but that is a different assertion. On Oct 17, 2005, at 12:42 PM, Per Heldal wrote: mon, 17,.10.2005 kl. 11.29 -0700, Fred Baker: OK. What you just described is akin to an enterprise network with a default route. It's also akin to the way DNS works. No default, just one or more *potential* routes. Your input is appreciated, and yes I'm very much aware that many people who maintain solutions that assume full/total control of the entire routing-table will be screaming bloody murder if that is going to change. Further details about future inter-domain-routing concepts belong in other fora (e.g. ietf's inter-domain-routing wg). The long-term operational impact is that the current inter-domain- routing concepts (BGP etc) don't scale indefinitely and will have to be changed some time in the future. Thus expect the size of the routing-table to be eliminated from the list of limiting factors, or that the bar is considerably raised. --- Note that I'm not saying that nothing should be done to preserve and optimise the use of the resources that are available today just because there will be something better available in a distant future. I'm in favor of the most restrictive allocation policies in place today. The development of the internet has for a long time been based on finding better ways to use available resources (CIDR anyone). To me a natural next-step in that process is for RIR's to start reclaiming unused v4 address-blocks, or at least start collect data to document that space is not being used (if they're not already doing so). E.g prevously announced address-blocks that has disappeared from the global routing-table for more than X months should go back to the RIR-pool (X=6). //Per
Re: And Now for Something Completely Different (was Re: IPv6 news)
Thus spake Fred Baker [EMAIL PROTECTED] The RIRs have been trying pretty hard to make IPv6 allocations be one prefix per ISP, with truly large edge networks being treated as functionally equivalent to an ISP (PI addressing without admitting it is being done). Make the bald assertion that this is equal to one prefix per AS (they're not the same statement at all, but the number of currently assigned AS numbers exceeds the number of prefixes under discussion, so in my mind it makes a reasonable thumb-in-the-wind- guesstimate), that is a reduction of the routing table size by an order of magnitude. If we are able to reduce the routing table size by an order of magnitude, I don't see that we have a requirement to fundamentally change the routing technology to support it. We may *want* to (and yes, I would like to, for various reasons), but that is a different assertion. If we reduce the average number of prefixes per AS by an order of magnitude, IMHO the result will be that there will be an order of magnitude growth in the number of ASes. We're just going to trade one problem for another. What we need is an interdomain routing system that can either (a) drastically reduce the incremental cost of additional prefixes in the DFZ, or (b) move the exist cost out of the DFZ to the people who want to multihome. Both probably mean ditching BGP4 and moving to some sort of inter-AS MPLS scheme, but it will never see the light of day unless it allows leaving hosts and intra-site routing intact (i.e. hop-by-hop routing and a single prefix per site). This last is why shim6 is DOA. S Stephen SprunkStupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them. --Aaron Sorkin
Re: And Now for Something Completely Different (was Re: IPv6 news)
What we need is an interdomain routing system that can either (a) drastically reduce the incremental cost of additional prefixes in the DFZ, or (b) move the exist cost out of the DFZ to the people who want to multihome. Both probably mean ditching BGP4 and moving to some sort of inter-AS MPLS scheme, but it will never see the light of day unless it allows leaving hosts and intra-site routing intact (i.e. hop-by-hop routing and a single prefix per site). This last is why shim6 is DOA. or... drop the idea of A/THE default free zone and recognize that the concept is based on a flawed assumption. --bill S Stephen SprunkStupid people surround themselves with smart CCIE #3723 people. Smart people surround themselves with K5SSS smart people who disagree with them. --Aaron Sorkin
Re: And Now for Something Completely Different (was Re: IPv6 news)
Fred, So the routing problem was looked at, and making a fundamental routing change was rejected by both the operational community and the routing folks. No, IPv6 doesn't fix (or even change) the routing of the system, and that problem will fester until it becomes important enough to change. From this end of the elephant, we looked at Nimrod and saw potential there, but did not buy off on it. We also looked at GSE and the routing folks at the very least seemed bought into that, but it died, under what I would characterize as a purely political hailstorm. Yes, the lack of a scalable routing architecture will continue to fester until it has sufficient political visibility that it exceeds our pain threshold and we decide to make the change. The problem with this model is that the pain of change grows daily. Each and every v6 system that is deployed is yet another bit of installed base that will need to be updated someday. The Internet community needs the IETF leadership to understand this very clearly and to take action to resolve this issue sooner, not later. As others have said, this is a pay now or pay later situation, and the pay later portion is MUCH more expensive. Specifically, the IAB should call for a halt to IPv6 deployment until consensus is reached on a scalable routing architecture. I realize that this is painful, but continuing to deploy is simply creating a v6 mortgage that we cannot afford to pay off. Regards, Tony
Re: And Now for Something Completely Different (was Re: IPv6 news)
[EMAIL PROTECTED] (Tony Li) writes: Specifically, the IAB should call for a halt to IPv6 deployment until consensus is reached on a scalable routing architecture. I realize that this is painful, but continuing to deploy is simply creating a v6 mortgage that we cannot afford to pay off. well, maybe so. but IAB could never make deployment start, and IAB cannot make deployment stop. deployment happens on its own terms. leadership has a built in time delay and a couple layers of indirection between intent and action and results. if you want IAB to lead somehow, give them some runway. -- Paul Vixie
Re: And Now for Something Completely Different (was Re: IPv6 news)
On Mon, 17 Oct 2005, Tony Li wrote: True. Even better, you get to change this binding (mobility) or have multiple bindings (multihoming). Indeed. True enough, but unfortunately, it's not done in a way that we can make use of the identifier in the routing subsystem or in the transport protocols. Well, if the idea is that the global routing subsystem should not have to burdened with the overwhelming details of who-where, then this mightn't matter much - all routing needs to know is where. The transport protocols, well they generally act on behalf of something which can do the lookup and supply transport with right address, as long the DNS server does not require who-where indirection ;). The other way of course is to carry who-where as routes in our current routing system. Then we just have to figure out how to confine things topologically. Would require changes in how providers peer though, but it is possible. And has been done in other networks, eg GSM in Ireland. We have provider-assigned, but provider-mobile prefixes. Just as with IP multihoming there was much protest that it couldn't be done, would be too problematic, would be too burdensome. However the regulator told the operators I don't care, you have till X to figure it out and implement. The operators did figure out, presumably including how to do billing for the differentials of any traffic carried for customers who had moved to other providers... The rest of the world has no clue that a large set of Irish GSM telephone numbers are essentially /32 routed between Irish providers. ;) A possibility anyway (but whether it's the least worst way - i don't know ;) ). It does though keep operators fully involved in all aspects of routing. Otherwise end-hosts will just work around the 'dumb' providers themselves, if there's no solution operators like. (Not a bad thing either really). Tony regards, -- Paul Jakma [EMAIL PROTECTED] [EMAIL PROTECTED] Key ID: 64A2FF6A Fortune: Money isn't everything -- but it's a long way ahead of what comes next. -- Sir Edmond Stockdale
Re: And Now for Something Completely Different (was Re: IPv6 news)
On Mon, 17 Oct 2005, Mikael Abrahamsson wrote: Also, if everybody got their equal size subnet delegation from each ISP then it shouldnt be that much of a problem to run two networks side-by-side by using the subnet part of the delegation equal to both networks, but keep the prefix separate. If you switch providers you change the prefix part. This means we need new mechanisms to handle this, Hmm, one thing that would be nice would be to seperate the prefix from the host identifier in DNS, oh... wait.. but I feel that's better than doing the routing mistake again. It's all routing, just shifting different portions of it to different places. regards, -- Paul Jakma [EMAIL PROTECTED] [EMAIL PROTECTED] Key ID: 64A2FF6A Fortune: panic (Splunge!); linux-2.2.16/drivers/scsi/psi240i.c
And Now for Something Completely Different (was Re: IPv6 news)
Tony, On Oct 15, 2005, at 11:26 PM, Tony Li wrote: Paul is correct. Things that looked like NAT were rejected because NAT is evil. Religion is so much fun. Shifting the NAT to end system removed the objection to NAT, tho it's not entirely clear why. Shifting NAT to the end system also happened to simplify the entire solution as well. Except for the part about having to rewrite all existing implementations to take full advantage of the technology. VJ compression should not be considered a violation of the end-to- end principle, as it is a per-link hack and performs a function that CANNOT be performed in the end systems. However, I'm not entirely sure that this is relevant. Well, if you NAT the destination identifier into a routing locator when a packet traverses the source edge/core boundary and NAT the locator back into the original destination identifier when you get to the core/destination edge boundary, it might be relevant. The advantages I see of such an approach would be: - no need to modify existing IPv6 stacks in any way - identifiers do not need to be assigned according to network topology (they could, in fact, be allocated according to national political boundaries, geographic boundaries, or randomly for that matter). They wouldn't even necessarily have to be IPv6 addresses just so long as they could be mapped and unmapped into the appropriate locators (e.g., they could even be, oh say, IPv4 addresses). - locators could change arbitrarily without affecting end-to-end sessions in any way - the core/destination edge NAT could have arbitrarily many locators associated with it - the source edge/core NAT could determine which of the locators associated with a destination it wanted to use Of course, the locator/identifier mapping is where things might get a bit complicated. What would be needed would be a globally distributed lookup technology that could take in an identifier and return one or more locators. It would have to be very fast since the mapping would be occurring for every packet, implying a need for caching and some mechanism to insure cache coherency, perhaps something as simple as a cache entry time to live if you make the assumption that the mappings either don't change very frequently and/ or stale mappings could be dealt with. You'd also probably want some way to verify that the mappings weren't mucked with by miscreants. This sounds strangely familiar... Obviously, some of the disadvantages of such an approach would be that it would require both ends to play and end users wouldn't be able to traceroute. I'm sure there are many other disadvantages as well. However, if an approach like this would be technically feasible (and I'm not entirely sure it would be), I suspect it would get deployed _much_ faster than an approach that requires every network stack to be modified. Again. Particularly given the number of folks who care about multi-homing are so small relative to the number of folks on the Internet. Can two evils make a good? :-) Rgds, -drc (speaking only for myself, of course)
Re: And Now for Something Completely Different (was Re: IPv6 news)
Hi David, snip Well, if you NAT the destination identifier into a routing locator when a packet traverses the source edge/core boundary and NAT the locator back into the original destination identifier when you get to the core/destination edge boundary, it might be relevant. The advantages I see of such an approach would be: - no need to modify existing IPv6 stacks in any way - identifiers do not need to be assigned according to network topology (they could, in fact, be allocated according to national political boundaries, geographic boundaries, or randomly for that matter). They wouldn't even necessarily have to be IPv6 addresses just so long as they could be mapped and unmapped into the appropriate locators (e.g., they could even be, oh say, IPv4 addresses). - locators could change arbitrarily without affecting end-to-end sessions in any way - the core/destination edge NAT could have arbitrarily many locators associated with it - the source edge/core NAT could determine which of the locators associated with a destination it wanted to use Of course, the locator/identifier mapping is where things might get a bit complicated. What would be needed would be a globally distributed lookup technology that could take in an identifier and return one or more locators. It would have to be very fast since the mapping would be occurring for every packet, implying a need for caching and some mechanism to insure cache coherency, perhaps something as simple as a cache entry time to live if you make the assumption that the mappings either don't change very frequently and/ or stale mappings could be dealt with. You'd also probably want some way to verify that the mappings weren't mucked with by miscreants. This sounds strangely familiar... Certainly does. Apparently this or a similar idea was suggested back in 1997, and is the root origin of the 64 bits for host address space, according to Christian Huitema, in his IPv6 book - http://www.huitema.net/ipv6.asp. A google search found the draft : GSE - An Alternate Addressing Architecture for IPv6 M. O'Dell, INTERNET DRAFT, 1997 http://www.caida.org/outreach/bib/networking/entries/odell97GSE.xml Can two evils make a good? :-) Not sure, however, two wrongs don't make a right, but three lefts do. Regards, Mark. -- The Internet's nature is peer to peer.
Re: And Now for Something Completely Different (was Re: IPv6 news)
Shifting the NAT to end system removed the objection to NAT, tho it's not entirely clear why. Shifting NAT to the end system also happened to simplify the entire solution as well. Except for the part about having to rewrite all existing implementations to take full advantage of the technology. That was inevitable from the start. A real locator/identifier separation requires a rewrite. Any system that provided site-wide source address control was going to require a rewrite. The bigger issue is that given that rewrite was inevitable, why didn't we have more design freedom. Religion *is* so much fun. Obviously, some of the disadvantages of such an approach would be that it would require both ends to play and end users wouldn't be able to traceroute. I'm sure there are many other disadvantages as well. However, if an approach like this would be technically feasible (and I'm not entirely sure it would be), I suspect it would get deployed _much_ faster than an approach that requires every network stack to be modified. Again. Particularly given the number of folks who care about multi-homing are so small relative to the number of folks on the Internet. David, I should point out that if only a small number of folks care about multihoming, then only a small number of folks need to change their stacks. And even in your solution, there would need to be some changes to the end host if you want to support exit point selection, or carry alternate locators in the transport. It's just a mess. I think that we all can agree that a real locator/ identifier split is the correct architectural direction, but that's simply not politically tractable. If the real message that the provider community is trying to send is that they want this, and not IPv6 as it stands today, then that's the message that should be sent, without reference to shim6. Tony
Re: And Now for Something Completely Different (was Re: IPv6 news)
Certainly does. Apparently this or a similar idea was suggested back in 1997, and is the root origin of the 64 bits for host address space, according to Christian Huitema, in his IPv6 book - http://www.huitema.net/ipv6.asp. A google search found the draft : GSE - An Alternate Addressing Architecture for IPv6 M. O'Dell, INTERNET DRAFT, 1997 http://www.caida.org/outreach/bib/networking/entries/odell97GSE.xml Note that GSE is in no way a NAT, so is very different than David's proposal. GSE also has a direct impact on all implementations (e.g., only use the identifier bits in the TCP pseudo-header, so that is also an all- implementations change. Further, that is a flag day, worldwide, even for non-multi-homed sites. Tony
Re: And Now for Something Completely Different (was Re: IPv6 news)
Tony Li wrote: It's just a mess. I think that we all can agree that a real locator/ identifier split is the correct architectural direction, but that's simply not politically tractable. If the real message that the provider community is trying to send is that they want this, and not IPv6 as it stands today, then that's the message that should be sent, without reference to shim6. Tony How is a split between locator / identifier any different logicaly from the existing ipv4 source routing? I thought that got dead ended? Or is a table lookup going to be needed? Wont all those tables need to be in the exact (or close to) same place as the current routing tables? Appreciate any enlightenment. Joe
Re: And Now for Something Completely Different (was Re: IPv6 news)
How is a split between locator / identifier any different logicaly from the existing ipv4 source routing? IPv4 source routing, as it exists today, is an extremely limited mechanism for specifying waypoints along the path to the destination. This is completely orthogonal to a real identifier/locator split, which would divide what we know of as the 'address' into two separate spaces, one which says where the node is, topologically, and one which says who the node is. One might use the identifier in the TCP pseudo-header, but not the locator, for one example, immediately allowing both mobility and multi-homing. Tony
Re: And Now for Something Completely Different (was Re: IPv6 news)
On Sun, 16 Oct 2005, Joe Maimon wrote: Tony Li wrote: It's just a mess. I think that we all can agree that a real locator/ identifier split is the correct architectural direction, but that's simply not politically tractable. If the real message that the provider community is trying to send is that they want this, and not IPv6 as it stands today, then that's the message that should be sent, without reference to shim6. Tony How is a split between locator / identifier any different logicaly from the existing ipv4 source routing? I thought that got dead ended? Or is a table lookup going to be needed? Wont all those tables need to be in the exact (or close to) same place as the current routing tables? Appreciate any enlightenment. For example, if your goal was to have TCP-like sessions between identifiers survive network events without globally propagating full network topology information about your site (the gripe against classic IPv4 BGP) you could have multiple locators associated with any single identifier sort of like the same way you can have multiple A records for a domain name. If the location layer session times out then it would try the other locators listed (pick a method of selection) and if it suceeded would resume the session transparent to the identifier layer. Design the timeout and retransmit algorithm and parameters to achieve the convergence times of your choice. You would need a new protocol stack on the hosts at both ends of connections. By common convention classic TCP hosts could be told to use one of the locators (a transition hack, or just run the protocols in parallel). No change would be required to the network, and existing TCP could continue to be supported (no flag day). Of course support of this new protocol would be limited to the clients and servers that chose to implement it, however this is no less than the change required for IPv6 which some hoped would solve the multihoming problem (possibly defined as scalably supporting network topology change without sessions being interrupted). Mike. +- H U R R I C A N E - E L E C T R I C -+ | Mike Leber Direct Internet Connections Voice 510 580 4100 | | Hurricane Electric Web Hosting Colocation Fax 510 580 4151 | | [EMAIL PROTECTED] http://www.he.net | +---+
Re: And Now for Something Completely Different (was Re: IPv6 news)
Tony Li wrote: How is a split between locator / identifier any different logicaly from the existing ipv4 source routing? IPv4 source routing, as it exists today, is an extremely limited mechanism for specifying waypoints along the path to the destination. IOW the end stations were supposed to be able to tell eachother how to route to eachother. Obviously that does not work in todays internet. But that was a seperation between the endpoints ID and the routing of the packet. This is completely orthogonal to a real identifier/locator split, which would divide what we know of as the 'address' into two separate spaces, one which says where the node is, topologically, and one which says who the node is. One might use the identifier in the TCP pseudo-header, but not the locator, for one example, immediately allowing both mobility and multi-homing. Do you mean adding a second address space to be used by all l3 protocols? Or adding a second address space for every L3 protocol? Or adding a layer 2.5 address space? That appears to be what shim6 is. Also my original question -- How do I send my packet to the other node? I cant just address my packet to the ID, I have to use either information supplied by that node or by a third party. Source routing or routing tables. If this decoupling depends on inband negotiated information, than this allows survivability, but it is not multihoming where multihoming is described as what we do now. Tony
Re: And Now for Something Completely Different (was Re: IPv6 news)
Mike Leber wrote: On Sun, 16 Oct 2005, Joe Maimon wrote: For example, if your goal was to have TCP-like sessions between identifiers survive network events without globally propagating full network topology information about your site (the gripe against classic IPv4 BGP) you could have multiple locators associated with any single identifier sort of like the same way you can have multiple A records for a domain name. Real world shows that that doesnt work very well. Multiple A records is not usuable practicaly speaking for anything other than load balancing, today. If the location layer session times out then it would try the other locators listed (pick a method of selection) and if it suceeded would resume the session transparent to the identifier layer. Design the timeout and retransmit algorithm and parameters to achieve the convergence times of your choice. DNS is a good example of something that was designed that way, but few people rely on common implementations actualy performing it properly. You would need a new protocol stack on the hosts at both ends of connections. By common convention classic TCP hosts could be told to use one of the locators (a transition hack, or just run the protocols in parallel). No change would be required to the network, and existing TCP could continue to be supported (no flag day). Appears to me thats what shim6 is (cursory reading + nanog discussions) Of course support of this new protocol would be limited to the clients and servers that chose to implement it, however this is no less than the change required for IPv6 which some hoped would solve the multihoming problem (possibly defined as scalably supporting network topology change without sessions being interrupted). Long story short, seperating endpoint/locator does nothing to allow multiple paths to a single IP6 address/prefix to scale.
Re: And Now for Something Completely Different (was Re: IPv6 news)
On Sun, 16 Oct 2005, Joe Maimon wrote: Mike Leber wrote: On Sun, 16 Oct 2005, Joe Maimon wrote: For example, if your goal was to have TCP-like sessions between identifiers survive network events without globally propagating full network topology information about your site (the gripe against classic IPv4 BGP) you could have multiple locators associated with any single identifier sort of like the same way you can have multiple A records for a domain name. Real world shows that that doesnt work very well. Multiple A records is not usuable practicaly speaking for anything other than load balancing, today. You are missing the point. Currently multihomed sites have multiple path entries in the routing table for a specific multihomed prefix. Instead of having multiple paths, you would have multiple location records in DNS. (Which are A records and any possible reordering by round robbin is not part of the actual path selection algorithm and which are associated with indentifiers though a standard to be designed as part of the new protocol.) The process of how you select which one to use (and what knobs you have for tuning) is a design decision, the same way it was with BGP and OSPF. If the location layer session times out then it would try the other locators listed (pick a method of selection) and if it suceeded would resume the session transparent to the identifier layer. Design the timeout and retransmit algorithm and parameters to achieve the convergence times of your choice. DNS is a good example of something that was designed that way, but few people rely on common implementations actualy performing it properly. BGP and OSPF have timeouts and select other paths. They give you the convergence you have now in the event of router failure. For example, a BGP session with a peer accross a public exchange has to time out, your interface is still up. Yes, you have to engineer the protocol for the convergence time you want. Define the goal and figure out the algorithm to achieve it. You would need a new protocol stack on the hosts at both ends of connections. By common convention classic TCP hosts could be told to use one of the locators (a transition hack, or just run the protocols in parallel). No change would be required to the network, and existing TCP could continue to be supported (no flag day). Appears to me thats what shim6 is (cursory reading + nanog discussions) Perhaps a shim6 advocate will explain the differences. Does shim6 provide separate identifiers from locators? Does shim6 require new protocol stacks on the hosts at both ends of a session? (If not then the source is not making its own path selection decisions.) Of course support of this new protocol would be limited to the clients and servers that chose to implement it, however this is no less than the change required for IPv6 which some hoped would solve the multihoming problem (possibly defined as scalably supporting network topology change without sessions being interrupted). Long story short, seperating endpoint/locator does nothing to allow multiple paths to a single IP6 address/prefix to scale. Um, this is equivalent to saying it doesn't work because I say so. How doesn't it work? For example you could claim (and then try to defend your claim): * It can't possibly converge quick enough because the genius that went into BGP and OSPF was lost and can never be found again. * (ok seriously) It can't converge quick enough because the timeout would have to be X and based on a guestimate of network topology entropy that would result in Y percent more traffic as each host tries to reestablish locator sessions. (Well, then define what percentage of sessions you think get interruped and support your claim.) * You throw away real topology information and rely on latency (or whatever), and using latency doesn't work because it doesn't allow traffic engineering according to policy. (Who said you have to give everybody the same set of locators? Paul might say e. FWIW, if you want the ability to tell different peers different answers like with BGP you will need the ability to give different answers with the new protocol.) Mike. +- H U R R I C A N E - E L E C T R I C -+ | Mike Leber Direct Internet Connections Voice 510 580 4100 | | Hurricane Electric Web Hosting Colocation Fax 510 580 4151 | | [EMAIL PROTECTED] http://www.he.net | +---+
Re: And Now for Something Completely Different (was Re: IPv6 news)
On Sun, 16 Oct 2005, Mike Leber wrote: Does shim6 require new protocol stacks on the hosts at both ends of a session? (If not then the source is not making its own path selection decisions.) As I understood it, shim6 is a way for two hosts to communicate between each other that they have multiple IPv6 addresses. So if a timeout occurs to the last used address, you can try another and try to resume the communication. So if the web-server has two different IP:s (from two different providers), both would be in DNS (preferrably) and the TCP session would be established with one of them. If shim6 detects that the original path is broken, it will try to use another and if it succeeds, the application won't notice anything as shim6 will abstract this to the TCP layer. I think this is a really good idea, having the network know about all multihomed companies just doesn't scale. With less prefixes and less AS numbers, network convergance would be much better. Think in the future, do we really want routers that'll handle millions of prefixes and hundreds of thousands of AS numbers, just because people want resiliance? If this can be solved on the end-user layer instead, it's more scalable. I can also see a loadbalancing scheme coming out on top of shim6 that'll be usable to end users as well. -- Mikael Abrahamssonemail: [EMAIL PROTECTED]
Re: And Now for Something Completely Different (was Re: IPv6 news)
Think in the future, do we really want routers that'll handle millions of prefixes and hundreds of thousands of AS numbers, just because people want resiliance? Something will have to provide it and I don't want it to be each of my hosts. I'd rather the hundreds of hosts handle payload and a few proper network devices handle where to move it If this can be solved on the end-user layer instead, it's more scalable. Sadly most end users will have no idea and cause more trouble that it's worth if hosts do it brandon
Re: And Now for Something Completely Different (was Re: IPv6 news)
Forgot to subscribe to nanog-post first time round... Forwarded Message On Sun, 2005-10-16 at 05:31 -0400, Joe Maimon wrote: Long story short, seperating endpoint/locator does nothing to allow multiple paths to a single IP6 address/prefix to scale. I may be wrong - I haven't been following shim6 for long, but to me it does appear to scale because it is moving the host identity problem from being an IP address that exists in a single long prefix in the core routing table to being an identifier (128 bit number) which maps to a number of existing IP addresses which each already live in much shorter prefixes in the core routing tables. i.e. move the problem from the core to the edge nodes. Those edge node only need to locator lookup tables for the hosts they are talking to - not all that they may talk to. e.g. Say there is a host a::1 and my server has 3 IP addresses b::1, c::1 and d::1, via service providers B, C and D. As it stands, obviously a::1 can talk directly to the server using any of the addresses. Now, say I want to multi-home. Obviously in the past, I would have gotten my own prefix, say e:: and ASN and announced it. But now with shim6, I could use e::1 as the identifier for my host and use b::1, c::1 and d::1 as the locators. So now someone requests a for my host and gets back e::1. Now when a::1 tries to connect to e::1, the shim does a lookup and sees three possible locators - b::1, c::1 and d::1. It selects one of them and packets are then routed between a::1 and one of b::1, c::1 and d::1 in the same way that they would today. How are there traffic engineering problems when at the end of the day the packets will still be routed in the same way? Or am I missing some crucial point? Regards, John
Re: And Now for Something Completely Different (was Re: IPv6 news)
On 16-Oct-2005, at 03:37, David Conrad wrote: Shifting the NAT to end system removed the objection to NAT, tho it's not entirely clear why. Shifting NAT to the end system also happened to simplify the entire solution as well. Except for the part about having to rewrite all existing implementations to take full advantage of the technology. Thought experiment: how many different software vendors need to change their shipping IPv6 code in order for some new feature like shim6 to be 80% deployed in the server and client communities of hosts? I'm thinking it's probably less than 5, but I'd be interested to hear opinions to the contrary. Joe
Re: And Now for Something Completely Different (was Re: IPv6 news)
GSE also has a direct impact on all implementations (e.g., only use the identifier bits in the TCP pseudo-header, so that is also an all- implementations change. Further, that is a flag day, worldwide, even for non-multi-homed sites. a flag day only for the very small number of ipv6 sites pay me now or pay me later randy
Re: And Now for Something Completely Different (was Re: IPv6 news)
# ... # # Obviously, some of the disadvantages of such an approach would be that it # would require both ends to play and end users wouldn't be able to # traceroute. I'm sure there are many other disadvantages as well. ... ok, so here's the problem. we don't have what the iab thinks of as end-to-end and we have not had it for a long time and it's not coming back under any circumstances. but the people willing to serve on the iab, as filtered down to the set of people willing to be put on the iab by any particular nomcom, do not believe this, or they believe it but they behave like a supreme court nominee who gets an inevitable question about roe-v-wade and their knee jerks and they say i support the constitution. so even though NAT is here to stay and firewalls are here to stay and proxies are here to stay and most ipv6 deployment by the end of its useful lifetime will have used RFC1918-like private addressing, or be behind firewalls that limit flows to what a security administrator can predict and protect and understand... officially the IAB can never, ever recognize this or act on it or make decisions or interpretations or recommendations based on it. that's how politics just is and our proper course is to build and deploy technology that works even if it goes against what the IAB has writ and seems a little bit subversive at the time it comes out (as with firewalls, and NAT), and let the political world play catch-up to the real world that we actually live in. # However, if an approach like this would be technically feasible (and I'm not # entirely sure it would be), I suspect it would get deployed _much_ faster # than an approach that requires every network stack to be modified. Again. # Particularly given the number of folks who care about multi-homing are so # small relative to the number of folks on the Internet. right. # Can two evils make a good? :-) definitely.
Re: And Now for Something Completely Different (was Re: IPv6 news)
# ... # # You are missing the point. # # Currently multihomed sites have multiple path entries in the routing table # for a specific multihomed prefix. # # Instead of having multiple paths, you would have multiple location records # in DNS. (Which are A records and any possible reordering by round robbin is # not part of the actual path selection algorithm and which are associated # with indentifiers though a standard to be designed as part of the new # protocol.) # # The process of how you select which one to use (and what knobs you have for # tuning) is a design decision, the same way it was with BGP and OSPF. yes. we called this the A6/DNAME/SRV approach. # * You throw away real topology information and rely on latency (or # whatever), and using latency doesn't work because it doesn't allow traffic # engineering according to policy. (Who said you have to give everybody the # same set of locators? Paul might say e. ... some applications need best-latency. others, best-isochrony. still others, highest-bandwidth. i don't think the network should have a single policy, but i don't think every application should have to test for latency, isochrony, and bandwidth toward each potential endpoint before it selects one. can't we collect this information as hint state and make it available to applications who will then make their decision privately? or failing that, just use SRV? # FWIW, if you want the ability to tell different peers different answers like # with BGP you will need the ability to give different answers with the new # protocol.) that's how i envision that the NSID option would be used from the client side, though i recognize that this will make ed lewis's head explode, that's OK too.
Re: And Now for Something Completely Different (was Re: IPv6 news)
On Sun, 16 Oct 2005 10:55:38 EDT, Joe Abley said: Thought experiment: how many different software vendors need to change their shipping IPv6 code in order for some new feature like shim6 to be 80% deployed in the server and client communities of hosts? I'm thinking it's probably less than 5, but I'd be interested to hear opinions to the contrary. Client end, if Microsoft, MacOS X, and the various Linuxoids shipped, you'd have pretty good coverage. Maybe Solaris 11 if they're still relevant by then. A few vendor Unixoids (AIX, Irix, etc), and proprietary systems (z/OS), but those vendors will either read the writing on the wall or fade away... Router end, probably same number - Cisco, Juniper, Linksys and a few other SOHO-class vendors, plus a few I've overlooked. The number is certainly more than 5, very likely close to 1 dozen, unlikely to be more than 2 dozen. Of course, even if everybody shipped a *working* *interoperable* product today (quit giggling - we're being hypothetical here), you'd still have a 3-5 year timeframe before all the stuff sold yesterday and years previous got upgraded or replaced. pgpYMQVSiuBH0.pgp Description: PGP signature
Re: And Now for Something Completely Different (was Re: IPv6 news)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 16-Oct-2005, at 16:20, [EMAIL PROTECTED] wrote: On Sun, 16 Oct 2005 10:55:38 EDT, Joe Abley said: Thought experiment: how many different software vendors need to change their shipping IPv6 code in order for some new feature like shim6 to be 80% deployed in the server and client communities of hosts? I'm thinking it's probably less than 5, but I'd be interested to hear opinions to the contrary. Client end, if Microsoft, MacOS X, and the various Linuxoids shipped, you'd have pretty good coverage. Maybe Solaris 11 if they're still relevant by then. A few vendor Unixoids (AIX, Irix, etc), and proprietary systems (z/ OS), but those vendors will either read the writing on the wall or fade away... To get 80%, I think on the client side you just need support in Microsoft v6-capable operating systems. Router end, probably same number - Cisco, Juniper, Linksys and a few other SOHO-class vendors, plus a few I've overlooked. I don't think you need any support at all on routers for shim6 to be functional for services that users and content providers care about. On the server side, windows plus Solaris plus linux seems like it might give 80%. So, that makes windows, Solaris and Linux. Whether the answer is three, five or twelve, the point I was attempting to make was that it's not necessarily a huge deployment obstacle to roll out shim6 across a good proportion of the network's hosts from the coding point of view. Since no flag day is required, this does not seem necessarily unmanageable. Of course, even if everybody shipped a *working* *interoperable* product today (quit giggling - we're being hypothetical here), you'd still have a 3-5 year timeframe before all the stuff sold yesterday and years previous got upgraded or replaced. Sure. In some ways it's fortunate that Microsoft has yet to ship an operating system where v6 is turned on by default (right? I heard that Vista will be the first?) On the server side there's the additional upgrade carrot that upgrades will facilitate multi-homing, which may make for an easier sale. Anyway. Thought experiment. Joe -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (Darwin) iD8DBQFDUs2a/f+PWOTbRPIRAsBUAJ0XSgeNfOsVJDt5kOyb0YReiwYnowCgxGCK yb5mI6ijhwUFKTwrZ2OSQyw= =F648 -END PGP SIGNATURE-
Re: And Now for Something Completely Different (was Re: IPv6 news)
On Sun, 16 Oct 2005, Mikael Abrahamsson wrote: Think in the future, do we really want routers that'll handle millions of prefixes and hundreds of thousands of AS numbers, just because people want resiliance? If this can be solved on the end-user layer instead, it's more you are getting these anyway, thank network convergence for that... or curse it, your call. things like 2547 'vpn' and the like are driving prefix numbers up regardless of what the Internet is doing. Hardware will be required to handle million(s) of prefixes sooner than large scale v6 deployment IMHO. Note, just cause it's there (or will be) doesn't mean I'm advocating that solution for v6, I just had questions (and thought others might as well) about shim6 or the direction of 'site multihoming' in v6... (not even specificly shim6)
Re: And Now for Something Completely Different (was Re: IPv6 news)
On Mon, 17 Oct 2005, Christopher L. Morrow wrote: you are getting these anyway, thank network convergence for that... or curse it, your call. things like 2547 'vpn' and the like are driving prefix numbers up regardless of what the Internet is doing. Hardware will be required to handle million(s) of prefixes sooner than large scale v6 deployment IMHO. Both MPLS and any tunneled VPN over IP means the core won't have to know about all those prefixes (think aggregation of addresses regionally in the IP case and outer label in the MPLS case). So if you're building a 100G capable platform that'll do IPv6 and MPLS, how much difference would it be if you only had to support 16000 labels and 16000 IPv6 prefixes, rather than 2 million? Then of course I guess the argument can be made to put everything on MPLS to avoid the core knowing about anything but outer labels. -- Mikael Abrahamssonemail: [EMAIL PROTECTED]
Re: And Now for Something Completely Different (was Re: IPv6 news)
On Mon, 17 Oct 2005, Mikael Abrahamsson wrote: On Mon, 17 Oct 2005, Christopher L. Morrow wrote: you are getting these anyway, thank network convergence for that... or curse it, your call. things like 2547 'vpn' and the like are driving prefix numbers up regardless of what the Internet is doing. Hardware will be required to handle million(s) of prefixes sooner than large scale v6 deployment IMHO. Both MPLS and any tunneled VPN over IP means the core won't have to know about all those prefixes (think aggregation of addresses regionally in the IP case and outer label in the MPLS case). 'core' doesn't matter so much, somewhere there has to be this knowledge... Perhaps you'll get lucky with some 'edge' devices not having to know about every destination, but I think that might be more rare than you'd like. So if you're building a 100G capable platform that'll do IPv6 and MPLS, how much difference would it be if you only had to support 16000 labels and 16000 IPv6 prefixes, rather than 2 million? not sure, I'm a chemical engineer :) Seriously though, is the break 1-2M or is it 10-20m? (doubling or orders of magnitude?) Then of course I guess the argument can be made to put everything on MPLS to avoid the core knowing about anything but outer labels. oh yes, mpls everywhere! wait... did I say that? yuck.
