Re: Backbone IP network Economics - peering and transit
If direct connecting != peering then definitely. Maybe we need to say differentiate between: - Connected transit - Remote transit - Connected peering - Remote peering And agree that, by default, transit ~= remote transit peering ~= direct peering Without getting too complicated. transit is always direct connection to a single AS, and indirect to all others. For simplicity's sake, single-homed customer ASes behind the transit AS are not considered apart from the transit AS. It is indirect for the rest of the internet, including the sum of all peering (read: direct connection without any indirect connections) connectivity. peering is a always direction connection to a single AS and no indirect connections are expected. Again, single-homed customer ASes are considered part of the peering AS. ASes that can only be reached from a single AS can only be reached by those with a direction connection to the upstream AS. --- This model [good or bad] allows people who pay for customer-only routes from a transit provider they can't settlement-free peer with be considered in the same breath as true peers. For technology concerns, I think this is valid. For business reasons there is probably some difference. DJ
Re: Backbone IP network Economics - peering and transit
Deepak Jain wrote: If direct connecting != peering then definitely. Maybe we need to say differentiate between: - Connected transit - Remote transit - Connected peering - Remote peering And agree that, by default, transit ~= remote transit peering ~= direct peering Without getting too complicated. transit is always direct connection to a single AS, and indirect to all others. For simplicity's sake, single-homed customer ASes behind the transit AS are not considered apart from the transit AS. It is indirect for the rest of the internet, including the sum of all peering (read: direct connection without any indirect connections) connectivity. peering is a always direction connection to a single AS and no indirect connections are expected. Again, single-homed customer ASes are considered part of the peering AS. Slight error, there.. While the first always is true, the second statement may not be. customers of peers are visible between two AS's peering. ASes that can only be reached from a single AS can only be reached by those with a direction connection to the upstream AS. --- This model [good or bad] allows people who pay for customer-only routes from a transit provider they can't settlement-free peer with be considered in the same breath as true peers. For technology concerns, I think this is valid. For business reasons there is probably some difference. DJ
Re: Backbone IP network Economics - peering and transit
Hmm. Interesting. I am (here is SFO area) DSL customer and DialUp customer. But I never received a notification from my provider(s), possible with free CD, explaining me (if I am a homewife, not an engineer, of course) what to do and how to prevent a problems. We have a lot of room for improvement. In includes not only viiruses, but wireless (when I work in my friend's home, I use his neighbour WiFi, because it have a bettr quality, withouut /of course/ his even knowing it -:) - or, better to say, his notebook prefer his neighbour, for some reasons), and so on. On Apr 20, 2004, at 10:32 AM, Daniel Golding wrote: On 4/20/04 1:34 AM, Michel Py [EMAIL PROTECTED] wrote: Patrick W.Gilmore wrote: Unless they have cheap access to a free NAP (TorIX, SIX, etc.), transit, even at higher prices, is probably be the best / cheapest way to reach the Internet. This is true, but there are plenty of other opportunities for peering, such as: both parties buy DS-3 class transit from the same tier-2 or even maybe tier-3 provider in a colo (which will likely be a BFM, other problem) not a formal IX. In other words, peering in an IX does cost money, but peering at a colo might not, as these messy colos are mostly unmanaged and nobody cares about that 25ft cross-over cable :-) Michel. This is a classical mistake. Peering always costs money and its never free. Maybe Free is the wrong word. Perhaps No additional cost over transit/whatever. Or, for those of us who think that the time it takes to plug a patch cable into an unused switch port and do some configuration changes are irrelevant, maybe free is the right word. Either way, it is not NEARLY as bad as you or many other people make it out to be. Allow me to explain The question is, how much, and is it cheaper than transit? Costs incurred in peering: - Port Costs (capex) Pth. In many, many cases, especially for smaller providers, this is a spare FE on a switch which already exists. For mid-sized providers, it is frequently a spare GE port on an existing switch, which means perhaps $500 for GBIC or something. For large providers, there is a cost here. But large providers are a different beast, and there is no way a simple e-mail could possibly capture the complexities implicit in peering between Very Large Providers. So we'll let them figure out their own costs. - A share of a router's backplane capacity corresponding to the port Irrelevant. The traffic has to go somewhere, if it does not go out the peering port, it will go out transit, but it is definitely going across the router's backplane. A better thing to put here would be possible use of a router which would not be used. Specifically, if I get a bit in a POP which has transit, I do not have to use the router out at the Peering Point. But how many people have router backpanes which are saturated? At worst you are running out of slots for ports in most cases. (Remember, we left the really big providers to their own devices.) - Cross connect costs (one time or recurring) Largely irrelevant - if you are really going to go out-of-biz for a $150/Month x-conn, you have bigger problems. - Operational costs such as legal review for BLPAs, NOC monitoring, troubleshooting when it flaps, putting MD5 on, etc These costs are frequently quoted as reasons not to peer by the larger providers. Strangely enough, if you are not a Tier 1 (or hoping to be a Tier 1), peering sessions are usually set up and forget. Networks who have 10s of gigabits of traffic but are not looking for reasons to deny peering requests see nearly no cost in these (especially compared to the overall cost of running the network). BLPAs are only required by people who think they mean something. Putting on MD5 is a bug/unique situation, which affect peering perhaps once every half-decade or so. Most small and mid-sized providers can handle the NOC monitoring, trouble shooting, etc. with single-digit-hours a month, max. And sometimes that time is handled by people who are sitting on their ass waiting for something to break anyway. (Read sunk cost.) So, unless you are looking for reasons to *not* peer, these are mostly BS. - Administration Think we covered this one. - Public Switch costs This is a cost and should be considered. Unless, of course, you are at TorIX, SIX, or any of the other very fine free NAPs available. Or if you can x-conn between your rack and someone else's rack in the same colo facility without going to a public switch. Or if you are in a FR or ATM cloud with other providers and can get uber-cheap PVCs between your routers with no additional hardware and a simple configuration change. Or I think you get the point. :) It is difficult to defend peering strategies today unless your network is of a fairly significant size (gigabits of traffic) and you are
RE: Backbone IP network Economics - peering and transit
On Tue, 20 Apr 2004, Michel Py wrote: Stephen J. Wilcox wrote: I assume Vijay meant the cost of a port for private peering, in which case if you private with all your peers and you have a lot of small peers thats going to be a lot of cost for a few kbps of traffic I'm having trouble parsing this. You connect your FE or GE port to an ISL/802.11q trunk to the colo's/IX switch. Then either a)everyone is in the same broadcast domain (dumb but no config), or there's a VLAN on that trunk from/to you to your peer(s). Save for the colo's/IX administrative/xconnect fee, where's the lot of cost? This is private vs public.. Steve
RE: Backbone IP network Economics - peering and transit
where's the lot of cost? Stephen J. Wilcox This is private vs public.. Even if it's private, and assuming that you're clever enough not to peer for a modem's worth of traffic, the cost is a no-brainer, IMHO. Someone checks my math please: At $20 per megabit for transit (which I find very low, but let's go for it anyway) a GE link for peering with an average use of 10% means $24000 per year saved; pays for the xconnect. Michel.
Re: Backbone IP network Economics - peering and transit
where's the lot of cost? Stephen J. Wilcox This is private vs public.. Even if it's private, and assuming that you're clever enough not to peer for a modem's worth of traffic, the cost is a no-brainer, IMHO. Someone checks my math please: At $20 per megabit for transit (which I find very low, but let's go for it anyway) a GE link for peering with an average use of 10% means $24000 per year saved; pays for the xconnect. If you have a gig of traffic to peer out to a single AS, you need quite a bit of infrastructure to support the peering and that infrastructure does not come cheap. Alex
Re: Backbone IP network Economics - peering and transit
[EMAIL PROTECTED] wrote: where's the lot of cost? Stephen J. Wilcox This is private vs public.. Even if it's private, and assuming that you're clever enough not to peer for a modem's worth of traffic, the cost is a no-brainer, IMHO. Someone checks my math please: At $20 per megabit for transit (which I find very low, but let's go for it anyway) a GE link for peering with an average use of 10% means $24000 per year saved; pays for the xconnect. If you have a gig of traffic to peer out to a single AS, you need quite a bit of infrastructure to support the peering and that infrastructure does not come cheap. But that structure doesn't vary vastly if you'd traffic out that gig via transit vs direct connect. It does vary (and add lots of infrastructure) if you don't aggregate your traffic at IXes and instead use loops to bring transit to you instead of going to it. (say a few 100Mb/s or OC3s in a few places instead of a GigE at an IX). Perhaps we should (for technical reasons) describe peering as direct connecting. Business reasons aside, technically the difference is that with transit you are expecting access via indirect connections to networks. With peering you expect direct connections into a network. Deepak Jain AiNET
RE: Backbone IP network Economics - peering and transit
Deepak Jain wrote: But that structure doesn't vary vastly if you'd traffic out that gig via transit vs direct connect. It does vary (and add lots of infrastructure) if you don't aggregate your traffic at IXes and instead use loops to bring transit to you instead of going to it. (say a few 100Mb/s or OC3s in a few places instead of a GigE at an IX). Indeed. Perhaps we should (for technical reasons) describe peering as direct connecting. This makes a lot of sense to me (although I would suggest a different name later). Since the beginning I have been trying to make the point that direct connecting was typically a no-brainer in terms of money. Peering when you have to buy the local loop is not such a slam dunk. Business reasons aside, technically the difference is that with transit you are expecting access via indirect connections to networks. I'm not so sure about this. There are lots of people that buy transit and are directly connected to their provider in an IX for example. With peering you expect direct connections into a network. If direct connecting != peering then definitely. Maybe we need to say differentiate between: - Connected transit - Remote transit - Connected peering - Remote peering And agree that, by default, transit ~= remote transit peering ~= direct peering Michel.
Re: Backbone IP network Economics - peering and transit
On Apr 22, 2004, at 9:29 PM, Michel Py wrote: Deepak Jain wrote: But that structure doesn't vary vastly if you'd traffic out that gig via transit vs direct connect. It does vary (and add lots of infrastructure) if you don't aggregate your traffic at IXes and instead use loops to bring transit to you instead of going to it. (say a few 100Mb/s or OC3s in a few places instead of a GigE at an IX). Indeed. Perhaps we should (for technical reasons) describe peering as direct connecting. This makes a lot of sense to me (although I would suggest a different name later). Since the beginning I have been trying to make the point that direct connecting was typically a no-brainer in terms of money. Peering when you have to buy the local loop is not such a slam dunk. Business reasons aside, technically the difference is that with transit you are expecting access via indirect connections to networks. I'm not so sure about this. There are lots of people that buy transit and are directly connected to their provider in an IX for example. With peering you expect direct connections into a network. If direct connecting != peering then definitely. Maybe we need to say differentiate between: - Connected transit - Remote transit - Connected peering - Remote peering And agree that, by default, transit ~= remote transit peering ~= direct peering Michel. The kind of relative cost dynamics described in this thread leave a measurable geographic imprint on the Internet. Big network operators make deployment decisions explicitly to optimize capex/opex over a relatively short horizon, with proximate peering opportunities often justifying higher upfront costs. Conversely, there are plenty of places where lack of public IX facilities, and/or exploitive metro/regional infrastructure costs make remote interconnection not-economically-viable -- so very little peering or multihoming in general. Regions or countries fitting the latter description typically have very few autonomous networks, because there's really very little be gained from running your own network. Infrastructure (layer2, basic telecom, etc.) was once highly regulated everywhere, and didn't/doesn't really become affordable anywhere unless/until someone in authority compels sharing or underwrites the development of competing infrastructures. I don't think it was just a coincidence that EGP was developed during the same period that Ma Bell was being broken up into regional and national independent operating entities... Voila: The origin and evolving structure of IDR is a product of layer 8/9. There's a time dimension to this dynamic as well, as infrastructure savings belatedly give rise to reduced transit costs; once and future operators jump into and out of the game at different points in the cycle. Anyone else notice how many content providers are now suddenly looking for peering coordinators? It's not because they expect other operators to come to their isolated data center(s)...they are building out, because that's what makes sense for them at this point in the cycle. Now I will go back to hunkering down until SF. Tom
RE: Backbone IP network Economics - peering and transit
The question is too simplistic ... It is not (simply) a matter of small vs. big or being on your own network from source-to-destination. Peering is an enabler ... and gives all an opportunity to share content globally ... kinda' fundamental to the Internet consortium. Is your question, 'Since fiber is so cheap, why doesn't everyone build an autonomous, facilities-based, global Internet network that competes for narrowband/broadband pullers of data and hosting/data centers/etc. for content providers (pulled-fromers or pushers of data)? Gary -Original Message- From: Michel Py [mailto:[EMAIL PROTECTED] Sent: Monday, April 19, 2004 10:46 PM To: Gordon Cook; [EMAIL PROTECTED] Subject: RE: Backbone IP network Economics - peering and transit Peering? Who needs peering if transit can be had for $20 per megabit per second? The smaller guys that don't buy transit buy the gigabit. Michel.
Re: Backbone IP network Economics - peering and transit
On 4/20/04 1:34 AM, Michel Py [EMAIL PROTECTED] wrote: Patrick W.Gilmore wrote: Unless they have cheap access to a free NAP (TorIX, SIX, etc.), transit, even at higher prices, is probably be the best / cheapest way to reach the Internet. This is true, but there are plenty of other opportunities for peering, such as: both parties buy DS-3 class transit from the same tier-2 or even maybe tier-3 provider in a colo (which will likely be a BFM, other problem) not a formal IX. In other words, peering in an IX does cost money, but peering at a colo might not, as these messy colos are mostly unmanaged and nobody cares about that 25ft cross-over cable :-) Michel. This is a classical mistake. Peering always costs money and its never free. The question is, how much, and is it cheaper than transit? Costs incurred in peering: - Port Costs (capex) - A share of a router's backplane capacity corresponding to the port - Cross connect costs (one time or recurring) - Operational costs such as legal review for BLPAs, NOC monitoring, troubleshooting when it flaps, putting MD5 on, etc - Administration - Public Switch costs It is difficult to defend peering strategies today unless your network is of a fairly significant size (gigabits of traffic) and you are collocated in an advantageous location(s). Otherwise, low cost transit is hard to beat. Some of the low cost transit providers will not last, and there WILL be a second round of bankruptcies and consolidations - Googin's words should be heeded. However, if you are multihomed and have sufficient transit diversity (and you don't assume any local loop liabilities), the low cost transit should be exploited while it lasts. -- Daniel Golding Network and Telecommunications Strategies Burton Group
Re: Backbone IP network Economics - peering and transit
On 4/20/04 8:45 AM, Gary Hale [EMAIL PROTECTED] wrote: The question is too simplistic ... It is not (simply) a matter of small vs. big or being on your own network from source-to-destination. Peering is an enabler ... and gives all an opportunity to share content globally ... kinda' fundamental to the Internet consortium. Is your question, 'Since fiber is so cheap, why doesn't everyone build an autonomous, facilities-based, global Internet network that competes for narrowband/broadband pullers of data and hosting/data centers/etc. for content providers (pulled-fromers or pushers of data)? Gary -Original Message- From: Michel Py [mailto:[EMAIL PROTECTED] Sent: Monday, April 19, 2004 10:46 PM To: Gordon Cook; [EMAIL PROTECTED] Subject: RE: Backbone IP network Economics - peering and transit Peering? Who needs peering if transit can be had for $20 per megabit per second? The smaller guys that don't buy transit buy the gigabit. Michel. Gary, Peering is an enabler gives all an opportunity to share content globally fundamental to the Internet consortium This is like the greatest hits compendium collected from various failed 1990's service provider business plans :) People should be careful. Peering is a business/networking arrangement that can save them money (or not). Those who try to imbue it with philosophical significance must be viewed with skepticism. Daniel Golding Network and Telecommunications Strategies Burton Group
Re: Backbone IP network Economics - peering and transit
On Tue, Apr 20, 2004 at 05:15:48AM +, Paul Vixie wrote: Peering? Who needs peering if transit can be had for $20 per megabit per second? anyone whose applications are too important to risk dependency on OPNs (other people's networks). OPNs also carry some of the consumers of your bits and you consume some of theirs. Unless you're peering with every laptop directly, somewhere, somehow, you'll be traveling on OPNs, wether it is dark fiber, a circuit, or a wavelength. Time to bust out the cardinal vs ordinal optimization argument again. option a) getting the best decision for certain (cost $1 million) option b) Getting a decision within the top 5% With probability = 0.99 (cost $1 million/x), In real life, we often settle for such a tradeoff with x=100 to 10,000 Under independent sampling, variance decreases as 1/sqrt(n). Each order of magnitude increase in certainty requires 2 orders of magnitude increase in sampling cost. To go from p=0.99 to certainty (p=0.9) implies a 1,000,000 fold increase in sampling cost. So, instead of creating very nice soundbites like OPNs (which I will be shamelessly appropriating for my own use thank you very much), I suggest we spend a bit more time actually _analysing_ using techniques from operations research as to _what_ gives us the most bang for the buck. Dan golding has it right re: peering not being a philosophy, but rather an _engineering_ decision. I touched upon this at the Great Peering Debate at the NANOG Miami, which was hosted by Bill Norton. /vijay
Re: Backbone IP network Economics - peering and transit
On Apr 20, 2004, at 10:32 AM, Daniel Golding wrote: On 4/20/04 1:34 AM, Michel Py [EMAIL PROTECTED] wrote: Patrick W.Gilmore wrote: Unless they have cheap access to a free NAP (TorIX, SIX, etc.), transit, even at higher prices, is probably be the best / cheapest way to reach the Internet. This is true, but there are plenty of other opportunities for peering, such as: both parties buy DS-3 class transit from the same tier-2 or even maybe tier-3 provider in a colo (which will likely be a BFM, other problem) not a formal IX. In other words, peering in an IX does cost money, but peering at a colo might not, as these messy colos are mostly unmanaged and nobody cares about that 25ft cross-over cable :-) Michel. This is a classical mistake. Peering always costs money and its never free. Maybe Free is the wrong word. Perhaps No additional cost over transit/whatever. Or, for those of us who think that the time it takes to plug a patch cable into an unused switch port and do some configuration changes are irrelevant, maybe free is the right word. Either way, it is not NEARLY as bad as you or many other people make it out to be. Allow me to explain The question is, how much, and is it cheaper than transit? Costs incurred in peering: - Port Costs (capex) Pth. In many, many cases, especially for smaller providers, this is a spare FE on a switch which already exists. For mid-sized providers, it is frequently a spare GE port on an existing switch, which means perhaps $500 for GBIC or something. For large providers, there is a cost here. But large providers are a different beast, and there is no way a simple e-mail could possibly capture the complexities implicit in peering between Very Large Providers. So we'll let them figure out their own costs. - A share of a router's backplane capacity corresponding to the port Irrelevant. The traffic has to go somewhere, if it does not go out the peering port, it will go out transit, but it is definitely going across the router's backplane. A better thing to put here would be possible use of a router which would not be used. Specifically, if I get a bit in a POP which has transit, I do not have to use the router out at the Peering Point. But how many people have router backpanes which are saturated? At worst you are running out of slots for ports in most cases. (Remember, we left the really big providers to their own devices.) - Cross connect costs (one time or recurring) Largely irrelevant - if you are really going to go out-of-biz for a $150/Month x-conn, you have bigger problems. - Operational costs such as legal review for BLPAs, NOC monitoring, troubleshooting when it flaps, putting MD5 on, etc These costs are frequently quoted as reasons not to peer by the larger providers. Strangely enough, if you are not a Tier 1 (or hoping to be a Tier 1), peering sessions are usually set up and forget. Networks who have 10s of gigabits of traffic but are not looking for reasons to deny peering requests see nearly no cost in these (especially compared to the overall cost of running the network). BLPAs are only required by people who think they mean something. Putting on MD5 is a bug/unique situation, which affect peering perhaps once every half-decade or so. Most small and mid-sized providers can handle the NOC monitoring, trouble shooting, etc. with single-digit-hours a month, max. And sometimes that time is handled by people who are sitting on their ass waiting for something to break anyway. (Read sunk cost.) So, unless you are looking for reasons to *not* peer, these are mostly BS. - Administration Think we covered this one. - Public Switch costs This is a cost and should be considered. Unless, of course, you are at TorIX, SIX, or any of the other very fine free NAPs available. Or if you can x-conn between your rack and someone else's rack in the same colo facility without going to a public switch. Or if you are in a FR or ATM cloud with other providers and can get uber-cheap PVCs between your routers with no additional hardware and a simple configuration change. Or I think you get the point. :) It is difficult to defend peering strategies today unless your network is of a fairly significant size (gigabits of traffic) and you are collocated in an advantageous location(s). Otherwise, low cost transit is hard to beat. I think you mean or you are colocated in an advantageous location, not and. If I am in 151 Front street, for a small one-time fee, I can connect to TorIX. The amount of the fee and the time it takes to set up peering is probably in the noise, even for a relatively small provider. Obviously if my entire traffic fits on a T1, things might be different, but I do not need anywhere near a gigabit of traffic to justify peering. You are probably at least an order of magnitude off. In general, Peering is a Good Thing [tm]. It increases performance, can lower costs,
Re: Backbone IP network Economics - peering and transit
On Tue, 20 Apr 2004, Patrick W.Gilmore wrote: In many, many cases, especially for smaller providers, this is a spare FE on a switch which already exists. I assume Vijay meant the cost of a port for private peering, in which case if you private with all your peers and you have a lot of small peers thats going to be a lot of cost for a few kbps of traffic - Operational costs such as legal review for BLPAs, NOC monitoring, troubleshooting when it flaps, putting MD5 on, etc These costs are frequently quoted as reasons not to peer by the larger providers. BLPAs are only required by people who think they mean something. Well theyre a good excuse thats for certain :) But I would say they do mean something.. if you're BigISP-A and you are peering with BigISP-B you want to make sure that continues reliably and that means a formal arrangement. Even if your a small ISP its worthwhile considering a formal arrangement particularly with the larger peers to make sure they dont ditch you without some good notice or that they will upgrade without cost if your traffic increases In general, Peering is a Good Thing [tm]. It increases performance, can lower costs, and might even increase your network reliability. Hmm, we're fairly open on peering and have a bunch of small peers, in fact most of our new peerings are with small peers (small is something like announcing a single /24 and doing almost no traffic). We occasionally see performance problems with these small peers, where they maybe drop the session without warning raising an alarm here or do something screwy with their config and leak or whatever. They also tend to only have one connection, this forces how we route traffic to them, as we're in the process of expanding I really want to have multiple equal paths so that we can be sure the traffic is taking the best way to them. My summary of these points is that I'm seriously considering what our policy will be in the future and for good reason (altho it will undoubtedly continue to be fairly relaxed). If your monthly costs are lower with peering than transit alone, it is probably a good idea to peer and ignore the NOC costs. In some instances I'm willing to pay more for a connection (eg paid peering or costs of backbone circuits) to ensure I'm receiving quality. There are a couple other issues not raised... One is the cost on the router in terms of memory and cpu of maintaining such a large number of sessions (usually less of an issue with your big multiprocessor routers) The other is our new hot topic of security, not sure if anyone has thought of this yet (or how interesting it is) but the nature of the bgp attack means that if you can view a BGP session you can figure things about a peer that would otherwise be hidden from you in particular the port numbers in use.. and I'm not entirely clear on the details but it sounds like when you hit the first session, you can take the rest out very easily. We cant take BGP out of band (yet!), perhaps we can keep it better hidden from view tho.. Steve
RE: Backbone IP network Economics - peering and transit
Daniel, That is way too cynical ... and does not address the question of whether building your own transport ever runs counter to the Internet as a consortium. There are business justifications that underpin peering relationships ... and they are based on understanding (or ... philosophy) Gary -Original Message- From: Daniel Golding [mailto:[EMAIL PROTECTED] Sent: Tuesday, April 20, 2004 10:36 AM To: Gary Hale; Michel Py; Gordon Cook; [EMAIL PROTECTED] Subject: Re: Backbone IP network Economics - peering and transit On 4/20/04 8:45 AM, Gary Hale [EMAIL PROTECTED] wrote: The question is too simplistic ... It is not (simply) a matter of small vs. big or being on your own network from source-to-destination. Peering is an enabler ... and gives all an opportunity to share content globally ... kinda' fundamental to the Internet consortium. Is your question, 'Since fiber is so cheap, why doesn't everyone build an autonomous, facilities-based, global Internet network that competes for narrowband/broadband pullers of data and hosting/data centers/etc. for content providers (pulled-fromers or pushers of data)? Gary -Original Message- From: Michel Py [mailto:[EMAIL PROTECTED] Sent: Monday, April 19, 2004 10:46 PM To: Gordon Cook; [EMAIL PROTECTED] Subject: RE: Backbone IP network Economics - peering and transit Peering? Who needs peering if transit can be had for $20 per megabit per second? The smaller guys that don't buy transit buy the gigabit. Michel. Gary, Peering is an enabler gives all an opportunity to share content globally fundamental to the Internet consortium This is like the greatest hits compendium collected from various failed 1990's service provider business plans :) People should be careful. Peering is a business/networking arrangement that can save them money (or not). Those who try to imbue it with philosophical significance must be viewed with skepticism. Daniel Golding Network and Telecommunications Strategies Burton Group
Re: Backbone IP network Economics - peering and transit
On Apr 20, 2004, at 2:15 PM, Stephen J. Wilcox wrote: On Tue, 20 Apr 2004, Patrick W.Gilmore wrote: In many, many cases, especially for smaller providers, this is a spare FE on a switch which already exists. I assume Vijay meant the cost of a port for private peering, in which case if you private with all your peers and you have a lot of small peers thats going to be a lot of cost for a few kbps of traffic It was Dan, not Vijay. And clearly we are not talking about running a pair of fiber to everyone who has a modem's worth of traffic. He mentioned the cost of the port. I said many people have spare FEs / GEs on existing switches. And if they do not, a few hundred dollars will get them one. - Operational costs such as legal review for BLPAs, NOC monitoring, troubleshooting when it flaps, putting MD5 on, etc These costs are frequently quoted as reasons not to peer by the larger providers. BLPAs are only required by people who think they mean something. Well theyre a good excuse thats for certain :) But I would say they do mean something.. if you're BigISP-A and you are peering with BigISP-B you want to make sure that continues reliably and that means a formal arrangement. Even if your a small ISP its worthwhile considering a formal arrangement particularly with the larger peers to make sure they dont ditch you without some good notice or that they will upgrade without cost if your traffic increases I specifically left out BigISP-*. The complexities of peering on a Tier 1 network are not really describable in a single e-mail. As for the smaller ISPs, read every peering agreement you've signed. They all say they can cancel with at most 30 days notice, for no reason, with no recourse, and nothing you can do about it. Furthermore, many include the ability to shut down peering if they even *think* you are doing something funny, and again you have no recourse. Peering agreements are not worth anything to keep peering up. They are only worth something if you are worried about the peer doing something like pointing default. In general, Peering is a Good Thing [tm]. It increases performance, can lower costs, and might even increase your network reliability. Hmm, we're fairly open on peering and have a bunch of small peers, in fact most of our new peerings are with small peers (small is something like announcing a single /24 and doing almost no traffic). The second number there is important, the first is not. There are peers which announce a /24 or few and have gigabits of traffic. We occasionally see performance problems with these small peers, where they maybe drop the session without warning raising an alarm here or do something screwy with their config and leak or whatever. Nowhere was I saying it is a good idea to peer with someone who hurts your network. But most of the peers, even the small ones, can keep their network stable. They also tend to only have one connection, this forces how we route traffic to them, as we're in the process of expanding I really want to have multiple equal paths so that we can be sure the traffic is taking the best way to them. Perfectly valid concern. Which is why I specifically told people to find out who would peer with them before paying to go to a peering point. Don't count your chickens until they're hatched and all that. :) My summary of these points is that I'm seriously considering what our policy will be in the future and for good reason (altho it will undoubtedly continue to be fairly relaxed). And I see nothing you mentioned which in any way goes against what I was saying. Your particular situation is very different than the next networks, as the next networks is unique to that network, etc. But that doesn't make peering bad. If your monthly costs are lower with peering than transit alone, it is probably a good idea to peer and ignore the NOC costs. In some instances I'm willing to pay more for a connection (eg paid peering or costs of backbone circuits) to ensure I'm receiving quality. It is nice to ensure quality. But if quality is your primary goal, then directly peering with a network will give you better quality from an end user (read paying customer) PoV than transit in most cases. Extra latency is usually not viewed as better quality. If you are worried about the connection being flaky, well, like I said, don't peer with flaky networks. Besides, most small to medium guys have enough headroom on their transit connections to take down many of their peers and push it over transit without congestion. There are a couple other issues not raised... One is the cost on the router in terms of memory and cpu of maintaining such a large number of sessions (usually less of an issue with your big multiprocessor routers) Agreed. But since we are not talking to the one-T1-ISP (which I also said would not fit the model), people probably have enough CPU to handle a few extra BGP sessions.
Re: Backbone IP network Economics - peering and transit
Cynical? Gee, I hope so. Anyone who reads that sort of fluff needs to be cynical. Lack of appropriate cynicism led, in part, to the recent unpleasantness in the telecommunications industry. Words like enabling, leveraging, mindshare, b2b, e-*, i-*, et al, are considered harmful to fruitful operational discussion :) -- Daniel Golding Network and Telecommunications Strategies Burton Group On 4/20/04 2:17 PM, Gary Hale [EMAIL PROTECTED] wrote: Daniel, That is way too cynical ... and does not address the question of whether building your own transport ever runs counter to the Internet as a consortium. There are business justifications that underpin peering relationships ... and they are based on understanding (or ... philosophy) Gary -Original Message- From: Daniel Golding [mailto:[EMAIL PROTECTED] Sent: Tuesday, April 20, 2004 10:36 AM To: Gary Hale; Michel Py; Gordon Cook; [EMAIL PROTECTED] Subject: Re: Backbone IP network Economics - peering and transit On 4/20/04 8:45 AM, Gary Hale [EMAIL PROTECTED] wrote: The question is too simplistic ... It is not (simply) a matter of small vs. big or being on your own network from source-to-destination. Peering is an enabler ... and gives all an opportunity to share content globally ... kinda' fundamental to the Internet consortium. Is your question, 'Since fiber is so cheap, why doesn't everyone build an autonomous, facilities-based, global Internet network that competes for narrowband/broadband pullers of data and hosting/data centers/etc. for content providers (pulled-fromers or pushers of data)? Gary -Original Message- From: Michel Py [mailto:[EMAIL PROTECTED] Sent: Monday, April 19, 2004 10:46 PM To: Gordon Cook; [EMAIL PROTECTED] Subject: RE: Backbone IP network Economics - peering and transit Peering? Who needs peering if transit can be had for $20 per megabit per second? The smaller guys that don't buy transit buy the gigabit. Michel. Gary, Peering is an enabler gives all an opportunity to share content globally fundamental to the Internet consortium This is like the greatest hits compendium collected from various failed 1990's service provider business plans :) People should be careful. Peering is a business/networking arrangement that can save them money (or not). Those who try to imbue it with philosophical significance must be viewed with skepticism. Daniel Golding Network and Telecommunications Strategies Burton Group
RE: Backbone IP network Economics - peering and transit
I disagree ... but sure do appreciate your tone ... :) Regards, Gary -Original Message- From: Daniel Golding [mailto:[EMAIL PROTECTED] Sent: Tuesday, April 20, 2004 4:32 PM To: Gary Hale; Michel Py; Gordon Cook; [EMAIL PROTECTED] Subject: Re: Backbone IP network Economics - peering and transit Cynical? Gee, I hope so. Anyone who reads that sort of fluff needs to be cynical. Lack of appropriate cynicism led, in part, to the recent unpleasantness in the telecommunications industry. Words like enabling, leveraging, mindshare, b2b, e-*, i-*, et al, are considered harmful to fruitful operational discussion :) -- Daniel Golding Network and Telecommunications Strategies Burton Group On 4/20/04 2:17 PM, Gary Hale [EMAIL PROTECTED] wrote: Daniel, That is way too cynical ... and does not address the question of whether building your own transport ever runs counter to the Internet as a consortium. There are business justifications that underpin peering relationships ... and they are based on understanding (or ... philosophy) Gary -Original Message- From: Daniel Golding [mailto:[EMAIL PROTECTED] Sent: Tuesday, April 20, 2004 10:36 AM To: Gary Hale; Michel Py; Gordon Cook; [EMAIL PROTECTED] Subject: Re: Backbone IP network Economics - peering and transit On 4/20/04 8:45 AM, Gary Hale [EMAIL PROTECTED] wrote: The question is too simplistic ... It is not (simply) a matter of small vs. big or being on your own network from source-to-destination. Peering is an enabler ... and gives all an opportunity to share content globally ... kinda' fundamental to the Internet consortium. Is your question, 'Since fiber is so cheap, why doesn't everyone build an autonomous, facilities-based, global Internet network that competes for narrowband/broadband pullers of data and hosting/data centers/etc. for content providers (pulled-fromers or pushers of data)? Gary -Original Message- From: Michel Py [mailto:[EMAIL PROTECTED] Sent: Monday, April 19, 2004 10:46 PM To: Gordon Cook; [EMAIL PROTECTED] Subject: RE: Backbone IP network Economics - peering and transit Peering? Who needs peering if transit can be had for $20 per megabit per second? The smaller guys that don't buy transit buy the gigabit. Michel. Gary, Peering is an enabler gives all an opportunity to share content globally fundamental to the Internet consortium This is like the greatest hits compendium collected from various failed 1990's service provider business plans :) People should be careful. Peering is a business/networking arrangement that can save them money (or not). Those who try to imbue it with philosophical significance must be viewed with skepticism. Daniel Golding Network and Telecommunications Strategies Burton Group
RE: Backbone IP network Economics - peering and transit
Stephen J. Wilcox wrote: I assume Vijay meant the cost of a port for private peering, in which case if you private with all your peers and you have a lot of small peers thats going to be a lot of cost for a few kbps of traffic I'm having trouble parsing this. You connect your FE or GE port to an ISL/802.11q trunk to the colo's/IX switch. Then either a)everyone is in the same broadcast domain (dumb but no config), or there's a VLAN on that trunk from/to you to your peer(s). Save for the colo's/IX administrative/xconnect fee, where's the lot of cost? Michel.
RE: Backbone IP network Economics - peering and transit
Peering? Who needs peering if transit can be had for $20 per megabit per second? The smaller guys that don't buy transit buy the gigabit. Michel.
Re: Backbone IP network Economics - peering and transit
On Apr 19, 2004, at 10:45 PM, Michel Py wrote: Peering? Who needs peering if transit can be had for $20 per megabit per second? The smaller guys that don't buy transit buy the gigabit. Then their traffic will not justify 1000s of $$ per month for lines, racks, and NAP connection. Unless they have cheap access to a free NAP (TorIX, SIX, etc.), transit, even at higher prices, is probably be the best / cheapest way to reach the Internet. OTOH, for the guys who do buy a lot of traffic, a NAP connection might be worth it. For instance, if you have a node in 151 Front Street, it would be silly not to connect to the TorIX for a one-time fee and send free traffic to a lot of good eyeballs in Canada - not to mention the performance benefits. The same might be true of an PAIX / Equinix location. Saying who needs [foo] is not a good question without supplying the other variables. It all depends on your traffic mix, locations, deals you can make with the NAPs, networks who will peer with you, etc. -- TTFN, patrick
Re: Backbone IP network Economics - peering and transit
Peering? Who needs peering if transit can be had for $20 per megabit per second? anyone whose applications are too important to risk dependency on OPNs (other people's networks). -- Paul Vixie
Re: Backbone IP network Economics - peering and transit
On Mon, 19 Apr 2004, Gordon Cook wrote: Peering? Who needs peering if transit can be had for $20 per megabit per second? Isnt the companies still doped from the bubble in 2000-2001? Price example: You have three cities. Two 12400 GSRs per city, and OC192 to connect them, that's a total of 12 $150k ($225k minus rebate) cards and let's say $100k per router for customer facing interfaces etc (unrealistically low). If you want to pay this investment back over three years and let's say you'll push 10gigs of customer paying traffic (because of redundancy etc). You end up with close to $10 per megabit in just equipment fee to cisco so you have half of the money left from the initally stated price of $20 per megabit, this for a small inter-metro network. Since Cisco basically hasnt lowered the price per megabit on any interface cards for the GSR platform, it cannot be used apart from doing very long distance transfer via DWDM where the links are full of revenue-generating traffic all the time. Juniper is even more expensive. We like these platforms, they're very stable and well performing, but I just cannot see where they can be justified investing in at todays megabit price. -- Mikael Abrahamssonemail: [EMAIL PROTECTED]
RE: Backbone IP network Economics - peering and transit
Patrick W.Gilmore wrote: Unless they have cheap access to a free NAP (TorIX, SIX, etc.), transit, even at higher prices, is probably be the best / cheapest way to reach the Internet. This is true, but there are plenty of other opportunities for peering, such as: both parties buy DS-3 class transit from the same tier-2 or even maybe tier-3 provider in a colo (which will likely be a BFM, other problem) not a formal IX. In other words, peering in an IX does cost money, but peering at a colo might not, as these messy colos are mostly unmanaged and nobody cares about that 25ft cross-over cable :-) Michel.
