Re: Open Source BGP Route Optimization?
Andrew - Supernews [EMAIL PROTECTED] wrote: Per == Per Gregers Bilse [EMAIL PROTECTED] writes: Per But that wasn't really the point. If I telnet to all border Per routers and do 'sh ip b' I can get all tables too; likewise if I Per have a starting point and do a lot of LS traceroutes; and maybe Per even via SNMP (haven't checked what various MIBs support). You can get the received routes via SNMP. I've done this manually on occasion for the purposes of doing what-if analysis of potential traffic plans - take a dump of all available external routes via SNMP, apply to that the proposed policy with regard to selecting the best route, then correlate the resulting route choices with known traffic statistics to determine the resulting utilisation levels of each external link. This has proven useful in a number of situations where radical changes to external routing were being made, to avoid unexpectedly overloading particular links. I would had liked to had been able to have done such things in the past. I was thinking about having a go at writing something, but it's not like I have enough time as it is, and our network isn't really big enough to warrant it. Though I am interested in what tools already exist to support this (more out of curiousity than need). A quick google search threw up a couple of research papers and IP/MPLSView (www.wandl.com), but I presume others on this list will know of more? Sam
Re: Open Source BGP Route Optimization?
Bruce Pinsky [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Per Gregers Bilse wrote: On May 28, 10:37am, Sam Stickland [EMAIL PROTECTED] wrote: Are there any BGP extensions that would cause a BGP speaker to foward all of it's paths, not just it best? I believe quagga had made some recent attempts It has been discussed and been on wish lists, but: And as I said in my other post, there were two drafts submitted that never went anywhere. in this direction. IIRC the problem isn't to do with the route annoucements, it's the route withdrawals. I believe BGP only specifies the prefix being withdrawn and not the path, so if it's advertised multiple paths to a prefix it's impossible to know which has been withdrawn. But the optimizing device is in need of receiving all potential paths from the border routers. Essentially, it needs a complete picture of all viable paths, not just the best that each border has. It would not be advertising multiple paths. No, that's not what I meant. I simply meant that the optimising device couldn't just be a iBGP peer - it wouldn't get enough information. It occurs to me now that walking the BGP4-MIB could be enough, but I'm wouldn't like to bet on the efficently of detecting prefix withdrawals by constantly rewalking the table! To bring this back on topic, I imagine I'd be happy with a tool that simply identified top traffic flows and automatically provided me with traceroute and ping results. Though admittedly I'm not sure how useful it would end up in real life, it sounds like it could be relatively useful tool in the hands of a someone that understands it. Thinking about the potential problems with it, I wonder if there could be any milage in the idea of preformance beacons: Points at key points in an AS (possibly registered in an RIR) that are garanteed to be useable for prefined traffic metrics. Hmm.. maybe it's late and I haven't had enough coffee - one of the route optimisation does something like this already? Sam
Re: Open Source BGP Route Optimization?
Are there any BGP extensions that would cause a BGP speaker to foward all of it's paths, not just it best? I believe quagga had made some recent attempts in this direction. IIRC the problem isn't to do with the route annoucements, it's the route withdrawals. I believe BGP only specifies the prefix being withdrawn and not the path, so if it's advertised multiple paths to a prefix it's impossible to know which has been withdrawn. Sam Per Gregers Bilse [EMAIL PROTECTED] wrote: At first I wasn't sure what a route optimizer was supposed to do -- the term is rather generic and could have a lot of different interpretations. A multi-path traffic balancing solution in the style of Cisco's OER has to be tightly integrated with the routing infrastructure. Specifically, it needs first hand BGP peer data in order to work reliably. There will be a number of cases where an add-on solution might be able to improve on certain things, but there is one major hurdle: a BGP speaker only forwards its own best paths, so an add-on analyzer might well never learn about alternative paths. The only way for any implementation to reliably learn (all) alternative paths and otherwise maintain routing integrity is by receiving BGP data first hand, ie directly peer with transit providers and other peers. Best, -- Per
Re: Open Source BGP Route Optimization?
Per Gregers Bilse [EMAIL PROTECTED] wrote: On May 28, 10:37am, Sam Stickland [EMAIL PROTECTED] wrote: Are there any BGP extensions that would cause a BGP speaker to foward all of it's paths, not just it best? I believe quagga had made some recent attempts It has been discussed and been on wish lists, but: in this direction. IIRC the problem isn't to do with the route annoucements, it's the route withdrawals. I believe BGP only specifies the prefix being withdrawn and not the path, so if it's advertised multiple paths to a prefix it's impossible to know which has been withdrawn. That is 100% correct, yes. Selective withdrawal is not supported. Another issue is that there isn't much point, as far as regular BGP and routing considerations go. Whichever is the best path for a border router is the best path; telling other routers about paths it will not use serves no (or at best very little) point in this context. Well something came up recently on a transit router. It takes multiple Tier-1 feeds, but management wanted to sell a just MFN to a customer. It's possible to policy route all of their traffic to the MFN interface and only advertise their prefixes to MFN, but not possible to only feed them the MFN routes without starting to use VRFs etc. Of course this is a great perversion of resources ;) Sam
Re: Open Source BGP Route Optimization?
On Fri, 28 May 2004, Bruce Pinsky wrote: But the optimizing device wouldn't be advertising multiple paths. It would be advertising its selected path from all viable paths based on the selection criteria/policy implemented by the user. The optimizing device can then keep track of what it has advertised and withdraw as appropriate/necessary. But how do the edge routers, which are passing on all their paths, pass on withdraws? The edge routers would have resend all routes after a withdraw (which current implementations do not do, obviously). Also, the optimising device would not be BGP compliant, an update of a route is an implicit withdraw, so the optimising device could not know, if given additional UPDATE messages for additional prefixes, whether the edge device still had the previous route or not. Essentially, it's impossible to do right with BGP at the moment. If it is possible, I'd love to hear how having previously tried to implement this (and failed). regards, -- Paul Jakma [EMAIL PROTECTED] [EMAIL PROTECTED] Key ID: 64A2FF6A warning: do not ever send email to [EMAIL PROTECTED] Fortune: If you wish to succeed, consult three old people.
RE: Open Source BGP Route Optimization?
Per Gregers Bilse [EMAIL PROTECTED] wrote: On May 28, 10:37am, Sam Stickland [EMAIL PROTECTED] wrote: Are there any BGP extensions that would cause a BGP speaker to foward all of it's paths, not just it best? I believe quagga had made some recent attempts It has been discussed and been on wish lists, but: in this direction. IIRC the problem isn't to do with the route annoucements, it's the route withdrawals. I believe BGP only specifies the prefix being withdrawn and not the path, so if it's advertised multiple paths to a prefix it's impossible to know which has been withdrawn. That is 100% correct, yes. Selective withdrawal is not supported. Another issue is that there isn't much point, as far as regular BGP and routing considerations go. Whichever is the best path for a border router is the best path; telling other routers about paths it will not use serves no (or at best very little) point in this context. Well something came up recently on a transit router. It takes multiple Tier-1 feeds, but management wanted to sell a just MFN to a customer. It's possible to policy route all of their traffic to the MFN interface and only advertise their prefixes to MFN, but not possible to only feed them the MFN routes without starting to use VRFs etc. Of course this is a great perversion of resources ;) Indeed. Makes me somehow wonder how come that customer did not think of buying transit directly from MFN? KISS, that is. Or am I missing something? mh Sam
Re: Open Source BGP Route Optimization?
Sam, You can get the received routes via SNMP. I've done this manually on occasion for the purposes of doing what-if analysis of potential traffic plans - take a dump of all available external routes via SNMP, apply to that the proposed policy with regard to selecting the best route, then correlate the resulting route choices with known traffic statistics to determine the resulting utilisation levels of each external link. This has proven useful in a number of situations where radical changes to external routing were being made, to avoid unexpectedly overloading particular links. I would had liked to had been able to have done such things in the past. I was thinking about having a go at writing something, but it's not like I have enough time as it is, and our network isn't really big enough to warrant it. Though I am interested in what tools already exist to support this (more out of curiousity than need). C-BGP (http://cbgp.info.ucl.ac.be) developped by Bruno Quoitin, can be used to perform this kind of analysis. It can import the BGP routes in MRTD format, but other formats can be easily converted to the MRTD format. C-BGP allows you to build a model of your network that considers both the IGP, the iBGP and eBGP sessions as well as the BGP routing policies that you use. Each C-BGP router is configured by using a Cisco-like syntax that should be familiar to most network engineers. One the model has been written, you can use it to perform different types of what-if analysis such as : - impact of IGP weight change - impact of the loss of one eBGP session - impact of the addition of a new peer/provider If there are other types of analysis that you would like to be able to perform in your network, let us know... Best regards, Olivier Bonaventure -- CSE Dept. UCL, Belgium - http://www.info.ucl.ac.be/people/OBO/
Re: best effort has problems
GC Date: Fri, 28 May 2004 15:58:06 -0400 GC From: Gordon Cook GC I published a two month issue last weekend with the bottom GC line conclusion that there can be no telecom recovery as GC long as the industry relies solely on the best effort GC business model which I believe is not economically GC sustainable. The PSTN doesn't offer guaranteed end-to-end transmission, and certainly statmuxes based on expected load. Looks like similar capacity planning. Perhaps you refer to latency. Most people don't care as long as HTTP and POP3 latency is good enough -- and server response time is often a substantial consideration. SMTP really isn't picky about latency or jitter. Maybe you mean packet loss. Most everyone here can recall the days of 30% packet loss across congested MAE FDDI fabric, but that went away what seems like eons ago. Eddy -- EverQuick Internet - http://www.everquick.net/ A division of Brotsman Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita _ DO NOT send mail to the following addresses : [EMAIL PROTECTED] -or- [EMAIL PROTECTED] -or- [EMAIL PROTECTED] Sending mail to spambait addresses is a great way to get blocked.
Re: best effort has problems
The PSTN doesn't offer guaranteed end-to-end transmission, and certainly statmuxes based on expected load. Looks like similar capacity planning. The PSTN does guarantee a certain service level, latency, call completion etc. Perhaps you refer to latency. Most people don't care as long as HTTP and POP3 latency is good enough -- and server response time is often a substantial consideration. SMTP really isn't picky about latency or jitter. Latency Jitter are very important when dealing with sound video. Or anything realtime for that matter. The Internet isn't just HTTP, NNTP, SMTP any more. Maybe you mean packet loss. Most everyone here can recall the days of 30% packet loss across congested MAE FDDI fabric, but that went away what seems like eons ago. I remember quite a bit of packet loss when the last series of worms hit
Re: Open Source BGP Route Optimization?
We used such system in Russia for many years, with a few exceptions: -- did not used SNMP (because it is a sux!), used 'ssh/rsh router show ...' commands instead; - not top 10 traffic flows, but top 10 traffic flows + top 10 unusual traffic flows. Worked effectively. To bring this back on topic, I imagine I'd be happy with a tool that simply identified top traffic flows and automatically provided me with traceroute and ping results. Though admittedly I'm not sure how useful it would end up in real life, it sounds like it could be relatively useful tool in the hands of a someone that understands it.
Re: best effort has problems
MC Date: Sat, 29 May 2004 14:26:01 -0400 MC From: Matthew Crocker MC The PSTN does guarantee a certain service level, latency, MC call completion etc. As do many Internet providers. (s/call completion/packet loss/) MC Latency Jitter are very important when dealing with sound MC video. Or anything realtime for that matter. The Internet MC isn't just HTTP, NNTP, SMTP any more. Nitpicking: Latency isn't that important with unidirectional communication. However, VoIP users seem reasonably happy with current latency and jitter -- and the Internet still is _largely_ xxTP, anyway... particularly if one ignores peer-to-peer file- swapping programs. MC I remember quite a bit of packet loss when the last series of MC worms hit As do I, but I consider that an exception, not part of standard operation. Admittedly, that may well be an error on my part, considering the increasing popularity of worms and viruses. The PSTN doesn't have botnets of pwned phones making prank calls. (Further note that MAE FDDI congestion was more frequent than the current malware field days.) However, I see this as a problem of securing machines, not one of best effort delivery. Choke trunks are used to prevent radio call-ins from overloading the PSTN. Perhaps throttling bandwidth using a slow-moving exponential decay, over a long averaging period, is a good idea. One could allow short bursts of line-rate traffic. End-user duty cycles are low. This is what facilitates current levels of statmuxing, and why packet loss skyrockets when many systems try to operate at line rate for extended durations. Eddy -- EverQuick Internet - http://www.everquick.net/ A division of Brotsman Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita _ DO NOT send mail to the following addresses : [EMAIL PROTECTED] -or- [EMAIL PROTECTED] -or- [EMAIL PROTECTED] Sending mail to spambait addresses is a great way to get blocked.
Re: best effort has problems
On Sat, 29 May 2004, Edward B. Dreger wrote: Nitpicking: Latency isn't that important with unidirectional communication. However, VoIP users seem reasonably happy with current latency and jitter -- and the Internet still is _largely_ xxTP, anyway... particularly if one ignores peer-to-peer file- swapping programs. Latency is fine for VOIP as long as you dont interact with the PSTN network, if you want to interact with PSTN then you need echo cancellation if you have high latency on the IP part. Most VOIP applications can handle 40ms jitter, so that's normally no problem unless your local access is full. Packet loss is normally no problem for VOIP if you use a proper (non-telco developed) codec. VOIP is actually better off with high packet loss and low jitter than the other way around (throwing off the old truth that core equipment should have lots of buffers). -- Mikael Abrahamssonemail: [EMAIL PROTECTED]
Re: best effort has economic problems
may I make just a passing observation? From a technology point of view the best effort internet certainly works. Not surprisingly the comments here are primarily debating the finer points of the technology. The point I am making in my report is NOT that the best effort network has technology problems but rather that it has ECONOMIC PROBLEMS. That it might support 2 or 3 players not 2 or 3 HUNDRED. That until companies begin to go chapter seven and vanish, the best effort net will be a black hole that burns up capital because, for many players, the OPERATIONAL expense is more than they get for bandwidth never mind cap-ex. best effort won't go away. many best effort players will. for the time being, best effort bandwidth prices as an absolute commodity cannot sustain networks over the long haul. A network that can deliver QoS the report hypothesizes may be able to attract enough revenue to become profitable. How to to this my group is still discussing. We don't pretend that QoS is easy or any kind of mature collection of technologies, but increasingly it looks as though the industry, if it is ever going to be self sustaining, really needs to look at QoS services and solutions. On Sat, 29 May 2004, Edward B. Dreger wrote: Nitpicking: Latency isn't that important with unidirectional communication. However, VoIP users seem reasonably happy with current latency and jitter -- and the Internet still is _largely_ xxTP, anyway... particularly if one ignores peer-to-peer file- swapping programs. Latency is fine for VOIP as long as you dont interact with the PSTN network, if you want to interact with PSTN then you need echo cancellation if you have high latency on the IP part. Most VOIP applications can handle 40ms jitter, so that's normally no problem unless your local access is full. Packet loss is normally no problem for VOIP if you use a proper (non-telco developed) codec. VOIP is actually better off with high packet loss and low jitter than the other way around (throwing off the old truth that core equipment should have lots of buffers). -- Mikael Abrahamssonemail: [EMAIL PROTECTED] -- = The COOK Report on Internet Protocol, 431 Greenway Ave, Ewing, NJ 08618 USA 609 882-2572 (PSTN) 703 738-6031 (Vonage) Subscription info prices at http://cookreport.com/subscriptions.shtml Report on economic black hole of best effort networks at: http://cookreport.com/13.04.shtml E-mail [EMAIL PROTECTED] =
Re: best effort has economic problems
On Sat, 29 May 2004, Gordon Cook wrote: discussing. We don't pretend that QoS is easy or any kind of mature collection of technologies, but increasingly it looks as though the Tier 1 operators do not do best effort really, at least not in their cores (and they have the SLAs to back it up). They buy hugely expensive top notch gear (Cisco 12000 (and now CRS:s) and Junipers) to get the big packet buffers, the fast reroutes and the full routing table lookups for each packet to avoid the pitfalls of flow forwarding the cheaper platforms have. With the advent of 10GE WAN PHY (Force10, Foundry, Riverstone, Extreme Networks, Cisco 7600) and full L3 lookup for each packet on their newer platforms, we'll see very much cheaper L2/L3 equipment being able to take advantage of existing OC192 infrastructure and that's where I think you'll start to see the real best effort networks operating at. At least the L2/L3 equipment will be much cheaper for the operators choosing this equipment, at approx 1/5 the initial investment of similar capacity 12400 and Juniper equipment. Now, how will this translate in cost compared to DWDM equipment and OPEX part of the whole equation? Well, the bubble effect is still doing fine, so I think we wont see any stability for yet another 2-3 years, I'll definately give you that in your analysis. As long as there is equipment and unused installed capacity left from 2000-2001 out there, the price equation will be skewed compared to what it actually costs to replentish the capacity when you've sold it. -- Mikael Abrahamssonemail: [EMAIL PROTECTED]
Re: best effort has economic problems
Tier 1 operators do not do best effort really, at least not in their cores (and they have the SLAs to back it up). They buy hugely expensive top notch gear (Cisco 12000 (and now CRS:s) and Junipers) to get the big packet buffers, the fast reroutes and the full routing table lookups for each packet to avoid the pitfalls of flow forwarding the cheaper platforms have. When 12016s are on ebay for $12,000, even a low budget tier 3 can afford proper routing gear... It's not as if the Internet is still powered by 7507s! (Well, a large part still is. :-) Now, how will this translate in cost compared to DWDM equipment and OPEX part of the whole equation? I am starting to see some interesting long-distance 2.5Gbps CWDM gear offered by European manufacturers, with 70km and 100km distance ratings. This stuff sells for a fraction of the price of equivalent Nortel/Ciena/Cisco ONS gear. Lots of optics companies are making 70km rated SFPs in 8 or 16 wavelengths now. So far it only runs at OC-48 speeds, but 10Gbps will be coming soon.
ICGCOMM (Intelcom Group) contact?
Could somebody from ICG please contact me off-list?
Re: best effort has economic problems
GC Date: Sat, 29 May 2004 16:53:17 -0400 GC From: Gordon Cook GC The point I am making in my report is NOT that the best GC effort network has technology problems but rather that it has GC ECONOMIC PROBLEMS. That it might support 2 or 3 players not GC 2 or 3 HUNDRED. Best effort is cheaper to provide. Cheaper sells. Is there enough of a market to sustain premium services? IP-based VPNs haven't replaced FR and PtP WAN links, but FR and PtP haven't thwarted IP-based VPNs. GC That until companies begin to go chapter seven and vanish, GC the best effort net will be a black hole that burns up GC capital because, for many players, the OPERATIONAL expense is GC more than they get for bandwidth never mind cap-ex. Definitely true about opex and capex... but I'm not convinced that QoS is the magic bullet that will make the marketplace big enough and profitable enough. I don't see service offerings fixing the woes of screwball pricing. GC best effort won't go away. many best effort players will. If all best effort players provided QoS/guaranteed services, would the survival rate be significantly higher as a result? GC for the time being, best effort bandwidth prices as an GC absolute commodity cannot sustain networks over the long GC haul. A network that can deliver QoS the report hypothesizes GC may be able to attract enough revenue to become profitable. That's where I'm not convinced. Current IP delineates the lower reliability boundary and a benchmark price point. Premium services won't have a lower cost than best-effort, so they must sell for more. Would the incremental service improvements be high enough to draw customers away from cheap BE _and_ support sufficient margins? First class hasn't stopped the cycle of airline bankruptcies and government bailouts. I don't see first class data as much different. GC How to to this my group is still discussing. We don't GC pretend that QoS is easy or any kind of mature collection of GC technologies, but increasingly it looks as though the GC industry, if it is ever going to be self sustaining, really GC needs to look at QoS services and solutions. Perhaps, but only if the price is right. DSL sells better than Internet T1 lines, which sell better than end-to-end private lines and packet clouds. There's a reason for that. Eddy -- EverQuick Internet - http://www.everquick.net/ A division of Brotsman Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita _ DO NOT send mail to the following addresses : [EMAIL PROTECTED] -or- [EMAIL PROTECTED] -or- [EMAIL PROTECTED] Sending mail to spambait addresses is a great way to get blocked.
Re: best effort has economic problems
interesting reading http://mail.internet2.edu:8080/guest/archives/qbone-arch-dt/log200205/msg0.html regards, /vicky Edward B. Dreger wrote: GC Date: Sat, 29 May 2004 16:53:17 -0400 GC From: Gordon Cook GC The point I am making in my report is NOT that the best GC effort network has technology problems but rather that it has GC ECONOMIC PROBLEMS. That it might support 2 or 3 players not GC 2 or 3 HUNDRED. Best effort is cheaper to provide. Cheaper sells. Is there enough of a market to sustain premium services? IP-based VPNs haven't replaced FR and PtP WAN links, but FR and PtP haven't thwarted IP-based VPNs. GC That until companies begin to go chapter seven and vanish, GC the best effort net will be a black hole that burns up GC capital because, for many players, the OPERATIONAL expense is GC more than they get for bandwidth never mind cap-ex. Definitely true about opex and capex... but I'm not convinced that QoS is the magic bullet that will make the marketplace big enough and profitable enough. I don't see service offerings fixing the woes of screwball pricing. GC best effort won't go away. many best effort players will. If all best effort players provided QoS/guaranteed services, would the survival rate be significantly higher as a result? GC for the time being, best effort bandwidth prices as an GC absolute commodity cannot sustain networks over the long GC haul. A network that can deliver QoS the report hypothesizes GC may be able to attract enough revenue to become profitable. That's where I'm not convinced. Current IP delineates the lower reliability boundary and a benchmark price point. Premium services won't have a lower cost than best-effort, so they must sell for more. Would the incremental service improvements be high enough to draw customers away from cheap BE _and_ support sufficient margins? First class hasn't stopped the cycle of airline bankruptcies and government bailouts. I don't see first class data as much different. GC How to to this my group is still discussing. We don't GC pretend that QoS is easy or any kind of mature collection of GC technologies, but increasingly it looks as though the GC industry, if it is ever going to be self sustaining, really GC needs to look at QoS services and solutions. Perhaps, but only if the price is right. DSL sells better than Internet T1 lines, which sell better than end-to-end private lines and packet clouds. There's a reason for that. Eddy -- EverQuick Internet - http://www.everquick.net/ A division of Brotsman Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita _ DO NOT send mail to the following addresses : [EMAIL PROTECTED] -or- [EMAIL PROTECTED] -or- [EMAIL PROTECTED] Sending mail to spambait addresses is a great way to get blocked.
Re: best effort has economic problems
On Sat, 29 May 2004, Eric Kuhnke wrote: When 12016s are on ebay for $12,000, even a low budget tier 3 can afford proper routing gear... It's not as if the Internet is still powered by 7507s! (Well, a large part still is. :-) 12016 will only do OC48 speeds and the OC48 cards that used to be had at $3500-$5000 on eBay now is $10k+ and quickly drying up. Then there is the whole legal and support issue, if you need Cisco support and want to be legal, you can effectively double the above prices. I am starting to see some interesting long-distance 2.5Gbps CWDM gear offered by European manufacturers, with 70km and 100km distance ratings. This stuff sells for a fraction of the price of equivalent Nortel/Ciena/Cisco ONS gear. Lots of optics companies are making 70km rated SFPs in 8 or 16 wavelengths now. So far it only runs at OC-48 speeds, but 10Gbps will be coming soon. At least in Sweden there is still plenty of unlit dark fiber than can be had cheaply. It's not the fiber that'll cost you, it's the rent of floorspace and power in the amplifier stations, plus the manpower OPEX of keeping it running. Still, at prices of $25-40 per meg per month, it's hard to produce new bandwidth even by buying gear on ebay and doing everything the cheap way. Let's say you average 1 gig of paying traffic per month on your OC48 links, that'll only give you let's say $40k revenue per month. Not much to build an operation on. -- Mikael Abrahamssonemail: [EMAIL PROTECTED]
