RIPE NCC publishes case study of youtube.com hijack
for those interested in the matter tom Dear Colleagues, As you may be aware from recent news reports, traffic to the youtube.com website was 'hijacked' on a global scale on Sunday, 24 February 2008. The incident was a result of the unauthorised announcement of the prefix 208.65.153.0/24 and caused the popular video sharing website to become unreachable from most, if not all, of the Internet. The RIPE NCC conducted an analysis into how this incident was seen and tracked by the RIPE NCC's Routing Information Service (RIS) and has published a case study at: http://www.ripe.net/news/study-youtube-hijacking.html The RIPE NCC RIS is a service that collects Border Gateway Protocol (BGP) routing information from roughly 600 peers at 16 Internet Exchange Points (IXPs) across the world. Data is stored in near real-time and can be instantly queried by anyone to provide multiple views of routing activity for any point in time. The RIS forms part of the RIPE NCC's suite of Information Services, which together provide a deeper insight into the workings of the Internet. The RIPE NCC is a neutral and impartial organisation, and commercial interests therefore do not influence the data collected. The RIPE NCC Information Services suite also includes the Test Traffic Measurement (TTM) service, the DNS Monitoring (DNSMON) service and Hostcount. All of these services are available to anyone, and most of them are offered free of charge. More information about RIPE NCC Information Services can be found at: http://is-portal.ripe.net Regards, Daniel Karrenberg Chief Scientist, RIPE NCC
Re: RIPE NCC publishes case study of youtube.com hijack
The report states: Sunday, 24 February 2008, 20:07 (UTC): AS36561 (YouTube) starts announcing 208.65.153.0/24. With two identical prefixes in the routing system, BGP policy rules, such as preferring the shortest AS path, determine which route is chosen. This means that AS17557 (Pakistan Telecom) continues to attract some of YouTube's traffic. It's worth noting that from where I sit, it appears as though none of Youtube's transit providers accepted this announcement. Only their peers. The point is -- Restrictive customer filtering can also bite you in the butt. Trying to require your providers to do a ge 19 le 25 (or whatever your largest supernet is), rather than filters for specific prefix sizes seems a worthwhile endeavor so you can de-aggregate on the fly, as necessary. -David Tom Quilling wrote: for those interested in the matter tom Dear Colleagues, As you may be aware from recent news reports, traffic to the youtube.com website was 'hijacked' on a global scale on Sunday, 24 February 2008. The incident was a result of the unauthorised announcement of the prefix 208.65.153.0/24 and caused the popular video sharing website to become unreachable from most, if not all, of the Internet. The RIPE NCC conducted an analysis into how this incident was seen and tracked by the RIPE NCC's Routing Information Service (RIS) and has published a case study at: http://www.ripe.net/news/study-youtube-hijacking.html The RIPE NCC RIS is a service that collects Border Gateway Protocol (BGP) routing information from roughly 600 peers at 16 Internet Exchange Points (IXPs) across the world. Data is stored in near real-time and can be instantly queried by anyone to provide multiple views of routing activity for any point in time. The RIS forms part of the RIPE NCC's suite of Information Services, which together provide a deeper insight into the workings of the Internet. The RIPE NCC is a neutral and impartial organisation, and commercial interests therefore do not influence the data collected. The RIPE NCC Information Services suite also includes the Test Traffic Measurement (TTM) service, the DNS Monitoring (DNSMON) service and Hostcount. All of these services are available to anyone, and most of them are offered free of charge. More information about RIPE NCC Information Services can be found at: http://is-portal.ripe.net Regards, Daniel Karrenberg Chief Scientist, RIPE NCC
Re: RIPE NCC publishes case study of youtube.com hijack
On Feb 29, 2008, at 7:46 AM, David Ulevitch wrote: The report states: Sunday, 24 February 2008, 20:07 (UTC): AS36561 (YouTube) starts announcing 208.65.153.0/24. With two identical prefixes in the routing system, BGP policy rules, such as preferring the shortest AS path, determine which route is chosen. This means that AS17557 (Pakistan Telecom) continues to attract some of YouTube's traffic. It's worth noting that from where I sit, it appears as though none of Youtube's transit providers accepted this announcement. Only their peers. A simple artifact of shortest AS path route selection. In addition, many providers employ policies that apply preference for prefixes learned from customers over those learned from peers, assuming they're of the same length. Had those same providers explicitly not accepted the /24 announcement from AS 17557 via their peers you wouldn't have been affected at all. The point is -- Restrictive customer filtering can also bite you in the butt. Trying to require your providers to do a ge 19 le 25 (or whatever your largest supernet is), rather than filters for specific prefix sizes seems a worthwhile endeavor so you can de- aggregate on the fly, as necessary. Deaggregation in order to mitigate less specific route hijacking is a hack that in most cases only half fixes the problem, if that. If providers didn't have those policies in place it'd be /32s that were being hijacked and route table growth and churn would be far worse than it already is. You prevent this by ubiquitous deployment of explicit customer and inter- provider prefix filters, you don't open things up more so that when problems occur, folks can try to hack around them. -danny
Re: RIPE NCC publishes case study of youtube.com hijack
On Fri, Feb 29, 2008 at 06:46:15AM -0800, David Ulevitch wrote: The point is -- Restrictive customer filtering can also bite you in the butt. Trying to require your providers to do a ge 19 le 25 (or whatever your largest supernet is), rather than filters for specific prefix sizes seems a worthwhile endeavor so you can de-aggregate on the fly, as necessary. If you support community-based blackholes, your customers want/need to be able to advertise up to /32. At a previous job we defined customer prefix-filters as prefix/mask upto 32 and then applied a reasonable max-prefix setting[1]. This allowed customers to send us a reasonable number of deaggregates for blackholing or TE purposes but protected us from a full-on leak/deaggregation event. Needless to say, each prefix with a mask longer then /24 was tagged with no-export as well, so those longer prefixes weren't propagated beyond our network. [1] We had a limited number of customer buckets... IIRC something like 2500, 5000, 15000, and 25000. That keeps the number of different configurations to a minimum number but still gives adequate protection. --Jeff
Re: RIPE NCC publishes case study of youtube.com hijack
Danny McPherson wrote: On Feb 29, 2008, at 7:46 AM, David Ulevitch wrote: It's worth noting that from where I sit, it appears as though none of Youtube's transit providers accepted this announcement. Only their peers. A simple artifact of shortest AS path route selection. Well, we (youtube and opendns) share some common transit providers -- and so I had expected to see all announcements from one customer to another customer directly downstream from the provider. But you very well could be right. Had those same providers explicitly not accepted the /24 announcement from AS 17557 via their peers you wouldn't have been affected at all. Of course... In fact, wouldn't it even providers benefit from having some logic that says don't ever accept a more specific of a customer-announced prefix? Customers might not like that though... :-) You prevent this by ubiquitous deployment of explicit customer and inter- provider prefix filters, you don't open things up more so that when problems occur, folks can try to hack around them. Like most things, ymmv. -David
Re: RIPE NCC publishes case study of youtube.com hijack
On Feb 29, 2008, at 11:49 AM, David Ulevitch wrote: Of course... In fact, wouldn't it even providers benefit from having some logic that says don't ever accept a more specific of a customer-announced prefix? Sure, that'd suck less, I guess, although then you have to punch holes for multi-homed customers, etc.., which is actually trivial if policy is generated automatically based on RPSL or the like and the policy is registered accordingly. But I still prefer explicit route policy where what an AS is permitted to announce is all that's permitted and any other prefixes are discarded. Any policy that allows lots of more specifics to be announced in case of route hijacking to me is like putting a band-aid on a headache. Customers might not like that though... :-) Right, it's breaks fail-over with multi-homing, in particular. As far as other more specifics for TE and the like, well, they're certainly welcome to register those prefixes such that they can be reflected in routing policies, but this would also ease announcements of unintentional more-specifics. I don't consider this one of those 'YMMV' things. Today, if providers explicitly filter at all they filter customer routes based on some IRR data or other internal database. They may put a few safety nets in place for bogon prefixes and certain prefix length policies or ASNs, or perhaps not accept their own aggregate or more specifics from peers. However, they accept everything else from peers, which means tomorrow, when this happens again, all they can do is get pissed because some monkey on the other side of the world fat-fingered a 2 instead of a 3, or forget to attached a no_advertise, no_export or other explicit non-transit community to a blackhole route .. and now some other site that presumably matters is offline, or half reachable, or whatever... Further, we can keep experiencing more extraneous route table bloat because of folks advertising more specifics of their own aggregates in order to minimize any impact a potential hijacking might have to their own space.. Or, we could start implementing explicit inter-provider filtering. Explicit policy on all inter-domain peers, customer or provider, based on RIR allocations, IRR objects and RPSLish language, and work on removing deployment barriers (e.g., stale IRR data, allocation authentication, IRR update vulnerabilities, router configuration scale and load issues, TTM for newly announced prefixes, etc..), with real deployment likely in an incremental bi-lateral manner between ISPs that employ IRR data for customer route policy today and already have tools to manage and deploy new policy. I challenge providers to step up here, the onus is on you and nothing else is going to make this problem go away.There's tangible incremental benefit to any provider that institutes such a policy, and by it's very nature, the right ISPs will encourage other sites on the Internet to begin employing IRRs and similar mechanisms, if for no reason other than to enable propagation of their own legitimate routes more quickly. -danny
