Re: YouTube IP Hijacking
At 05:31 AM 25-02-08 +, Steven M. Bellovin wrote: Seriously -- a number of us have been warning that this could happen. More precisely, we've been warning that this could happen *again*; we all know about many older incidents, from the barely noticed to the very noisy. (AS 7007, anyone?) Something like S-BGP will stop this cold. Yes, I know there are serious deployment and operational issues. The question is this: when is the pain from routing incidents great enough that we're forced to act? It would have been nice to have done something before this, since now all the world's script kiddies have seen what can be done. we've been warning that this could happen *again* - this is happening every day - just look to: http://cs.unm.edu/~karlinjf/IAR/prefix.php?filter=most http://cs.unm.edu/~karlinjf/IAR/subprefix.php?filter=most for samples. Thing is - these prefix hijacks are not big ticket sites like Youtube or Microsoft or Cisco or even whitehouse.gov - but rather just sites that never make it onto the NANOG radar. -Hank
Re: YouTube IP Hijacking
On Mon, 25 Feb 2008 01:49:51 -0500 (EST) Sean Donelan [EMAIL PROTECTED] wrote: On Mon, 25 Feb 2008, Steven M. Bellovin wrote: How about state-of-the-art routing security? The problem is what is the actual trust model? Are you trusting some authority to not be malicious or never make a mistake? There are several answers to the malicious problem. There are fewer answers to never making a mistake problem. The state of the art routing security proposals let the trusted securely make mistakes. At one time or another, I think every router vendor, every ASN operator, every RIR, and so on has made a mistake at some time. Yeah, I know some of those mistakes may have actually been malicious, but so far the mistakes have outnumbered the malicious. If someone comes up with the anti-mistake routing protocol ... Right. Everyone makes mistakes, but not everyone is malicious.And the RIRs and the big ISPs are *generally* more clueful than the little guys and the newcomers. Note also that secured BGP limits the kinds of mistakes people can make. If I have a certificate from my RIR for 192.0.2.0/24, I can't neither announce 10.0.0.0/8 nor delegate it to you, no matter how badly I type. Secured BGP still strikes me as a net win. --Steve Bellovin, http://www.cs.columbia.edu/~smb
Re: YouTube IP Hijacking
On Feb 25, 2008, at 2:27 AM, Hank Nussbacher wrote: At 07:15 PM 24-02-08 -0500, Randy Epstein wrote: More importantly, why is PCCW not prefix filtering their downstreams? Why? - Lack of clue - Couldn't care less - No revenue Take your pick - or add your own reason. PCCW is not alone. They just happen to be the latest in a long line of ISPs that follow the same rules - their own. All good, er, bad reasons. Fixing the filter your downstreams problem is very important. It would also solve 90-something percent of the problems mentioned in this thread. E.g. as7007. :) -- TTFN, patrick
Re: YouTube IP Hijacking
On Feb 25, 2008, at 2:32 AM, Hank Nussbacher wrote: At 05:31 AM 25-02-08 +, Steven M. Bellovin wrote: Seriously -- a number of us have been warning that this could happen. More precisely, we've been warning that this could happen *again*; we all know about many older incidents, from the barely noticed to the very noisy. (AS 7007, anyone?) Something like S-BGP will stop this cold. Yes, I know there are serious deployment and operational issues. The question is this: when is the pain from routing incidents great enough that we're forced to act? It would have been nice to have done something before this, since now all the world's script kiddies have seen what can be done. we've been warning that this could happen *again* - this is happening every day - just look to: http://cs.unm.edu/~karlinjf/IAR/prefix.php?filter=most http://cs.unm.edu/~karlinjf/IAR/subprefix.php?filter=most for samples. Thing is - these prefix hijacks are not big ticket sites like Youtube or Microsoft or Cisco or even whitehouse.gov - but rather just sites that never make it onto the NANOG radar. How many of those would be stopped with transit providers filtering their downstreams? Which doesn't require rolling out a new technology like SBGP. And, I would argue, if we cannot even get transit providers to filter their downstreams, there is no way in hell we can get transit providers to filter on some RR or doing authentication on individual prefixes. Let's start with the easy stuff. Walk before run and all that. -- TTFN, patrick
Re: YouTube IP Hijacking
On Sun, 24 Feb 2008, Sargun Dhillon wrote: I don't know how large Pakistani Telecom is, but it I bet its not large enough that PCCW should be allowing it to advertise anything. I think you're failing to take into account how multihoming generally works. The real fallacy here is that PCCW/BTN refuses to prefix-list filter their customers, as evidenced by this and past leaks. If something productive can come from today's outage, it would be PCCW beginning to do their part as responsible Internet citizens, given (excuse the pun) peer pressure. I'd also focus on the lessons learned from the un-official IP Hijacking BOF held in San Jose, during which engineers and researchers studied the extent to which obviously-bogus route advertisements propagated across the public Internet. At these events, prefixes such as 1/8 and 100/7 were advertised, and, by Renesys/bgplay/route-views/etc data, accepted by 99% (?) of the internet. IP blocks that were hijacked before (like 146.20/16) were announced with similar outcome. Results were planned to be presented at the next NANOG, but they shouldn't be a surprise to anyone in the industry: nobody filters. Paul Wall
Re: YouTube IP Hijacking
having built an ISP or two in pakistan, PTCL (Pakistan Telecom) is not the sole provider of bandwidth to the country, although it likely carries the bulk of traffic to the country. operationally, there are a number of jurisdictions which filter content and connectivity on a variety of basis. adjusting the BGP announcements is a fairly quick and sure way to hobble connectivity to specific content. although, it is quickly bypassed by shifting the content to other addresses and domain names. i'm sure that this was an accidental leakage, and that appropriate corrections were/are taken in due course. -- Jim Mercer[EMAIL PROTECTED]+971 55 410-5633 I'm Prime Minister of Canada, I live here and I'm going to take a leak. - Lester Pearson in 1967, during a meeting between himself and President Lyndon Johnson, whose Secret Service detail had taken over Pearson's cottage retreat. At one point, a Johnson guard asked Pearson, Who are you and where are you going?
Re: YouTube IP Hijacking
Interesting that (according to Renesys) BT reconnected about 500 networks in Pakistan after the big fibre cut. I wonder if there's any data around that would tell us who filters and who doesn't? On Mon, Feb 25, 2008 at 9:02 AM, Jim Mercer [EMAIL PROTECTED] wrote: having built an ISP or two in pakistan, PTCL (Pakistan Telecom) is not the sole provider of bandwidth to the country, although it likely carries the bulk of traffic to the country. operationally, there are a number of jurisdictions which filter content and connectivity on a variety of basis. adjusting the BGP announcements is a fairly quick and sure way to hobble connectivity to specific content. although, it is quickly bypassed by shifting the content to other addresses and domain names. i'm sure that this was an accidental leakage, and that appropriate corrections were/are taken in due course. -- Jim Mercer[EMAIL PROTECTED]+971 55 410-5633 I'm Prime Minister of Canada, I live here and I'm going to take a leak. - Lester Pearson in 1967, during a meeting between himself and President Lyndon Johnson, whose Secret Service detail had taken over Pearson's cottage retreat. At one point, a Johnson guard asked Pearson, Who are you and where are you going?
Re: YouTube IP Hijacking
On Mon, Feb 25, 2008 at 09:13:23AM +, Alexander Harrowell wrote: Interesting that (according to Renesys) BT reconnected about 500 networks in Pakistan after the big fibre cut. I wonder if there's any data around that would tell us who filters and who doesn't? based on my experience of routing (and de-routing) my own legacy space as well as some RIPE space through PTCL, i know they have procedures in place to restrict what their customers can send to them, so it makes sense that they have a clue as to how to control what they send out. probably fat fingers, and probably fat wobbly fingers in a rush to comply with a government directive. On Mon, Feb 25, 2008 at 9:02 AM, Jim Mercer [EMAIL PROTECTED] wrote: having built an ISP or two in pakistan, PTCL (Pakistan Telecom) is not the sole provider of bandwidth to the country, although it likely carries the bulk of traffic to the country. operationally, there are a number of jurisdictions which filter content and connectivity on a variety of basis. adjusting the BGP announcements is a fairly quick and sure way to hobble connectivity to specific content. although, it is quickly bypassed by shifting the content to other addresses and domain names. i'm sure that this was an accidental leakage, and that appropriate corrections were/are taken in due course. -- Jim Mercer[EMAIL PROTECTED]+971 55 410-5633 I'm Prime Minister of Canada, I live here and I'm going to take a leak. - Lester Pearson in 1967, during a meeting between himself and President Lyndon Johnson, whose Secret Service detail had taken over Pearson's cottage retreat. At one point, a Johnson guard asked Pearson, Who are you and where are you going? -- Jim Mercer[EMAIL PROTECTED]+971 55 410-5633 I'm Prime Minister of Canada, I live here and I'm going to take a leak. - Lester Pearson in 1967, during a meeting between himself and President Lyndon Johnson, whose Secret Service detail had taken over Pearson's cottage retreat. At one point, a Johnson guard asked Pearson, Who are you and where are you going?
Re: YouTube IP Hijacking
Patrick W. Gilmore [EMAIL PROTECTED] wrote On Feb 25, 2008, at 2:27 AM, Hank Nussbacher wrote: At 07:15 PM 24-02-08 -0500, Randy Epstein wrote: More importantly, why is PCCW not prefix filtering their downstreams? Why? - Lack of clue - Couldn't care less - No revenue Take your pick - or add your own reason. PCCW is not alone. They just happen to be the latest in a long line of ISPs that follow the same rules - their own. All good, er, bad reasons. Fixing the filter your downstreams problem is very important. It would also solve 90-something percent of the problems mentioned in this thread. E.g. as7007. :) I am in the APRICOT meeting in Taipei now, and met a guy from PCCW/AS3491. I have showed him this thread, and have suggested 1) validating prefixes from downstreams before accept, and 2) setting an inbound prefix-filter to their downstreams. regards, - Matsuzaki Yoshinobu [EMAIL PROTECTED] - IIJ/AS2497 INOC-DBA: 2497*629
Re: YouTube IP Hijacking
On 25 feb 2008, at 9:14, Paul Wall wrote: I don't know how large Pakistani Telecom is, but it I bet its not large enough that PCCW should be allowing it to advertise anything. I think you're failing to take into account how multihoming generally works. The real fallacy here is that PCCW/BTN refuses to prefix-list filter their customers, as evidenced by this and past leaks. If something productive can come from today's outage, it would be PCCW beginning to do their part as responsible Internet citizens, given (excuse the pun) peer pressure. Well, if they had problems like this in the past, then I wouldn't trust them to get it right. Which means that it's probably a good idea if EVERYONE starts filtering what they allow in their tables from PCCW. Obviously that makes it very hard for PCCW to start announcing new prefixes, but I can't muster up much sympathy for that. So basically, rather than generate routing registry filters for the entire world, generate routing registry filters for known careless ASes. This number should be small enough that this is somewhat doable. (Although better tools for generating the filters would help, I tried generating filters from the RIPE db a few times but I couldn't figure it out using available tools and I didn't have the time to write my own.)
Peering Survey 2008 (http://tinyurl.com/3xoa6g)
As a follow up to the presentations introducing the peering survey at NANOG and APRICOT, we'd like to announce it to the NANOG mailing list in order to get as many people as possible to participate. What is it? - New survey on how people configure peering! - Featuring technical questions on what protocols and features are used - It will take you under 10 minutes to answer 24 easy questions - No questions on facilities, IXs, or policies - No personal or identifying information will be used How does it work? - You take the survey here: http://tinyurl.com/3xoa6g (http://www.surveymonkey.com/s.aspx?sm=hy_2bvVcxxszICLYRFzPAjMw_3d_3d) - Results will be presented at the GPF 3.0 (and future conferences) Example Questions: - Do you use IPv6 unicast BGP peering? - Do you use BFD? - Do you use four byte ASNs (RFC 4893)? - What is the largest frame size you use on peering links? - What is your biggest concern when deploying a new feature? - Note: is it OK to say that you don't know what a feature is What is the purpose of the survey? - Identify trends in feature and protocol deployment - Educate the community on features and best practices Thanks for your participation! Greg Hankins (Force10) Ren Provo (Comcast) Tom Scholl (ATT)
RE: YouTube IP Hijacking
This candidate list of requirements is for route sources that North American Operators should trust to propagate long prefix routes, nothing more, nothing less. All operators already have some kind of criteria which they use to decide whether or not to trust a particular source of routes whether long prefixes or short. You are suggesting that these operators should give up this role to a trusted third party so that al North American network operators share fate in terms of BGP trust relationships. It seems that you feel this is an improvement since some network operators have very lax policies and trust people that they shouldn't. In that case, I wonder whether these operators would even bother joining such a shared-fate arrangement. But the big downside is for the operators who have carefully honed filtering policies and who are careful about who they trust. For them there is no upside to joining a shared-fate arrangement, and a potential downside if management decides that their internal BGP filtering practices can now be made more lax. And, of course, the shared fate arrangement is now a single point of failure and a very juicy target for bad guys to attack. The real solution to the YouTube issue is for people to pressure other network operators to raise their game and pay attention to how they manage their BGP trust relationships and filter announcements. In addition, more people need to get involved in information sharing arrangements like Routing Registries, MyASN, alert services and so on. None of these things create a single point of failure and some of them would be useful even if your Super AS is created. Because all of this activity is done by humans, even your Super AS can make mistakes so it would be bad for people to trust it just because it is big. Alert services, RRs, MyASN, etc., can protect against human failures whether in the Super AS or Pakistan Telecom. Perhaps you might like to propose criteria you would find useful in setting a level of trust, or some alternative method to avoid a recurrence of a site that is widely visited being black holed through another ISP advertising a more specific route? Haven't you noticed that the definition of widely visited site changes regularly, and often quite abruptly? How much traffic did YouTube get 3 years ago? Facebook? MySpace? There is no shortcut for eternal vigilance, i.e. manage your BGP relationships don't just configure and forget. Item 2: in this context, is specific to the needs of North American Network Operators accepting long prefix routes. I am not advocating not accepting routes from the ROW, just not very specific ones. It's entirely possible for North American Operators to rely on law enforcement in say, the EU and Australia. In case you hadn't noticed, there is no North American law enforcement agency and no North American courts and no North American laws outside of NAFTA. So I'm not sure what you are getting at here. Do you want to reopen NAFTA negotiations to include Internet peering? I think it would be better to propose some constructive ideas as to how we can avoid what happened today from recurring, and also deal with the issue of hijacked IP space in general. The tools and techniques are out there. All that is needed is for people to follow best practices. Seems to me that educational activity would be more productive than building castles in the sky. --Michael Dillon
Re: Secure BGP (Was: YouTube IP Hijacking)
[EMAIL PROTECTED] wrote: [..] Pushing this task off to a server that does not have packet-forwarding duties also allows for flexible interfaces to network management systems including the possibility of asking for human confirmation before announcing a new route. There is no (direct) requirement for most of these solutions to do it in the router that forwards actual packets, just add a special BGP box for this. This box then 'verifies' if the update looks OK. When the update looks fishy, it can either, depending on what you want either notify your favourite $nocmonkey to look at it and/or at least instruct the real routers to not use that path. You can take (S-)BGP(-S) for verification, but you can also use IRR data or whatever source you have for stating 'this prefix from there over this path is trusted', compare against that and voila, you got a report when the assumed vectors don't match and you can at least react to them. These kind of systems already exist, see previous emails, but clearly not too many actually make use of them, now that is too bad for your customers who couldn't see their lolcats or worse who couldn't reach their stock house for quickly selling their shares before that company went down the drain completely... Greets, Jeroen signature.asc Description: OpenPGP digital signature
Secure BGP (Was: YouTube IP Hijacking)
Right. Everyone makes mistakes, but not everyone is malicious.And the RIRs and the big ISPs are *generally* more clueful than the little guys and the newcomers. Note also that secured BGP limits the kinds of mistakes people can make. If I have a certificate from my RIR for 192.0.2.0/24, I can't neither announce 10.0.0.0/8 nor delegate it to you, no matter how badly I type. Secured BGP still strikes me as a net win. I suspect that a major part of the problem with implementing Secured BGP is that it is put forth as a solution that you implement in your routers. Network Operators are very careful about the stuff that goes into routers, even to the extent that many of them do not use SSH to manage them. Instead, they run SSH on trusted and secured servers inside their PoPs and configure their routers to only accept telnet sessions from those trusted and secured servers. Is there some way of deploying a solution like Secure BGP without actually requiring that it go into the routers? Perhaps something that allows the routers to still maintain BGP sessions that can withdraw routes, or announce routes which were recently withdrawn, but require a separate encrypted session between two servers, each one in a trust relationship with one of the BGP speaking routers, to handle announcements of new routes? Pushing this task off to a server that does not have packet-forwarding duties also allows for flexible interfaces to network management systems including the possibility of asking for human confirmation before announcing a new route. --Michael Dillon
Re: YouTube IP Hijacking
On Mon, Feb 25, 2008 at 10:12:47AM -, [EMAIL PROTECTED] wrote: In case you hadn't noticed, there is no North American law enforcement agency and no North American courts and no North American laws outside of NAFTA. So I'm not sure what you are getting at here. Do you want to reopen NAFTA negotiations to include Internet peering? the law process within NAFTA is no more than a delay tactic used by various big business and big government to defer any real resolution to the problems. the laws of Canada, Mexico and the US are still largely seperate, and the laws of one do not necessarily follow in another. -- Jim Mercer[EMAIL PROTECTED]+971 55 410-5633 I'm Prime Minister of Canada, I live here and I'm going to take a leak. - Lester Pearson in 1967, during a meeting between himself and President Lyndon Johnson, whose Secret Service detail had taken over Pearson's cottage retreat. At one point, a Johnson guard asked Pearson, Who are you and where are you going?
RE: YouTube IP Hijacking
the laws of Canada, Mexico and the US are still largely seperate, and the laws of one do not necessarily follow in another. Not to mention other North American countries such as France(1), Bermuda, Cuba, Haiti, etc., etc. --Michael Dillon (1) The islands of St. Pierre and Miquelon, Martinique, Guadeloupe P.S. The point of this is that there is nothing nice and neat about the environment in which the Internet operates therefore we really can't expect to find any nice and neat solutions to the messy problems we run across. All we can do is to limit the damage, detect the aberrations, and put them right as quickly as we can. The public has voted and their opinion is that 5 nines does not matter, best effort is good enough, if it really is *BEST* effort.
Re: YouTube IP Hijacking
On Sun, Feb 24, 2008 at 10:49 PM, Sean Donelan [EMAIL PROTECTED] wrote: On Mon, 25 Feb 2008, Steven M. Bellovin wrote: How about state-of-the-art routing security? The problem is what is the actual trust model? Are you trusting some authority to not be malicious or never make a mistake? There are several answers to the malicious problem. There are fewer answers to never making a mistake problem. [snip] +5, Insightful. The focus thus far seems to have been on establishing security on the basis of trusted senders (SPF for BGP, if you'll pardon my rather crude analogy). The impact of a mistake-based failure in a trusted scenario could actually be quite a bit higher than what we've seen thus far: 1) if data (e.g. routes) from a trusted source is allowed into a network (or used as a basis for decision-making) with minimal screening, attackers will immediately shift focus to spoofing trusted sources, just as they currently do in other scenarios; 2) the impact of a typo or other operator error when operating in trusted mode is much higher than that of mistakes from non-trusted sources (if 17557's upstream had trusted a little less - by not automatically propagating any new routes that showed up at the front door - the current incident could very well have amounted to little more than a log entry somewhere, and perhaps an email). I think what you and Steve Bellovin had to say about anti-mistake protocol and belt-and-suspenders should be garnering at least as much consideration as prevention of malicious attacks/forgeries/etc., considering the percentage of outages caused by operator error vs those caused by malice ... -- [EMAIL PROTECTED],darkuncle.net} || 0x5537F527 http://darkuncle.net/pubkey.asc for public key
Re: YouTube IP Hijacking
At 06:17 PM 25-02-08 +0900, Matsuzaki Yoshinobu wrote: All good, er, bad reasons. Fixing the filter your downstreams problem is very important. It would also solve 90-something percent of the problems mentioned in this thread. E.g. as7007. :) I am in the APRICOT meeting in Taipei now, and met a guy from PCCW/AS3491. I have showed him this thread, and have suggested 1) validating prefixes from downstreams before accept, and 2) setting an inbound prefix-filter to their downstreams. Now if only everyone here on NANOG were to do what Matsuzaki has done, and take the time to educate those less clueless, the world would be a better place. -Hank
Re: YouTube IP Hijacking
At 03:14 AM 25-02-08 -0500, Paul Wall wrote: Results were planned to be presented at the next NANOG, but they shouldn't be a surprise to anyone in the industry: nobody filters. Incorrect. Some do filter and do it well. Problem is that it is in general a minority - many of which can be found here on NANOG. -Hank
BGP prefix filtering, how exactly? [Re: YouTube IP Hijacking]
Changed the subject line a little... On Mon, 25 Feb 2008, Hank Nussbacher wrote: At 03:14 AM 25-02-08 -0500, Paul Wall wrote: Results were planned to be presented at the next NANOG, but they shouldn't be a surprise to anyone in the industry: nobody filters. Incorrect. Some do filter and do it well. Problem is that it is in general a minority - many of which can be found here on NANOG. In a lot of this dialogue, many say, you should prefix filter. However, I'm not seeing how an ISP could easily adopt such filtering. Let's consider the options: 1) manually maintained prefix-filters. OK for small ISPs or small users where the prefix churn is minimal. 2) build the filters based on IRR data. But which IRRs to use? some points here: a) only RIPE IRR uses a sensible security model [1], so if you use others, basically anyone can add route objects to the registry. How exactly would this model be useful? b) use your own IRR where you require your customers to add the route objects and verify that they're OK. This means a lot of work for you and even more for your customers. So, this is no excuse for not doing prefix filtering if you only do business in the RIPE region, but anywhere else the IRR data is pretty much useless, incorrect, or both. (Yeah, we prefix filter all our customers. Our IPv6 peers are also prefix filtered, based on RIPE IRR data (with one exception). IPv4 peers' advertisements seem to be too big a mess, and too long filters, to fix this way.) [1] Joe Abley's explanation on SIDR list on 20 Jun 2007: http://www.ietf.org/mail-archive/web/sidr/current/msg00201.html -- Pekka Savola You each name yourselves king, yet the Netcore Oykingdom bleeds. Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
Re: ISP's who where affected by the misconfiguration: start using IRR and checking your BGP updates (Was: YouTube IP Hijacking)
On Mon, 25 Feb 2008, Hank Nussbacher wrote: For us who actually have customers we care about, we probably find it better for business to try to make sure our own customers can't announce prefixes they don't own, but accept basically anything from the world that isn't ours. You are a distinct minority. My experience has shown that most ISPs don't give a sh*t about filtering what their customers can announce so what has happened, will continue to happen. I've only dealt with a handful of the bigger networks, but every transit BGP session I've ever been the customer role on has been filtered by the provider. From memory and in no particular order, that's UUNet, Level3, Digex, Intermedia, Global Crossing, Genuity, Sprint, Above.net, Time Warner, CW, MCI, XO, Broadwing, and a few smaller ones nobody's likely to have heard of. As an ISP providing transit, all of our customers get prefix-filtered. -- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net| _ http://www.lewis.org/~jlewis/pgp for PGP public key_
Re: YouTube IP Hijacking
Christopher Morrow wrote: On Sun, Feb 24, 2008 at 8:42 PM, Patrick W. Gilmore [EMAIL PROTECTED] wrote: except that even the 'good guys' make mistakes. Belt + suspenders please... is it really that hard for a network service provider to have a prefix-list on their customer bgp sessions?? L3 does it, ATT does it, Sprint does it, as do UUNET/vzb, NTT, GlobalCrossing ... seriously, it's not that hard. I know of one tier-1 on that list that made a filtering mistake on one of my own circuits no more than a few months ago. Even some of the biggest players make mistakes. Justin
RE: YouTube IP Hijacking
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Steven M. Bellovin How about state-of-the-art routing security? Seriously -- a number of us have been warning that this could happen. More precisely, we've been warning that this could happen *again*; we all know about many older incidents, from the barely noticed to the very noisy. (AS 7007, anyone?) Something like S-BGP will stop this cold. Yes, I know there are serious deployment and operational issues. The question is this: when is the pain from routing incidents great enough that we're forced to act? It would have been nice to have done something before this, since now all the world's script kiddies have seen what can be done. BCPs stops this problem. soBGP may make it easier.
hijack chronology: was [ YouTube IP Hijacking ]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Greetings, Late last night, after poring through our data, I posted a detailed chronology of the hijack as seen from our many peering sessions. I would add to this that the speed of YouTube's response to this subprefix hijack impressed me. As discussed earlier in this thread, this is really the same old song--it simply has a new verse now. (How many of our troubadors know all of the verses since AS 7007?) Best, - -Martin [0] http://www.renesys.com/blog/2008/02/pakistan_hijacks_youtube.shtml - -- Martin A. Brown --- Renesys Corporation --- [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (GNU/Linux) Comment: pgf-0.72 (http://linux-ip.net/sw/pine-gpg-filter/) iD8DBQFHwtYodXQGngQsWbkRAv1PAKDB0w1OjRmKjF7027ccZJzuzyboKQCgwxFt V8CNr0Nzw2lqx0O5vk3uHKo= =GUic -END PGP SIGNATURE-
Re: YouTube IP Hijacking
y'all, On Mon, Feb 25, 2008 at 06:49:35AM -0800, Barry Greene (bgreene) wrote: Seriously -- a number of us have been warning that this could happen. More precisely, we've been warning that this could happen *again*; we all know about many older incidents, from the barely noticed to the very noisy. (AS 7007, anyone?) Something like S-BGP will stop this cold. Yes, I know there are serious deployment and operational issues. The question is this: when is the pain from routing incidents great enough that we're forced to act? It would have been nice to have done something before this, since now all the world's script kiddies have seen what can be done. BCPs stops this problem. soBGP may make it easier. except in the most meaningless of interpretations, i don't see how BCPs solve this problem. Barry: did you mean: If only everyone on the Internet would perfectly filter their customers prefix announcements on every connection, then everything would be fine? Or perhaps: If everyone would register all of their prefixes in some applicable routing registry with such a degree of accuracy that we could build customer and peer filters out of it then this problem would go away. ? both are true statements, but neither is ever going to happen (i'm not sure that sBGP or soBGP is going to happen ever either). in the mean time all we can do is watch and try to respond to events as quickly as they occur. In fact, we would all be better off if more people were just watching their prefix announcement progapations. Although this hijacking/blackholing was caught quickly, i have seen many other cases where the event lasted for hours (or days). by the way: my colleague Martin A. Brown (who spoke at NANOG on the Taiwan Earthquakes), has written a fairly detailed blog post on the this event. It includes a detailed timeline and some information for people who might find that useful. http://www.renesys.com/blog/2008/02/pakistan_hijacks_youtube_1.shtml Clearly this is not the first interesting accidental hijacking and certainly won't be the last. t. -- _ todd underwood +1 603 643 9300 x101 renesys corporationgeneral manager babbledog [EMAIL PROTECTED] http://www.renesys.com/blog
Rép : YouTube IP Hijacking
Le 25 févr. 08 à 02:42, Patrick W. Gilmore a écrit : On Feb 24, 2008, at 7:36 PM, Tomas L. Byrnes wrote: 1: Hosted at a Tier 1 provider. That is a silly requirement. (I am sorry, I tried hard to find a nicer way to say this, but I really feel strongly about this.) 2: Within a jurisdiction where North American operators have a good chance of having the law on their side in case of any network outage caused by the entity. This is also a bit strange. Do your users never attach to a host outside the USofA? Wasn't it humour / joke / troll :-) ? If not, maybe we've to follow discussion on nnsquad (Network Neutrality Squad) mailing list ? :-)) - Jean-Michel Planche blog: http://www.jmp.net Chairman and co-founder Witbe web : http://www.witbe.net Follow me http://www.twitter.com/jmplanche --- 2.0 Monitoring : relevant End to End monitoring for critical app. and carrier class services
Rep : YouTube IP Hijacking
If someone comes up with the anti-mistake routing protocol ... We could try to invent more idiot proof protocols, but the more control (and centralization), the more it will be a kind of Internet. Not sure the founding principles and factors that made the Internet successful would resist anymore. Anyhow, you can think all you want about security, such as Apple with its concept of his locking iPhone, but you can’t anticipate the unexpected, like the ‘jailbreaker’ success ... and peoples acting on their routers with their feet and not with their brain and their hands. It's the nature of Internet technology: something could always fail and the ability to prepare for the unexpected is one of the reason why it works and is so scalable. It's also why best effort / real knowledge / education are better approaches than searching for yet another killer secure protocol. But maybe I'm a dreamer :-)) Anyhow, I’m not saying that nothing can be done. I can see at least two possibilities: 1/ What measures can be taken to prevent such things from happening and great discussion about it on the list. 2/ How can we take a more proactive approach and be informed of such incidents as soon as they occur and not after the first customer complaint On first issue,a lot to do ... If ‘best effort’ is something that always exist in the today business world, I think we’ll arrive at an equilibrium without waiting for geni.net (certainly good) answers. On second issue, there are plenty of possibilities and it's not difficult now to be informed to the minute when big destination / AS seems to be in trouble. FYI, just see: a very interesting TCP traceroute yesterday, during the mistake on Youtube (seen from our system / AS15436 on US East coast, Europe (Paris and London) http://www.flickr.com/photos/jmplanche/2291442426/ http://www.flickr.com/photos/jmplanche/2290636351/ : routing changed http://www.flickr.com/photos/jmplanche/2290636277/ : Youtube.com unreachable http://www.flickr.com/photos/jmplanche/2290636131/ : very interesting, there is an abnormally high response time, just before severe breakdown. Is Pakistan trying to announce more and more Youtube address and less and less Youtube servers available to answer ? http://www.flickr.com/photos/jmplanche/2291429544/ : not the same overload just before crash as seen from FR and UK : http://www.flickr.com/photos/jmplanche/2291429414/ - Jean-Michel Planche blog: http://www.jmp.net Chairman and co-founder Witbe web : http://www.witbe.net Follow me http://www.twitter.com/jmplanche --- 2.0 Monitoring : relevant End to End monitoring for critical app. and carrier class services
Re: ISP's who where affected by the misconfiguration: start using IRR and checking your BGP updates (Was: YouTube IP Hijacking)
On Mon, Feb 25, 2008 at 09:28:47AM -0500, Jon Lewis wrote: I've only dealt with a handful of the bigger networks, but every transit BGP session I've ever been the customer role on has been filtered by the provider. From memory and in no particular order, that's UUNet, Level3, Digex, Intermedia, Global Crossing, Genuity, Sprint, Above.net, Time Warner, CW, MCI, XO, Broadwing, and a few smaller ones nobody's likely to have heard of. We take transit from some of these providers, and I we have a slightly different experience. While it's not quite a free-for-all, some have implemented a limit on the number of announced prefixes without any restriction to specific space. We found this out after AboveNet dampened us for announcing too many routes. No one there could ever produce any substantial evidence of that, or provide us a single example of one of these routes - but we were told it was strictly the number of prefixes that mattered. I know that I provide newly assigned prefixes to our providers, which includes PCCW. If those make it into a prefix-list at PCCW though, I don't really know for sure. -- Ross Vandegrift [EMAIL PROTECTED] The good Christian should beware of mathematicians, and all those who make empty prophecies. The danger already exists that the mathematicians have made a covenant with the devil to darken the spirit and to confine man in the bonds of Hell. --St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
Re: ISP's who where affected by the misconfiguration: start using IRR and checking your BGP updates
I've only dealt with a handful of the bigger networks, but every transit BGP session I've ever been the customer role on has been filtered by the provider. From memory and in no particular order, that's UUNet, Level3, Digex, Intermedia, Global Crossing, Genuity, Sprint, Above.net, Time Warner, CW, MCI, XO, Broadwing, and a few smaller ones nobody's likely to have heard of. There's at least one reasonably big transit provider that does *not* do prefix filtering: TeliaSonera (AS 1299). They *do* perform as-path filtering, but the effectiveness is disputable... As an ISP providing transit, all of our customers get prefix-filtered. Same here. Steinar Haug, Nethelp consulting, [EMAIL PROTECTED]
RE: YouTube IP Hijacking
DO NOT sign up at that site until the site admin fixes a major issue - I thought it looked interesting but now I'm in an embarrassing situation. I signed up like anyone would do and the moment I validated my email address, postings started to showup under my account that are weeks old - these postings are of sexual nature ... lovely Not impressed to say the least emailed the site admin asking something be done... Paul Stewart Senior Network Administrator Nexicom 5 King St. E., Millbrook, ON, LOA 1GO Phone: 705-932-4127 Web: http://www.nexicom.net Nexicom - Connected. Naturally. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jason Sent: Sunday, February 24, 2008 8:13 PM To: nanog@merit.edu Cc: Paul Ferguson Subject: Re: YouTube IP Hijacking This is similar, and available for all regions/ASNs. http://cs.unm.edu/~karlinjf/IAR/index.php -- Jason Paul Stewart wrote: Very nice.. is there an ARIN equal that anyone knows of OR can you use the RIPE one for ARIN registered space? Just curious.. thanks.. Paul -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Paul Ferguson Sent: Sunday, February 24, 2008 7:07 PM To: [EMAIL PROTECTED] Cc: nanog@merit.edu Subject: Re: YouTube IP Hijacking -- Daniel Roesen [EMAIL PROTECTED] wrote: On Sun, Feb 24, 2008 at 10:41:26PM +, Paul Ferguson wrote: The best you can _probably_ hope for is a opt-in mechanism in which you are alerted that prefixes you have registered with the aforementioned system are being originated by an ASN which is not authorized to originate them. http://www.ris.ripe.net/myasn.html Nice. :-) - ferg -- Fergie, a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawg(at)netzero.net ferg's tech blog: http://fergdawg.blogspot.com/ The information transmitted is intended only for the person or entity to which it is addressed and contains confidential and/or privileged material. If you received this in error, please contact the sender immediately and then destroy this transmission, including all attachments, without copying, distributing or disclosing same. Thank you.
RE: YouTube IP Hijacking
The Site admin got back to me right away I jumped the gun slightly.. Anyways, a spammer had signed into that site previously with the same username and posted lots of crap - when I signed up, those posts came back online hence my panic Should be fine now - interesting site ;) Paul -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Paul Stewart Sent: Monday, February 25, 2008 11:48 AM To: [EMAIL PROTECTED]; nanog@merit.edu Cc: Paul Ferguson Subject: RE: YouTube IP Hijacking DO NOT sign up at that site until the site admin fixes a major issue - I thought it looked interesting but now I'm in an embarrassing situation. I signed up like anyone would do and the moment I validated my email address, postings started to showup under my account that are weeks old - these postings are of sexual nature ... lovely Not impressed to say the least emailed the site admin asking something be done... Paul Stewart Senior Network Administrator Nexicom 5 King St. E., Millbrook, ON, LOA 1GO Phone: 705-932-4127 Web: http://www.nexicom.net Nexicom - Connected. Naturally. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jason Sent: Sunday, February 24, 2008 8:13 PM To: nanog@merit.edu Cc: Paul Ferguson Subject: Re: YouTube IP Hijacking This is similar, and available for all regions/ASNs. http://cs.unm.edu/~karlinjf/IAR/index.php -- Jason Paul Stewart wrote: Very nice.. is there an ARIN equal that anyone knows of OR can you use the RIPE one for ARIN registered space? Just curious.. thanks.. Paul -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Paul Ferguson Sent: Sunday, February 24, 2008 7:07 PM To: [EMAIL PROTECTED] Cc: nanog@merit.edu Subject: Re: YouTube IP Hijacking -- Daniel Roesen [EMAIL PROTECTED] wrote: On Sun, Feb 24, 2008 at 10:41:26PM +, Paul Ferguson wrote: The best you can _probably_ hope for is a opt-in mechanism in which you are alerted that prefixes you have registered with the aforementioned system are being originated by an ASN which is not authorized to originate them. http://www.ris.ripe.net/myasn.html Nice. :-) - ferg -- Fergie, a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawg(at)netzero.net ferg's tech blog: http://fergdawg.blogspot.com/ The information transmitted is intended only for the person or entity to which it is addressed and contains confidential and/or privileged material. If you received this in error, please contact the sender immediately and then destroy this transmission, including all attachments, without copying, distributing or disclosing same. Thank you.
RE: YouTube IP Hijacking
This is a very interesting site. However, I notice that, in the all in the last 24 hours it doesn't show the YouTube hijack. It does have a lot of entries for 17557, most recently on 2/17. How reliable is this system? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Hank Nussbacher Sent: Sunday, February 24, 2008 11:33 PM To: Steven M. Bellovin; nanog@merit.edu Subject: Re: YouTube IP Hijacking At 05:31 AM 25-02-08 +, Steven M. Bellovin wrote: Seriously -- a number of us have been warning that this could happen. More precisely, we've been warning that this could happen *again*; we all know about many older incidents, from the barely noticed to the very noisy. (AS 7007, anyone?) Something like S-BGP will stop this cold. Yes, I know there are serious deployment and operational issues. The question is this: when is the pain from routing incidents great enough that we're forced to act? It would have been nice to have done something before this, since now all the world's script kiddies have seen what can be done. we've been warning that this could happen *again* - this is happening every day - just look to: http://cs.unm.edu/~karlinjf/IAR/prefix.php?filter=most http://cs.unm.edu/~karlinjf/IAR/subprefix.php?filter=most for samples. Thing is - these prefix hijacks are not big ticket sites like Youtube or Microsoft or Cisco or even whitehouse.gov - but rather just sites that never make it onto the NANOG radar. -Hank
Re: YouTube IP Hijacking
Tomas: It's primarily a proof of concept site, to show that such an idea would be useful, but it has been running for over a year now and discovered many interesting hijacks (such as eBay/google/etc..). You're right that there is a glaring ommission, which is yesterday's youtube hijack. This is due to a bug in the sub-prefix lookup code (which can cause the IAR to miss some sub-prefix hijacks), which I'm currently fixing. Once that is done I'll rerun the IAR over yesterday's logs and it will show up. Josh On Mon, Feb 25, 2008 at 10:37 AM, Tomas L. Byrnes [EMAIL PROTECTED] wrote: This is a very interesting site. However, I notice that, in the all in the last 24 hours it doesn't show the YouTube hijack. It does have a lot of entries for 17557, most recently on 2/17. How reliable is this system? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Hank Nussbacher Sent: Sunday, February 24, 2008 11:33 PM To: Steven M. Bellovin; nanog@merit.edu Subject: Re: YouTube IP Hijacking At 05:31 AM 25-02-08 +, Steven M. Bellovin wrote: Seriously -- a number of us have been warning that this could happen. More precisely, we've been warning that this could happen *again*; we all know about many older incidents, from the barely noticed to the very noisy. (AS 7007, anyone?) Something like S-BGP will stop this cold. Yes, I know there are serious deployment and operational issues. The question is this: when is the pain from routing incidents great enough that we're forced to act? It would have been nice to have done something before this, since now all the world's script kiddies have seen what can be done. we've been warning that this could happen *again* - this is happening every day - just look to: http://cs.unm.edu/~karlinjf/IAR/prefix.php?filter=mosthttp://cs.unm.edu/%7Ekarlinjf/IAR/prefix.php?filter=most http://cs.unm.edu/~karlinjf/IAR/subprefix.php?filter=mosthttp://cs.unm.edu/%7Ekarlinjf/IAR/subprefix.php?filter=most for samples. Thing is - these prefix hijacks are not big ticket sites like Youtube or Microsoft or Cisco or even whitehouse.gov - but rather just sites that never make it onto the NANOG radar. -Hank
Re: BGP prefix filtering, how exactly? [Re: YouTube IP Hijacking]
On Feb 25, 2008, at 6:08 AM, Pekka Savola wrote: In a lot of this dialogue, many say, you should prefix filter. However, I'm not seeing how an ISP could easily adopt such filtering. So, this is no excuse for not doing prefix filtering if you only do business in the RIPE region, but anywhere else the IRR data is pretty much useless, incorrect, or both. Agreed. (Yeah, we prefix filter all our customers. Our IPv6 peers are also prefix filtered, based on RIPE IRR data (with one exception). IPv4 peers' advertisements seem to be too big a mess, and too long filters, to fix this way.) Do you explicitly filter routes from your upstream or transit providers? E.g., if one were to announce, say, a more specific of one of your customer's routes to you would you accept it? What about someone else's address space? The only full set of prefix filtering I've ever seen implemented (i.e., BGP customers AND peers) was b y ANS during my days at iMCI ~95. It was extremely painful at times, even for us, if we wanted to advertise new address space we had to update IRR objects and wait on their nightly push of updated routing policies at ANS. We generated our own routing policies automatically off our IRR, which mirrored others as well, and explicitly prefix filtered customers with some fixed prefix and AS path-based policies applied to peers. If it became really urgent, then we'd call ANS and have them manually update their policy, and subsequently 'bounce' the route announcement to trigger transmission of a new update. This was long before incrementally updated filters and things like BGP route refresh ever existed. Prefixes and AS-MACROs had to be right in the IRRs or the policies wouldn't be updated. It's to bad other folks didn't follow suit. As for this event, a slightly different spin here: http://tinyurl.com/3y3pzl -danny
Re: BGP prefix filtering, how exactly? [Re: YouTube IP Hijacking]
On Mon, 25 Feb 2008, Danny McPherson wrote: (Yeah, we prefix filter all our customers. Our IPv6 peers are also prefix filtered, based on RIPE IRR data (with one exception). IPv4 peers' advertisements seem to be too big a mess, and too long filters, to fix this way.) Do you explicitly filter routes from your upstream or transit providers? E.g., if one were to announce, say, a more specific of one of your customer's routes to you would you accept it? What about someone else's address space? Our own or our singlehomed customers' address space -- we would reject such an advertisement. The same inbound consistency check applies to peers and upstreams/transits. If it's someone else's or a more specific or the same prefix as our multihomed customers -- we accept it. There isn't anything else we can do in practise which would not hurt legitimate routing.. It was extremely painful at times, even for us, if we wanted to advertise new address space we had to update IRR objects and wait on their nightly push of updated routing policies at ANS. We generated our own routing policies automatically off our IRR, which mirrored others as well, and explicitly prefix filtered customers with some fixed prefix and AS path-based policies applied to peers. If it became really urgent, then we'd call ANS and have them manually update their policy, and subsequently 'bounce' the route announcement to trigger transmission of a new update. Sounds like a procedure that should be applied today (whether or not you want to use IRR and/or autogenerated configs is a matter of taste) but the principle seems sound. -- Pekka Savola You each name yourselves king, yet the Netcore Oykingdom bleeds. Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
[admin] [summary] RE: YouTube IP Hijacking
A bit of administrativia: This thread generated over a hundred posts, many without operational relevance or by people who do not understand how operators, well, operate, or by people who really don't have any idea what's going on but feel like posting. I'd like to briefly summarize the important things that were said. If you would like to add something to the thread, make sure you read this post in entirety. Sorry if I didn't attribute every suggestion to a poster. Facts: * AS17557 announced more specific /24 to 3491, which propagated to wider internets * Chronology (by [EMAIL PROTECTED]) http://www.renesys.com/blog/2008/02/pakistan_hijacks_youtube.shtml * Things suggested to possibly address the problem: ** IRR filtering (using IRRPT http://sourceforge.net/projects/irrpt/ to generate filter lists) ** Notification when origin of a given route changes http://www.cs.ucla.edu/~mohit/cameraReady/ladSecurity06.pdf http://www.ris.ripe.net/myasn.html http://cs.unm.edu/~karlinjf/IAR/index.php (from pgBGP) ** pgBGP to depref suspicious routes http://www.nanog.org/mtg-0606/pdf/josh-karlin.pdf (unclear the number of false positives that will adversely affect connectivity) ** sbgp/sobgp - require full authentication for each IP block, and thus unlikely to be implemented until certificate chains are in place, and vendors release code that does verification, and operators are happy enough running it. Other things addressed: * Fragility of Internet: ** Nobody brought up the important point - the BGP announcement filtering are only as secure as the weakest link. No [few?] peers or transits are filtering large ISPs (ones announcing few hundred routes and up). There are a great many of them, and it takes only one of them to mess up filtering a downstream customer for the route to be propagated. ** Paul Wall brought up the fact that even obviously bogus routes (1/8 and 100/7) were accepted by 99% of internet during an experiment. Will it take someone announcing 9/11 to get us to pay attention? (ok, bad joke) ** What I'd like to see discussed: Issues of filtering your transit downstream customers, who announce thousands of routes. Does *anyone* do it? * Typos vs Malicious announcements ** Some ways of fixing the problem (such as IRR filtering) only address the typos or unintentional announcements. There's full agreement that IRR is full of junk, which is not authenticated in any sort. ** Things like PHAS won't work if hijacker keeps the origin-AS same (by getting their upstream to establish session with different ASN) ** What I'd like to see discussed: Who (ICANN/RIRs/LIRs) is actively working on implementing chain of trust of IP space allocations? * Ways to address the issue without cooperation of 3491: ** Filtering anything coming out of 17557 ** Suggestions given: ** What I'd like to see discussed: Can an network operator, *today*, filter the possibly bogus routes from their peers, without manual intervention, and without false positives? * Yelling at people who don't filter ** Per above, 3491 isn't the only one who filters. In fact, claims were made that *nobody* filters large enough downstreams. (beyond aspath/maxpref) ** *please* do not post additional comments about pccw bad, etc. * Malicious vs mistaken on part of AS17557 and 3491: ** *please* do not post speculation unless you have facts to back it up. ** Any discussions of cyber-jihad are off-topic unless you can produce the fatwa to back it up.
Re: [admin] [summary] RE: YouTube IP Hijacking
On Feb 25, 2008, at 12:51 PM, Alex Pilosov wrote: ** Nobody brought up the important point - the BGP announcement filtering are only as secure as the weakest link. No [few?] peers or transits are filtering large ISPs (ones announcing few hundred routes and up). There are a great many of them, and it takes only one of them to mess up filtering a downstream customer for the route to be propagated. Yes, that was my implicit point to Pekka. Even if you do everything feasible today (i.e., explicitly filter customers, some amount of policy for peers, and perhaps a few hacks for multi-homed customers), you're still pretty much screwed if someone announces your address space. Heck, you're as likely to accept the announcement as anyone. ** Paul Wall brought up the fact that even obviously bogus routes (1/8 and 100/7) were accepted by 99% of internet during an experiment. I'm not sure why this would surprise anyone. ** What I'd like to see discussed: Issues of filtering your transit downstream customers, who announce thousands of routes. Does *anyone* do it? Lots of folks do. The interesting bit is that even then, those same providers would accept perhaps even those customer routes from their peers implicitly. * Typos vs Malicious announcements ** Some ways of fixing the problem (such as IRR filtering) only address the typos or unintentional announcements. You mean as opposed to intentionally malice acts? Well, not completely. See Pekka's email, for example. Of course, it does vary widely across IRRs, etc.. There's full agreement that IRR is full of junk, which is not authenticated in any sort. Mostly, though not completely. ** Things like PHAS won't work if hijacker keeps the origin-AS same (by getting their upstream to establish session with different ASN) NO, that's not even necessary. Simple originate the route from the legit AS, and then transit it with the local AS as a transit AS. AS path manipulation is trivial. ** What I'd like to see discussed: Who (ICANN/RIRs/LIRs) is actively working on implementing chain of trust of IP space allocations? * Ways to address the issue without cooperation of 3491: ** Filtering anything coming out of 17557 Bad idea. ** Suggestions given: ** What I'd like to see discussed: Can an network operator, *today*, filter the possibly bogus routes from their peers, without manual intervention, and without false positives? Sure, if they want to dedicate an engineer to it, automate policy deployment and deal with brokenness by turning steam valves. * Yelling at people who don't filter That's been productive for over a decade now. ** Per above, 3491 isn't the only one who filters. In fact, claims were made that *nobody* filters large enough downstreams. (beyond aspath/maxpref) Wrong. -danny
Re: [admin] [summary] RE: YouTube IP Hijacking
On Mon, 25 Feb 2008, Danny McPherson wrote: ** Paul Wall brought up the fact that even obviously bogus routes (1/8 and 100/7) were accepted by 99% of internet during an experiment. I'm not sure why this would surprise anyone. To me and you, it's not surprising. To public, it might be. Even the majority of nanog attendees I think would be surprised. ** What I'd like to see discussed: Issues of filtering your transit downstream customers, who announce thousands of routes. Does *anyone* do it? Lots of folks do. The interesting bit is that even then, those same providers would accept perhaps even those customer routes from their peers implicitly. Well, in this case, they *aren't* filtering! (unless I am misunderstanding what you are saying, due to repeated use of 'their'). ** Things like PHAS won't work if hijacker keeps the origin-AS same (by getting their upstream to establish session with different ASN) NO, that's not even necessary. Simple originate the route from the legit AS, and then transit it with the local AS as a transit AS. AS path manipulation is trivial. Oh yeah, d'oh! Thanks for correction. But that is also an important point against PHAS and IRRPT filtering - they are powerless against truly malicious hijacker (one that would register route in IRR, add the right origin-as to AS-SET, and use correct origin). ** What I'd like to see discussed: Who (ICANN/RIRs/LIRs) is actively working on implementing chain of trust of IP space allocations? * Ways to address the issue without cooperation of 3491: ** Filtering anything coming out of 17557 Bad idea. Obviously :) ** Suggestions given: ** What I'd like to see discussed: Can an network operator, *today*, filter the possibly bogus routes from their peers, without manual intervention, and without false positives? Sure, if they want to dedicate an engineer to it, automate policy deployment and deal with brokenness by turning steam valves. I'd hear to see who does it, and get them to present the operational lessons at the next nanog! * Yelling at people who don't filter That's been productive for over a decade now. ** Per above, 3491 isn't the only one who filters. In fact, claims were made that *nobody* filters large enough downstreams. (beyond aspath/maxpref) Wrong. Likewise, I'd like to know who does this (names) and how can we get them to present best practices at the next nanog! -alex
RE: BGP prefix filtering, how exactly? [Re: YouTube IP Hijacking]
clip Our own or our singlehomed customers' address space -- we would reject such an advertisement. The same inbound consistency check applies to peers and upstreams/transits. If it's someone else's or a more specific or the same prefix as our multihomed customers -- we accept it. There isn't anything else we can do in practise which would not hurt legitimate routing.. clip What do you do when one of your multi-homed customers on your IP space has an outage on their connection to your network? How would your customers then reach that customer? Although this wouldn't be THAT BIG of a deal for small networks, if say a larger or a Tier-1 provider practiced this (AFAIK, the only somewhat large network to do this is, believe it or not, PCCW), your customer would experience a major outage. There must be a better way. :) Pekka Savola Regards, Randy
Re: [admin] [summary] RE: YouTube IP Hijacking
On Feb 25, 2008, at 1:22 PM, Alex Pilosov wrote: Well, in this case, they *aren't* filtering! (unless I am misunderstanding what you are saying, due to repeated use of 'their'). What I'm saying is that best case today ISPs police routes advertised by their customers, yet they accept routes implicitly (including routes from address space that may belong to their customers) from peers. Seems a little hokey, eh? Oh yeah, d'oh! Thanks for correction. But that is also an important point against PHAS and IRRPT filtering - they are powerless against truly malicious hijacker (one that would register route in IRR, add the right origin-as to AS-SET, and use correct origin). Yep, pretty much. Sure, if they want to dedicate an engineer to it, automate policy deployment and deal with brokenness by turning steam valves. I'd hear to see who does it, and get them to present the operational lessons at the next nanog! Maybe Curtis V. would present what ANS was doing in 1994 :-) But now we've even got things like BGP route refresh, incrementally updatable filters, and BGP soft reconfiguration to ease the deployment burden. There have been two or three panels on this exact topic in the past, you can find them in the index of talks. Unfortunately, the problem hasn't changed at all. Perhaps we could just replay those video streams :-) -danny
Re: [admin] [summary] RE: YouTube IP Hijacking
I'd hear to see who does it, and get them to present the operational lessons at the next nanog! On second thought, I guess one thing has changed considerably since 15 years ago. Rather than ~5000 monkeys with keyboard access to manipulate global routing tables, there are likely well North of 250,000 (25k active ASNs * 10 meat computers per), which is surely well on the conservative side. The bottom line is [still] that ISPs should at least explicitly filter prefixes from customers and networks from which they provide transit services. -danny
Re: BGP prefix filtering, how exactly? [Re: YouTube IP Hijacking]
On Mon, 25 Feb 2008 15:29:01 EST, Randy Epstein said: Our own or our singlehomed customers' address space -- we would reject ^^^ such an advertisement. The same inbound consistency check applies to peers and upstreams/transits. What do you do when one of your multi-homed customers on your IP space has an outage on their connection to your network? How would your customers then reach that customer? He explicitly said single-homed. Of course, multi-homed requires different handling, because you may hear their other home announce them (although again, you probably shouldn't listen to *THAT* announcement either if *your* link to them is up). And I posit that if you don't know if your customer is single or multi-homed, you have *bigger* issues to deal with. pgpReSJzdoydy.pgp Description: PGP signature
RE: [admin] [summary] RE: YouTube IP Hijacking
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 There have been two or three panels on this exact topic in the past, you can find them in the index of talks. Unfortunately, the problem hasn't changed at all. Perhaps we could just replay those video streams :-) My $.02 - http://www.getit.org/wordpress/?p=82 The irony to one of those, is that in NANOG 25 right before my session which pointed out this continued threat vector, we had Protecting the BGP Routes to Top Level DNS Servers http://www.nanog.org/mtg-0206/bush.html. -BEGIN PGP SIGNATURE- Version: PGP 8.1 iQA/AwUBR8M5ib/UEA/xivvmEQISnwCgojEcUx7dyMBQPP59gZjdLgQeqqMAoL0p seCMm+llF8tkr+pGf7LlyXrW =6jfG -END PGP SIGNATURE-
Re: Secure BGP (Was: YouTube IP Hijacking)
Is there some way of deploying a solution like Secure BGP without actually requiring that it go into the routers? The IETF SIDR wg (shameless plug as I'm wg co-chair) is working on a way to say with strong assurance who holds what prefixes, and therefore who can authorize the origination of what prefixes. This could be used in creating filter lists, answering customer request (please announce this for me...), checking the RIB out-of-band, etc. Such info is also the foundation of any yet proposed mechanism for doing in-band bgp security (S-BGP, soBGP, psBGP, SPV, etc., etc.), but the sidr work by itself does not need to be done in the router. Maybe some of you could take a look and comment. Look for the drafts at http://www.ietf.org/html.charters/sidr-charter.html --Sandy
RE: BGP prefix filtering, how exactly? [Re: YouTube IP Hijacking]
Valdis wrote: He explicitly said single-homed. Of course, multi-homed requires different handling, because you may hear their other home announce them (although again, you probably shouldn't listen to *THAT* announcement either if *your* link to them is up). And I posit that if you don't know if your customer is single or multi-homed, you have *bigger* issues to deal with. My bad, I misread his multi-homed comment. From what I understand (and have seen in practice) PCCW does not listen to their address space from their peers no matter what the status of the connection to their customer is. I find this policy flat out flawed. Randy
Re: [admin] [summary] RE: YouTube IP Hijacking
On Mon, Feb 25, 2008, Alex Pilosov wrote: A bit of administrativia: This thread generated over a hundred posts, many without operational relevance or by people who do not understand how operators, well, operate, or by people who really don't have any idea what's going on but feel like posting. I'd like to briefly summarize the important things that were said. If you would like to add something to the thread, make sure you read this post in entirety. .. and, since I have 5 minutes to spare between disk array rebuilds, I bring you: http://nanog.cluepon.net/index.php/MailTopics For now it just links into Alex's summary post. I'll go trawl the archives now for more summaries. Adrian
Re: YouTube IP Hijacking
On Mon, Feb 25, 2008 at 2:32 AM, Hank Nussbacher [EMAIL PROTECTED] wrote: we've been warning that this could happen *again* - this is happening every day - just look to: http://cs.unm.edu/~karlinjf/IAR/prefix.php?filter=most http://cs.unm.edu/~karlinjf/IAR/subprefix.php?filter=most So, as someone that once subscribed to josh's alerts... for a large ISP, or an ISP with lots of customers using PA/PI space I got loads and loads of alerts about customer allocations popping out in other ASN's, it turns out those were almost always intended by the customer even... So I'm not sure that the above links are the best judge of this problem space. I agree though that the problem is there to some extent... much more often seen than the 1/2yr events we've noted on nanog-l over the last 8 years. -Chris
