Peering with the Internet Alert Registry
All, Some of you are aware of the site for network operators: http://iar.cs.unm.edu/ which has running for two years now. The purpose of the site is to detect and distribute network anomaly information to the network operators that need to know. The flip side of our proposed security system, Pretty Good BGP (PGBGP), lowers the local preference of anomalous routes on BGP routers for 24 hours, giving operators time to respond to anomalous routes before they can fully propagate. Now, PGBGP is in actual routing software (Quagga), which we soon hope to distribute. As an initial means of test, we will switch the IAR to it (instead of scraping RIPE/RouteViews with a script). This means that we will need peers to provide the IAR with BGP updates (we will not propagate any route updates to your routers). Currently we have three BGP streams, more would be appreciated. If you would like to contribute to our research project, please reply directly to me. More information about the project can be found here: http://cs.unm.edu/~karlinjf/pgbgp/ http://cs.unm.edu/%7Ekarlinjf/pgbgp/ Thanks! Josh
Re: Peering with the Internet Alert Registry
Chris, That's a good question. IAR peers that also wish to run PGBGP will transmit their anomalous routes out of band to the IAR. This will likely be done via logs and a simple forwarding script. Josh On Mon, Mar 10, 2008 at 4:01 PM, Christopher Morrow [EMAIL PROTECTED] wrote: On Mon, Mar 10, 2008 at 11:01 AM, Josh Karlin [EMAIL PROTECTED] wrote: All, Some of you are aware of the site for network operators: http://iar.cs.unm.edu/ which has running for two years now. The purpose of the site is to detect and distribute network anomaly information to the network operators that need to know. The flip side of our proposed security system, Pretty Good BGP (PGBGP), lowers the local preference of anomalous routes on BGP routers for 24 hours, giving operators time to respond to anomalous routes before they can fully propagate. does pgbgp toss out alerts/snmp-traps/log-messages when these anomalous announcements arrive? if not, how does one know they are inside the 24hr window?
Re: YouTube IP Hijacking
Tomas: It's primarily a proof of concept site, to show that such an idea would be useful, but it has been running for over a year now and discovered many interesting hijacks (such as eBay/google/etc..). You're right that there is a glaring ommission, which is yesterday's youtube hijack. This is due to a bug in the sub-prefix lookup code (which can cause the IAR to miss some sub-prefix hijacks), which I'm currently fixing. Once that is done I'll rerun the IAR over yesterday's logs and it will show up. Josh On Mon, Feb 25, 2008 at 10:37 AM, Tomas L. Byrnes [EMAIL PROTECTED] wrote: This is a very interesting site. However, I notice that, in the all in the last 24 hours it doesn't show the YouTube hijack. It does have a lot of entries for 17557, most recently on 2/17. How reliable is this system? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Hank Nussbacher Sent: Sunday, February 24, 2008 11:33 PM To: Steven M. Bellovin; nanog@merit.edu Subject: Re: YouTube IP Hijacking At 05:31 AM 25-02-08 +, Steven M. Bellovin wrote: Seriously -- a number of us have been warning that this could happen. More precisely, we've been warning that this could happen *again*; we all know about many older incidents, from the barely noticed to the very noisy. (AS 7007, anyone?) Something like S-BGP will stop this cold. Yes, I know there are serious deployment and operational issues. The question is this: when is the pain from routing incidents great enough that we're forced to act? It would have been nice to have done something before this, since now all the world's script kiddies have seen what can be done. we've been warning that this could happen *again* - this is happening every day - just look to: http://cs.unm.edu/~karlinjf/IAR/prefix.php?filter=mosthttp://cs.unm.edu/%7Ekarlinjf/IAR/prefix.php?filter=most http://cs.unm.edu/~karlinjf/IAR/subprefix.php?filter=mosthttp://cs.unm.edu/%7Ekarlinjf/IAR/subprefix.php?filter=most for samples. Thing is - these prefix hijacks are not big ticket sites like Youtube or Microsoft or Cisco or even whitehouse.gov - but rather just sites that never make it onto the NANOG radar. -Hank
Re: odd hijack
Wouldn't they want to hijack more specifics to spam? no. see nick feamster's work, and the lightning talk i proxied for him in dallas. randy Right, you might want to announce less specifics so that you go unnoticed and then you can spam from blocks not in use. I'm just somewhat surprised that they would announce /3's, /6's, and /7's and be so incredibly obvious about it rather than actually be covert and just announce a few more specifics that likely noone would notice. But I guess at this point they're still not being caught so why worry. Josh
odd hijack
I recently brought up a prefix hijack that the NANOG community solved, the AS had accidentally started announcing their bogon list. Here is one that is somewhat the opposite, the AS announced a significant portion of IANA allocated space. Note, they are large blocks and as such probably did not cause much damage because most networks announce more specifics. My question to the community is, what kind of misconfiguration could cause this set of prefixes to be announced? I asked the AS responsible, but have not had a response. If you would like more information on the hijack, please see the Internet Alert Registry forums at http://cs.unm.edu/~karlinjf/IAR/ Following are the AS's announced prefixes during the ~10 minute hijack from earlier today: 11.0.0.0/8 12.0.0.0/7 121.0.0.0/8 122.0.0.0/7 124.0.0.0/7 126.0.0.0/8 128.0.0.0/3 15.0.0.0/8 151.99.190.0/24 16.0.0.0/6 160.0.0.0/5 168.0.0.0/6 172.0.0.0/8 188.0.0.0/8 189.0.0.0/8 190.0.0.0/8 191.0.0.0/8 192.0.0.0/8 193.0.0.0/8 194.0.0.0/7 196.0.0.0/8 198.0.0.0/8 199.0.0.0/8 20.0.0.0/7 200.0.0.0/8 201.0.0.0/8 202.0.0.0/7 204.0.0.0/7 206.0.0.0/7 208.0.0.0/8 209.0.0.0/8 210.0.0.0/7 210.170.0.0/18 212.0.0.0/7 214.0.0.0/7 216.0.0.0/8 217.0.0.0/8 218.0.0.0/7 22.0.0.0/8 220.0.0.0/7 222.0.0.0/8 24.0.0.0/8 25.0.0.0/8 26.0.0.0/8 28.0.0.0/7 30.0.0.0/8 32.0.0.0/6 32.1.21.168/32 38.0.0.0/8 40.0.0.0/8 41.0.0.0/8 43.0.0.0/8 44.0.0.0/6 48.0.0.0/6 56.0.0.0/7 58.0.0.0/8 59.0.0.0/8 6.0.0.0/8 60.0.0.0/7 62.0.0.0/8 63.0.0.0/8 64.0.0.0/5 72.0.0.0/7 74.0.0.0/7 76.0.0.0/8 77.0.0.0/8 78.0.0.0/7 8.0.0.0/7 80.0.0.0/7 82.0.0.0/8 82.143.0.0/18 82.143.0.0/20 82.143.0.0/21 82.143.10.0/23 82.143.12.0/24 82.143.16.0/20 82.143.32.0/19 82.143.32.0/24 82.143.33.0/25 82.143.8.0/23 83.0.0.0/8 84.0.0.0/6 88.0.0.0/7 90.0.0.0/8 91.0.0.0/8 96.0.0.0/6 Thanks, Josh
Re: odd hijack
Wouldn't they want to hijack more specifics to spam? I doubt much of that space is going to correctly route for spamming purposes. On 11/9/06, Hank Nussbacher [EMAIL PROTECTED] wrote: On Thu, 9 Nov 2006, Josh Karlin wrote: Here is one that is somewhat the opposite, the AS announced a significant portion of IANA allocated space. Note, they are large blocks and as such probably did not cause much damage because most networks announce more specifics. My question to the community is, what kind of misconfiguration could cause this set of prefixes to be announced? I asked the AS responsible, but have not had a response. Misconfiguration? :-) That's a nice word for spammer. See Joe's PPT at: http://www.uoregon.edu/~joe/maawg8/maawg8.ppt AS29449 is not the problem. It is the upstreams of AS5602 (KPNQwest Italia) and AS286 (KPN) that let this crap leak. -Hank Nussbacher http://www.interall.co.il
AS 8437 announced a quarter of the net for half of an hour
Greetings, Today (Aug 14th 2006) AS 8437 announced 63 /8 nets from 14:30 to 15:00 UTC. I don't believe that this is normal, but please correct me if I am wrong. More info can be found at the Internet Alert Registry here: http://cs.unm.edu/~karlinjf/IAR/prefix.php?filter=most If you come to this 24 hours of the event, you can go here: http://cs.unm.edu/~karlinjf/IAR/search.php and do an Hijacker AS search on 8437. If you would like to see the routes as they happened from a RIPE viewpoint, please check out this very cool site: http://stats.sunet.se/bgpsearch/ Following is the list of announced nets: 1/8 2/8 5/8 7/8 23/8 27/8 31/8 36/8 37/8 39/8 42/8 49/8 50/8 77/8 78/8 79/8 92/8 93/8 94/8 95/8 96/8 97/8 98/8 99/8 100/8 101/8 102/8 103/8 104/8 105/8 106/8 107/8 108/8 109/8 110/8 111/8 112/8 113/8 114/8 115/8 116/8 117/8 118/8 119/8 120/8 173/8 174/8 175/8 176/8 177/8 178/8 179/8 180/8 181/8 182/8 183/8 184/8 185/8 186/8 187/8 197/8 198/8 223/8
Re: AS 8437 announced a quarter of the net for half of an hour
Yes but no response yet. On 8/14/06, Randy Bush [EMAIL PROTECTED] wrote: Today (Aug 14th 2006) AS 8437 announced 63 /8 nets from 14:30 to 15:00 UTC. I don't believe that this is normal, but please correct me if I am wrong. have you written to tele2uta in asutria? randy
Re: AS 8437 announced a quarter of the net for half of an hour
Ah, I believe you're right. Thanks for clearing it up! I had looked up a couple of the prefixes to see if they had owners and I thought I had seen one, but I must have made a typo. I like my swimming pool of lava thank you very much :p Josh On 8/14/06, Richard A Steenbergen [EMAIL PROTECTED] wrote: On Mon, Aug 14, 2006 at 01:36:36PM -0600, Josh Karlin wrote: Greetings, Today (Aug 14th 2006) AS 8437 announced 63 /8 nets from 14:30 to 15:00 UTC. I don't believe that this is normal, but please correct me if I am wrong. Note they're all unallocated blocks, so probably someone's attempt at bogon filtering got leaked inadvertently. Since they're all unallocated blocks, it shouldn't have done any harm, and anyone with reasonably intelligent routing policies should have blocked those routes anyways. :P And may there be a special circle of hell reserved for the weenies who do stupid unnecessary shit that breaks more than it fixes in the name of security. :) -- Richard A Steenbergen [EMAIL PROTECTED] http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
Re: a fun hijack: 1/8, 2/8, 3/8, 4/8, 5/8, 7/8, 8/8, 12/8 briefly announced by AS 23520 (today)
I am happy folks like at RIPE and the IETF are looking at solutions, but sBGP isn't a new idea, and well, how LONG have we been waiting for DNS-SEC now? I just read a paper yesterday from '97 that suggested complete registries would be available within the next couple of years ;)
a fun hijack: 1/8, 2/8, 3/8, 4/8, 5/8, 7/8, 8/8, 12/8 briefly announced by AS 23520 (today)
Check out the IAR for Potential Prefix Hijacks and if you're coming to this more than 24 hours after the post, do a search on AS 23520 as the hijacking AS. I don't know how long the routes were announced, but they seem to be gone now. Or maybe the IAR is horribly broken, in which case I will be lynched :) IAR: http://cs.unm.edu/~karlinjf/IAR/ Josh
Re: a fun hijack: 1/8, 2/8, 3/8, 4/8, 5/8, 7/8, 8/8, 12/8 briefly announced by AS 23520 (today)
Wonder if it was intentional or a 'classful' issue. This is why we (Level 3) and ATT announce the /9s of 4/8, 8/8, and 12/8 :) -Kevin The /9s were stolen too, as well as a host of other prefixes. I just listed the biggies that I was pretty sure didn't belong to 23520. No clue if it was intentional or not, but I would also like to know. Josh
Re: So -- what did happen to Panix?
Here is what we propose in PGBGP. If you have a more specific route and its AS Path does not contain any of the less specific route's origins, then ignore it for a day and keep routing to the less specific origin. If it's legitimate the less specific origin should forward the data on for the day. We see about 30 of these suspicious routes per day. I imagine some of you will not like this sceheme. Please let me know why. Josh On 2/8/06, Jeffrey Haas [EMAIL PROTECTED] wrote: On Wed, Feb 08, 2006 at 04:37:31AM +, Christopher L. Morrow wrote: I had thought Josh's paper (or maybe not josh, whomever it was) said something along the lines of: 1) if more than one announcement prefer 'longer term', 'older', 'more usual' route 2) if only one route take it and run! FWIW, this sort of mechanism was discussed among the IETF RPSEC WG task group that is working on BGP security requirements. On the presumption that some database of stable routes and paths is present, you could bias your preference in your routes for more stable routes and paths. You would also need to decide what to do about more specific routes covered by stable routes. Do you ignore them? This is a harder question. -- Jeff Haas NextHop Technologies
Re: So -- what did happen to Panix?
Chris has it! And to be clear, we only require a slow (1 day) provider changeover in the case that you want to announce your old provider's sub-prefix at a new provider. For instance, if you are an ATT customer using a 12/8 sub-prefix and change providers but keep the prefix, the prefix will look funny coming from another originator for the first day and be delayed. All other methods of changing providers will not be interfered with. Josh I had thought Josh's paper (or maybe not josh, whomever it was) said something along the lines of: 1) if more than one announcement prefer 'longer term', 'older', 'more usual' route 2) if only one route take it and run! So.. provided Connexion withdraws from 'as-germany' and announces in 'as-atlantic ocean', and so on there would only be 1 route, and you'd fall to step 2. (yes, the paper was more detailed and there were more steps...)
Re: So -- what did happen to Panix?
Hasn't that been said for years? Wouldn't perfect IRRs be great? I couldn't agree more. But in the meanwhile, why not protect your own ISP by delaying possible misconfigurations.Our proposed delay does *not* affect reachability, if the only route left is suspicious, it will be chosen regardless. Depending on the threat model, then, one attack would be to cause an AS to damp the non-suspicious route. This seems bad, right? A flapping, correct route seems better than a stable, suspicious one. A flapping route would only be considered suspicious if it disappears for many consecutive days and no other known route for the prefix originates at the same AS. At which point the attacker has already won. Our primary concern is with keeping BGP stable until its replacement (e.g. sBGP) is ready for deployment. Perhaps I am missing something, but how does imposing a delay help in ascertaining a route's correctness? Even looking at some of the suspicious routes I see by hand in the anomalies we detect, I can't personally tell what's incorrect/actionable vs. simply unusual (again, this goes back to the problem of inaccurate registries). In the case of Panix/ConEd, I can imagine that an operator would have responded to the alarms, checked the registry information and said, these routes look reasonable; go for it! Or, as human nature suggests, the operator might have even just ignored the alarms (particularly if origin changes are as frequent as they seem to be). Ascertaining correctness is only half of the work. If you correctly classify a malicious route, but do not take some measure to prevent its spread, you have just done yourself and your customers harm. In the case of PGBGP, there is a lot that an operator can do to verify correctness. Multiple viewpoints of anomalous routes can be collected into a single database in which operators can, once per day, check to make sure that their own address space is not being announced elsewhere. This can easily be automated for both the NOC and the collection process. Relationship information need not be revealed as only the originator of the suspicious route is needed. If, in the worst case, the route is not detected as malicious before it is considered normal, the next wave of routers will be introduced to the route and consider it suspicious. The first wave will then notice the problem and fix it, still protecting a significant portion of the network. Josh
Re: So -- what did happen to Panix?
Wouldn't a well-operated network of IRRs used by 95% of network operators be able to meet all three of your requirements? -certified prefix ownership -certified AS path ownership -dynamic changes to the above two items It seems to me that most of the pieces needed to do this already exist. RPSL, IRR softwares, regional addressing authorities (RIRs). If there are to be certified AS paths in a central database this also opens the door to special arrangements for AS path routing that go beyond peering, i.e. agreements with the peers of your peers. Hasn't that been said for years? Wouldn't perfect IRRs be great? I couldn't agree more. But in the meanwhile, why not protect your own ISP by delaying possible misconfigurations.Our proposed delay does *not* affect reachability, if the only route left is suspicious, it will be chosen regardless. If you are changing providers, which takes awhile anyway, just advertise both for a day and you have no problems. Or, if you are concerned about speed, simply withdraw one and the new one will have to be used. If you are anycasting the prefix and a new origin pops up that your view has not seen before, then you might have a temporary load balance issue, but there is absolutely no guarantee of what routers many hops away from you will see anyway. Josh
Re: So -- what did happen to Panix?
The noise of origin changes is fairly heavy, somewhere in the low hundreds of alerts per day given a 3 day history window. Supposing a falsely originated route was delayed, what is the chance of identifying and fixing it before the end of the delay period? Do operators commonly catch misconfigurations on their own or do they usually find out about it from other operators due to service disruption?
Re: So -- what did happen to Panix?
I unfortunately don't have answers to those questions, but you've piqued my interest so I will try to look into it within the next couple of days. Josh On 1/26/06, Jared Mauch [EMAIL PROTECTED] wrote: On Thu, Jan 26, 2006 at 04:22:29PM -0700, Josh Karlin wrote: The noise of origin changes is fairly heavy, somewhere in the low hundreds of alerts per day given a 3 day history window. Supposing a falsely originated route was delayed, what is the chance of identifying and fixing it before the end of the delay period? Do operators commonly catch misconfigurations on their own or do they usually find out about it from other operators due to service disruption? Are the origin changes for a small set of the prefixes that tend to repeat (eg: connexion as planes move), or is it a different set of prefixes day-to-day or week-to-week? I suspect there are the obvious prefixes that don't change (eg: 12/8, 18/8, 35/8, 38/8) but subparts of that may change, but for most people with allocations in the range of 12-17 bits, I suspect they won't change frequently. - jared -- Jared Mauch | pgp key available via finger from [EMAIL PROTECTED] clue++; | http://puck.nether.net/~jared/ My statements are only mine.
preventing future situations like panix
Short of perfect filters, or perfect IRRs combined with PKI, it's a difficult problem to stop prefix hijacks such as we've seen this weekend. Myself, Jennifer Rexford, and Stephanie Forrest have been looking at feasible and incrementally deployable solutions to the problem and we would really like to have operator input on our proposed solution. The idea is simply to consider 'suspicious' looking routes as a last resort in the decision process (~1 day). Thus if no alternative route for a prefix exists, the suspicious route is used regardless, no harm done. Of course, relationship preference must be preserved when possible so very few routes should be considered suspicious if possible. Suspicious routes are those that originate at an AS that has not originated the prefix in the last few days and those that introduce sub-prefixes. Sub-prefixes are always considered suspicious (~1 day) and traffic will be routed to the super-prefix for the suspicious period. We have not yet addressed man-in-the-middle style of attacks where an AS announces a false route and places itself in the middle. We also realize that nobody likes to have announcements delayed, and we explain in detail how few routes will actually be delayed by our mechanism in the linked paper. http://www.cs.princeton.edu/~jrex/papers/pgbgp.pdf Your input is most welcome. Thanks, Josh Karlin
Re: preventing future situations like panix
It seems like most of the routers which would need to make this decision wouldn't have adequate information upon which to do so... not necessarily. the decision could be made in near real time by building prefix filters based on the algorithms that josh and co have worked on and leaving a 'default deny' in place. this moves the routing decision off of the router (which i agree does not have the history or resources to take these additional vectors of information into account) and over to a server with more storage and computational capacity. The 'core' routers are definitely the best informed, though other ASs which are multi-homed also come across a substantial bit of information through updates. Yet if only the core ASs were to run such a solution, it would be sufficient to suppress most attacks for at least a day. The paper has more detail on that situation.
Re: preventing future situations like panix
To what extent does the route object validation in the RIPE database (for routes covering RIPE-allocated space), together with maintainer object authentication, provide a perfect IRR, according to your research? (I realise the step from useful, authenticated source of data to universally-deployed import filters is non-trivial.) My understanding is that RIPE, while quite good, still contains a significant amount of old data that needs to be regularly flushed. It certainly seems reasonable to use its information as a good first approximation of the validity of a route, and I think that would go quite well with our recommendation, reducing the number of routes flagged as suspicious.
Re: preventing future situations like panix
For those prefixes announced by ConEd within the last 3 days that it no longer owns, correct, it would not of helped. But saving some is certainly better than none. For the second statement things get a little more subtle. We have considered allowing the trusted originator of a prefix to split the space among itself and those downstream of it without considering that suspicious behavior. This allows ASs to protect themselves via such methods. Thanks for your comments! Josh On 1/23/06, Thor Lancelot Simon [EMAIL PROTECTED] wrote: On Mon, Jan 23, 2006 at 12:47:38PM -0700, Josh Karlin wrote: Suspicious routes are those that originate at an AS that has not originated the prefix in the last few days and those that introduce sub-prefixes. Sub-prefixes are always considered suspicious (~1 day) and traffic will be routed to the super-prefix for the suspicious period. So, if you consider the recent Cone-D hijacking incident, it seems to me that: 1) Cone-D's announcement of _some_ of the prefixes they announced would have been considered suspicious -- but not all, since some of the prefixes in question were for former customers or peers who had only recently terminated their business arrangements with Cone-D. 2) Panix's first, obvious countermeasure aimed at restoring their connectivity -- announcing their own address space split in half -- would *also* have been considered suspicious, since it gave two sub-prefixes of what Cone-D was hijacking. Unless I misunderstand what you're proposing -- which is entirely possible, in fact perhaps even likely -- it seems to me that it might well have done at least as much harm as good. Thor
