Re: 12/8 problems? (fwd)
-- Forwarded message -- Date: Wed, 14 Sep 2005 16:39:54 -0400 (EDT) From: Manish Karir [EMAIL PROTECTED] To: nanog@merit.edu Subject: Re: 12/8 problems? I'm sorry I'm a bit late on this thread but wanted to point out that you can view some of the relevant information on this, as seen from routeviews at: http://bgpinspect.merit.edu - At the bottom half of the page, select routeviews peer, e.g. sprint - select Prefix-Exact as query type - Enter prefix into the Query box: 12.0.0.0/8 (without the quotes) - Select start date as Aug 25, select end date as Sept 14 - Click submit-query. Alternately you can just get the pdf files of the results page from the above queries from the following 2 links: http://bgpinspect.merit.edu/reports/glx-12-8.pdf http://bgpinspect.merit.edu/reports/sprint-12-8.pdf After a few more cleanups and changes, we will be re-announcing the bgp-inspect project hopefully before the next nanog mtg. as a generally available service. thanks manish Date: Mon, 12 Sep 2005 18:02:29 -0400 From: Richard A Steenbergen [EMAIL PROTECTED] Subject: Re: 12/8 problems? On Sat, Sep 10, 2005 at 06:15:38AM -0700, Eric Louie wrote: FYI, happened again this morning for (at least) 12/8 duration approx 30 minutes starting at 5:45 AM PDT. Notice that ATT is no longer taking chances, and is announcing 2 /9s. - -- Richard A Steenbergen [EMAIL PROTECTED] http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC) --
Re: 12/8 problems?
On Sat, Sep 10, 2005 at 06:15:38AM -0700, Eric Louie wrote: FYI, happened again this morning for (at least) 12/8 duration approx 30 minutes starting at 5:45 AM PDT. Notice that ATT is no longer taking chances, and is announcing 2 /9s. -- Richard A Steenbergen [EMAIL PROTECTED] http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
12/8 problems?
Apologies for a post of an operational nature, but is anyone else seeing problems with ATTs 12/8 block? From a New York router connected to Global Crossing and Peer 1: border-1.nycmny sh ip bgp 12.xxx.xxx.xxx BGP routing table entry for 12.0.0.0/8, version 86901457 Paths: (2 available, best #1) Not advertised to any peer 3549 12956 26210 64.213.176.97 from 64.213.176.97 (208.50.59.1) Origin incomplete, metric 2602, localpref 100, valid, external, best, ref 2 Community: 232589665 232618104 13768 12956 26210, (received-only) 64.34.84.117 from 64.34.84.117 (216.187.124.10) Origin incomplete, localpref 100, external, ref 2 Route views is showing a 12/8 with a fair amount of dampening/flap penalties in the last 10-12 minutes. -- Drew Linsalata The Gotham Bus Company, Inc. Dedicated Servers and Colocation Solutions Long Island, New York http://www.gothambus.com
Re: 12/8 problems?
On Fri, Sep 09, 2005 at 11:12:25AM -0400, Drew Linsalata wrote: Apologies for a post of an operational nature, but is anyone else seeing problems with ATTs 12/8 block? From a New York router connected to Global Crossing and Peer 1: border-1.nycmny sh ip bgp 12.xxx.xxx.xxx BGP routing table entry for 12.0.0.0/8, version 86901457 Paths: (2 available, best #1) Not advertised to any peer 3549 12956 26210 64.213.176.97 from 64.213.176.97 (208.50.59.1) Origin incomplete, metric 2602, localpref 100, valid, external, best, ref 2 Community: 232589665 232618104 13768 12956 26210, (received-only) 64.34.84.117 from 64.34.84.117 (216.187.124.10) Origin incomplete, localpref 100, external, ref 2 Route views is showing a 12/8 with a fair amount of dampening/flap penalties in the last 10-12 minutes. Looks like 12956 is announcing some /8s to every peer and transit. Worse still, Sprint and GX are propagating it. This is not the first time that Telefonica has leaked a lot of garbage routes with serious network impact as a result (nor is it the second or third, actually). 12.0.0.0/8 64.0.0.0/8 65.0.0.0/8 I'd say both GX and Sprint have a lot to answer for right about now. -- Richard A Steenbergen [EMAIL PROTECTED] http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
Re: 12/8 problems?
On Fri, 9 Sep 2005, Richard A Steenbergen wrote: On Fri, Sep 09, 2005 at 11:12:25AM -0400, Drew Linsalata wrote: Apologies for a post of an operational nature, but is anyone else seeing problems with ATTs 12/8 block? From a New York router connected to Global Crossing and Peer 1: border-1.nycmny sh ip bgp 12.xxx.xxx.xxx BGP routing table entry for 12.0.0.0/8, version 86901457 Paths: (2 available, best #1) Not advertised to any peer 3549 12956 26210 64.213.176.97 from 64.213.176.97 (208.50.59.1) Origin incomplete, metric 2602, localpref 100, valid, external, best, ref 2 Community: 232589665 232618104 13768 12956 26210, (received-only) 64.34.84.117 from 64.34.84.117 (216.187.124.10) Origin incomplete, localpref 100, external, ref 2 Route views is showing a 12/8 with a fair amount of dampening/flap penalties in the last 10-12 minutes. Looks like 12956 is announcing some /8s to every peer and transit It looks like 12956 is propagating announcements from their customer 26210 of these /8 routes. It looks like 12956 does not have correct policies in place to block such announcements from their customers as many of the large ISPs in US do (mostly by requiring customers to pre-authorize and give list of blocks that they would be announcing) and that is why from time-time things like this leak out (which they deal with each time after the fact). It does seem appropriate that if 12956 is unable to put appropriate policies in place to make sure things like this do not happen, then all its announcements will have to be double-checked and pre-authorized by its transits i.e. GBLX and Sprint. --- William Leibzon Elan Networks [EMAIL PROTECTED]
Re: 12/8 problems?
On Fri, Sep 09, 2005 at 11:25:25AM -0400, Richard A Steenbergen wrote: Looks like 12956 is announcing some /8s to every peer and transit. Worse still, Sprint and GX are propagating it. This is not the first time that Telefonica has leaked a lot of garbage routes with serious network impact as a result (nor is it the second or third, actually). 12.0.0.0/8 64.0.0.0/8 65.0.0.0/8 I'd say both GX and Sprint have a lot to answer for right about now. Minor apologies to GX, it looks like Telefonica isn't a customer any more, just a direct peer. I'm still annoyed from the last outage caused when Telefonica leaked routes to GX as a transit customer. Sprint on the other hand propagated this as full transit. I'm glad to see no one has learned from AS7007. :) As for how to prevent this from happening again... I know many people who aren't able to implement full peer filtering are at least enforcing simple as-path checks on the largest ASNs (making sure that customers and peers don't reannounce paths which have 7018 in them, for example), but it doesn't look like anyone is trying to filter things on a largest prefix basis. When AS26210 decides to start originating the prefixes themselves instead of just leaking it from 7018, boom. -- Richard A Steenbergen [EMAIL PROTECTED] http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
Re: 12/8 problems?
Richard A Steenbergen wrote: Looks like 12956 is announcing some /8s to every peer and transit. Worse still, Sprint and GX are propagating it. This is not the first time that Telefonica has leaked a lot of garbage routes with serious network impact as a result (nor is it the second or third, actually). 12.0.0.0/8 64.0.0.0/8 65.0.0.0/8 I'd say both GX and Sprint have a lot to answer for right about now. Looks like 26210 is originating the prefixes and Telefonica is happily passing them along to the world, at least some portion of which is glad to go along for the ride. Q. How does the Internet work? A. Spit and glue. -- Drew Linsalata The Gotham Bus Company, Inc. Dedicated Servers and Colocation Solutions Long Island, New York http://www.gothambus.com
Re: 12/8 problems?
On Fri, Sep 09, 2005 at 11:44:05AM -0400, Drew Linsalata wrote: Looks like 26210 is originating the prefixes and Telefonica is happily passing them along to the world, at least some portion of which is glad to go along for the ride. Q. How does the Internet work? A. Spit and glue. $10 says someone forgot ip classless. $20 says this devolves into a discussion about pgp key signed bgp announcements or some other impractical soapbox within less than 10 emails. :) Now if only they made no ip clueless. -- Richard A Steenbergen [EMAIL PROTECTED] http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
Re: 12/8 problems?
Richard A Steenbergen wrote: $10 says someone forgot ip classless. Is there a valid argument for making ip classless the default in the IOS? Seems to me that it would only solve problems, but I don't profess to be a routing guru, especially in comparison to folks in this forum. -- Drew Linsalata The Gotham Bus Company, Inc. Dedicated Servers and Colocation Solutions Long Island, New York http://www.gothambus.com
RE: 12/8 problems?
Richard A Steenbergen wrote on Friday, September 09, 2005 11:57 AM: On Fri, Sep 09, 2005 at 11:44:05AM -0400, Drew Linsalata wrote: Looks like 26210 is originating the prefixes and Telefonica is happily passing them along to the world, at least some portion of which is glad to go along for the ride. Q. How does the Internet work? A. Spit and glue. $10 says someone forgot ip classless. I'll take that bet. My $10 says they turned on auto-summary. $20 says this devolves into a discussion about pgp key signed bgp announcements or some other impractical soapbox within less than 10 emails. :) Actually, my practical solution to this one is max-prefixing your peers. It means you have to watch your peers slow growth, but frankly, you should be watching that anyway. Now if only they made no ip clueless. Or at least correctly set the evil bit on appropriate packets.
Re: 12/8 problems?
12.0.0.0/8 64.0.0.0/8 65.0.0.0/8 And wouldn't you know it, we have an application that needs to reach servers in 12/8 and 65/8, and someone just came over to me asking for help in figuring out why that application isn't working. I guess I should have checked my NANOG mail before I told them I had no idea what was going on. :) It just so happens that our two providers are the ones previously mentioned that are accepting the offending routes from Telefonica. John --
RE: 12/8 problems?
On Fri, 9 Sep 2005, Israel, David B. wrote: Richard A Steenbergen wrote on Friday, September 09, 2005 11:57 AM: On Fri, Sep 09, 2005 at 11:44:05AM -0400, Drew Linsalata wrote: Looks like 26210 is originating the prefixes and Telefonica is happily passing them along to the world, at least some portion of which is glad to go along for the ride. $10 says someone forgot ip classless. I'll take that bet. My $10 says they turned on auto-summary. Here's what Telefonica is or has been announcing to peers with a _12956_26210_ AS path (note that a bunch of these are currently history entries, so aren't being announced at the moment). It looks like a corresponding classful announcement for every CIDR announcement, plus a few more. From http://lg.pch.net: equinix-ashburnsh ip bgp regex _12956_26210_ BGP table version is 8836678, local router ID is 157.22.13.84 Status codes: s suppressed, d damped, h history, * valid, best, i - internal, r RIB-failure, S Stale Origin codes: i - IGP, e - EGP, ? - incomplete Network Next HopMetric LocPrf Weight Path h 12.0.0.0 206.223.115.1070 12956 26210 ? h 12.144.80.0/24 206.223.115.1070 12956 26210 i h 12.144.82.0/23 206.223.115.1070 12956 26210 ? h 12.144.84.0/22 206.223.115.1070 12956 26210 i h 64.0.0.0 206.223.115.1070 12956 26210 ? h 65.0.0.0 206.223.115.1070 12956 26210 ? h 65.173.56.0/21 206.223.115.1070 12956 26210 i h 200.11.68.0 206.223.115.1070 12956 26210 ? h 200.58.64.0/21 206.223.115.1070 12956 26210 ? * 200.58.160.0/23 206.223.115.1070 12956 26210 25620 i * 200.58.162.0 206.223.115.1070 12956 26210 25620 i * 200.58.163.0 206.223.115.1070 12956 26210 25620 i * 200.58.164.0 206.223.115.1070 12956 26210 25620 i * 200.58.165.0 206.223.115.1070 12956 26210 25620 i * 200.58.166.0 206.223.115.1070 12956 26210 25620 i * 200.58.167.0 206.223.115.1070 12956 26210 25620 i * 200.58.168.0 206.223.115.1070 12956 26210 25620 i Network Next HopMetric LocPrf Weight Path * 200.58.169.0 206.223.115.1070 12956 26210 25620 i * 200.58.170.0 206.223.115.1070 12956 26210 25620 i * 200.58.171.0 206.223.115.1070 12956 26210 25620 i * 200.58.172.0 206.223.115.1070 12956 26210 25620 i * 200.58.173.0 206.223.115.1070 12956 26210 25620 i * 200.58.174.0 206.223.115.1070 12956 26210 25620 i * 200.58.175.0 206.223.115.1070 12956 26210 25620 i * 200.58.176.0 206.223.115.1070 12956 26210 25620 i * 200.58.177.0 206.223.115.1070 12956 26210 25620 i * 200.58.178.0 206.223.115.1070 12956 26210 25620 i * 200.58.179.0 206.223.115.1070 12956 26210 25620 i * 200.58.180.0 206.223.115.1070 12956 26210 25620 i * 200.58.181.0 206.223.115.1070 12956 26210 25620 i * 200.58.182.0 206.223.115.1070 12956 26210 25620 i * 200.58.183.0 206.223.115.1070 12956 26210 25620 i * 200.58.184.0 206.223.115.1070 12956 26210 25620 i * 200.58.185.0 206.223.115.1070 12956 26210 25620 i * 200.58.186.0 206.223.115.1070 12956 26210 25620 i * 200.58.187.0 206.223.115.1070 12956 26210 25620 i * 200.58.188.0 206.223.115.1070 12956 26210 25620 i * 200.58.189.0 206.223.115.1070 12956 26210 25620 i * 200.58.190.0 206.223.115.1070 12956 26210 25620 i Network Next HopMetric LocPrf Weight Path * 200.58.191.0 206.223.115.1070 12956 26210 25620 i * 200.75.160.0/20 206.223.115.1070 12956 26210 22541 22541 i * 200.85.128.0 206.223.115.1070 12956 26210 27714 i * 200.85.128.0/23 206.223.115.1070 12956 26210 ? * 200.85.129.0 206.223.115.1070 12956 26210 27714 i * 200.85.130.0 206.223.115.1070 12956 26210 27714 i * 200.85.131.0 206.223.115.1070 12956 26210 27714 i * 200.85.132.0 206.223.115.1070 12956 26210
Re: 12/8 problems?
Israel, David B. [EMAIL PROTECTED] writes: Actually, my practical solution to this one is max-prefixing your peers. It means you have to watch your peers slow growth, but frankly, you should be watching that anyway. Max-prefix is part of the battle. A corollary max-aggregate where for instance one could say shut the connection down if these guys try to announce more than a /14 worth of space total to me - I don't believe it would be convenient. None of this works of course if people don't turn it on though. ---rob
Re: 12/8 problems?
Drew Linsalata wrote: Richard A Steenbergen wrote: $10 says someone forgot ip classless. Is there a valid argument for making ip classless the default in the IOS? Seems to me that it would only solve problems, but I don't profess to be a routing guru, especially in comparison to folks in this forum. It has been that way for a while now? Pete
