Re: F.ROOT-SERVERS.NET moved to Beijing?
120K domains - basically cnnic seems to have finally got tired of russian botmaster types registering thousands of domains at a time, and put in a rule that says you need business registration in China / ID in china to register a .cn Beyond that, that's one ccTLD - however large. There are multiple gTLDs that have already done a great job of cleanup (biz, info for example) and so far I haven't heard of .us having an infestation of botmasters / spammers. And of course there are all the registrars out there that need to be reached out to / handled etc etc - but that's another kettle of fish. We're discussing two different things here - apples and oranges, though it does look like they're all part of the same fruit salad. 1. Action by different registrars / registries [in .cn's case, a government controlled registry, to be sure] 2. State policy to route internet access and DNS through an inspecting + rewriting firewall that blocks or replaces politically unacceptable content --srs On Mon, Oct 3, 2011 at 11:17 AM, Randy Bush ra...@psg.com wrote: china nukes 120,000 domains for going against the policy of the state. oops! that wasn't china, was it? perhaps, we should postpone telling others what to do until our side of the street is clean? randy -- Suresh Ramasubramanian (ops.li...@gmail.com)
Re: F.ROOT-SERVERS.NET moved to Beijing?
On Mon, 03 Oct 2011 11:29:43 +0530, Suresh Ramasubramanian said: 120K domains - basically cnnic seems to have finally got tired of russian No, I think Randy was referring to this sort of thing: http://www.theregister.co.uk/2011/02/18/fed_domain_seizure_slammed/ pgpZ8g15XRvjK.pgp Description: PGP signature
Re: F.ROOT-SERVERS.NET moved to Beijing?
Sure - but what was being discussed in this thread was transparent / on the fly rewrites of root server responses getting exposed to people beyond china. Whether these responses should be altered / censored within china or not is a different can of worms, and that too has nothing at all to do with either registry policy, or law enforcement mandated domain seizure. On Mon, Oct 3, 2011 at 11:42 AM, valdis.kletni...@vt.edu wrote: No, I think Randy was referring to this sort of thing: http://www.theregister.co.uk/2011/02/18/fed_domain_seizure_slammed/ -- Suresh Ramasubramanian (ops.li...@gmail.com)
Re: F.ROOT-SERVERS.NET moved to Beijing?
On Sun, Oct 02, 2011 at 05:40:23PM +, Janne Snabb sn...@epipe.com wrote a message of 32 lines which said: I happened to notice the following at three separate sites around the US and one site in Europe: Good analysis at http://bgpmon.net/blog/?p=540
Re: F.ROOT-SERVERS.NET moved to Beijing?
On Sun, Oct 02, 2011 at 04:06:44PM -0700, Leo Bicknell bickn...@ufp.org wrote a message of 107 lines which said: We have found networks where a query sent to F-Root never reaches an ISC run server. For details on such behavior, i highly recommend the excellent paper Identifying and Characterizing Anycast in the Domain Name System http://www.isi.edu/~xunfan/research/anycast_Tech_Report_ISI_TR_671.pdf, which shows, among other things, that such masquerading (by a false root name server) happens.
Re: F.ROOT-SERVERS.NET moved to Beijing?
- Original Message - From: valdis.kletni...@vt.edu On Mon, 03 Oct 2011 11:29:43 +0530, Suresh Ramasubramanian said: 120K domains - basically cnnic seems to have finally got tired of russian No, I think Randy was referring to this sort of thing: http://www.theregister.co.uk/2011/02/18/fed_domain_seizure_slammed/ Our government has gone rogue on us, Eric Goldman, a professor at Santa Clara University School of Law, said. Our government is going into court with half-baked facts and half-baked legal theories and shutting down operations. This is exactly what we thought the government couldn't do. I'm scratching my head why we aren't' grabbing the pitchforks. ® I.C.E., our very own Gestapo-Without-Borders. Makes me proud.sigh
Re: F.ROOT-SERVERS.NET moved to Beijing?
On Sun, Oct 02, 2011 at 05:40:23PM +, Janne Snabb sn...@epipe.com wrote a message of 32 lines which said: $ dig +short +norec @F.ROOT-SERVERS.NET HOSTNAME.BIND CHAOS TXT pek2a.f.root-servers.org The next time, I suggest to also run data queries such as A www.facebook.com or A www.twitter.com to see if there is hard evidence of an actual security problem. (Most articles on this case mentioned that we have no proof there was a rewriting of answers from the F-root instance.)
Re: F.ROOT-SERVERS.NET moved to Beijing?
Todd Underwood toddun...@gmail.com wrote: sure, but DNSSEC is still basically unused. If you are running BIND 9.8 there is really no reason not to turn on DNSSEC validation, then you won't have to worry about anycast routes leaking from behind the great firewall. dnssec-validation auto; dnssec-lookaside auto; Tony. -- f.anthony.n.finch d...@dotat.at http://dotat.at/ Viking, North Utsire: Southerly veering southwesterly 6 to gale 8, occasionally severe gale 9 at first in northwest Viking. Moderate or rough becoming very rough or high. Rain then squally showers. Moderate or good, occasionally poor.
Ashburn Va. Datacenter Movers
Can anyone recommend a vendor in the D.C. metro region with experience moving equipment between datacenters? I'm looking for an experienced, bonded, team of professionals to help with un-racking, moving and re-racking network equipment, servers and all of the related gear. Feel free to contact me offlist. Thanks, ~Matt
Re: F.ROOT-SERVERS.NET moved to Beijing?
On Oct 3, 2011, at 7:29 AM, Tony Finch wrote: If you are running BIND 9.8 there is really no reason not to turn on DNSSEC validation, then you won't have to worry about anycast routes leaking from behind the great firewall. User Exercise: What happens when you enable integrity checking in an application (e.g., 'dnssec-validation auto') and datapath manipulation persists? Bonus points for analysis of implementation and deployment behaviors and resulting systemic effects. Network layer integrity techniques and secure routing infrastructure are all that's going to fix this. In the interim, the ability to detect such incidents at some rate faster than the speed of mailing lists would be ideal. -danny
Re: Ashburn Va. Datacenter Movers
ask your account manager if he has any agreement in place with any supplier From: Matt Ryanczak ryanc...@gmail.com To: nanog@nanog.org Sent: Monday, October 3, 2011 2:22 PM Subject: Ashburn Va. Datacenter Movers Can anyone recommend a vendor in the D.C. metro region with experience moving equipment between datacenters? I'm looking for an experienced, bonded, team of professionals to help with un-racking, moving and re-racking network equipment, servers and all of the related gear. Feel free to contact me offlist. Thanks, ~Matt
Re: F.ROOT-SERVERS.NET moved to Beijing?
User Exercise: What happens when you enable integrity checking in an application (e.g., 'dnssec-validation auto') and datapath manipulation persists? Bonus points for analysis of implementation and deployment behaviors and resulting systemic effects. i agree with danny here. ignoring randy (and others) off-topic comments about hypocrisy, this situation is fundamentally a situation of bad (or different) network policy being applied outside of its scope. i would prefer that china not censor the internet, sure. but i really require that china not censor *my* internet when i'm not in china. t
...my Internet... snicker :)
On Mon, Oct 03, 2011 at 10:30:47AM -0400, Todd Underwood wrote: User Exercise: What happens when you enable integrity checking in an application (e.g., 'dnssec-validation auto') and datapath manipulation persists? Bonus points for analysis of implementation and deployment behaviors and resulting systemic effects. i agree with danny here. ignoring randy (and others) off-topic comments about hypocrisy, this situation is fundamentally a situation of bad (or different) network policy being applied outside of its scope. i would prefer that china not censor the internet, sure. but i really require that china not censor *my* internet when i'm not in china. t well, not to disagree - BUT the sole reason we have BGP and use ASNs the way we do is to ensure/enforce local policy. It is, after all, an AUTONOMOUS SYSTEM number. One sets policy at its boundaries on what/how to accept/reject/modify traffic crossing the boundary. If you dont -like- the ASN policy - then don't use/traverse that ASN. and rPKI has the same problems as DNSSEC. lack of uniform use/implementation is going to be a huge party - full of fun games. /bill
Re: F.ROOT-SERVERS.NET moved to Beijing?
On 03/10/2011 09:03, Stephane Bortzmeyer wrote: On Sun, Oct 02, 2011 at 05:40:23PM +, Janne Snabb sn...@epipe.com wrote a message of 32 lines which said: I happened to notice the following at three separate sites around the US and one site in Europe: Good analysis at http://bgpmon.net/blog/?p=540 We used DNSMON data to analyse this event, and found an earlier leak on 29 and 30 September: https://labs.ripe.net/Members/emileaben/f-root-route-leak-the-dnsmon-view best regards, Emile Aben RIPE NCC
Re: F.ROOT-SERVERS.NET moved to Beijing?
ignoring randy (and others) off-topic comments about hypocrisy actually, if you had followed the thread in its sad detail, at that point of jingoism they were on. this situation is fundamentally a situation of bad (or different) network policy being applied outside of its scope. kink is gonna leak. rfc1918 is gonna leak. ula-foo is gonna leak. pakistani kink is gonna leak. anycast 'local' cones are gonna leak. chinese kink is gonna leak. american kink is gonna leak. s/are gonna/has already/g are people gonna stop doing kink? sadly, not likely. so all we are left is Danny McPherson wrote: Network layer integrity techniques and secure routing infrastructure are all that's going to fix this. and Danny McPherson wrote: In the interim, the ability to detect such incidents at some rate faster than the speed of mailing lists would be ideal. is not a lot of good unless you insert and fix. watching train wrecks is about as fun as reading pontification on nanog. qed :) randy
Re: Facebook insecure by design
On 02/10/2011 19:01, Michael Thomas wrote: William Allen Simpson wrote: On 10/2/11 12:36 PM, Jimmy Hess wrote: On Sun, Oct 2, 2011 at 10:38 AM, Michael Thomasm...@mtcc.com wrote: I'm not sure why lack of TLS is considered to be problem with Facebook. The man in the middle is the other side of the connection, tls or otherwise. That's where the X509 certificate comes in. A man in the middle would not have the proper private key to impersonate the Facebook server that the certificate was issued to. My understanding of his statement is that Facebook itself is the MITM, collecting all our personal information. Too true. Bingo. Mike +1
Re: F.ROOT-SERVERS.NET moved to Beijing?
In a message written on Mon, Oct 03, 2011 at 09:27:46AM -0400, Danny McPherson wrote: User Exercise: What happens when you enable integrity checking in an application (e.g., 'dnssec-validation auto') and datapath manipulation persists? Bonus points for analysis of implementation and deployment behaviors and resulting systemic effects. I think this is a (to some on the list) cryptic way of asking If all your routes to the server go to someone masquerading, what happens when you try to validate that data? The question being if you configure your nameserver to validate the root, but don't get signed answers back will your nameserver refuse to serve up any data, effectively taking you and your users offline? The answer should be no. This is part of why there are 13 root servers. If a nameserver is told the root is signed and it gets unsigned answers from one of the 13, it should ignore them and move on. I do not off the top of my head know all the timeouts and implementation dependant behaviors, but also remember that a up caching resolver will make approximately 1 query to the root per day for _valid_ names, but many queries per day for invalid names. Thus the impact to valid names should be minimal, even in the face of longer timeouts. Is there enough operational experience with DNSSEC? No. Can we fix that by saying it's not good enough yet? No. Run it. The people who write nameserver software are comitted to fixing any issues as quickly as possible, because it is our best way to secure DNS. Network layer integrity techniques and secure routing infrastructure are all that's going to fix this. In the interim, the ability to detect such incidents at some rate faster than the speed of mailing lists would be ideal. Network layer integrity and secure routing don't help the majority of end users. At my house I can choose Comcast or ATT service. They will not run BGP with me, I could not apply RPKI, secure BGP, or any other method to the connections. They may well do NXDOMAIN remapping on their resolvers, or even try and transparently rewrite DNS answers. Indeed some ISP's have even experimented with injecting data into port 80 traffic transparently! Secure networks only help if the users have a choice, and choose to not use bad networks. If you want to be able to connect at Starbucks, or the airport, or even the conference room Wifi on a clients site you need to assume it's a rogue network in the middle. The only way for a user to know what they are getting is end to end crypto. Period. As for the speed of detection, its either instantenous (DNSSEC validation fails), or it doesn't matter how long it is (minutes, hours, days). The real problem is the time to resolve. It doesn't matter if we can detect in seconds or minutes when it may take hours to get the right people on the phone and resolve it. Consider this weekend's activity; it happened on a weekend for both an operator based in the US and a provider based in China, so you're dealing with weekend staff and a 12 hour time difference. If you want to insure accuracy of data, you need DNSSEC, period. If you want to insure low latency access to the root, you need multiple Anycasted instances because at any one point in time a particular one may be bad (node near you down for maintenance, routing issue, who knows) which is part of why there are 13 root servers. Those two things together can make for resilliance, security and high performance. -- Leo Bicknell - bickn...@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/ pgpNfaKxvGaS0.pgp Description: PGP signature
Re: Facebook insecure by design
On Mon, Oct 3, 2011 at 4:27 AM, William Allen Simpson william.allen.simp...@gmail.com wrote: On 10/2/11 12:36 PM, Jimmy Hess wrote: On Sun, Oct 2, 2011 at 10:38 AM, Michael Thomasm...@mtcc.com wrote: I'm not sure why lack of TLS is considered to be problem with Facebook. The man in the middle is the other side of the connection, tls or otherwise. That's where the X509 certificate comes in. A man in the middle would not have the proper private key to impersonate the Facebook server that the certificate was issued to. My understanding of his statement is that Facebook itself is the MITM, collecting all our personal information. Too true. I assume that any MITM is actually going to try and prevent our data from making it to the end point i.e the real attacker. -- Regards, Jason Leschnik. [m] 0432 35 4224 [w@] jason dot leschnik at ansto dot gov dot aujason.lesch...@ansto.gov.au [U@] jml...@uow.edu.au
Re: F.ROOT-SERVERS.NET moved to Beijing?
On Oct 3, 2011, at 11:20 AM, Leo Bicknell wrote: Thus the impact to valid names should be minimal, even in the face of longer timeouts. If you're performing validation on a recursive name server (or similar resolution process) expecting a signed response yet the response you receive is either unsigned or doesn't validate (i.e., bogus) you have to: 1) ask other authorities? how many? how frequently? impact? 2) consider implications on _entire chain of trust? 3) tell the client something? 4) cache what (e.g., zone cut from who you asked)? how long? 5) other? minimal is not what I was thinking.. Network layer integrity and secure routing don't help the majority of end users. At my house I can choose Comcast or ATT service. They will not run BGP with me, I could not apply RPKI, secure BGP, or any other method to the connections. They may well do NXDOMAIN remapping on their resolvers, or even try and transparently rewrite DNS answers. Indeed some ISP's have even experimented with injecting data into port 80 traffic transparently! Secure networks only help if the users have a choice, and choose to not use bad networks. If you want to be able to connect at Starbucks, or the airport, or even the conference room Wifi on a clients site you need to assume it's a rogue network in the middle. The only way for a user to know what they are getting is end to end crypto. Period. I'm not sure how end to end crypto helps end users in the advent of connectivity and *availability* issues resulting from routing brokenness in an upstream network which they do not control. crypto, OTOH, depending on what it is and where in the stack it's applied, might well align with my network layer integrity assertion. As for the speed of detection, its either instantenous (DNSSEC validation fails), or it doesn't matter how long it is (minutes, hours, days). The real problem is the time to resolve. It doesn't matter if we can detect in seconds or minutes when it may take hours to get the right people on the phone and resolve it. Consider this weekend's activity; it happened on a weekend for both an operator based in the US and a provider based in China, so you're dealing with weekend staff and a 12 hour time difference. If you want to insure accuracy of data, you need DNSSEC, period. If you want to insure low latency access to the root, you need multiple Anycasted instances because at any one point in time a particular one may be bad (node near you down for maintenance, routing issue, who knows) which is part of why there are 13 root servers. Those two things together can make for resilliance, security and high performance. You miss the point here Leo. If the operator of a network service can't detect issues *when they occur* in the current system in some automated manner, whether unintentional or malicious, they won't be alerted, they certainly can't fix the problem, and the potential exposure window can be significant. Ideally, the trigger for the alert and detection function is more mechanized than notification by services consumer, and the network service operators or other network operators aware of the issue have some ability to institute reactive controls to surgically deal with that particular issue, rather than being captive to the [s]lowest common denominator of all involved parties, and dealing with additional non-determinsitic failures or exposure in the interim. Back to my earlier point, for *resilience* network layer integrity techniques and secure routing infrastructure are the only preventative controls here, and necessarily to augment DNSSEC's authentication and integrity functions at the application layer. Absent these, rapid detection enabling reactive controls that mitigate the issue are necessary. -danny
Re: F.ROOT-SERVERS.NET moved to Beijing?
On Mon, Oct 3, 2011 at 12:38 PM, Danny McPherson da...@tcb.net wrote: If the operator of a network service can't detect issues *when they occur* in the current system in some automated manner, whether unintentional or malicious, they won't be alerted, they certainly can't fix the problem, and the potential exposure window can be significant. Ideally, the trigger for the alert and detection function is more mechanized than notification by services consumer, and the network service operators or other network operators aware of the issue have Does ISC (or any other anycast root/*tld provider) have external polling methods that can reliably tell when, as was in this case, local-anycast-instances are made global? (or when the cone of silence widens?) Given that in the ISC case the hostname.bind query can tell you at least the region + instance#, it seems plausible that some system of systems could track current/changes in the mappings, no? and either auto-action some 'fix' (SHUT DOWN THE IAD INSTANCE IT's ROGUE!) or at least log and notify a hi-priority operations fixer. Given something like the unique-as work Verisign has been behind you'd think monitoring route origins and logging 'interesting' changes could accomplish this as well? (I suppose i'm not prescribing solutions above, just wondering if something like these is/could-be done feasibly) -chris
Re: Facebook insecure by design
Jason Leschnik wrote: On Mon, Oct 3, 2011 at 4:27 AM, William Allen Simpson william.allen.simp...@gmail.com wrote: On 10/2/11 12:36 PM, Jimmy Hess wrote: On Sun, Oct 2, 2011 at 10:38 AM, Michael Thomasm...@mtcc.com wrote: I'm not sure why lack of TLS is considered to be problem with Facebook. The man in the middle is the other side of the connection, tls or otherwise. That's where the X509 certificate comes in. A man in the middle would not have the proper private key to impersonate the Facebook server that the certificate was issued to. My understanding of his statement is that Facebook itself is the MITM, collecting all our personal information. Too true. I assume that any MITM is actually going to try and prevent our data from making it to the end point i.e the real attacker. What fun would that be? Seriously though, a MITM doesn't have to be disruptive; there are a zillion and three other reasons. Like getting a big budg hollywood movie made about you. Mike
Re: F.ROOT-SERVERS.NET moved to Beijing?
In a message written on Mon, Oct 03, 2011 at 12:38:25PM -0400, Danny McPherson wrote: 1) ask other authorities? how many? how frequently? impact? 2) consider implications on _entire chain of trust? 3) tell the client something? 4) cache what (e.g., zone cut from who you asked)? how long? 5) other? minimal is not what I was thinking.. I'm asking the BIND team for a better answer, however my best understanding is this will query a second root server (typically next best by RTT) when it gets a non-validating answer, and assuming the second best one validates just fine there are no further follow on effects. So you're talking one extra query when a caching resolver hits the root. We can argue if that is minimal or not, but I suspect most end users behind that resolver would never notice. You miss the point here Leo. If the operator of a network service can't detect issues *when they occur* in the current system in some automated manner, whether unintentional or malicious, they won't be alerted, they certainly can't fix the problem, and the potential exposure window can be significant. In a message written on Mon, Oct 03, 2011 at 01:09:17PM -0400, Christopher Morrow wrote: Does ISC (or any other anycast root/*tld provider) have external polling methods that can reliably tell when, as was in this case, local-anycast-instances are made global? (or when the cone of silence widens?) Could ISC (or any other root operator) do more monitoring? I'm sure, but let's scope the problem first. We're dealing here with a relatively wide spread leak, but that is in fact the rare case. There are 39,000 ASN's active in the routing system. Each one of those ASN's can affect it's path to the root server by: 1) Bringing up an internal instance of a root server, injecting it into its IGP, and hijacking the route. 2) Turning up or down a peer that hosts a root server. 3) Turning up or down a transit provider. 4) Adding or removing links internal to their network that change their internal selection to use a different external route. The only way to make sure a route was correct, everywhere, would be to have 39,000+ probes, one on every ASN, and check the path to the root server. Even if you had that, how do you define when any of the changes in 1-4 are legitimate? You could DNSSEC verify to rule out #1, but #2-4 are local decisions made by the ASN (or one of its upstreams). I suppose, if someone had all 39,000+ probes, we could attempt to write algorythms that determined if too much change was happening at once; but I'm reminded of events like the earthquake that took out many asian cables a few years back. There's a very real danger in such a system shutting down a large number of nodes during such an event due to the magnitude of changes which I'd suggest is the exact opposite of what the Internet needs to have happen in that event. (I suppose i'm not prescribing solutions above, just wondering if something like these is/could-be done feasibly) Not really. Look, I chase down several dozen F-Root leaks a year. You never hear about them on NANOG. Why? Well, it's some small ISP in the middle of nowhere leaking to a peer who believes them, and thus they get a 40ms response time when they should have a 20ms response time by believing the wrong route. Basically, almost no one cares, generally it takes some uber-DNS nerd at a remote site to figure this out and contact us for help. This has tought me that viewpoints are key. You have to be on the right network to detect it has hijacked all 13 root servers, you can't probe that from the outside. You also have to be on the right network to see you're getting the F-Root 1000 miles away rather than the one 500. Those 39,000 ASN's are providing a moving playing field, with relationships changing quite literally every day, and every one of them may be a leak. This one caught attention not because it was a bad leak. It was IPv6 only. Our monitoring suggests this entire leak siphoned away 40 queries per second, at it's peak, across all of F-Root. In terms of a percentage of queries it doesn't even show visually on any of our graphs. No, it drew attention for totally non-technical reasons, US users panicing that the Chinese goverment was hijacking the Internet which is just laughable in this context. There really is nothing to see here. DNSSEC fixes any security implications from these events. My fat fingers have dropped more than 40qps on the floor more than once this year, and you didn't notice. Bad events (like earthquakes and fiber cuts) have taken any number of servers from any number of operators multiple times this year. Were it not for the fact that someone posted to NANOG, I bet most of the people here would have never noticed their 99.999% working system kept working just fine. I think all the root ops can do better, use more monitoring services, detect more route hijacks faster, but none of us will ever get 100%.
Re: F.ROOT-SERVERS.NET moved to Beijing?
On Oct 3, 2011, at 1:09 PM, Christopher Morrow wrote: Given that in the ISC case the hostname.bind query can tell you at least the region + instance#, it seems plausible that some system of systems could track current/changes in the mappings, no? and either auto-action some 'fix' (SHUT DOWN THE IAD INSTANCE IT's ROGUE!) or at least log and notify a hi-priority operations fixer. That sort of capability at the application layer certainly seems prudent to me, noting that it does assume you have a measurement node within the catchment in question and are measuring at a high enough frequency to detect objective incidents. Given something like the unique-as work Verisign has been behind you'd think monitoring route origins and logging 'interesting' changes could accomplish this as well? I'm a fan of both routing system consumer-esque monitoring, and do believe that a discriminator in the routing system associated with globally anycasted prefixes makes this simpler - for both detection, and possibly even reactive or preventative controls IF necessary. A unique origin AS is not the only place you can do this in the routing system, as I'm sure some will observe, but it seems an ideal location to me. -danny
Re: F.ROOT-SERVERS.NET moved to Beijing?
On Oct 3, 2011, at 1:34 PM, Leo Bicknell wrote: I'm asking the BIND team for a better answer, however my best understanding is this will query a second root server (typically next best by RTT) when it gets a non-validating answer, and assuming the second best one validates just fine there are no further follow on effects. So you're talking one extra query when a caching resolver hits the root. We can argue if that is minimal or not, but I suspect most end users behind that resolver would never notice. I'm not talking one extra query, and it's not simply about subsequent transaction attempts either - so conjecture aiming to marginalize the impact isn't particularly helpful. I.e., have that look, get back to us... :-) -danny
Re: F.ROOT-SERVERS.NET moved to Beijing?
Leo, On Mon, Oct 3, 2011 at 7:34 PM, Leo Bicknell bickn...@ufp.org wrote: The only way to make sure a route was correct, everywhere, would be to have 39,000+ probes, one on every ASN, and check the path to the root server. Even if you had that, how do you define when any of the changes in 1-4 are legitimate? You could DNSSEC verify to rule out #1, but #2-4 are local decisions made by the ASN (or one of its upstreams). I suppose, if someone had all 39,000+ probes, we could attempt to write algorythms that determined if too much change was happening at once; but I'm reminded of events like the earthquake that took out many asian cables a few years back. There's a very real danger in such a system shutting down a large number of nodes during such an event due to the magnitude of changes which I'd suggest is the exact opposite of what the Internet needs to have happen in that event. This sounds an awfully lot like the notary concept: - http://perspectives-project.org/ - http://convergence.io/ Furthermore, changing network paths used to reach information probably should not be reason to shut down a service, in general. More interesting than which path is used, I suppose, is whether or not the data being returned has been changed in some unexpected/undesired way. Regards, Martin
FW: [arin-announce] Whois Query Behavior is Updated
Hi Apologies for the cross post from ARIN-Announce. Thought that many of you would be interested in hearing about the recent ARIN Whois change given the recent discussion on NANOG. Regards, Mark Kosters ARIN CTO On 10/2/11 8:51 PM, ARIN i...@arin.net wrote: ARIN has changed the Whois query behavior on port 43. A query for an IP address in the ARIN region will return only that assignment/allocation within the ARIN region, and will no longer included ARIN's parent record (the /8). Both the Whois web interface and Whois-RWS behavior were not affected by this configuration change. Regards, Mark Kosters Chief Technical Officer American Registry for Internet Numbers (ARIN)
Re: F.ROOT-SERVERS.NET moved to Beijing?
On 2011-10-03, at 13:39, Danny McPherson wrote: On Oct 3, 2011, at 1:09 PM, Christopher Morrow wrote: Given that in the ISC case the hostname.bind query can tell you at least the region + instance#, it seems plausible that some system of systems could track current/changes in the mappings, no? and either auto-action some 'fix' (SHUT DOWN THE IAD INSTANCE IT's ROGUE!) or at least log and notify a hi-priority operations fixer. That sort of capability at the application layer certainly seems prudent to me, noting that it does assume you have a measurement node within the catchment in question and are measuring at a high enough frequency to detect objective incidents. In principle there seems like no reason that a DNS client sending queries to authority-only servers couldn't decide to include the NSID option and log changes in declared server identity between subsequent queries (or take some other configured action). We support 5001 on L-Root (which runs NSD), for what that's worth, as well as HOSTNAME.BIND/CH/TXT, VERSION.BIND/CH/TXT, ID.SERVER/CH/TXT and VERSION.SERVER/CH/TXT, but those require separate queries. I appreciate NSID support is not universal, but perhaps that's ok in the sense of better than nothing. I'm a fan of both routing system consumer-esque monitoring, and do believe that a discriminator in the routing system associated with globally anycasted prefixes makes this simpler - for both detection, and possibly even reactive or preventative controls IF necessary. A unique origin AS is not the only place you can do this in the routing system, as I'm sure some will observe, but it seems an ideal location to me. Whether it's the right-most entry in the AS_PATH or a bigger substring, you still need more measurement points than you have if you want to catch every leak. Joe signature.asc Description: Message signed with OpenPGP using GPGMail
Re: F.ROOT-SERVERS.NET moved to Beijing?
Furthermore, changing network paths used to reach information probably should not be reason to shut down a service, in general. cool. then we can get rid of dynamic routing. it always has been a pain in the ass. randy
Re: he.net down?
My peering with them at PAIX has been down for almost 50 minutes. Not able to get to them via other paths... On Mon, Oct 3, 2011 at 3:37 PM, chris tknch...@gmail.com wrote: Down here as well chris On Oct 3, 2011 6:36 PM, Aiden Sullivan ai...@sullivan.in wrote: www.he.net seems to be down on both IPv4 and IPv6 -- does anyone know what is going on? -- Aiden
Re: he.net down?
In message 9b29642e-ea4d-4aec-aa45-866fd5829...@oitc.com, TR Shaw writes: Fine here in FL $ ping6 www.he.net PING6(56=3D40+8+8 bytes) 2001:470:5:4ed:cabc:c8ff:fea1:560c -- = 2001:470:0:76::2 16 bytes from 2001:470:0:76::2, icmp_seq=3D0 hlim=3D55 time=3D178.017 ms On Oct 3, 2011, at 6:39 PM, Christopher Morrow wrote: On Mon, Oct 3, 2011 at 6:37 PM, chris tknch...@gmail.com wrote: Down here as well =20 =20 ~$ ping6 www.he.net PING www.he.net(he.net) 56 data bytes 64 bytes from he.net: icmp_seq=3D2 ttl=3D54 time=3D124 ms =20 chris On Oct 3, 2011 6:36 PM, Aiden Sullivan ai...@sullivan.in wrote: www.he.net seems to be down on both IPv4 and IPv6 -- does anyone = know what is going on? =20 -- Aiden =20 =20 =20 =20 Looks like fremont/lax is down which is the normal path to my tunnel endpoint (currently unreachable). There were issue late last week as well. Mark traceroute to 64.71.128.82 (64.71.128.82), 64 hops max, 44 byte packets 1 10.72.0.1 (10.72.0.1) 8.059 ms 6.195 ms 7.196 ms 2 bla1-ge0-0-1.gw.optusnet.com.au (198.142.160.181) 7.895 ms 7.501 ms 8.788 ms 3 bla5-ge2-1.gw.optusnet.com.au (211.29.126.77) 8.692 ms 6.985 ms 7.314 ms 4 203.208.190.85 (203.208.190.85) 162.929 ms 162.457 ms 163.349 ms 5 10gigabitethernet1-3.core1.lax1.he.net (206.223.123.37) 163.099 ms 164.391 ms 174.027 ms 6 10gigabitethernet1-1.core1.phx1.he.net (72.52.92.250) 178.051 ms 173.395 ms 174.653 ms 7 10gigabitethernet1-3.core1.dal1.he.net (72.52.92.254) 197.036 ms 196.807 ms 202.276 ms 8 * 10gigabitethernet4-4.core1.chi1.he.net (184.105.213.118) 223.289 ms 224.630 ms 9 10gigabitethernet3-2.core1.den1.he.net (184.105.213.86) 247.460 ms 247.256 ms * 10 * * * 11 * * * 12 * * * 13 * * * 14 * * * 15 * * * 16 * * * 17 * * * 18 * * * -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org
Re: F.ROOT-SERVERS.NET moved to Beijing?
On Mon, 03 Oct 2011 15:07:02 PDT, Leo Bicknell said: If we went back to hosts.txt this pesky DNS infrastructure would be totally unnecessary. You're just saying that because you're hoping your employer will get to sell bandwidth to SRI-NIC.ARPA ;) pgpjuWWXhHkU5.pgp Description: PGP signature
Re: he.net down?
Our session with them is up and down at Any2 at OWB. --Original Message-- From: Aiden Sullivan To: nanog@nanog.org Subject: he.net down? Sent: Oct 3, 2011 3:35 PM www.he.net seems to be down on both IPv4 and IPv6 -- does anyone know what is going on? -- Aiden Sent from my Verizon Wireless BlackBerry
he.net down?
www.he.net seems to be down on both IPv4 and IPv6 -- does anyone know what is going on? -- Aiden
Re: he.net down?
Down here as well chris On Oct 3, 2011 6:36 PM, Aiden Sullivan ai...@sullivan.in wrote: www.he.net seems to be down on both IPv4 and IPv6 -- does anyone know what is going on? -- Aiden
Re: he.net down?
On Mon, Oct 3, 2011 at 6:37 PM, chris tknch...@gmail.com wrote: Down here as well ~$ ping6 www.he.net PING www.he.net(he.net) 56 data bytes 64 bytes from he.net: icmp_seq=2 ttl=54 time=124 ms chris On Oct 3, 2011 6:36 PM, Aiden Sullivan ai...@sullivan.in wrote: www.he.net seems to be down on both IPv4 and IPv6 -- does anyone know what is going on? -- Aiden
Re: he.net down?
Fine here in FL $ ping6 www.he.net PING6(56=40+8+8 bytes) 2001:470:5:4ed:cabc:c8ff:fea1:560c -- 2001:470:0:76::2 16 bytes from 2001:470:0:76::2, icmp_seq=0 hlim=55 time=178.017 ms On Oct 3, 2011, at 6:39 PM, Christopher Morrow wrote: On Mon, Oct 3, 2011 at 6:37 PM, chris tknch...@gmail.com wrote: Down here as well ~$ ping6 www.he.net PING www.he.net(he.net) 56 data bytes 64 bytes from he.net: icmp_seq=2 ttl=54 time=124 ms chris On Oct 3, 2011 6:36 PM, Aiden Sullivan ai...@sullivan.in wrote: www.he.net seems to be down on both IPv4 and IPv6 -- does anyone know what is going on? -- Aiden
Re: he.net down?
On 10/03/2011 12:35 PM, Aiden Sullivan wrote: www.he.net seems to be down on both IPv4 and IPv6 -- does anyone know what is going on? Linode's Fremont location was effected too, HE are their network providers, was down for about an hour. Paul
Re: he.net down?
On Mon, Oct 03, 2011 at 11:14:03PM +, Michael J McCafferty wrote: Our session with them is up and down at Any2 at OWB. --Original Message-- From: Aiden Sullivan To: nanog@nanog.org Subject: he.net down? Sent: Oct 3, 2011 3:35 PM www.he.net seems to be down on both IPv4 and IPv6 -- does anyone know what is going on? -- Aiden Sent from my Verizon Wireless BlackBerry Blaming DDOS. http://status.linode.com The incident was a probable DDOS attack, but its behavior was unusual and difficult to identify. Our network engineers made some adjustments to the DOS countermeasures acquired after last week's incident, and that seems to have stabilized traffic flow. We apologize for the inconvenience. -Ben Larsen Hurricane Electric Internet Services Some supporting evidence would be nice. - Nate Itkin
Re: he.net down?
On Oct 3, 2011, at 7:25 PM, Nate Itkin wrote: On Mon, Oct 03, 2011 at 11:14:03PM +, Michael J McCafferty wrote: Our session with them is up and down at Any2 at OWB. --Original Message-- From: Aiden Sullivan To: nanog@nanog.org Subject: he.net down? Sent: Oct 3, 2011 3:35 PM www.he.net seems to be down on both IPv4 and IPv6 -- does anyone know what is going on? -- Aiden Sent from my Verizon Wireless BlackBerry Blaming DDOS. http://status.linode.com The incident was a probable DDOS attack, but its behavior was unusual and difficult to identify. Our network engineers made some adjustments to the DOS countermeasures acquired after last week's incident, and that seems to have stabilized traffic flow. We apologize for the inconvenience. -Ben Larsen Hurricane Electric Internet Services Some supporting evidence would be nice. Exactly what do you expect a network which is attacked to post to NANOG, or a random web page, to prove they were attacked? Given the 1000s of network outages over the last decade, I can think of maybe a handful that supplied supporting evidence. As I said before, Mike the gang at HE are stand-up people. If they said it was a DoS, it was a DoS - although I note they did not say it was a DoS, just probably a DoS. But I extend my faith if their lack of prevarication to even statement as well. In fact, it speaks well that they are being equivocal until they are certain themselves. -- TTFN, patrick
RE: he.net down?
Since we're on the topic of DoS. What best practice actions can be taken AFTER such an attack? Subject: Re: he.net down? From: patr...@ianai.net Date: Mon, 3 Oct 2011 19:33:10 -0400 To: nanog@nanog.org On Oct 3, 2011, at 7:25 PM, Nate Itkin wrote: On Mon, Oct 03, 2011 at 11:14:03PM +, Michael J McCafferty wrote: Our session with them is up and down at Any2 at OWB. --Original Message-- From: Aiden Sullivan To: nanog@nanog.org Subject: he.net down? Sent: Oct 3, 2011 3:35 PM www.he.net seems to be down on both IPv4 and IPv6 -- does anyone know what is going on? -- Aiden Sent from my Verizon Wireless BlackBerry Blaming DDOS. http://status.linode.com The incident was a probable DDOS attack, but its behavior was unusual and difficult to identify. Our network engineers made some adjustments to the DOS countermeasures acquired after last week's incident, and that seems to have stabilized traffic flow. We apologize for the inconvenience. -Ben Larsen Hurricane Electric Internet Services Some supporting evidence would be nice. Exactly what do you expect a network which is attacked to post to NANOG, or a random web page, to prove they were attacked? Given the 1000s of network outages over the last decade, I can think of maybe a handful that supplied supporting evidence. As I said before, Mike the gang at HE are stand-up people. If they said it was a DoS, it was a DoS - although I note they did not say it was a DoS, just probably a DoS. But I extend my faith if their lack of prevarication to even statement as well. In fact, it speaks well that they are being equivocal until they are certain themselves. -- TTFN, patrick
Re: he.net down?
On 04/10/2011, at 10:08 AM, Brandon Kim wrote: Since we're on the topic of DoS. What best practice actions can be taken AFTER such an attack? Notifying NANOG/*NOG lists? ;)
Re: F.ROOT-SERVERS.NET moved to Beijing?
In a message written on Tue, Oct 04, 2011 at 07:00:52AM +0900, Randy Bush wrote: cool. then we can get rid of dynamic routing. it always has been a pain in the ass. If we went back to hosts.txt this pesky DNS infrastructure would be totally unnecessary. -- Leo Bicknell - bickn...@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/ pgpAqU4PG4Kpm.pgp Description: PGP signature
Re: he.net down?
On 10/3/11 4:38 PM, Brandon Kim wrote: Since we're on the topic of DoS. What best practice actions can be taken AFTER such an attack? Wait to hear from Roland. ;) ~Seth
RE: he.net down?
We are back up in LAX, 16:39PDT. Hopefully, it stays up. ~ Blaine -Original Message- From: Seth Mattinen [mailto:se...@rollernet.us] Sent: Monday, October 03, 2011 4:47 PM To: nanog@nanog.org Subject: Re: he.net down? On 10/3/11 4:38 PM, Brandon Kim wrote: Since we're on the topic of DoS. What best practice actions can be taken AFTER such an attack? Wait to hear from Roland. ;) ~Seth
Re: he.net down?
On Oct 3, 2011, at 6:54 PM, Dave hartzell wrote: My peering with them at PAIX has been down for almost 50 minutes. Not able to get to them via other paths... Just posted to outages@ (where this discussion should likely be taking place): HE's entire network is intermittently down. They claim it is a DoS: https://twitter.com/#!/henet/status/120951537974513665 It is not the first time this week. Mike the team are good engineers, they'll do what they can to bring it backup quickly. There are lots of other rumors floating around, but I think you should wait for official word. HE is not a telco, they will tell you what happened. (Right Mike? :) -- TTFN, patrick
Re: he.net down?
Here's what Linode at Fremont looked like from our network (m5hosting.com) and the Linode sites in Newark, and London. http://smokev.m5hosting.com/smokeping/smokeping.cgi?target=World.USA.Hurricane The smokeping from our network at m5hosting.com is through the Any2 at OWB. On Mon, 2011-10-03 at 13:17 -1000, Paul wrote: On 10/03/2011 12:35 PM, Aiden Sullivan wrote: www.he.net seems to be down on both IPv4 and IPv6 -- does anyone know what is going on? Linode's Fremont location was effected too, HE are their network providers, was down for about an hour. Paul -- Michael J. McCafferty Principal M5 Hosting http://www.m5hosting.com You can have your own custom Dedicated Server up and running today ! RedHat Enterprise, CentOS, Ubuntu, Debian, OpenBSD, FreeBSD, and more
Re: [arin-announce] Whois Query Behavior is Updated
'twas a consummation devoutly to be wished .. thanks, mark! --srs(iPad) On 04-Oct-2011, at 2:03, Christopher Morrow morrowc.li...@gmail.com wrote: On Mon, Oct 3, 2011 at 3:11 PM, Mark Kosters ma...@arin.net wrote: Hi Apologies for the cross post from ARIN-Announce. Thought that many of you would be interested in hearing about the recent ARIN Whois change given the recent discussion on NANOG. thanks! :)
Re: he.net down?
Hi Michael, Our smokeping shows a similar thing. http://smokeping.a2b-internet.com/smokeping/?target=US.HE A lot of packetloss around the 30th / 1st on IPv4 while V6 was unaffected. Their support stated it was a fibercut in the usa somewhere. From Amsterdam all shows good and green again. Erik Verstuurd vanaf mijn iPad Op Oct 4, 2011 om 2:43 heeft Michael J McCafferty m...@m5computersecurity.com het volgende geschreven: Here's what Linode at Fremont looked like from our network (m5hosting.com) and the Linode sites in Newark, and London. http://smokev.m5hosting.com/smokeping/smokeping.cgi?target=World.USA.Hurricane The smokeping from our network at m5hosting.com is through the Any2 at OWB. On Mon, 2011-10-03 at 13:17 -1000, Paul wrote: On 10/03/2011 12:35 PM, Aiden Sullivan wrote: www.he.net seems to be down on both IPv4 and IPv6 -- does anyone know what is going on? Linode's Fremont location was effected too, HE are their network providers, was down for about an hour. Paul -- Michael J. McCafferty Principal M5 Hosting http://www.m5hosting.com You can have your own custom Dedicated Server up and running today ! RedHat Enterprise, CentOS, Ubuntu, Debian, OpenBSD, FreeBSD, and more
Re: he.net down?
Its working fine from India as well... -Thanks, Viral. On 4 October 2011 04:05, Aiden Sullivan ai...@sullivan.in wrote: www.he.net seems to be down on both IPv4 and IPv6 -- does anyone know what is going on? -- Aiden
